Measuring the success of a learning health system: development of an evaluation framework for the Sydney Kid’s Learning Health Initiative

Abstract

Background: Accessibility of electronic patient data should enable more hospitals to become ‘learning health systems’ (LHS).

Aims: We developed an evaluation framework for measuring the success of the Sydney Kid’s Learning Health Initiative, a strategic program at the Sydney Children’s Hospitals Network (SCHN), Australia, which aims to implement a LHS for better use of clinical data and therefore improved patient care.

Methods: We developed a logic model evaluation framework by conducting a series of electronic surveys and a final workshop with a working group of key stakeholders at SCHN. Stakeholders comprised representatives from the program’s operations team, clinicians, executive board, clinical governance, research ethics and governance, consumer involvement and academics in implementation science, and clinical research.

Results: The logic model describes the key inputs, activities, outputs and impacts of the early phase of the program. Key impacts included: 1) children’s health, the number of children reached; 2) social, the number of clinicians trained; 3) economic, the societal costs for improving models of care; and 4) knowledge, changes to local or wider models of care. We will develop several proof-of-concept projects which aim to test our logic model and demonstrate the potential value of implementing a LHS at SCHN. In this presentation, we summarise key considerations for implementing a LHS within an Australian context.

Conclusions: Learning health systems interact with and impact many components across the health system. We have identified several key categories for measuring success of our program that may assist other institutions to develop similar evaluation frameworks.