Curriculum critique – Rural Research Capacity Building Program

Abstract

Purpose: This paper provides a critique of the curriculum of the Rural Research Capacity Building Program, a novice researcher training program conducted by New South Wales Health in Australia.

Approach: Using a framework approach, aspects of the curriculum are explored with reference to existing literature and 10 years of program coordinator experience implementing the curriculum.

Findings: This critique has found that the curriculum provides structure and flexibility for teaching and learning practices. A competency-based framework provides clarity for learners in knowing when program requirements have been met. By encouraging learners to compile a portfolio of evidence, achievement of competency can be demonstrated against all aspects of the curriculum. As the program targets motivated self-directed learners, the ability to tailor learning experiences within the curriculum is a critical component.

Research implications: This critique provides a foundation for further research exploring specific aspects of curriculum delivery.

Practical implications: Formal critique of a curriculum allows for ongoing refinement of teaching practices.

Originality/value: Formal curricula of workplace-based training programs are rarely explored in the literature. Critiques of curricula provide an important tool for the examination of teaching methods and program outcomes.

Limitations: The Rural Research Capacity Building Program is a single program, which may limit the generalisability of these findings

Keywords: curriculum, adult learning, research education

1 Health Education and Training Institute, NSW Health, St Leonards, Australia
Corresponding author: Dr David Schmidt, Senior Program Manager – Rural Research, Health Education and Training Institute, Level 2, 1 Reserve Road, St Leonards NSW 2065, Australia
[email protected]

Background

The aim of the Rural Research Capacity Building Program (RRCBP) is to guide development of research and evaluation skills of experienced health workers who are novice researchers, while simultaneously contributing to the rural health research evidence base. Candidates of the program work within the public health sector in public health executive, hospital, community, aged care or domiciliary settings. Health District executive sponsors provide input into the selection process for candidates. On average a cohort of 18 candidates (range 13–26) are accepted into the two-year program, with intakes held every 12–18 months. As a result, cohorts overlap by 6–12 months and more than one cohort is underway the majority of the time.

A structured curriculum guides RRCBP teaching, learning, milestones and demonstration of competency of skills for these health workers who undertake workplace-based research within their rural environment with the support of the program.

The RRCBP curriculum was designed by the Health Education and Training Institute (HETI) with input from external educational consultants from the University of Newcastle in 2014 in response to an internal review undertaken by Smyth (2014). The curriculum built upon the existing processes and goals of the program that had been developed and internally evaluated since its inception in 2006. The curriculum was endorsed by HETI’s executive and implemented with the commencement of the 2015 cohort of the program. Since this implementation, six cohorts (total 110 enrolments) have completed their two-year program. By comparison, eight cohorts (total 162 enrolments) completed the program prior to implementation of the curriculum.

The curriculum was constructed with a flexible linear design that matches the needs of the health workforce and educational context. Within this design, individual key areas or units of competency are compiled into the overarching learning experience. These units of competency are designed to supplement the learning journey, which combines experiential learning and cultural constructivism, underpinned by capacity building principles (Schmidt, Webster & Duncanson 2019).

Program coordinators have constructed and refined the program’s learning resources to encompass the extensive curriculum, then deliver the content in a flexible manner to meet the diverse needs of learners. As learners are geographically dispersed across rural NSW, the program employs multi-modal delivery. Program material is delivered using a combination of distance education, teleconferencing, webconferencing and face-to-face workshops.

The flexible nature of the program means the curriculum itself can be interpreted and applied differently from cohort to cohort. Program coordinators can account for differences in selected study designs chosen by learners to answer their research questions, both with and between cohorts. For example, within each cohort there are quantitative, qualitative and mixed methods studies, but the proportions of each study design differ between cohorts, as do the topic areas and disciplines of the learners.

Critiquing curricula is an important part of ensuring educational quality (Lock, Hill & Dyjur 2018) and a guiding framework is a common approach to curriculum review processes (Lock, Hill & Dyjur 2018; Dyjur & Kalu 2016).

This critique is guided by the following framework, which was first encountered by the lead author (DS) as part of a postgraduate education qualification (Kerr 2012) and has previously been used by the lead author for a curriculum critique assessing a qualification set at the Certificate IV level under the Australian Qualifications Framework (Schmidt 2012):

1. Foundations of the curriculum and the philosophy on which the curriculum is based

2. Which curriculum model has been chosen and how that fits with educational philosophy

3. Curriculum congruence in integrating theory and practice

4. How learning progression is demonstrated throughout the curriculum

5. How the curriculum allows for different learning styles and rates of progression

6. How the curriculum allows for different teaching methods and multiple methods of competency attainment

7. How the curriculum is assessed

8. How the model empowers learners to take responsibility for their own learning

While this framework of guiding questions was selected based on previous experience, the use of Miller’s Pyramid (Williams et al. 2016), both within the curriculum and in this critique, provided additional theoretical rigour.

The curriculum can be found in Supplement A. The potential for future curriculum development is discussed with reference to alternative philosophical frameworks. While there is a clear and coordinated approach to teaching and learning throughout the program, this experiential learning program focuses on clinician research education, skill development and translation of research into evidence-based practice. As the program is widely viewed as a flagship for programs of this type, an understanding of this curriculum is likely to have broader applicability. The RRCBP is not yet aligned with a particular level of the Australian Qualifications Framework (Australian Qualifications Framework Council 2013) and currently does not lead to a nationally recognised qualification.

1. Foundations Of The Curriculum And The Philosophy On Which The Curriculum Is Based

The educational philosophies underpinning the RRCBP curriculum are threefold. The program was conceived in 2006 on capacity building principles (Webster et al. 2011). This was enhanced with implementation of a curriculum in 2015, with specific reference to the principles of research capacity building as outlined by Cooke (2005). Subsequent integration of Miller’s Pyramid (Williams et al. 2016) now allows the delivery of content to ensure the ability for learners to “know”, “know how”, “show” or “do” as appropriate for their learning needs and the project being undertaken.

The role of the educator in the curriculum is to scaffold the learning experience. Candidates often begin their research learning as novices, despite often being very experienced clinicians. Of the 110 candidates enrolled in the program since 2015, none had prior research higher degrees. As the learning experiences progress and learners grow in confidence, the emphasis moves from knowledge to application, consistent with Miller’s Pyramid (Williams et al. 2016). The philosophies underlying the curriculum encompass the learner’s aspirational goal and capacity building, with practical how-to steps, milestones and measurable outcomes that facilitate progress toward that aspiration. The ‘cohort’ structure of the program is aligned with positive psychology, in that novice researchers will perform ‘better together’ in the unfamiliar and initially daunting research environment (Gable & Haidt 2005). Thus, the transition from novice towards expert occurs both individually and collectively.

Previous research on the RRCBP has demonstrated that graduates of the program moved from novice researchers to the equivalent of researchers with small grant funding or two peer-reviewed publications, according to self-rated research experience (Schmidt, Webster & Duncanson 2019) as measured using a validated tool, the Research Spider (Smith et al. 2002), although it should be noted that the study examined graduates of the program that were enrolled prior to the curriculum implementation in 2015. Repeating this analysis to compare pre- and post-curriculum implementation will add evidence to the impact of the curriculum and is planned as a future study.

2. Which Curriculum Model Has Been Chosen And How That Fits With Educational Philosophy

The curriculum model has been structured on the areas of research experience outlined in Smith’s Research Spider (Smith et al. 2002), with two additional domains added to cover contextual factors and mixed methods approaches. Domains are outlined in Table 1.

Table 1: Domains of research experience in RRCBP curriculum

These domains allow learners to reflect on areas of experience and then use competencies to meet the required educational targets and their own learning objectives (Iwasiw, Goldenberg & Andrusyszyn 2005). The curriculum is also sequential to match stages of the research process – understanding, planning, implementing and communicating/disseminating research.

The modular structure of the curriculum allows flexible entry and exit points. While standard program progression allows a direct linear pathway for many learners, the modules allow individual learners to enter later in the program if prior experience and learning can be demonstrated. Similarly, individuals may have met their own learning goals at the completion of some but not all modules, and the structure of the curriculum allows a defined stepping off point. Figure 1 shows the modular design of the curriculum. Since implementing the curriculum, 32 of the 110 (29%) individuals enrolled in the program have left prior to completion, which is a small, non-significant improvement (p = 0.68, Fisher’s Exact test) over the previous intakes (53/162, 33%).

Figure 1: Modular design of the RRCBP

The capacity building framework is a pragmatic and practical underlying philosophy in the RRCBP, and a ‘learning-by-doing’ approach provides a good fit with the competency-based structure. This fit is important in ensuring learning goals are met (Iwasiw, Goldenberg & Andrusyszyn 2005).

Introductory short education modules on research processes, available on the NSW Health MyHealthLearning platform are used in the early stages of the program. The program does not use a set textbook, instead recommendations are tailored to individual need, ranging from introductory practical texts for piloting surveys (Bell 2005) through to more complex qualitative works in a specific qualitative methodology (Edwards, O'Mahoney & Vincent 2014). The majority of learning resources used within workshops and webconferences have been developed from existing expertise and have not undergone a formal validation process. This allows the practical knowledge of teaching staff in the real-world experience of conducting research in a rural environment to inform the teaching processes. Ongoing feedback on teaching resources is sought, offered and provided by learners in each cohort, many of whom have education and teaching expertise.

One aspect of the learning philosophy that is not explicit within the curriculum but is made clear within program documents and program delivery is recognition of student learning independent of, and in addition to, the program content. Examples include additional learning about the research topic or research processes of particular interest to individuals. This learner-centred approach of recognising what one knows, rather than how one has achieved the knowledge, is consistent with adult learning principles (Breier 2005; Cooper & Harris 2013).

3. Curriculum congruence in integrating theory and practice

The primary goals of the program are:

· To increase the number of rural and remote health workers with knowledge and skills in evaluation and research methods, and

· To contribute to the literature on both innovation and evidence-based practice around rural and remote health care.

The use of a capacity building framework and experiential learning processes ensure that theory is linked to practice (eds Doel & Shardlow 2009) in the RRCBP curriculum. Program coordinators translate and apply the program’s theoretical underpinnings into the education content and to each learner’s individual project. Learning activities such as pre-program learning tasks and group activities, evidence portfolio short-answer questions and group activities built into workshops ensure that theory and applicability are intertwined. For example, just-in-time learning (Baruah 2013) is applied so that learning tasks are immediately applicable. Some content is repeated so that learners who are at different stages in their projects are exposed to the content ‘just-in-time’ within their journey. For example, some learners join sessions with subsequent cohorts if the curriculum content better matches their milestone trajectory.

Theory components within the learning resources have been adapted to the rural health research context. Program graduates return as mentors in the program, sharing lived experiences of how they applied their theoretical research concepts into the real-world context. Educational approaches such as the use of problem-based learning encourage engagement within workshops and web conferences, while motivating learners to look for opportunities to meet their own learning needs (Rotgans & Schmidt 2011). The ‘application-through-experience’ integration of theory and practice through the curriculum is also consistent with clinicians’ experiences of combined formal and informal learning within the workplace (Jennings & Wargnier 2012; Misko 2008).

Experienced educators are critical to the application of the curriculum via experiential learning (Spouse 2001). The coordinators of this program are both graduates of the program and experienced research mentors. The coordinators provide support, mentorship and education for each individual and their project and the cohort of learners, in addition to coordination and management of the program as a whole. Both coordinators were trained to manage the program while job-sharing with an existing program coordinator, ensuring continuity of historical and corporate knowledge, consistency in program management and delivery, and maintenance of relationships with key stakeholders.

Post the curriculum introduction, there have been 57 graduates, 98% of which remain working in a rural area. These graduates have produced over 16 peer-reviewed publications and a minimum of 45 conference presentations on their close-to-practice research, thus contributing to the rural health evidence base. Projects that do not proceed to publication have their research reports archived for open access on the program’s website.

4. How learning progression is demonstrated throughout the curriculum

The portfolio of evidence is used to demonstrate achievement against the units of competency. The portfolio can be completed contemporaneously with the workshops and their corresponding units of competency, by using prior learning, or as a reflective tool at the completion of the learning experience. The curriculum has been carefully designed to capture existing tasks that occur during the experiential process, with additional short-answer questions used to expand the learner’s scope beyond the domain of a single project. Table 2 shows how each domain of research is demonstrated against the levels of knowledge in Miller's pyramid, using activities from the workshops and elements of the portfolio of evidence.

Table 2: Demonstrating evidence using Miller's Pyramid against domains of the Research Spider (Smith et al. 2002) and additional domains

There are a range of options for demonstrating learning throughout the program. For most learners this occurs as a linear progress through a single research project using experiential learning supplemented by formal learning tasks. For some learners however, a more flexible structure may be required. Learners can combine recognition of prior learning and contribute to different ‘domains’ across multiple research projects. For example, a learner may design one study to protocol stage, collect data for a different study, contribute to another ethics application and co-write a research report on a different study. The ability to combine these different approaches to completing the units of competency is demonstrated in Figure 2.

Figure 2: Combining different approaches to demonstrate learning against units of competency

While the relationship between the units of competency and the assessment process is depicted as unidirectional, in practice the process is much more iterative. The units of competency provide the structural framework for additional learning that supplements the experiential learning component. This leads the learner to use the units as a checklist against which they ensure they have completed all the learning tasks, while the learning tasks direct the learner back to the units of competency and the underlying curriculum.

One risk with a modular curriculum is that there may be limited opportunity for the knowledge of each unit to build directly on previous learning (Ellis 2019; Morrison 1993; Young 2002). In this program the sequential process inherent in developing and undertaking a research project ensures that early learning underpins later tasks. Only those with a heavy reliance on prior learning or alternative completion of the portfolio via multiple projects would be at risk of a disconnect where modules do not build on preceding activities. This risk is also mitigated by the learners being assigned one of the program coordinators as a primary contact to ensure continuity. Building linkages with their research mentor, their cohort peers and graduates of the program also provides learners with support to achieve each unit of competency.

5. How the curriculum allows for different learning styles and rates of progression

The curriculum of the RRCBP outlines the approaches used to deliver content. The curriculum refers to workshops, peer learning via group activities, independent learning and a mentoring system to support and scaffold this learning. The combination of face-to-face and distance education could be described asa hybrid multimodal approach (Iwasiw, Goldenberg & Andrusyszyn 2005). Learning methods are not prescribed within the curriculum, and program coordinators are expected to identify and address particular learning needs of candidates. The absence of prescriptiveness suits adult learners and allows flexibility to accommodate different learning styles (Wheelahan & Carter 2001). Such flexibility means a range of methods can be used to engage learning, enhance leaner satisfaction and boost interactivity. This combination of approaches allows support while creating an autonomous learning environment at the same time.

The process step of undertaking formal research-training-needs assessment (Brown 2002; Iqbal & Khan 2011) and the self-assessed learning-styles checklist (Hawk & Shah 2007; Morse, Oberer & Mitchell 1998; Neuhauser 2002; Pashler et al. 2008; Smith & Dalton 2005) as part of the pre-program activities are critical steps in ensuring coordinators and learners alike understand both learning needs and learning styles on entry into the program. This open engagement with learners about their preferred learning style and learning needs is reliant on both learner and program coordinator being honest and willing to adapt (Smith & Blake 2005).

The RRCBP is delivered in a way that is milestone- rather than timeline-driven. It is based on the understanding that planning, implementing, analysing and reporting on the outcomes of research projects is not always linear, and that the time and resources that clinician-researchers can devote to their project will vary across their candidature. This is achieved by having a spectrum of topics for weekly webinars, each of which is focused on one cohort or cohort subset, but of interest to those in other cohorts, and sometimes to program graduates. Recording these sessions accommodates self-paced learners and allows candidates that need to revisit or make up for a missed session to do so.

Similarly, while the content of the program’s first workshop is delivered to the entire cohort, subsequent workshops include some whole-cohort sessions, and other sessions targeted to different stages of the research process and different research methodologies. By the program’s fourth and final workshop, some candidates will be working on writing their reports while others will be supported in finalising ethics or analysis, depending on their study status.

Using the submission date for the draft report as a consistent milestone, it can be seen that five of the six completing cohorts since 2015 have achieved median on-time completion, something that was achieved in only 50% of eight completing cohorts prior to the implementation of the curriculum.

6. How the curriculum allows for different teaching methods and multiple methods of competency attainment

The curriculum for the RRCBP does not dictate how skills are to be taught but provides a range of options and approaches for flexible delivery of content. Teaching through directive approaches, facilitation and inquiry are all established approaches derived from adult learning theory (Knowles, Holton & Swanson 2005), which program coordinators apply during the program. Choosing which approach to use at different times is partially in response to learner’s expressed needs, learner growth and also in response to the changing nature of the relationship between the learner and program coordinators (Duncanson, Schmidt & Webster 2020). This flexibility is important, given criticisms of competency-based curricular structures as being inflexible or rigid (Tritton 2008).

Overall, program coordinators tend to use less directive teaching approaches to deliver the RRCBP, as learners within the program are frequently motivated, high-achieving and curious. Such characteristics are more closely associated with learners who perform well in self-directed learning environments (de Bruijn & Leeman 2011; Edmondson, Boyer & Artis 2012).

As the program coordinators have lived experience as graduates of the program and remain research active, they are able to teach through modelling (Knowles, Holton & Swanson 2005). By reflecting and embedding their own relevant real-world experience as rural clinician-researchers in their teaching and facilitation, program coordinators have established credibility and relatability.

Most learners achieve competency in all four units using the same research project. Under some circumstances this is not possible, and candidates can submit evidence of competency from other research projects they have undertaken. This method is consistent with ‘recognition of prior learning’, although for some candidates this is more accurately described as ‘parallel learning’ completed during their RRCBP candidature. Of the 57 graduates since 2015, 51 have followed the linear process of a single research project and six have used a ‘parallel learning’ approach to demonstrating competency.

7. How the curriculum is assessed

The RRCBP curriculum does not contain information on how the curriculum itself is assessed and reviewed, hence the need for this critique. It does however set a minimum benchmark that all learners attain a Novice level according to Miller’s Pyramid (Williams et al. 2016) across all domains of the curriculum. Table 2 demonstrates how this may be evidenced.

RRCBP program outcomes and outputs relative to the curriculum and the program’s establishment goals have been explored through peer-reviewed research conducted by program coordinators. This has included directly measuring change in the 10 elements contained in the Research Spider (Smith et al. 2002) in terms of self-rated research experience (Schmidt, Webster & Duncanson 2019), evidencing achievement against specific elements of the curriculum (Duncanson, Schmidt & Webster 2018) and examining the impact of the program as a whole on research activity and career growth (Schmidt et al. 2022).

Resources associated with delivery of the curriculum: portfolio documents, teaching resources and the application forms are reviewed annually by program coordinators. This annual review is based on feedback received from executive sponsors of the program within partner health districts and learner feedback obtained from workshops, annual reviews and graduation interviews.

Discussions about how to measure the outcomes of a RRCB program in a rural health context are ongoing, with program coordinators currently exploring approaches with jurisdictional network peers on how best to capture and describe the ‘value’ of rural research capacity building initiatives. This pragmatic and academic piece of work could use the RRCBP curriculum as a benchmark document for the development of domains of assessment.

8. How the curriculum empowers learners to take responsibility for their own learning

The RRCBP program allows learners to take responsibility for their learning in a number of ways. The research-training-needs assessment, completed prior to commencement, encourages learners to identify areas in which learning is needed and to work with coordinators to devise learning strategies to address these needs. Annual reviews are conducted with learners, with opportunity provided to revise individual learning goals that were set at program commencement.

Education topics for weekly webconferences are selected with input from learners and in line with self-directed and just-in-time learning principles (Baruah 2013).

The mentoring package provided to learners, in combination with the education on mentoring, emphasises the need for the learner to take an active role and greater responsibility for their learning, for example, setting agendas and ensuring mentoring needs are met. This is also recognised as essential in addressing any power imbalance in the mentoring relationship (Hudson 2013).

Effectiveness of the curriculum: a discussion

The RRCBP curriculum has formalised existing teaching and learning practices developed over time within the program, contributing both structure and flexibility. The decision to move to a competency-based framework has made explicit the steps required for learners to meet the curriculum. The program’s portfolio of evidence provides a way for learners to demonstrate achievement of competency and allows for supplemental learning to ensure all aspect of the curriculum are met. The flexibility inherent in this design is important for this group of motivated self-directed learners (de Bruijn & Leeman 2011).

While this critique has focused on educational processes, it is important to consider education outcomes. Research has demonstrated that the program is achieving its foundational goal of building research experience (Schmidt, Webster & Duncanson 2019), although whether the curriculum has further enhanced this is yet to be assessed. Change in research experience is one way to demonstrate the transition from “knowing” to “knowing how” or “showing” (Williamset al . 2016), or demonstrate that learning has occurred (Kirkpatrick & Kirkpatrick 2006).

The program continues to add to the rural health evidence base (Duncanson, Schmidt & Webster 2018) and improve research capability in the long-term (Schmidt et al. 2022). It should be noted that the program has provided evidence of capacity building prior to the implementation of the formal curriculum (Webster et al. 2011) and the curriculum has built on an existing foundation of success.

One aspect of workplace-based education to be considered is the enculturation or “hidden curriculum” that occurs within the health workplace (Chuang et al. 2010). While research-training programs can act as a means of cultural constructivism (Schmidt et al. 2022; Schmidt & Kirby 2016), the influence of the learner’s workplace remains active and can influence the learner’s experience and even their continuation in the program (Schmidt, Robinson & Webster 2014). For example, if a learner’s research is not supported by their line manager, it may not be possible to allocate time to research activities. While this is may present a challenge to workplace-based education, the opportunities and importance of training-in-place and close-to-practice research are important facets of research capacity building (Cooke 2005; Sarre & Cooke 2009) and offset this risk. The curriculum encompasses scenario and role-play sessions on building research into the workplace and managing workplace relationships to reinforce the importance of this aspect of the learner experience.

The program was originally designed to target the individual learner, and the curriculum continues this focus, with emphasis on building supports around an individual rather than a team-based approach to research. There is increasing recognition of the value of research teams (Cooke et al. 2008; Curry et al. 2012; Holden et al. 2012; Lakhani, Benzies & Hayden 2012) in health research. While program coordinators have been open to team-based projects, and a number of projects with two researchers have been conducted, the use of teams was identified in the 2014 program review as a potential future growth area (Smyth 2014). Future revision of the curriculum could ensure that training allows for individual, pair or team-based learning experiences. This could be achieved through formal applications by teams, or increased emphasis on candidates building a team around their project at a local level. Program coordinators have also consulted with other RCB program managers who support primarily team-based projects. Consideration of how team projects could be financially supported could accompany any curriculum changes. While individual researchers remain the primary focus, the cohort itself also provides aspects of a team environment such as collaborative problem solving, peer support and shared goals.

One element that the curriculum does not currently privilege is Indigenous research methodologies. While Indigenist methods are mentioned in the context of qualitative research, it is acknowledged that this is embedded within Western research philosophies. In the interests of decolonising the curriculum (Bullen & Flavell 2022; McLaughlin & Whatman 2007), future development of the curriculum should value Indigenous research methods as a valid and reliable path to knowledge creation alongside Western research traditions. Learning how to conduct culturally safe research with the Indigenous community, using either Indigenist or Western methods, is essential for many RRCBP projects and would allow a greater ability to understand and explore Indigenous culture (Gorman & Toombs 2009). Engaging Indigenous researchers as part of the teaching team will be essential for this knowledge to be appropriately delivered. Ensuring the cultural safety of learners and teaching staff is paramount in this process. Future revision of the curriculum provides opportunity to decolonise the curriculum and privilege Indigenous methodologies.

Future curriculum development can also embrace the possibility of articulating with formal qualifications, such as a research higher degree. This is an activity that occurs in practice but is not explicitly stated in the curriculum beyond encouraging learners to engage with higher degree studies. There is evidence that the program itself does lead some learners to undertake further university studies (Schmidt et al. 2022) and there is opportunity within the curriculum to outline where and how an articulation with the university sector could occur. Aligning the curriculum with the Australian Qualifications Framework (AQF) would support HETI to seek credit recognition/advanced standing for the RRCBP against units of study within postgraduate awards. A demonstrated alignment to courses delivered by higher education providers and associated AQF level would be required by each higher education provider to meet their credit recognition/advanced standing process requirements. This would support participants with credit toward postgraduate qualifications and allow them to potentially graduate with a formal qualification, for example, at the Graduate Diploma or Graduate Certificate level. This is a logical step and was an original recommendation of the review that led to the development of the curriculum (Smyth 2014).

Limitations

The RRCBP is a single program, which may limit the generalisability of these findings. As there are limited curricula in the published literature, a single program critique may still provide learnings to other research-training providers.

Conclusions

The curriculum for the RRCBP as delivered by the NSW HETI is effective and functional. The structure provides assurance that learning goals are met while allowing flexibility to deliver content in a way that enhances self-directed learning and outcomes. The outcomes of the program are well evidenced and highlight the value of a formal curriculum that builds on a successful workplace education program.

Future revision of the curriculum is recommended with a view to clearly stating and measuring RRCB outcomes and outputs, exploring team-based research approaches, articulation with the higher education sector and alignment with the AQF with the goal of credit recognition, advanced standing or a formal qualification. An enhanced program curriculum will increase value for learners and their organisations, provide a professional development pathway for rural health professionals and strengthen partnerships with higher education providers.

Funding

No specific funding was received for the completion of this manuscript.

References

Australian Qualifications Framework Council 2013, Australian qualifications framework , 2 edn, Australian Qualifications Framework Council, South Australia.

Baruah, HK 2013, ‘Just-in-time learning’, IOSR Journal of Humanities and Social Science ,vol. 12, no. 4, pp. 53-7.

Bell, J 2005, Doing your research project: a guide for first-time researchers in education, health and social science, 4 edn, McGraw-Hill Education, Berkshire.

Breier, M 2005, ‘A disciplinary-specific approach to the recognition of prior informal experience in adult pedagogy:‘rpl’ as opposed to ‘RPL’’, Studies in Continuing Education,vol. 27, no. 1, pp. 51-65.

Brown, J 2002, ‘Training needs assessment: A must for developing an effective training program’, Public Personnel Management,vol. 31, no.4, pp. 569-78.

Bullen, J & Flavell, H 2022, ‘Decolonising the indigenised curricula: preparing Australian graduates for a workplace and world in flux’, Higher Education Research & Development , vol. 41, no. 5, pp. 1402-16.

Chuang, AW, Nuthalapaty, FS, Casey, PM, Kaczmarczyk, JM, Cullimore, AJ, Dalrymple, JL &. Peskin, EG 2010, ‘To the point: reviews in medical education-taking control of the hidden curriculum’, American Journal of Obstetrics and Gynecology , vol. 203, no. 4, pp. 316.e311.

Cooke, J 2005, ‘A framework to evaluate research capacity building in health care’ BMC Family Practice,vol.6, pp. 44.

Cooke, J, Nancarrow, S, Dyas, J & Williams, M 2008, ‘An evaluation of the 'Designated Research Team' approach to building research capacity in primary care’, BMC Family Practice,vol. 9, pp. 12p.

Cooper, L & Harris, J 2013, ‘Recognition of prior learning: exploring the ‘knowledge question’’, International Journal of Lifelong Education ,vol. 32, no. 4, pp. 447-63.

Curry, LA, O’Cathain, A, Clark, VLP, Aroni, R, Fetters, M & Berg, D 2012, ‘The role of group dynamics in mixed methods health sciences research teams’, Journal of Mixed Methods Research,vol. 6, no. 1, pp. 5-20.

de Bruijn, E & Leeman, Y 2011, ‘Authentic and self-directed learning in vocational education: challenges to vocational educators’, Teaching and Teacher Education , vol. 27, no. 4, pp. 694-702.

Doel, M & Shardlow, S (eds.) 2009, Educating professionals: practice learning in health and social care . Ashgate.

Duncanson, K, Schmidt, D & Webster, E 2020, ‘Giving and receiving written feedback on research reports: a narrative review and guidance for supervisors and students. Health Education in Practice: Journal of Research for Professional Learning , vol. 3, no. 2, pp. 2.

Duncanson, K, Webster, EL & Schmidt, DD 2018, ‘Impact of a remotely delivered, writing for publication program on publication outcomes of novice researchers’, Rural and Remote Health, vol. 18, no. 2.

Dyjur, P & Kalu, F 2016, Introduction to curriculum review, Taylor Institute for Teaching and Learning, University of Calgary, Clagary.

Edmondson, DR, Boyer, SL & Artis, AB 2012, ‘Self-directed learning: a meta-analytic review of adult learning constructs’, International Journal of Education Research , vol. 7, no. 1, pp. 40-8.

Edwards, PK, O'Mahoney, J & Vincent, S 2014, Studying organizations using critical realism: a practical guide. Oxford University Press, Oxford.

Ellis, R 2019, ‘Towards a modular language curriculum for using tasks’, Language Teaching Research, vol. 23, no. 4, pp. 454-75.

Gable, S. L., & Haidt, J. (2005). What (and why) is positive psychology? Review of General Psychology, 9(2), 103-110.

Gorman, D & Toombs, M 2009, ‘Matching research methodology with Australian indigenous culture’, Aboriginal and Islander Health Worker Journal , vol. 33, no. 3, pp. 4-7.

Hawk, TF & Shah, AJ 2007, ‘Using learning style instruments to enhance student learning’, Decision Sciences Journal of Innovative Education , vol. 5, no. 1, pp. 1-19.

Holden, L, Pager, S, Golenko, X, Ware, RS & Weare, R 2012, ‘Evaluating a team-based approach to research capacity building using a matched-pairs study design’, BMC Family Practice, vol.13, pp. 16.

Hudson, P 2013, ‘Developing and sustaining successful mentoring relationships’, Journal of Relationships Research, vol. 4, pp. e1.

Iqbal, MZ & Khan, RA 2011, ‘The growing concept and uses of training needs assessment: a review with proposed model’, Journal of European Industrial Training , vol. 35, no. 5, pp. 439-66.

Iwasiw, C, Goldenberg, D & Andrusyszyn, M-A 2005, Curriculum development in nursing education, Jones & Bartlett Publishers.

Jennings, C & Wargnier, J 2012, Effective Learning with 70:20:10; the new frontier for the extended enterprise, CrossKnowledge.

Kerr, R 2012, ‘Clinical Education Planning’, in HLT404 Subject Outline, Charles Sturt University, Bathurst.

Kirkpatrick, D & Kirkpatrick, J 2006, Evaluating training programs: the four levels, Berrett-Koehler Publishers.

Knowles, MS, Holton, EF, & Swanson, RA 2005, The adult learner: the definitive classic in adult education and human resource development , 6th edn, Elsevier.

Lakhani, J, Benzies, K & Hayden, KA 2012, ‘Attributes of interdisciplinary research teams: a comprehensive review of the literature’, Clinical and Investigative Medicine, vol. 35, no. 5, pp. E226.

Lock, J, Hill, L & Dyjur, P 2018, 'Living the curriculum review: Perspectives from three leaders', Canadian Journal of Higher Education, vol. 48, no. 1, pp. 118-31.

McLaughlin, JM & Whatman, SL 2007, ‘Embedding indigenous perspectives in university teaching and learning: lessons learnt and possibilities of reforming/decolonising curriculum’, in Proceedings 4th International Conference on Indigenous Education : Asia/Pacific.

Misko, J 2008, Combining formal, non-formal and informal learning for workforce skill development . NCVER, Adelaide.

Morrison, K 1993, ‘Building progression into modular higher degrees in education’, Journal of In-Service Education, vol. 19, no. 3, pp. 5-11.

Morse, J, Oberer, J & Mitchell, D 1998, ‘Understanding learning styles: implications for staff development educators’, Journal of Nursing Staff Development , vol. 14, no. 1, pp. 41-6.

Neuhauser, C. 2002, ‘Learning style and effectiveness of online and face-to-face instruction’, The American Journal of Distance Education , vol. 16, no. 2, pp. 99-113.

Pashler, H, McDaniel, M, Rohrer, D & Bjork, R 2008, ‘Learning styles: concepts and evidence’, Psychological Science in the Public Interest , vol. 9, no. 3, pp. 105-19.

Rotgans, JI & Schmidt, HG 2011, ‘Cognitive engagement in the problem-based learning classroom’, Advances in Health Sciences Education , vol. 16, no. 4, pp. 465-79.

Sarre, G & Cooke, J 2009, ‘Developing indicators for measuring Research Capacity Development in primary care organizations: a consensus approach using a nominal group technique’, Health & Social Care in the Community , vol. 17, no. 3, pp. 244.
https://doi.org/10.1111/j.1365-2524.2008.00821.x

Schmidt, D 2012, Certificate IV in Allied Health Assistance: a curriculum review and discussion document New South Wales Rural Health and Research Congress, Wagga Wagga. https://www.voced.edu.au/content/ngv%3A56389

Schmidt, D, Duncanson, K, Webster, E, Saurman, E & Lyle, D 2022, ‘Critical realist exploration of long-term outcomes, impacts and skill development from an Australian Rural Research Capacity Building Programme: a qualitative study’, BMJ Open, vol. 12, no. 12, pp. e065972. https://doi.org/10.1136/bmjopen-2022-065972

Schmidt, D, Robinson, K & Webster, E 2014, ‘Factors influencing attrition from a researcher training program’, International Journal for Researcher Development , vol. 5, no. 1, pp. 56-67.

Schmidt, DD & Kirby, S 2016, ‘A modular approach to rural and remote research education: a project report’, Rural and Remote Health, vol. 16, no. 1, pp. 3609.

Schmidt, DD, Webster, E & Duncanson, K 2019, ‘Building research experience: impact of a novice researcher development program for rural health workers’, Australian Journal of Rural Health, vol. 27, no. 5, pp. 392-7. https://doi.org/10.1111/ajr.12520

Smith, H, Wright, D, Morgan, S, Dunleavey, J & Moore, M 2002, ‘The 'Research Spider': a simple method of assessing research experience’, Primary Health Care Research and Development , vol. 3, no. 3, pp. 139-40. https://doi.org/10.1191/1463423602pc102xx

Smith, P & Blake, D 2005, Facilitating learning through effective teaching . NCVER, Adelaide.

Smith, P & Dalton, J 2005, Getting to grips with learning styles . NCVER, Adelaide.

Smyth, T 2014, Rural research capacity building program review. GAP Ltd., Sydney.

Spouse, J 2001, ‘Work- based learning in health care environments’, Nurse Education in Practice ,vol. 1, no. 1, pp. 12-8.

Tritton, B 2008, ‘Competency-based learning in higher education’ PhD thesis, Victoria University, viewed 4 November, 2024, https://vuir.vu.edu.au/2014/1/Brian_Tritton.pdf

Webster, E, Thomas, M, Ong, N & Cutler, L 2011, ‘Rural research capacity building program: capacity building outcomes’, Australian Journal of Primary Health ,vol.17, no. 1, pp. 107-13.

Wheelahan, L & Carter, R 2001, ‘National training packages: a new curriculum framework for vocational education and training in Australia’, Education + Training,vol. 43, no. 6, pp. 303-16.

Williams, BW, Byrne, PD, Welindt, D & Williams, MV 2016, ‘Miller's pyramid and core competency assessment: a study in relationship construct validity’, Journal of Continuing Education in the Health Professions , vol. 36, no. 4, pp. 295-9.

Young, M 2002, ‘Modularization and the outcomes approach: towards a strategy for a curriculum of the future', in J Burke (ed.) Outcomes, Learning and the Curriculum, Routledge.

Supplementary Material

This supplementary material was part of the submitted manuscript and is presented as supplied by the authors.

Appendix: Supplement A: Core Curriculum for the Rural Research Capacity Building Program

Competencies and Core Curriculum for the Rural Research Capacity Building Program (RRCBP)

This document outlines the aims, competencies and curriculum of HETI’s Rural Research Capacity Building Program.

A review of the program conducted by Smyth in May 2014 contained a series of recommendations have led to a review of the program including the development of a ‘core curriculum’, the implementation of a more phased approach to focus on the establishment of linkages to research networks within NSW Health as well as supporting the delivery of content necessary to enable the candidates to complete their research projects.

This document will outline the approach of developing competencies and delivering content that will be required of novice researcher in the context increasing research capacity in a rural clinical or health services setting.

Core Competencies and Curriculum

Background and Theoretical Framework

The Rural Research Capacity Building Program aims to increase the number and range of clinicians with knowledge and skills in rural health care evaluation and research. The program intends to develop rural based researchers:

• With knowledge, skills and abilities in applied research and evaluation;
• With the ability to address health care issues in policy and practice; and
• Who can work collaboratively with practitioners, consumers, and researchers from other disciplines

It should be noted that the program considers ‘research’ to include a wide range of scholarly investigation including clinical research and other studies within a variety of health services.

Candidates who complete the program are encouraged, but are not required, to undertake further accredited higher education studies. It is expected that this program will help them continue their interest and involvement in health research.

The Rural Research Capacity Building program seeks to:

• increase the number of rural clinicians in NSW able to undertake and contribute to research;
• encourage participants to continue to be actively engaged in research, including in the higher education sector;
• build a critical mass to strengthen rural health research; and
• contribute to translate research into improving clinical practice, health services and rural community health outcomes.

The Rural Research Capacity Building Program currently provides workshops designed to provide novice researchers with the competencies and skills to complete their own research projects. The workshops focus on providing a general level of research knowledge that enable the candidates to be aware of a broad range of clinical and health services research approaches and should result in effective consumers of research papers. At the end of these workshops, candidates should be able to initiate their own research and should have started to establish a support network to support their research. The material covered within the workshops, therefore, should provide the basics necessary to set a firm foundation that will provide the novice researchers with the capacity to develop research within their LHDs and their clinical and health services practices.

In conjunction to the material covered in the workshops, candidates are provided supplement training through individual mentorship specifically focused on their research projects. The content covered in these sessions is not included in this core curriculum but are part of the skills that are provided through the RRCBP through mentors, the consultant biostatistician and program officers.

Almost concurrently, candidates work on their own research project and therefore apply their workshop training. Candidates define and refine their research proposals, create the research protocol, get their projects approved by the ethics committee, collect and analyse data and create a research project report. This process, again, is supported by the program through the networking provided by the program’s infrastructure of mentors and program officers.

The purpose of any research capacity building program is to provide novice researchers with the skills and support in order to enable them to become productive and proficient researchers. Cooke (2005) has define the six principles of a research capacity building program as

• Skills and confidence building
• Close to practice
• Linkages and collaboration
• Appropriate dissemination
• Continuity and sustainability
• Infrastructure.

In the case of HETI’s Rural Research Capacity Building program many of these principles have been inherently included. Of note, mentors are assigned to each candidate to help foster the skills and confidence. Research projects have been developed to suit the candidate’s professional background and have been supported by their Local Health Districts. Networking has been encouraged. Summary reports have been disseminated through the HETI Websites. Candidates have been supported financially through backfill and educationally through workshops and ongoing delivery of online and distance support of the HETI Research Support Officers.

Given this, the competency framework and the curriculum has been developed and adapted to include the strengths of the current program and to enhance Cooke’s six principles. Changes include a greater focus on establishing linkages and collaboration with NSW Health’s six pillars, validation of the core competencies covered, increased communication and dissemination of research projects, and an emphasis on the unique aspects of rurality.

As stated, the aim provides the highest level goals of the program and the expected outcomes of the program. The intent of the competency framework is to bridge the high level aims of the RRCBP to the concrete curriculum. The competency framework states the generic goals of developing good novice researchers and provides a definition of the traits that we hope to imbue in our successful graduates. This then links to the core curriculum which outlines the actual topics that each candidate will be presented and those content areas that are expected to be attained on completion.

The curriculum includes topics that were previously covered during the short course in 2013. This content is appropriate for developing the novice researcher providing them with the fundamentals of research including both quantitative and qualitative approaches and basic concepts of bias, etc.

In order to measure the level to which candidates meet the curriculum goals, Miller’s Pyramid (1990) can be applied. At minimum, candidates will be expected to gain a level of cognition (or “knows” according to Miller’s Pyramid) for all curriculum content areas. Through working with their research project, it is expected that some of the candidates will reach the ability to apply curriculum areas (or “shows” or “does” according to Miller). The level of research skills per candidate can be quickly assessed by reviewing the curriculum with Miller’s pyramid in mind. This can be assessed with a table including the curriculum content areas in the first column and “Knows”, “Knows How”, “Shows” and “Does” across the other columns.

Figure 1: Miller’s Pyramid (2005)

Competency Framework

The Competency Framework is a high level refinement of the program aims and is intended to provide a framework to guide teachers and candidates toward successful outcomes.

In order to foster their potential as rural researchers, the following core competencies are critical to the success of the candidates:

1. Understanding the role of research in health and health systems
2. Designing Research
3. Conducting Research
4. Communicating Research

Understanding the role of research in health and health systems

Novice researchers need to be able to better understand the context of their research. Given the intent of the program is to foster rural research and that candidates work in and are supported by their LHDs. Further, novice researchers need to be aware of the networks that are available around them to support and fund research and the activity of research.

Designing Research

This competency includes developing a viable research questions, scoping the research project and confirming the research question. Research should be developed with a valid and potentially viable study methodology.

Conducting Research

Successful researchers are able to operationalize and undertake research in order to answer their research questions. As part of this, researchers are able to determine the most appropriate method of study, acquire funding and support for implementing the study, organize study participants as appropriate, and collect, analyse and interpret data.

Communicating Research

Upon completion of the research project, the dissemination of the research is critical to a successful researcher. This includes verbally presenting research outcomes within the LHD, academic conferences and other platforms and written presentations of their research as publications in academic journals and other forums.

Core Curriculum

The curriculum outlines the core content areas that all candidates of the RRCBP will cover throughout the program. It is assumed that at the initial stage of the candidate’s training, the workshops will provide the candidates with the basic knowledge and awareness of these research techniques (as per the first step of ‘knows’ in Millar’s Pyramid). It is expected that during the course of the RRCBP, candidates will be able to apply their knowledge of curriculum material as they proceed through their research projects.

The framework for the core curriculum is based on Smith, Wright and Dunleavey’s (2005) Research Spider. The Research Spider has been used in the RRCBP as a formative self-report mechanism that helps determine the confidence developing researcher across 10 research perspectives. In addition to these 10 categories, two additional categories were added to include the content of research “Understanding the context of Rural Clinical and Health Services Research in NSW” and consideration for mixed research methodologies. As training for novice researchers within a program, the concept of applying for research funding has been replaced by applying for Ethics Approval.

1. Understanding the context of Rural Clinical and Health Services Research in NSW

1. Understand the context of Clinical and Health Services Research and support system for rural research in NSW
2. Understand the terminology used for research in NSW
3. Compare differences between Clinical and Health Services research to compared to other research
4. Understand the importance and impact of research within the rural health system and its effect on health quality and the health care system
5. Identify stages in the health research process
6. Understand operational aspects of their local research hubs and their local ethics committees
7. Understand reporting responsibilities of their local research hubs

2. Generating Research Ideas

1. Examine current research in the candidate’s area of interest
2. Explore research being done within the ministry pillars and LHDs to determine projects that can complement and contribute to the candidate’s research activities.
3. Begin to establish linkages and partnerships key program areas in the Ministry and the NSW Health pillars:
4. Agency for Clinical Innovation (ACI)
5. Bureau of Health Information (BHI)
6. Cancer Institute NSW
7. Clinical Excellence Commission (CEC)
8. Health Education and Training Institute (HETI)
9. NSW Kids and Families
10. Be able to create a research question with a rural clinical/health services relevance that can be explored with a scientific research approach
11. Continue to foster research links with LHDs and your local practice to generate research ideas
12. Develop an appreciation for research feasibility and the appropriateness of research or other methodologies to answer a question of interest

3. Finding Relevant Literature

1. Identify resources available to them such as CIAP and Endnote basic
2. Use appropriate databases to find relevant literature that addresses the research question
3. Develop links to research resource support services such as health services librarians
4. Gain experience developing literature search strategies.

4. Critically Reviewing the Literature

1. Distinguish the basic difference between quantitative and qualitative research methods
2. Use appropriate qualitative checklists such as COREQ to assess the value of a qualitative study.
3. Use a systematic approach to establish the validity of a quantitative and qualitative research paper.
4. Given a research paper, be able to identify research approach implemented.

5. Writing a Research Protocol

1. Prepare a research protocol to undertake their desired research with a sound research design
2. Prepare a timeline for their proposed research study
3. Create a budget for their research project

6. Applying for Ethics Approval

1. Understand the requirements of submitting their research proposal for Ethics review
2. Create a research submission to ethics and, as necessary, make changes to their research implementation plan based on the suggestions from the Ethics Committee

7. Using Quantitative Research Methods

1. Describe and identify the main types of quantitative study types
1. Descriptive Research
1. Case Study
2. Naturalistic Observation
3. Survey Research Design
4. Observational Study
2. Covariance
1. Case-Control Study
2. Cohort Study
3. Longitudinal Study
4. Cross Sectional Study
5. Correlational Study
3. Semi-Experimental
1. Field Experiments
2. Quasi-Experimental Design
3. Identical Twins Study
4. Experimental
4. Experimental Design
1. True Experimental Design /
2. Randomised Control Trials
3. Double Blind Experiment
4. Factorial Design
2. Identify the major sources of bias in quantitative research
3. Understand methods of minimizing bias in research
4. Interpret basic measures of frequency and association
5. Be able to define prevalence, incidence, relative risk, risk difference and number needed to treat.
6. Interpret confidence intervals
7. Interpret p-values in research and statistical reporting
8. Define and differentiate accuracy and precision
9. Use appropriate descriptive statistics to describe a dataset
10. Apply tables, charts and graphs to communicate research findings
11. Understand how to use and optimise surveys and questionnaires for research

8. Qualitative Research Methods

1. Consider what qualitative research is and what it is useful for
2. Discuss some of the core concepts in qualitative research
3. Describe how ontology, epistemology, methodology and methods combine with theoretical frameworks to inform all stages of a qualitative study
4. Describe procedural and theoretical rigour and explain why these concepts are central to qualitative research
5. Discuss why reflexivity is preferred to bias in qualitative research
6. Choosing from phenomenology, feminist or indigenist design, participatory action research, ethnography, grounded theory, or Delphi technique, describe two types of theoretical frameworks used in qualitative research
7. Become familiar with some sampling frameworks used in qualitative research including maximum variation sampling, extreme or deviant case sampling, homogenous sampling, typical case sampling, snowball sampling.
8. Become familiar with methods of data collection common in qualitative research including interviewing, focus groups, photo voice, observation, field notes and drawing.
9. Practice preparing for, collecting and coding data
10. Describe approaches to coding qualitative data
11. Outline the purpose of memo writing in qualitative research

9. Mixed Methods

1. Describe the use of mix methods approaches to research and evaluation

10. Analysing and Interpreting Results

See quantitative, qualitative and mixed methods

11. Writing and Presenting a Research Report

1. Write a cogent literature review
2. Analyse and interpret research findings
3. Develop an appreciation for research feasibility and the appropriateness of research or other methodologies to answer a question of interest
4. Present their findings to a colleague
5. Present their findings to a group of peers
6. Present their findings to their LHDs

12. Publishing and Communicating Research

*NOTE the research spider only mentions publishing but our competency framework suggest that ‘communicating’ is more relevant for the program (perhaps this should be joined with the section above?)

1. Select a suitable forum for the dissemination of their research project
2. Know the steps required to submit an article to an academic journal
3. Able to submit summary reports to HREC and HETI in a timely manner
4. Able to communicate research findings in an academic setting through written submissions
5. Able to communicate research findings in an academic setting through oral presentations

References

Cooke, J. (2005). A framework to evaluate research capacity building in health care. BMC Family Practice, 6(44), 1471-2296.

Miller GE (1990). The assessment of clinical skills/ competence/ performance. Acad Med, 65:s63-s67.

NSW Health Department. (2001). A Framework for Building Capacity to Improve Health.

Smith, H., Wright, D., & and Dunleavey, J. (2002). The 'Research Spider': a simple method of assessing research experience. Primary Health Care Research and Development, 3, 139-140.