General practice registrars’ clinical uncertainty, and in-consultation information- and assistance-seeking

Parker Magin ¹, Amanda Tapley ² , Georga Cooke¹, Susan Wearne ⁴ , Elizabeth Holliday ¹ , Simon Morgan⁵, Jean Ball ⁶ , Neil Spike ⁷ , Mieke van Driel ³

Abstract

Purpose: To explore the association of Australian general practitioner (GP) registrars’ responses to uncertainty with their in-consultation information-, advice- and assistance-seeking.

Design/methodology/approach: A cross-sectional analysis of data from the Registrar Clinical Encounters in Training (ReCEnT) cohort study in four Australian states. In ReCEnT, GP registrars record details of 60 consecutive consultations, six-monthly, three times during training.

Outcome factors in logistic regression models included whether the registrar sought in-consultation information or assistance from (i) their supervisor or (ii) an electronic or paper-based source. Independent variables were the four independent subscales of the Physicians’ Reaction to Uncertainty (PRU) instrument, as well as registrar, practice and consultation variables.

Findings: 589 registrars contributed details of 70,412 consultations.

On multivariable analysis, scores on the two ‘affective’ PRU subscales ‘anxiety regarding diagnosis/management’ (OR 1.03; 95% confidence intervals [CIs] [1.01, 1.05], p = 0.003) and ‘concern about a bad outcome’ (OR 1.03; 95% CIs [1.01, 1.06], p = 0.008) were significantly associated with seeking supervisor assistance. There was no association with ‘behavioural’ subscales ‘reluctance to disclose uncertainty to patients’ and ‘reluctance to disclose mistakes to physicians’.

None of the PRU subscales were significantly associated with information-seeking from electronic or hard copy sources.

Research implications: Further research is required to explore the role of uncertainty within registrar–supervisor interactions and to define the role of supervisors in registrars’ functional adaptation to clinical uncertainty (including how best to support and train supervisors in this role).

Practical implications: GP registrars’ ‘affective’ responses to clinical uncertainty are associated with assistance-seeking from clinical supervisors. While in-consultation assistance-seeking may promote registrars’ tolerance of uncertainty, it may also contribute to supervisor workload.

Originality/value: This is the first study to examine trainees’ levels of uncertainty and their seeking of information and assistance.

Limitations: We have not investigated whether registrars’ seeking assistance resolved or attenuated, for the index problem, their anxiety or concern.

Keywords: Family Practice; General Practice; Education, Medical, Graduate; Uncertainty; Preceptorship; Clinical Decision-Making.

Background

Uncertainty is unavoidable in clinical practice (Domen 2016; Gerrity et al. 1992; Han, Klein & Arora 2011), particularly general practice (Gerrity et al. 1992; O’Riordan et al. 2011). Undifferentiated illness and presentation earlier in the course of illness are more common in generalist practice than in specialist settings, increasing levels of uncertainty (Alam et al. 2017). Uncertainty also arises from general practitioners ([GPs], family physicians) applying single disease guidelines in the setting of generalist care of multimorbidity (Wallace et al. 2015). Clinical uncertainty can have deleterious effects across multiple domains (Strout et al. 2018), including effects on both the clinician (influencing professional satisfaction and burnout [Bovier and Perneger 2007; Cooke et al. 2013]), and on the health system in which they practise (e.g., greater health costs [Allison et al. 1998] including increased test-ordering [Pedersen et al. 2015; van der Weijden et al. 2002]).

Management of the uncertainty intrinsic to general practice is a core clinical skill of GPs (Malterud et al. 2017), but both established GPs (Stone 2014) and GP registrars (vocational trainees/residents in general practice) struggle with the effects of uncertainty (Cooke et al. 2013; Danczak & Lea 2014). Registrars’ tolerance of uncertainty may influence decisions to seek information and assistance, including during consultations (Sturman, Jorm & Parker 2020). Answering clinical questions generated during clinical consultations is a vital aspect of patient care (Del Fiol, Workman & Gorman 2014; Ely, Burch & Vinson 1992). Failure to find answers to the questions may lead to suboptimal patient care decisions (Del Fiol, Workman & Gorman 2014). Some questions can be pursued post-consultation but some must be answered in-consultation (Gonzalez-Gonzalez et al. 2007). Given the breadth of their practice, generalist clinicians have a particular need for answering in-consultation clinical questions, and GPs when using online resources are more likely to seek answers to patient-related questions than are specialist physicians (Bennett et al. 2005).

As well as informing immediate patient care, answering in-consultation clinical questions is a rich source of clinician learning (Brown et al. 2018; Phillips & Glasziou 2008). Registrars are early-career generalist clinicians and have limited expertise and experience, needing ‘real-time’ answers to address knowledge gaps in immediate patient care (Brown et al. 2018; Phillips & Glasziou 2008). They also have an overarching educational need to improve their clinical knowledge levels and move towards competence in independent practice (Brown et al. 2018).

In many countries, GP registrars or trainees learn within an apprenticeship-like model whereby they undergo a structured program of centralised education (in Australia, a minimum total of 125 hours in the first year of training). However, most learning takes place in individual (mainly small, geographically dispersed) general practices under the supervision of designated experienced GP supervisors (Thomson et al. 2011; Wearne et al. 2012). Australian GP registrars practise with considerable clinical autonomy, but have recourse to advice or assistance from their supervisor if requested. The responsibility for initiation of this assistance lies with the registrar (Brown et al. 2018).

Registrars seek answers in-consultation to clinical questions (in 21% of consultations) more often than do established GPs (Magin et al. 2015). The most common sources of information or advice are the supervisor (9.2% of consultations [Morgan et al. 2015]; 6.9% of individual problems managed [Magin et al. 2015; Morgan et al. 2015]) and electronic sources (6.5% of problems managed [Magin et al. 2015]). Supervisors are preferentially consulted for more complex problems (Magin et al. 2015). Appropriateness of advice- and assistance-seeking has implications for registrar learning, patient safety, and efficient use of resources (supervisor time) (Ingham et al. 2020; Morrison et al. 2015; Partanen 2018).

It is axiomatic that seeking in-consultation answers to clinical questions entails some element of uncertainty on the GP registrar’s part (Clement et al. 2015). It is also plausible that the registrar’s individual response to clinical uncertainty influences decisions to seek information and assistance. In this study, we sought to establish the association of registrars’ responses to uncertainty with their in-consultation information-, advice- and assistance-seeking.

Methods

This paper provides a cross-sectional analysis of data from the Registrar Clinical Encounters in Training (ReCEnT) study.

Recent

ReCEnT is an ongoing, multicentre cohort study of GP registrars’ in-practice clinical experiences. Data included in the current analysis were collected in four of Australia’s then 17 Regional Training Providers (RTPs) spanning four states. A total of five six-monthly rounds of data collection were conducted from 2011 to 2013. RTPs during this period were government-funded, not-for-profit, geographically defined educational and training organisations. Participants were GP registrars in general practice-based training terms.

The detailed ReCEnT methodology has been described previously (Morgan et al. 2012). Briefly, registrars complete paper-based forms recording details of 60 consecutive consultations around the midpoint of each of their three general practice training terms (six-monthly for full-time registrars) as part of their training. This exercise is part of their routine educational program, with registrars receiving detailed feedback on their recorded clinical and educational activity. As well as this educational use, registrars may also provide signed consent for research use of their data. Some registrars at one of the four RTPs also collected data during an optional fourth training term. Patient demographics, clinical details and educational actions (including in-consultation information- and assistance-seeking) are recorded for each of the 60 patient encounters per term.

Registrar and practice demographics are documented in each six-monthly collection period through a separate questionnaire. During five data collection rounds (2011–2013), clinical uncertainty scales were included in this questionnaire.

Outcome variable

The outcome variables in analyses were whether during a consultation:

i. the registrar sought advice or assistance from their supervisor or the supervisor’s delegate GP if the supervisor was unavailable (hereafter, ‘supervisor’)

ii. the registrar sought information from an electronic or paper-based source.

Main variables of interest: Responses to clinical uncertainty

The variables of interest in this analysis were scores on the Physicians’ Reaction to Uncertainty (PRU) subscales (Gerrity et al. 1995). These subscales, each ranked on a 6-point Likert scale, measure a doctor’s ‘affective’ response to uncertainty (the first two subscales) and a ‘behavioural’ response of coping in response to uncertainty (the third and fourth subscales). The PRU subscales are:

i. anxiety due to uncertainty about diagnosis/treatment: ‘anxiety’ (5 items)

ii. concern about a bad outcome for the patient: ‘concern’ (3 items)

iii. reluctance to disclose diagnosis/treatment uncertainty to patients: ‘reluctance to disclose to patients’ (5 items)

iv. reluctance to disclose mistakes to physicians: ‘reluctance to disclose to physicians’ (2 items).

Responses to each item are scored from ‘strongly disagree’ (scored 1) to ‘strongly agree’ (scored 6), with relevant items reverse scored and items summed to create total subscale scores. The subscales are independent constructs, and no overall ‘uncertainty’ score is calculated. The PRU subscales have shown good reliability and validity (Gerrity et al. 1990; Gerrity et al. 1995; Schneider et al. 2007).

Other independent variables

Other independent variables included registrar, patient, practice and consultation variables. These variables are included in Supplementary Table S1. Practice postcode was used to define the Australian Standard Geographical Classification-Remoteness Area classification (the degree of rurality) of the practice location, and to define the practice location’s Socio-Economic Index for Areas’ ‘Index of Relative Socio-Economic Disadvantage’ decile.

The unit of analysis was the individual consultation. Proportions of consultations for which (i) supervisor advice or assistance were sought and (ii) information was sought from electronic or hard copy sources were calculated with 95% confidence intervals (CIs), adjusted for clustering within registrars.

To test associations of a registrar seeking (i) supervisor advice or assistance and (ii) information from electronic or hard copy sources, simple and multiple logistic regression were used within a generalised estimating equations framework to account for clustering of patients within registrars. An exchangeable working correlation structure was assumed. Covariates with a p-value of < 0.2 on univariate analysis were included in the multivariable analyses. Covariates that had a small effect size and a p-value > 0.2 in the multivariable model were tested for removal from the model. If the covariate’s removal did not substantively change the resulting model, the covariate was not included in the final multivariable model.

We conducted separate analyses for each of the four separate PRU subscales for each of the two outcomes. For the fourth PRU subscale, ‘reluctance to disclose mistakes to physicians’, a printing error resulted in only two rounds of complete data being collected. Only these complete data were used in analyses involving this subscale.

Mean substitution was used to reduce the number of missing values for the three uncertainty scores ‘anxiety’, ‘concern’ and ‘reluctance to disclose to patients’, dependent on no more than half of the items being missing. Revised total scores were created for all outcomes using the recoded items. For ‘reluctance to disclose to physicians’, mean substitution was not employed, as there are only two items in this scale.

To assess the magnitude of associations with information- or assistance-seeking that were statistically significant, we calculated Cohen’s d as a standardised measure of effect (using univariate findings).

Analyses were programmed using STATA 13.1 and SAS V9.4. P-values < 0.05 were considered statistically significant.

Ethical approval was obtained from the University of Newcastle Human Research Ethics Committee (Ref. No. H-2009–0323).

Results

A total of 589 individual registrars (response rate 93.6%) contributed details of 70,412 individual consultations. The characteristics of the participating registrars and practices are presented in Table 1.

Table 1: Characteristics of participating registrars and participating general practices.

^* Numbers may not add up to 1184 for registrar/practice variables by term due to missing data.

^† General Practitioners (GPs)

^‡ SEIFA – Socio-economic Index for Area (Index of Disadvantage).

Advice or assistance was sought from the registrars’ supervisor in 8.8% (95% CI: 8.1–9.5) of consultations (n = 6,184). Information was sought from electronic (8.4%) or hard copy (1.7%) sources in 9.8% (95% CI: 8.8–10.7) of consultations (n = 6,869). More than one source of information or assistance could be sought in the one consultation.

Univariate analyses

The characteristics associated with seeking advice or assistance from a supervisor and seeking information from an electronic or hard copy source are presented in Supplementary Tables S1 and S2, respectively. ‘Anxiety’ (p < 0.001), ‘concern’ (p < 0.001) and ‘reluctance to disclose to patients’ (p = 0.019), but not ‘reluctance to disclose to physicians’ (p = 0.98), were significantly associated with seeking help from a supervisor on univariate analysis. ‘Anxiety’ (p = 0.002), but neither ‘concern’ (p = 0.12) nor ‘reluctance to disclose to patients’ (p = 0.84), nor ‘reluctance to disclose to physicians’ (p = 0.92), was significantly associated with seeking information from an electronic or hard copy source on univariate analysis.

Multivariable analyses

Seeking supervisor advice or assistance

The regression models including ‘anxiety’, ‘concern’, ‘reluctance to disclose to patients’ and ‘reluctance to disclose to physicians’, respectively, are presented in Tables 2, 3, 4 and 5. On multivariable analysis, ‘anxiety’ (OR 1.03; 95% CIs [1.01, 1.05], p = 0.003) and ‘concern’ (OR 1.03; 95% CIs [1.01, 1.06], p = 0.008) were significantly associated with seeking supervisor advice or assistance. ‘Reluctance to disclose to patients’ (OR 1.00; 95% CIs [0.98, 1.02], p = 0.90) and ‘reluctance to disclose to physicians’ (OR 1.01; 95% CIs [0.96, 1.06], p = 0.73) were not significantly associated with seeking supervisor advice or assistance.

Table 4: Univariate and adjusted associations withseeking advice or assistance from a supervisor and with seeking information from an electronic or hard-copy source, including associations with the scores on the Physicians’ Reaction to Uncertainty subscale ‘Reluctance to disclose diagnosis/treatment uncertainty to patients’

^* RTP – Regional Training Provider

Table 5: Univariate and adjusted associations withseeking advice or assistance from a supervisor and with seeking information from an electronic or hard-copy source, including associations with the scores on the Physicians’ Reaction to Uncertainty subscale ‘Reluctance to disclose mistakes to physicians’

*FTE – full-time equivalent

†RTP – Regional Training Provider

Seeking information from an electronic or hard copy source

The regression models including ‘anxiety’, ‘concern’, ‘reluctance to disclose to patients’ and ‘reluctance to disclose to physicians’, respectively, are also presented in Tables 2, 3, 4 and 5. On multivariable analyses, neither ‘anxiety’ (OR 1.00; 95% CIs [0.99, 1.02], p = 0.61), nor ‘concern’ (OR 1.00; 95% CIs [0.98, 1.03], p = 0.76), nor ‘reluctance to disclose to patients’ (OR 0.99; 95% CIs [0.97, 1.01], p = 0.26), nor ‘reluctance to disclose to physicians’ (OR 1.03; 95%CIs [0.98, 1.09], p = 0.20) were significantly associated with seeking information from an electronic or hard copy source.

Effect sizes

For advice- or assistance-seeking from a supervisor, Cohen’s d for ‘anxiety’ and ‘concern’ were 0.32 and 0.21, respectively.

Conclusions

Main findings

We found that ‘affective’ responses to uncertainty (‘anxiety’ and ‘concern’), but not ‘behavioural’ responses (reluctance to disclose uncertainty to patients or mistakes to physicians), were associated with registrars seeking in-consultation advice or assistance from their supervisor. The effect sizes for these associations were modest (small or small-to-moderate Cohen’s d of 0.32 and 0.21). There were no significant associations of responses to uncertainty with seeking information from electronic or hard copy sources.

We are not aware of any previous studies examining the association of clinical uncertainty and information- or assistance-seeking.

Implications for clinical and educational practice

‘Affective’ responses to uncertainty

‘Direct supervision’ is central to the registrar–supervisor educational model (Cottrell et al. 2002; Ingham et al. 2020; Partanen 2018). An initial implication of our findings is that high levels of registrar ‘affective’ responses to uncertainty create work for supervisors. How they interpret or manage this work is likely to be context dependent. In Australia, supervisors are engaged in care of their own patients concurrently with supervising registrars and have finite remunerated teaching time (Ingham et al. 2020).

This interpretation may suggest high levels of ‘affective’ responses to uncertainty may be problematic for supervisors (Sturman, Jorm & Parker 2020). However, the association may also reflect a functional response of the supervisor–registrar dyad to registrar uncertainty. Higher ‘affective’ responses with less tolerance of uncertainty in doctors (including trainees) are associated with less professional satisfaction (Bovier & Perneger 2007) and higher risk of burnout (Cooke et al. 2013). Lower tolerance of uncertainty has also been associated with generation of greater health costs (Allison et al. 1998) including increased test-ordering. As well as financial consequences, increased test-ordering has patient safety implications (Deyo 2002). A particular consideration concerning the uncertainty–anxiety nexus in trainee clinicians is that anxiety and stress can impair learning (Conrad et al. 2012; Pekrun et al. 2002). Observation of interactions between registrars and supervisors suggests registrars often seek reassurance that their plans for patients are appropriate rather than seek information per se (Brown et al. 2018). Thus, if registrars seeking in-consultation supervisor assistance were to allay anxiety and concern arising from uncertainty, there would be benefits to registrars, patients and health systems. This would be especially so if these registrar–supervisor interactions educationally addressed coping with uncertainty generically (O’Riordan et al. 2011; Sturman, Jorm & Parker 2020), as well as the specific uncertainty prompting the assistance-seeking.

Our previous analyses in this registrar population have demonstrated that seeking advice or assistance from a supervisor declines markedly as registrars progress through training (Morgan et al. 2015). The decline in seeking information from an electronic or hard copy source is not as marked (Magin et al. 2015). The causes for the decline in seeking supervisor assistance are likely to include greater experience in the general practice clinical environment (Sturman, Jorm & Parker 2020) and greater knowledge levels (leading to less uncertainty). Given our findings of an association with ‘affective’ responses to uncertainty, any declines in levels of these responses to uncertainty might also lead to less assistance-seeking. However, a further likely cause of the decline in registrars’ recourse to supervisor assistance may be ‘supply-driven’ rather than ‘demand-driven’. The time supervisors within the Australian general practice training program are remunerated for registrar teaching decreases appreciably for each term of their registrar’s training program. This creates benchmarks for approximately how much time is appropriate for registrars to require (and for supervisors to provide) at each stage of training. A schedule of reducing supervisor–registrar interaction is consistent with the need for registrars’ progression to autonomy within the apprenticeship-like model (Wearne et al. 2012) in preparation for unsupervised practice (Kennedy et al. 2005). But any mismatch in individual registrars between decreases in uncertainty and/or responses to uncertainty and reduced supervisor availability could be problematic (if supervisory support does, indeed, attenuate the negative effects of responses to uncertainty on registrars and their practice behaviours).

The lack of association of responses to clinical uncertainty with information-seeking from electronic or hard copy sources contrasts with the associations of ‘affective’ responses we found with seeking supervisor assistance. This may reflect electronic and hard copy (non-human) resources being better at addressing clinical uncertainty itself, rather than the affective responses to uncertainty. These ‘affective’ responses may be best addressed within the supportive context of the registrar–supervisor ‘community of practice’ (Clement et al. 2015; Morrison et al. 2015)—although, for some registrars this may be more comfortable out of the patient’s hearing (Sturman et al. 2020).

‘Behavioural’ responses to uncertainty

Reluctance to disclose uncertainty to patients and reluctance to disclose mistakes to physicians, as suggested by some qualitative research (Sturman, Jorm & Parker 2020), would be problematic in terms of patient safety (and registrar learning) if they led to registrars failing to seek appropriate advice or assistance from their supervisor (Kennedy et al. 2009; Partanen 2018). However, we found no evidence for such an association of ‘reluctance’ responses and seeking supervisor advice or assistance.

Addressing the problem programmatically

How specialist GP vocational training programs should address the issue of responses to clinical uncertainty is an important question. Reducing uncertainty itself is desirable. Providing access to, and training in, utilisation of clinical information sources may reduce clinical uncertainty (Axelson et al. 2007). We have found ‘affective’ responses to uncertainty to be associated with inappropriate antibiotic prescribing by GP registrars (manuscript in preparation). Reducing uncertainty (e.g., by point-of-care testing) could improve antibiotic prescribing rates (Stanton, Francis & Butler 2010). However, a certain amount of uncertainty is inevitable in medicine, and helping registrars learn to manage uncertainty and their own affective responses to uncertainty is key.

It has been noted that attenuation of responses to uncertainty with time in practice, rather than formal educational intervention, may be the essential element (White & Williams 2017). Later training terms in our registrar population are certainly associated with lower scores on the PRU (Cooke, Doust & Steele 2017), but it is unclear how much of this attenuation of PRU scores may be due to educational intervention rather than amount of in-practice experience. It is certainly proposed that more functional responses to uncertainty can be taught within educational programs (Danczak & Lea 2018; Domen 2016; O’Riordan et al. 2011; Taylor et al. 2018; Wray & Loo 2015). Particular educational methodologies have been proposed to develop tolerance of uncertainty. For example, this includes small group structured exercises designed to promote reflection (Danczak & Lea 2018). Educational methodologies have also been proposed to facilitate teaching (e.g., ‘tactical decision games’ [Drummond et al. 2016]) and assessment (e.g., script concordance testing [Lubarsky et al. 2013]) within the context of clinical uncertainty.

GP supervisors are identified as having a vital role in education around management of clinical uncertainty (O’Riordan et al. 2011; Sturman, Jorm & Parker 2020). It has also been proposed that assessment of learners’ level of responses to uncertainty (using the PRU) would facilitate education to enhance tolerance of uncertainty (Wray & Loo 2015). Even in the absence of individual–registrar-level information, our previous findings (Cooke, Doust & Steele 2017) on the demographic ‘phenotypes’ of registrars with higher levels of affective responses to uncertainty may inform educational approaches.

Implications for future research

We have established a role for ‘affective’ responses to uncertainty in registrars electing to access in-consultation advice and assistance. Further research is required to explore the role of uncertainty within the ‘social space’ of the resulting registrar–supervisor interaction (Brown et al. 2018) and to define the role of supervisors in registrars’ functional adaptation to clinical uncertainty (including how best to support and train supervisors in this role). There may be a role for research examining supervisors’ affective responses to uncertainty and how this influences the interactions of the supervisor–registrar dyad. Research could also establish if educational interventions can reduce ‘anxiety’ and ‘concern’, and what effects this would have on the registrar–supervisor relationship, including frequency of advice- or assistance-seeking.

Strengths and limitations

A strength of this study is the linking of valid measures of registrars’ responses to uncertainty (the PRU subscales) with detailed data on registrars’ in-consultation educational behaviours. The large number of relevant independent variables measured and the large sample size of consultations allowed for fine-grained multivariable analyses. The high response rate, unusual in studies of GPs (Bonevski et al. 2011), is also a strength.

A limitation is that due to a printing error, complete data for one of the four PRU subscales were available for only two rounds of data collection. A further limitation is that while we have data on how often and for what problems/diagnoses registrars seek information or assistance (and can analyse these in relation to constitutional responses to uncertainty), we do not know how satisfactorily their seeking assistance addressed their anxiety or concern in that consultation for that problem.

Conclusions

GP registrars’ ‘affective’ responses to clinical uncertainty are associated with frequency of advice- or assistance-seeking from their clinical supervisor. The registrar–supervisor relationship may help registrars respond functionally to clinical uncertainty, but increased demands on supervisors related to ‘affective’ response to uncertainty also create extra work for supervisors.

Acknowledgements

The authors thank the participating registrars and training practices for their contributions to the ReCEnT project.

Conflict of Interest

A/Prof. Susan Wearne is also an employee of the Department of Health. The views expressed in this article are her own and are not necessarily those of the Australian Government.

The other authors report no conflicts of interest.

Funding

The ReCEnT project was funded during the collection of the data presented in this paper by the participating educational organisations (General Practice Training—Valley to Coast, the Victorian Metropolitan Alliance, General Practice Training Tasmania, and Adelaide to Outback GP Training Program), which were funded by the Australian Government. From 2016 to 2019, ReCEnT was funded by an Australian Department of Health-commissioned research grant, and was supported by the GP Synergy Regional Training Organisation. GP Synergy is funded by the Australian Government.

References

Alam, R, Cheraghi-Sohi, S, Panagioti, M, Esmail, A, Campbell, S & Panagopoulou, E 2017, ‘Managing diagnostic uncertainty in primary care: A systematic critical review’, BMC Family Practice, vol. 18, article no. 79.

Allison, JJ, Kiefe, CI, Cook, EF, Gerrity, MS, Orav, EJ & Centor, R 1998, ‘The association of physician attitudes about uncertainty and risk taking with resource use in a Medicare HMO’, Medical Decision Making, vol. 18, no. 3, pp. 320–329.

Axelson, C, Wardh, I, Strender, L-E & Nilsson, G 2007, ‘Using medical knowledge sources on handheld computers: A qualitative study among junior doctors’, Medical Teacher, vol. 29, no. 6, pp. 611–618.

Bennett, NL, Casebeer, LL, Kristofco, R & Collins, BC 2005, ‘Family physicians’ information seeking behaviors: A survey comparison with other specialties’, BMC Medical Informatics & Decision Making, vol. 5, article no. 9.

Bonevski, B, Magin, PJ, Horton, G, Foster, M & Girgis, A 2011, ‘Response rates in GP surveys: Trialling two recruitment strategies’, Australian Family Physician, vol. 40, no. 6, pp. 427–430.

Bovier, PA & Perneger, TV 2007, ‘Stress from uncertainty from graduation to retirement: A population-based study of Swiss physicians’, Journal of General Internal Medicine, vol. 22, no. 5, pp. 632–638.

Brown, J, Nestel, D, Clement, T & Goldszmidt, M 2018, ‘The supervisory encounter and the senior GP trainee: Managing for, through and with’, Medical Education, vol. 52, no. 2, pp. 192–205.

Clement, T, Brown, J, Morrison, J & Nestel, D 2015, ‘Ad hoc supervision of general practice registrars as a “community of practice”: Analysis, interpretation and re-presentation’, Advances in Health Sciences Education, vol. 21, no. 2, pp. 415–437.

Conrad, C, Konuk, Y, Werner, PD, Cao, CG, Warshaw, AL, Rattner, DW, Stangenberg, L, Ott, HC, Jones, DB, Miller, DL & Gee, DW 2012, ‘A quality improvement study on avoidable stressors and countermeasures affecting surgical motor performance and learning’, Annals of Surgery, vol. 255, no. 6, pp. 1190–1194.

Cooke, G, Tapley, A, Holliday, E, Morgan, S, Henderson, K, Ball, J, van Driel, M, Spike, N, Kerr, R & Magin, P 2017, ‘Responses to clinical uncertainty in Australian general practice trainees: A cross-sectional analysis’, Medical Education, vol. 51, no. 12, pp. 1277–1288.

Cooke, GPE, Doust, JA & Steele, MC 2013, ‘A survey of resilience, burnout, and tolerance of uncertainty in Australian general practice registrars’, BMC Medical Education, vol. 13, article no. 2.

Cottrell, D, Kilminster, S, Jolly, B & Grant, J 2002, ‘What is effective supervision and how does it happen? A critical incident study’, Medical Education, vol. 36, no. 11, pp. 1042–1049.

Danczak, A & Lea, A 2014, ‘What do you do when you don’t know what to do? GP associates in training (AiT) and their experiences of uncertainty’, Education for Primary Care, vol. 25, no. 6, pp. 321–326.

Danczak, A & Lea, A 2018, ‘Developing expertise for uncertainty; Do we rely on a baptism of fire, the mills of experience or could clinicians be trained?’ Education for Primary Care, vol. 29, no. 4, pp. 237–241.

Del Fiol, G, Workman, TE & Gorman, PN 2014, ‘Clinical questions raised by clinicians at the point of care: A systematic review’, JAMA Internal Medicine, vol. 174, no. 5, pp. 710–718.

Deyo, RA 2002, ‘Cascade effects of medical technology’, Annual Review of Public Health, vol. 23, pp. 23–44.

Domen, RE 2016, ‘The ethics of ambiguity: Rethinking the role and importance of uncertainty in medical education and practice’, Academic Pathology, vol. 3.

Drummond, I, Sheikh, G, Skinner, J & Wood, M 2016, ‘Exploring the feasibility and acceptability of using tactical decision games to develop final year medical students’ non-technical skills’, Medical Teacher, vol. 38, no. 5, pp. 510–514.

Ely, JW, Burch, RJ & Vinson, DC 1992, ‘The information needs of family physicians: Case-specific clinical questions’, Journal of Family Practice, vol. 35, no. 3, pp. 265–269.

Gerrity, MS, Earp, JL, Devellis, RF & Light, DW 1992, ‘Uncertainty and professional work: Perceptions of physicians in clinical practice’, American Journal of Sociology, vol. 97, no. 4, pp. 1022–1051.

Gerrity, MS, Devellis, RF & Earp, JA 1990, ‘Physicians’ reactions to uncertainty in patient care: A new measure and new insights’, Medical Care, vol. 28, no. 8, pp. 724–736.

Gerrity, MS, White, KP, Devellis, RF & Dittus, RS 1995, ‘Physicians’ reactions to uncertainty: Refining the constructs and scales’, Motivation and Emotion, vol. 19, no. 3, pp. 175–191.

Gonzalez-Gonzalez, AI, Dawes, M, Sanchez-Mateos, J, Riesgo-Fuertes, R, Escortell-Mayor, E, Sanz-Cuesta, T & Hernandez-Fernandez, T 2007, ‘Information needs and information-seeking behavior of primary care physicians’, Annals of Family Medicine, vol. 5, no. 4, pp. 345–352.

Han, PKJ, Klein, WMP & Arora, NK 2011, ‘Varieties of uncertainty in health care: A conceptual taxonomy’, Medical Decision Making, vol. 31, no. 6, pp. 828–838.

Ingham, G, Plastow, K, Kippen, R & White, N 2020, ‘Closer supervision in Australian general practice training: Planning major system change’, Australian Journal of Primary Health, vol. 26, no. 2, pp. 184–190.

Kennedy, TJT, Regehr, G, Baker, GR & Lingard, LA 2005, ‘Progressive independence in clinical training: A tradition worth defending?’ Academic Medicine, vol. 80, no. 10, pp. S106–S111.

Kennedy, TJT, Regehr, G, Baker, GR & Lingard, LA 2009, ‘Preserving professional credibility: Grounded theory study of medical trainees’ requests for clinical support’, BMJ, vol. 338, b128.

Lubarsky, S, Dory, V, Duggan, P, Gagnon, R & Charlin, B 2013, ‘Script concordance testing: From theory to practice: AMEE guide no. 75’, Medical Teacher, vol. 35, no. 3, pp. 184–193.

Magin, P, Morgan, S, Wearne, S, Tapley, A, Henderson, K, Oldmeadow, C, Ball, J, Scott, J, Spike, N, McArthur, L & van Driel, M 2015, ‘GP trainees’ in-consultation information-seeking: Associations with human, paper and electronic sources’, Family Practice, vol. 32, pp. 525–532.

Malterud, K, Guassora, AD, Reventlow, S & Jutel, A 2017, ‘Embracing uncertainty to advance diagnosis in general practice’, British Journal of General Practice, vol. 67, no. 659, pp. 244–245.

Morgan, S, Magin, PJ, Henderson, KM, Goode, SM, Scott, J, Bowe, SJ, Regan, CM, Sweeney, KP, Jackel, J & van Driel, ML 2012, ‘Study protocol: The Registrar Clinical Encounters in Training (ReCEnT) study’, BMC Family Practice, vol. 13, article no. 50.

Morgan, S, Wearne, S, Tapley, A, Henderson, K, Oldmeadow, C, Ball, J, van Driel, M, Scott, J, Spike, N, McArthur, L & Magin, P 2015, ‘In-consultation information and advice seeking by Australian GP trainees from GP trainers: A cross-sectional analysis’, Education for Primary Care, vol. 26, no. 3, pp. 155–165.

Morrison, J, Clement, T, Nestel, D & Brown, J 2015, ‘Perceptions of ad hoc supervision encounters in general practice training: A qualitative interview-based study’, Australian Family Physician, vol. 44, no. 12, pp. 926–932.

O’Riordan, M, Dahinden, A, Akturk, Z, Ortiz, JMB, Dagdeviren, N, Elwyn, G, Micallef, A, Murtonen, M, Samuelson, M, Struk, P, Tayar, D & Thesen, J 2011, ‘Dealing with uncertainty in general practice: An essential skill for the general practitioner’, Quality in Primary Care, vol. 19, no. 3, pp. 175–181.

Partanen, R 2018, ‘Don’t miss the boat: Maximise ad hoc teaching with general practice trainees’, Medical Education, vol. 52, no. 2, pp. 143–145.

Pedersen, AF, Carlsen, AH & Vedsted, P 2015, ‘Association of GPs’ risk attitudes, level of empathy, and burnout status with PSA testing in primary care’, British Journal of General Practice, vol. 65, no. 641, pp. e845–e851. [‘Erratum’, British Journal of General Practice, 2016, vol. 66, no. 642, p. 15.]

Pekrun, R, Goetz, T, Titz, W & Perry, RP 2002, ‘Academic emotions in students’ self-regulated learning and achievement: A program of qualitative and quantitative research’, Educational Psychologist, vol. 37, no. 2, pp. 91–105.

Phillips, R & Glasziou, P 2008, ‘Evidence based practice: The practicalities of keeping abreast of clinical evidence while in training’, Postgraduate Medical Journal, vol. 84, no. 995, pp. 450–453.

Schneider, A, Szecsenyi, J, Barie, S, Joest, K & Rosemann, T 2007, ‘Validation and cultural adaptation of a German version of the Physicians’ Reactions to Uncertainty scales’, BMC Health Services Research, vol. 7, article no. 81.

Stanton, N, Francis, NA & Butler, CC 2010, ‘Reducing uncertainty in managing respiratory tract infections in primary care’, British Journal of General Practice, vol. 60, no. 581, pp. e466–e475.

Stone, L 2014, ‘Blame, shame and hopelessness: Medically unexplained symptoms and the “heartsink” experience’, Australian Family Physician, vol. 43, no. 4, pp. 191–195.

Strout, TD, Hillen, M, Gutheil, C, Anderson, E, Hutchinson, R, Ward, H, Kay, H, Mills, GJ & Han, PKJ 2018, ‘Tolerance of uncertainty: A systematic review of health and healthcare-related outcomes’, Patient Education and Counseling, vol. 101, no. 9, pp. 1518–1537.

Sturman, NJ, Jorm, C & Parker, M 2020, ‘With a grain of salt? Supervisor credibility and other factors influencing trainee decisions to seek in-consultation assistance: A focus group study of Australian general practice trainees’, BMC Family Practice, vol. 21, article no. 28.

Sturman, NJ, Tapley, A, van Driel, ML, Holliday, E, Ball, J, Davey, A, Fielding, A, Fitzgerald, K, Spike, N & Magin P 2020, ‘Configurations for obtaining in-consultation assistance from supervisors in general practice training, and patient-related barriers to trainee help-seeking: A survey study’, BMC Medical Education, vol. 20, article no. 369.

Taylor, D, Picker, B, Woolever, D, Thayer, EK, Carney, PA & Galper, AB 2018, ‘A pilot study to address tolerance of uncertainty among family medicine residents’, Family Medicine, vol. 50, no. 7, pp. 531–538.

Thomson, JS, Anderson, KJ, Mara, PR & Stevenson, AD 2011, ‘Supervision—growing and building a sustainable general practice supervisor system’, Medical Journal of Australia, vol. 194, no. 11, pp. S101–S104.

van Der Weijden, T, van Bokhoven, MA, Dinant, G-J, van Hasselt, CM & Grol, RPTM 2002, ‘Understanding laboratory testing in diagnostic uncertainty: A qualitative study in general practice’, British Journal of General Practice, vol. 52, no. 485, pp. 974–980.

Wallace, E, Salisbury, C, Guthrie, B, Lewis, C, Fahey, T & Smith, SM 2015, ‘Managing patients with multimorbidity in primary care’, BMJ , vol. 350, h176.

Wearne, S, Dornan, T, Teunissen, PW & Skinner, T 2012, ‘General practitioners as supervisors in postgraduate clinical education: An integrative review’, Medical Education, vol. 46, no. 12, pp. 1161–1173.

White, G & Williams, S 2017, ‘The certainty of uncertainty: Can we teach a constructive response?’ Medical Education, vol. 51, no. 12, pp. 1200–1202.

Wray, CM & Loo, LK 2015, ‘The diagnosis, prognosis, and treatment of medical uncertainty’, Journal of Graduate Medical Education, vol. 7, no. 4, pp. 523–527.