Evaluating structured assessment of anaesthesiologists' non-technical skills

Research output: Contribution to journal › Journal article › Research › peer-review

Standard

Evaluating structured assessment of anaesthesiologists' non-technical skills. / Jepsen, R M H G; Dieckmann, Gerhard Peter; Spanager, L; Lyk-Jensen, H T; Konge, L; Ringsted, C; Østergaard, D.

In: Acta Anaesthesiologica Scandinavica, Vol. 60, No. 6, 07.2016, p. 756-766.

Research output: Contribution to journal › Journal article › Research › peer-review

Harvard

Jepsen, RMHG, Dieckmann, GP, Spanager, L, Lyk-Jensen, HT, Konge, L, Ringsted, C & Østergaard, D 2016, 'Evaluating structured assessment of anaesthesiologists' non-technical skills', Acta Anaesthesiologica Scandinavica, vol. 60, no. 6, pp. 756-766. https://doi.org/10.1111/aas.12709

APA

Jepsen, R. M. H. G., Dieckmann, G. P., Spanager, L., Lyk-Jensen, H. T., Konge, L., Ringsted, C., & Østergaard, D. (2016). Evaluating structured assessment of anaesthesiologists' non-technical skills. Acta Anaesthesiologica Scandinavica, 60(6), 756-766. https://doi.org/10.1111/aas.12709

Vancouver

Jepsen RMHG, Dieckmann GP, Spanager L, Lyk-Jensen HT, Konge L, Ringsted C et al. Evaluating structured assessment of anaesthesiologists' non-technical skills. Acta Anaesthesiologica Scandinavica. 2016 Jul;60(6):756-766. https://doi.org/10.1111/aas.12709

Author

Jepsen, R M H G ; Dieckmann, Gerhard Peter ; Spanager, L ; Lyk-Jensen, H T ; Konge, L ; Ringsted, C ; Østergaard, D. / Evaluating structured assessment of anaesthesiologists' non-technical skills. In: Acta Anaesthesiologica Scandinavica. 2016 ; Vol. 60, No. 6. pp. 756-766.

Bibtex

@article{f18d3239ec844de1b86a2709a58c16c5,

title = "Evaluating structured assessment of anaesthesiologists' non-technical skills",

abstract = "BACKGROUND: Non-technical skills (NTS) are essential for safe and efficient anaesthesia. Assessment instruments with appropriate validity evidence can be used to ensure that anaesthesiologists possess the NTS necessary to deliver high-standard patient care. The aims were to collect validity evidence using a contemporary validity framework for the assessment instrument Anaesthesiologists' Non-Technical Skills in Denmark (ANTSdk) regarding response process and internal structure (including reliability), and to investigate the effect of rater training on these properties.METHODS: An explorative study was undertaken at the Danish Institute for Medical Simulation, Copenhagen, Denmark. In a 1-day session, using ANTSdk, a convenience sample of 19 anaesthesiologists rated trainee anaesthesiologists' NTS in nine video-recorded simulation scenarios before and after a 3-h training session.RESULTS: Response process evidence: participants considered ANTSdk useful and feasible for NTS assessment. Internal structure evidence: inter-rater reliability (single measures) largely expressed substantial agreement (ICC ≥ 0.55 and ICC ≥ 0.60 for pre- and post-training ratings respectively). Strong internal consistency of ratings was found (Spearman's correlation coefficient ≥ 0.82). Accuracy of participants' ratings compared with reference ratings (± 1 scale point) was notable (76% and 78% for pre- and post-training ratings, respectively). The results indicate that the elements 'Demonstrating self-awareness', 'Reassessing decisions', 'Assessing competencies', and 'Supporting others' need more attention in future rater training.CONCLUSION: The validity evidence collected on content, response process, and internal structure, suggests that ANTSdk is easy to use on video-recorded simulation scenarios, indicating that ANTSdk is a feasible instrument for NTS assessment during anaesthesia training.",

keywords = "Anesthesiologists, Anesthesiology, Clinical Competence, Denmark, Female, Humans, Male, Reproducibility of Results",

author = "Jepsen, {R M H G} and Dieckmann, {Gerhard Peter} and L Spanager and Lyk-Jensen, {H T} and L Konge and C Ringsted and D {\O}stergaard",

note = "{\textcopyright} 2016 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd.",

year = "2016",

month = jul,

doi = "10.1111/aas.12709",

language = "English",

volume = "60",

pages = "756--766",

journal = "Acta Anaesthesiologica Scandinavica",

issn = "0001-5172",

publisher = "Wiley-Blackwell",

number = "6",

}

RIS

TY - JOUR

T1 - Evaluating structured assessment of anaesthesiologists' non-technical skills

AU - Jepsen, R M H G

AU - Dieckmann, Gerhard Peter

AU - Spanager, L

AU - Lyk-Jensen, H T

AU - Konge, L

AU - Ringsted, C

AU - Østergaard, D

PY - 2016/7

Y1 - 2016/7

N2 - BACKGROUND: Non-technical skills (NTS) are essential for safe and efficient anaesthesia. Assessment instruments with appropriate validity evidence can be used to ensure that anaesthesiologists possess the NTS necessary to deliver high-standard patient care. The aims were to collect validity evidence using a contemporary validity framework for the assessment instrument Anaesthesiologists' Non-Technical Skills in Denmark (ANTSdk) regarding response process and internal structure (including reliability), and to investigate the effect of rater training on these properties.METHODS: An explorative study was undertaken at the Danish Institute for Medical Simulation, Copenhagen, Denmark. In a 1-day session, using ANTSdk, a convenience sample of 19 anaesthesiologists rated trainee anaesthesiologists' NTS in nine video-recorded simulation scenarios before and after a 3-h training session.RESULTS: Response process evidence: participants considered ANTSdk useful and feasible for NTS assessment. Internal structure evidence: inter-rater reliability (single measures) largely expressed substantial agreement (ICC ≥ 0.55 and ICC ≥ 0.60 for pre- and post-training ratings respectively). Strong internal consistency of ratings was found (Spearman's correlation coefficient ≥ 0.82). Accuracy of participants' ratings compared with reference ratings (± 1 scale point) was notable (76% and 78% for pre- and post-training ratings, respectively). The results indicate that the elements 'Demonstrating self-awareness', 'Reassessing decisions', 'Assessing competencies', and 'Supporting others' need more attention in future rater training.CONCLUSION: The validity evidence collected on content, response process, and internal structure, suggests that ANTSdk is easy to use on video-recorded simulation scenarios, indicating that ANTSdk is a feasible instrument for NTS assessment during anaesthesia training.

AB - BACKGROUND: Non-technical skills (NTS) are essential for safe and efficient anaesthesia. Assessment instruments with appropriate validity evidence can be used to ensure that anaesthesiologists possess the NTS necessary to deliver high-standard patient care. The aims were to collect validity evidence using a contemporary validity framework for the assessment instrument Anaesthesiologists' Non-Technical Skills in Denmark (ANTSdk) regarding response process and internal structure (including reliability), and to investigate the effect of rater training on these properties.METHODS: An explorative study was undertaken at the Danish Institute for Medical Simulation, Copenhagen, Denmark. In a 1-day session, using ANTSdk, a convenience sample of 19 anaesthesiologists rated trainee anaesthesiologists' NTS in nine video-recorded simulation scenarios before and after a 3-h training session.RESULTS: Response process evidence: participants considered ANTSdk useful and feasible for NTS assessment. Internal structure evidence: inter-rater reliability (single measures) largely expressed substantial agreement (ICC ≥ 0.55 and ICC ≥ 0.60 for pre- and post-training ratings respectively). Strong internal consistency of ratings was found (Spearman's correlation coefficient ≥ 0.82). Accuracy of participants' ratings compared with reference ratings (± 1 scale point) was notable (76% and 78% for pre- and post-training ratings, respectively). The results indicate that the elements 'Demonstrating self-awareness', 'Reassessing decisions', 'Assessing competencies', and 'Supporting others' need more attention in future rater training.CONCLUSION: The validity evidence collected on content, response process, and internal structure, suggests that ANTSdk is easy to use on video-recorded simulation scenarios, indicating that ANTSdk is a feasible instrument for NTS assessment during anaesthesia training.

KW - Anesthesiologists

KW - Anesthesiology

KW - Clinical Competence

KW - Denmark

KW - Female

KW - Humans

KW - Male

KW - Reproducibility of Results

U2 - 10.1111/aas.12709

DO - 10.1111/aas.12709

M3 - Journal article

C2 - 26988291

VL - 60

SP - 756

EP - 766

JO - Acta Anaesthesiologica Scandinavica

JF - Acta Anaesthesiologica Scandinavica

SN - 0001-5172

IS - 6

ER -

ID: 173672276