Competency assessment of Pediatrics Residents in an Amazonian public health institution with a multi-source method
DOI:
https://doi.org/10.33448/rsd-v12i3.40761Keywords:
Medical residency; Medical education; Educational evaluation.Abstract
Objective: To validate and introduce a method for assessing competencies in Pediatrics for residents of the Medical Residency Program in Pediatrics of a Public Health Institution in the Amazon. Methods: This is a cross-sectional and analytical study with a quali-quantitative approach with teachers, preceptors, pediatrics residents, nurses and patients' families in three sectors of the Pediatrics Medical Residency Program, between April and July 2018. Results: The instruments were validated by seven judges with an average of 15 years in the teaching profession; 139 evaluations were obtained from 26 residents, mostly women (80.8%) and first year of training (R1); 33.8% by teachers/preceptors and nurses (Mini-CEX instrument), 38.2% by family members and 28% by residents through self- and/or peer evaluation. For feedback, there was no record in the mini-CEX and, among family members, there was a predominance of compliments, as well as in self/peer evaluations. Better evaluations occurred by peers regarding self-assessments on the topics: psychosocial aspects of the disease and medical knowledge (R1); and management in outpatient care, management of multiple complex problems, and management of hospitalized patients (R2). In the assessment of family members, the R2 scored higher in two items than the R1: talks all about his disease; as for the miniCEX, the R2 performed better in clinical skills, clinical reasoning, anamnesis and physical examination. Reliability in the three instruments, with Cronbach's alpha coefficient: self/peers assessment (0.72), miniCex (0.9) and family members' assessment (0.86). Conclusion: Use of the method allowed qualities to be observed in the individual and collective context of this group of professionals.
References
Almeida, M. M. G. de., & Amaral, C. G. (2021). Feedback formativo e aprendizagem do aluno de Medicina no contexto pediátrico: uma revisão narrativa da literatura. Interface - Comunicação, Saúde, Educação. Interface (Botucatu).
Ansari, A. A., Donnon, T., Khalifa, K. A., Darwish, A & Violato, C. (2014). The construct and criterion validity of the multisource feedback process to assess physician perfomance: a meta-analysis. Advances in Medical Education and Practice, 5, 39-51.
Antunes, M. M. A. (2014). Técnica Delphi: metodologia para pesquisas em educação no Brasil. Rev. educ. PUC-Camp,19 (1), 63-71.
Bogetz, A. L., Rassbach, C. E., Chan, T & Blankenburg, R. L (2017). Exploring the Educational Value of Patient Feedback: A Qualitative Analysis of Pediatric Residents’ Perspectives. Academics Pediatrics, 17 (1), 4-8.
Borges, M. C, Miranda, C. H., Santana, R. C & Bollela, V. R (2014). Avaliação formativa e feedback como ferramenta de aprendizado na formação de profissionais da saúde. Medicina, 47 (3), 324-331.
Brasil. Ministério da Educação (2003). Resolução Comissão Nacional de Residência Médica no 004/2003.
Brasil. Ministério da Educação (2016). Resolução Comissão Nacional de Residência Médica no 01/2016, de 30 de dezembro de 2016. Dispõe sobre os requisitos mínimos do Programa de Residência Médica em Pediatria e dá outras providencias. Diário Oficial da União no 251 30 dez 2016; Seção 1.
Brinkman, W. B., Geraghty, S. R., Lanphear, B. P., Khoury, J. C., Gonzalez & Dewitt, T. G. (2007). Effect of multisource feedback on resident communication skills and professionalism: a randomized controlled trial. Arch Pediatr Adolesc Med, 161 (1), 44 – 49
Chandler, N., Henderson, G., Park, B., Byerley, J., Brown, W. D & Steiner, M. J. (2010). Use of a 360-degree evaluation in the outpatient setting: The usefulness of nurse, faculty, patient/family, and resident self-evaluation. Journal of Graduate Medical Education, 2 (3), 430-434.
Colthart, I., Bagnall, G., Evans, A., Allbutt, H., Haig, A &, Illing, J. (2008). The effectiveness of self-assessment on the identification of learner needs, learner activity, and impact on clinical practice: BENE Guide n10. Medical teacher, 30, 124-145
Davis, D. A., Mazmanian, P. E., Fordis, M. & Van, H. (2006). Accuracy of Physician Self-Assessment Compared with Observed Measures of Competence: A Systematic Review. JAMA, 296 (9), 1094-1102.
Donnon, T., Ansari, A & Violato, C. (2014). The Reliability, Validity, and Feasibility of Multisource Feedback Physician Assessment: A Systematic Review. Academic Medicine, 89 (3).
Estrela, C. (2018). Metodologia Científica: Ciência, Ensino, Pesquisa. Editora Artes Médicas.
Evans, R. G., Edwards, A., Evans, S. & Elwyn, G. (2007). Assessing the practising physician using patient surveys: A systematic review of instruments and feedback methods. Fam Pract, 24 (2), 117–127.
Fabricio, T. N. B. (2018). Experiência de implantação da avaliação 360 graus e feedback entre residentes de ginecologia e obstetrícia; 2018. Mestrado [Dissertação] - Universidade Federal do Rio Grande do Norte.
Feitosa, T. A. F (2015). Proposta de implantação do método avaliativo de competências Clínicas (MINI-CEX) na residência médica em doenças infecciosas da Universidade Federal do Rio Grande do Norte – UFRN. Natal/RN. Mestrado [Dissertação] - Centro de Ciências da Saúde da Universidade Federal do Rio Grande do Norte.
Fernandes, C. R., Filho, A. F., Gomes, J. M. A., Filho, W. A. P., Cunha, G. K. F & Maia, F. L. (2012). Currículo Baseado em Competências na Residência Médica. Rev. Brasileira de Educação médica, 1 (36), 129-136.
Fernandes, B. C et al. (2020) Desenvolvimento de MiniCEX para avaliação de competências clínicas do enfermeiro em medicamentos. Mundo da Saúde,44:465-474.
Gálvez, F. G. M. (2011). Assessment of clinical competence in a pediatric residency with the Mini-Clinical Evaluation Exercise (Mini-CEX). Arch Argent Pediatr, 109 (4), n. 4, 314-320.
Goel, A & Singh, T. (2015). The usefulness of Mini Clinical Evaluation Exercise as a learning tool in different pediatric clinical settings. International Journal of Applied and Basic Medical Research, 5 (1), 32 – 34.
Goodyear, H. M., Lakshminarayana, I., Wall, D & Bindal, T. (2015). A multisource feedback tool to assess ward round leadership skills of senior paediatric trainees: (2) Testing reliability and practicability. Postgrad Med J, 91, 268–273.
Gupta, S., Sharma, M & Singh, T (2017). The Acceptability and Feasibility of Mini-clinical Evaluation Exercise as a Learning Tool for Pediatric Postgraduate Students. International Journal of Applied and Basic Medical Research,7 (1), 19 – 22.
Honji, V. Y. (2014). Avaliação de Competências Clínicas de Médicos Residentes de Urologia na Realização de Exame Urodinâmico. Mestrado [Dissertação] - Pontifícia Universidade Católica de São Paulo.
Joshi, R., Ling, F. W & Jaeger, J. (2004). Assessment of a 360-degree instrument to evaluate resident’s competency in interpersonal and communication skills. Acad Med,79, 458 – 463.
Lottenberg, M. P., Bichuette, L. D., Bortolotto, L. A., Gowdak, L. H. W., Darrieux, F. C. da C., Binotto, M. A., Kalil Filho, R., & Caramelli, B. (2022). Programa de Residência Médica em Cardiologia de Adultos do InCor em 2022: 40 Anos Formando Cardiologistas para as Demandas do Brasil. Arquivos Brasileiros De Cardiologia, 119. Arq. Bras. Cardiol., 119(5) suppl 1), 43–51.
Megale, L., Gontijo, E. D & Motta, J. A. C. (2009). Avaliação de Competência Clínica em Estudantes de Medicina pelo Mini exercício Clínico Avaliativo (Miniex). Revista Brasileira de Educação Médica, 33 (2),166–175.
Maroco, J & Marques, T. G. (2006). Qual a fiabilidade do alfa de Cronbach? Questões antigas e soluções modernas? Laboratório de Psicologia, 4 (1), 65-90.
Moonen-van, L. J. M., Overeem, K., Govaerts, M. J., Verhoeven, B. H. & Van, D. V. C. P. (2015). The Reliability of Multisource Feedback in Competency-Based Assessment Programs: The Effects of Multiple Occasions and Assessor Groups. Academic Medicine, 90; 8.
Moreno, M. L., Santos, N. R & Leonor, M (2021). Investigação empírica sobre feedback de múltiplas fontes na avaliação de desempenho: revisão de literatura. Rev. Psicol., Organ. Trab. [online]. 21(4)
Norcini, J. J., Blank, L. L., Arnould, G. K & Kimball, H. R. (1995). The Mini-CEX (Clinical Evaluation Exercise): A preliminar Investigation. Ann Intern Med, 123 (10), 795 – 799.
Norcini, J. J., Blank, L. L., Arnould, G. K & Kimball, H (1997). Examiner differences in the mini-CEX. Adv Health Sci Educ Theory Pract, 2 (1), 27-33.
Norcini, J. J., Blank, L. L & Duffy, F. D. (2003). The mini-CEX: a method for assessing clinical skills. Ann Intern Med, 138 (6), 476 – 481.
Oliveira, C. L. B et al. (2020). MiniCex como Instrumento para Avaliação de Programa no Internato de um Curso de Medicina. Revista Brasileira de Educação Médica, 44 (1).
Oliveira, D. P. (2014). Associação do Miniex e prova teórica como método de avaliação para residentes de Ortopedia. Mestrado [Dissertação] - Pontifícia Universidade Católica de São Paulo.
Poulos, A & Mahony, M. J. (2008). Effectiveness of feedback: the students’ perspective. Assessment & Evaluation in Higher Education, 33 (2),143–154.
Ponte, M. F., Farias, L. A. B. G., Carneiro, R. J., Sales, I. C. B., Oliveira, O. J. N & Carvalho, M. C. (2018). Construção do currículo por competências para a Residência Médica em Clínica Médica do hospital Geral de Fortaleza. Jornal de Ciências da Saúde do Hospital Universitário da Universidade Federal do Piauí, 1 (2), 4-17.
Probyn, L., Lang, C., Tomlinson, G., & Bandiera, G. (2014). Multisource Feedback and Self-Assessment of the Communicatior, Collaborator, and Professional CanMEDS Roles for Diagnostic Radiology Residents. L. Canadian Association of Radiologists Journal, 65 (4), 379 – 384.
Qu, B., Zhao, Y., & Sun, B. Z. (2012). Assessment of Resident Physicians in Professionalism, Interpersonal and Communication Skills: A Multisource Feedback. Int J Med Sci, 3 (9), 228-236.
Raee, H., Amini, M., Nasab, A. M., Pour, A. M & Jafari, M. M. (2014). Team-based assessment of professional behavior in medical students. J Adv Med Educ Prof, 2 (3), 126-130.
Reinders, M. E., Ryan, B. L., Blankenstein, A. H, Sterwart, M. A & Van Marwijk, H. W. (2011). The effect of patient feedback on physicians’ consultation skills: a systematic review. Acad Med, 86 (11), 1426 – 1436.
Riveros, R., Kimatian, S., Castro, P., Dhumak, V., Honar, H & Mascha, E. (2016). Multisource Feedback in professionalism for anesthesia residents. Journal of Clinical Anesthesia, 34, 32–40.
Roberts, M. J., Campbell, J. L., Richards, S. H & Wright, C. (2013). Self-other agreement in multisource feedback: The influence of doctor and rater group characteristics. Journal of Continuing Education in the Health Professions, 33 (1), 14-23.
Sargeant, J., Armson, H., Chesluk, B., Dornan, T., Eva, K & Holmboe, E. (2010). The Processes and Dimensions of Informed Self-Assessment: A Conceptual Model. Academic Medicine, 85 (7), 1212-1220.
Schiavon, J., & Detregiachi E. (2022). Mini-CEX: Revisão sobre um Método para Avaliação de Competências Clínicas na Residência Médica. Bol Curso Med, 8(2).
Senger, M. H & Campos, M. C. G. (2016). Matrizes para a Aquisição de Competências no Ensino de Urgências Clínicas / Ensino de Urgências Orientado por Competências. Revista Brasileira de Educação Médica, 40 (1), 172 – 182.
Souza, R. G. S. (2012). Atributos Fundamentais dos Procedimentos de Avaliação. In: Tiberfo IFLC, Daud-Gallotti RM, Trocon LEA, Martins MA. Avaliação Prática de Habilidades Clínicas em Medicina. São Paulo:Editora Atheneu; 2;11.
Sousa, J. C. V. (2015). Implementação da avaliação 360 graus do residente no Programa de Residência Médica em Cardiologia e Clínica Médica do Hospital Universitário Onofre Lopes (HOUL). Mestrado [Dissertação] - Universidade Federal do Rio Grande do Norte.
Sharma, R., Jain, A., Gupta, N., Garg, S., Batta, M & Dhir, S. K. (2016). Impact of self-assessment by students on their learning. Int J Appl Basic Med Res, 6 (3), 226-229.
Sorg, J. C., Wilson, R. D., Perzynski, A. T., Tran, D & Vargo, M. M. (2012). Simplifying the 360-degree peer evaluation in a physical medicine and rehabilitation residency program. Am J Phys Med Rehabil, 91, 797-803.
Tariq, M., Boulet, J., Motiwala, A., Sajjad, N & Ali, S. K. (2014). A 360-Degree Evaluation of the Communication and Interpersonal Skills of Medicine Resident Physicians in Pakistan. Education for Health, 27 (3), 269-276.
Vázquez, M., García, L., Pinchak, C., Peluffo, G., Grille, S & Weinberger, M. (2018). Evaluación de las competencias clínicas de estudiantes de Posgrado de Pediatría utilizando el Mini-Clinical Evaluation Excercise (Mini-CEX). Arch Pediatr Urug, 89 (1), 21-25.
Watling, C. J & Lingard, l. (2012). Toward meaningful evaluation of medical trainees: the influence of participants' perceptions of the process. Adv Health Sci Educ Theory Pract, 17 (2), 183-194.
Zhao, Y., Zhang, X., Chang, Q & Sun, B. (2013). Psychometric Characteristics of the 360o Feedback Scales in Professionalism and Interpersonal and Communication Skills Assessment of Surgery Residents in China. Journal of Surgical Education, 70 (5), 628-635.
Zehra, T., Tariq, M., Ali, S. K., Motiwala, A & Boulet, J. (2015). Challenges of providing timely feedback to residents: Faculty perspectives. J Pak Med Assoc, 65 (10), 1069 – 1074.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Kamylly Reina Carneiro de Mendonça; Leonardo Teixeira de Mendonça ; Lizomar de Jesus Maues Pereira Moia ; Mariane Cordeiro Alves Franco; Aurimery Gomes Chermont
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
1) Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
2) Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
3) Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
