Q method in research on health professions education and digital technologies: a systematic review

Q method, a mixed methods research approach, is used to explore points of view and attitudes towards a specific phenomenon from subjective human perspectives. There has been an increase in the use of digital technologies in education and it has become necessary to investigate the difficulties and facilities of health professionals and students in order to improve the use of such technologies for teaching and learning. We aimed to identify and evaluate studies that employed Q method to investigate the use of digital technologies in Health Professions Education. To achieve this, a systematic review was conducted according to the PRISMA Statement Guidelines. The selection of articles was based on the search strategy (("Q-sort" OR "Q-methodology" OR "Q-technique")) AND (("Teaching") OR ("Learning")). Of the 1,398 articles found, 13 were selected in accordance with the adopted inclusion criteria. The Research, Society and Development, v. 10, n. 10, e471101019154, 2021 (CC BY 4.0) | ISSN 2525-3409 | DOI: http://dx.doi.org/10.33448/rsd-v10i10.19154 2 articles successfully applied Q method to health issues, which expands its application possibilities and provides a contribution to mixed methods research. Another contribution is the use of the Mixed Methods Appraisal Tool in this type of review. In view of the pressing need for education changes, using mixed methods research, particularly Q method, to investigate teaching culture and practice, can successfully support the renewal of Health Professions Education.


Introduction
Q methodology, referred to in this study as Q method, was created in the 1930s by William Stephenson, an English physicist and psychologist who was interested in finding a scientific way to study subjectivity (Stephenson, 1935;Watts & Stenner, 2012). It is currently considered an important method to understand the meanings lent to a set of arrangements of statements distributed among a group of participants. Based on non-positivistic philosophical and epistemological premises, it correlates response arrangements and uses by-person factor analysis to identify groups of participants who think similarly (Dune, Mengesha, Buscemi, & Perz, 2019;Novaes, 2016Novaes, , 2020Watts & Stenner, 2012).
Explaining it in a summarized way, in Q method, participants rank a set of statements about a topic in a quasi-normal distribution according to their opinions, from 'least agree' to 'most agree' (for example, from -4 or -5 to + 4 or + 5). Then, factor analysis is performed from the individual points of view (or classifications) (Qurtas & Shabila, 2020;Yau, Babovič, Liu, Gugel & Monrouxe, 2021).
Exploring scientific evidence of the use of Q method in the health area, we found studies on the relative value of life extensions for people with terminal diseases from the perspective of the general population in the United Kingdom (Mason et al., 2018;McHugh et al., 2015); essential care aspects at the end of life in the opinion of people with dementia and their caregivers (Hill et al. 2017); views of young people with chronic conditions on transition from pediatric to adult care (Hislop, Mason, Parr, Vale & Colver, 2016); and literature reviews on the advantages and disadvantages of using this methodology (Dziopa & Ahern, 2001;Simons, 2013). Cross (2005) states that Q method is a robust technique to explore individuals' beliefs and experiences in health education and health promotion research. In the field of Health Professions Education, there are examples of studies that employ this method in nursing education (e.g., Ha (2015), Lim, Wynaden, Baughman & Heslop (2021), Petit dit Dariel, (2013)). According to Ha (2015), Q method is a crucial tool to identify, assess and reflect on nursing students' experiences and attitudes towards clinical practice, providing a useful insight to facilitate the development of effective clinical teaching strategies in nursing education.
In addition, the current diversity of university students -digital natives (individuals who have grown up in the digital age) and digital immigrants (individuals who have grown up before or partially before the digital age) -has led to an increasing use of technologies in health education. The evolution of web 3.0 has changed the forms of communication, increasing network collaboration skills, allowing self-guided and individualized learning for students, and enabling teachers to engage in short, succinct, fast, and frequent interactions with students (Chicca & Shellenbarger, 2018;Hays, 2018;Rocha & Sampaio, 2020;Sampson & Karagiannidis, 2002).
Conducted for methodological purposes (Munn, Stern, Aromataris, Lockwood & Jordan, 2018;Vidal & Fukushima, 2021), our study aimed to carry out a systematic review in order to identify and evaluate studies that employed Q method to investigate the use of digital technologies in Health Professions Education.

Study design
This systematic review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Moher et al., 2009). It is also in line with the recommendations of the Cochrane Handbook for systematic reviews. The study followed the Studies, Data, Methods and Outcomes structure (SDMO) (Clarke, Oxman, Paulsen, Higgins & Green, 2011), seeking to clarify the impact of the methodology on the quality of the research within the specific field (Munn et al., 2018).

Search strategies
We used the SDMO structure (Clarke et al., 2011) to construct the research question: S: experiences in Health Professions Education; D: scientific articles; M: Q method; and O: use of digital technologies. Thus, the question directing the review was: "How has the Q method been used in research on the use of digital technologies in health education?"

Identification of Studies
Our search for published articles was carried out in six databases: PubMed, Scopus, CINAHL, PsycInfo, Web of Science, and Latin American and Caribbean Center on Health Sciences Information (LILACS). For the search strategy, we used the following descriptors combined with Boolean operators: "Q-sort" OR "Q-methodology" OR "Q-technique" AND learning OR teaching in five databases. Specifically in LILACS, "Q-sort" OR "Q-methodology" OR "Q-technique" OR "Metodologia Q" were used, as detailed in Box 1. We had the technical support of a librarian (M.S.) to design the strategy and to perform the article search.

Study Selection Criteria
The articles found in the search were transferred to the Rayyan QCRI software (Ouzzani Hammady, H., Fedorowicz, Z., & Elmagarmid, et al., 2016). Two researchers eliminated duplications and read titles and abstracts independently (S.S.S. and N.R.B.). Subsequently, they read the full-text articles that met the inclusion criteria. Any discrepancies between the reviewers' assessment in the phase of full-text reading and in the selection of the articles that would be included were solved by consensus and the mediation of another researcher (M.N.F.F.). The researchers excluded articles that merely assessed Q method instead of applying it to Health Professions Education with digital technologies.
Articles that described Health Professions Education using digital technologies and employed the Q method in their research, published up to December 2019, in Portuguese, English, or Spanish, were considered eligible.
For this, Q method was classified into the MMR category of exploratory sequential study. In studies in this category, a

Database
Search strategy
MMAT is used to analyze whether the sources of qualitative (QUAL) data are relevant, whether the appropriate resources were employed for quantitative (QUAN) studies, such as measurements (clear origin, validity known, standard instrument), and whether the integration of qualitative and quantitative data (MMR) answers the research question (Pluye et al., 2011).
We decided to use MMAT to detail the studies included in this review. The use of MMAT enabled to create a global quality score of each article, considering it as the lowest score of the study components. The score is 25% when QUAL=1 or QUAN=1 or MMR=0; 50% when QUAL=2 or QUAN=2 or MMR=1; 75% when QUAL=3 or QUAN=3 or MMR=2; and 100% when QUAL=4 and QUAN=4 and MMR=3 (Box 2).

Data Extraction and Synthesis
Data extraction was carried out by two researchers (S.S.S and N.R.B.). The eligible texts were read and summarized, observing aspects of authorship, year of publication, location, participants' characteristics, journal's area of knowledge, the digital technologies that were used, the main findings related to Health Professions Education, and particular aspects of Q method. The relevant information about each study was extracted and included in a spreadsheet made in Microsoft Word.

Afterwards, this information was summarized.
Box 2 -MMAT 1 quality assessment for mixed methods studies according to the criteria 2

Criteria QUAL Domain QUAN Domain Mixed Domain
1 Are the sources of qualitative data (archives, documents, informants, observations) relevant to address the research question (objective)?
Is the sampling strategy relevant to address the quantitative research question (quantitative aspect of the mixed methods question)?
Is the mixed methods research design relevant to address the qualitative and quantitative research questions (or objectives), or the qualitative and quantitative aspects of the mixed methods question (or objective)? 2 Is the process for analyzing qualitative data relevant to address the research question (objective)?
Is the sample representative of the population understudy?
Is the integration of qualitative and quantitative data (or results) relevant to address the research question (objective)?
3 Is appropriate consideration given to how findings relate to the context, e.g., the setting, in which the data were collected?
Are measurements appropriate (clear origin, or validity known, or standard instrument)?
Is appropriate consideration given to the limitations associated with this integration, e.g., the divergence of qualitative and quantitative data (or results) in a triangulation design?
4 Is appropriate consideration given to how findings relate to researchers' influence, e.g., through their interactions with participants?
Is there an acceptable response rate (60% or above)?
Because this study contains qualitative and quantitative data, meta-summarization was adopted to present the results and discussion. Meta-summarization is a type of meta-synthesis in which one describes qualitative findings in the quantitative form of statistics. Using topics, we pointed to data frequency and their prevalence, in order to validate them. In addition to extraction, we performed the abstraction of findings and calculated the amplitude of prevalence (Sandelowski & Barroso, 2003).

Results
Overall, 1,398 articles were identified, of which 370 were duplicated. After reading titles and abstracts, the researchers (S.S.S and N.R.B.) selected 44 articles, 12 articles in common. After reaching a consensus, the researchers selected 27 articles to be read in full. The application of the inclusion criteria resulted in 13 articles that made up the final sample of this study. The diagram of the selection of the articles that compose this systematic review can be found in Figure 1.
The reasons for the exclusion of 14 articles were: article written in Korean (n=2); article unavailability after additional searches in other databases and contact with authors (n=2); the study did not use digital technologies (n=4); the article was informative and did not develop educational activities (n=3); the study was not from the health area (n=2); the study did not use Q method (n=1).  Table 1 presents a summary of the main aspects that were found, organized according to authorship and year of publication, place, sample characteristics, the type of technology that was used, the particular aspects of Q method in relation to how the qualitative and quantitative phases were carried out, and the main findings of the included articles.
Studies published from 1998 to 2018 were found and, to facilitate identification, they were arranged in chronological order. We observed that there was a predominance of studies conducted in the following countries: Canada (30.8%; n=4) and USA (30.8%; n=4), followed by Korea (23%; n=3) and United Kingdom (15.4%; n =2) ( Table 1).
In the assessment of methodological quality, 11 studies can be considered as having high methodological quality (score 75% or 100% -MMAT), whereas two can be considered of lower methodological quality (score 50% -MMAT). The global quality score of the 13 studies is presented in Table 2. Research, Society and Development, v. 10, n. 10, e471101019154, 2021 (CC BY 4. It identified three points of view (response arrangement or factor): pragmatists (factor 1), positive communicators (factor 2A) and shy enthusiasts (factor 2B). These factors explained 28% (factor 1) and 11% (factor 2) of total variance, respectively.

Qualitative
Step

Quantitative
Step Key Results MMAT Quality Score* The perspective "Stand by me", factor 3 -The perspective "The agony of defeat", factor 4 -The perspective "Let me think it through", factor 5 -The perspective "I'm engaging and so should you". 12 Note: Each domain is formed by 4 criteria, which were assessed as: 1, criterion met; 0, criterion not met or unable to determine. *MMAT: Mixed Methods Appraisal Tool (Pluye et al. 2011). Source: Articles included in this review.  Table 1 presents the specific characteristics of the method used by the researchers to develop the studies: how the statements were constructed; some authors consider that the sample data characterize the qualitative step of Q method (Couto et al., 2011;Ramlo, 2016); the method for extracting the correlation of the arrangements of the response sets and factorial rotation were characterized in these studies as the quantitative step of Q method.

Description of Q method in the Studies
Q sample varied from a minimum of 29 to a maximum of 60 statements, with 46% (n=6) of the studies using between 40 and 49 statements. The method of data extraction was Centroid in 38% (n=5) of the studies, and Principal Component Analysis (PCA) was used in 54% (n=7). Data analyses are performed by different programs: PQMethod in 61% (n=8) of the studies, PC-QUANL in 13.4% (n=2), PCQ in 7.7% (n=1), and Qanalyze in 7.7% (n=1). One study (7.7%) used the STATA software to analyze data.
Despite being involved in philosophical discussions since its origin (Stephenson, 1935), Q method is presented as MMR in all the studies (Table 1) (Akhtar-Danesh et al., 2009;Coogan, Dancey & Attree, 2005). Its qualitative component appears in the first phase of the method through the construction of the concourse, and subsequently, in the Q set or Q sample (Coogan et al., 2005). In addition, manual rotation of the factors and interviews and open-ended questions could be inserted after the quantitative phase in the application of the Q set (data collection), to enable the collection of more information and assist in the process of analysis and interpretation of these factor matrices, or to better estimate Q sorts (Petit dit Dariel et al., 2013).
Concerning the construction of the Q sample, Q set, or Q statements, the authors used several techniques, such as interviews with the target audience in 30.8% of the studies (Ha, 2016;Miller et al. 1998 The participant group or P set were the names attributed to the participants of the Q study. The size varies, but for the authors who created the method, the gold standard is a group formed by 40 to 60 participants (Watts & Stenner, 2012). Of the studies gathered in our review, 54% had between 40 and 60 participants (Akhtar-Danesh et al., 2009;Ha, 2014Ha, , 2016Landeen et al. 2015;Miller et al. 1998;Paige & Morin, 2015a, 2015b, 30.8% had 20 to 39 participants (Baxter et al., 2009;Petit dit Dariel et al., 2013;Valaitis et al., 2007;Yeun et al., 2014), and 15% had more than 100 participants (Coogan et al., 2005;Roberts et al., 2018).
The quantitative component is characterized by the analysis of the response arrangements (Q sort), in which each statement receives a score corresponding to the rank assigned to it by each respondent; for example, "strongly agree" to "strongly disagree". The different arrangements are correlated and their factor analysis is carried out. When correlations are found between different arrangements, it is considered that there is a tendency to give importance to the same statements (Novaes, 2020;Stenner & Stainton-Rogers, 2004;Watts & Stenner, 2012).
In the response arrangements, the Centroid method was used for the extraction of correlations (Coogan et al., 2005; Petit dit Dariel et al., 2013;Valaitis et al., 2007), or the PCA extraction method (Ha, 2014(Ha, , 2016Landeen et  to rotate them (Paige & Morin, 2015a). In addition, in some studies factors were identified and labeled by the judgment of a team of experts in the researched domain, in a process called manual rotation of the factors (Coogan et al., 2005;Ha, 2016;Valaitis et al., 2007). This latter method is defended by classical and theoretical researchers as being the difference in relation to R methodology, as it enables to have a qualitative lens to explore factors, transforming the study into a hybrid one (Ramlo, 2016).
The researchers who use Q method highlight that it is not merely a measurement method for testing validity and reliability; rather, it is an interactive process in which participants classify a series of content-sensitive statements, creating a forced and normal distribution, and this distribution allows a more sophisticated analysis of the respondents' data (Akhtar-Danesh et al., 2009;Cross, 2005).
The studies emphasize that Q method is a strategy to find different perception patterns and disregard the importance of its number distribution in a larger population (Baxter et al., 2009;Petit dit Dariel et al., 2013;Valaitis et al., 2007); thus, it does not focus on the normal distribution pattern. However, Roberts et al., (2018) used Q sort data from a preliminary study (Roberts et al., 2015), and conducted only a factorial analysis to apply different statistical tests.

Using digital technologies in Health Professions Education
In the studies gathered in this systematic review, the primary experience we found regarding the use of digital technologies was simulation, and students and professors perceive it as a supportive learning approach that complements clinical practice (Yeun et al., 2014). However, it is necessary to offer support to students and professors, so that they know the process in which they are included and the tools they are using (Ha, 2016).
The use of e-learning as a support to education and learning in the health area (Coogan et al., 2005;Petit dit Dariel et al., 2013;Valaitis et al., 2007) through virtual learning platforms (Coogan et al., 2005) and videoconference (Valaitis et al., 2007) was also characterized as a facilitating tool. Nevertheless, there are time and training barriers and cultural perception constraints for the adoption of e-learning.
The employment of digital technologies enabled a new cultural training, similarly to what happened with the emergence of books, printed media, and the radio. Today, with the development of the internet, there are several types of readers (contemplative, moving, immersive, and omnipresent) in cyberspace, which is dynamic and in constant transformation (Santaella, 2013). To mediate health education, it is necessary to recognize these reader profiles in the classroom and the forms of communication they use.
The profile patterns of students (Baxter et al., 2009;Landeen et al., 2015;Valaitis et al., 2007;Yeun et al., 2014;), professors (Akhtar-Danesh et al., 2009;Baxter et al., 2009;Petit dit Dariel et al., 2013;Valaitis et al., 2007), and employees (Valaitis et al., 2007) (Defenders of digital technologies, Humanists, Skeptics and Pragmatists) are characterized as groups of response arrangements with a tendency to give importance to the same statements, as presented in Table 1, and ratify the need to understand how digital culture is experienced in the education of health professionals in the 21 st century.

Discussion
In summary, the results showed that Q method can be used to explore students' and professors' attitudes and viewpoints towards the inclusion of digital technologies in the classroom, converting human subjective perspectives into objective results about the phenomenon of digital culture. From this perspective, learning this method and the nature of its mixed research type seems to be appropriate for developing studies on health sciences and Health Professions Education.
Assessing the quality of MMR such as the Q method is challenging and has limitations, but MMAT (Pluye et al., 2011) enabled us to assess qualitative and quantitative components concomitantly. By analyzing the articles' quality, it is possible to identify in what aspects the research using this method needs to be improved.
By performing the analysis and interpretation through a qualitative meta-summarization, our study enabled us to relate studies on the same phenomenonthe use of digital technologies in Health Professions Educationto heterogeneous applications of Q method.
There are variations in the way in which Q method is developed in the analyzed articles: for the construction of concourse and Q set, many techniques were used, like interviews, previous studies, gathering of literature, and focus groups; the extraction of correlations was performed by Centroid or Principal Component Analysis (PCA). After the correlations were extracted from the respondents' response arrangements, submission to factorial rotation or the non-utilization of this procedure were identified.
The method has some disadvantages, like the length of time spent in the Q classification process (one or two hours per individual (Ha, 2016;Yeun et al., 2014)) and difficulties in recruitment and participant adherence (Landeen et al., 2015). In spite of its long history, Q method still is an innovative methodology, unknown in many disciplines, journals and countries (Lim et al., 2021;Van Exel & Graaf, 2005). For instance, recent studies developed in Brazil, where the main authors of the present research work and carry out their studies, have reported that the use of Q method is still considered incipient in the country (Brandão et al., 2017;Novaes, 2016).
We conducted a search for studies developed in the area of health in Brazil using the descriptors and databases selected for this review. Only one article was found (Santos & Schor, 2003). In a search conducted without defining descriptors, we found another article in the health area (Serralta et al., 2007). In these studies (Santos & Schor 2003;Serralta et al. 2007), the authors adapted Q method; they did not use its traditional concepts (Brown, 1980;Watts & Stenner, 2012). Therefore, they were not included in this review, as they did not meet the eligibility criteria.

Strengths and limitations
For some authors, the fragility of the method lies in the fact that study participants are few and selected by convenience, which limits data extrapolation (Kampen & Tamás 2013). However, other authors (Ramlo, 2019;Stenner & Stainton-Rogers, 2004) defend that this feature is, in fact, the advantage of the method, as it enables to know different perspectives of the investigated phenomenon through a hybrid combination, the combination of qualitative and quantitative dimensions. This frequently causes discomfort in the researchers, because they must pay more attention to the subjective and interpretative dimensions.
There are divergences in the views and consensuses regarding Q method within the mixed methods, quantitative methods and qualitative methods communities (Ramlo, 2019), which generates diversity in the vocabulary related to the method, both in the descriptors and in the keywords. This may have restricted the scope of the searches, returning the ones that used only general descriptors, which is a limitation of our study. In addition, the heterogeneity present in the stages of the methodfor example, in the construction of the concourse, in the development of the Q sample, and in the analysis and interpretation of the Q sort -may be a barrier to the adoption of this method by young researchers.

Implications for practice and research
Nowadays, the unexpected pandemic that was started in 2020 by the Severe Acute Respiratory Syndrome Coronavirus 2 (Sars-CoV-2), causing the disease COVID-19, has accelerated the introduction of digital technologies in the teaching and learning process in higher education. The need for emergency remote teaching arose in order to maintain academic activities because of the imposition of quarantines and social distancing (Hodges, Moore, Lockee, Trust & Bond, 2020;Rocha & Sampaio, 2020). In the area of health, the main challenges are to develop the faculty's set of skills and competencies in the use of digital technologies, to include ICT in health curricula and clinical placements, and to promote the professional development of the personnel involved in the educational mission of colleges and universities (Cox, Seaman, Hyde, Freire, & Mansfield, 2020).

Conclusion
Our findings showed that Q method is often cited as appropriate to be applied to the study of Health Professions Education mediated by digital technologies. We believe that the methodological design of this study can serve as a model for others investigating similar research questions and stimulates further research in the field of Health Professions Education with the use of Q method. Health educators face the challenges of intentionality in relation to the concept of health, to the proposed care model, to the strategies used to develop skills and produce knowledge in health education, and to the inclusion of digital technologies in its teaching. In view of the pressing need for education changes, we believe that using MMR, particularly Q method, to investigate teaching culture and practice, can successfully support the renewal of Health Professions Education. The authors suggest that future investigations use the Q Method to understand which elements of online technology are adopted in pedagogical practice by Brazilian public university professors and identify evidence on the recognition and appreciation of digital technologies in the teaching and learning environment.