Reflections on evaluation of scientific production – a special look at Brazil
DOI:
https://doi.org/10.33448/rsd-v11i15.35924Keywords:
Scientific output; Funding; Open access; Evaluation; Graduate; Science in Brazil.Abstract
In this article, we bring some aspects of scientific output evaluation with a look at Brazil. Initially, we present a brief context of scientific production in Brazil and address metrics, such as the impact factor, used to assess the impact of this production on the national and international scientific community and to support decision-making on research funding. Next, we discuss factors that influence the citation of articles and other indicators, which also assess the quality of scientific production. Considering characteristics of the Areas of Knowledge features of scientific journals are punctuated, the Open Access movement points to the need for a public policy to guide the participation of Brazilian research in this new model of making science. Finally, we bring up the issue of the funding needed to encourage scientific production. The objective of reflecting, in this study, on the evaluation of Brazilian scientific output is to raise awareness and mobilize the different actors in the process in favor of a more accessible science that recognizes the specificities of the areas and values their transfer of knowledge, contributing to solving problems of society.
References
ABCD – Agência de Bibliotecas e Coleções Digitais da Universidade de São Paulo. (c2015-2022). Entenda o que é Acesso Aberto. https://www.abcd.usp.br/apoio-pesquisador/acesso-aberto-usp/entenda-o-que-e-acesso-aberto.
Abreu, M., Grinevich, V. Hughes, A., & Kitson, M. (2009). Knowledge Exchange between Academics and the Business, Public and Third Sectors. Centre for Business Research and UK∼IRC, University of Cambridge. http://www.cbr.cam.ac.uk/pdf/AcademicSurveyReport.pdf
Aiello, E., Donovan, C., Duque, E., Fabrizio, S., Flecha, R., Holm, P., ... & Reale, E. (2021). Effective strategies that enhance the social impact of social sciences and humanities research. Evidence & Policy, 17(1), 131-146.
Almeida, E. C. E. (2006). O portal de periódicos da CAPES: estudo sobre a sua evolução e utilização. Dissertação de Mestrado em Desenvolvimento Sustentável, Universidade de Brasília, Brasília, DF, Brasil.
Almeida, E. C. E., Guimarães, J. A., & Alves, I. T. G. (2010). Dez anos do Portal de Periódicos da Capes: histórico, evolução e utilização. Revista brasileira de pós-graduação, 7 (13), 218-246. https://doi.org/10.21713/2358-2332.2010.v7.194.
Andrade, J. D. S., Cassundé, F. R. D. S. A., & Barbosa, M. A. C. (2019). Da liberdade à “gaiola de cristal”: sobre o produtivismo acadêmico na pós-graduação. Perspectivas em Gestão & Conhecimento, 9 (1), 169-197. http://hdl.handle.net/20.500.11959/brapci/112427.
Archambault, E. (2010). The outputs and impacts of social sciences and humanities research: Evidence from bibliometrics research and two large-scale web surveys. Science-Metrix. https://www.science-metrix.com/pdf/SM_SSHRC_Brown_Bag_SSH_Research_Impacts.pdf.
Azevedo, M. (2016). The evaluation of the social impacts of culture: culture, arts and development. Thèse, Economies and finances, Université Panthéon-Sorbonne - Paris I, França. https://tel.archives-ouvertes.fr/tel-01804118v1.
Barata, R. B. (2019). Mudanças necessárias na avaliação da pós-graduação brasileira. Interface - Comunicação, Saúde, Educação, 23, e180635. https://doi.org/10.1590/Interface.180635.
Bekele, A., Chu, K., D'Ambruoso, L., Davies, J. I., Ferriolli, E., Greig, C., ... & Siddiqi, S. (2022). Global health research funding applications: brain drain under another name?. The Lancet Global Health, 10(1), e22-e23.
Bezerra, W. R. P., & Fernandes, N. D. C. M. (2021). Análise dos índices de inovação e os resultados recentes da balança comercial brasileira. Revista Pensamento Contemporâneo em Administração, 15 (2), 181-209. https://doi.org/10.12712/rpca.v15i2.49234.
Biblioteca Universitária (2017, 15 dezembro). Você sabe o que é Indicadores bibliométricos? A gente te conta!. Periódicos UFMG, Notícias. https://www.ufmg.br/periodicos/voce-sabe-o-que-e-indicadores-bibliometricos-a-gente-te-conta.
Brighenti, A. M. (2019). Umwelt-measures. On extensive and intensive measures: Introduction to the special issue ‘Theorising measures, rankings and metrics’. Social Science Information, 58 (2), 224-237. https://doi.org/10.1177/0539018419858816.
Björk, B. C., & Solomon, D. (2015). Article processing charges in OA journals: relationship between price and quality. Scientometrics, 103 (2), 373-385. https://doi.org/10.1007/s11192-015-1556-z.
Blind, K., Pohlisch, J., & Zi, A. (2018). Publishing, patenting, and standardization: Motives and barriers of scientists. Research Policy, 47 (7), 1185-1197. https://doi.org/10.1016/j.respol.2018.03.011.
Bornmann, L., & Leydesdorff, L. (2013). Macro-Indicators of Citation Impacts of Six Prolific Countries: InCites Data and the Statistical Significance of Trends. PLoS ONE, 8, e56768. https://doi.org/10.1371/journal.pone.0056768.
Brainard, J. (2021). Open access takes flight.
Brasil, A. (2021 Julho). Beyond the Web of Science: an overview of Brazilian papers indexed by regionally relevant databases. Proceedings of International Conference on Scientometrics & Informetric, Leuven, Belgium, 18. Disponível em: https://kuleuven.app.box.com/s/kdhn54ndlmwtil3s4aaxmotl9fv9s329. Acesso em: 04 ago. 2021.
Brownell, K. D., & Roberto, C. A. (2015). Strategic science with policy impact. The Lancet, 9986 (385), 2445-2446. https://doi.org/10.1016/S0140-6736(14)62397-7.
Bu, Y., Lu, W., Wu, Y., Chen, H., & Huang, Y. (2021). How wide is the citation impact of scientific publications? A cross-discipline and large-scale analysis. Information Processing & Management, 58 (1), 102429. https://doi.org/10.1016/j.ipm.2020.102429.
Bulaitis, Z. (2017). Measuring impact in the humanities: Learning from accountability and economics in a contemporary history of cultural value. Palgrave Communications, 3 (1), 1-11. https://doi.org/10.1057/s41599-017-0002-7.
CAPES. (2021). Geocapes. https://geocapes.capes.gov.br/geocapes/.
Chankseliani, M., & McCowan, T. (2021). Higher education and the sustainable development goals. Higher Education, 81(1), 1-8.
Chavarro, D., Tang, P., & Ràfols, I. (2017). Why researchers publish in non-mainstream journals: Training, knowledge bridging, and gap filling. Research Policy, 46 (9), 1666–1680.
Chi, P. S., Gorraiz, J., & Glänzel, W. (2019). Comparing capture, usage and citation indicators: An altmetric analysis of journal papers in chemistry disciplines. Scientometrics, 120 (3), 1461-1473.
Coccia, M., Falavigna, G., & Manello, A. (2015). The impact of hybrid public and market-oriented financing mechanisms on the scientific portfolio and performances of public research labs: a scientometric analysis. Scientometrics, 102, 151-168. https://doi.org/10.1007/s11192-014-1427-z.
Coccia, M. (2019). Why do nations produce science advances and new technology?. Technology in society, 59, 101124.
Confraria, H., & Godinho, M. M. (2014). The impact of African science: a bibliometric analysis. Scientometrics, 102 (2), 1241-1268. https://doi.org/10.1007/s11192-014-1463-8.
D’Este, P., & Patel, P. (2007). University–industry linkages in the UK: What are the factors underlying the variety of interactions with industry? Research policy, 36 (9), 1295-1313. https://doi.org/10.1016/j.respol.2007.05.002.
Darbier, M. (2020, 21 setembro). A Researcher’s Complete Guide to Open Access Papers. Clarivate Analytics Blog. https://clarivate.com/article/a-researchers-complete-guide-to-open-access-papers/.
Dewan, P., & Shah, D. (2016). A writer’s dilemma: Where to publish and where not to?. Indian pediatrics, 53 (2), 141-145. https://doi.org/10.1007/s13312-016-0809-0.
DFG - Deutsche Forschungsgemeinschaft. (2019). Cluster of Excellence (2005-2017/19). https://www.dfg.de/en/research_funding/programmes/excellence_initiative/clusters_excellence/#:~:text=Clusters%20of%20excellence%20enable%20German,cooperation%20among%20the%20participating%20institutions.
Dias, A. M. I., Therrien, J., & Farias, I. M. S. (2017). As áreas da educação e de ensino na Capes: identidade, tensões e diálogos. Revista Educação e Emancipação, 10 (1), 34-57. https://doi.org/10.18764/2358-4319.v10n1p34-57.
Fabiani, S., & Sbragia, R. (2014). Tax incentives for technological business innovation in Brazil: the use of the Good Law-Lei do Bem (Law No. 11196/2005). Journal of technology management & innovation, 9, (4), 53-63. http://dx.doi.org/10.4067/S0718-27242014000400004.
Fávero, A. A., Consaltér, E., & Tonieto, C. (2019). A avaliação da pós-graduação e a sua relação com a produção científica: dilemas entre a qualidade e a quantidade. EccoS – Revista Científica, 51, e14508. https://doi.org/10.5585/EccoS.n51.14508.
Fazenda, I., Tavares, D., & Godoy, H. (2018). Interdisciplinaridade na pesquisa científica. Papirus Editora.
Feldman, L. (2016). Effects of TV and cable news viewing on climate change opinion, knowledge, and behavior. Climate Science. https://doi.org/10.1093/acrefore/9780190228620.013.367.
Fortunato, S., Bergstrom, C. T., Börner, K., Evans, J. A., Helbing, D., Milojević, S., ... Barabási, A. L. (2018). Science of science. Science, 359 (6379), eaao0185. https://doi.org/10.1126/science.aao0185.
Frischtak, C. (2019). Science and innovation in Brazil. Where to now? In Frischtak, C. Innovation in Brazil. (pp. 93-119). Londres: Routledge.
Gargouri, Y., Hajjem, C., Larivière, V., Gingras, Y., Carr, L., Brody, T., & Harnad, S. (2010). Self-Selected or Mandated, Open Access Increases Citation Impact for Higher Quality Research. PLoS ONE, 5 (10), e13636. https://doi.org/10.1371/journal.pone.0013636.
Glänzel, W. (2001). National characteristics in international scientific co-authorship relations. Scientometrics, 51 (1), 69–115. https://doi.org/10.1023/A:1010512628145.
Google Scholar. (2022). Google Scholar Metrics. https://scholar.google.com.br/intl/pt-BR/scholar/metrics.html#metrics.
Gu, X., Hua, S., McKenzie, T., & Zheng, Y. (2022). Like father, like son? Parental input, access to higher education, and social mobility in China. China Economic Review, 72, 101761.
Hall, C. M. (2011). Publish and perish? Bibliometric analysis, journal ranking and the assessment of research quality in tourism. Tourism Management, 32 (1), 16-27. https://doi.org/10.1016/j.tourman.2010.07.001.
Hall, S. A., & Wilcox, A. J. (2007). The fate of epidemiologic manuscripts: a study of papers submitted to epidemiology. Epidemiology, 18 (2), 262-265. https://doi.org/10.1097/01.ede.0000254668.63378.32.
Hammarfelt, B., & Rijcke, S. de (2015). Accountability in context: Effects of research evaluation systems on publication practices, disciplinary norms, and individual working routines in the faculty of Arts at Uppsala University. Research Evaluation, 24 (1), 63-77. https://doi.org/10.1093/reseval/rvu029.
Hayden, M. C., Weiβ, M., Pechriggl, A., & Wutti, D. (2018). Insights into university knowledge transfer in the social sciences and humanities (SSH) and other scientific disciplines—More similarities than differences. Front: Frontiers Research Metrics and Analytics, 3, 32. https://doi.org/10.3389/frma.2018.00032.
Jamal, T., Smith, B., & Watson, E. (2008). Ranking, rating and scoring of tourism journals: Interdisciplinary challenges and innovations. Tourism Management, 29 (1), 66-78. https://doi.org/10.1016/j.tourman.2007.04.001.
James, J. E. (2017). Free‐to‐publish, free‐to‐read, or both? Cost, equality of access, and integrity in science publishing. Journal of the Association for Information Science and Technology, 68 (6), 1584-1589. https://doi.org/10.1002/asi.23757.
Jeong, S., Choi, J. Y., & Kim, J. Y. (2014). On the drivers of international collaboration: The impact of informal communication, motivation, and research resources. Science and Public Policy, 41 (4), 520-531. https://doi.org/10.1093/scipol/sct079.
Jordan, G. B., Streit, L. D., & Matiasek, J. (2003). Attributes in the Research Environment That Foster Excellent Research: An Annotated Bibliography. United States. https://doi.org/10.2172/808620.
Jugend, D., Fiorini, P. D. C., Armellini, F., & Ferrari, A. G. (2020). Public support for innovation: A systematic review of the literature and implications for open innovation. Technological Forecasting and Social Change, 156, 119985.
Kamdem, J. P., Fidelis, K. R., Nunes, R. G., Araujo, I. F., Elekofehinti, O. O., Cunha, F. A., … Barros, L. M. (2017). Comparative research performance of top universities from the northeastern Brazil on three pharmacological disciplines as seen in scopus database. Journal of Taibah University Medical Sciences, 12 (6), 483-491. https://doi.org/10.1016/j.jtumed.2017.03.003.
Kamdem, J. P., Roos, D. H., Sanmi, A. A., Calabró, L., Abolaji, A. O., Oliveira, C. S., … Rocha, J. B. T. (2019). Productivity of CNPq researchers from different fields in biomedical sciences: The need for objective bibliometric parameters - A report from Brazil. Science and Engineering Ethics, 25 (4), 1037-1055. https://doi.org/10.1007/s11948-018-0025-5.
King, D. A. (2004). The scientific impact of nations. Nature, 430 (6997), 311-316. https://doi.org/10.1038/430311a.
Krawczyk, F., & Kulczycki, E. (2021). How is open access accused of being predatory? The impact of Beall's lists of predatory journals on academic publishing. The Journal of Academic Librarianship, 47(2), 102271.
Larivière, V., Gingras, Y., & Archambault, É. (2006). Canadian collaboration networks: A comparative analysis of the natural sciences, social sciences and the humanities. Scientometrics, 68 (3), 519–533. https://doi.org/10.1007/s11192-006-0127-8.
Larivière, V., Macaluso, B., Mongeon, P., Siler, K., & Sugimoto, C. R. (2018). Vanishing industries and the rising monopoly of universities in published research. PLoS ONE, 13 (8), e0202120. https://doi.org/10.1371/journal.pone.0202120.
Link, A. N., Siegel, D. S., & Bozeman, B. (2007). An empirical analysis of the propensity of academics to engage in informal university technology transfer. Industrial and corporate change, 16 (4), 641-655. http://dx.doi.org/10.1093/icc/dtm020.
Linton, J. D., Tierney, R., & Walsh, S. T. (2011). Publish or perish: How are research and reputation related?. Serials Review, 37 (4), 244-257. https://doi.org/10.1016/j.serrev.2011.09.001.
Lundberg, J. (2017). Does academic research affect local growth? Empirical evidence based on Swedish data. Regional Studies, 51 (4), 586-601. https://doi.org/10.1080/00343404.2015.1107182.
Mali, F., Pustovrh, T., Platinovšek, R., Kronegger, L., & Ferligoj, A. (2017). The effects of funding and co-authorship on research performance in a small scientific community. Science and Public Policy, 44 (4), 486-496. https://doi.org/10.1093/scipol/scw076.
Marginson, S. (2022). What drives global science? The four competing narratives. Studies in higher education, 47 (8), 1566-1584. https://doi.org/10.1080/03075079.2021.1942822.
Marques, F. (2013, maio). Os limites do índice-h. Revista Pesquisa Fapesp. https://revistapesquisa.fapesp.br/os-limites-do-indice-h/.
Mathies, C., Kivistö, J., & Birnbaum, M. (2020). Following the money? Performance-based funding and the changing publication patterns of Finnish academics. Higher Education, 79(1), 21-37. https://doi.org/10.1007/s10734-019-00394-4.
McManus, C., & Baeta Neves, A. A. (2021a). Funding research in Brazil. Scientometrics, 126(1), 801-823.
McManus, C., & Baeta Neves, A. A. (2021b). Production Profiles in Brazilian Science, with special attention to social sciences and humanities. Scientometrics, 126 (3), 2413-2435. https://doi.org/10.1007/s11192-020-03452-2.
McManus, C et al. (2021c) Profiles not metrics: the case of Brazilian universities. Anais da Academia Brasileira de Ciências [online]. v. 93, n. 4, e29290261. https://doi.org/10.1590/0001-3765202120200261
McManus, C., Baeta Neves, A.A. & Prata, A.T. (2021d) Scientific publications from non-academic sectors and their impact. Scientometrics 126, 8887–8911 https://doi.org/10.1007/s11192-021-04159-8
Melo, J. H. N., Trinca, T. P., & Maricato, J. M. (2021). Limites dos indicadores bibliométricos de bases de dados internacionais para avaliação da Pós-Graduação brasileira: a cobertura da Web of Science nas diferentes áreas do conhecimento. Transinformação, 33, e200071. https://doi.org/10.1590/2318-0889202133e200071.
Mingers, J., & Harzing, A. W. (2007). Ranking journals in business and management: A statistical analysis of the Harzing data set. European Journal of Information Systems, 16 (4), 303-316.
Moed, H. F. (2005). Citation Analysis in Research Evaluation, vol. 9, Berlin/Heidelberg: Springer.
Mongeon, P., & Paul-Hus, A. (2016). The journal coverage of Web of Science and Scopus: a comparative analysis. Scientometrics, 106 (1), 213–228. https://doi.org/10.1007/s11192-015-1765-5.
Mugnaini, R., Damaceno, R. J. P., Digiampietri, L. A., & Mena-Chalco, J. P. (2019). Panorama da produção científica do Brasil além da indexação: uma análise exploratória da comunicação em periódicos. Transinformação, 31, e190033. https://doi.org/10.1590/2318-0889201931e190033.
Mugnaini, R., Digiampetri, L. A., & Mena-Chalco, J. P. (2014). Comunicação científica no Brasil (1998-2012): indexação, crescimento, fluxo e dispersão. Transinformação, 26, 239-252. https://doi.org/10.1590/0103-3786201400030002.
Nature. (2018). The best research is produced when researchers and communities work together. Nature, 562 (7), editorial. https://doi.org/10.1038/d41586-018-06855-7.
Nederhof, A. J., & Visser, M. S. (2004). Quantitative deconstruction of citation impact indicators: Waxing field impact but waning journal impact. Journal of Documentation, 60 (6), 658-672. http://dx.doi.org/10.1108/00220410410568142.
Neiva, S. C. P. C., Dutra, A. C., Silva, V. de S., Fonseca, M. C. C. da, & Silva, C. M. (2022). Perspectivas da ciência brasileira: um estudo sobre a distribuição de bolsas de pesquisa em produtividade do CNPq ao longo do ano de 2019. Revista Interdisciplinar Científica Aplicada, 16 (3), 51–71. https://portaldeperiodicos.animaeducacao.com.br/index.php/rica/article/view/18090.
Nobre, L. N., & Freitas, R. R. (2017). A evolução da pós-graduação no Brasil: histórico, políticas e avaliação. Brazilian Journal of Production Engineering - BJPE, 3 (2), 26–39. https://doi.org/10.0001/v3n2_3.
OECD. (2012). Research Universities: Networking the Knowledge Economy. October, p. 1–87.
O’Grady, C. (2022). Upheaval in Norwegian science funding threatens grants. Science News. Available at: https://www. science. org/content/article/upheaval-norwegianscience-funding-threatens-grants.
Oliveira Jr, O. N. (2016). Research landscape in Brazil: Challenges and opportunities. The Journal of Physical Chemistry C, 120 (10), 5273-5276. http://dx.doi.org/10.1021/acs.jpcc.6b01958.
Olmos-Peñuela, J., Castro-Martínez, E., & d’Este, P. (2014). Knowledge transfer activities in social sciences and humanities: Explaining the interactions of research groups with non-academic agents. Research Policy, 43 (4), 696-706. https://doi.org/10.1016/j.respol.2013.12.004.
Packer, A. L., Biojone, M. R., Antonio, I., Takenaka, R. M., García, A. P., Silva, A. C. D., ... & Delbucio, H. C. R. F. (1998). Scielo: Uma metodologia para publicação eletrônica. Ciência da Informação, 27 (2), 109-121. https://doi.org/10.1590/S0100-19651998000200001.
Pan, R. K., Kaski, K., & Fortunato, S. (2012). World citation and collaboration networks: uncovering the role of geography in science. Scientific reports, 2(1), 1-7. https://doi.org/10.1038/srep00902.
Patrus, R., Shigaki, H. B., & Dantas, D. C. (2018). Quem não conhece seu passado está condenado a repeti-lo: distorções da avaliação da pós-graduação no Brasil à luz da história da Capes. Cadernos EBAPE.BR, 16 (4), 642-655. https://doi.org/10.1590/1679-395166526.
Pavan, C. (2018). Produção científica do Brasil: relações entre o acesso aberto à informação científica e a política de financiamento público para a publicação de artigos mediante o pagamento de Article Processing Charge (APC). Tese de Doutorado em Educação em Ciências, Universidade Federal do Rio Grande do Sul, Porto Alegre, RS, Brasil. http://hdl.handle.net/10183/185052.
Perkmann, M., & Walsh, K. 2007. University–industry relationships and open innovation: towards a research agenda. International Journal of Management Reviews, 9 (4), 259-280. https://doi.org/10.1111/j.1468-2370.2007.00225.x.
Pinfield, S., Salter, J., & Bath, P. A. (2017). A “Gold‐centric” implementation of open access: Hybrid journals, the “Total cost of publication,” and policy development in the UK and beyond. Journal of the Association for Information Science and Technology, 68 (9), 2248-2263. https://doi.org/10.1002/asi.23742.
Piwowar, H., Priem, J., Larivière, V., Alperin, J. P., Matthias, L., Norlander, B., ... & Haustein, S. (2018). The state of OA: a large-scale analysis of the prevalence and impact of Open Access articles. PeerJ, 6, e4375. https://doi.org/10.7717/peerj.4375.
Puuska, H. M. (2014). Scholarly publishing patterns in Finland: A comparison of disciplinary groups. Academic Dissertation, University of Tampere, Tampere, Finland. https://tampub.uta.fi/bitstream/handle/10024/95381/978-951-44-9480-2.pdf?sequence=1.
Quan, W., Chen, B., & Shu, F. (2017). Publish or impoverish: An investigation of the monetary reward system of science in China (1999-2016). Aslib Journal of Information Management, 69 (5), 486-502. https://doi.org/10.1108/AJIM-01-2017-0014.
Quintanilha, T. L., & Cardoso, G. (2018). The impact factor as a legitimator of the scientific knowledge produced: a review of the literature. JANUS. NET e-journal of International Relations, 9 (2), 32-44.
Ramírez-Castañeda, V. (2020). Disadvantages in preparing and publishing scientific papers caused by the dominance of the English language in science: The case of Colombian researchers in biological sciences. PloS one, 15(9), e0238372. https://doi.org/10.1371/journal.pone.0238372.
Ravelli, A. P. X., Fernandes, G. C. M., Barbosa, S. D. F. F., Simão, E., Santos, S. M. A. D., & Meirelles, B. H. S. (2009). A produção do conhecimento em enfermagem e envelhecimento: estudo bibliométrico. Texto & Contexto-Enfermagem, 18 (3), 506-512. https://doi.org/10.1590/S0104-07072009000300014.
Razumova, I. K., & Kuznetsov, A. (2019). Impact of open access models on citation metrics. Journal of Information Science Theory and Practice, 7 (2), 23-31, 2019. https://doi.org/10.1633/JISTaP.2019.7.2.2
Reale, E., Avramov, D., Canhial, K., Donovan, C., Flecha, R., Holm, P., ... & Van Horik, R. (2018). A review of literature on evaluating the scientific, social and political impact of social sciences and humanities research. Research Evaluation, 27(4), 298-308.
REF - Research Evaluation Framework. (c2020). About the REF. https://www.ref.ac.uk/about/.
Robinson-Garcia, N., Costas, R., & van Leeuwen, T. N. (2020). Open Access uptake by universities worldwide. PeerJ, 8, e9410. https://doi.org/10.7717/peerj.9410.
Sampaio, R. M., & Bonacelli, M. B. M. (2018). Biodiesel in Brazil: Agricultural R&D at petrobras biocombustível. Journal of technology management & innovation, 13 (1), 66-74. https://doi.org/10.4067/S0718-27242018000100066.
Schneider, E. M., Fujii, R. A. X., & Corazza, M. J. (2017). Pesquisas quali-quantitativas: contribuições para a pesquisa em ensino de ciências. Revista Pesquisa Qualitativa, 5(9), 569–584. Recuperado de https://editora.sepq.org.br/rpq/article/view/157
SCImago. (2021). Scimago Journal & Country Rank. https://www.scimagojr.com/countryrank.php?year=2021.
Serenko, A., & Dohan, M. (2011). Comparing the expert survey and citation impact journal ranking methods: Example from the field of Artificial Intelligence. Journal of Informetrics, 5 (4), 629-648. https://doi.org/10.1016/j.joi.2011.06.002.
Silberberg, S. D., Crawford, D. C., Finkelstein, R., Koroshetz, W. J., Blank, R. D., Freeze, H. H., ... & Seger, Y. R. (2017). Shake up conferences. Nature, 548 (7666), 153-154. https://doi.org/10.1038/548153a.
Silva, F. C. C. D., & Silveira, L. D. (2019). O ecossistema da Ciência Aberta. Transinformação, 31, e190001. https://doi.org/10.1590/2318-0889201931e190001.
Silva, M. Z, Venturini, J. C., & De Nez, E. (2018). Quali x Quanti – Quanti x Quali: Desevendando Mitos e verdades sobre as Abordagens na Pesquisa em Ciências Contábeis. XVIII USP International Conference in Accounting. Disponível em: https://congressousp.fipecafi.org/anais/18UspInternational/ArtigosDownload/1220.pdf
Singh, V. K., Singh, P., Karmakar, M., Leta, J., & Mayr, P. (2021). The journal coverage of Web of Science, Scopus and Dimensions: A comparative analysis. Scientometrics, 126(6), 5113-5142. https://doi-org.ez1.periodicos.capes.gov.br/10.1007/s11192-021-03948-5
Sivertsen, G. (2016). Patterns of internationalization and criteria for research assessment in the social sciences and humanities. Scientometrics, 107, 357–368. https://doi.org/10.1007/s11192-016-1845-1
Sivertsen, G. (2018). The Norwegian Model in Norway. Journal of Data and Information Science, 3 (4), 2-19. https://doi.org/10.2478/jdis-2018-0017.
Sousa, A. G. de, Braga, M. J., & Meyer, L. F. (2015). Impact of cooperation on the R&D activities of Brazilian firms. Procedia Economics and Finance, 24, 172-181. https://doi.org/10.1016/S2212-5671(15)00641-3.
Stahlschmidt, S., & Stephen, D. (2022). From indexation policies through citation networks to normalized citation impacts: Web of Science, Scopus, and Dimensions as varying resonance chambers. Scientometrics, 127(5), 2413-2431.
Strasser, B. J. (2009). The coproduction of neutral science and neutral state in Cold War Europe: Switzerland and international scientific cooperation, 1951–69. Osiris, 24 (1), 165-187. https://doi.org/10.1086/605974.
Suber, P. (2012). Ensuring open access for publicly funded research. BMJ, 345, e5184. https://doi.org/10.1136/bmj.e5184.
Tahamtan, I., Afshar, A. S., & Ahamdzadeh, K. (2016). Factors affecting number of citations: a comprehensive review of the literature. Scientometrics, 107 (3), 1195-1225. https://doi.org/10.1007/s11192-016-1889-2.
The British Academy. (2008). Punching our weight: The humanities and social sciences in public policy making. A British Academy Report. https://heranet.info/assets/uploads/2017/11/Punching-our-weight-the-humanities-and-social-sciences-in-public-policy-making-British-Academy.pdf.
Toledo, E. G. (2018). Research assessment in the Humanities and the Social Sciences in review. Revista Española de Documentación Científica, 41 (3), e208. https://doi.org/10.3989/redc.2018.3.1552.
Van den Akker, W., & Spaapen, J. (2017). Productive interactions: Societal impact of academic research in the knowledge society. LERU position paper, https://www.leru.org/files/Productive-Interactions-Societal-Impact-of-Academic-Research-in-the-Knowledge-Society-Full-paper.pdf
Visser, G. (2009). Tourism geographies and the South African National Research Foundation's Researcher Rating System: international connections and local disjunctures. Tourism Geographies, 11(1), 43-72. https://doi.org/10.1080/14616680802643243.
Vogel, M. J. M. (2017). Uso de indicadores bibliométricos na avaliação da capes: o qualis periódicos. Encontro Nacional de Pesquisa e Pós-graduação em Ciência da Informação, Marília, SP, Brasil, 18. http://hdl.handle.net/20.500.11959/brapci/105459.
Wagner, C. S., Park, H. W. & Leydesdorff, L. (2015). The continuing growth of global cooperation networks in research: A conundrum for national governments. PloS one, 10 (7), e0131816. https://doi.org/10.1371/journal.pone.0131816.
Walters, W. H. (2017). Do subjective journal ratings represent whole journals or typical articles? Unweighted or weighted citation impact? Journal of Informetrics, 11 (3), 730-744. http://dx.doi.org/10.1016/j.joi.2017.05.001.
Waltman, L. (2016). A review of the literature on citation impact indicators. Journal of informetrics, 10 (2), 365-391. https://doi.org/10.1016/j.joi.2016.02.007.
Wutti, D., & Hayden, M. (2017). Knowledge transfer in the social sciences and humanities (SSH)—Definition, motivators, obstacles, and visions. Colloquium New Philologies, 2 (1), 87–101. https://doi.org/10.23963/cnp.2017.2.1.7.
Yang, S., & Zheng, M. (2019). Performance of citations and altmetrics in the social sciences and humanities. Proceedings of the Association for Information Science and Technology, 56 (1), 326-335. https://doi.org/10.1002/pra2.69.
Yang, S., Zheng, M., Yu, Y., & Wolfram, D. (2021). Are Altmetric. com scores effective for research impact evaluation in the social sciences and humanities? Journal of informetrics, 15 (1), e101120. https://doi.org/10.1016/j.joi.2020.101120.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Andréa Carvalho Vieira; Diogo Onofre Gomes de Souza
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
1) Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
2) Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
3) Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
