Scientific Production of Researchers Recipiente of CNPq Productivity Grants in the Field of Production Engineering in the 2013-2016 period

This work has the purpose of mapping the scientific production of researchers recipient of CNPq productivity grants in research (PQ) and productivity grants in technological development and innovative extension (DT), in the Production Engineering field, in the four-year period from 2013 to 2016. The information was extracted from the Lattes Platform, on which we obtained quantitative data about the individual scientific production of 190 researchers, through the software scripLattes v8.10. The data analysis was conducted through descriptive statistics, the Kolgomorov-Smirnov data normality test, the Kruskal-Wallis test, and the Mann-Whitney test. The results show that 76% of the researchers compose the PQ grant category. Within the four-year period, among PQ researchers, 1B and 1A researchers had the best average of total published articles, with 19.42 and 15.75 articles per researcher, respectively. Among DT researchers, 1D and level 2 researchers had, in average, 21.5 and 12.57 published articles, respectively. PQ researchers maintain their production of scientific articles homogeneous at all Qualis levels. On the other hand, DT researchers present greater heterogeneity in their publications. It is possible to conclude that level 1 researchers do not focus on journals that present low Qualis ratings for the publication of their research. In contrast, level 2 researchers are less experienced and expect to publish in higher-level journals, but when their works are rejected, they start “betting” on journals of lesser quality.


Introduction
Brazil is a developing country that has been performing well in several knowledge areas, including in scientific production, especially in comparison with European countries and the United States (Coutinho et al., 2012). This advance is related to the increase in academic degrees, students, graduates and institutions recognized by the Coordination for the Improvement of Higher Education Personnel (Capes 1 ) of the Department of Education (MEC 2 ), through postgraduate programs (Capes, 2006) that deliver the greatest portion of the Brazilian scientific production , approximately 85% of the national total.
This attribution takes place because the university plays an important role in society: "educating people to work with the knowledge" (Brito Cruz, 2010, p. 15). In Brazil, higher education institutions have three functions within the community: education, research, and extension (Durham, 2004). These functions are supported by the Federal Constitution of 05 October 1988 (Brasil, 1988), according to the article 207: "the universities possess didactic-scientific, administrative, financial and assets autonomy, and will obey to the principle of inseparability between education, research, and extension".
In order to disseminate their research, universities take support on research financier agencies (Ebadi and Schiffauerova, 2016;Leta et al., 2006), created between the 1950s and 60s (Guimarães, 2002;Leta et al., 2006;Silva, 2011), still characterized as a recent event in the country (Link and Scott, 2004;Leta et al., 2006). Most research projects are financed by public agencies (Hagedoorn et al., 2000), and the first ones created were the Coordination for the Improvement of Higher Education Personnel (Capes) and the National Council of Technological and Scientific Development (CNPq 3 ) (Guimarães, 2002;De Meis et al., 2007;Silva, 2011).
Among the research grants offered by the CNPq in Brazil are the PQ and DT grants, in which this study will focus.
They are both reserved for standout researchers. The PQ grants focus on the scientific production valorization and the DT grants value scientific production in technological development and innovation. The PQ and DT grants are divided into categories: category 1, with levels A, B, C, and D; and category 2, which is not subdivided into levels. One of the requirements for receiving those grants is the title of doctor: for category 1, the researcher must have had their doctorate in the period of eight years ago; for category 2, three years (CNPq, 2017a).
The PQ and DT grant researchers represent the elite of Brazilian research and science, because they represent considerable scientific contribution to several knowledge areas, through scientific articles, publications in congresses, books, orientations to students, and so on (Oliveira et al., 2011;Silva, 2011;Picinin et al., 2013). This representativeness comes from government incentives, viewed as essential for the Brazilian scientific research (Hermes-Lima et al., 2008) and impactful in its growth (Payne and Siow, 2003;Blume-Kohout et al., 2009). On the other hand, the researchers are highly dependent on financier agencies in order to continue their research projects, and thus the agencies are responsible for the distribution of funds, making the efficient distribution among the most competent researchers (Ebadi and Schiffauerova, 2016).
Thus, the government defines metrics to verify the results achieved by researchers who were financed (Beaudry and Allaoui, 2012) and those who may be financed in the future (Picinin, 2014), seeing that the financing generates an increase in the researchers' production (Latour and Woolgar, 1979) and influences the amount of scientific articles and patents (Zucker et al., 2007;Beaudry and Allaoui, 2012).
The number of published articles is one of the indices of investment in science, especially in emerging countries (Moya-Anegon and Herrero-Solana, 1999;Glaünzel et al., 2006). In Brazil, the growth of scientific production is evident when its evolution is examined: in 1980, the total of publications was 2,000 documents (Helene and Ribeiro, 2011);in 1993, 5,547 documents (Leite, 2014);in 1998, 8,000 documents (Adams andKing, 2009);in , 19,108 documents (Leite, 2014; in 2009, more than 35,000 documents (Helene and Ribeiro, 2011); and in 2013, the total of publications was 42,931 documents (Leite, 2014). This increase is also clear when considering the ranking of nations with the highest scientific production according to Thomson Reuters: in 1993, Brazil was ranked in 24 th , ascending to the 13 th position in 2013 (Leite, 2014).
Despite this growth, it is worth considering not only the number of publications but also their quality (Ferreira et al., 2003). Given that, the scientific community is willing to search for new ways to evaluate scientific production and compare researchers (Yaminfirooz and Gholinia, 2015). Quality evaluation may be conducted in different ways, but the most utilized method is the Impact Factor (IF) (Goméz-Sancho and Mancebón-Torrubia, 2010). The IF assesses journals through the relevance of the scientific works published and their impact on the literature over a period of time. The IF is calculated annually based on the total amount of times that an article is cited, in the two previous years to its publication. This value is divided by the total of published articles (by the journal) in those two years (Garfield, 2006). The journal's IF is calculated to compose the Journal Citation Reports (JCR©), a resource that enables to check the most cited journals in a certain area and the relevance of the publication for academia. According to Miller (2002), journals with high IF have a good reputation in the 3 CNPq -Conselho Nacional de Desenvolvimento Científico e Tecnológico. scientific community. Another widely accepted metric way is the h-index, which evaluates the relevance of the researcher.
Only two years after its disclosure, it had already become a standard rating in global literature for researchers and institutes (Van Eck and Waltman, 2008).
In Brazil, the Qualis/Capes metric is also utilized, which according to Jukemura and Diniz (2015) is employed by Capes through the application Coleta and the Platform Sucupira, which presents a ranking of journals from several areas employed to disseminate the country's research. The ranking is based on criteria to measure the journal's quality encompassing qualitative and quantitative aspects. Through the Qualis rating, the journals are classified in indices in descending order of quality, coded under the levels: A1, A2, B1, B2, B3, B4, B5, and C.
Researchers from different fields investigate the scientific production of productivity grants researchers such as in other contexts (Scarpelli et al., 2008;Martelli-Junior et al., 2010;Spilki, 2013;Grasel et al., 2020). These researches allows tracing quantitative trends in production and the understanding of the requirements to obtain the productivity grant (Mueller, 2006;Picinin et al., 2015).
Within this context, this work has the purpose of mapping the scientific production of researchers recipient of productivity grants in research (PQ) and productivity grants in technological development and innovative extension (DT) of CNPq, in the field of Production Engineering, over the four-year period from 2013 to 2016.

Methodology
Based on Gil (2008) and Pereira et al. (2018), this study is classified from the nature point of view as applied; from objectives point of view as exploratory and from the approach of the problem point of view as quantitative.
The documentary corpus of the research was constructed by selecting the researchers who receive the PQ and DT productivity grants from CNPq in the Production Engineering field. The PQ grant researchers were found on the CNPq website http://cnpq.by/bolsistas-vigentes, selecting as filters the field "engineering" and the sub-field "production engineering". The search for DT grant researchers took place on the Lattes Platform in "search curriculum vitae", with the filter "Other Grant Researchers of CNPq" in addition to the mode "Productivity in Technical Drawing and Innovative Extension", combined with the filter "Professional performance" in the field "Engineering" and sub-field "Production Engineering". The search aiming to list the PQ and DT grant researchers was conducted on December 31 st , 2016.
To map the scientific production, the data from the four-year period 2013-2016 at Capes were considered (extracted from the Lattes curriculum vitae of the researchers). The scientific production was obtained through the software scriptLattes v8.10, which collected quantitative data about the individual scientific production of the researchers in terms of: (i) scientific publications in scientific journals rated by Qualis-Capes; (ii) technical production: technological products, processes or techniques, technical works, and other technical activities; (iii) published books, collections, and book chapters; (iv) congresses: complete works, expanded abstracts, and abstracts; (v) orientations to students at different levels: post-doctorate, doctorate, masters, final term papers (at undergrad level), and scientific initiation; (vi) list of the journals that published those works.
The software scriptLattes is a free system that automatically extracts information from a set of researchers registered on the Lattes Platform. The system is composed by the following modules: (i) data selection, (ii) data pre-processing, (iii) redundancy treatment, (iv) generating graphs of collaboration among the members of the group, (v) generating research maps based on geographic information, and (vi) automatic creation of reports of bibliographical, technical, and artistic production and academic supervision (Mena-Chalco, 2009). After their collection through the software scriptLattes, the data were grouped on an electronic spreadsheet for further statistical analysis.
The list of journals that published works was established manually, from the data previously collected by the software scriptLattes. The journals were ranked according to their number of publications on an electronic spreadsheet.
For the data analysis, descriptive statistics calculations were utilized (in addition to graphs), such as the mean and standard deviation. Through those calculations, it was possible to verify the preliminary discrepancies and similarities existent between the groups of researchers.
The other calculations carried out in this study were based on the literature of Dancey and Reidy (2006). The Kolgomorov-Smirnov normality test, recommended for samples greater than 30 cases, was employed. In the result analysis, the significance level adopted was of p>0.05 to find a normal data distribution (parametric), and a significance level of p<0.05 for a non-normal distribution (non-parametric).
The test results showed data non-linearity (p<0.05). From that presupposition, the Kruskal-Wallis test was employed in order to attest if there was a significant difference between the average points (Dancey and Reidy, 2006). The significance level adopted was of p>0.05 for non-significant differences and p<0.05 for significant differences between the following groups: Publication of articles in journals of Group 1 (A1 and A2 articles), Group 2 (B1 and B2 articles), Group 3 (B3, B4, and B5 articles), and Group 4 (C articles and articles unrated by Qualis); for PQ grant researchers in the category/level 1A, 1B, 1C, 1D, and 2, as well as DT grant researchers in the category/level 2 and 1D.
In the same manner, the aforementioned test was applied in the publications of books, articles in congresses, and other productions (technical and artistic production) by PQ grant researchers in the category/level 1A, 1B, 1C, 1D, and 2, as well as by DT grant researchers in the category/level 2 and 1D. It is worth highlighting that in both the case of articles published in journals and the case of books, articles in congresses and other productions, the analysis was conducted within the category/level of grant researchers.
In the cases that pointed to significant differences between the averages, the Mann-Whitney test was employed as a complement. Its objective is to verify the existence of statistically significant differences between the average points of two conditions (Dancey and Reidy, 2006). A significance level of p>0.05 was adopted for non-significant differences and p<0.05 for significant differences. The Mann-Whitney test was complementary to the Kruskal-Wallis test.
Furthermore, in order to verify the relation between the scientific productions of each category and/or level of the PQ and DT grant researchers' group, Spearman's rank correlation test was employed, seeing that the data were found as being nonnormal (p<0.05).
Regarding the publications in scientific journals, Boxplot graphs and a correlation matrix were elaborated, which were interpreted alongside Spearman's rank correlation so as to ascertain the relation between the publications, researchers, and the Qualis/Capes rating. For the rank correlation analyses, the criteria proposed by Dancey and Reidy (2006) were employed to find the correlations strength, in which: ρ=0.10 to 0.30 weak strength, ρ=0.40 to 0.60 moderate strength; ρ=0.70 to 0.90 strong strength, and ρ=1 perfect correlation, both for the negative and positive correlations. Moreover, a significance level of 95% or p≤0.05 was considered for significant results.
For the statistical calculations of averages differences (Kruskal-Wallis complemented by Mann-Whitney), the scientific articles published by PQ and DT grant researchers were grouped by their Qualis/Capes rating, as shown in Table 1.

Results and Discussion
According to CNPq (2017c), there was a total of 2,859 researchers in the field of Production Engineering, corresponding to 1.2% of the total of researchers in Brazil. In 2016, this number increased to 3,135 researchers, again corresponding to 1.2% of the total of researchers. This research examined 190 grant researchers over the period of time at hand, distributed according to Table 2.  In both grant categories it may be seen that, level 2 holds the greatest number of researchers: 84 researchers in the PQ grant and 42 researchers in the DT grant. Similar results were found by the studies of Barata and Goldbaum (2003) in the Collective Health field, Cavalcanti and Pereira (2008) in Dentistry, Silva (2011) in Science and Math Teaching, Picinin et al. (2013) in Production Engineering in the 2007period, and Picinin (2014 in Administration, which point out that most researchers belong to level 2. The high concentration in level 2 is due to its nature as the first grant category, in which the doctor initiates as a researcher, and in order to attain renown and progression, researchers attempt to stand out with their scientific production. So that they are able to ascend to the 1D category, researchers must show a growing autonomy in their works, that is, a relevant international scientific production. From the category 1C on, the researcher's progress is linked to a regular and independent scientific production, a display of leadership and acknowledgment from their peers (CNPq, 2017d). In view of the aforementioned aspects, the number of scientific productions of the researchers was ascertained, so as to express their representativeness in the field of Production Engineering, following the criteria of CNPq, which evaluates scientific production mainly through the Qualis/Capes rating, focusing on the field at hand. Graph 1 exhibits the number of scientific productions published in journals and the average per researcher. Research, Society and Development, v. 10, n. 2, e15010212378, 2021 (CC BY 4.0) | ISSN 2525-3409 | DOI: http://dx.doi.org/10.33448/rsd-v10i2.12378 8 The lowest average belonged to PQ 2 researchers, with 11.6. In both cases, the highest averages are significantly greater than the ones found in the Production Engineering field.
In this study, it is possible to verify that there is a larger amount of articles published in category 2, especially in the PQ grant. However, the average does not stand out, due to the fact that this category holds a higher concentration of researchers.
Despite the quantitative representativeness in some modalities, a more qualitative approach is also required. Graph 2 displays the number of publications as rated by Qualis/Capes.

Graph 2.
Representation of scientific production classified by the Qualis/Capes rating of the published journals.

Source: Authors (2017).
Graph 2 is important because it shows the number of articles per Qualis/Capes rating published by each group of productivity grants. PQ 2 researchers possess the largest amount of A1 publications, as it is possible to analyze, but the number of researchers per level should also be taken into account. Given that, PQ 2 researchers published an average of 1.86 articles per researcher in the A1 rating. That average is not the highest, which belongs to PQ 1B researchers, but it is greater than the one of PQ 1A researchers, whose 1.67 articles per researcher is the lowest average among PQ researchers.
PQ 2 and DT 2 researchers presented the largest number of unrated articles: 269 and 180, respectively, corresponding to an average of 3.2 and 4.29 articles per researcher. The non-rating of a journal, according to Capes (2017a), happens due to its not appearing in the Qualis/Capes system, generally for not having been indicated by a postgraduate program as a publication vehicle for scientific production or having been marked by some program as a "non-journal" according to the criteria established by the field of knowledge. After the indication of the journal to the Qualis/Capes system, it goes through a triage and enters the Qualis listing if accepted. The rating and divulgation happen annually.
Observing the average per Qualis/Capes rating, it is possible to notice that PQ 1A researchers hold the worst averages in journals rated by Qualis/Capes as B2, B4, B5, C and unrated. In contrast, DT 2 researchers hold the worst averages in the highest levels of the Qualis/Capes rating, A1, A2, B1 and B3. Level 1 researchers, especially 1A, do not focus on low-level journals to publish their researches, due to their experience and intention to remain in the already attained level. Category 2 researchers are less experienced and expect to publish in high-quality journals, but when their works are rejected they start "betting" on lesser quality journals. The study of Barata and Goldbaum (2003) compares the categories 1 and 2 and highlights that publications of higher quality belong to category 1. The study of Picinin et al. (2013Picinin et al. ( ), over 2007Picinin et al. ( -2009 shows that PQ 1A researchers obtained the highest averages in the A2 and B2 Qualis/Capes rating, which partially agrees with this study, in which PQ 1A researchers only obtain the highest average in the A2 rating. This situation is explained by the fact that publications rated as B2 are saturated in the four-year period contemplated in this study (Capes, 2017b), which demotivates high-level researchers to invest in B2 publications.
After that observation, the Kruskal-Wallis test was employed to verify whether there is a significant difference between the scientific production of the Qualis/Capes researcher groups per category and level.  (2017). Table 3 shows if there was a significant difference between the means of publication of articles considering the Qualis/Capes group. The results obtained in the Kruskal-Wallis test identify that DT researchers, 1 and 2D, presented p<0.05, that is, these two researcher groups display significant difference regarding the publication distribution in one of the Qualis/Capes groups. Therefore, the Mann-Whitney calculations were performed to find in which of the groups is the higher concentration of publications by DT 1D and DT 2 grant researchers.
It is possible to highlight that, generally speaking, PQ grant researchers, although presenting a greater publication average in the highest Qualis levels, maintain certain homogeneity in the scientific production throughout all levels. In contrast, DT grant researchers display more heterogeneity in their publications, with a tendency towards lower Qualis levels.  H0: The averages of the groups 1 and 2, 1 and 3, 1 and 4, 2 and 3, 2 and 4 do not present significant difference.

H0 rejected
Grouping variable: Qualis/Capes Source: Authors (2017). Table 4 shows in which comparisons there was significant difference between the means of publication of articles considering the Qualis/Capes group. In the DT 1D category of grant researchers, there is a difference in the scientific production between all the publication groups. The same occurs with the grant researchers in the DT 2 category.
The DT 1D researchers published more in Group 4 than in other groups. The average of publications per group was also obtained: Group 1 with 1.8 articles, Group 2 with 1.8 articles, Group 3 with 3.93 articles, and Group 4 with 8.6 articles. It is possible to observe that DT 1D researchers publish more in the lesser qualified group.
The DT 2 researchers obtained the following averages: Group 1 with 6.6 articles, Group 2 with 11.8 articles, Group 3 with 26.13 articles, and Group 4 with 4.48 articles.
Regarding the scientific production from recipients of PQ grants per Qualis/Capes level (A1, A2, B1, B2, B3, B4, B5, C, and unrated), a boxplot graph was generated to evidence the existence of outliers and an analysis through the median, as shown in Graph 3. Graph 3 shows that the publications in the group unrated by Qualis was greater than the others, seeing the amplitude of its box. Nevertheless, the upper whisker is bigger than the others. The medians of the A1, A2 and unrated publications are similar, and the box size of the first quartile of the A1 and unrated publications are also similar, as may be verified too.
Moreover, only the B1 and B2 publications did not present outliers.
Along this line, the larger number of publications in journals unrated by Qualis draws attention, seeing that publications in this level tend to not score in postgraduate programs (CNPq, 2013). Thus, it is assumed that grant researchers do not seek only journals rated by Qualis to publish scientific articles, but journals that might possess JCR and that, often times, have not been rated by Qualis yet. In general, there is a slight tendency of publications being higher in the A1, A2 and B2 levels. There is a notable decrease in the B2 level, possibly due to the saturation imposed by the field and the difficulty of publishing in this level.
Regarding the scientific production of recipients of DT productivity grants per Qualis/Capes level (A1, A2, B1, B2, B3, B4, B5, C, and unrated), a boxplot graph was generated to evidence the existence of outliers and an analysis through the median, as shown in Graph 4. The DT researchers also display a predominance of publications in journals unrated by Qualis, as evidently shown by its box size. The scenario observed in the PQ researcher group is replicated in the DT researcher group, for both groups seem to not only seek journals rated by Qualis to publish scientific articles.
There is also low representativeness in the highest Qualis levels -A1 and A2. This low representativeness reinforces the perspective of Wainer (2013), who affirms that the grant awards the effort of the researchers, but not necessarily due to the fact of their production causes impact in its scientific field.
Even considering that assumption, this phenomenon may be a reflex of the productivity grants in DT purpose. In theory, DT grants value scientific production in technological development and innovation (CNPq, 2017a) and it must influence the amount of scientific articles and patents (Zucker et al., 2007;Beaudry and Allaoui, 2012). Thus, researchers tend to invest their innovative studies in publications under the form of patents, which limits the number of publications in the A1 and A2 levels. This discussion does not analyze publications of patents, which is a limitation from the present study.
None of the Qualis/Capes levels had outliers, that is, in contrast with the PQ grant researchers, the group is homogeneous and there is not a grant researcher who is elevating the group publication average. As it had already been detected in the Kruskal-Wallis and Mann-Whitney tests, a tendency of DT grant researchers in publications of low Qualis rating may be verified, going against the logic imposed by the PQ researcher group.
Spearman's correlation test was conducted between each Qualis/Capes level of PQ grant researchers, as shown in Table 5. It is worth highlighting that the test was chosen due to the previous application of the Kolgomorov-Smirnov normality test, which resulted in non-linear data.   Table 5 shows the cases where there was interrelationship between PQ grant level and the number of publications in journals of the highest levels of Qualis/Capes. In one case, a significant correlation returned. In eight scenarios, the correlation presented a negative result, as follows: the correlation between the levels A1 and A2, A2 and B1, A2 and B2, A2 and B3, A2 and B4, A2 and B5, A2 and C, and A2 and Unrated. It means to say that, in these scenarios, the variables are inversely proportionalas the number of publications at the A2 level increases, it tends to decrease the other level.
There were strong correlations between A1 and A2, A1 and B1, A1 and B2, A1 and B4, A1 and B5, A1 and Unrated, A2 and B1, A2 and B5, A2 and Unrated, B1 and B2, B1 and B5, B2 and B5, B3 and B5, B3 and C, B3 and B5, B4 and C, and B5 and Unrated. Creating large correlation groups, it is possible to conclude that the researchers who publish at the A1 level tend to publish more at the A2 and B2 levels, that is, high levels. In contrast, the researchers who publish at lower levels, such as B4, tend to also publish at the B5, C, and Unrated levels.
The analysis of the aforementioned results highlights that, as the publications at a level increase, they tend to increase at another level as well. It also becomes clear that publications at the B5 level presented a strong relation with the other levels, except for level C. Thus, it is possible to assume that, in the group examined, the individuals who published many times at the B5 level tend to perform similarly at the other levels.
In order to deepen the analysis of the productivity in research grant researchers' scientific production, a correlation matrix was generated between every Qualis/Capes level (A1, A2, B1, B2, B3, B4, B5, C, and Unrated), as shown in Graph 5.
Research, Society and Development, v. 10, n. 2, e15010212378, 2021 (CC BY 4.0) | ISSN 2525-3409 | DOI: http://dx.doi.org/10.33448/rsd-v10i2.12378 The graph makes it easy to visualize the negative tendency of A2 publications over the others. However, it is noticeable that this tendency occurs only over a portion of the PQ researcher group.
Spearman's correlation test was conducted between each Qualis/Capes level of the productivity in technological development grant researchers, shown in Table 6. The test was selected due to the previous application of the Kolgomorov-Smirnov normality test, which returned nonlinear data.  (2017). Table 6 shows the cases where there was interrelationship between DT grant level and the number of publications in journals of the highest levels of Qualis/Capes. All of the significant correlations returned positive values and, with the exception of the moderate correlation between B2 and C, they were all strong.
Publications at the B2 level presented relations to the A1, B1, B3, B4, B5, and Unrated levels were verified. In addition, publications at the B3 level presented relations to the B1, B2, B4, and B5 levels. Finally, publications at the B5 level presented relations to the B2, B3, B4, and C levels. Analyzing these results, it is possible to assume that the publications at the B2, B3, and B5 levels tend to increase when the publications at other levels do the same.
Generically, it is possible to infer that the DT researchers who publish at the B2 level tend to publish at the B1, A2, and A1 levels as well. On the other hand, the researchers who publish at the B1 level tend to also publish at lower Qualis levels, and not at the A1 and A2 levels.
Graph 6 displays the correlation matrix between every Qualis/Capes level of DT grant researchers.
The correlation matrix exhibits the linearity of the largest correlations that are between A2 and B2, B1 and B4, and B1 and B5.
In order to illustrate the scenario in which the scientific articles are divulged, the journals with the highest number of publications at each researcher level are listed in Table 7.  In addition to having articles published in journals, CNPq demands that researchers, especially those at level 1, indicate five relevant technical productions or human resources productions (CNPq, 2017d). Thus, the technical production of these researchers was ascertained in terms of technological products, technological processes, technical works or other technical activities, as shown in Table 8.  (2017). Table 8 shows the technical production of PQ and DT grant researchers. As expected, the DT researchers are more expressively represented in the technical production category as it is possible to notice, especially in technical works and other technical activities. That is explained by the fact that their technological production goes beyond publications in journals, also encompassing activities such as registered patents and development of non-patented products, processes, or softwares, which focus on innovation and technological development, creating possibilities and interactions with firms and the society as a whole. The same result was found by Picinin et al. (2016)  To better visualize the difference between researchers, the standard deviation was calculated for each type of technical production, verifying that the technical works category presents the largest dispersion between technical productions, it is worthy of attention the high number of works by level 2 researchers (especially DT) standing out, in contrast with the low representativeness of PQ 1B and DT 1D researchers.
The Kolgomorov-Smirnov normality test was also conducted to test the data linearity, returning a value of p=0.003 (p<0.05). The data distribution was verified as being characterized as non-normal. In order to test the existence of a significant difference in the technical production of those groups, the Kruskal-Wallis test was conducted, as shown in Table 9.  (2017). Table 9 presents a comparison between the technical production of PQ and DT grant researchers and the productivity grant category/level. A value of p<0.05 was observed, meaning that there is a significant difference between the technical production of the researchers. In order to identify the difference, the researcher groups were compared through the Mann-Whitney test, as shown in Table 10.  In this case, it stands out that the grant researchers from the DT2 group presented an average of technical production Research, Society andDevelopment, v. 10, n. 2, e15010212378, 2021 (CC BY 4.0) | ISSN 2525-3409 | DOI: http://dx.doi.org/10.33448/rsd-v10i2.12378 21 significantly higher (7.43) than all of the other DT groups. At first glance, this phenomenon could indicate that DT grant researchers realize a high number of technical productions. However, examining the mean of the DT 1D group (0.93), that hypothesis is not proved, due to the non-existence of a pattern (see Table 8). Thus, even though the technical production by the DT2 group stands out, it is not possible to affirm that this scenario is a specific trajectory of the DT group. In order to prove this phenomenon, it would be necessary to conduct longitudinal studies in comparison with other knowledge areas.
Another type of scientific production considered in this study were books, collections, and book chapters written by the researchers, as shown in Table 11. Source: Authors (2017). Table 11 shows the descriptive statistics of the production of books, collections and book chapters book compilations of PQ and DT grant researchers. The mean shows that, both for books and collections, none of the researcher groups reached the average of one work published per author. However, the DT category attained the highest averages, with the DT 2 group presenting the best average in book publications and the DT 1D group presenting the best average in the publication of collections and book chapters. The lowest averages, on the other hand, belong to the PQ 1B group regarding books and the PQ 1C group in terms of collections and book chapters. Through the standard deviation, it is possible to analyze that the largest difference in the productions is in book chapters, in which the category 2 holds a greater representativeness than the other researcher groups. The standard deviation indicates that some researchers publish several book chapters, whereas others dispense this type of production.
In all three cases is possible to verify the low representativeness of the field is noticeable, especially in comparison with the study of Silva (2011), in the Sciences and Math Teaching field, which found that the best average of published books by the researchers was of 12.5 books per researcher belonging to the 1A category. In terms of book chapters, the highest representativeness was of 1C researchers, with 18.8 book chapters per researcher.
In complement, the Kolgomorov-Smirnov normality test was applied and, seeing that the data were nonlinear (p<0.05), the Kruskal-Wallis test was conducted as well in order to compare the publication averages of books, book chapters, and collections between each researcher category. The null hypothesis (H0) indicates that there is no difference in the publication of books, book chapters, and collections between the following researcher groups: PQ1A, PQ1B, PQ1C, PQ1D, PQ2, DT1D, and DT2. On the other hand, the alternative hypothesis (H1) indicates that there is a difference in the publication of books, book chapters, and collections between the groups. The Kruskal-Wallis test returned a value of p=0.081, which allows us to accept H0 and conclude that there was no significant difference in the publication of books, book chapters, and collections between the groups PQ1A, PQ1B, PQ1C, PQ1D, PQ2, DT1D, and DT2.
The low engagement by the researchers in this way of divulging their research is explained, according to Silva et al. (2003), by the fact that the production of books requires more time and effort, whereas works published in events are smaller productions realized to divulge their research or part of it, which eventually may be published in books, more rapidly.
The low value attributed to book publications is also observed when Capes (2017b) exposes its evaluation criteria for the professors who are part of postgraduate programs in the Engineering III field, which encompasses Production Engineering.
It establishes that there is no specific score for books, which are evaluated individually, and also that their production must not outweigh the production of articles published in journals.
In addition to the aforementioned productions, for the CNPq, articles published in congresses and other scientific events are also considered for the researcher progress on it or at least keeping the grant for the researcher in the postgraduate program. However, they weigh less in the evaluation (CNPq, 2017d). Table 12 displays the participation in congresses of each grant researcher category regarding complete works, expanded abstracts, and abstracts. Source: Authors (2017). Table 12 shows the descriptive statistics of the production of publications in congresses, conference short papers and conference paper abstracts of PQ and DT grant researchers. For all the grant researcher categories, it is desirable that the researcher divulge their scientific production in national and international congresses of a good scientific level (CNPq, 2017d).
The results show that all the grant researcher categories had works published in congresses. Regarding complete works, the DT 1D researchers presented the highest average of works divulged. The PQ 1D researchers stood out in the publication of expanded abstracts, and the PQ 1A researchers had the best performance in abstracts. Thought the standard deviation is possible to observe a larger dispersion in the complete works, although the value was also high for expanded abstracts and abstracts.
If compared to the results found by Silva (2011), in the Sciences and Math Teaching field in 2011, there is a great discrepancy between the fields, seeing that the best average in complete works belonged to the 1A researchers with 102.5 articles per researcher, and the lowest average was of 19 articles per researcher in the category 2. Regarding expanded abstracts, the best average is held by the 1B researchers with 6 expanded abstracts per researcher, and the lowest average was of 3.2 expanded abstracts per researcher, again for category 2. In both situations may be verified a significant discrepancy. The inferior representativeness of the Production Engineering field is noticeable in publications in congresses. However, the differences between the fields of knowledge being compared should be taken into account.
To analyze the publication of articles in congresses, the Kolgomorov-Smirnov normality test was conducted and, given the data nonlinearity (p<0.05), the Kruskal-Wallis test was applied as well. The Kruskal-Wallis test comparing the publications in congresses of the groups PQ1A, PQ1B, PQ1C, PQ1D, PQ2, DT1D, and DT2 returned a value of p=0.059.
Considering that the null hypothesis (H0) indicates that there is no difference in the publications of books, book chapters and collections; and the alternative hypothesis (H1) indicates that there is a significant difference, the result p=0.059 allows us to accept H0. Therefore, no significant difference among the production in congresses was found between the mentioned groups.
The orientations to students concluded by each researcher group were also ascertained, as shown in Table 13.  It must be observed that not every researcher is in a position to instruct students at doctorate and post-graduate levels, which would increase the standard deviation at those levels of orientation. One possible situation is that the researcher who is attached to both masters and doctorate programs has the possibility of distributing their orientation vacancies among doctoratelevel students with priority, not instructing masters-level students. Moreover, some researchers choose to not instruct final term papers and/or scientific initiation. However, other researchers/professors opt to give orientation to students through their final term papers and scientific initiations in order to create a bond or verticalization with students who may be interested in entering to postgraduate programs.
The masters-level was verified and the best average is held by PQ 1D researchers. Regarding final term papers, PQ 1A researchers have the best average and, for scientific initiations, DT 2 researchers.
If observed the PQ2 researchers, it is possible to notice their largest participation at masters-level with an average of 5.98 orientations. For DT 2 researchers, it is in scientific initiation with 5.86 orientations, followed by masters-level with 5.45 orientations. Similar results were found by Picinin et al. (2016), also in the Production Engineering field over the [2007][2008][2009] period, in which PQ 2 researchers held an average of 5.27 orientations at masters-level and DT 2 researchers, an average of 6.11.
When calculated the highest standard deviation, the higher standard deviation was observed as being in the orientations at masters-level. There is also a concentration of orientations in scientific initiation at level 2, corroborating with the findings of Cavalcanti and Pereira (2008), who studied the Dentistry field over the 2004-2006 period.
In order to verify whether there was a significant difference between the researchers of each category/level of productivity grant in orientations, the ANOVA was calculated with Tukey's Post Hoc test (after confirming the data linearity through the Kolgomorov-Smirnov test, p>0.05). Table 14 shows the results obtained in the ANOVA test.  There was no significant difference between the grant researchers in the number of orientations conducted at post-doctorate and doctorate levels (p>0.05). On the other hand, there was significant difference between the productivity grant researchers in the orientations at masters, final term papers, and scientific initiation levels.
Tukey's Post Hoc test was conducted to carry out a pair-to-pair comparison of the orientations at masters, final term papers, and scientific initiation levels. Table 15 displays the groups whose pair-to-pair comparison presented significant differences and presents the results of Tukey's Post Hoc test in orientations for scientific initiations.  DT2 researchers are predominant in scientific initiation orientations, presenting a significantly higher average than PQ1A, PQ1B, and PQ1C researchers. Apparently, scientific initiation orientation attracts DT researchers in larger proportions.
The possibility of innovation in scientific initiation orientations may contribute to this scenario.
In general, it becomes clear that there are categories of scientific and technical productions that present a larger total production than others. Considering the average per researcher (relative production), however, this value decreases, displaying no representativeness in relation to the others. Graph 4 displays the total of productions in terms of researcher category.
Graph 7. Total of scientific productions comparison.
Graph 7 illustrates a compilation of the different types of production (articles, technical, artistic, congresses and books) of PQ and DT grant researchers. Due to the high concentration of researchers in category 2, the number of productions is relatively higher than in category 1. The scientific productions represented in Graph 4 were: 41% in congresses, 29% in scientific articles, 25% in technical productions, and 5% in books. Thus, it is possible to infer that the main means of divulgation selected by the researchers are those that are faster and more far-reaching, in addition to being considered in the postgraduate programs scoring system in which they are inserted.
Despite the considerable production in the Production Engineering field, it is understood that its production through scientific articles is lower than that of other knowledge areas. According to Leite (2014), from 2008 to 2012, the Production Engineering field occupied the 8 th position in the scientific production ranking, behind Clinical Medicine, Vegetable and Animal Sciences, Agricultural Sciences, Chemistry, Physics, Social Sciences, Biology, and Biochemistry. The performance issue worries the researchers, seeing that CNPq (2017d) establishes that, if the researcher's performance does not correspond to their level, the agency may not renew the research grant, that is, the researcher loses their financing. That is expressively bad for the researcher, given that Cavalcanti and Pereira (2008) regard the researchers who receive the productivity grant from CNPq as the elite of Brazilian research, reinforcing the importance and difficulty of joining this select group.

Final Considerations
This work had the purpose of mapping the scientific production of the researchers recipient of CNPq's grants for productivity in research (PQ) and productivity in technological development and innovative extension (DT), in the Production Engineering field in the 2013-2016 period.
We found that PQ researchers are predominant (76%), but in comparisons with previous studies, DT researchers are ascending. For both categories, level 2 holds the largest number of researchers, due to it is the first level in which the doctor enters. After the researchers start receiving the grant, they must to stand out and progress in their scientific production, ascending to higher levels.
Regarding published articles in the PQ category, PQ 1B researchers had the best average. However, it was still lower than that of grant researchers in the DT category, where DT 1D researchers held the best average, 21.5 published articles.
Despite the representativeness in article production, PQ 2 researchers had a greater number of A1 publications than others. If we consider the articles average per researcher, however, the PQ 1B level had the best performance in A1 publications. We also found that level 1 researchers do not focus on journals of low Qualis levels to publish their researches. In turn, level 2 researchers are less experienced and expect to publish in high-quality journals, but when their works are rejected, they start "betting" on lesser-quality journals.
Another highlight is that PQ grant researchers maintain a homogeneity in the production of scientific articles at all Qualis levels. In contrast, DT researchers display more heterogeneity in their publications, with a tendency towards publishing in lower Qualis levels.
For both categories, PQ and DT, the journals unrated by Qualis/Capes held the highest number of publications, which implies that the grant researchers do not only seek journals indexed in the Qualis system, but also those that may have JCR.
Regarding technical production, DT researchers are more expressive. DT2 researchers presented a technical production average significantly higher than those of the other DT groups. From that analysis, we identified a gap in the study, in which we cannot affirm that the predominance of the DT category in technical productions is a specific trajectory; further studies are required to confirm that hypothesis.
We also found that, for both books and collections, none of the researcher groups reached the average of one work published per researcher. Nevertheless, the DT category attained the highest averages: DT2 in book production and DT 1D in collections and book chapters.
Concerning works published in congresses, while every category was represented, DT 1D researchers had, in average, more complete works published. PQ 1D researchers stood out in the publication of expanded abstracts, and towards the published abstracts, PQ 1A researchers presented the best average.
Regarding orientations to students, PQ 1A researchers stood out both at post-doctorate and undergrad (final term papers) levels. While PQ 1B researchers were predominant in doctorate-level orientations, PQ 1D researchers at masters level, and DT 2 researchers in scientific initiations.
Another finding was that some levels of researchers are more representative in terms of the number of scientific productions, which is justified by the larger number of members. However, if the average of production per researcher is taken into account, that value decreases, not being representative in relation to the other levels.
Finally, we established that congresses and scientific articles are the preferred means of divulgation of the researchers, due to their fastness and more far-reaching.
As limitations of this work, we list: (i) the study did not analyze the publication of patents by the researchers; (ii) only a 4-year period was examined, which could be broadened to check the evolution of the productivity by the grant researchers in the field; and (iii) the study focused only in the Production Engineering field, which also could be expanded to other fields, in order to compare different knowledge areas.
As possible future studies, we suggest: (i) to investigate later periods and comparing it with the previous 4-year interval performed in this study; (ii) to analyze in depth the patents produced by the researchers since differently of the article publications where exists the Qualis/Capes and other indicators, there is not any publicized indicator to classify the relevance of technical production, extremely important in the context of DT grant researchers; (iii) to compare the scientific production of PQ and DT grants of the Production Engineering with researchers of other fields.