Linguistic and semantic characteristics of articles and peer review reports in social and medical sciences: analysis of articles published in Open Research Central (Pubmet, 2022)
Recently, initiatives to share data on peer review (Squazzoni et al, 2020) have brought about better understanding of the peer review process across different disciplines (Buljan et al, 2020, Squazzoni et al 2021a, Squazzoni et al 2021b). Some studies (Strang & Siler, 2015; Siler & Strang, 2016) examined the changes from initial submissions to journals to publication. Another study (Teplitskiy & Bakanic, 2016) explored how well peer reviews predict the article’s citation impact. A study of almost half a million peer review reports from 61 journals (Buljan et al., 2020) showed that peer review reports were longer in social science than in medical journals. It also showed that their language characteristics differed across disciplines (Buljan et al., 2020).The aim of our study was to further examine structural and linguistic differences between articles and the characteristics of the peer review process, including the language used and outcomes of peer review reports, in medicine and health sciences vs. social sciences.To select the articles we used the search engine of the Open Research Central (ORC) platform, which covers different fields, and applied the following filters: “Subject area: Medical and health sciences”, and “Subject area: Social science”. “Article type(s): Research article” filter was also applied. We ended up with 1912 medical and 477 social science articles. To create the samples of articles with clear medical/health vs. social sciences content, we excluded those with a tag for medicine and health sciences and any other disciplinary field except biology and life sciences and those with a tag both for social sciences, and biology and life sciences. This left 408 medical and 54 social science articles. Using the DOIs of filtered articles, we downloaded them manually in an XML format. Articles that had not been reviewed were excluded, yielding a total of 51 articles with a social sciences tag and 361 articles with a medicine and health sciences tag, as well as their peer review texts.We analysed them using the Linguistic Inquiry Word Count (LIWC) text analysis software and word embeddings, a method in which words are given mathematical vector representation. To assess the differences between the articles and their reviews across medical and social sciences, we used the one-way ANOVA and post hoc Tukey’s test. For multivariate frequency distribution of the variables we used the contingency test. All analyses were carried out in JASP, version 0.14.1.Articles from medicine and health sciences and those from social sciences differed in structure. The median word count in the introduction section was 674 (IQR=308.0–637.0) for social sciences and 431 (IQR=420.5–1029.0) for medical sciences. The conclusion section was also longer in social sciences, with 263 words (IQR=135.5–516.0) vs. 94 words (IQR=64.0–168.0) in medical sciences (p