Multiplicity arises when data analysis involves multiple simultaneous inferences, increasing the chance of spurious findings. It is a widespread problem frequently ignored by researchers. In this paper, we perform an exploratory analysis of the Web of Science database for COVID-19 observational studies. We examined 100 top-cited COVID-19 peer-reviewed articles based on p -values, including up to 7100 simultaneous tests, with 50% including >34 tests, and 20% > 100 tests. We found that the larger the number of tests performed, the larger the number of significant results (r = 0.87, p < 10 −6 ). The number of p -values in the abstracts was not related to the number of p -values in the papers. However, the highly significant results ( p < 0.001) in the abstracts were strongly correlated (r = 0.61, p < 10 −6 ) with the number of p < 0.001 significances in the papers. Furthermore, the abstracts included a higher proportion of significant results (0.91 vs. 0.50), and 80% reported only significant results. Only one reviewed paper addressed multiplicity-induced type I error inflation, pointing to potentially spurious results bypassing the peer-review process. We conclude the need to pay special attention to the increased chance of false discoveries in observational studies, including non-replicated striking discoveries with a potentially large social impact. We propose some easy-to-implement measures to assess and limit the effects of multiplicity.
【저자키워드】 SARS-CoV-2, Epidemiology, multiple hypotheses testing, multiple testing problem, false discovery rate (FDR), environmental research, health geography, 【초록키워드】 COVID-19, database, observational studies, Data analysis, Web of Science, Abstract, Exploratory analysis, type I, P -value, measure, p -values, article, multiplicity, inferences, Effect, widespread, limit, researchers, performed, examined, proportion, reported, correlated, addressed, arise, peer-reviewed, bypassing, 【제목키워드】 COVID-19, multiplicity, peer,