Research Observatory: Bibliometric experiment with the full text of research papers

BIBLIOMETRIC EXPERIMENT WITH THE FULL TEXT OF RESEARCH PAPERS

Are self-citations a normal feature of knowledge accumulation?
By Vincent Larivière

Presentation slides

Science is a cumulative activity, in which past knowledge serves as a foundation for new knowledge. One of the mechanisms through which the cumulative nature of science manifests itself is the act of citing. However, citations are also central to research evaluation, thus creating incentive for researchers to cite their own work. Therefore, such self-citations have been one of the most constant criticism against the use of citation indicators for the measurement of research impact. Using a dataset containing millions of papers and disambiguated authors, this talk will examine the relative importance of self-citations and self-references in the scholarly communication landscape, their relationship with age and gender of authors, as well as their effects on various research evaluation indicators. It will also present the results of a comparison of the content of cited and citing papers, thus making it possible to test whether researchers cite their own work in order to inflate their impact indicators. The talk with conclude with a discussion of the role of self-citations in the research ecosystem.

Understanding scientific disagreement
By Dakota Murray

Presentation slides

Healthy disagreement among scientists drives the creation of new knowledge and is a necessary precursor to consensus upon which technologies, policies, and new knowledge can be built. Yet, in spite of its prominence in popular and theoretical models of scientific progress, disagreement has received little empirical attention, with progress stymied by a lack of appropriate data and widely-accepted quantitative indicators. In this talk, we outline progress in overcoming these challenges, illustrating how increasingly-available full-text data and new approaches to measuring disagreement are paving the way for a more comprehensives, empirical, and quantitative understanding of the salience and features of disagreement in science at multiple levels of analysis. Using a rigorously-validated cue-word based approach, instances of disagreement are identified from the citation sentences of millions of publications, and incorporated into a singular indicator of disagreement. Using this indicator, we simultaneously reveal the structure of disagreement between macro-level fields and the enormous heterogeneity across meso-level subfields. At the micro-level, we complement these data with published comments—the most unambiguous instance of criticism in science—in order to better understand the sociological drivers of disagreement, including author gender, seniority, prestige, and more. This project establishes a firm methodological and empirical foundation for a science of scientific disagreement, which will prove essential for validating theories of scientific progress, building tools for scholarly search and discovery, designing consensus-aware science policy, and for effectively communicating epistemic uncertainty and consensus to the public.