Abstract
An increasing number of national and international funding and statistical agencies utilize Web of Science (WOS) as a source of data influencing their decisions and analyses of research outcome. However, currently existing data sources for scientometric research, including WOS, are far from being perfect.
Most of the imperfections are caused by uneven coverage, errors or changes in indexing policies, or mistaken or ineffective retrieval strategies employed by the users. Thus, it is important to be aware of the critical elements of scientometric evaluation, as inappropriately designed search procedures may lead to confusing or false-positive results.
This paper presents the analysis of a series of previously published papers, which were affected by errors of omission and commission due to changes in WOS abstracting policies. When comparing WOS Topic search with WOS Title search, substantial differences arose.
Number of papers published every year on cruciate ligament was shown to remain unchanged since early 1980s, when employing WOS Title search. Similarly, trends in number of citations on this topics remain unchanged through the long period of time, reflecting only increasing amount of citable papers available.
The findings differ from those reported previously based on WOS Topic search, as improvement in the search protocol fully explained and rejected the previously reported steep increase in publications on cruciate ligament, air pollution, and oral lesions since 1991. The different outcomes compared to the other search protocols were caused by variations in WOS abstracting policies, such as exclusion of the address field, keywords, and exclusion or changes of the country codes or names.
Despite the percentage of WOS records lacking these fields is decreasing in time, inclusion of such records hinders the ability to use the respective fields in any long-term searches using the WOS database. The results suggest that WOS Topic search is not the appropriate tool to search for time-dependent changes in publication productivity.