Usage of statistical and substanive significance in Pedagogika and Pedagogicka orientace in the last 10 years: statistician's insight
Abstract
The of this paper is to highlight the problems in the use of statistical and substantive significances in the last decade in two leading journals of teachers, particularly in the journal Pedagogika and Pedagogicka orientace. The main part of the studz is devoted to content analysis of two mentioned journals.
The article offers the results of the research in three areas: incorrect use of statistical tests, the incorrect interpretation and application of effect sizes. In conclusion the author in addition to traditional summary of results offers potential proposals for improvements in publishing and educational practice in the Czech educational community.
The research carried out showed that quantitatively oriented texts are relatively few (two pedagogical journals were published 84 texts in 10 years). More than half of those texts uses the concept of statistical significance, although it presented data do not allow.
Quite a few are frequent errors in the interpretation of statistical significance, the most common include mechanical work with the data (most often use asterisks). Contrary to current guidelines in the journals surveyed taken quite a few effect sizes, in addition to taking only the simplest level of substantive significance.
It is also alarming that about a third of the articles analyzed lacked substantive interpretation of the results.