Claims about how high-achieving schools promote stronger academic progress of individual pupils need to be questioned – matters of self-concept may be relevant
When choosing a school, parents often assume that a higher achieving intake will benefit their child’s learning – and this may sound conventional wisdom. Indeed, educational research has, at times, reported a small and positive association of school-average achievement with individual students’ academic progress. However, how much should we trust these findings?
With our work, we argue that reported positive associations between school-average achievement and students’ academic progress may often be an artefact of inadequacies in the statistical methodology upon which relevant research has been based: in fact, these associations are non-existent or even, negative. The term “phantom effects” has been used to describe this phenomenon— here you see it, here you don’t. Importantly, we claim that the relationship between school-average achievement and student academic progress cannot be considered independently of its relationship with students’ academic self-concept. In this respect, we verify a well-known result in educational psychology, the “Big-Fish-Little-Pond effect”, and we do so with students as young as seven to nine years of age.
What is the so-called “Big-Fish-Little-Pond effect”?
Academic self-concept refers to that specific component of self-concept (a person’s perception of self) that denotes how individuals perceive their own academic abilities and competencies in a specific subject. Cultivating academic self-concept has been recognised as a fundamental responsibility of schools for over fifty years. Indeed it is highly valued as an educational outcome in its own right. However, it is also related to academic achievement and it facilitates the development of other desirable outcomes. Importantly, there is solid evidence that academic self-concept and academic achievement are reciprocally related, so that higher academic self-concept leads to higher academic achievement and vice versa. For this reason, the recommendation of educational researchers is, if possible, to consider both outcomes in studies searching for factors enhancing student learning.
Self-concept research is well-grounded in a rich theoretical literature drawing from educational psychology, social psychology, psychophysical research, and sociology. It consistently predicts a negative association between school-average achievement and academic self-concept, even though individual student achievement positively predicts academic self-concept: the so-called “Big-Fish-Little-Pond Effect”. The phenomenon (see Figure 2) refers to the tendency of equally able schoolchildren to demonstrate lower academic self-concept when attending a school with a higher-achieving intake than equally able students who attend a school with an overall low- or medium- achieving intake. Its theoretical explanation draws upon social comparison theory and frame of reference effects. These perspectives attribute this phenomenon to comparisons that students use to evaluate themselves, including comparisons with their school peers. Thus, when students compare their performance at school with higher-achieving children, they conceive themselves as less able academically.
Suppose we accept that the intake of higher achieving schools serves to boost individual student academic progress. In that case, this should also predict higher self-concept, given the reciprocally positive effects between academic achievement and academic self-concept. Why is it then that persistent research evidence, based on the Big-Fish-Little-Pond Effect, implies the contrary? Our study shows that such paradoxical findings may arise due to measurement error and uncontrolled pre-existing differences distorting the interpretation of the results.
How is error of measurement defined?
Inherent in assessing student achievement in an academic major (e.g., mathematics or science) is the prevalence of random measurement error. For instance, whenever a multiple-choice test is used to assess students’ performance in mathematics, the scores obtained do not reflect students’ true knowledge in a perfectly reliable way. While it is impossible to observe a student’s true academic achievement, the magnitude of the error of measurement can be estimated and corrected for in research analysis. This is good news, since theoretical and empirical research evidence suggests that unless educational researchers do so, any observed associations between school-average achievement and student academic progress will be estimated more positive than they actually are.
What did we show with our research study?
We based our study on a large sample of English primary school students (19,059 from 593 schools) that were followed through their first four years of schooling. The data were kindly provided to us by the Performance Indicators at Primary School (PIPS) project, run by the Curriculum, Evaluation and Monitoring Centre at Durham University. We evaluated the association of school-average achievement in mathematics in Year 1 with students’ individual progress in mathematics from Year 1 to Year 4. No evidence for a positive association was found – in fact, when we used models that corrected measurement error in students’ achievement score, a small, negative association was retrieved. Moreover, with our study we provided evidence for Big-Fish-Little-Pond effects for the students in our sample, and we showed that the Big-Fish-Little-Pond effect could be a potential mechanism leading to the negative association between school-average achievement in Year 1 and students’ progress from Year 1 to Year 4. Our findings question previous research suggesting that attending a school with higher average achievement necessarily advances students’ outcomes
How did we control measurement error ?
We used a set of statistical models referred to in the literature as “multilevel latent variable models”. These have specifically been developed to correct for measurement error in students’ test scores and correct for sampling error in observed school-average achievement. Sampling error results from using a finite number of students from each school to obtain an approximation of school-average achievement, which is only a sampling from the total number of students in the school. Such models are also called “doubly latent models”: they “doubly” correct for two different sources of error (measurement error and sampling error). They are also “multilevel”; they take into account both “levels” operating in the analysis: that of the students (observed student achievement), and that of the school (school-average achievement).
What makes our study original?
Given the widespread misconceptions about both the direction and the appropriate methodology for testing the association between school-average achievement and individual academic outcomes, the basing of our results on a large nationally representative sample of young UK students makes an important contribution to existing research that reported similar findings with US students. What is important is that, whilst previous methodological studies clearly demonstrate the phantom effect, only a few propose or apply any statistical models to correct for measurement error, reasonably, since the methodology that we employed has only recently been readily available to applied researchers.
Another important contribution of our study to what is known so far about how studying among high-achieving peers influences students’ academic outcomes is that we verified the Big-Fish-Little-Pond effect. Relatively few studies have done so with students in the early stages of primary and especially so for students as young as the first year of primary school. Indeed, one would not expect social comparison processes underpinning the Big-Fish-Little-Pond effect to be evident at such a young age, since students’ self-concept becomes more aligned with their achievements as students grow older.
Why should our findings be of concern to parents and practitioners?
Our findings raise concerns about the validity of previous and current educational studies reporting a positive association between school-average achievement and students’ academic progress, without taking into account the error of measurement in research analysis. In fact, the prevalence of measurement error in baseline achievement (students’ achievement on school entry) has already been shown to seriously bias the value-added measures of schools’ ‘progress’ that are used for accountability purposes in England, thereby deeming the putative superiority of grammar schools’ performance ambiguous.
In interpreting our study’s findings and understanding their implications for policy and practice, a distinction should be made between the school peer effect and institutional effects. The school peer effect purely reflects the association between students’ individual achievements and the way in which they interact with each other in the school’s context. Institutional effects on the other hand, refer to the influence that the school processes and practices (e.g. the classroom’s instruction) and the existing school structures (e.g. school resources and facilities, school size) may have on students’ progress. Both effects contribute to the observed association between school-average achievement and individual student achievement. While our study suggests that the overall association may not necessarily be positive, it does not imply that institutional effects do not exist. In fact institutional effects may result in better outcomes, so that students in schools with a low school-average achievement can achieve higher, had these schools been given the same school resources and teaching quality advantages as schools with a high-average achievement.
Authored by:
Ioulia Televantou
Ioulia Televantou a researcher in quantitative social science, with a special interest in the application of multilevel modelling and latent variable models in large-scale, empirical driven educational research, seeking to assess the impact of peers/peer assisted teaching on students’ learning. She is currently a lecturer in “ Research Methodology & Statistics” at the Theological School of Cyprus, and also a research associate at the University of Nicosia Medical School.
Herbert Marsh
Herbert Marsh is Professor Emeritus at the Department of Education, University of Oxford
Theresa Dicke
Theresa Dicke is Senior Research Fellow at the Australian Catholic University
Christos Nicolaides
Christos Nicolaides is a Marie S. Curie Fellow and an Assistant Professor at the Department of Business and Public Administration at University of Cyprus
Leave a Reply