Raj Chetty at GEN
15/12/2011 § Leave a comment
The Government Economics Network held a conference yesterday. Raj Chetty from Harvard University presented on his research into schooling, test scores, teacher abilities, and pupils’ subsequent earnings.
His research (summary here) found, not surprisingly, that kindergarten test scores correlated with later earnings. More interestingly, the earnings correlated with the scores of the other children in the kindergarten class. What made this finding intriguing was that children were randomly assigned to their kindergartens. Because of the design of the education policy, the researchers could discount selection bias. This was a great example of quasi-experimental research following public policy changes, exactly the sort of thing Tim Harford has advocated.
Chetty also presented on later research that concerned teacher quality and its effects, using a longitudinal dataset of pupils’ performance. They constructed a measure of teacher ‘value added’ from changes in pupils’ test scores. That measure correlated with later earnings (+), teen pregnancies (-), home ownership (+), and more. Again, really fascinating quasi-experimental design and interesting techniques to separate signal from noise.
Also, he had a somewhat hopeful message. Better teachers can improve pupils’ outcomes after controlling for parental characteristics. So, education should be able to improve intergenerational class mobility. What’s more, the test scores produced a measure of ‘value added’ that provides information about teacher quality. That gives us a way to measure teacher performance that isn’t hopelessly correlated with pupils’ abilities and socio-economic status.
But, I have some quibbles:
- as tests have become more linked to pay and performance, the incentive to manipulate results has increased; teachers and administrators have responded to those incentives. So, this research may be a good look at the past, but may not help for the future. (To be fair, the presentation linked above does contain ‘Test manipulation’ as a concern, but IIRC it wasn’t on the GEN presentation)
- they show that parent characteristics do not correlate with teacher value add (slide 24). I found this worrying, rather than a vindication of their metric. If high incomes buying homes in nice neighbourhoods with high property taxes doesn’t also buy good teachers, then they have uncovered a serious government&market failure
- Chetty discussed the fact that the results were on the order of a percentage point or two in income. He pointed out that the difference between trend and actual GDP is a few percentage points (in the US) and we think that’s a problem. Therefore, the impact of good teachers is also economically significant. That wasn’t a bad line to take, but I am more interested in how ‘good teacher’ lines up against other influences on pupil success, like parental income and parental education
- in particular, I would be interested to know how teacher value added compares with or interacts with expectations of pupils. The Pygmalion effect (in both directions) could be either a major contributor to ‘value added’, or could be an additional effect that is either more or less important.
A big hand to GEN for their first conference. By the metric ‘questions generated’, it certainly had value-add.