Education policy: solving the wrong problem

24/01/2014 § 1 Comment

The big political news yesterday was the announcement by National of a new education policy. It will put in place a few experts — super-principals and super-teachers — to oversee and guide schools and regular principals and teachers. And, it will pay them a premium.

The usual economic models I use don’t seem to apply here. This is more about management, for which I have no ready models. If I fall back on thinking about incentives, I don’t see anything in here that changes the incentives for teachers. In fact, it creates incentives for the incoming managers to (a) overstate the deficiencies of the existing situation, and (b) institute new policies and procedures so as to be seen to be doing something. But I don’t have a great economic insight for you, sorry.

I do know a little about the education system, though, and it seems like this policy is solving the wrong problem, or a problem that does not exist. Education in New Zealand is generally good. Overall, students fare well on standardised exams and international rankings. They are able to go to overseas universities and do well. On average and in the main, education is on par with the rest of the developed world.

Whether warehousing kids for 13 years is actually good for their development as creative human beings, that’s another argument, but we’ll skip it here.

Despite the good average performance, the country has two problems:

  • the long, fat tail of low performance, which correlates with race and poverty
  • lack of public resources for top students, to help them achieve their potentials.

The first problem is that some kids in some communities are disadvantaged in all kinds of ways, and that shows up in school results. Improving their performance is partly about the schools themselves, and partly about helping the communities and families overcome the disadvantages. It also involves some reflection on why the disadvantage is there and continues, which is a massive social conversation that the inequality campaigners are attempting to have (but, in my opinion, going about the wrong way).

The second problem comes out of the drive for standards. If teachers are given incentives to lift kids to some arbitrary level, they will work towards it. Any kids who naturally get there will be ignored, because teachers aren’t paid to help them excel. This imposes costs on the families and communities, who have to pay for all the extracurricular stuff that helps the kids excel, or move them to the expensive schools that offer more options. It also creates losses for individuals and societies, who aren’t as creative and inventive as they could have been.

The new policy addresses neither problem. Creating a national system of administrators and consultants (which is what the policy seems to do) isn’t necessary. We need targeted intervention to help the poorest performing kids, and we need a shift in focus and policy to create incentives for teachers to nurture the best and brightest.

Inequality, mental models, and counterfactuals

09/09/2013 § 4 Comments

In the recent book Inequality: A New Zealand Crisis, Jonathan Boston has a really useful chapter. It probably should have started the book as a way of laying some intellectual groundwork, instead of being Chapter 5. It raises the same point as Matt Nolan does at TVHE, quoting Amartya Sen, but with more detail. Here’s Boston:

As highlighted in this chapter, there is almost universal acceptance that equality matters. Yet there is no consensus on what kind of equality should be championed.

And here’s Nolan:

 Everyone, especially those who are more extreme in any given “political dimension” cares about equality of something – and the underlying reason why there are trade-offs stems from (as Sen discusses in the book) the heterogeneity of individuals!

Boston tries to cope with the different equality targets in two ways. First, he says that some equalities are more important than others, arguing that there is an inequality of equalities. With this, I believe he steps right back from his initial understanding — that reasonable people may reasonably disagree. Secondly, he suggests a certain pragmatism, or specific egalitarianism. While this seems attractive, it papers over the real conflict amongst theories of egalitarianisms with a poorly defined set of concrete goals.

In doing this, Boston shows how hard it really is to have evidence-based policies. When it comes time to make a decision, we have to take values and preferences into account. A particular set of evidence can be made to suit a range of policies, once you introduce different values. I’ve not read Peter Gluckman’s new report on evidence-based policies (pdf), but the reporting I’ve seen suggests that it is insufficiently attuned to this problem.

There is a second problem less well understood. When we make judgements about inequality, we are using mental models of social systems to create counterfactuals. For example, if one says, ‘they wouldn’t be poor if they weren’t lazy’, there is a mental model of society that underpins the judgement. That model has a weighting on ‘effort’, as well as weightings on other things like ‘education’, ‘social network’, ‘ethnicity’, ‘gender’, and more. The ‘effort’ weighting is sufficiently large to counteract any negatives from the other factors. We could, through interviews and surveys, estimate the parameters that people apply to those factors.

We all have these models. They are all wrong. I say that with conviction because ‘all models are wrong.’ They are always partial — they have missing variables — and the parameters are estimated from a sample of observations rather than the population. So, in my interpretation of my experience, it may be that ‘effort’ is sufficient to make a person not-poor. That doesn’t make it so, either for my experience (which suffers from observer error) or for the wider world.

In economic analysis, counterfactuals are hard to construct. You have to decide which factors are important and how important and over what time. If I’m being cynical, I might say that the counterfactual is the most important part of any cost-benefit analysis, and it is literally something we just make up. With data and evidence, mind you. But we make it up just the same.

How much harder is it, then, to understand the counterfactuals that people create from poorly-specified mental models of complex social systems?

More evidence would help, so it’s good to see Gluckman encouraging the government to find and use more. The findings will rarely be conclusive, however, as evidenced by Boston’s discussion of equality.

Science stumble with young students

01/09/2013 Comments Off on Science stumble with young students

The science sector in New Zealand wants to get more people — particular young people — interested in science. It believes that science careers get short shrift when students are planning their education. It also wants to encourage more girls into STEM subjects. Don’t take my word for it. There’s a 2008 Science Maniesto from the Royal Society explaining all this.

I’m certainly in favour of my daughters having interesting, rewarding jobs. If a science career provides that, great. There’s been some science talent in the family, so it’s a possibility.

We’ve been supporting what science is available for primary and intermediate girls. Recently, one daughter participated in the NIWA Wellington Science Fair. From our experience, the event didn’t help get kids fired up for science.

The most important thing to realise is that these kids have choices. Sure, science is one possibility, but so are medicine, law, finance and more. Science has to be appealing. So let’s compare:

  • My daughter’s division had over 50 entries. Only four kids won prizes. Most of those kids won more than one prize. By comparision, a singing competition might have four prizes in a division with maybe a dozen entries. One of their maths competitions has five prizes for the 30 or so schools who participate.
  • There was no feedback. The kids have no idea what they did well and where they fell down. They don’t know what they could do better. By contrast, performance judges fill out sheets for each performer. They typically give positive and critical comments, which helps kids both understand their mark on the day and identify things to improve.
  • The best we can figure is that the judges liked some combination of science, application and presentation. But there’s no way to gauge how much those things contribute to the rankings. On the other hand, a maths competition is judged in terms of right and wrong answers. If your team gets it right, you get the point. The kids are competing against the maths problems as much as they are competing against each other. The ICAS and Australian Mathematics Competitions are similarly based on getting the answers right, not tickling the judges’ fancy.
  • My daughter’s girls’ school had one of the largest contingents from any school in her division. They won nothing. Nada, zippo, zilch. We can talk about other schools having more experience with the competition and larger schools having more resources for extra-curricular activities and the rest. But none of those explanations changes the experience this girls’ school had of sending a big group to a city-wide event and coming back empty-handed.

What my daughter and her classmates experienced seemed to be a subjective, secretive, winner-take-all tournament. Now, obviously, these folks can run any kind of competition they want. They just shouldn’t be surprised when these girls don’t rush back to do it again, and find something better to do with their time.

Organising some thoughts on process and goals

22/11/2012 § 1 Comment

There is, in Wellington and elsewhere, a push to make government more innovative. The idea is that the private sector succeeds through innovation, so the public sector should follow suit.

Sometimes, the focus is on getting good feedback loops. If each policy initiative is viewed as an experiment of sorts, then bureaucrats should be collecting data from each experiment to determine what works. This is one of the messages of Adapt.

Sometimes, the focus is on being innovative for the sake of innovation, on challenging the status quo because it needs to be shaken up. Thus, I’ve heard that some Ministers have challenged their Ministries to ‘scare me’ or ‘surprise me’. Or, to rely on a cliche, to think outside the square.

Here’s a 2×2 matrix that’s helping me organise my thinking:

Private businesses have pretty clear goals. Mainly, they make money. They might also provide some intangible benefits — being master of your own destiny, steward of the land, conqueror of markets, etc. But mostly, the goal is pretty clear.

How they achieve that goal is wide open. They try stuff out, make big changes and incremental changes, keep what works and discard the rest. Once they figure out what works, that might get turned into codified knowledge. Businesses produce standard operating procedures and policy manuals of all sorts. Franchises are an example of making business processes regular and predictable.

Government, on the other hand, doesn’t necessarily have clear goals. It’s a bit about keeping people happy, keeping things ticking over, improving the living standards of some people while not harming others too much, responding to pressures from all sides. The goals are fuzzy and changing. The bureaucracy compensates by creating clear processes. When things go wrong, bureaucracies often defend their actions by saying, ‘we followed the correct procedures.’

The push for innovation puts government in a new quadrant. Now, bureaucrats are asked to challenge their own processes, to think continuously about how they can do better. The goals are still fuzzy — that’s the nature of governing — but now the process is, too. This quadrant creates a quandary: how are they to know what ‘better’ is?

Framing the problem with the 2×2 matrix leads me to three thoughts:

  • experimenting with governance is hard, as hard as research is to do well. Researchers are highly skilled, highly qualified, and relatively well paid. If we want good experimental governance, then we have to provide the necessary resources. Otherwise, it will be like research done by people without training or skills — think term papers written by undergraduates
  • to help staff, the senior managers should try to provide clear interim goals. This will allow staff to focus on experimenting with processes. Otherwise, staff will be experimenting with both goals and processes, which is a recipe for chaos
  • alternatively, setting up clear measuring and monitoring processes can allow experimenting around goals.

Graphically, I’m suggesting that Ministries shouldn’t stay in the upper left quadrant if they can help it. It’s difficult and expensive. Instead, they should create some clarity around either goals or processes (move right or down), and then experiment around whatever is left.

Where Am I?

You are currently browsing entries tagged with policy at Groping towards Bethlehem.