Exam difficulty control knob
Recently, there was a discussion at Abi’s blog about the JEEs alleged bias against women.
So, everyone went off and came back with statistics that hammered home the fact that very few girls are in the IITs / BITS (post BITSAT)/ etc. Even the west is not an exception it seems. What do these facts tell you about the selection machines ? Not much, really. If you want to prove bias in the JEE, this is the wrong kind of data.
The JEE selection machine cannot distinguish between women and men, unlike the IIM selection machine which can; due to the group discussions and interviews. Given this, how does one go about proving bias in the JEE ? It is possible, but not easy. You have to show that the questions asked in the JEE are somehow more likely to be answered correctly by men. In doing so, you have to factor out effects of differential preparation and coaching. If you dont, you will only show that the JEE is biased against “uncoached girls”.
Allow me to drive home this point further. If we use the same yardstick of checking the output of the JEE selection machine as an indicator of bias, it would seem like the exam is heavily biased against Mallus !! (Out of 979 candidates from the south in a recent year – 769 were from AP compared to just 32 from Kerala). There were probably more girls from AP than from any gender from Kerala or TN or Karnataka. (See an earlier post on this) Can we then claim that the JEE is biased in favour of APites ? Yes, but you have to prove it. You could start by examining whether the JEE has questions that give the AP student an unfair edge. For example : are there questions on Chiranjeevi films ? It may sound frivolous – but if you want to assert bias in the JEE – you have to pick up this line. After all, all exams are biased in favour of those who prepare for it.
Anyway, this blog post is not about the JEE. It is about the easyness of exams and its impact on representation.
Consider a mythical exam toughness knob.
Establish the boundary conditions first.
1. What is the toughest possible exam (level 10) ?
2. What is the easiest possible exam (level 1) ?
Think about it.
Here are my answers. Forgive my pretentious tone (if you havent already gotten used to it by now).
The toughest exam is one in which no amount of preparation will give you even a tiny bit of advantage. The best example is surprise! “a lottery”.
The easiest exam is one in where you will gain a huge advantage with the minimum amount of work. The key is the “minimum amount of work” part. If there was no work at all then it would be akin to a lottery because everyone will do no work.
Now consider a population of candidates with different individual characteristics (Sex, Economic Background, Caste, State, Rural/Urban, School board, etc).
Now, play with the toughness knob and look at the output of the exams. Is each individual characteristic represented in the same proportion in the output as in the input ?
Ironically, both extremes will throw up comparable representation figures among all variables. A lottery is cleaner than a ridiculously easy exam because it does not have a bunching problem (tie breaker).
If we want to measure representation – we have to measure all the free variables. Otherwise, the exercise is flawed or the exercise shows interest in the results. For example, if we miraculously solve only the gender “bias” – then we will just have ten new ones. Would it be ok if the new JEE was biased against lower-caste women ? What about men from a poor rural background from TN state board ? The complexities are mind boggling.
This is why we must be very careful not to pass on the task of correcting societies’ real or perceived ills on to an hapless entrance exam. We have reservations for that.