Reality Check India

Exam difficulty control knob

Posted in Uncategorized by realitycheck on May 2, 2007

Recently, there was a discussion at Abi’s blog about the JEEs alleged bias against women. 

So, everyone went off and came back with statistics that hammered home the fact that very few girls are in the IITs / BITS (post BITSAT)/ etc. Even the west is not an exception it seems. What do these facts tell you about the selection machines ? Not much, really. If you want to prove bias in the JEE, this is the wrong kind of data.

The JEE selection machine cannot distinguish between women and men, unlike the IIM selection machine which can; due to the group discussions and interviews. Given this, how does one go about proving bias in the JEE ? It is possible, but not easy. You have to show that the questions asked in the JEE are somehow more likely to be answered correctly by men. In doing so, you have to factor out effects of differential preparation and coaching. If you dont, you will only show that the JEE is biased against “uncoached girls”.

Allow me to drive home this point further. If we use the same yardstick of checking the output of the JEE selection machine as an indicator of bias, it would seem like the exam is heavily biased against Mallus !! (Out of 979 candidates from the south in a recent year – 769 were from AP compared to just 32 from Kerala). There were probably more girls from AP than from any gender from Kerala or TN or Karnataka.    (See an earlier post on this) Can we then claim that the JEE is biased in favour of APites ? Yes, but you have to prove it. You could start by examining whether the JEE has questions that give the AP student an unfair edge. For example : are there questions on Chiranjeevi films ? It may sound frivolous – but if you want to assert bias in the JEE – you have to pick up this line. After all, all exams are biased in favour of those who prepare for it.

Anyway, this blog post is not about the JEE.  It is about the easyness of exams and its impact on representation.

dial.jpg 

Consider a mythical exam toughness knob.

Establish the boundary conditions first.

1. What is the toughest possible exam (level 10) ? 

2. What is the easiest possible exam (level 1) ?

Think about it. 

Here are my answers. Forgive my pretentious tone (if you havent already gotten used to it by now).

The toughest exam is one in which no amount of preparation will give you even a tiny bit of advantage.  The best example is surprise! “a lottery”.

The easiest exam is one in where you will gain a huge advantage with the minimum amount of work. The key is the “minimum amount of work” part. If there was no work at all then it would be akin to a lottery because everyone will do no work.

Now consider a population of candidates with different individual characteristics (Sex, Economic Background, Caste, State, Rural/Urban, School board, etc).

Now, play with the toughness knob and look at the output of the exams. Is each individual characteristic represented in the same proportion in the output as in the input ?

Ironically, both extremes will throw up comparable representation figures among all variables.  A lottery is cleaner than a ridiculously easy exam because it does not have a bunching problem (tie breaker).

If we want to measure representation – we have to measure all the free variables. Otherwise, the exercise is flawed or the exercise shows interest in the results. For example, if we miraculously solve only the gender “bias” – then we will just have ten new ones. Would it be ok if the new JEE was biased against lower-caste women ? What about men from a poor rural background from TN state board ? The complexities are mind boggling.

This is why we must be very careful not to pass on the task of correcting societies’ real or perceived  ills on to an hapless entrance exam. We have reservations for that.

Advertisements

34 Responses

Subscribe to comments with RSS.

  1. Abi said, on May 3, 2007 at 8:48 am

    Let’s say A (Board Exams) is a prerequisite for B (IITs). [A -> B].

    Now, you introduce an extra layer C (the entrance exam, JEE) in between, to make the route [A -> C -> B].

    Your may say both A and C are gender-neutral. And I would tend to agree.

    But, that’s not the situation today. The intermediate stpe C (JEE) requires an additional input D (Coaching classes)! “Requires” is the keyword here.

    [Aside: The standard of the TN Entrance exam (now defunct, of course) never ‘required’ coaching classes. ]

    If coaching classes (D) introduce a bias (we know the bias against the poor, but my case is that there may be a bias against women), then JEE too becomes biased. Indirectly, of course.

    The stuff about “correcting societies’ real or perceived ills on to an hapless entrance exam” is unnecessary boilerplate. It is enough if (a) the entrance exam corrects itself, or, (b) admission criteria take into account the known deficiencies in the entrance exams.

    Which is better, RC? Tweaking admission criteria or an externally imposed corrective mechanism such as reservation?

  2. realitycheck said, on May 3, 2007 at 9:51 am

    Abi,

    My disagreement is not with your idea for a “New and Improved JEE”. Nothing is sacrosanct about the current system.I welcome the debate you have initiated on your blog.

    Unfortunately, most people including myself, latched on to the part where you strove to establish a gender bias in the current system. This resulted in your main theme, which I think was a call for a new and improved JEE, getting lost in the noise.

    It is true that the JEE today does absolutely nothing about the inherent gender disadvantages or alleged lack of motivation among women to go over the top in coaching. It is also important to note that it does not introduce a new bias of its own.

    The easier you make the exams, the better the representation figures will be ( this is my hypothesis ). The trade off is the bunching problem and the lack of a determinism.

    The now defunct TN entrance also had coaching classes. Quite a few of them. This was one of the reasons cited by the government in court against continuation of the exam. Many come from smaller towns to Chennai to attend the two month or so coaching classes. The scale and toughness are not the same as the JEE.

    What do you think about the decision to do away with the entrance exam in TN ?

    Slightly off topic – the crucial role played by reevaluation.

    Last year 14000 students applied for reevaluation because even a single mark can pull them down 200 ranks. It was surprising that there were many who got upto 30 extra marks. Hundreds more obtained a lesser change.

    In 2005, 179 students scored 100% marks in Phy and Chem and Bio (all three). How can they be accomodated in the top college, eg MMC which has only 85 seats ? Is there any other state with this output ?

    Source Deccan Chronicle print edition (cant find it online)

    >> Which is better, RC? Tweaking admission criteria or an externally imposed corrective mechanism such as reservation? >>

    Good question. Let me think about it.

  3. Barbarindian said, on May 3, 2007 at 1:05 pm

    I think coaching centers are being knocked a little too much. Coaching centers can not beat a donkey into a horse. I know many students who made it to the IITs without any coaching.

    There is no stat on how many students went to a real serious coaching center or how many were really expensive and beyond the means of the average student. As a matter of fact, it can not be easily proven that the coaching centers really introduce an unfair distortion.

    The JEE has already undergone changes, at least twice. First they eliminated the English paper (apparently it was hurting rural students). I think recently they made two changes, format and consolidation. I believe th last two were done to correct for the coaching center effect to an extent.

    This is why we need private colleges and lots of them. They will implement their own admission system. The market will evaluate the output and correct distortions. That is the only fair system possible. But the socialists will continue to harp on a point and insist on adding worse distortions than the current system has.

    The concept of fairness can never be agreed upon. No matter how you design the JEE, there will be skewed numbers from different sections. In India the possibilities are enormous. Religion, caste, community, socialist/capitalist, gender, rich/poor, urban/rural/semi-urban, veg/non-veg. Let’s say non-veg middle-class Bania Gujaratis from rural areas are James Bonds of chemistry. They will insist Chemistry should have a better weightage.

    Women are doing relatively well in Business in India. Vinita Bali, Kiran Majumdar, the Women VPs of ICICI and other banks. Let’s not get too excited about a socialist professor’s crazy ideas.

  4. Barbarindian said, on May 3, 2007 at 1:09 pm

    If coaching centers are the issue, then the Government should set up coaching centers for OBCs, modelled after the Ramanujan school in Bihar and ask Profs like Abi to volunteer.

    http://www.telegraphindia.com/1060601/asp/jamshedpur/story_6296472.asp

    These kids study hard, lead an austere life and at the end are rewarded. Year after year. Boring, right? Now, give OBCs IIT degress, directly. How exciting! Yummy!

    Once again, no one cares about opportunities in India. It is about equalization of outcome.

  5. rc said, on May 3, 2007 at 1:38 pm

    >> This is why we need private colleges and lots of them. They will implement their own admission system. The market will evaluate the output and correct distortions. >>

    I have no issues with private colleges having any admissions norms.

    >> There is no stat on how many students went to a real serious coaching center or how many were really expensive and beyond the means of the average student. >>

    We desparately need this info to take this forward.

    >> As a matter of fact, it can not be easily proven that the coaching centers really introduce an unfair distortion >>

    What about the recent revelation that 85% of IIT students from the south are from AP ? Could it be due to the Ramiah style coaching factories in Hyd ?

  6. Barbarindian said, on May 3, 2007 at 2:02 pm

    What about the recent revelation that 85% of IIT students from the south are from AP ? Could it be due to the Ramiah style coaching factories in Hyd ?

    RC,

    Is there any info about the actual number of ethnic Keralites in IITs? What I am getting at is that perhaps the best of Kerala have moved out of the state and hence don’t show up in South stats. This is true for TN. Tam Brahms are one the largest representative communities in IIT but they don’t show up in your stats.

    I think the lure of Gulf etc. is much more pronounced in Kerala. I am just making wild guesses here.

  7. Reason said, on May 3, 2007 at 2:57 pm

    With OBC quota in IITs pretty much a done deal, it is obviously time to turn to JEE. And just like how OBC quota is sneaked behind sob stories of ‘oppression of dalits’, the project on JEE too would have to be sneaked in behind something else.

    On the question of Andhra, some good info comes up if you google. Appears like Andhra’s education board is called board of intermediate education.

    They seem to take their pass percentages pretty seriously – even in the first year of intermediate (Plus one in TN), pass percentages are in the 40-50% range. In TN stateboard, first year (Plus one) pass percentage must be like 100%. Second year % too does not cross 60% in andhra – mostly less than 50%.
    http://bieap.gov.in/resultanalysis.html

    Some more –
    http://www.hindu.com/2006/05/29/stories/2006052902730500.htm
    “For those intending to study Maths and Biology as base subjects, corporate colleges have package deals that come with IIT, EAMCET and AIEEE coaching. The Government colleges too have decided to follow suit with special classes for competitive examinations.”

    http://www.hindu.com/2006/01/28/stories/2006012809130500.htm
    “The Board of Intermediate Education (BIE) is proposing to increase the number of teaching hours in all Government and aided junior colleges from next academic year to keep pace with corporate colleges in quality education and good results.”

    Studies conducted by BIE officials have revealed that the Government and aided colleges are giving 1,050 to 1,100 hours of teaching in the two-year intermediate course while corporate colleges engage students for 2,500 to 2,800 hours. The gap is to be filled by increasing the number of instruction hours, besides introducing study hours in each and every college.

    Ofcourse, socialist experiments are more fun stuff to do than all this hard work. When socialists rule, why even bother with the hard work?

  8. srinias said, on May 4, 2007 at 4:44 am

    In AP,the dominant castes like Kammas and Reddys are very hardworking.They never asked for reservation like their sudra brethern in TN and Karnataka.They benefitted from green revolution and they invested their returns in industries,quality corporate colleges and hospitals.The literacy rate among them is very high.They surpassed the local Brahmins in education. The so called OBCs of TN and karnataka should learnn a lesson from them and prosper rather then suffering from mediocrity.

  9. Bruno said, on May 4, 2007 at 4:56 am

    //MMC which has only 85 seats//

    You are wrong…..

    Deccan Chronicle and Rediff are known to give 100 % false Information. Don’t depend on those two

    And my thoughts about this are given at
    http://bruno.penandscale.com/2007/05/answer-to-professor-abi.html

  10. Bruno said, on May 4, 2007 at 4:59 am

    A related discussion about “Bias” is also given here
    http://bruno.penandscale.com/2006/02/about-entrance-and-exit-exams.html

  11. Bruno said, on May 4, 2007 at 5:02 am

    my thoughts about this are given at
    http://bruno.penandscale.com/2007/05/answer-to-professor-abi.html

  12. realitycheck said, on May 4, 2007 at 5:14 am

    Bruno,

    Interesting post. I too would be keen to know if your prediction (1100 girls out of 1500) comes true.

    Any idea how the TN colleges are planning to normalize the CBSE board exam results ? Will it be (your mark) / (toppers’ mark) ?

  13. realitycheck said, on May 4, 2007 at 5:22 am

    srinias,

    Good points about the AP communities.

    It is critical to remember that just because some communities in TN are labelled OBCs, it does not mean they dont have abilities or resources to compete. If you take a community as a unit, there are many who will outscore the brahmins (whatever is left of them) in any exam.

    Also the communities you mention are Vaishya.

  14. srinias said, on May 4, 2007 at 5:47 am

    No,the communities kammas and reddys depend mainly on agriculture and they are not vaishayas.In AP vaishya caste is termed as Komati.Except for a few companies like GMR,the vaishyas are not as dominant as kammas and reddys who belong to the fourth varnamcalled sudras.

    But their political administration is not upto the required standards as the Governments in Andhra suffer from corruption and nepotism.

  15. Reason said, on May 4, 2007 at 6:26 am

    Srinias,
    what are these ‘corporate’ colleges in AP that I keep hearing about? Do they refer to ‘private’ colleges like in other states, or is there more?
    ‘Private’ colleges are typically owned by minorities (christians), politicians, or some other assorted trusts. ‘Corporate’ college sounds different, and it is only recently that I see that name – after I started searching about AP following RC’s comments.

  16. srinias said, on May 4, 2007 at 7:18 am

    Corporate colleges are pure private colleges owned by individuals (not specific to any social group).Ther main motto is to earn money but by imparting good education.They spend sufficient money to provide good labs,reading rooms,hostel gacilities and employ highly paid teaching staff including retired professors.These colleges offer packeges mixing intermediate with IIT entrance, all india Engg exam,or AP’s EAMCET for engineering/medicine etc.So they train the students on IIT/other competitive exams along with intermediate course.

    But is quite evident that these colleges are able to get good number of IIT ranks for students.To join Ramaihs college specialised in IIT coaching(along with intermediate) students have to write an entrance exam.There are coaching centeres for this purpose too.Competition is quite high in Hyderabad.

    But i dont know whether these college are producing students with alround personality?

  17. Revathi said, on May 4, 2007 at 7:29 am

    I cant speak for non hindu women but historically, there have been a great deal of atrocities against hindu women.very few were allowed secondary school education, Widows had to shave their heads ( incredible violation of privacy), wear only white and were barred from remarriage, and just like the SCs couldnt appear in public or at relegious functions! This kind of practice continues even today. Hindu widows cannot give away their children in marriage, cannot take part in social events. I dont see why they shouldnt get reservation.

  18. Reason said, on May 4, 2007 at 8:08 am

    Srinias,
    >> To join Ramaihs college specialised in IIT coaching(along with intermediate) students have to write an entrance exam

    Are you saying that students passing out of SSLC have to write a entrance test to join this college for intermediate first year admission?

    If so, socialists would have to start with that entrance test to address biases before they even get to JEE 🙂

    Does this college (and other popular ‘corporate’ colleges) follow CBSE syllabus or AP intermediate board syllabus?

    You seem to be aware of AP (probably from AP yourself?) I would like to know about the condition of Government-run intermediate colleges too in AP.

    Also, looks like the pass percentage in both intermediate first year and second year is around 50%. Are the standards of these exams tough to pass?

    Is the syllabus followed by the AP intermediate board similar to or better than CBSE? If so, how do the teaching staff in Government intermediate colleges cope with that?

  19. srinias said, on May 4, 2007 at 12:10 pm

    These colleges follow AP intermediate board syllabus.But inaddition to that they teach whatever is required for the specific exam like IIT entrance.

    The standard of Mathematics is high comapred to that of CBSE but it is not so with Physics.

    The intermediate exams are not that tough to pass.But somehow we see that most of the passed out students get first class or distinction.

    The Govt colleges in districts are still doing fine.They are able to cope with the sylabus.

  20. anonymous coward said, on May 4, 2007 at 12:28 pm

    the answer against coaching classes would not be reducing the standard of the paper but to provide free/subsidized coaching.

    whenever something is achieved without hard work, it is difficult to realise its true value.

  21. Bruno said, on May 5, 2007 at 12:08 pm

    //Interesting post. I too would be keen to know if your prediction (1100 girls out of 1500) comes true.//

    Me too waiting 🙂 🙂 🙂

    Whatever they do, it is going to be disaster for CBSE Board students ….

    Because there will be at least 1000 guys in each subject (Physics, Chemistry, Biology) taking centum…. (I am expecting more centum in Physics this year….)

    Summing all these you will have at least 1000 girls / guys with a total of 595/600 (and that is more than 99%)

    So the Cut Off is going to be like this

    OC / BC – 99%
    MBC – 98.5 %
    SC – 97.5 %
    ST – 92 %

    SO, the chance for CBSE guys are virtually ruled out

    May be some 10 to 15 guys can get that

    Let us wait and watch

    //Any idea how the TN colleges are planning to normalize the CBSE board exam results ? Will it be (your mark) / (toppers’ mark) ?//

    There was a very detailed normalisation procedure laid out. I did not read through that…

    Any way, if it is MY Mark / Topper’s Mark – then CBSE Guys are going to suffer.

  22. realitycheck said, on May 5, 2007 at 3:38 pm

    //OC / BC – 99%
    MBC – 98.5 %
    SC – 97.5 %
    ST – 92 %//

    Not good at all. Revaluation will become even more critical.

    I can see ties in almost all top medical school programs, selection of disciplines in Anna. Here is the proposed tie breaking mechanism. (Source The Hindu http://www.hindu.com/2007/05/05/stories/2007050518460100.htm )

    If more than one student had obtained the same marks, the following aspects will be taken into account to determine their seniority: percentage of marks in Mathematics; percentage of marks in Physics; percentage of marks in the elective subject; date of birth and random allotment of numbers.

  23. Barbarindian said, on May 5, 2007 at 4:47 pm

    Guess we are converging towards a system of lottery. This is actually great in the spirit of democracy, I bet a referendum to replace the IIT-JEE will pass with record majority.

  24. Bruno said, on May 6, 2007 at 11:55 am

    That is if TWO Candidates BORN on the Same Date scores the same mark, the ranking is done based on Toss//

    I don’t think that it has happened so far (when we had entrance exams), but “THIS” time, we are in for some head aches..

    There are some 2 lakhs students writing this exam
    Assuming that 95 % are born between 1st June 1989 and 31st May 1991, we have 1.9 lakhs students born on 730 days

    That is approximately 260 guys per day…. If any two of them takes the same Mark in Physics, Chem, Biology (or P, C, Maths), we are in for some trouble….

  25. Bruno said, on May 6, 2007 at 12:04 pm

    Look, there is an amendment to my original sentence

    If TWO Candidates from the same COMMUNITY and BORN on the Same Date scores the same marks in all of the each subjects (Physics, Chemistry, Biology or Physics, Chemistry, Maths), the ranking is done based on Toss

  26. xyz said, on May 6, 2007 at 5:25 pm

    bruno,
    you must be celebrating that cbse students are going to suffer.

    i wrote jee 20 years back and failed.That was the end of it.
    But thugs like abi and bruno have to bring caste into it.
    RC,
    every one knows the true nature of the likes of bruno,abi,karu,baalu,ramadoss.It is known as envy,hatred.
    These debased Shu….

  27. Bruno said, on May 7, 2007 at 5:16 am

    //you must be celebrating that cbse students are going to suffer.

    That is your DELUSION…..

    You have a lot of delusions… This is just one of them … Try to see everything with a open mind…

    I cannot imagine as to why i should celebrate when CBSE students suffer. What is the logic…. Why are your imaginations running wild… why can’t you come to reality instead of harbouring imaginary fears

  28. Bruno said, on May 7, 2007 at 5:25 am

    //xyz said,//

    Any one reading the comments will know as to whose messages have more hatred and whose message have more reason

  29. Barbarindian said, on May 7, 2007 at 5:35 am

    Well, there are many people who are delusional enough to believe that Brahmins and Jews are collaborating with multinationals to oppress Muslims and OBCs in India. An ex-IITian (incidentally Brahmin) has a graphic on his blog that says all NRIs based in the USA are parts of a cult group of Hindutvas. This in turn is being controlled by RSS/VHP/BJP. Massive scale recruitment of Indian students is going on in US campuses. The NRIs are funding “progroms” against Muslims in India.

    The only difference is, we put forward our delusions as a personal vent on the blogs, which are our last refuge. Your delusions are published on front pages of India’s leading dailies and discussed in the parliament.

    At some level or the other we have all become delusional or at best disillusioned.

  30. Barbarindian said, on May 7, 2007 at 5:36 am

    Look, how delusional we are:
    https://realitycheck.wordpress.com/2006/12/09/another-statue-story/

    I must be seeing things. Can you prescribe me some medication?

  31. Revathi said, on May 8, 2007 at 7:44 am

    No one has commented on the issue of reservation for hindu women. Their plight was comparable to the SCs in the past and even now. Why are they being denied reservation?

  32. Reason said, on May 8, 2007 at 10:29 am

    // No one has commented on the issue of reservation for hindu women. Their plight was comparable to the SCs in the past and even now //

    Such blanket statements may not be the best way to address an issue. For example, OBC reservation supporters start with 2000, 3000, 5000 years etc of oppression.

    It may not be entirely true that all hindu women were deprived of education in all of our past – how would you define ‘past’? There were Hindu women poets (Avvayyar, Karaikkal ammaiyar) and Rishis. You could argue their numbers were smaller compared to men poets, but it does disprove a claim that Hindu women were totally denied education (not every educated man or woman achieved enough literary skills to become a famous poet, so to produce two women poets the society must have educated a lot more women).

    All stories of ‘historical oppression’ need to take into account that the Indian/Hindu society was under foreign assault and foreign oppression for several centuries. A society living under foreign rule might be forced to keep their women locked up for other fears.

    Even if you start with addressing ‘Hindu women’, assuming there is a fair case there, pretty soon politicians would bring in caste factor there too – providing reservation for women in parliament is caught in that loop.

    What is the purpose of reservation? correcting real or imagined stories of historical oppressions? ensuring representation? Rationing all available engineering, medical and management seats strictly according to proportion of caste/religion/male/female population shares?

  33. Revathi said, on May 9, 2007 at 8:02 am

    Dear Reason,

    Hindu women,even brahmins were denied secondary school education except in the case of exceptions. Both my grand mothers (my grand father was one of the the first few indian postgraduates in physics) didnt get beyond primary school. So you can imagine the plight of lesser educated families. We know the story of avvaiyyar what she had to go through in order to “escape” the clutches of the society. In fact, she is a good example of what women had to go through. Maithreyi and Gargi if and when they existed, were taught privately by their fathers or husbands at home. So this proves that hindu women were denied the right to education.
    I am just pointing out that their status in terms of education was on par with the OBCs and SCs. So according to the existing criteria for reservation, they should get reservation since we increase their numbers in IITs and IIMs so that they could provide adequate role models for other women.

  34. […] See related post : The exam difficulty control knob (What do dead easy exams mean ? ) […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: