What's new

Quartiles and weights - Exam results

hellohi

Active Member
Thread starter #1
I asked GARP about quartiles and weights in the exam result,,,, and my question was as follow:

according to my notes from the last exams results, when I were look at the exam quartiles results ... I saw 4 quartiles (FRM part 1)....if we look at two candidates quartile results for example 1233 and 1332, the frist candidate faild and the second who got the second result passed, if they got the same frequancy quartiles why one passed and the other faild, me and many agree that the weight of the book (30% or 20%) and the better quartile result related to this book that has more weight effect the final result (pass or fail).
for clarification if we took the above example we see that the first candidatie took 1233 thats mean 33 in the third and fourth more weight books (30% not the 20%)so he is faild despit he got the same four quartiles as the one who passed but in different orderlines?????

and they answerd me three times by three emails as follow:

the first email

A 20% subject area is worth 20 Pts of the total exam
a 30% subject area is worth 30 Pts of the total Exam.
A 3rd quartile (vertical is worth between .26 and .50.
Therefore a 20 % subject area is worth between 5.2 pts and 10 points
Therefore a 30% subject are is worth between7.8 and 15 points.
Sincerely,
Frank Weber

the second email

In addition to your calculation with regard to horizontal value of the subject areas, you must remember the vertical differential in each quartile has a range of 25 points.
Your comparison is not valid.
Sincerely,
Frank Weber
Vice President - MemberServices


the third email
Quartile Explanation from Chris Donohue


Thank you for your interest in and commitment to the FRM program. As you know, your exam results are released only with a pass or fail grade. Quartile data is provided only as a broad guideline to show you what your performance was relative to your peers taking the exam on the same day. This is valuable information if you did not pass the exam in that it will direct you to the areas that need improvement.

Please note that, by definition, each quartile number covers a 25% vertical range and your performance could be anywhere within that range. Additionally, subject area values are not evenly distributed. Performing well in smaller subject areas. Subject areas also have a different % value. Taken in context, this means many candidates may have the same or very similar quartiles with some passing and some failing because their scores vary by a sizeable margin. The quartiles are issued to provide you with a more in depth view of your performance so you may know what path you need to take in your studies; they cannot be used to make comparisons to other candidates ‘overall score.

may the memebers of this forum discuss and explain what GARP says related to my question above?

thanks
Nabil
 
Last edited:
#2
I think what they're trying to say is that there's a lot of variability not only with weighted percentage of each section, but also within the range of tthe Quartiles.

For example:

Student 1: Q2, Q2, Q3, Q3
Student 2: Q2, Q2, Q3, Q3

Perhaps Student 1 passes and Student 2 fails even though they have equivalent Quartile results because Student 1 was Top 51% for his Q3 results (top of Q3 Quartile) while Student 2 was Top 74% (bottom of Q3 Quartile but still in Q3). So if they are borderline to the cutoff, then Student 1 would have the statistical advantage over Student 2 in passing.
 

hellohi

Active Member
Thread starter #3
I think what they're trying to say is that there's a lot of variability not only with weighted percentage of each section, but also within the range of tthe Quartiles.

For example:

Student 1: Q2, Q2, Q3, Q3
Student 2: Q2, Q2, Q3, Q3

Perhaps Student 1 passes and Student 2 fails even though they have equivalent Quartile results because Student 1 was Top 51% for his Q3 results (top of Q3 Quartile) while Student 2 was Top 74% (bottom of Q3 Quartile but still in Q3). So if they are borderline to the cutoff, then Student 1 would have the statistical advantage over Student 2 in passing.
thanks a lot for your great effort,,,,is there any ideas else? or it is the only one?
 

David Harper CFA FRM

David Harper CFA FRM
Staff member
Subscriber
#4
@hellohi

Okay I built this small spreadsheet (because trying to talk about this soon becomes a word soup :eek: imo), please see https://www.dropbox.com/s/ikhbkm0571fdchh/1019-garp-frm-scoring-exam.xlsx?dl=0
... and below is a snapshot. This is just for P1 and you only input (change) the yellow cells, the rest is calculated.

Notice how I input an extreme version of the one you referenced in your email; i.e., Student #1 earns 3/3/2/2 and passes while Student #2 earns 2/2/3/3 but fails. I was deliberately provocative: notice how the seemingly subtle difference can lead to a difference between a final score of 62 and 36 (wow!). Caveat: I'm not sure my quantiles are exactly calibrated, but they can't be too far off. I hope that clarifies!

 
Last edited:
#6
Hi @David Harper CFA FRM .There is something which i cant understand.The Peak level of normal distribition which GARP send to show results is 1200 in my distribition but 1500 in the other members i wont to know what is that .I gave fist level FRM in 19 Novermber 2016 and i failed results 2334 .Please if you have any information about this write me .

Thansk
 

David Harper CFA FRM

David Harper CFA FRM
Staff member
Subscriber
#7
Hi @Elnur1 I admit that I am not familiar with this "normal distribution" reporting (peak at 1200 versus 1500) to which you refer; apologies, but I am only familiar with GARP's quantile scoring (eg, 2334). cc @Nicole Seaman I haven't analyzed the feedback thread, have you seen reference to a "normal distribution" reported by GARP along with the results?
 

David Harper CFA FRM

David Harper CFA FRM
Staff member
Subscriber
#9
Hi @Elnur1 I'm baffled by the y-axis units of the displayed normal distribution; given the density, I believe the max y is 1/[sqrt(2*pi)*σ]; i.e., the pmf density function without the effectively discounting exp(). I don't see how the axis can get that high. I suspect it is merely visual. Please note GARP's methodology is based on quartiles (quantiles): there is no need to assume or impose normality. GARP's method does not impose normality, as far as I understand (it already quite sufficiently "grades on a curve"). The graphs may just be visual signals. And perhaps they varied the y-axis values because according to the topic weights. @Nicole Seaman If @Elnur1 really prefers, do you mind reaching out to confirm my assumption that "the normal graphs displaying along with the quartile reporting are merely visual indicators to convey quartile status. Specifically, the y-axis values are not exactly meaningful, nor do the pictorial normals imply any necessary normality; ie, normality would be coincident, as our understanding is that GARP does not actually impose normality on the score distribution." Thanks!
 

Nicole Seaman

Chief Admin Officer
Staff member
Subscriber
#10
Hi @Elnur1 I'm baffled by the y-axis units of the displayed normal distribution; given the density, I believe the max y is 1/[sqrt(2*pi)*σ]; i.e., the pmf density function without the effectively discounting exp(). I don't see how the axis can get that high. I suspect it is merely visual. Please note GARP's methodology is based on quartiles (quantiles): there is no need to assume or impose normality. GARP's method does not impose normality, as far as I understand (it already quite sufficiently "grades on a curve"). The graphs may just be visual signals. And perhaps they varied the y-axis values because according to the topic weights. @Nicole Seaman If @Elnur1 really prefers, do you mind reaching out to confirm my assumption that "the normal graphs displaying along with the quartile reporting are merely visual indicators to convey quartile status. Specifically, the y-axis values are not exactly meaningful, nor do the pictorial normals imply any necessary normality; ie, normality would be coincident, as our understanding is that GARP does not actually impose normality on the score distribution." Thanks!
@David Harper CFA FRM and @Elnur1

I emailed GARP about this, and received this response today,

"David is right. The graphs are only from a visual perspective, so candidates can see their strengths and weakness. That’s it."

I hope this helps to clear up any confusion :)

Nicole
 
Top