Go to navigation
It is currently Sat Feb 22, 2020 11:09 am

All times are UTC




Post new topic Reply to topic  [ 4 posts ] 
Author Message
PostPosted: Wed Aug 30, 2017 11:14 am 
Offline

Joined: Sat May 06, 2017 8:54 pm
Posts: 1
My son came within 1% of the Wirral pass mark so we decided to look at the appeals process and gather some information.
We did this through subject access requests to Wirral Council and CEM/Durham.

Here is some of the information received - Apologies as not all in order but may be usefull.

"Thank you for your request for additional information from Durham University. Colleagues have provided the information below.

I note the average of the Age STD results is 234.18 (230.81+244.3+227.43) not the 233.34 as noted – please confirm the figure is an average or explain calculation.

The total weighted score is correct. The score is calculated as;
50% of their age standardised Verbal score, plus
25% of their age standardised Maths score, plus
25% of their age standardised NonVerbal score.

Also please provide multipliers / dividers used to modify results for each age group.
(I presume these are monthly based).
For example the modifier to change the Raw score for Age Std.
So I am looking for 36 Variables (12 Months / Verbal – Maths – Non-Verbal).


During the age standardisation process the relationship between age and raw score is calculated separately for each of the three test sections (Verbal, Maths and NonVerbal). Every pupil’s score for each test section is then adjusted by a small amount according to their age. This creates the age-adjusted raw scores, such that the mean age-adjusted raw score is the same across all ages. Age-adjusted raw scores are calculated 'on-the-fly' and not stored. CEM will not release the method used to calculate age-standardised scores as this could be used by a third party to deduce the method by which standardised scores are produced which could damage CEM’s commercial interests. As a result, we are refusing this element of your request under section 43(2) of the Freedom of Information Act 2000. Section 43 (2) of the FoIA exempts information from release where its disclosure under the Act would, or would be likely to, prejudice substantially the commercial interests of any person or public authority. As section 43 (2) is a qualified exemption, the University has performed a public interest test and has concluded that release of test administration information would prejudice the commercial interests of the Centre for Evaluation and Monitoring (CEM).

The University recognises that release of the requested information would allow individuals to better understand the 11+ tests. However, release of information relating to test format and administration would have a prejudicial effect upon the commercial value of the assessment for Durham University as it could weaken the effectiveness of our testing model, and jeopardise the security of test material. The Centre works hard to design tests that are as resistant to tutoring as possible, and to maintain the security of test material. Release of the information could potentially allow tutoring companies to tailor their teaching to the test: not only would this unfairly advantage tutored children it would also undermine the quality of our test and prejudice the commercial value of the assessment for the Centre.

FOI Reference 2017/132
Request
Please provide the number of children taking the 11+ by birth date month and also the number of those children reaching the required level for Grammar School entrance within the Wirral Consortium of Grammar Schools
Response
The table below shows the counts per month, for the standardisation group.
NB: Included is ‘percentage qualified’. It would be misleading to only look at the numbers qualifying as you would expect there to be more candidates to qualify from birth months where there are more candidates.
Birth Month
Count
Qualified
Percentage Qualified
September
163 69 42%
October
180 70 39%
November
163 69 42%
December
137 72 53%
January
151 69 46%
February
129 68 53%
March
141 63 45%
April
135 64 47%
May
120 56 47%
June
156 60 38%
July
138 61 44%
August
114 51 45%

This would suggest a Wirral pass rate of approximately 44.7%.
Can you confirm this is correct as seems very high given my knowledge of results in several top schools.


The relationship between age and raw score is calculated separately for each of the three test sections (Verbal, Maths and NonVerbal). Every pupil’s score for each test section is then adjusted by a small amount according to their age. This creates the age-adjusted raw scores, such that the mean age-adjusted raw score is the same across all ages. Hence, the youngest pupil in the cohort is not disadvantaged on the basis of their age. Age-adjusted raw scores are calculated 'on-the-fly' and not stored.
These age-adjusted raw scores (aa) are standardised for each section using a mean of approximately 234 (MEANstd) and standard deviation of approximately 15 (SDstd).

Internal Review of Freedom of Information Request
1. Background - Original Freedom of Information Act (FoIA) Request
On 15 March 2017 the University received the following request for information from in relation to his 10 year old son.
I have been directed to you by Wirral Mainstream Admissions. Please provide Examination scores before and after age standardization
Receipt of the request was acknowledged on 16 March 2017. Given the nature of the request the applicant was advised that it was a Subject Access Request (SAR) under the Data Protection Act and the necessary SAR form, fee and proof of identity was requested. There followed further correspondence with Mr to confirm for which School the examination results had been provided.
The University gathered information about the subject matter of the request and reviewed the advice provided by its Centre for Evaluation and Monitoring (CEM) in relation to the requested data.
The University sent its response to the request on 5 April 2017 and provided the applicant with results (raw scores) of the various elements of the 11+ exam undertaken by his son. For reasons of sensitivity the information was posted rather than emailed.
On 8 April 2017 the applicant submitted the following follow up request.
I note the average of the Age STD results is 234.18 (230.81+244.3+227.43) not the 233.34 as noted – please confirm the figure is an average or explain calculation.
Also please provide multipliers / dividers used to modify results for each age group.
(I presume these are monthly based).
For example the modifier to change the Raw score for Age Std.
So I am looking for 36 Variables (12 Months / Verbal – Maths – Non-Verbal).

On 27 April 2017 a PDF document was issued to the applicant providing the following data. This followed a referred request the applicant had made to CYPD- Secondary places.
The number of children taking the 11+ by birth date month and also the number of those children reaching the required level for Grammar School entrance.
On 27 April 2017, Mr submitted a further request.
This would suggest a Wirral pass rate of approximately 44.7%. Can you confirm this is correct as seems very high given my knowledge of results in several top schools.
On 28 April 2017, Mr submitted a follow-up request to the University
I have had the figures reviewed externally and it has been noted in certain birth months you have 39% more chance of passing the 11+. It will be interesting to see the modifiers for each subject per birth month.
On 10 May 2017 the following response was issued to the applicant.

The total weighted score is correct. The score is calculated as;
50% of their age standardised Verbal score, plus
25% of their age standardised Maths score, plus
25% of their age standardised NonVerbal score.

During the age standardisation process the relationship between age and raw score is calculated separately for each of the three test sections (Verbal, Maths and NonVerbal). Every pupil’s score for each test section is then adjusted by a small amount according to their age. This creates the age-adjusted raw scores, such that the mean age-adjusted raw score is the same across all ages. Age-adjusted raw scores are calculated 'on-the-fly' and not stored. CEM will not release the method used to calculate age-standardised scores as this could be used by a third party to deduce the method by which standardised scores are produced which could damage CEM’s commercial interests.
As a result, we are refusing this element of your request under section 43(2) of the Freedom of Information Act 2000. Section 43 (2) of the FoIA exempts information from release where its disclosure under the Act would, or would be likely to, prejudice substantially the commercial interests of any person or public authority. As section 43 (2) is a qualified exemption, the University has performed a public interest test and has concluded that release of test administration information would prejudice the commercial interests of the Centre for Evaluation and Monitoring (CEM).
The University recognises that release of the requested information would allow individuals to better understand the 11+ tests. However, release of information relating to test format and administration would have a prejudicial effect upon the commercial value of the assessment for Durham University as it could weaken the effectiveness of our testing model, and jeopardise the security of test material. The Centre works hard to design tests that are as resistant to tutoring as possible, and to maintain the security of test material. Release of the information could potentially allow tutoring companies to tailor their teaching to the test: not only would this unfairly advantage tutored children it would also undermine the quality of our test and prejudice the commercial value of the assessment for the Centre.
Request for an Internal Review
On 12 May 2017 Mr emailed the University’s and requested for an internal review on the following grounds.
Unfortunately the Age Adjusted Raw Score could in no way impact CEM’s Commercial Interests as you state.
The following comment is totally unacceptable and you are leaving yourself open to significant scrutiny “Age-adjusted raw scores are calculated 'on-the-fly' and not stored.”

2. Internal Review
There are 5 matters that I have investigated as part of this review:
i) The treatment of the request as a Data Protection Request and the treatment of the follow up request which should have been treated as a Freedom of Information request.
ii) The steps taken to respond to all parts of the FOI request and to provide assistance to the applicant.
iii) Whether the information provided sufficiently addressed all parts of the request.
iv) The use of FOI exemptions applied.
v) The time taken by the University to respond to the information request.

For the purposes of this review, I have spoken to the University’s Governance and Executive Support Department and I have also read through the relevant email exchanges and the Freedom of Information Act section on exemptions.
Each of these is addressed below:
i) The nature of the information requested in the follow up request dated 8 April fell within the FOI Act and should have been dealt with as such and within the relevant timeline of 20 working days.
ii) I am satisfied that the University took the appropriate steps to respond to all parts of the FOI request and to provide assistance to the applicant.
iii) I am satisfied that the information provided sufficiently addressed all parts of the request.
iv) The University applied the use of the S 43 exemption (release would, or would be likely to, substantially prejudice its commercial interests) to refuse release of the method used to calculate age-standardised scores on the basis that this could be used by a third party to deduce the method by which standardised scores are produced, which could damage CEM’s commercial interests. I am satisfied that this exemption was appropriately applied.
v) I am further satisfied that as required in the use of this qualified exemption, that the public interest test was appropriately applied, and that the conclusion reached on non-disclosure was justified.
vi) I have noted that the University also provided further explanation relating to the commercial value of the testing model and the need to ensure security of test material.
vii) The expression ‘on the fly’ could have been otherwise worded, but its meaning is clear from the context of its use.
viii) The University should have responded to the request by the 9 May 2017 to meet the 20 working days requirement of the FOI Act. The information was provided on 10 May 2017.
In terms of the decision to withhold the requested information, I have reviewed the arguments you put forward in your request for an internal review. I have also considered the public interest in release of information.
In light of the arguments above, I uphold the decision made on this occasion.
I have noted that the University made an error in handling this request. While the original request was correctly identified as a subject access request for personal data, the follow queries were of a generic nature and should have been treated as a Freedom of Information request. In effect the request was dealt with as such and the response cited a FOI exemption. The University wishes to apologise for this error, which I can confirm, has not impacted on the response provided. It also apologises for failing to meet the 20 day response period.

Dear Mr

The 11 plus tests have always been age-standardised. Your other email to CEM suggests that you believe the results are standardised by month of birth, this is incorrect, they are standardised by exact day of birth.

We have sent your email onto CEM for their comments, who have provided the following in response to your email, with reference to the table given to you in your FOI request which asked for data by month of birth.

“The 39% is basically a flawed logic, but calculated by the % increase of February over June: (53% - 38%)/38% = 39%. This is a highly inappropriate interpretation of the data. He states February children are some of the oldest, whereas actually, they are almost average because March-born children are average age (middle of the academic year). It may well be that some qualifying candidates were born on the last day of the preceding month, or the first day of the next month – just one day different would change the % qualify by birth month, so the percentages should only be taken as indicative.

Taking all children in the standardisation cohort, the effect of age has been removed, to make it so there is no correlation between age and score. The table provided was for those qualifying, so statistically there will be fluctuation in the numbers qualifying, just as there is fluctuation in the numbers born in each month.”

University of Durham CEM fully believe the age standardisation was carried out correctly. The Authority has every confidence in the standardisation process carried out by the University of Durham.

For information, Julia Hassall is the Director of Children’s Services and as such, responses to emails on subjects relating to the admission of children to school are provided accordingly by the Mainstream Admissions team as this is their area of responsibility within the Department.

Regards

Mainstream Admissions team

Hello,

While I appreciate the data can be interpreted to fit the requirement, the terminology “Flawed Logic” is actually wrong.
C.E.M supplied the data and I merely interpreted it given those birth month figures – basic statistics.
Percentages they have actually confirmed themselves in the below email
I believe this could be interpreted as “Clusters”, however I believe that is not correct and my interpretation is correct and valid.
The data was supplied by C.EM. and with the further data which is being supplied by C.EM shortly I understand this will be confirmed.
In fact the comment “so statistically there will be fluctuation in the numbers qualifying” is actually irrelevant even if correct as it does not impact the percentages which provide the true birth month statistics.

Regardless, my basic assertion is the new format VR has impacted to a greater extent on younger children and that the standardization has not been modified to account for this, given this is the first occasion of the use of this format this is understandable, however I am not prepared to accept this has impacted on my son’s ability to attend Grammar School.

As such I reserve the right to seek legal redress as appropriate in future and will certainly publish facts and figures obtained through FOI as this is entirely appropriate where educational injustice has taken place.

I do appreciate your time on this matter and hope we can resolve the issue shortly.

Best Regards,

The admission number for the grammar schools for Year 7 entry 2017 was as follows:

Calday Grange Grammar School: 240
West Kirby Grammar School: 180
Wirral Grammar School for Boys: 155
Wirral Grammar School for Girls: 173
St Anselm's Catholic College: 135
Upton Hall School FCJ: 140

The number allocated as at 1st March 2017 was as follows:

Calday Grange Grammar School: 235
West Kirby Grammar School: 180
Wirral Grammar School for Boys: 155
Wirral Grammar School for Girls: 168
St Anselms Catholic College: 126
Upton Hall School FCJ: 133

There is an admin charge of £70 for tests to be remarked. If you wish to go ahead you can do so via our website http://www.wirral.gov.uk by going to the Make a Payment section. If you click on the Pay online by Card button there should be an option for 11-Plus which is where you can provide your child's details and make the payment.


Last edited by Hiutsuri on Mon Sep 04, 2017 5:42 am, edited 1 time in total.

Top
 Profile  
 
PostPosted: Sat Sep 02, 2017 3:29 pm 
Offline

Joined: Mon Dec 12, 2005 5:26 pm
Posts: 8259
Quote:
..... we decided to look at the appeals process and gather some information.
We did this through subject access requests to Wirral Council and CEM/Durham.

Welcome to Appeals, and many thanks for sharing the correspondence regarding your SARs.

On the wider issue of general advice about challenging 'the system' at an appeal, forum readers might be interested in an additional entry in the Q&As:
https://www.elevenplusexams.co.uk/appeals/general#a57

_________________
Etienne


Top
 Profile  
 
PostPosted: Sun Sep 03, 2017 10:55 am 
Offline

Joined: Tue Feb 24, 2009 9:50 am
Posts: 137
Location: surrey
And here is another example of challenging the system that had little or no effect .

https://www.gov.uk/government/uploads/s ... t_2017.pdf


Top
 Profile  
 
PostPosted: Tue Nov 26, 2019 8:20 pm 
Offline

Joined: Tue Sep 24, 2019 12:09 pm
Posts: 4
Hiutsuri - can you remember what your sons raw scores were?


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 4 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Privacy Policy | Refund Policy | Disclaimer | Copyright © 2004 – 2020