[page 207]
CHAPTER 3
The Method of Estimating and the Reliability of the Estimates
LINKING THE RATES
1. The sample data were first used to calculate Pr and Qr for each category of student. Pr is the proportion of the outflow from the rth stage at the end of the session who entered the next stage at the beginning of the next session. Qr is the proportion of the outflow who qualified. Consequently (Qr - Pr) is the proportion of the outflow who left the course after qualifying at the rth stage and (1 - Qr) the proportion who left the course after failing to qualify at the rth stage. It should be noted that these proportions are all proportions of the outflow, and not of the enrolment for the stage, which consists of the outflow plus the repeaters.
2. The first part of each table was then written down as
The second part was obtained from the first by taking differences and the third part by ascertaining from the sample the proportion of students obtaining each certificate who were not retarded, or retarded by one, two, or three or more years.
3. It will be seen that this process, based on the passage of several cohorts through one session, would give the same results as following one cohort through several sessions, if a steady state existed - that is to say, if each cohort were the same size as its predecessor, and had the same history. For each P and Q is a weighted average, of the form,
where wa, wb ... etc. are the proportions in which the outflow from the stage is divided between students with no retardation, one year's retardation, ... etc., and pa pb ... etc. (or qa, qb etc.) are the proportions continuing (or qualifying). A steady state means a state in which the p q and w do not change, either from session to session or from cohort to cohort. Consequently in a steady state (1) and (2) can represent either the passage of the survivors of a single cohort through the stage in different years, or the passage of the survivors of different cohorts in the same year. Thus if a single cohort were
[page 208]
being followed, and wd pd represented the transit in 1957 of survivors retarded 3 years, then wc pc, wb pb, wa pa, would represent the transits in 1956, 1955 and 1954 of survivors retarded 2, 1, and 0 years. But if the reckoning were by the session and wd pd related to the cohort starting in 1952, then wc pc, wb pb and wa pa would relate to the cohorts starting in 1953, 1954, and 1955.
BIAS?
4. The actual state is not steady: the cohorts in some categories are known to be increasing in size, and there may be changes with time in p and q. Consequently, (1) and (2) cannot in practice stand for both the reckoning by the session and the reckoning by the cohort. If we allot them to the session we need
for the cohorts. The equations we can actually use are (1) and (2), because the sample gives the values of the p, q, w, and not of the p' q' and w'. So far as the difference between the first pair and the second pair of equations resides in the differences between the w and the w' this is a disadvantage: it means that the estimates are biased, though it is shown below that the bias is not serious. But, so far as the difference resides in the differences between the p, q and the p', q' it is a gain, since the p, q are all recent while the p' q' are obsolescent, and in particular the pa', qa' to which the largest weights attach, are obsolete.
5. The rate of increase in the size of cohorts varies from category to category, but is unlikely to exceed 10 per cent yearly for any well established course. This enables an upper limit to the bias to be calculated. Thus for one category we have the following table:
Table F. National Certificate. Engineering. Students with no Exemption. Values of wa, pa, qa, etc.
[page 209]
It is clear that p and q do not vary enough from row to row for small changes in the weights to produce much effect. If P'' and Q'' are calculated by adjusting the weights to offset an annual increase of 10 per cent in the size of cohorts the ratios in the last row of the table are obtained (when enough figures are kept throughout the computation).
Table G.
6. The results of the complete calculation, for an annual expansion of 10 per cent, are
[page 210]
Thus if the annual increase has been 10 per cent the O.N.C. and H.N.C. estimates are too large by amounts ranging from 0.3 per cent to 0.9 per cent. The actual increase, and therefore the bias, is probably somewhat less than this. In any case a comparison with the standard errors (Table 5 and 6) shows that this bias is not a serious contributor to the risk of large error.
7. There is a small systematic error, already considered in paragraphs 7, 8 and 9 of Chapter 1, arising from the fact that a success gained by a student who has resumed the course after leaving it for an interval is not taken into account by the method of estimating. No direct evidence as to the number of such students can be obtained from the survey, but they are known to be very rare among part-time day students, and quite rare among evening students, up to the O.N.C. stage. Consequently, so far as the O.N.C. estimates are concerned, it seems unlikely that this error can do much more than cancel the bias in the opposite sense arising from the annual growth in the size of cohorts.
8. For the H.N.C. estimates upper limits have already been given, in Table A in Chapter 1, first by using the H.N.C./O.N.C. ratio from the Annual Report*, and secondly by assuming that all students who left the course after qualifying at a stage subsequently returned. This is a grossly optimistic assumption. A more realistic procedure is to suppose first that the proportion of students who return after leaving the course before they have reached the O.N.C. stage is negligibly small, and that of those who leave after qualifying at the O.N.C. or a later stage, a half return. The corresponding estimates are given in the third line in Table H below. The second line contains the estimates when the half is reduced to a third. Neither of these assumptions makes any allowance for students who have left after failing to qualify returning after an interval. However, the assumption that a third of the qualifiers and also a third of the non-qualifiers return after an interval produced much the same estimate as the assumption that half the qualifiers (and no non-qualifiers) do so.
Table H. Effect on the H.N.C. Estimates of various Assumptions about the Proportion of Students returning after an Interval
(1) Students with no exemption (2) Students exempt from first stage
*Annual Report by the Minister of Education
[page 211]
All the estimates in this table are a good deal higher than those reached by the method of following a cohort (Cf. Table A, p. 192), but this is to be expected, since the difference represents delayed successes. The estimates in the fourth row, and particularly those for students with no exemption, are too high, for the reason given in paragraph 7 of Chapter 1.
9. The bias arising from the growth of the cohorts, and estimated in paragraph 6 above, has not been laid off in calculating Table H. Taking both kinds of bias together it seems reasonable to hold that the O.N.C. estimates, as given in the main tables, are unbiased, and that the H.N.C. estimates need a small upward adjustment of about 2 per cent.
SAMPLING VARIATION
10. The first step in estimating the sampling variation was to use the division of the sample into five independent subsamples and obtain some of the principal tables in five-fold form. Table 5 is an example. This gives five independent estimates for the content of each cell in the tables, from which the corresponding standard errors can be estimated on four degrees of freedom, either by the usual process or, more simply and with sufficient accuracy, by dividing the range by 5.2.
11. Unless this process is supplemented it suffers from two disadvantages. In the first place some of the estimates on four degrees of freedom will be much too large, and others much too small. This risk is discounted in the table of t, but at the expense of having a wide spread in the tails. In the second place it would hardly be practicable to produce all the tables in quintuplicate, while to increase the number of degrees of freedom by having more subsamples would involve an impossible mass of calculation.
12. A way out is to look for a simple general formula, suggested by the nature of the sampling design and the method of estimating the variables, and to test it against the estimates of error provided by the subsamples, with due regard to the fact that these are on only four degrees of freedom. Any such simple formula has some initial probability of giving a proper representation of the sampling errors: if it agrees well with a large number of estimates from the subsamples its final probability becomes very high.
13. It is clear from the nature of the sampling design and the estimating process that there is a fair initial probability that good estimates of the standard errors will be given by
where n is the number of students in a category, m the number of
[page 212]
colleges, σ2 the variance between colleges, and k a constant to allow for the facts that each student who is not a repeater appears in only one of the ratios that are multiplied to produce the estimates, and that repeaters appear in none of the ratios. If the items in the "Enter" rows of the tables are summed and multiplied by (1 + x) where x is the ratio of repeaters to outflow the result is never very far from 3, which may therefore be adopted tentatively as the value of k. Experience suggests 6 per cent as a suitable value for σ. With these values inserted the formula gives very close agreement with the estimates of standard error on 4 degrees of freedom in Table 5 and several similar tables for both the National Certificate and the City and Guilds. For example, Table 5 provides 34 comparisons altogether. The averages are almost equal, the formula values being larger by 0.06 per cent, and the largest disparity is between 2.1, the last figure but one in the lowest row of Table 5 (ii), and 1.4 given by the formula. This gives F = 2.25 which is below the 5 per cent point, 2.45, for 4 and 113 degrees or freedom. The agreement between the formula and the subsample tables makes the final probability very much higher than the initial probability, so that the formula may reasonably be adopted outside the range for which it has been tested against the subsamples, and in particular to cover all the remaining results in Tables 1-4. For this purpose it has been tabulated for suitable values of p, q, and n in Table 6.
14. It may also be applied to Dr. E. C. Venables' results and to those in the survey of the National Certificate in Chemistry mentioned in Chapter 1, though it must be remembered that in these cases k is 1, because the cohort method was followed. Of Dr. Venables' 214 students 11 transferred, and 34 obtained the O.N.C. in 5 years or less. This is 17 per cent, or say 18 per cent when allowance is made for those who take 6 years or more. With k = 1 and 4 colleges in the sample the standard error is 4.0 per cent which brings the result well within the range of the Council's 24.6 per cent. Combining the results, with weights inversely as their variances, gives a joint estimate of 24.1 per cent, with a standard error of 1.1 per cent.
15. The Chemistry survey gives 26 per cent as the proportion of students with no exemption who obtain the O.N.C. This is on 230 students followed through in eleven colleges. Consequently k = 1, m = 11 and the standard error is 3.4 per cent. The corresponding estimate from the Council's sample is 39 per cent with a standard error of 4.7 per cent, on 329 students in 114 colleges with k = 3. The differences between the two estimates is 2¼ times its standard error, which suggests that there may have been some improvement between 1955, the middle point of the Chemistry survey, and 1957 when the Council's survey took place. If the two estimates are combined, with weights inversely as the variances, they give a joint estimate of 31
[page 213]
per cent, with a standard error of 2.8 per cent, for 1956, whether or not an improvement has occurred. For students with one exemption the survey estimate is 57 per cent, on 11 colleges and 135 students, which gives a standard error of 4.6 per cent. The Council's estimate is (70 ± 5.4) per cent. The difference of 13 per cent is just under twice its standard error of 7.1 per cent, which also suggests that an improvement may have taken place. The combined estimate for 1956 is (63 ± 3.5) per cent.
16. A similar comparison with Lady Williams' results* for City and Guilds courses may be set out as follows:
Table J.
In Lady Williams' column the first bracket gives the number of students, followed by the number of colleges, in her sample. In the Council's column the first bracket gives the number of students, the number of colleges being 114 throughout. For Lady Williams' k is 1, for the Council it is 3. The "Combined Estimate" column is obtained by weighting the results inversely as their variances. It will be seen that the average difference between the Council's and the combined estimates is less than 1 per cent, and that the largest individual difference is 6 per cent for Brickwork, where the Council's result has an unusually large standard error (5.3 per cent) owing to the small number of students taking the 3 year course. Lady Williams' estimates by themselves are considerably lower on the average than the combined estimates, which suggests that there may have been an improvement over the intervening period. The estimates of the National Institute of Industrial Psychology are much higher than the Council's, but they are based on a different population - the population of students from firms with an unusual interest in the progress of their apprentices.
*See page 191
[page 214]
Main Tables
Table 1. All National Certificate Courses.
(i) Percentages entering and qualifying at each stage whether retarded or not.
(ii) Percentages
(a) leaving the course at each stage after qualifying at that stage
(b) entering each stage
(c) leaving the course at each stage after failing to qualify at that stage.
[page 215]
Table 1 (continued)
(iii) Percentages obtaining the Ordinary National Certificate and the Higher National Certificate in the minimum time, and in longer times, and the average time taken by students who are ultimately successful.
[page 216]
Table 2. Six City and Guilds Courses.
(i) Percentages entering and qualifying at each stage, whether retarded or not.
(ii) Percentages
(a) leaving the course at each stage after qualifying at that stage
(b) entering each stage
(c) leaving the course at each stage after failing to qualify at that stage.
[page 217]
Table 2 (continued)
(iii) Percentages obtaining the Intermediate Certificate in the minimum time, and in longer times, and the average time taken by students who are ultimately successful.
[page 218]
Table 3. National Certificate Engineering Courses by various Factors.
Table 3.1
(i) Percentages entering and qualifying at each stage, whether retarded or not, by Age of Leaving School.
(ii) Percentages
(a) leaving the course at each stage after qualifying at that stage
(b) entering each stage
(c) leaving the course at each stage after failing to qualify at that stage.
[page 219]
Table 3.1 (continued)
(iii) Percentages obtaining the Ordinary National Certificate and the Higher National Certificate in the minimum time, and in longer times, and the average time taken by students who are ultimately successful.
[page 220]
National Certificate Engineering Courses
Table 3.2
(i) Percentage entering and qualifying at each stage, whether retarded or not, by Type of School.
(ii) Percentages
(a) leaving the course at each stage after qualifying at that stage
(b) entering each stage
(c) leaving the course at each stage after failing to qualify at that stage.
[page 221]
Table 3.2 (continued)
(iii) Percentages obtaining the Ordinary National Certificate and the Higher National Certificate in the minimum time, and in longer times, and the average time taken by students who are ultimately successful.
[page 222]
National Certificate Engineering Courses.
Table 3.3
(i) Percentages entering and qualifying at each stage, whether retarded or not, by Type of Attendance.
(ii) Percentages
(a) leaving the course at each stage after qualifying at that stage
(b) entering each stage
(c) leaving the course at each stage after failing to qualify at that stage.
[page 223]
Table 3.3 (continued)
(iii) Percentages obtaining the Ordinary National Certificate and the Higher National Certificate in the minimum time, and in longer times, and the average time taken by students who are ultimately successful.
[page 224]
National Certificate Engineering Courses.
Table 3.4
(i) Percentages entering and qualifying at each stage whether retarded or not, by Father's Occupation.
(ii) Percentages
(a) leaving the course at each stage after qualifying at that stage
(b) entering each stage
(c) leaving the course at each stage after failing to qualify at that stage.
*See page 201 for symbols and page 114 for definition.
[page 225]
Table 3.4 (continued)
(iii) There are no appreciable differences between these groups in the time taken by successful candidates.
[page 226]
National Certificate Engineering Courses.
Table 3.5
(i) Percentages entering and qualifying at each stage, whether retarded or not, by subject found very difficult.
(ii) Percentages
(a) leaving the course at each stage after qualifying at that stage
(b) entering each stage
(c) leaving the course at each stage after failing to qualify at that stage.
[page 227]
Table 3.5 (continued)
(iii) Percentages obtaining the Ordinary National Certificate and the Higher National Certificate in the minimum time, and in longer times, and the average time taken by students who are ultimately successful.
[page 228]
National Certificate Engineering Courses.
Table 3.6
(i) Percentages entering and qualifying at each stage, whether retarded or not, by Games versus No Games.
(ii) Percentages
(a) leaving the course at each stage after qualifying at that stage
(b) entering each stage
(c) leaving the course at each stage after failing to qualify at that stage.
[page 229]
Table 3.6 (continued)
(iii) Percentages obtaining the Ordinary National Certificate and the Higher National Certificate in the minimum time, and in longer times, and the average time taken by students who are ultimately successful.
[page 230]
National Certificate Engineering Courses.
Table 3.7
(i) Percentages entering and qualifying at each stage, whether retarded or not, by method of securing exemption from the first stage.
(ii) Percentages
(a) leaving the course at each stage after qualifying at that stage
(b) entering each stage
(c) leaving the course at each stage after failing to qualify at that stage.
[page 231]
Table 4. City and Guilds Courses by Type of Attendance.
(i) Percentages entering and qualifying at each stage.
[page 232]
Table 4 (continued)
(ii) Percentages
(a) leaving the course at each stage after qualifying at that stage
(b) entering each stage
(c) leaving the course at each stage after failing to qualify at that stage.
[page 233]
Table 4 (continued)
[page 234]
Table 5. National Certificate Engineering Courses.
(i) Percentages entering and qualifying at each stage, whether retarded or not, by Subsamples.
[page 235]
Table 5 (continued)
(ii) Percentages
(a) leaving the course at each stage after qualifying at that stage
(b) entering each stage
(c) leaving the course at each stage after failing to qualify at that stage.
[page 236]
Table 6. Standard Errors.
The standard error or any or the estimates in Tables 1-4 can be round in this table by interpolation. The cell entries are the values of
throughout.
For the justification of the rule see paragraph 13 in Chapter 3.
Example: Table 1 gives the number n of Engineering students with no exemption as 4,757, and the proportion obtaining the Ordinary National Certificate as 25 per cent. The relevant items in Table 7 are:
and interpolation gives 1.2 per cent as the standard error of the 25 per cent, on 113 degrees of freedom.
[page 237]
Index
1. References are generally to Tables on the pages indicated.
2. The Roman number preceding the page number indicates the Survey.
I - The Social Survey.
II - The National Service Survey.
III - The Technical Courses Survey.
3. II and III give information only on boys: "I" generally covers boys and girls in separate tables. It follows that if information is sought on girls, it will be found only in references prefixed by "I".
Subject, Part and Page Number
Ability/Ability Group
defined, II, 113
age on leaving school, II, 118, 119, 135
apprenticeships, II, 148-149
educational level, II, 122, 123
family size, II, 125, 126, 176-178, 182
levels of ability, II, 113
part-time further education. II, 144
General Certificate of Education, II, 122-124
types of secondary school, II, 120-121
Apprentices, II. 146-153
ability, II, 148-149
earnings, I, 43. II, 155, 157
parental occupation, II, 151
school background, II, 150
Board and Lodging
payment by young workers, I, 28-29
Boy Scouts
(see Uniformed Organisations)
Cadets
(see Uniformed Organisations)
Careers, I, 42-43
Cinema, I, 92-95
City and Guilds of London Institute I. 65, 75. III. 213, 216, 217, 231-233
Clubs, I, 81-91
ages of club members, I, 90-91
part-time further education, I, 82, 84, 88
membership by rural and urban areas, I, 89
type of organisations, I, 87
types of club activity, I, 84, 86 et pass
(see also Games, Youth Organisations)
Commercial Colleges/Courses, I, 62, 65, 66. II. 140-142
Craft Apprentices
(see Apprentices)
Dancing, I, 84, 86, 87, 91-94, 96-97
Day Release
(see Part-time Day Release)
Earnings
(see under Parents, Employment, Educational level)
Educational Level
defined, II, 115
ability, II, 122, 123
earnings, II, 155-157
[page 238]
Subject, Part and Page Number
Employment, I, 34 ff. II, 154 ff
earnings
age on leaving school, II, 155, 157
apprentices, I, 43. II, 155, 157
educational level, II, 155, 157
parental occupation, II, 155, 157
school attended, I, 42, 43. II, 155, 157
further education
compulsory attendance, I, 71-72, 76
inducements offered, I, 73
jobs
number held by school-leavers, I, 46 et pass
reasons for taking particular job, I, 42-43
travel time, I, 44
levels of occupation
age on leaving school, I, 37
G.C.E. performance, I, 39
occupation of father, I, 36. II, 159-160
Evening Classes, I, 76, 78. II, 143. III, 201, 222, 223, 231-233
Further Education,
full-time courses
age on leaving school, I, 37, 56
area, I, 60
duration of attendance, I, 63
home back ground of students, I, 36, 57. II, 138
student's occupation, I, 59
institution attended, I, 62. II, 138
part-time further education courses, I. 56-80. II, 139-146. III, 188 et pass
age on leaving school, I, 56
area, I, 60. II, 144
part-time further education abandoned/continued courses, I, 66 et pass. U, 142, 145
duration of part-time education, I, 68. II, 141-142, 145
non-vocational courses, II, 145
parental occupation of student, I, 57. II, 140
reasons for giving up part-time classes, I, 80
suggested by, I, 70
types of course, I, 65, 66. II, 140-142, 145
type of school, I, 9, 56 ff
voluntary/compulsory attendance, I, 71-72
(see also: City and Guilds, National Certificate Courses)
club membership, I, 82, 84, 88
examinations, I. 65 et pass
(see also: City and Guilds, General Certificate of Education, National Certificate Courses)
Games, I, 84, 86-87, 91, 93, 94, 96-97. II, 172. III, 201, 228
General Certificate of Education
ability, II, 122-124
National Certificate Engineering Courses, III, 230
part-time courses, I, 65-66. II, 140,142
present occupation, I, 39
Girl Guides (see under Uniformed Organisations)
Hobbies, I, 84, 86, 91. II, 173
Leisure activities, I, 81 ff. II, 166 ff
(see Clubs, Youth Organisations)
Libraries, I, 98
[page 239]
Subject, Part and Page Number
National Certificate Courses, I, 66, 75. III, 188 et pass
age on leaving school, III, 218-219
difficult subjects, III, 226-227
exemptions, III, 196-197, 201, 230-231
games, III, 228-229
occupation of student's father, III, 224-225
part-time day release, I, 75. III, 222, 223
success and progress rates, III, 192, 196-197, 201, 214-215 et pass
type of school attended, III, 220-221
Non-Vocational education, II, 145
Occupational Group
defined, I, 11. II, 114
parental occupation, I, 12, 13, 17, 36, 57, 62. II, 111, 118-123, 127, 129, 130, 132, 133, 138, 140, 151, 152, 153, 155, 157, 159, 160, 162, 164, 167, 168, 169, 172, 176 ff, 179. III, 201, 224-225
child's age on leaving school, I, 17. II, 132, 133
full-time further education, I,57, 59, 62. II, 138
games, II, 172
parent's income, I, 13, 57
part-time further education, II, 140
size of family, II, 127, 129, 176-178, 179
type of school attended by child, II, 130
training in industry, II, 151-153
youth organisations, II, 167-169
son/daughter's occupation, I, 36-37, 39, 42-43, 48, 50-53. II, 159-164
Parents
age on leaving school, I, 14
attitude to child's leaving school, I, 20-21
income, I, 13, 19
(see also Occupational Group (a))
Part-Time Classes (see under Further Education, Part-time courses)
Part-Time Day Release, I, 72, 73, 75 et pass. II, 142, 145. III, 201, 222, 223
employer's inducements, I, 73
length of working week, I, 77, 79
loss of earnings, I, 76
non-vocational courses, II, 145, 146
type of course, I, 75. II, 142
Part-Time Jobs (see under Secondary Schools, paid jobs)
Red Cross (see Uniformed Organisations)
Rural Areas
club membership, I, 89
further education, I, 60. II, 144
travel to and from work, I, 44
Secondary Schools
ability, II, 120-121
age on leaving school
ability, II, 118-119
attitude of parents, I, 20-21
present occupation by age on leaving school, I, 37
earnings, II, 155, 157
income of father, I, 19, 28
occupation of father, I, 17. II, 127, 132-133
parents' leaving age, I, 14, 16
school attended, I, 9. II, 131-133
selective schools, I, 15-17. II, 131-133
size of family, II, 128-129
technical courses, III, 201- 218
attitude to leaving school, I, 23-24, II, 136-137
[page 240]
Subject, Part and Page Number
Secondary Schools - continued
employment, I, 36-37, 48 et pass
paid jobs held while at school, I, 30-33
parental occupation of pupils, I, 9, 12-13. II, 112, 130
reasons for leaving school, I, 25, 28. II, 135
industrial training, II, 150, 155- 157
youth organisations, I, 82 et pass. II, 169
Sixth Forms, II, 132
Sports (see Games)
Teacher Training Colleges, I, 62
Training in Industry, II, 146-153
(see also Apprentices)
Travel to and from work, I. 44
Uniformed Organisations, I, 84 et pass. II, 166 et pass
University, I, 62. II, 138
Urban Areas
youth club membership, I, 89
further education, I, 60. II, 144
travel to and from work, I, 44
Youth Employment Officer, I, 70
Youth Organisations, I, 81-91. II, 166-171
ability, II, 170
parental occupation of members II, 167-169
type of school, II, 169
(see also Clubs)
Youth Service (see Youth Organisations, Clubs, Uniformed Organisations)

