[page 31]
APPENDIX
An Investigation into Post-War Reading Ability
CONTENTS
Foreword
I Introduction: Previous Surveys of Reading Attainment
II Definition and Measurement of Literacy
III Calibration of the New Reading Test
IV Distributions of Reading Ability
V Discussion of Results
(a) General Extent of Illiteracy
(b) Differences at Different Ages
(c) School Differences
(d) Sex Differences
(e) Service Differences
VI Summary and Conclusions
APPENDICES
A Reading test distributions in 1948-9 for 18-year adult males, 15.0 and 11.0 year pupils
B Corrected standards on Burt, Vernon and Schonell Graded Word Reading Tests
C Numbers of 15.0 year pupils tested in some 150 schools, and median reading test scores
D The factorial content of the Watts-Vernon Reading Test.
[page 32]
APPENDIX
FOREWORD
THE REPORT that follows is based on the work of a committee appointed by the Minister of Education. The main responsibility for planning and carrying out the investigation and for recording its results rests with Professor P. E. Vernon who at the time the work was done was a member of the Senior Psychologist's Department of the Admiralty. To Professor Vernon must also be accorded proportionate credit for the smooth and successful completion of a complex investigation. It could not however have been done without the careful work of the many teachers who assisted the committee in the testing of some six thousand children in over two hundred schools in many parts of England and Wales, and of the staffs of the personnel selection departments in the three Services who arranged and carried out the testing of over three thousand recruits.
I. INTRODUCTION: PREVIOUS SURVEYS OF READING ATTAINMENT
On the basis of reading tests given to 256 boys and girls and 227 adults, Professor Sir Cyril Burt (1) estimates that 1 per cent of 16-year-olds and 1½ to 2 per cent of 20-25-year-old adults are illiterate, having reading ages below 6½ years; and in addition that 10 per cent and 15-20 per cent respectively are semi-literate, having reading ages below 9 years. Though these samples are small he states that their results are confirmed by researches in the Services and elsewhere. They are accepted as correct by another authority on reading, Professor F. J. Schonell (2). His colleague, Dr. W. D. Wall (3), quotes 20 per cent and 1 per cent as the proportions of adult males with reading ages below 10 and 7½ years respectively, but does not mention the samples tested. (If Burt's border lines are substituted the figures would drop to approximately 12 per cent and ½ per cent.) By applying individual tests of reading comprehension and mechanical reading to backward Army recruits, he shows that the deficit is far more marked on the former than the latter.
One extensive investigation of primary school children is that of Miss Hammond (4), who arranged the testing of practically the whole of the school population of Brighton between the ages of 6 years 6 months and 11 years 11 months, numbering 8292 children, with Burt's Graded Word Reading Test. Her tabulations suggest that the average amount
(1) Burt, C., The Education of Illiterate Adults. British Journal of Educational Psychology, 1945, 15, 20-27.
(2) Schonell, F. J., Problems of Illiteracy. Times Educational Supplement, February 23rd, 1946.
(3) Wall, W. D., Reading Backwardness Among Men in the Army. British Journal of Educational Psychology, 1945, 15, 28-40.
(4) Hammond, D., Attainment in Reading. Times Educational Supplement, Aug. 14, 1948.
[page 33]
of backwardness in 1948 is small; thus the median reading age of all 7:0-11 children is 7.0, and of 10:0-11 children 10.9; boys are slightly poorer than girls. She claims that the proportion of backward readers increases with age, but this appears to be due merely to the inability of the test to measure very backward reading in 6 year pupils, and to measure very advanced reading in 11 year pupils. Unfortunately the test, first published in 1921, is almost certainly too lenient for 1939 standards. Vernon (5) has issued new norms for Scottish children, and if these hold for English children (6), it may be deduced that the median performances of 7-plus and 10-plus children in Brighton are equivalent to reading ages of 6.7 and 9.8 respectively. In other words the average amount of retardation in the primary schools is between two-thirds and three-quarters of a year at all ages. If we define backwardness as the possession of a reading quotient below 80, then some 32 per cent of the 10 and 11-plus children are backward, as contrasted with the accepted pre-war figure of about 10 per cent. It should also be pointed out that Brighton is likely to be superior to the country as a whole. If rural and slum areas were investigated the amount of backwardness might be considerably greater.
In September 1947 the London Head Teachers Research Committee, using the Holborn Reading Test (7), tested 1972 pupils of average age 11½ entering London secondary modern schools mainly in poor areas. As many as 15.9 per cent failed to reach a reading age higher than 7½ years; that is their reading quotients were 65 or below. This would suggest that the median reading age is little greater than 9.0 and that nearly 50 per cent are backward, having reading quotients below 80. The figures are worse than those just quoted for Brighton, but the difference is probably due not merely to the larger proportion of slum areas in London, but to the exclusion of the brightest third of London pupils who go to secondary schools other than modern ones.
The London results are confirmed by an investigation of some 2500 pupils aged 14-plus in Birmingham secondary modern schools by Dr. Wall and Miss Lampard (8). Details have not yet been published, but here too some 50 per cent fall in the backward grade, and it is found that approximately 10 per cent have reading ages below 8½ years. We may reasonably deduce that, when they leave school at 15.0, much the same proportion will have reading ages below Burt's borderline of 9, i.e. that 10 per cent represents the figure for semi-literacy and illiteracy in such schools.
In any discussion of backwardness and illiteracy it is essential to realise how great the extent of these phenomena depends upon the
(5) Vernon, P. E., The Standardisation of a Graded Word Reading Test, University of London Press, 1938.
(6) This assumption might be challenged. However it is shown later in the Report that Vernon's Scottish standards do coincide with English ones as measured by Schonell's Graded Reading Vocabulary test, at least over the 8.0-12.0 range. A conversion table for all three tests is given in Appendix B.
(7) Watts, A. F. The Holborn Reading Scale. Harrap. 1948.
(8) Lampard, D. M., The Relationship Between Levels of Reading Comprehension and Word Recognition in Secondary Modern Pupils. Paper read to Section J, British Association for the Advancement of Science, Sept. 1948.
[page 34]
criterion adopted. For example, Vernon (9) found that roughly half of a large group of Ordinary Seamen in 1943, who were fairly representative of young male adults, were "clearly incapable of writing an intelligible and reasonably grammatical, even if simple, sentence", when answering a short examination paper. On the other hand the Army personnel selection staff defines as illiterates those recruits who cannot produce legible answers to the questionnaire or Qualification Form filled up on entry to the Service; and such recruits have consistently numbered between 1 and 2 per cent. Similarly it was stated in the House of Commons (Feb. 15, 1949) that some 2 per cent of National Service recruits are illiterate or of extremely low literacy. This refers to the number who fail an examination set by the Education Officers, and are sent to Preliminary Education Centres for basic schooling.
These are extreme instances of divergent estimates. But we shall see below that quite considerable variations occur even between the results of different standardised reading tests. In addition, since backwardness is a matter of degree, the proportions of backward or illiterate individuals will naturally depend on just where the dividing lines are drawn. Hence an attempt has been made in this, and later, sections to adhere to a few uniform divisions, as follows:
Backward readers are those whose reading ages are more than 20 per cent below their chronological ages, in the case of children, i.e. those whose reading quotients are below 80. In the case of adults, the expected average reading age may be taken as 15.0 years, hence backward readers are those with reading ages below 12.0 years.
Illiterate readers are those whose reading age (regardless of chronological age) is less than 7.0 years. (This figure differs from Burt's 6½ years merely because the reading test described below did not readily measure the 6½ year level.)
Semi-literate readers are those whose reading age is 7.0 or greater, but less than 9.0 years.
II. DEFINITION AND MEASUREMENT OF LITERACY
In order to obtain really reliable estimates of the extent of illiteracy, four conditions must be fulfilled. First we must decide what we mean by literacy-illiteracy; secondly we need valid tests for measuring this quality. Thirdly, if we are to gauge any change in the amount of illiteracy since 1939, the tests must be standardised or calibrated in terms of pre-war norms. Fourthly the tests must be applied to adequately representative samples.
Many and varied definitions of literacy have been given, but present-day conceptions normally imply both competent reading and intelligible self-expression on paper. It seemed better in the first place to keep to reading, since if a test involved a mixture of reading, handwriting, spelling and composition, there would be no means of deciding which ability was up to standard or retarded. A separate investigation of writing ability is being planned. The definition of reading ability also raises many problems, and there is an extraordinary
(9) Vernon, P. E., An Experiment on the Value of the Film and Film-strip in the Instruction of Adults. British Journal of Educational Psychology, 1946, 16, 149.
[page 35]
lack of conclusive research. We know that the ability is not a single unitary entity which can be adequately measured by a conventional graded word (mechanical) test or a paragraph comprehension test, but there is no agreement as to what distinctive factors are involved.
Batteries of tests for measuring different aspects of reading have been published by several educational psychologists including Burt (10, Schonell (11), Watts (12), Gates (13), Triggs (14), and others, but their lists of tests show little concordance (15). In other words, each author analyses the total complex of reading skills into different components. Considerable doubt is cast on all a priori analyses by recent American investigations at the college level, using the technique of factor analysis. Davis (16) described nine components of adult reading ability, and on devising tests to measure these and analysing their inter-correlations, he claimed to find two independent factors in reading corresponding to word knowledge and verbal reasoning, together with several smaller factors. But Thurstone (17) re-analysed his data and concluded that all the correlations could be well accounted for by a single general factor - a blend of knowledge of word meanings and comprehension of sentences and paragraphs. Clearly it is most desirable that similar investigations should be carried out in this country, at several age levels, with a variety of reading and other literacy tests, in order to establish both the underlying essence or general factor in literacy, and any distinguishable group-factors or sub-types of ability (cf. Appendix D).
In default of such information it was decided that the test employed for surveying literacy should aim to measure the general factor described by Thurstone, by means of silent-reading sentences, probably this vocabulary plus comprehension ability overlaps with, but is relatively distinct from, mechanical word-pronouncing ability. It is more difficult to differentiate it from general intelligence, for several intelligence test batteries have actually included both vocabulary and
(10) Burt, C. L., Mental and Scholastic Tests. King, 1921.
(11) Schonell, F. J., Backwardness in the Basic Subjects. Oliver and Boyd, 1942. The Psychology and Teaching of Reading. Oliver and Boyd, 1945.
(12) Watts, A. F., The Language and Mental Development of Children. Harrap, 1944.
(13) Gates, A. I., Silent Reading Tests, Grades 3 to 8. Bureau of Publications, Teachers College, Columbia University, 1935.
(14) Triggs, F. O., Description of the Purposes and Functions of the Diagnostic Reading Tests. Educational and Psychological Measurement, 1948, 8, 3-14.
(15) Burt and Schonell agree in providing tests of Graded Word Pronunciation, Continuous Prose Pronunciation, Speed and Comprehension. Gates's tests for Grades 3 to 8 claim to measure Reading to Appreciate General Significance, to Predict the Outcome of Given Events, to Understand Precise Directions, and to Note Details. Triggs's series for Grades 7 to 12 includes Vocabulary, Visual and Auditory Comprehension, Rate of Reading three types of material, and two tests of Word Attack.
(16) Davis, F. B., Fundamental Factors of Comprehension in Reading. Psychometrika, 1944, 9, 185-197.
(17) Thurstone, L. L., Note on a Re-analysis of Davis's Reading Tests. Psychometrika, 1946, 11, 185-188.
[page 36]
sentence-comprehension items. Nevertheless it was thought that sentences could be devised which would be sufficiently straight-forward not to involve much reasoning capacity or g-factor, and which would also avoid specialised literary or technical terms. The content of the test should be based on concrete, everyday life situations, familiar to 15 year children and adults. To ensure objectivity of scoring it should have multiple-choice or selective responses, though there is much to be said in favour of creative-response items among backward children and adults. Two examples follow of the kind of questions which, it was hoped, would give a fair indication of the effectiveness of a child's or adult's reading ability, that is of his literacy for practical purposes of daily life.
1. We hope to see my favourite film-star tonight at the local (pub, cafe, review, station, cinema).
2. The passengers injured in the railway accident were taken by ambulance to the General (Omnibus, Canteen, Hospital, Terminus, Post Office).
Several early versions were devised mainly by Dr. A. F. Watts, and were tried out by him on classes of 14-plus pupils in London schools. Surprisingly, in view of the large extent of backwardness and illiteracy that had been expected, many of the initial items were too easy. Thus all but 2 per cent of London secondary modern pupils could manage the above sentences. Some more difficult items were contributed by the experienced test-constructing staff of the Civil Service Commission's Research Unit. Inevitably these were more abstract and involved than had originally been planned. Two examples of items needed to cover the average and superior levels are:
3. The delegates to the conference committed themselves enthusiastically to a far-reaching programme of economic planning, ranging from abolition of obstacles in the way of trade to joint plans for agricultural (areas, subsidies, Labour, development, co-operation).
4. It is difficult to make him see reason. The most convincing arguments fail to influence his (omnipotent, obtuse, obsequious, obverted, obdurate) character.
A 60-item test was now applied to 485 children aged around 15 years, 11 years and 8 years, and to 200 R.A.F. recruits. A thorough item analysis was carried out. Each of these four groups was split into top, middle and bottom thirds on the test as a whole, and the responses of each group to each item were tabulated. Thirty-five items were selected for the final test, which fulfilled the following, requirements:
(i) The difficulty level of the items in the 15 year and adult groups should be such that the test would discriminate over almost the whole range of reading ability.
(ii) There should be a large rise in the pass-rate for each item from 8 year to adult readers.
(iii) Within each age group there should be a large rise in pass-rates from the bottom to the top third.
(iv) There should be a good distribution of wrong responses, no one alternative being too popular. For example:
5. Halt, Major Road Ahead means (turn back, turn right, drive faster, stop, drive slowly).
was rejected because almost all children who got it wrong chose 'drive
[page 37]
slowly' as their answer. The four examples already quoted were also rejected because they failed to meet one or other of these requirements.
The Watts-Vernon 35-item Silent Reading Test occupies front and back of a foolscap sheet, so that its paper consumption is not large. The time limit is 10 minutes: It can be applied and scored by teachers not specially trained in psychological testing. It measures reading ages from approximately 7 year to 18 or 19 year (pre-war) levels, that is up to the highest levels encountered in secondary modern and technical schools, though it is not difficult enough for the best grammar school pupils or for university-educated adults. The reliability or internal consistency co-efficients were 0.89 both in the R.A.F. and the 15 year secondary modern groups (18). A higher co-efficient would be expected in a complete age group. Thus although the test was intended only for comparing groups of children or adults, it also gives fairly trustworthy reading ages for individuals. It is not to be published meanwhile, but copies and instructions may be obtained from the Secretary of the Sub-Panel on Illiteracy, Ministry of Education, Curzon Street, London, W.1., by qualified research workers.
III. CALIBRATION OF THE NEW READING TEST
In order to find what would have been the performance of children in 1938 on the new test, it was applied to groups of pupils in 1948 along with other reading tests for which pre-war norms (believed to be accurate) were available. An investigation was planned with the assistance of Dr. Watts, and visits were paid to six secondary and three primary schools in London or the Home Counties, chosen to be fairly representative of all types. Three, four, or five of the following tests (depending on level of ability of the pupils) were given to 432 children with ages of about 14:9 to just over 15:0, and to 358 pupils in 11-plus or 8-plus classes. The numbers of boys and girls were approximately equal.
1. A Number and Letter Series, or an Abstraction, test of general intelligence (both unpublished); 15 mins.
2. Watts-Vernon Silent Reading Test; 10 mins.
3. Schonell Silent ReadingTest B; 15 mins.
4. Vernon Spelling Test (re-writing a mis-spelt word in each of 30 sentences, unpublished); 10 mins.
5. Cattell Midland Attainment Test No. 2 (Comprehension): 10 mins.
6. Vernon Graded Arithmetic-Mathematics Test; 20 mins.
The order of application of the tests was varied to offset practice or fatigue effects. All the group testing was done by one person, Sergeant John Lewis, an experienced tester lent by the War Office Directorate for Selection of Personnel. In addition 284 children picked
(18) The co-efficients were obtained by the Kuder-Richardson Formula 20.
[page 38]
at random in each class were given two individual tests, in alternating order, by Mrs. D. A. F. Vernon or by Professor Vernon:
7. SchoneIl's Graded Reading Vocabulary Test.
8. Vernon Graded Word Reading Test.
Analysis of the individual test results showed that there was no significant difference between the two testers when testing comparable groups of children. A graph was drawn of Schonell vs.Vernon reading ages, and these were found to be, on the average, identical over the 8.0 to 12.0 years range. Inter-correlations in the 11-plus and 8-plus groups were .95 to .96, and the (arithmetical) median discrepancy was 0.4 years. Below 8.0 years the Vernon test is probably a little too difficult, having been standardised originally on Scottish children, who tend to make an earlier start on reading than English children. Incidentally the order of difficulty of the words was also less appropriate in the Vernon test; a considerably larger range of words had to be attempted between the first failed word and the last successful word. Above 12.0 years, the Schonell test probably becomes too difficult. It only claims to measure up to a reading age of 15.0, but comparison with the Vernon test (which goes up to an arbitrary reading age of 21.0) suggested that Schonell's last ten words actually correspond to the 15½ to 19½ year levels. A correction table was therefore built up for converting Vernon scores at the bottom end of the scale, and Schonell scores at the top end, into what are considered to be correct reading ages (cf. Appendix B). The two tests were then averaged to give an individual reading age for each pupil.
These ages were compared with the Watts-Vernon test scores and an approximate set of pre-war norms was reached for the new test by the method of equivalent percentiles. A second approximation was made similarly by comparison with the Schonell B test. This had been taken by 550 children, and yields reading ages from 7 to 13½ years. Third and fourth comparisons were made with the Cattell Midland Test (157 cases, 6-14 year norms), and, the Vernon Spelling Test (all cases, 7-19 year Scottish norms). The results are given in Table I. They show fairly close concordance, except for the norms derived via the Cattell test. Clearly this is far too lenient, which is unfortunate since it is the easiest of any of the tests used to 'get across' to younger children. But the agreement between the approximations is not perfect. Thus a child with a Watts-Vernon score of 13 is likely to have a reading age of 11.0 on the Graded Word tests, a reading age of 10½ on Schonell B, and a spelling age of 9¾ on the Vernon test. Presumably the post-war English child tends to do a little better on individual word-reading than on comprehension, and he is relatively low in spelling because Scottish spelling standards are exceptionally high. The final pre-war norms for the Watts-Vernon test, given in the last column, are based on an average of the three approximations (excluding Cattell), and conform most closely to the Schonell B standards over the range which this test covers. Reading ages higher than 14-15 are, like mental ages, arbitrary units for expressing superior ability. A reading age of 21.0 would correspond roughly to the level of university arts graduates.
[page 39]
TABLE I. DERIVATION OF PRE-WAR AGE NORMS FOR WATTS-VERNON TEST
IV. DISTRIBUTIONS OF READING ABILITY
The standardised test was now applied by teachers to the following school groups, through the co-operation of many local education authorities and by trained testers in the Forces:
The distributions of results are given in Table II in two ways. First the 90th, 50th and 10th percentile scores are shown, that is the scores reached by the top 10 per cent of readers, the middle or median score, and the score cutting off the bottom 10 per cent. Next the percentages are shown with pre-war reading ages of:
17.0-plus (Superior) scoring 31-35
13.8-plus (Average plus) scoring 23-30
[page 40]
This dividing line was taken because the median score for aIl 15 to 18 year-olds was estimated as 22½.
12.0-plus (Average minus) scoring 18-22
9.0-plus (Backward) scoring 9-17
7.0-plus (Semi-literate) scoring 3-8
Under 7.0 (Illiterate) scoring 0-2
In the case of 11.0 pupils, the divisions are taken at reading quotients corresponding to those of 15.0 year olds, namely at reading ages of 12.5, 10.1, 8.8 and 6.6 respectively. That is, the proportions with test scores of 19.5-plus, 12.3-plus, 8.4-plus, 2.2-plus are reached by interpolation. The two bottom groups, corresponding to semi-literate and illiterate, are combined.
TABLE II. WATTS-VERNON READING TEST DISTRIBUTIONS
[click on the image for a larger version]
[page 41]
It would be useless to attempt to estimate the reading test distributions for all 15 or 11 year children or for 18 year males merely by combining the results of the groups actually tested. A representative 15 year sample must, for example, include the correct proportion of grammar school pupils for the country as a whole. Synthetic samples were therefore built up as follows. From official sources (19) it was found that children approaching 15 attend the main types of schools in these proportions:
| per cent |
Secondary Grammar (L.E.A.) | 18.6 |
Secondary Grammar (Direct Grant) | 2.2 |
Recognised Efficient Secondary | 3.1 |
Secondary Technical | 3.1 |
Secondary Modem Urban | 47.3 |
Secondary Modern Rural | 3.6 |
Primary All-Age | 20:5 |
Other Direct Grant and Recognised Efficient | 0.7 |
Special Schools | 0.9 |
| 100.0 |
A few types of schools are omitted from this list, but the numbers of pupils thus involved are small, and they are unlikely to upset the relative proportions shown here. It seemed fair therefore to accept the figures of 20 per cent of our grammar pupils and 7 per cent of our technical plus central pupils. The official figures for rural secondary modern schools are based on local government boundaries, not on size of schools or concentration of population. The generally accepted estimate of the rural population is about 20 per cent. Since the standards of most primary all-age schools are likely to be similar to those of rural schools, it was decided to include 47 per cent of our urban sample and 25 per cent of our rural and small-town sample. This leaves 1 per cent for pupils in special schools for the educationally retarded.
Part of the 11 year sample had already transferred to secondary schools (20). Among those in primary schools it was decided to include 5 per cent private, 65 per cent urban and 30 per cent rural and small-town schools.
Among adults, we know the approximate proportions called up by the three Services at the date of testing, namely Army 45 per cent, R.A.F. 11 per cent and Navy 4 per cent, but some 40 per cent are not called up, consisting (in roughly equal numbers) of: (1) medical rejects; (2) deferred apprentices; (3) exempted farm workers, miners and members of the mercantile marine; (4) volunteers to the regular Forces. From previous intelligence test figures there is reason to believe that medical rejects and deferred apprentices do not differ markedly from called-up recruits in intellectual and educational level; also that farm workers are of much the same level as labourers. An approximate distribution for farm workers, miners and mercantile marine was
(19) These figures are based on the number of children between 13 and 14 in January, 1948, in grant-aided and other schools recognised as efficient.
(20) It was not possible to obtain samples with median age exactly 11.0, but as the Gloucestershire and London groups averaged about one third of a year older, and the Huddersfield group one third to one half a year younger, adjustment was made by subtracting 1 mark from the scores of the former and adding one mark to the scores of the latter.
[page 42]
therefore estimated from that of men actually called up who were labourers, and this was added to the Service distribution in the correct proportion.
The actual distributions of reading test scores of these three synthetic samples are given in Appendix A. These are onIy doubtfully representative of the total population, but at least an attempt has been made to eliminate possible biases in the light of the best available information. If they err, it is more likely to be on the strict than the lenient side; for the proportions of rural pupils may have been overestimated, and the educational level of men whose call-up is deferred under-estimated.
V. DISCUSSION OF RESULTS
(a) GENERAL EXTENT OF ILLITERACY
Though there is certainly an increase in backwardness compared with pre-war norms, the percentages of semi-literates and illiterates are much smaller than some of the estimates. Approximately 30 per cent of 15-year-olds, 23 per cent of 11-year-olds and 16 per cent of adults fall in the backward or lower groups, instead of the expected pre-war 10 per cent. Moreover the median reading ages of 15 and 11-year-olds are 13:2 and 10:0, that is retardations of 1:10 and 1:0 years (21). But the proportions of nearly or completely illiterate are only 3.6 per cent in adults and 5.7 per cent in 15-year-olds, and the proportion of 11-year-olds with correspondingly low reading quotients is 3.8. Why then are these figures so much smaller than some of those put forward by educational psychologists and others? Any or all of the following factors may be involved:
(i) Ascertainment in the present survey may have been incomplete; some of the most backward pupils may have been withheld because they were not in the top classes of the secondary modern or primary all-age schools. It cannot be guaranteed that this never occurred, though precautions were taken to prevent it. Another possibility is that, as most of the testing of 15-year-olds took place towards the end of the summer session of 1948, there may have been a number of absentees, and duller children who were shortly to leave school for good might be more likely to play truant.
Neither of these factors is likely to have biased the 11 year or adult samples. A further check on the Army figures (which constituted a major part of the adult sample) is provided by the proportion of men found by the Directorate of Army Education to be very low in literacy. As already mentioned, this proportion was 2 per cent in 1948-9. But the Army discharges another 2 to 2½ per cent of its intakes, and most of those men score very low on intelligence tests. Were they retained, most of them would probably qualify for Preliminary Education Centres. Thus the recruits of very low literacy in the total Army intakes probably number between 3 and 4 per cent and this agrees precisely with the proportion scoring below 9.0 years on the Watts-Vernon test.
(ii) The samples tested by other investigators may also have been unrepresentative. For example it is not generally realised that so large
(21) Had the test been given to 15.0 year olds in 1938, the average reading age might, of course, have been lower than 15.0 because so large a proportion of children then left school at 14. But it is reasonable to assume that the extra year of schooling should add one year to reading ability.
[page 43]
a proportion of the 14-15-year-old population as 27 per cent is in grammar or technical schools. If we omitted them our proportions of semi-literates and illiterates among 15-year-olds would rise to 6.1 per cent and 1.7 per cent or nearly 8 per cent in all. And the backward pupils (including illiterates) would rise to 48 per cent. These figures do not fall far below those quoted in the Introduction for secondary modern schools in London and Birmingham.
(iii) The extra year at school and the present re-organisation of post-primary education may already have made some difference, though it is difficult to believe that these could have much effect on the pupils of very low literacy.
(iv) The tests used in some earlier investigations, such as Ballard's probably have unreliable norms. But Burt's Graded Word test is, if anything, too lenient; and Schonell's B test has, as we have seen, standards almost identical with those of the Watts-Vernon test.
(v) Perhaps the chief factor is the extent to which different tests spread out those tested. It is well known that some intelligence tests yield many more extreme (high or low) I.Q.s and fewer middling I.Q.s than others. Their Standard Deviations may range from about 15 to 30. The same may happen with reading tests. Two tests might be equally well standardised, and yet if the Standard Deviations of their reading quotients were 20 and 30, they would yield 2.3 per cent and 9.2 per cent of 15 year readers as having reading ages below 9.0 years (22). Schonell and Wall have repeatedly established greater amounts of backwardness on comprehension than on mechanical reading tests, showing therefore that mechanical tests spread out readers to a smaller extent. Can we show that the Watts-Vernon test has a smaller spread than other commonly used tests?
The new test, together with Schonell's B test and Vernon-Schonell graded word tests were all taken, during the calibration experiment, by 81 15-year-olds and 44 11-year-olds. The proportions of 15-year-old scoring below the 11 year medians were 30.9, 34.8 and 42 per cent respectively. Similar though smaller differences were obtained by contrasting 11-year and 8-year-olds. To provide further verification, a group of 51 very backward Army recruits in Preliminary Education Centres were given Watts-Vernon and Schonell B; they had also taken Burt's Graded Word test. The results are shown in Table III. It may be seen that 51 per cent are below the 9 year borderline on Burt's test, 66 per cent on Schonell B, and only 39 per cent on Watts-Vernon; also that the median Schonell score is 1¾ years lower than the median
TABLE III. PROPORTIONS OF BACKWARD ADULTS WITH VARIOUS READING AGES ON THREE READING TESTS
(22) This calculation assumes that reading ability is normally distributed. Almost certainly it is negatively skewed, so that the correct proportions below a given reading age cannot be predicted from the standard deviation of reading quotients. It nevertheless provides a valid illustration of the differences yielded by tests possessing different spreads.
[page 44]
Watts-Vernon. Clearly the difference in spreads is more marked among adults than children, but it is almost certainly large enough in children to explain any discrepancies between the results of this and of the Brighton or London surveys.
The reasons for these differences among intelligence or reading tests are obscure, but 'straight-forwardness' vs. 'unusualness' seems to have something to do with them. In other words, tests involving familiar operations give smaller spreads than those involving complicated and novel mental manœuvres. Mechanical reading of words is so well drilled in children that a test based on it yields fairly small proportions of semi-literates. But many comprehension tests, including Schonell's, may involve a good deal of what Professor Vernon has called 'test-sophistication' (23), and so prove more troublesome to the unscholastically-minded secondary pupil or adult. Such tests might therefore show larger proportions of semi-literacy. The Watts-Vernon test is not, of course, wholly familiar in type, but it calls for less complex operations than most paragraph comprehension tests. Another common difference between tests with high or low spread is that the latter tend to be more variegated, confined less to a single function. Now reading in everyday life is much more variegated than reading as measured by any single test in existence. Thus we would expect the proportions of semi-literates to decrease the more nearly our tests approximate to real-life situations. This too may explain why mechanical reading tests, which are extremely homogeneous, sometimes give larger spreads, than does the Watts-Vernon test.
The above arguments may appear somewhat technical and hypothetical. But actually they have vital practical significance. For they imply that there is no one 'true' proportion of backward or illiterate readers in the population. It all depends on the type of test used, in other words on what we mean by, or use as our measure of, literacy. We have given reasons for regarding a test of the Watts-Vernon type as a better measure among adults but many teachers might justifiably regard a test of word pronunciation as more appropriate for primary-school children. It follows that the chief finding of this investigation is not that a large proportion of the population is backward in reading compared with pre-war standards, nor that the proportion is less big than some estimates that have been put forward, but that standards have been established for fairly representative groups at three age levels against which subsequent educational progress can be assessed. Investigations are already planned to show in a few years' time whether the schools have made good their losses, whether the extra year at school produces more literate recruits, and so forth.
(b) DIFFERENCES AT DIFFERENT AGES
The average retardation in years of reading age is smaller at 11 than 15, but in terms of reading quotients there is little difference. Thus the mean reading quotient of older children is 88, of the younger ones 91. A more valid comparison can perhaps be made by taking 265
(23) Vernon, P. E. Intelligence Test Sophistication. British Journal of Educational Psychology, 1938, 8, 237-244.
[page 45]
11-year-old children in certain all-age schools in Gloucestershire, where 125 15-year-olds were also tested. These samples should be exactly comparable. Both were slightly above average, their median reading ages being 13:8 and 10:4. Their median chronological ages were 15:0 and 11:2; thus their reading quotients are 91 and 92.
The results for the 147 children of median age 8:8 tested in the calibration experiment in two London schools are also worth quoting. Though they do not, of course, constitute a representative sample, they are probably quite close to average, since the median reading quotient of the 11-year-olds in the same schools was 88. Their median reading age was 7:4, giving them an average quotient of 85. In the Brighton Survey too the mean quotients of the 6 to 7-year-olds were distinctly lower than that of the 10 to 11-year-olds (even when Burt's norms were adjusted), though in terms of years of reading age the younger children were less retarded.
We must conclude therefore that there is no evidence that backwardness exists only among the older pupils whose education was upset by war-time conditions. It seems to be as marked throughout the primary schools as the secondary schools in 1948.
Comparing the distributions for adults and 15-year-olds, we find slightly smaller proportions in the lowest categories, but far more adults in the superior group. As more than half of these adults have not had any schooling beyond 14, it is clear that reading ability tends to improve between 14 and 18, except perhaps among the poorest readers. In a Report to the British Association for the Advancement of Science in 1947 Professor Vernon gave evidence of a 'fanning downwards' in arithmetic and spelling abilities between 14 and 18. The superior pupils appeared to retain their level, the average ones dropped slightly, while the duller ones lost their scholastic skills quite rapidly. Here we have the reverse phenomenon - fanning upwards. Good readers increase their reading ages by about 2 years during the 3 to 4 years after leaving school. Those whose reading age was 10-12 years on leaving increase by about 1 year. Those whose reading age was under 8 do not on the average show any appreciable increase; possibly indeed many of them decline. It might be expected that, although reading and arithmetic are little used except by the well-educated after 14-15, reading would continue to be practised by all but the most incapable. We can guess also why it is that Professor Burt claims a greater proportion of illiterates at 20-25 than at 16. If his Graded Word scale was used, it is quite possible that skill in word-pronunciation drops, as do arithmetic and spelling, although ability to comprehend reading matter increases.
(c) SCHOOL DIFFERENCES
Differences in the 15 year groups between different types of schools are enormous, ranging from grammar schools with 37 per cent superior readers and less than 1 per cent backward or worse, to rural secondary modern and mixed schools with less than 1 per cent superior and over 50 per cent backward. While this conforms to expectations it also shows that the test is quite a good measure of educational attainment, and suggests that selection for different types of secondary education is fairly efficient. Thus only 3 per cent of all secondary modern pupils read up to the level of the average grammar school pupil.
[page 46]
Differences between schools within any one main type are comparatively small, apart from chance fluctuations of sampling (24). This indicates that our sampling is fairly adequate although only limited areas of the country have been tested. The differences are shown in Appendix C, which lists the numbers in each school opposite the median score obtained by that school. Thus the best grammar schools, where 95, 93 and 32 were tested, obtain a median of 30. The poorest grammar school obtains a median of 23, but this is in a small town in a rural area. A well-known boys' public school lies just at the median for all grammar schools. Technical and selective central schools seem to occupy much the same level, and pupils in private commercial colleges are only slightly poorer; (they are almost all girls and include 12.6 per cent backward, compared with 7.2 per cent in publicly maintained schools). Secondary modern schools show a wider range, though the more extreme values occur chiefly in small groups. No consistent differences could be discerned between London and the provincial towns. Home Office school boys are only slightly poorer than rural secondary, but one special school for physically handicapped gave a median score of 14½ (10 boys), and one for the educationally sub-normal gave a median score of 8 (9 boys).
In the 11 year groups the small sample of preparatory or private school children is appreciably superior to the others, but not so much so as are 15 year grammar to secondary modern pupils.
(d) SEX DIFFERENCES
At 15 years girls average about 1 point lower than boys. The 90th and 50th percentiles are the same for the two sexes in grammar and and technical schools, but girls range lower at the bottom end. In secondary modern and all-age schools, the 90th and 50th percentiles are lower, but the 10th percentiles score is slightly higher. In other words, very good readers among girls are as good as boys, average ones are a little poorer, and very poor readers are somewhat less frequent. It will be seen indeed that there are about twice as many boys as girls in the lowest, illiterate, grade. The lower average performance is curious, since the norms quoted by Schonell for his test B are higher for girls. The reason probably lies in the content of the tests. Schonell's consists largely of fairy and juvenile fictional stories, whereas the Watts-Vernon test draws on more realistic material such as jobs, politics, etc., which may be more interesting to boys.
In the 11 year groups, the sexes were not always distinguished. But in the large Huddersfield sample the median scores for boys and girls are identical, while girls show a smaller range, i.e, fewer very good and very poor readers. Their 90th percentiles is one point lower, and their 10th percentile one higher.
(e) SERVICE DIFFERENCES
It will be seen that the Army National Service intake is closely similar to the estimated total population. The R.A.F. and Navy secure
(24) Analysis of variance did in fact yield highly significant differences between schools within any one of the four main types, but these accounted for only about 10 per cent of total variance as contrasted with 48 per cent due to differences between types, and 42 per cent due to differences between individuals within schools.
[page 47]
intakes of decidedly better quality. The Navy has scarcely any backward readers and no illiterates, and the R.A.F. has only 1½ per cent of men of very low literacy. Among regular recruits (volunteers) both in the Navy and R.A.F. the average level is at least as good as that in the total population, though inferior to that of the National Service recruits to these Services. The range of ability is, however, restricted. Volunteers contain no illiterates and scarcely any semi-literates, but also fewer superior readers. These findings all accord quite well with expectations.
VI. SUMMARY AND CONCLUSIONS
1. Recent investigations and public statements have suggested the existence of far more backwardness and illiteracy among schoolchildren and adults than before the 1939-45 war. The estimates vary markedly, however, and adequate attention has not always been paid to the correctness of the norms or standards of tests used, or to the representativeness of the samples tested.
2. In an attempt to survey the proportions of illiterates and semi-literates among 11.0 and 15.0 year children and young adult recruits, and to compare with pre-war standards, a new group silent reading test was devised by Dr. A. F. Watts and Professor Vernon.
3. Pre-war reading age norms from 7.0 to 19.0 years were established by comparing the test scores of some 800 secondary and primary school pupils with their scores on several other tests standardised before the war.
4. The test was applied to some 3500 children aged close to 15.0 and some 2800 aged close to 11.0 in over 200 schools in many parts of England and Wales, also by the personnel selection departments of the three Services to some 3000 recruits, mostly aged 18 years. Results were classified as Superior (Reading Age 17.0+), Average + (13.8+), Average - (12.0+), Backward (9.0+), Semi-literate (7.0+) and Illiterate (under 7.0.).
5. A synthetic sample, representative of the whole 15.0 year population, was found to contain only 1.4 per cent of illiterates and 4.3 per cent of semi-literates. In a similar sample of adult males the proportions were 1.0 per cent and 2.6 per cent respectively. The much larger estimates given by other writers may be ascribed mainly to differences in the spread of the tests used.
6. Reading ability continues to improve after leaving school except among the very poorest readers.
7. The median reading age for the 15.0 year sample was 13.2, that is a drop of about 1¾ years below pre-war level. A median drop of 1.0 years in the 11.0 sample suggests that the amount of retardation is very similar in primary schools. The proportions of backward readers (including illiterates) are 30 per cent, 23 per cent and 16 per cent in the 15, 11 and adult samples.
[page 48]
8. Very large differences in attainment are demonstrated between different types of school - e.g. grammar and public, technical, urban secondary modern or church schools, and rural or small-town schools. Girls are on the average slightly poorer at this test, but show almost as many superior readers as boys, and fewer illiterate ones.
9. Army National Service recruits at the beginning of 1949 are similar in reading to the total 18 year population, and naval and R.A.F. recruits are much superior. Volunteers to the two latter Services are inferior to conscripts, but are so much more homogeneous that they contain scarcely any men or women of very low literacy.
10. It may be concluded that the proportions of backward readers in English and Welsh primary and secondary modern schools are larger than they were before the war, but that the numbers of pupils leaving school and of young adults who are nearly or completely illiterate, namely about 5 per cent, is less serious than is often stated. But in view of the fact that the proportion varies considerably both with the type of test and the standards adopted, it is suggested that the question as to the amount of illiteracy is somewhat artificial, and cannot be finally answered. The main value of this investigation lies in the establishment of standards on a new and unpublished test for various samples of adults and children, against which subsequent improvements or declines can be measured.
[page 49]
Appendices to the Report on the Investigation
APPENDIX A. Reading Test Distributions in 1948-9 for 18 year Adult Males, 15.0 and 11.0 year Pupils.
APPENDIX B. Corrected Standards on Burt, Vernon and Schonell Graded Word Reading Tests.
Scores on the three tests are listed in the Table, and the corresponding corrected reading ages are given down the left-hand side (years} and along the top (tenths).
For example, a reading age of 14.6 is believed to correspond to a score of 101 words on Burt, 96 on Vernon and 86 on Schonell. These are not new norms; they are intended merely to make the tests equivalent. The Vernon test is too difficult at the bottom end, the Burt test too easy; while the Burt and Schonell tests are too difficult at the top end.
[page 50]
APPENDIX C. Numbers of Pupils (15.0) Tested in some 150 Schools and Median Reading Test Scores
[click on the image for a larger version]
APPENDIX D. The Factorial Content of the Watts-Vernon Reading Test
Two preliminary studies of the content of the new test were made by factorial analyses. In the calibration experiment, 81 secondary modern and central school 15-year-olds took five tests. Their inter-correlations are tabulated below. Two factors were extracted by the centroid method, and were rotated through an angle which gave the expected v-saturation to the Number Series intelligence test. This test is not a very reliable one for children, hence its low communality. It may be seen that the g or general intelligence loadings of the Watts-Vernon and Schonell tests are very high,
[page 51]
though both of them do to some extent measure the same verbal-educational factor as the individual (mechanical) reading and the spelling tests.
[click on the image for a larger version]
Ten tests were inter-correlated among 994 Army recruits; (Number Series, however, was taken by 758 only, Mechanical Assembly by 701 and Abstraction by 233). Four factors were extracted by the centroid method and rotated to yield a g-factor, and verbal-educational, number and mechanical group factors. The reading test is shown here too to depend very largely on g, but its v:ed saturation is as high as that of a test of verbal fluency and of the educational standard or schooling index.
[click on the image for a larger version]