McCarthy Scales of Children's Abilities - Verbal Memory

The Verbal Memory subscale of the McCarthy Scales of Children's Abilities was last administered in the NLSY79 Child survey in 1994. This assessment taps a child's short-term memory in response to auditory stimuli. The Verbal Memory subtest selected for use in the NLSY79 Child is only one of six scales that form the complete McCarthy assessment battery. Verbal Memory was administered by first asking the child, age three through six years, to repeat words or sentences said by the interviewer (Parts A and B). Then the child listens to and retells the essential aspects of a short story read aloud by the interviewer (Part C).

Administration of Verbal Memory

From 1986 to 1990, both the word and sentence components as well as the story part of the assessment were administered. In 1992 and 1994, administration was limited to the word/sentence component of the assessment. This means that in 1992 and 1994, only the first two parts (A and B) of Verbal Memory were administered. After 1994, due to cost reasons and concerns about data quality, administration of this assessment was discontinued.

Scoring Verbal Memory

In the first half of the word-sentence component of the assessment (Part A), the score that the child received was contingent on the child repeating a series of words, ideally in the same sequence that they were uttered by the interviewer. In Part B of this first section, the child was scored according to the number of key words that he or she repeated from a sentence read by the interviewer. The combined total score for Parts A and B determined whether the story (Part C) was administered. In Part C, the child was read a story paragraph and then scored on the basis of his or her ability to recall key ideas from that story. National norms are available for this assessment, so children were assigned normed scores based on his or her performance in comparison with a nationally representative sample.

The number of correct responses to the words and sentences on pages 50 and 51 in the 1994 Child Supplement (the last year the assessment was administered) were combined to generate one total raw score. 

As noted in the 1986 through 1990 rounds of data collection, the Verbal Memory assessment included a "Part C" or a "Story" section. Children who received this assessment in 1986-1990 received two scores in each year. Entry into the "Story" was contingent upon receiving a minimum combined score of 8 on Part A plus Part B. The researcher may note that there were a few instances of children entering and receiving a score on Part C who had received an invalid skip score on Part A and Part B. While it may not have been possible to score A and B for various reasons, the available information was sufficient for the scorer to be confident that the A and B score was at least 8. Children who received a valid score of less than 8 on Part A and Part B were automatically assigned a zero on Part C. This explains the considerable heaping at the zero outcome for Part C.

The scoring on Part C is a simple summation of the number of key words/phrases identified correctly from the paragraph on page CS-36 of the 1990 Child Supplement. No proration was attempted for missing responses. The individual items appear on page CS-38 of that supplement. 

Age Eligibility for Verbal Memory

Verbal Memory has typically been completed by children between the ages of three and six, although in 1990, administration was limited to ages four to six. In all child survey years it was only administered to age-eligible children who had not previously (in a prior round) completed the assessment. The precise administration pattern by year is noted in Table 4 in the Child Assessments--Introduction section. 

Norms for Verbal Memory

Appropriate national norms are available in the McCarthy manual (McCarthy, 1972: 205). Thus, percentile and standard scores are available for linking with the raw scores. A total raw score and two normed scores were generated for Part C in 1986 through 1990. From an analytical perspective, the prospective user may note that the distributions of the percentile and standard scores for Part C are somewhat uneven, reflecting the fact that the Part C outcome allowed for only 12 possible responses (0 and 1 through 11) with a major heaping as noted, at the zero category. The fact that the percentile/standard scores assigned to the various raw scores vary by the age of the child helps to smooth the normed pattern somewhat. However, the user is encouraged to examine the pattern of normed responses before proceeding with his or her research. As with all of the assessments in the Child Supplement, the Child Supplement age variable (CSAGE) should be used when stratifying the sample by age of child.

Completion Rates for Verbal Memory 

The 1994 completion rate for Parts A and B was only about 82 percent, below the completion rate for all of the other child-administered assessments. Hispanic children had a completion rate of only 77 percent, substantially below that for other children. Thus, as with some of the other assessments, there is surface evidence that language constraints come into play when evaluating the reliability and potential validity of this assessment. With regard to this assessment, it is important to note that a Spanish translation was not utilized. Since this test measures English language verbal retention, a language bias is likely for at least some children. Hispanic children and children of less educated mothers are heavily over-represented among those who could not be scored in the "invalid response" subset.

Validity of Verbal Memory

While this subscale has a high face validity regarding what it purports to measure, the user should be sensitive to the fact that the scoring of Part C, the story section, undoubtedly includes an element of subjectivity. Interviewers can, in some instances, disagree regarding whether or not a child's specific response was indeed a "correct" or "incorrect" interpretation of an aspect of the story. Also, to some extent, the verbatim verbal responses recorded by the interviewer could, in some instances, be coded in different manners by different interviewers. In order to test this latter premise, NORC had the 1986 verbatim responses for about 400 children independently coded by two coders. There was complete agreement between coders for 92 percent of the respondents.

At a different level, there is also some possibility that the Part A response patterns reflect a lack of precision in the instruction, an ambiguity that also exists in the McCarthy manual. The instructions (for Part A) only ask the child to repeat the words that the interviewer reads to him or her, but do not specify that the words should be repeated in the same sequence. However, in the scoring, the respondent loses a point if the words are repeated out of sequence. Thus, the extent to which the words were repeated in or out of sequence may have been a function of how the instructions were understood, an artifact that could attenuate the reliability of the Part A score.

Additional Information about Verbal Memory

Verbal Memory has been one of the most difficult of the assessments to administer because of the ambiguity involved in determining whether a child does not know an answer or is just shy (see Baker and Mott, 1995 for a discussion of this issue and its impact on the assessment). This is primarily an issue with younger children who had not previously been tested or had not been in a formal school environment. With the introduction of the CAPI administration procedures in 1994, one additional problem became apparent. The number of cases scored "zero" increased substantially, but interviewer comments suggest that many of these cases really should have been "non-completions." This is discussed in detail in Baker and Mott (1995). For the reasons noted above, this assessment should be used cautiously. Additional discussion relating to the reliability and validity of this assessment, as well as how it has been used by other researchers, can be found in the NLSY Child Handbook: 1986-1990 and in The NLSY Children, 1992:Description and Evaluation, both available on the Research/Technical Reports page.