Item Nonresponse within Problem Sections
Item Nonresponse within Problem Sections
How much missing data are associated with particular questions? This section provides readers with an in-depth view of the questions within survey sections having a high amount of missing data. Like the previous parts, this section provides tables for each of the selected survey years. The first table (Table 1) examines questions from the 1979 survey's "Work Experience" section. This section has more missing data (14.5 percent) than any other 1979 survey section. The second set of tables (Tables 2 through 6) examines the most problematic section of the 1984 survey, "Fertility and Abortion." The third set of tables (Tables 7 and 8) examines the most problematic 1989 survey section, "Income and Assets." Since the 1994 "Income and Asset" section again ranked first in missing data, the next set of tables (Tables 9 and 10) substitutes the "Drug and Alcohol Use Supplements," given the high degree of research interest in understanding nonresponse in these sections. Table 11 highlights nonresponse in 1998 in the Marital History section. Table (12) tracks nonresponse problems in the over-40 health section.
To ensure the sets of tables are not overwhelming, all sections that could be naturally divided are split (Fertility, for instance). Additionally, only the most important question or questions with high rates of nonresponse are shown. Table 1, which examines the amount of missing data in the 1979 survey, shows the highest amount of missing data are associated with a pair of retrospective questions that asked respondents to remember what happened two years earlier. Interviewers incorrectly skipped slightly less than 1,750 respondents over R01150., weeks worked in 1977, and R01153., hours worked per week in 1977. Examining the 1979 questionnaire shows that these questions appear at the bottom of a page. Prior to these questions is a fairly complicated half page of instructions and questions that the interviewer must read, understand, and partially speak. It seems likely that many interviewers did not understand the instructions and skipped to the next page.
Table 1. Amount of Missing Data Per Question in the Work Experience Section, 1979 Survey
Reference # | Variable Title | Invalid Skip | Don't Know | Refusal |
R01150. | Weeks Work in 1977 | 1735 | 11 | 1 |
R01151. | Weeks Work in 1976 | 418 | 18 | 1 |
R01152. | Weeks Work in 1975 | 240 | 11 | 0 |
R01153. | Hours/Week Work in 1977 | 1749 | 13 | 0 |
R01154. | Hours/Week Work in 1976 | 459 | 16 | 0 |
R01165. | Industry of 1st Job after School | 628 | 4 | 1 |
R01166. | Occupation at 1st Job after School | 627 | 3 | 1 |
R01167. | Hours/Week Work at 1st Job after School | 631 | 6 | 1 |
R01168. | Hours/Day at 1st Job after School | 632 | 6 | 1 |
R01169. | Rate of Pay at 1st Job after School | 632 | 32 | 2 |
Tables 2-6, which examine the "Fertility" section, show a much lower number of invalid skips in all parts except in the abortion questions. While invalid skips do not reach the level seen in Table 1, on average 190 female respondents were not asked each abortion question (190 is an average from all abortion questions, not just those shown in the tables). The table also shows a number of other trends. First, respondents have higher levels of don't know answers the more precise the question being asked. For example, in Table 2, when males were asked the date of birth of their first child, only one did not know the year, three did not know the month and 10 did not know the day. This phenomena is most clearly seen in Table 5, which shows the year and month of the respondent's first sexual encounter. Only 43 respondents did not know the year, but 1,410 respondents did not know the month. This problem with dates is also seen in the abortion data where only four respondents did not know the year when they had their first abortion, but 13 did not know the month.
Refusal rates in the "Fertility" section are quite low except for a number of key questions. Asking the number of times they had sex in the last month elicited high rates of refusal for males and females. This question elicited 167 male and 135 female refusals. Interestingly, most individuals were willing to answer if they ever had sex since only 45 males and 54 females refused to answer these questions. Birth control questions did not have exceptionally high rates of refusal. Seventeen female respondents and no males refused to answer the birth control questions. Table 6 shows that 28 females refused to answer if they ever had an abortion and 28 more refused to state if they dropped out of school before they terminated the pregnancy.
Table 2. Amount of Missing Data Per Question in Male Fertility Section, 1984 Survey
Reference # | Variable Title | Invalid Skip | Don't Know | Refusal |
R13017. | Ever Had Any Children | 0 | 3 | 0 |
R13019. | Month Birth Child#1 Born | 41 | 3 | 0 |
R13020 | Day Birth Child #1 Born | 45 | 10 | 0 |
R13021. | Year Birth Child#1 Born | 39 | 1 | 0 |
R13022. | Sex of Child#1 Born | 3 | 0 | 0 |
R13115. | Total #Children Expect to Have | 12 | 45 | 3 |
R13117. | #Years Expect Have 1st/Next Child | 22 | 120 | 0 |
R13118. | Had Any Children/Expecting | 0 | 7 | 0 |
R13119. | Current Pregnancy Planned | 131 | 0 | 0 |
R13121. | Ever Had Sexual Intercourse | 12 | 0 | 45 |
R13122. | Age @First Sexual Intercourse | 28 | 19 | 23 |
R13123. | #Times Sexual Intercourse Past Month | 11 | 68 | 167 |
R13124. | Is Partner Now Pregnant | 0 | 1 | 0 |
R13125. | Use Any Birth Control During Last Month | 15 | 2 | 0 |
R13126. | #Times Try Prevent Pregnancy | 65 | 0 | 0 |
R13127.-R13141. | Method of Birth Control | 16 | 0 | 0 |
R13142. | Ever Have a Sex Education Course | 10 | 0 | 12 |
R13148. | Month Took Sex-Ed Course | 73 | 564 | 0 |
R13149. | Year Took Sex-Ed Course | 36 | 58 | 0 |
R13150. | Time When Pregnancy Most Likely | 19 | 1480 | 20 |
Table 3. Amount of Missing Data Per Question in Female Fertility Section, 1984 Survey
Reference # | Variable Title | Invalid Skip | Don't Know | Refusal |
R13191. | #Pregnancies | 8 | 0 | 0 |
R13251. | Use Any Birth Control before Preg#1 | 18 | 0 | 1 |
R13254. | Want Be Pregnant before Preg#1 | 20 | 0 | 0 |
R13255. | Husband/Partner Want Preg#1 | 19 | 20 | 0 |
R13283. | Get Prenatal Care Preg#1 | 57 | 0 | 0 |
R13286. | Frequency Alcohol Use Preg#1 | 58 | 0 | 0 |
R13288. | #Cigarettes Smoked Preg#1 | 56 | 0 | 0 |
R13297. | X-Rays Taken Preg#1 | 57 | 0 | 0 |
R13302. | Sonogram Preg#1 | 57 | 6 | 0 |
R13358. | Amniocentesis Preg#1 | 57 | 0 | 0 |
R13411. | Took Vitamins Preg#1 | 57 | 0 | 0 |
R13443. | C-Section Child#1 Born | 52 | 0 | 0 |
R13445. | Weight at Delivery, Preg#1 | 53 | 5 | 1 |
R13446. | Weight before Preg#1 | 51 | 5 | 1 |
R13449. | Length Child#1 Born at Birth | 53 | 20 | 0 |
R13667. | Weight of Child#1 @Birth Lbs | 25 | 6 | 0 |
Table 4. Amount of Missing Data Per Question in Feeding Part of Fertility Section, 1984 Survey
Reference # | Variable Title | Invalid Skip | Don't Know | Refusal |
R13670. | Child#1 Breastfed | 27 | 0 | 0 |
R13672. | Month Age Child#1 Breast Fed Ended | 27 | 1 | 0 |
R13674. | Month Age Child#1 Formula Fed | 38 | 3 | 0 |
R13693. | Wk Age Child#1 Formula Fed Ended | 57 | 0 | 0 |
R13694. | Month Age Child#1 Formula Fed Ended | 57 | 6 | 0 |
R13696. | Months Age Child#1 - Cow's Milk | 81 | 10 | 0 |
R13698. | Months Age Child#1 - Solid Food | 86 | 10 | 0 |
Tables 7 and 8 examine the "Income and Assets" section of the 1989 survey. While invalid skips are relatively rare in this section, refusals and don't know answers are fairly prevalent. The question with the highest amount of missing income data is R29822., which asks how much income was earned by other adults living in the household who were related to the respondent. While the previous questions showed that most respondents knew the type of income received by these family members, 958 could not come up with a specific amount. The second most problematic question with 11 invalid skips, 155 don't knows, and 113 refusals was R29714., which asked the respondent how much they earned from wages, salary, and tips.
Table 5. Amount of Missing Data Per Question in Child Part of Fertility Section, 1984 Survey
Reference # | Variable Title | Invalid Skip | Don't Know | Refusal |
R13791. | Age Had 1st Menstrual Period | 8 | 14 | 22 |
R13792. | Year 1st Menstrual Period | 0 | 7 | 0 |
R13793. | Month Had 1st Menstrual Period | 17 | 2207 | 1 |
R13794. | R Ever Been Pregnant | 0 | 1 | 0 |
R13795. | Ever Had Sexual Intercourse | 4 | 0 | 54 |
R13796. | Age First Sexual Intercourse | 5 | 26 | 78 |
R13797. | Year 1st Sexual Intercourse | 0 | 43 | 66 |
R13798. | Month Sexual Intercourse 1st Time | 19 | 1410 | 75 |
R13799. | #Times Sexual Intercourse Past Month | 9 | 104 | 135 |
R13802. | #Times Try Prevent Pregnant Past Month | 17 | 0 | 2 |
Table 6. Amount of Missing Data Per Question in Abortion Questions of Fertility Section, 1984 Survey
Reference # | Variable Title | Invalid Skip | Don't Know | Refusal |
R13827. | Ever Had An Abortion | 135 | 0 | 28 |
R13828. | # of Abortions | 143 | 0 | 0 |
R13830. | Year of 1st Reported Abortion | 196 | 4 | 0 |
R13837. | Drop out School #1 Pregnant | 155 | 0 | 28 |
R13839. | Year Left School 1st Time Pregnant | 164 | 0 | 0 |
R13841. | Year Return School Time#1 after Pregnant | 258 | 0 | 0 |
Other questions with high numbers of don't knows are R29813., which asked about the amount of money received from other sources like interest and dividends, R29825., which asks about a partner's income, and R29827., which asks the number of exemptions used when filing a Federal tax return.
The asset table (Table 8) also shows invalid skips are rare but don't know and refusal rates are not. Surprisingly, one of the questions with the highest amount of missing data (315 missing answers) asks, "how much is your car worth (R29852.)?" Another question missing many observations asks the amount of the respondent's savings (R29835.). While the car worth question primarily elicits don't knows, the savings question resulted in 160 refusals. Three other questions elicited high numbers of don't knows: value of stocks and bonds (R29837.) - 219 don't knows; amount taken out of savings last year (R29842.) - 222 don't knows; and the market value of other items such as jewelry (R29854.) - 151 don't knows.
Table 7. Amount of Missing Data Per Question in Income Section, 1989 Survey
Reference # | Variable Title | Invalid Skip | Don't Know | Refusal |
R29714. | Amount Rec from Wages/Salary/Tips | 11 | 155 | 113 |
R29715. | In 1988 Receive Income from Own Business | 1 | 0 | 11 |
R29717. | How Much Did R Receive after Expenses | 6 | 49 | 23 |
R29732. | Amount Rec'd Per Week from Unemployment | 0 | 5 | 1 |
R29736. | Amount Sp Rec'd 1988 from Wages | 16 | 17 | 70 |
R29754. | How Much Did Sp Receive from Unemployment | 8 | 12 | 0 |
R29758. | R/Spouse Rec'd Money for Child Support | 1 | 1 | 10 |
R29759. | Amount R/Spouse Rec'd Child Support | 2 | 14 | 2 |
R29760. | R/Spouse Rec'd AFDC Payments | 0 | 4 | 9 |
R29774. | R/Spouse Rec'd Food Stamps | 0 | 2 | 10 |
R29788. | R/Spouse Rec'd SSI/Public Assistance | 0 | 4 | 9 |
R29808. | Rec'd Veteran Benefits | 1 | 1 | 10 |
R29812. | R/Spouse Rec'd Money from Oth So | 0 | 2 | 16 |
R29822. | Income Rec'd by Adults Related To R | 7 | 958 | 8 |
R29825. | Total Income Rec'd before Deduct | 2 | 200 | 4 |
R29826. | Sp File Federal Income Tax R | 0 | 2 | 13 |
R29827. | R'S Filing Status on Federal Ret | 11 | 8 | 2 |
R29828. | Exemptions Filed on 1988 Federal Tax | 62 | 92 | 3 |
Table 8. Amount of Missing Data Per Question in Asset Section, 1989 Survey
Reference # | Variable Title | Invalid Skip | Don't Know | Refusal |
R29831. | Amount Property Selling for on Today | 5 | 53 | 10 |
R29832. | Amount R Owes on Property | 4 | 85 | 25 |
R29833. | Amount Other Debt R Owes on Property | 12 | 26 | 27 |
R29835. | Amount of Savings | 7 | 166 | 160 |
R29837. | Current Market Value of Stocks | 2 | 219 | 23 |
R29838. | R/Spouse Have Rights to Estate | 2 | 3 | 18 |
R29839. | Total Value of Estate | 3 | 90 | 6 |
R29840. | Put Money in/out of Savings | 1 | 3 | 28 |
R29841. | How Much More Money Put in | 6 | 110 | 53 |
R29842. | How Much More Money Take out | 5 | 222 | 21 |
R29843. | R Have Business Investment | 0 | 1 | 12 |
R29844. | R Have Investment in a Farm | 4 | 0 | 0 |
R29847. | Total Market Value of Business | 4 | 75 | 10 |
R29848. | Total Amount of Business Debt | 1 | 55 | 8 |
R29851. | How Much Does R Owe on Vehicle | 0 | 56 | 17 |
R29852. | Amount Vehicle Sells for Today | 11 | 293 | 11 |
R29854. | Market Value of Other Items | 5 | 151 | 25 |
R29856. | Total Amount R Owes | 1 | 73 | 13 |
Table 9 and 10 examine the drug and alcohol use supplements in the 1994 survey. In these CAPI modules, there are no invalid skips. Interestingly, there are extremely low refusal and don't know rates within the "Alcohol" section (Table 9). The question with the highest refusals (nine respondents) asks if the individual had a drink since the 1989 interview. The typical question in the "Alcohol" section received only two refusals. Don't know rates are also low. The maximum number of don't knows at nine occurs in R49803., which asks if the respondent needs to drink more alcohol now in order to get drunk. On average, the "Alcohol" section records only 1.5 don't knows per question.
Table 9. Amount of Missing Data Per Question in Alcohol Use Section, 1994 Survey
Reference # | Variable Title | Invalid Skip | Don't Know | Refusal |
R49791. | R Had Drink of Alcohol since 1989 | 0 | 3 | 9 |
R49792. | Had Alcoholic Beverage in Last 30 | 0 | 0 | 5 |
R49793. | Times Had 6/More Drinks Last | 0 | 0 | 1 |
R49794. | How Many of Last 30 Days Drank A | 0 | 6 | 2 |
R49795. | No. of Drinks on Avg. Day When R | 0 | 8 | 3 |
R49803. | Need More to Get Drunk Than Before | 0 | 9 | 0 |
R49808. | Arrested, in Police Trouble | 0 | 0 | 3 |
R49809. | Drink More Than Before | 0 | 4 | 3 |
These low numbers of refusals and don't knows are not seen in Table 10, which examines the "Drug Use" section. On average, the typical question in this supplement elicited 23 don't knows and 48 refusals. Readers should understand that this supplement was generally filled in directly by the respondent, not by the interviewer. To provide respondents with practice using a computer, the questionnaire asked them two practice questions not related to drug use. Refusal rates are even high for these two test questions, which ask how many more children the respondent expects to have and what type of entertainment, such as movies, concerts, or plays, the respondent went to last year.
The highest number of refusals (119) occurs in R50532., which asks the age the respondent first used marijuana. The second largest number of refusals occurs in a similar question, R50536., which asks the age of first cocaine use. These same questions have very high don't know responses (113 marijuana and 48 cocaine). One other question with a very high don't know rate is R50525., which asks if the respondent ever smoked cigarettes daily. Almost 80 individuals did not know the answer to this question. Given that the question wording is straightforward, it is likely a number of respondents are using don't know as a polite way of refusing to answer the question.
Table 10. Amount of Missing Data Per Question in Drug Use Section, 1994 Survey
Reference # | Variable Title | Invalid Skip | Don't Know | Refusal |
R50524. | R Smoked at Least 100 Cigrtts in Life? | 0 | 24 | 38 |
R50525. | R Ever Smoked Daily? | 0 | 79 | 49 |
R50526. | Age When R 1st Started Smoking Daily? | 0 | 33 | 12 |
R50531. | Total Occasion R Use Marijuana | 0 | 33 | 89 |
R50532. | Age 1st Time Used Marijuana | 0 | 113 | 119 |
R50533. | Most Recent Time Used Marijuana | 0 | 35 | 89 |
R50535. | How Many Occasions Used Cocaine | 0 | 19 | 86 |
R50536. | Age 1st Time Used Cocaine | 0 | 48 | 103 |
R50537. | Most Recent Time Used Cocaine | 0 | 15 | 78 |
R50539. | How Many Occasions Used Crack | 0 | 15 | 77 |
R50540. | Age 1st Time Used Crack | 0 | 33 | 82 |
R50541. | Most Recent Time Used Crack | 0 | 16 | 74 |
R50553. | R Used Heroin w/o Doctor's Instr | 0 | 9 | 53 |
The top ten questions show that a large number of respondents (ranging from 119 to 181 respondents, depending on the question) have difficulty with questions asking them about their spouse's rate and amount of pay, hours worked and weeks worked. In addition, questions which ask details about a spouse's previous marriage are also quite difficult for many respondents to answer.
Table 11. Amount of Missing Data Per Question in Marital History Section, 1998 Survey
Reference # | Variable Title | Invalid Skip | Don't Know | Refusal |
R58067. | Rate of Pay for Spouse Main Job (Time Unit) | 0 | 181 | 49 |
R58204. | Age of Spouse at 1st Marriage | 0 | 213 | 2 |
R58125. | Spouse's Weekly Earnings at Main Job | 0 | 159 | 29 |
R58068. | Spouse Receive Overtime at Main Job | 0 | 151 | 26 |
R58127. | Estimate Spouse's Weekly Earning Main Job | 0 | 149 | 26 |
R58178. | House Spouse Works Per Week Usually | 0 | 170 | 1 |
R58177. | Number of Weeks Worked by Spouse in Last Year | 0 | 140 | 24 |
R58179. | Number Weeks Not Working by Spouse Last Year | 0 | 130 | 24 |
R58176. | Spouse Hourly Rate of Pay | 0 | 119 | 28 |
R58208. | Duration of Spouse's Previous Marriage? | 0 | 109 | 16 |
Table 12 examines the top questions with missing data problems from the health section in 2004. In this table, reference numbers starting with "R" are for questions asked of all respondents in the survey, while reference numbers starting with "H" represent questions in the "over 40 health module." This module was designed to provide researchers with more information about the health of the respondent when they turned 40 years old and is asked of respondents in the first interview after they turn 40.
While other data from the survey show that many people know if they are covered by health insurance, Table 12 reveals that many do not know details about this coverage. For example, one question with a large number of don't knows is R83036., which asks if the respondent's health insurance plan is an HMO, a preferred provider plan (PPO) or a network of affiliated doctors. This question had 428 missing responses out of 6,175 total responses (a 7% missing response rate). Other questions with high don't know rates ask if the respondent's children are covered by health insurance. The health question with the highest refusal rate asks the respondent how much they weigh, with 114 people refusing to divulge the number. Finally, in the 40+ health module a number of NLSY79 respondents have difficulty answering questions about the health and life status of their biological father. This is not surprising given a small but significant number of respondents stated in the past that they have never met their biological father.
Table 12. Amount of Missing Data Per Question in Health Section, 2004 Survey
Reference # | Variable Title | Invalid Skip | Don't Know | Refusal |
R83036. | Primary Insurance Plan HMO, Network, PPO | 0 | 426 | 2 |
R83037. | Is Primary Plan a PPO? | 0 | 388 | 2 |
R83070. | Children Have Health/Hospitalization Plan? | 0 | 328 | 15 |
R83038. | R's Primary Plan Need Authorization? | 0 | 301 | 0 |
H00015. | Date Most Recent General Physical Exam | 0 | 189 | 0 |
R82983. | How Much Does R Weigh? | 0 | 50 | 114 |
H00014. | Ever Had A General Physical Exam? | 0 | 147 | 2 |
H00017. | Cause Of Biological Dads Death | 0 | 133 | 10 |
H00019. | Bio Dad Have Major Health Problems? | 0 | 134 | 8 |
R82982. | Since What Date R Had This Health Limit | 0 | 120 | 0 |
R82992. | Length Light Moderate Activities 10 Min | 0 | 105 | 5 |
H00047. | Date Hypertension Diagnosed | 0 | 91 | 0 |
H00016. | Is R's Biological Dad Living? | 0 | 83 | 4 |
R82989. | Frequency of Light Mod Exercise 10 > Min | 0 | 75 | 6 |
H00018. | Age Of Biological Dad At Death | 0 | 68 | 1 |
H02445. | Date Most Recent Visit to Health Professional | 0 | 52 | 11 |
H00012. | R Ever Visit Health Care Professional? | 0 | 58 | 0 |
R83042. | Spouse Have Health/Hospital Plan | 0 | 32 | 24 |
R83048. | Spouse Employer Pay All Health Plan Cost? | 0 | 49 | 2 |
Note: Reference numbers that begin with the letter H are variables that are combined from different years of the over-40 health module. Researchers wanting to see the results from just the 2004 survey should use variable H00002.00, which is titled "Source Year for 40+ Health Module Data." Use this variable to select just those cases which answered the questions in 2004. |
References
Mott, Frank L. "Patterning of Child Assessment Completion Rates in the NLSY: 1986-1996." CHRR, The Ohio State University, 1998.
Mott, Frank L. "The Patterning of Female Teenage Sexual Behaviors and Attitudes." CHRR, The Ohio State University, 1994.
Mott, Frank L. "Fertility-Related Data in the 1982 National Longitudinal Surveys of Work Experience of Youth: An Evaluation of Data Quality and Some Preliminary Analytical Results." CHRR, The Ohio State University, 1983.
Olsen, Randall J. "The Effects of Computer Assisted Interviewing on Data Quality." CHRR, The Ohio State University, 1992.