Sample Weights

This section is divided into a discussion of the procedures used to develop sample weights and a section on the practical application of these weights. Before using NLS data in analysis, researchers should consult the practical usage discussion to determine when weighting of data is appropriate. Sample-based weights in each of the NLS cohorts were designed to reflect the underlying population in the year in which each cohort was originally surveyed. Individual case weights were assigned to yearly interviews in such a way as to produce group estimates which are demographically representative of each cohort's base year population when used in tabulations.

Base-year sampling weights

Population data derived from the NLS are based on multi-stage ratio estimates. The first step was to assign each sample case a basic weight consisting of the reciprocal of the final probability of selection. This probability reflects the differential sampling by race within each stratum of the four cohorts.

The base year weights for all those interviewed were adjusted to account for the overrepresentation of blacks in the sample as well as for persons who were not interviewed in the initial survey. This adjustment was made separately for each of eight groupings for the Older Men (based on the four Census regions [Northeast, North Central, South, West] by urban-rural residence) and 24 groupings for the Young Men (based on the four Census regions, race [non-black/black], and three place of residence groupings [urban, rural farm, and rural nonfarm]).

In the first stage of ratio weight adjustment, differences at the time of the 1960 Census between the distribution by race and residence of the population as estimated from the sample PSUs and that of total population in each of the four major regions of the country were taken into account. Using 1960 Census data, estimated population totals by race and residence for each region were computed by appropriately weighting the Census counts for PSUs in the sample. Ratios were then computed between these estimates (based on sample PSUs) and the actual population totals for the region as shown by the 1960 Census.

In the second stage ratio adjustment, sample proportions were adjusted to independent current estimates of the civilian noninstitutionalized population by age, sex, and race. These estimates were prepared by carrying forward the most recent Census data (1960) to take account of subsequent aging of the population, mortality, and migration between the United States and other countries (Census Bureau 1966). The adjustment was made by race within three age groups for the Older Men and in four age groups for the Young Men.

Sampling weight nonresponse adjustment

Subsequent to the initial interview of each cohort, reductions in sample size occurred because of noninterviews. To compensate for these losses, the sampling weights of the interviewed individuals were revised. Each cohort of the NLS consists of a panel of individuals in which no new individuals were permitted to enter after the base year. As a result, all reweighting of the sample after the initial survey round was calibrated to base year population parameters. This revision was done in two stages. First, out-of-scope noninterviews in each of the years were identified by Census and eliminated from the sample of noninterviews. This group consisted of individuals who were institutionalized, who had died, who were members of the armed services, or who had moved outside the United States, i.e., individuals who were no longer members of the noninstitutionalized civilian population of the United States.

The second stage in the adjustment acknowledged the possible nonrepresentative characteristics of the in-scope interviews. For each survey year, those who were eligible but not interviewed, as well as those who were interviewed, were distributed into nonresponse adjustment cells. For the Older Men, there were 24 nonresponse adjustment cells based on 1966 data regarding race (black and non-black), length of time in residence at first interview (nine or fewer years, ten or more years, N/A), and education (N/A, eight or fewer years, nine to eleven years, twelve or more years). The Young Men cohort was divided into 30 nonresponse cells based on 1966 data using the same race and residence variables as above, but with father's occupation (white collar, service, blue collar, farm, N/A) instead of the education variables used with the Older Men. Within each of the cells, the base year sampling weights of those interviewed were increased by a factor equal to the reciprocal of the reinterview rate (using base year weights) in that year.

For the Young Men cohort, the sampling weights of those interviewed were further adjusted to account for the return to the civilian population of men who were in the armed services at the time of initial interview. This final adjustment made use of the first stage estimates described above and independent Census Bureau estimates of the civilian population by selected age categories and race.

Practical usage

The Older and Young Men cohorts were based upon stratified, multi-stage random samples with oversamples of blacks. Each case in each interview year is assigned a weight specific to that year. This weight can be interpreted as an estimate of the number of people in the population of interest that the individual in the sample represents. The following is a discussion of the ramifications of the weights when used for data analysis.

To tabulate characteristics of the sample (sample means, totals, or proportions) for a single interview year in order to describe the population being represented, it is necessary to weight the observations using the weights provided. For example, to estimate the average hours worked in 1976 by men born in 1957 through 1964, researchers would simply use the weighted average of hours worked, where weight is the 1976 sample weight. These weights are approximately correct when used in this way, with item nonresponse possibly generating small errors. Other applications for which users may wish to apply weighting, but for which the application of weights may not correspond to the intended result, include:

Samples generated by dropping observations with item nonresponse

Often users confine their analysis to subsamples for which respondents provided valid answers to certain questions. In this case, a weighted mean will not represent the entire population, but rather those persons in the population who would have given a valid response to the specified questions. Item nonresponse because of refusals, don't knows or invalid skips is usually quite small, so the degree to which the weights are incorrect is probably quite small. In the event that item nonresponse constitutes a small proportion of the variables under analysis, population estimates (i.e., weighted sample means, medians and proportions) would be reasonably accurate. However, population estimates based on data items that have relatively high nonresponse rates, such as family income, may not necessarily be representative of the underlying population of the cohort under analysis.

Data from multiple waves

Because the weights are specific to a single wave of the study, and because respondents occasionally miss an interview but are contacted in a subsequent wave, a problem similar to item nonresponse arises when the data are used longitudinally. In addition, occasionally the weights for a respondent in different years may be quite dissimilar, leaving the user uncertain as to which weight is appropriate. In principle, if a user wished to apply weights to multiple wave data, weights would have to be recomputed based upon the persons for whom complete data are available. If the sample is limited to respondents interviewed in a terminal or end point year, the weights for that year can be used.

Regression analysis

A common question is whether one should use the provided weights to perform weighted least squares when doing regression analysis. Such a course of action may lead to incorrect estimates. If particular groups follow significantly different regression specifications, the preferred method of analysis is to estimate a separate regression for each group or to use dummy (or indicator) variables to specify group membership. If one wishes to compute the population average effect of, for example, education upon earnings, one may simply compute the weighted average of the regression coefficients obtained for each group, using the sum of the weights for the persons in each group as the weights to be applied to the coefficients. While least squares is an estimator that is linear in the dependent variable, it is nonlinear in explanatory variables, and so weighting the observations will generate different results than taking the weighted average of the regression coefficients for the groups. The process of stratifying the sample into groups thought to have different regression coefficients and then testing for equality of coefficients across groups using an F-test is described in most statistics texts.

If one is unsure of the appropriate grouping, one should consult a statistician or other person knowledgeable about the data set before specifying the regression model. Note that if subgroups have different regression coefficients, a regression on a random sample of the population would be misspecified.

Custom weighting program

Every Older and Young Men survey contains a created variable that is the respondent's cross-sectional weight. Using these weights provides a simple method for users to correct the raw data for the effects of over-sampling of blacks and the initial clustering of respondents at the survey's beginning. Unfortunately, while each set of weights provides an accurate adjustment for any single year, none of the weights provide an accurate method of adjusting multiple years' worth of data. Users analyzing more than one year of Older or Young Men's data should use longitudinal weights, which improve a researchers' ability to accurately calculate summary statistics from multiple years of data.

Users can create longitudinal weights for the Older or Young Men by going to the NLS Custom Weights page. To create a set of custom weights, users select the survey years corresponding to their research and pick the "Download" button. The custom weighting program will generate a set of longitudinal weights and open a download dialog box so that users can save the weights to their computer. The resulting file contains two columns of data, with the columns separated by a blank space. The first column is the public identification (ID) number of each respondent. The second column is the weight. If the respondent did not participate in every survey checked off, then the respondent is given a weight of zero. If the respondent did participate, he is given a positive longitudinal weight.

The custom weighting program is an Internet version of the program used to create the cross-sectional weights for the original cohorts since the 1990s. The primary difference between the cross-sectional and longitudinal weighting programs is in how the list of respondents is created. In the cross-sectional case the weighting program is given a list of all people who participated in a particular survey round. In the longitudinal case the weighting program creates a "dummy" survey round where the user specifies who participated and who did not. This "dummy" round is based on the set of surveys selected. It then calculates which respondents participated in every survey round chosen by the researcher and uses that list to generate weights.

The original cohorts weighting is derived from the base year weights via a two-step process. First, all out-of-scope noninterviews, which are respondents who have died, been institutionalized, or moved outside the U.S. are eliminated from the pool of respondents who are classified as noninterviews. Second, those who are in-scope, whether or not they do an interview, are distributed into 24 cells based on race (black/non-black), length of residence at the time of the first interview (nine or less years, ten or more years, or unknown) and education (eight or less years, nine to eleven years, twelve or more years, or unknown).

These cells are then examined to see if the cells have too few respondents. If a cell has too few respondents, it is collapsed with an adjoining cell. Once the optimal number of cells is created, all of the weights associated with respondents in a particular cell are totaled. These totals are then divided to create an adjustment factor. This adjustment factor is then multiplied by each respondent's base year weight, which results in the custom longitudinal weight for a respondent.

Reference

Census Bureau. Current Population Reports. Series P-25, No. 352, November 18, 1966.