Sample Weights

This section is divided into a description of the procedures used to develop sample weights and a discussion of the practical application of these weights. Before using NLS data in an analysis, the user should consult the practical usage discussion to determine when weighting of data is appropriate. Sample-based weights are designed to reflect the underlying population in the year in which the cohort was initially surveyed. Individual weights are assigned after each interview; these weights produce group estimates that are demographically representative of each cohort's base-year population when used in tabulations. Sampling weights for each respondent can be found on the corresponding public data release. For the 2003 release (Young Women) the cross sectional weights were revised because some respondents who were originally coded as "can't locate" were later found to be deceased.

Base-year sampling weights

Population data derived from the NLS are based on multi-stage ratio estimates. The first step was to assign each sample case a basic weight consisting of the reciprocal of the final probability of selection. This probability reflects the differential sampling by race within each stratum. The base-year weights for all those interviewed were adjusted to account for the overrepresentation of blacks in the sample as well as for persons selected after screening who were not interviewed in the initial survey. This adjustment was made separately for each of:

• Mature Women. 16 groupings based on the four Census regions (Northeast, North Central, South, and West), race (non-black/black), and urban/rural residence.
• Young Women. 24 groupings based on the four Census regions (Northeast, North Central, South, and West), race (non-black/black), and three place of residence groupings (urban, rural farm, and rural non-farm).

In the first stage of ratio weight adjustment, differences at the time of the 1960 Census between the distribution by race and residence of the population as estimated from the sample PSUs and that of total population in each of the four major regions of the country were taken into account. Using 1960 Census data, estimated population totals by race and residence for each region were computed by appropriately weighting the Census counts for PSUs in the sample. Ratios were then computed between these estimates (based on sample PSUs) and the actual population totals for the region as shown by the 1960 Census.

In the second stage ratio adjustment, sample proportions were adjusted to independent current estimates of the civilian noninstitutionalized population by age, sex, and race. These estimates were prepared by carrying forward the most recent Census data (1960) to take account of subsequent aging of the population, mortality, and migration between the United States and other countries (Census Bureau 1966). The adjustment was made by race within three Mature Women age groups and five Young Women age groups.

Sampling weight nonresponse adjustment

Since the initial interview, reductions in sample size have occurred due to noninterviews. To compensate for these losses, the sampling weights of the individuals who were interviewed have been revised. The Mature and Young Women cohort is a panel of individuals into which no new individuals were added after the base year. As a result, all reweighting after the initial survey was calibrated to base-year population parameters. This revision was done in two stages. First, out-of-scope noninterviews in each year were identified by the Census Bureau and eliminated from the sample of noninterviews. This group consisted of individuals who were institutionalized, had died, were members of the armed services, or had moved outside the United States--that is, individuals who were no longer members of the U.S. noninstitutionalized civilian population. (Note: In 2003, an attempt was made to interview some of the institutionalized respondents).

The second stage in the adjustment acknowledges the possible nonrepresentative characteristics of the in-scope interviews. For each survey year, those who are eligible but not interviewed, as well as those who are interviewed, were distributed into:

• Mature Women. 24 nonresponse adjustment cells based on race (non-black/black), length of residence in the United States at first interview (nine or fewer years, ten or more years, N/A), and education (N/A, eight or fewer years, nine to eleven years, twelve or more years) reported in 1967.
• Young Women. 30 nonresponse adjustment cells based on race (non-black/black), length of residence in the United States at first interview (nine or fewer years, ten or more years, N/A) and father's occupation (white collar, service, blue collar, farm, N/A) reported in 1968.

Within each of the cells, the base-year sampling weights of those interviewed were increased by a factor equal to the reciprocal of the reinterview rate (using base-year weights) in that year.

In 1991, NLS staff began investigating the effects of differential nonresponse on sampling weights as then calculated. The original weighting routine was designed to minimize an increase in variance caused by large weights for individuals with certain characteristics. One effect of this original procedure was that certain subsegments of the sample were assigned identical sampling weights. NLS staff adjusted the weights to avoid this problem.

Practical usage

The Mature and Young Women cohorts were based upon stratified, multi-stage random samples with an oversample of blacks. Each case in each interview year was assigned a weight specific to that year. This weight can be interpreted as an estimate of the number of people in the corresponding population that the individual in the sample represents. This section discusses some ramifications of the weights when used for data analysis.

To tabulate characteristics of the sample (i.e., sample means, totals, or proportions) for a single interview year in order to describe the population being represented, it is necessary to weight the observations using the weights provided. For example, to estimate the average hours worked in 1987 by women age 14-24 as of December 31, 1967, researchers would simply use the weighted average of hours worked, where weight is the 1987 sample weight. These weights are approximately correct when used in this way, with item nonresponse possibly generating small errors. Other applications for which users may wish to apply weighting, but for which the application of weights may not produce the intended result, include:

Samples generated by dropping observations with item nonresponses

Users often confine their analysis to subsamples of respondents who provided valid answers to certain questions. In this case, a weighted mean will not represent the entire population, but rather those persons in the population who would have given a valid response to the specified questions. Item nonresponse because of refusals, don't knows, or invalid skips is usually quite small, so the degree to which the weights are incorrect is probably quite small. In the event that item nonresponse constitutes a small proportion of the variables under analysis, population estimates (i.e., weighted sample means, medians, and proportions) would be reasonably accurate. However, population estimates based on data items that have relatively high nonresponse rates, such as family income, may not necessarily be representative of the underlying population of the cohort.

Data from multiple waves

Because the weights are specific to a single wave of the study, and because respondents occasionally missed an interview but were contacted in a subsequent wave, a problem similar to item nonresponse arises when the data are used longitudinally. In addition, the weights for a respondent in different years may occasionally be quite dissimilar, leaving the user uncertain about which weight is appropriate. In principle, if a user wished to apply weights to multiple wave data, weights would have to be recomputed based upon the persons for whom complete data are available. If the sample is limited to respondents interviewed in a terminal or end point year, the weight for that year can be used. Users with a more complex sample selection often can obtain reasonably accurate results by using the base-year weights.

Regression analysis

A common question is whether one should use the provided weights to perform weighted least squares when doing regression analysis. Such a course of action may lead to incorrect estimates. If particular groups follow significantly different regression specifications, the preferred method of analysis is to estimate a separate regression for each group or to use dummy (or indicator) variables to specify group membership. If one wishes to compute the population average effect of, for example, education upon earnings, one may simply compute the weighted average of the regression coefficients obtained for each group, using the sum of the weights for the persons in each group as the weights to be applied to the coefficients. While least squares is an estimator that is linear in the dependent variable, it is nonlinear in explanatory variables, so weighting the observations will generate different results than taking the weighted average of the regression coefficients for the groups. The process of stratifying the sample into groups thought to have different regression coefficients and then testing for equality of coefficients across groups using an F-test is described in most statistics texts.

Researchers unsure of the appropriate grouping may wish to consult a statistician or other person knowledgeable about the data set before specifying the regression model. Note that if subgroups have different regression coefficients, a regression on a random sample of the population would be misspecified.

Custom weighting program

Every Mature and Young Women survey contains a created variable that is the respondent's cross-sectional weight. Using these weights provides a simple method for users to correct the raw data for the effects of over-sampling of blacks and the initial clustering of respondents at the survey's beginning. Unfortunately, while each set of weights provides an accurate adjustment for any single year, none of the weights provide an accurate method of adjusting multiple years' worth of data. Users analyzing more than one year of Mature and Young Women's data should use longitudinal weights, which improve a researchers' ability to accurately calculate summary statistics from multiple years of data.

Users can create longitudinal weights for the Mature and Young Women by going to the Custom Weighting page. To create a set of custom weights, users select the survey years corresponding to their research and pick the "Download" button. The custom weighting program will generate a set of longitudinal weights and open a download dialog box so that users can save the weights to their computer. The resulting file contains two columns of data, with the columns separated by a blank space. The first column is the public identification (ID) number of each respondent. The second column is the weight. If the respondent did not participate in every survey checked off, then the respondent is given a weight of zero. If the respondent did participate, she is given a positive longitudinal weight.

The custom weighting program is an Internet version of the program used to create the cross-sectional weights for the original cohorts since the 1990s. The primary difference between the cross-sectional and longitudinal weighting programs is in how the list of respondents is created. In the cross-sectional case the weighting program is given a list of all people who participated in a particular survey round. In the longitudinal case the weighting program creates a "dummy" survey round where the user specifies who participated and who did not. This "dummy" round is based on the set of surveys selected. It then calculates which respondents participated in every survey round chosen by the researcher and uses that list to generate weights.

The original cohorts weighting is derived from the base year weights via a two-step process. First, all out-of-scope noninterviews, which are respondents who have died, been institutionalized, or moved outside the U.S. are eliminated from the pool of respondents who are classified as noninterviews. Second, those who are in-scope, whether or not they do an interview, are distributed into 24 cells based on race (black/non-black), length of residence at the time of the first interview (nine or less years, ten or more years, or unknown) and education (eight or less years, nine to eleven years, twelve or more years, or unknown).

These cells are then examined to see if the cells have too few respondents. If a cell has too few respondents, it is collapsed with an adjoining cell. Once the optimal number of cells is created, all of the weights associated with respondents in a particular cell are totaled. These totals are then divided to create an adjustment factor. This adjustment factor is then multiplied by each respondent's base year weight, which results in the custom longitudinal weight for a respondent.

Reference

Census Bureau. Current Population Reports. Series P-25, No. 352, November 18, 1966.