Methodology › 2016 Methodology

2016 Methodology Download PDF

The 2016 American Values Atlas (AVA) is a project of PRRI. Results for all demographic, religious affiliation, and political affiliation questions were based on 101,438 bilingual telephone interviews (including 60,355 cell phone interviews) conducted between January 6, 2016 and January 10, 2017 by professional interviewers under the direction of SSRS. Results for questions on specific issues (e.g. immigration and LGBT issues) are based on a subset of 40,509 telephone interviews (including 24,266 cell phone interviews) conducted between May 18, 2016 and January 10, 2017. The AVA was made possible by generous grants from The Ford Foundation, The Carnegie Corporation of New York, the Arcus Foundation, the Gill Foundation and the Nathan Cummings Foundation.

Throughout 2016, at least 1,000 interviews were completed each week, with about 600 interviews conducted among respondents on their cell phones. Each week, interviewing occurred over a five-day period, from Wednesday through Sunday or from Thursday through Monday. The selection of respondents within households was accomplished by randomly requesting to speak with the youngest adult male or female currently living in the household.

Data collection was based on stratified, single-stage, random-digit-dialing (RDD) of landline telephone households and randomly generated cell phone numbers. The sample was designed to represent the total U.S. adult population from all 50 states, including Hawaii and Alaska. The landline and cell phone samples were provided by Marketing Systems Group.

The weighting was accomplished in two separate stages. The first stage of weighting corrects for different probabilities of selection associated with the number of adults in each household and each respondent’s telephone usage patterns. In the second stage, sample demographics were balanced to match target population parameters for gender, age, education, race and Hispanic ethnicity, region (U.S. Census definitions), population density, and telephone usage. The population density parameter was derived from 2010 Census data. The telephone usage parameter came from an analysis of the July-December 2015 National Health Interview Survey. All other weighting parameters were derived from an analysis of the U.S. Census Bureau’s March 2016 Current Population Survey.

The sample weighting was accomplished using iterative proportional fitting (IFP), a process that simultaneously balances the distributions of all variables. Weights are trimmed so that they do not exceed 4.0 or fall below 0.25 to prevent individual interviews from having too much influence on the final results. The use of these weights in statistical analysis ensures that the demographic characteristics of the sample closely approximate the demographic characteristics of the target populations.

The table below shows the sample sizes for all 50 states and 30 metro areas for both sample and issue subsample. The margin of error for total sample is +/- 0.4 percentage points at the 95% level of confidence. The margin of error for issue subsample is +/- 0.6 percentage points at the 95% level of confidence. The design effect for both total sample and the issue subsample is 1.4. In addition to sampling error, surveys may also be subject to error or bias due to question wording, context, and order effects.

Sample Sizes for States and Metro Areas

State Total Sample Issue Subsample
United States 101,438 40,509
Alabama 1,485 562
Alaska 573 264
Arizona 2,042 868
Arkansas 1,008 404
California 9,640 3,909
Colorado 1,657 679
Connecticut 1,073 412
Delaware 302 120
Florida 6,076 2,371
Georgia 2,928 1,129
Hawaii 438 235
Idaho 609 262
Illinois 3,587 1,426
Indiana 2,288 894
Iowa 1,325 541
Kansas 1,091 424
Kentucky 1,463 558
Louisiana 1,410 539
Maine 594 260
Maryland 1,727 669
Massachusetts 1,952 812
Michigan 2,997 1,134
Minnesota 2,060 834
Mississippi 833 321
Missouri 2,171 893
State Total Sample Issue Subsample
Montana 524 212
Nebraska 747 308
Nevada 977 427
New Hampshire 432 146
New Jersey 2,779 1,121
New Mexico 726 297
New York 7,072 2,837
North Carolina 3,544 1,373
North Dakota 331 149
Ohio 4,074 1,604
Oklahoma 1,154 448
Oregon 1,648 687
Pennsylvania 4,610 1,757
Rhode Island 369 151
South Carolina 1,636 637
South Dakota 367 159
Tennessee 2,139 882
Texas 6,956 2,844
Utah 1,056 446
Vermont 313 113
Virginia 2,862 1,116
Washington 2,264 917
West Virginia 784 309
Wisconsin 2,213 836
Wyoming 244 102
Metro Area Total Sample Issue Subsample
United States 101,439 40,509
Atlanta 1,590 622
Boston 1,346 529
Charlotte 766 308
Chicago 2,439 939
Cincinnati 728 299
Cleveland 673 270
Columbus 662 260
Dallas 1,756 746
Denver 771 303
Detroit 1,135 424
Houston 1,476 603
Indianapolis 603 230
Kansas City 706 280
Las Vegas 687 295
Los Angeles 3,000 1,194
Miami 1,552 605
Milwaukee 565 200
Minneapolis-St. Paul 1,217 489
Nashville 538 217
New York 6,590 2,644
Orlando 659 255
Philadelphia 1,884 724
Phoenix 1,244 522
Pittsburgh 947 352
Portland 845 357
San Francisco 1,143 465
Seattle 976 382
St. Louis 875 367
Tampa-St. Petersburg 925 346
Washington, DC 1,865 731