1.Measurement Error in Relationship and Marital Status Questions
Nancy Bates
Senior Researcher for Survey Methodology
U.S. Census Bureau
COPAFS Quarterly Meeting
December 2, 2011
2.Research problem: definitions
Societal and legal definition of “marriage” has changed
New terms (e.g., same-sex husbands and wives, partner, civil unions, domestic partnerships)
State-to-state recognition and no federal level recognition
According to 2008 ACS: ~150,000 same-sex spousal couples but…
Only ~32,000 same-sex legally married couples in U.S. in 2008 (Williams Institute)
Why the discrepancy?
2
3.Classification “error”
Are cohabiting gays & lesbians selecting “husband/wife” (even if not married)?
Is primacy effect a factor (‘husband/wife’ is 1st in list, ‘unmarried partner’ is 13th)?
False positives caused by heterosexual married couples
mis-marking gender?
Some combination of all the above?
3
4.Research to date
Interagency Workgroup on Measuring Relationships in Federal Household Surveys (IWMRFHS)
Focus Groups (Bates et al., 2010)
Understand terms commonly used
Interpreted as legal or something else?
Shortcomings of current questions?
Come up with alternative relationship and marital status questions to cognitively test
Cognitive Interviews (DeMaio and Bates, 2011)
Test 2 alternatives of each question among target populations
Select one set for further quantitative testing
5.Methodology: focus groups
18 groups in 8 geographically varied sites (15 with gay couples; 3 straight)
Included legally married, domestic partners, and no legal status
Summary Findings:
Interpreted Qs in context of a federal form - that means legal status
Legally married anywhere = “husband/wife”
Very few selected “husband/wife” or “now married’ if not legally married
Need categories to reflect new legal unions
“Functional equivalence” problem
Most same-sex couples able to make selection in Relationship Q.
Marital status had bigger problems:
No place to indicate committed relationship for many same-sex couples
Consider measuring cohabitation/relationship status and marital status separately
6.Methodology: Cognitive Interviews
40 interviews - included both gays and straights
Married, domestic partnerships (DP)/civil unions (CU), no legal status
Cleveland, Charlotte, Las Vegas, DC metropolitan area
Observed as respondents completed one of two versions
Retrospective cognitive interviews
How and why respondents answered as they did?
Meaning of terms and categories?
Sensitive questions?
Showed alternate versions at end – preference?
7.New Relationship Questions
Version 1
How is this person related to Person 1? Mark (X) ONE box.
[ ] Husband/wife/spouse [ ] Parent-in-law
[ ] Unmarried partner [ ] Son-in-law or daughter-in-law
[ ] Biological son or daughter [ ] Other relative
[ ] Adopted son or daughter [ ] Roomer or boarder
[ ] Stepson or stepdaughter [ ] Housemate or roommate
[ ] Brother or sister [ ] Foster child
[ ] Father or mother [ ] Other nonrelative
[ ] Grandchild
8.New Relationship Questions
Version 2
How is this person related to Person 1? Mark (X) ONE box.
[ ] Opposite-sex husband/wife/spouse [ ] Grandchild
[ ] Same-sex husband/wife/spouse [ ] Parent-in-law
[ ] Opposite-sex unmarried partner [ ] Son-in-law or daughter-in-law
[ ] Same-sex unmarried partner [ ] Other relative
[ ] Biological son or daughter [ ] Roomer or boarder
[ ] Adopted son or daughter [ ] Housemate or roommate
[ ] Stepson or stepdaughter [ ] Foster child
[ ] Brother or sister [ ] Other nonrelative
[ ] Father or mother
9.New Marital Status Questions
Version 1
What is this person’s current marital status? Mark (X) ONE box.
[ ] Now married
[ ] In a registered domestic partnership or civil union
[ ] Widowed
[ ] Divorced
[ ] Separated
[ ] Never married
10.New Marital Status Questions
Version 2
1. What is this person’s current marital status? Mark (X) ONE box.
[ ] Now married - Skip to #4
[ ] Widowed
[ ] Divorced
[ ] Separated
[ ] Never married
2. Is this person currently living in a registered domestic partnership or civil union?
[ ] Yes - skip to #4
[ ] No
11.Last in series: cohabitation question
3. Is this person currently living with a boyfriend/girlfriend or partner
in this household?
[ ] Yes
[ ] No
Allows same-sex couples without legal recognition to indicate relationship status
Allows for identification of sub-families
Currently asked in Current Population Survey
12.Results: Relationship Question
Most survey reports aligned with “true” legal status
Category ordering effect of Version 2 caused misreporting
Version that delineates “same-sex” / “opposite-sex” had most mentions for being potentially sensitive (but was also most preferred)
Recommendation: Further test delineated Version 2
Acts as consistency check to reduce gender misreports among straight married couples
However, new category of “same-sex husband/wife/spouse” could inflate gay couple reports (reorder categories to minimize)
Also, “same-sex / opposite-sex” labels could have negative impact on unit response (need to empirically test)
13.Reactions to Relationship with Same-Sex and Opposite-Sex categories
“This [version with opposite-sex and same-sex labels] actually might, would offend me a little bit, because I think it’s …I feel like we’re kind of just lowering the standards..[as in] OK, we’ve got to conform to everyone. We want to be politically correct”
- married straight female
“...it does show how we grow as a country, that’s what the Census is, showing how we deposit ourselves all over the country and how we define our families”
- married gay man
“[that version] tries too hard…it’s making a point. [that version means] we’re gonna put it out there…that we’re including everybody. But, I get that..it’s fine…we’re in America”
- married straight female
14.Results: Marital Status Question
Most survey reports were consistent with “true” legal status
Inconsistent reports involved “domestic partner/civil union” in some way
Confusion /misunderstanding of “domestic partnership/civil union” (mostly among straights)
Overwhelming preference for embedded Version 1, but…
Recommendation: Further test Version 2 (separate questions)
Misunderstanding of DP/CU terms is widespread
Embedding may inflate DP/CU selection among straight couples
Reduce above by having separate question
Reorder to ask cohabitation first followed by DP/CU question
15.Reactions to DP/CU concept
“two people who have lived together for at least 7 years”
-straight married male
“I might instead put registered DP or CU..if I lived with somebody in a committed relationship [and] we weren’t married. Now I have another option”
-straight married female
“In California, you can do a domestic partnership, which is same-sex marriage” - straight married male
16.Cohabitation Item
When space allows, keep cohabitation question
Re-order to follow Marital Status and precede Domestic Partnership/Civil Union question
17.Recommended: Relationship Q.
How is this person related to Person 1? Mark (X) ONE box.
[ ] Opposite-sex husband/wife/spouse [ ] Grandchild
[ ] Opposite-sex unmarried partner [ ] Parent-in-law
[ ] Same-sex husband/wife/spouse [ ] Son-in-law or daughter-in-law
[ ] Same-sex unmarried partner [ ] Other relative
[ ] Biological son or daughter [ ] Roomer or boarder
[ ] Adopted son or daughter [ ] Housemate or roommate
[ ] Stepson or stepdaughter [ ] Foster child
[ ] Brother or sister [ ] Other nonrelative
[ ] Father or mother
18.Recommended: Marital Status Q.
1. What is this person’s current marital status? Mark (X) ONE box.
[ ] Now married - Skip to #4
[ ] Widowed
[ ] Divorced
[ ] Separated
[ ] Never married
2. Is this person currently living with a boyfriend/girlfriend or partner in this household?
[ ] Yes
[ ] No - skip to #4
3. Is this person currently living in a registered domestic partnership or civil union?
[ ] Yes - skip to #4
[ ] No
19.Next Steps
Recommended questions currently under review of interagency group
Piggybacking on the next ACS forms-length test
Mail only
Small test panel (5,000 mailout or only ~2,000 returns)
Need to test in other modes
Reinterview component would be ideal