Placement, Wording, and Interviewers: Identifying Correlates of Consent to Link Survey and Administrative Data

Joseph W. Sakshaug, Valerie Tutz, Frauke KreuterResearch 2012 · DOI

TL;DR

This paper investigates how survey design choices and interviewer characteristics influence respondents' consent to link their survey data with administrative records. They found that placing the consent question at the beginning of a telephone interview significantly increased consent rates, while the wording of the consent question had no discernible effect. Interviewers who were personally willing to link their own data were also more successful in obtaining consent from respondents.

What Problem Does This Paper Address?

As survey budgets shrink and the demand for richer data grows, linking survey responses with administrative data is becoming increasingly attractive. This linkage offers cost savings and opens up new avenues for research. However, a significant hurdle is obtaining respondent consent for this data linkage. When respondents refuse, it can lead to missing data and introduce bias into the findings derived from linked datasets. While previous research has identified respondent-level factors (like age or education) that correlate with consent, these are outside the control of survey designers. This paper focuses on modifiable design elements – specifically, the wording and placement of the consent question, and interviewer attributes – to identify strategies that can improve consent rates.

Methods

The study employed a mixed-methods approach combining experimental design and statistical modeling.

  • Wording and Placement Experiments: A telephone survey was conducted where respondents were randomly assigned to different conditions.
  • Wording: One group received a consent question that highlighted the potential benefit of a "shorter interview" (benefit wording), while another received a neutral wording.
  • Placement: Respondents were also randomized to have the consent question presented either at the beginning or the end of the interview. These experiments were partially crossed.
  • Interviewer Survey: Prior to the main data collection, interviewers completed a questionnaire about their demographics, expectations regarding consent, and their personal willingness to consent to various data linkage requests.
  • Statistical Analysis:
    • Bivariate analyses (chi-square tests) were used to compare consent rates between the wording and placement experimental conditions.
  • A mixed-effects logistic regression model with a random intercept was used to analyze the impact of interviewer characteristics on respondent consent, accounting for the clustering of respondents within interviewers.

Datasets

The study utilized data from a German telephone survey conducted from August to October 2011. The sample consisted of 12,000 individuals drawn from German federal databases used in social security administration. A total of 2,400 completed interviews were achieved, with a response rate of 19.4%.

Key Insights and Ideas

  • Placement is Paramount: The location of the consent question within the interview flow is a critical factor. Placing it early seems to leverage a respondent's initial willingness to participate.
  • Wording Matters Less Than Expected: Highlighting potential benefits like a "shorter interview" didn't significantly boost consent rates in this telephone setting. This suggests that for some respondents, the perceived risks or privacy concerns might outweigh perceived benefits.
  • Interviewer's Personal Stance is Key: Interviewers who were personally willing to link their own data were more successful in persuading respondents to consent. This hints at a "walk the talk" effect, where an interviewer's conviction can influence respondent trust and agreement.
  • Experience Can Be a Double-Edged Sword: Surprisingly, more experienced interviewers were less likely to obtain consent. This could be due to various factors, perhaps a more jaded perspective or a tendency to rush through sensitive questions.

Key Results

  • Placement Effect: Placing the consent question at the beginning of the telephone interview resulted in a significantly higher consent rate (95.6%) compared to placing it at the end (86.0%). This is an approximately 10 percentage point difference.
  • Wording Effect: The consent rate for the "benefit" wording (95.5%) was not statistically different from the "neutral" wording (95.6%).
  • Interviewer Willingness: Interviewers' willingness to consent to a range of hypothetical data linkage requests was positively correlated with respondent consent. However, their willingness to consent to the specific linkage requested in the survey was not.
  • Interviewer Expectations: Interviewers' expectations of consent rates did not predict their success in obtaining consent.
  • Interviewer Experience: Interviewers with 37 or more months of experience had a negative effect on linkage consent.
  • Interviewer Variance: Including interviewer covariates reduced the proportion of variance attributable to the interviewer by about half, indicating that interviewer characteristics do play a role.

Limitations and Future Work

The authors acknowledge several limitations:

  • Generalizability: The study was based on German residents meeting specific employment history criteria, which may limit generalizability to other populations or countries.
  • Refusal Conversion: No refusal conversion attempts were made, meaning the findings might not apply to more reluctant respondents.
  • Mode of Data Collection: The findings are specific to telephone surveys. The authors suggest that in web surveys, where break-offs are a concern, mentioning time savings might be more impactful. For face-to-face interviews, the dynamics of rapport building might favor different placement strategies.
  • Interviewer Training: While interviewer willingness to consent is a predictor, it's not feasible to remove interviewers. Future work could focus on providing targeted training and information to interviewers who may struggle with obtaining consent.
  • Differential Mechanisms: The study couldn't explore whether different types of respondents (e.g., cooperative vs. reluctant) respond differently to wording or placement. Future research could use contact history data to tailor consent strategies.

The paper concludes by emphasizing the importance of optimizing linkage consent rates, as data linkage is a valuable and cost-effective supplement to traditional survey data collection.

Summary updated

Abstract

Record linkage is becoming more important as survey budgets are tightening while at the same time demands for more statistical information are rising. Not all respondents consent to linking their survey answers to administrative records, threatening inferences made from linked data sets. So far, several studies have identified respondent-level attributes that are correlated with the likelihood of providing consent (e.g., age, education), but these factors are outside the control of the survey designer. In the present study three factors that are under the control of the survey designer are evaluated to assess whether they impact respondents' likelihood of linkage consent: 1) the wording of the consent question; 2) the placement of the consent question and; 3) interviewer attributes (e.g., attitudes toward data sharing and consent, experience, expectations). Data from an experiment were used to assess the impact of the first two and data from an interviewer survey that was administered prior to the start of data collection are used to examine the third. The results show that in a telephone setting: 1) indicating time savings in the wording of the consent question had no effect on the consent rate; 2) placement of the consent question at the beginning of the questionnaire achieved a higher consent rate than at the end and; 3) interviewers' who themselves would be willing to consent to data linkage requests were more likely to obtain linkage consent from respondents.
BibTeX
@inproceedings{sakshaug2012placement,
  author = {Joseph W. Sakshaug and Valerie Tutz and Frauke Kreuter},
  title = {{Placement, Wording, and Interviewers: Identifying Correlates of Consent to Link Survey and Administrative Data}},
  booktitle = {Research},
  year = {2012},
  doi = {10.18148/srm/2013.v7i2.5395},
}

Loading related content...

Figure