Behavioral Research Surveys Research Has Traditionally Been

¶ … Behavioral Research

Surveys research has traditionally been conducted in three ways: In-person, by mail, and over the telephone. A fourth option has been added to these modes through the survey opportunities available in digital media. This array of modes is a bit like the child's matching game in which the outlier is identified: To whit, "Which one does not belong?" Two of the four survey modes entail real-time interaction between a survey researcher and a respondent; the other two survey modes do not entail this "live" interaction. The presence of a "live" interviewer in telephone surveys research is believed to be an important variable in the non-response data. Typically, telephone survey participants are not offered a non-response option ("don't know"), but in this study, a non-response option was offered to the CATI respondents, since this was an important part of the investigation. Moreover, surveys research via the new digital media is becoming a predominant mode of surveys research. A review of a study follows in which differences in the responses of surveys research participants in a mixed internet-telephone study are described and analyzed.

Presentation. Lugtig, et al. (2011) have written a well-organized and clear article on nonresponse bias and mode effects in mixed-mode surveys research. The abstract provides an intriguing and well-stated description of the research, and lacks only the conventional key words that typically accompany an abstract in major periodicals today. Major headings are typical of APA (Introduction, Methods, Results, Conclusion and discussion, and References). The subheadings convention preferred by the journal make use of lower case letters in the headings, except for the first word, which is capitalized. The authors have followed APA format precisely in their manuscript, or else the submission was edited to conform to APA standards -- there are no evident issues in this regard.

Limitations of the study are thoroughly described and center on two main considerations: the tendency of mode effects to interact and the difficulty of achieving the right set of covariates. A discussion of the statistical modeling limitations is offered in the following subsection. The different mode effects -- which can include recency effects, primacy effects, and interviewer effects -- interact sufficiently so as to erode any certainty about which types of mode effects are in play. The authors found that their results were robust as long as their matching specifications did not attempt to match all the sample respondents. To support replication of their methods, details about steps taken to ensure robust matching processes were provided.

Statistical analysis. The authors have taken a different tack to surveys research sample matching through their use of propensity score matching instead of electing to conduct the more commonly employed procedures for weighting samples to make them equivalent. Propensity modeling enables the surveys researchers to identify unique respondents and note how they are distinguished, rather than having to treat unique respondents as a problem to be solved prior to data analysis. This distinction is clearly related to the purpose of the study, which is to explore the applicability of propensity analysis for mode effects research. Further, propensity score matching shows the same weaknesses as other types of regression methods, the most fundamental one of which is the specification or substitution of covariates. Propensity scores are calculated on the basis of the covariates selected, so it follows that covariate choice is critical. The two experimental groups were identified as: The Computer Assisted Telephone Interview (CATI; n = 2685) and the Web Assisted Personal Interview (WAPI; n = 1347). A third control group of 500 participants was drawn as a stratified sample from the TeamVier access panel of respondents. Various tables and graphs are employed for ease of data interpretability and alignment of data to the study conclusions.

Conclusion