Tool Development and Testing

¶ … Testing

Developing and Testing Data Collection Tools

In research and planning, data collections tools are essential because they are the media that bridges the researcher to the target respondents or groups of the study. These tools serve as "building-blocks" that enables the researcher to visualize the outcome of his/her study: how the tool aligns with the study's objectives and how it generates data and information that will be used for analysis later. It is critical, then, to establish how faithful and appropriate the tools are to the objectives of the research at hand. To achieve this goal, it is therefore imperative for a researcher to develop and test data collection tools that will be used in the study.

While developing a data collection tool, the goal is to maintain fidelity of the research objectives. Questions that will be asked should answer the objectives, but they must be also organized coherently and must flow seamlessly. These considerations are important especially when doing surveys, whether self-accomplished or face-to-face. An organized tool and seamless flow of questions are created largely for the benefit of the respondent, whom the researcher would get invaluable information from. Organization and flow of questions establishes a start, middle, and an end to the questionnaire or guide (tool) being used for the study. For the researcher, the start and end are just as critical as the middle part, which contains the core questions critical to answering the study objectives. For the respondents, the beginning and end of a questionnaire or interview enables him/her to warm-up to the line of questioning that is about to happen, and would also guide him/her towards the end as the interview/survey is coming to a close (Reisman et. al, 2007:33-35).

Integral to tool development is its testing to different factors that could affect data collection. Reliability testing ensures that the results or data collected can be consistently used or applied to different population or respondent groups being targeted. Validity, meanwhile, helps the researcher determine if data or information collected (i.e., responses from interviews or surveys) are authentic and therefore valid for data testing and analysis. Other factors also come into play when doing tool testing, including respondent bias and enumerator or assessor bias, both of which must be considered during testing as possible sources of error during data analysis. Respondent bias could happen when a respondent tend to respond differently from the rest of the respondent pool; this could affect the results of the study as the respondent becomes an "outlier" in the analysis. It is also possible that field enumerators or researchers gathering the data would record information differently or mistakenly record information due to confusion or fatigue in fieldwork. These factors must be taken into account during the testing of the tool and data analysis.