Reliability and Validity Trochim (2007) Examines Validity

Reliability and Validity

Trochim (2007) examines validity and reliability in the context of arriving at measures for constructs that firstly measure what they purport to measure. Secondly, the measures do what they purport to do in a consistent manner so that the researcher can have confidence in the measurement and hence the research project is not compromised. The elements of validity and reliability are pivotal concerns to research. Research thrives in a climate where these are dominant and progressive. Without effective mechanisms to achieve these objectives, the entire research process and the products of research become suspect.

The six types of validity examined by the author are separated into two different components of construct validity. Construct validity is a measure of the success of process of turning constructs into measures. This process identified by the author as operationalization requires that the measures and their underlying construct have a close affinity. The closer the affinity the more valid the measure is (Kennedy, 1984). Thus when the researcher is measuring deviance does the measures or questions used to examine deviance truly capture deviant behavior. Consequently, the author divides the six into two groups those that deal with translation validity, face validity and content validity. Those that address criterion validity, namely predictive validity, concurrent validity, convergent validity, and discriminant validity follow these.

Face validity is a very subjective assessment of the operationalization of the construct. Face validity is an assessment of the accuracy of the measure based on the knowledge available to the researcher and their examination of the measure by looking at it. Face validity can be bolstered by having experts in the field examine the measure; this process however is still highly subjective. Face validity is useful for examining measures to assess juvenile delinquency, by looking at the questions used to determine delinquent behavior the validity of the measure is established. For content validity, the researcher compares the operationalization of the construct to some existing content elements for the construct. This approach allows the researcher to compile a listing of the elements that should be part of a construct and then subsequently examine the operationalization for its consistency with the identified elements. An example of this is the designation of juvenile offenders; and what criteria should determine who is a juvenile offender.

For predictive validity, the measures are examined for its success in predicting something it is theoretically supposed to predict. A high correlation between the measure and its theoretically predicted companion demonstrates predictive validity. If a measure is developed to identify potential offenders, the measure should be positively correlated with socioeconomic status since many offenders come from low socioeconomic backgrounds. Concurrent validity is the ability of a measure to discriminate successfully between groups that it is theoretically constructed to do. A good example of concurrent validity would be the ability to distinguish between different types of sex offenders.

Convergent validity determines whether an operationalization is analogous to another measure that theory dictates it is similar to. Discriminant validity is the opposite of convergent validity as it assesses the measures divergence or difference from other operationalized constructs that according to the theory it should be different from. The expectation is that there will be high correlations for elements that should be convergent, and low correlations for elements that should be divergent. This is seen in the correlation between education level and degrees of incarceration even when ethnicity is controlled for.

Reliability relates to the nature of the measure itself, that is does the measure do what it purports to do on a consistent basis. This speaks to the repeatability of the measure. If the researcher is using a question to assess the potential of an individual to engage in criminal behavior, a reliable measure would provide similar results under different conditions. The measure given to children in one school should not differ too greater if it is given to children in a second school. The measure accurately measures the same construct in different locations it then becomes reliable. The author identifies four types of reliability, inter-rater or inter-observer reliability, test retest reliability, parallel forms reliability, and internal consistency reliability.

Inter-rater or inter-observer reliability provides a measure of the extent to which different individuals provide a consistent estimate of the same phenomenon. When behavior is observed, it is very possible that different persons will understand and interpret the behavior differently. In attempting to understand power relations within the prison system multiple observers can be employed to explore how prisoners are treated by guards and the response of the prisoners to the treatment. This particular component is important in qualitative research designs that depend heavily on participant observation, or other methods of data collection that rely on observation by researchers. Test retest reliability is demonstrated when the same test is applied to the same group or a similar group and the results examined. The correlation between the scores is a measure of the reliability of the measurement instrument. Test retest reliability can be seen in the examination of parental influence on delinquency. The parents could be given the same test to establish parental attitudes and its link to juvenile delinquency.

Parallel forms reliability requires that the researcher compare two forms of the same thing. If a researcher was interested in recidivism and created a measure to predict recidivism in inmates the components of the measure could be divided into two parts forming two parallel measures. When these measures are tested on the group, the results should be very similar. Internal consistency reliability looks at how well measures in an instrument measure the same construct. The instrument used to measure recidivism could be examined for internal consistency by not dividing the measures but looking at them in the same questionnaire. The data are collected from the group in a single instance but the measures are tested against each other for consistency.