Four major types of validity:
Construct, external, statistical, and internal
Construct
how well the variables in the study are measured or manipulated. The extent to which the operational variables used in the study are a good approximation of the conceptual variables
External
the extent to which the results of a study generalize to some larger population(ex: whether the results from a sample of children apply to all US children), as well as the other times or situations(ex: whether the results based on one type of music applie
Statistical
addresses the strength of an effect and its statistical significance (probability that the result could have been obtained by chance if there really is no effect). Also addresses the extent to which the study minimizes the probabilities of two errors(of f
Internal
In a relationship between variable (A) and another (B), the extent to which A, rather than some other variable (C) is responsible for the effect on (B)
Outliers can
give a line some pull in a different direction contributing to lack of stability
Threats to internal validity: history
History: unique experiences subjects have between measurements in an experiment
Threats to internal validity: maturation
Maturation: natural(not experimenter imposed) changes that occur as result of time passing
Threats to internal validity: testing
Testing: subject pretests to establish subject equivalency can contaminate test performance
Threats to internal validity: instrumentation
Instrumentation: changing the measurement methods/administration affects what is measured
Threats to internal validity: statistical regression
Statistical Regression: when subjects are selected because they scored high or low, they will almost always produce a different distribution of scores closer to the population's at retest
Threats to internal validity: selection
Selection: subjects in comparison are not functionally equivalent at the beginning of the study
Threats to internal validity: experimental mortality
Experimental Mortality: subjects drop out of studies
Threats to internal validity: selection interactions
Selection Interactions: in some studies the selection method interacts with
Type I Error
conclude there is an association when there is not
Type II Error
conclude there is no association when there is
Reliability
consistency or stability in measurement over time
Measurement Error
Measures typically have some amount of measurement error but strive to keep it small
Research Error
error introduced in the process of collecting data, human error, greater number of data collectors the more error is likely
Individual Differences
have the effect of spreading out the scores of people within each group
Situation Noise
external distractions of any kind, can add unsystematic variability
Inter-rater or inter-observer reliability:
two observers with exact same script to take measurements