parallel forms
method of establishing the reliability of a measurement instrument by correlating scores on two different but equivalent versions of the same instrument
Test-Retest
method of establishing reliability is assessed by measuring the same individuals at two points in time. Appropriate for measuring something stable and consistent (personality, ability; not mood)
Ex. Test A today and test b a month from now
Split-Half
This measure of internal consistency/reliability takes a test and divides it in two. Theoretically, it assumes that the subject should perform equally well on both parts., after arbitrarily dividing the test into two halves, you should compute examinees'
Kuder-Richardson 20
To measure internal consistency/reliability for True/False questionnaires using numbers to represent answer choices, analysis for internal consistency. Average all possible split halves. Can be used only with dichotomous responses. For the KR20 formula to
Internal Consistency
A measure of reliability; the degree to which a test yields similar scores across its different parts, such as on odd versus even items. an estimate of how reliable a test is when items on the test are compared to each other. see split-half and odd-even r
Dichotomous
division into two parts, groups or classes esp. when these are sharply distinguished or opposed (usually correct/incorrect)
Coefficient Alpha
symbolized as ?, is a commonly used estimate of a test's or measure's reliability/internal consistency and is calculated by taking the average of all possible split-half correlations; also called Cronbach's alpha. Use with any response format. A generaliz
Factor Analysis
A set of multivariate data analysis methods for reducing large matrixes of correlations to fewer variables. The variables are linear combinations of the variables that were in the original correlation matrix.
Reliability of Difference Scores
The Lower the correlation b/t X&Y, lower the reliability. Includes error from both measures. Can use alternative statistical techniques
Reliability of Behavioral Observations
Error=time/item sampling, internal consistency, observer differences. Estimate reliability: interrator reliability--consistency among judges (percentage agreement, kappa, phi-correlation between 2 observers)
Interrator Reliability
consistency/degree of agreement between 2 or more scores/judges/observers/raters
Standard Error of Measurement
inaccuracy of a test caused by chance, degree of error is reported in the use of the term standard deviation, formula is takes the square root of 1 minus the reliability coefficient. Used to determine confidence intervals...Probability that true score fal
What reliability is high enough?
Different for different purposes. Research: .70s, .80s. Decisions about ppl >.95. Not always at desired level
What if reliability is too low?
1) Increase # of items-Larger sample of domain; use Spearman-Bowen to estimate how long it needs to be.
2) Factor analysis & Item Analysis-with internal consistency estimates; assess whether measure is unidimensional or multidimensional
3)Correction for A
Correction for Attenuation
Correction of the reduction, caused by low reliability, in the estimated correlation between a test and another measure. The correction for attenuation formula is used to estimate what the correlation would have been if the variables had been perfectly re
Spearman & Brown
a mathematical formula that can be used with split-half or odd-even reliability estimates to increase the accuracy, which is impaired because of the shortening (splitting in half) of the test.
formula developed by Spearman and Brown that one can use to co