Reliability and validity should be considered...
before you start analyzing data
This explores the question "how do I know that the test, scale, instrument, etc. works every time I use it?
reliability
This explores the question "how do I know that the test, scale, instrument, etc. measures what it is supposed to?
validity
If data are not reliable or not valid, the results of any test or hypothesis
are not valid
Reliability and validity specifically relate to
the quality of the data themselves
The outcome variable in an analysis is called...
the dependent variable
The predictor variable in an analysis is called
the independent variable
This is the least precise level of measurement.
nominal
This is the most precise level of measurement.
ratio
This is the level of measurement where outcomes are based on some underlying continuum where it is possible to speak about how much more a higher performance is than a lower one:
interval level of measurement
This is the level of measurement where outcomes can only be placed into unranked categories
nominal level of measurement
This is the level of measurement where outcomes can be rank ordered.
ordinal level of measurement
This is the level of measurement where outcomes are based on some underlying continuum that also contains a true or absolute zero.
ratio level of measurement
Only this level of measurement has a true zero.
ratio
Moving from the nominal to the ratio levels of measurement, your data becomes...
more precise and detailed
Which of the following is concerned with monitoring estimates of present performance and predictions of future performance?
predictive validity
What type of validity is concerned with the adequate representation of test items?
content validity
If you want to know that a test measures some underlying psychological construct, what type of validity evidence would you want to collect?
constrict validity
If you correlate scores from your test with some other valid measure that assesses the same set of abilities, what type of validity evidence are your collecting?
criterion validity
If I can say that my weekly statistics quiz fairly assesses the material covered, what source of validity evidence should I have collected?
content validity
What is the key to establishing Criterion validity?
quality of criterion
Which of the following is NOT a measure of validity?
parallel forms validity
This type of validity is established by consultation with an expert on the topic focused upon by your instrument.
content validity
This type of validity is determined by determining the association between the test scores and some specified present or future criterion.
criterion validity
This type of validity is based on the judgment of how well a test reflects an underlying idea
construct validity
High school class rank is highly correlated with college GPA. This is an example of what type of validity
predictive validity
The score that you would actually record is the...
observed score
A measure of how stable a test is over time is an example of which of the following types of reliability?
test-re test
The correlation between scores from time 1 and time 2 is called:
test- re test reliability
The correlation between two forms of the same test is called:
parallel forms reliability
Cronbach'a Alpha is used to calculate the following
internal consistency reliaility
The number of agreements between your two raters divided by the total number of possible agreements is the way to calculate:
interrater reliability
This is the type of validity that examines how well a test outcome is consistent with a criterion that occurs in the present.
concurrent criterion validity
This is the type of validity that examines how well a test outcome is consistent with a criterion that occurs in the future.
predictive validity
This is the score that you would receive if the test contained no error.
true score
The actual or measured score is called:
the observed score
This is the difference between the observed and true scores:
the error score
Which coefficient reflects the occurrence of perfect reliability?
1.00
Two trained professionals observe the behavior of children in a classroom. They each rate observed behaviors using the same form and the number of items that were rated the same is calculated. This is an example of which type of reliability?
inter ratee
If the measures associated with a test are said to be consistent, you might conclude that the measure is ___________.
reliable
Which of the following is NOT a measure of reliability?
multiple forms reliability
When we calculate reliability, we know the observed score. What are the two unknown components of the reliability equation?
true and error scores
Inter-rater reliability measures consistency _________________.
from rater to rater
Which of the following is the Greek letter associated with Cronbach's Alpha, sometimes known as coefficient alpha?
A.
Test-Retest reliability is an example of which of the following?
measure of reliability over time
Parallel forms reliability is an example of which of the following?
measure of equivalence
Inter-rater reliability is an example of which of the following?
measure of agreement
What would the inter-rater reliability be for a 50 item measure in which the number of agreements between Rater 1 and Rater 2 was 45?
0.90
What are the four levels of measurement?
nominal, ordinal, interval, ratio
The maximum level of validity possible is equal to:
the square of the reliability coefficient
Which level of measurement is concerned only with the characteristics of an outcome that fits into a category?
nominal level of measurement
If you are interested in describing the order of variables along a continuum, what level of measurement would you use?
ordinal level of measurement
If the underlying continuum we are measuring assumed equal intervals, at what level of measurement is the associated variable being measured?
interval level of measurement
Which of the following levels of measurement provides the most information about a variable?
ratio level of measurement
Who is responsible for creating the formula for coefficient alpha?
Lee J. Cronbach
What is the assignment of values to outcomes known as?
measurement
What is the difference between the true score and the observed score?
observed score: what you actually get on the test
true score: the true , 100% accurate reflection of what you really know.
page 106
The outcome or predictive variable in an analysis is called the dependent variable. T/F
True
The treatment or predictor variable is called the independent variable. T/F
True
The less precise levels of measurement contain all the qualities of the scales above them. T/F
False