Research and Design Exam 1

� What information (values) do you need to calculate a z-score?

...

� How do you interpret a z-score? Be able to interpret z-values of various signs and values.

...

� What's a standardized distribution?

...

What happens to the mean, the SD, and the shape of the distribution when we calculate z-scores?

...

What happens to the mean, the SD, and the shape of the distribution when we start from z-scores and add, subtract, multiply, or divide by a constant?

...

What descriptive statistics should you always report in an APA-style paper? Where in your paper does this information belong?

...

� What does APA recommend about the use of first- vs. third-person writing and active vs. passive voice?

...

examples of active voice:

...

example of passive voice

...

� How many decimal places should you include in APA papers

...

� Be able to interpret common APA abbreviations - N/n, M, SD, r

...

What are tables

organized sets of numbers

what are figures

picture representations of numbers

what are appendices

...

when should you use a table

...

what is a crosstab and what is it useful for

...

what kind of data is most appropriate for crosstabs

...

� Why doesn't correlation equal causation?

...

� What are correlations useful for?

...

� What is the conceptual definition of a correlation? (This is the formula in words, not symbols!)

...

� What is a covariance?

...

� How is the formula for a covariance similar to the formula for a variance? (Again, conceptually -not the symbols but their meaning).

...

What is the sum of products? How is it similar to the sum of squares?

...

o What happens to the covariance when an observation has positive deviations on both variables? Negative deviations on both variables? One of each?

...

Can we have a negative covariance? If so, what does it mean?

...

� What is the difference between a correlation and a covariance?

...

� How do correlations change if we add, subtract, multiply, or divide one or both variables by a constant?

...

� What is restricted range, and how does it affect correlations?

...

� Where does error come from in research? Be able to identify several possible sources.

Error comes from:
-Research design (design confounds, poor choices about variables or measures)
-Study administration (lack of experimental control, variability in procedures, human errors, equipment error, outside factors)
-participants (lack of understa

� What is the big-picture definition of reliability (as given in class, not your textbook)?

...

o How does reliability relate to error?

...

o Why are there different types of reliability coefficients?

...

� When do you need to worry about interrater reliability?

...

� How do you calculate an interrater reliability coefficient for two raters?

...

What do you calculate if you have more than two raters?

...

What is the standard for considering an interrater reliability coefficient acceptable?

...

� How is interrater agreement different from interrater reliability?

...

How can we calculate interrater agreement?

...

What is Cohen's kappa, and why is it more informative than simply calculating percent agreement?

...

What is the standard for considering a kappa coefficient acceptable?

...

� When do you need to worry about test-retest reliability?

...

When would it not be helpful to calculate test-retest reliability?

...

How do we calculate a test-retest reliability coefficient?

...

What is the standard for considering a test-retest reliability coefficient acceptable?

...

What's a theory?

an explanation of relationships amongst variables at a conceptual/abstract level

What's a hypothesis?

-a specific prediction about a pattern you expect to see in your data.
-Involves a comparison
-A good hypothesis follows logically from a theory, but is much more specific and concrete

How are the scientific definitions of "theory" and "hypothesis" different from popular misconceptions?

It is not an "If... then" statement

What 3 things do we need to establish before we can make a claim that one variable causes another

To claim A causes B:
1. A must be related to B (covariation)
2. A must happen before B in time (temporal precedence)
3. We must be able to rule out plausible alternative explanations for the relationship between A and B.

what two characteristics define a true experiment?

1. at least one variable is manipulated- determined by the researchers
2. participants are randomly assigned to a value of that variable

how is true experimental research different from correlational research?

...

what is the main advantage of experimental research?

...

What is the difference between a population and a sample?

a POPULATION is the group of people you want to be able to make conclusions about
a SAMPLE is the group of people who actually participated in your study

What is discrete variables

a discrete variable consists of separate, indivisible categories. There are no in-between values (no decimals) and are usually (but not always) a limited number of possible values

what are continuous variables

a continuous variable is (theoretically) divisible into an infinite number of fractional parts. We need to choose real-limits or the level of precision. (age, weight, etc)

nominal/categorical variables:

a nominal variable consists of a set of categories that have different names... but do not make any quantitative distinctions. They have no natural or logical order to them. (major, ethnicity, favorite ice cream flavor)

ordinal variables:

consists of a set of categories that are organized in an ordered sequence. It does not tell us how much "more than" or "less than" one observation is compared to the other

interval variables:

consists of ordered categories that are all intervals of exactly the same size. Equal differences between numbers on the scale reflect equal distances in magnitude. No natural zero point. (temperature, comparisons to an average or standard)

ratio variables:

an interval variable with the additional feature of an absolute zero point. Can compare ratios, meaningful zero. (Kelvin temperature, counts of things like speeding tickets)

Technically, what type of variable is a rating scale? Practically, what level of measurement do we usually treat rating scales as being

ordinal (like a likert-type rating scale)

Why does the type or level of our variables matter? Why do we need to know this?

...

How does summation notation work?

The symbol of notation tells you to take all of the values and add them together

Know the rules for order of operations - as you work through an equation, what should you do in what order?

PEMD(summation)AS

What's the difference between a frequency distribution and a histogram? When and why would you use each one?

a frequency distribution is an organized tabulation of the number of values located in each category on the scale of measurement.

How do you read a frequency distribution?

Usually 2 columns discussing about 1 variable and how often it occurs.

What's the difference between a bar graph and a histogram? What type of data is each more appropriate for, and why?

-Bar graphs best displays frequency distributions for nominal or ordinal data. Spaces between values
-Histograms are a specific type of bar graph that works best with interval or ratio data and group observations into class intervals with many possible va

How do you read a histogram?

...

What is a frequency polygon?

Graphic depiction that uses a line to represent the distribution of frequencies of scores.
It is like a histogram but with connected dots instead of bars

For frequency polygons, when do you use jagged lines and when do you use smooth curves

Use smooth curves when youre visualizing a population, and do not know the exact frquencies.
Use jagged lines for sample data (what you actually observed)

What is the main idea behind all measures of central tendency?

...

What is the conceptual definition of the mean?

the mean of a distribution is the sum of the scores divided by the number of scores

What happens when we add or subtract a constant to a mean?

It is the same as adding or subtracting the same constant to the mean

What if we multiply or divide to a mean?

multiplying or dividing each score by a constant is also the same as multiplying or dividing the mean by that same constant.
Example:
original: (3+2+3+4)/4= 3 (times 3!)
New: (9+6+9+12)/4=9

What is the median

the median of a distribution is the midpoint of the scores, when they are arranged in order from smallest to largest. Use with ordinal values and skewed data

When is it most appropriate to report and use the mean

The mean is a great way to represent symmetric distribution. Interval or ratio variables show best choice

What is mode

the most common, or the peaks of a histogram. Has greatest frequency. We often use mode with nominal/categorical data.

When is it most appropiate to use mode

for nominal variables

what is bimodal distribution

It is a distribution that has two modes.
has two most popular categories with equal (or similar) frequencies

what is the major mode of bimodal distribution

...

what is the minor mode of bimodal distribution

...

What does it mean to say that a distribution is skewed?

A non-symmetric distribution where one end "pulls" the shape of the distribution

What is the difference between positive and negative skew?

+ skew: They are on the high end. as variable 1 increases, variable 2 will increase and vice versa
- skew: Those extreme observations are on the low end. As variable 1 increases, variable 2 will decrease

How are the mean and the median related in a skewed distribution?

- skew: mean < median
+ skew: mean > median

What is the skewness statistic? How do we interpret it?

...

What is the kurtosis statistic? How do we interpret it?

Kurtosis refers to the degree to which a distribution is relatively flat or peaked

describe positively skewed distributions

...

describe negatively skewed distributions

...

Conceptually, what is the main idea behind all measures of variability?

...

Why is variability important? Is variability good or bad?

...

What is the range? How do you find it? When is the range useful?

...

� Conceptually, what is the variance?

...

What's a deviation?

...

Why do we square deviations when we calculate the variance?

...

What is the sum of squared deviations and why is it important?

...

Conceptually, what's a standard deviation?

...

� How are the variance and standard deviation related (statistically)? How are they different?

...

Why do we more often report the SD than the variance?

...

� What symbols do we use to represent the mean, variance, and standard deviation?

...

� What adjustment do we have to make when we calculate the variance or SD for a sample rather than a population? Why do we make this adjustment?

...

Why do we often use multiple items to measure the same variable?

...

� What is internal consistency? When do you need to worry about internal consistency?

...

What is the conceptual definition of an internal consistency coefficient?

...

What is the name of the statistic we calculate to assess internal consistency?

...

What is the standard for considering an internal consistency coefficient acceptable?

...

� How do you choose which reliability coefficient(s) you should report?

...

� When do you need to worry about validity?

...

� What is the definition of validity according to the Standards for Educational and Psychological Testing (given in lecture)? How is this different from many textbook definitions of validity?

...

� How are reliability and validity different? How are they related?

...

In general terms, how do we get evidence about validity?

...

� When (in what kinds of studies) do you need to worry about inattentive responding?

...

� How can you encourage participants to pay attention before you collect data?

...

� How can you assess which participants paid attention?

...

What is the infrequency approach?

...

What is the response time approach?

...

What is the low variability approach?

...

What is the psychometric antonym approach?

...

What is the long strings approach?

...

� If you identify inattentive respondents, what do you do? How do you report this in your results?

...

� What is the difference between inattentive responding and socially desirable responding?

...

� What is the difference between data that is missing at random and data that is systematically missing?

...

� What is the difference between listwise deletion and pairwise deletion? Which is better? Why?

...

� What is imputation, generally?

...

o What is the difference between mean imputation and multiple imputation? Which is better? Why?

...

What is maximum likelihood estimation (in general terms)? Does it work?

...

� How can you check for systematically missing data?

...

� What is the response rate? When and how can you calculate it?

...

� What is attrition? Is attrition systematically missing data or missing at random?

Attrition is when participants fail to complete a study (usually missing at random?)

� What is an outlier?

data points that deviate markedly from others and can distort our analyses in several different ways

o What is the difference between an error outlier and an interesting outlier?

error outlier: values that look extreme because of a mistake and not real observations
interesting (e.g., data entry mistakes, impossible values)
outliers: values that are extreme but are not mistakes (exceptions to general trends or patterns)

� What is a boxplot? How does it help you identify outliers?

...

� What's the difference between a single-construct outlier and a multiple-construct outlier?

...

� How does a scatterplot help you identify outliers?

...

� What is a Q-Q plot? How does it help you identify outliers?

a q-q plot rank orders of both variables and calculate quantiles for each

� If you find an error outlier, what should you do about it?

...

� If you find an interesting outlier, what should you do about it?

...

� Why do we use probabilistic reasoning in psychology?

...

� What does it mean to say that something is "random" in the context of statistics?

...

What is a random sample? What is an independent random sample?

...

What is the difference between sampling with replacement and sampling without replacement?

...

� What is the connection between probability and inferential statistics?

In probability stats, we know the distribution of possible outcomes in the population and we use probability to predict the most likely outcome in a sample where inferential stats we know the outcomes we have in the same and we want to work backwards to e

� What is a normal distribution?

a function that represents the distribution of many random variables as a symmetrical bell-shaped graph.

How do you read a normal distribution? What does the height of the curve represent?

...

What is a standard normal distribution?

a standard normal distribution has a mean of 0 and a variance of 1

� What does the normal distribution tell us about the probability of observing any particular value?

...

How do we use z-scores to find this probability? Be able to interpret z-scores and probability values in the context of a normal distribution.

...

What is the body and what is the tail in a normal distribution?

body= the larger area (>50%)
tail= the smaller area

� What is the difference between probability and nonprobability samples?

Probability samples start with a sampling frame, non probability samples do not

What must you have to have a probability sample?

a sampling frame

o What are the advantages and disadvantages of each one?

...

� We discussed three types of probability samples and three types of nonprobability samples.

Probability samples:
1. simple random sampling
2. cluster sample
3. stratified random sample
Nonprobability samples:
1. convenience/haphard sample
2. purposive sample
3. quota sample

� What is sampling error?

sampling error is the natural discrepancy, or amount of error, between a sample statistic and its corresponding population parameter.

o What is the difference between sample statistics and population parameters?

...

� What is the distribution of sample means? How do we get it?

...

How does the mean of this distribution change as we take more samples?

...

How does the variability of this distribution change as we take bigger samples?

...

� What is the Central Limit Theorem?

It explains that a sampling distribution of possible sample means is approximately normally distributed, regardless of the shape of the distribution in the population.

What three implications of the CLT did we discuss in class? How does this affect the shape, mean, and variability of the distribution of sample means?

...

� What is the standard error?
How is it similar to and different from the standard deviation?

Standard error is the average sampling error

How is it similar to and different from sampling error?

sampling error is the difference between one sample mean and the population mean

� What is the Law of Large Numbers?

the larger the sample size (n), the more probable it is that the sample mean will be close to the population means

� How can we use the distribution of sample means to find the probability of a particular mean from a population? How is this different from finding the probability of a single score?

...

� What is restricted range, and how does it affect correlations?

...

� Where does error come from in research? Be able to identify several possible sources.

...

� What is the big-picture definition of reliability (as given in class, not your textbook)?

...