Methods & Stats Comp

Scientific Method

Results of any single investigation may be the result of influences other than the intervention or experimental manipulation Research cannot prove a hypothesis; it supports ReplicationExtend

Replicate extend

A model of research favoring the repetition of an earlier study with an additional feature that enhances its generalization

Random Assignment

assigning participants to experimental and control conditions by chance, thus minimizing preexisting differences between those assigned to the different groups

Internal Validity

Extent to which a study establishes a trustworthy cause-and-effect relationship between a treatment and an outcome

Internal Validity Threat

history, maturation, testing, instrumentation, testing, selection bias, regression to the mean, attrition

External validity

Extent to which conclusions of study can be generalized

External Validity Threat

selection bias, history, experimenter effect, Hawthorne effect (know being studied), situation

experimental

IV manipulated by experimenter

quasi-experimental

an experimental design that lacks random assignment

naturalistic observation

a descriptive technique of observing and recording behavior in naturally occurring situations without trying to manipulate and control the situation

Cross Sectional

one point in time of different age groups

Cohort Design

type of longitudinal cohorts of people observed overtime

Case-Control

observational study participants selected based on their outcome status

Solomon 4 Group Design

half receive pre/post test; half only post test; half control and half experimental

Single Subject

Case Study

Multiple Baseline

baseline data, intervention, take away intervention, introduce intervention

Control for cofounding variables

•Key to experimental method.•Want least ambiguous interpretations of results.•Manipulate IV and hold all other variables constant, either by experimental control or randomization

control group

if random assignment was done, group who does not receive intervention

comparison group

if random assignment was NOT done, the group who did not receive intervention/IV

Cross Over Design

repeated measurements design such that each experimental unit (patient) receives different treatments during the different time periods, i.e., the patients cross over from one treatment to another during the course of the trial

Counterbalancing

A method of controlling for order effects in a repeated measure design by either including all orders of treatment or by randomly determining the order for each subject

Determining Causation

1. Temporal Precedence: A occurs before B2. Covariation of cause and effect: when A changes, B changes3. Elimination of alternative explanation: control for C

Elimination of alternative explanation

control for C

Covariation of cause and effect

when A changes, B changes

Temporal Precedence

A occurs before B

Research ethical concerns

informed consentdebriefprotection of participantsdeceptionconfidentialitywithdrawal

Terminology in scientific inquiry

1. Question2. Research3. Hypothesis4. Experiment5. Data analysis6. Conclusions

Types of variables

IV/predictor: variable that is manipulated to examine its impact on the DVDV/outcome: what happens/measures after manipulationControls: kept the same throughout the experiment Covariates: no direct interest to the researcher, but one that may have an affect on the outcome

Reliability vs. Validity

A test may be reliable without being valid, but a test cannot be valid unless it is reliable

Types of samples

Probability: some form of random selectionNon-Probability: convenience Stratified: proportional representation of groups in the sample is the same as in the population

Single subject designs

-subjects serve as their own controls (compared only to themselves)

Scales of measurement

nominal: categories, no numeric scaleordinal: rank orderinterval: numerical ratio: 0 indicates absence

nominal

categories (male/female)

interval

space between numbers is important

ordinal

order

Ratio

true zero

equivalence reliability

Measurement reliability across indicators; a measure that yields consistent results using different specific indicators, assuming that all measure the same construct

stability reliability

test-retest; scores remain constant from the test to the retest assuming that the true ability of each person has not changed between testings

homogeneity reliability

Addresses the correlation of various items within the instrument or internal consistency; determined by split-half reliability or Cronbach's alpha coefficient

Inter-Rater Reliability

correlation between the observations of 2 different raters on a measure

Construct

adequacy of the operational definition of variables

Face

the content of the measure appears to reflect the construct being measured

Content

the extent to which a measure represents all facets of a construct

Criterion

measures how well one measure predicts an outcome for another measure

Convergent

scores on the measure are related to other measures of the same construct

Divergent Validity

scores on measure are not related to other measures that are theoretically different

Multitrait-multimethod matrix

approach to assess construct validity

Incremental Validity

used to determine whether a new psychometric assessment will increase the predictive ability beyond that provided by an existing method of assessment.

sampling distribution

a probability distribution of a statistic obtained from a larger number of samples drawn from a specific population

Statistical inference

statistics makes inferences and predictions about a population based on a sample of data taken from the population in question

Normal Distribution

68% within 1 SD; 97% within 2 SDs, 99% within 3 SDs of the mean (under the curve)

T distribution

if differences b/w groups is larger than we would expect based on standard error

F distribution

Compare amount of variability explained by the model (experiment) to the error in the model (indiv differences)

chi-square distribution

the sum of these random samples squared

Measures of central tendency

mean- sensitive to outliersmedian - less sensitive to outliersmode-possible to have multiples modals

measures of dispersion (variability)

range: highest-lowestvariance: SD^2standard deviation: average distance from the mean

Standard Deviation

Average distance from the mean

variance

SD^2

Range

highest-lowest

Parameter

Characteristic of a population

Statistic

characteristic of a sample

Skewness vs. kurtosis

skewness: measure of symmetry -positive: tail to the right-negative: tail to the leftkurtosis: measure of how heavily tailed

Frequency tables and distributions

how to identify outliers:

Scatterplots

data visualization that shows the relationship between different variablesHow to identify outliers: If one point of a scatter plot is farther from the regression line than some other point, then the scatter plot has at least one outlier.

Independent samples t test

Compares the means of two groups on one continuous DV

paired sample/dependent t test

Compares two means based on related data (same people; same dyad; pre-post); continuous DV

One-Way ANOVA

Test for differences between 2 or more groups; 1 IV w/ 1+ levelsIV=categoricalDV=continuousESTABLISHES DIFFERENCES EXIST; NOT WHICH GROUPS ARE DIFFERENT

Factorial ANOVA

Compare means of multiple groups; 2+ IVs, 1 DVmore power than multiple ANOVAS and can detect interactions

2 x 3 x 4 ANOVA

number of numbers=number of factors-3numbers themselves= number of levels-F1 has 2 levels-F2 has 3 levels-F4 has 4 levels

Main Effects

one IV has sig overall effect on the DV, regardless of the other IV

Interaction

Effects of 1 IV differ according to level of the other IV-sig tells THERE IS DIFFERENCE; NOT HOW

If lines of graph a factorial ANOVA cross each other is there an interaction?

Yes, as long as slopes are different (not parallel) there is an interaction

Repeated Measures ANOVA

Same group participants get all treatment/conditions-1 IV w/ 1+ levels-1 DV

Chi-Square

Whether distribution of CATEGORICAL variables differ from one another -2 categorical variables-use odds ratio for effect size

Pearson r

a measure of the strength and direction of a linear relationship; correlation between continuous variables

Linear Regression

predicting an association between two variables when outcome=continuousUses F-statistic Regression line: Y=Bo+B1(X)

ANCOVA

Test for differences b/w group means when know an extraneous variable affects the DV1 IV w/1+ level (categ)1 DV (cont)1 CV (cont)Reduces Error Variance & increases experimental control

MANOVA

compare group means when testing for more than 1 DV-1+ IV-2+ DVs

MANCOVA

-2+ DVs-1+ IV-I+ CV

Spearman rank correlation

nonparametric test that quantifies the strength of an association b/t 2 variables that does not assume normal distribution continuous data. Can be used for ordinal or nonnormally distributed continuous data.

Logistic regression

Predict CATEGORICAL outcome variable from 1+ predictors -predicts the probability of the outcome occurring Methods: forced entry, hierarchical, stepwise

coefficient of determination (r^2)

Measure of model fit What amount of variance in Y is explained by X?

Linearity

linear relationship

Normality

Population from which sample is drawn is normally distributed

Homogeneity of Variance

want variability in each group to be about the same

Homoscedasticity

error term (scatter) is the same across all values of the independent variables.

Do you want Levene's to be significant?

no; want >.05

Effect Sizes

how different the groups are from one anotherT-test: Cohen's d

Setting alpha

Probability of type 1 error; setting threshold usually set to .05

Moderating

affects the STRENGTH of the relationship b/w IV and DV

Mediating

EXPLAINS the relationship b/w IV and DV

P values

Tells how extreme the data isHow likely got the result if the null is true

Z scores

measure the distance of a score from the mean in units of standard deviation

type 1 vs type 2 error

Type 1: false positiveType 2 : false negativeDecreased by strong power (larger sample size)

Familywise Error

The likelihood of making at least one Type I error across a number of comparisons.

Planned Comparisons

find out which groups differ using hypothesis and theory; using chunks & defining weights

Post Docs

comparing means against each other while controlling for type 1 error; no theory -Bonferroni=conservative

Power

Tells how many participants need for study Can detect true differencesUsually set to .80

Relationship of effect size, sample size, power

depends upon effect size and sample sizeHuge sample sizes may detect differences that are quite small and possibly trivial.

Factorial ANOVA

multiple IVs

MANOVA

multiple DVs

Relationship B/W variables tests

PEARSON Correlation (both continuous)SPEARMAN Correlation (both rank)

Predictors Tests

One Predictor/outcome: -LOGISTIC regression (categorical)-LINEAR regression (continuous)Several Predictor/1 outcome:-LOGISTIC (categorical)-MULTIPLE (continuous)Several/Several:-Canonical correlation

Differences B/W groups Tests with 2 levels of IV

CHI-SQUARE (categorial)PAIRED T TEST (continuous AND same people/pairs)INDEPENDENT T-TEST (continuous AND diff. people)SINGLE SAMPLE T TEST (sample vs. norm)

Differences B/W groups Tests with >2 levels IV

CHI-SQUARE (categorical)1 WAY ANOVA RM (continuous AND same people)1 WAY ANOVA NO RM (continuous AND diff people)FACTORIAL (one DV and multiple IVs)MANOVA (multiple continuous DVs and 1+ categorical IVs)ANCOVA/MANCOVA (with covariate variable)