Common Features of Qualitative Research
-Emphasize observation about natural behavior.
-A focus on previously unstudied processes and unanticipated phenomena.
-Researchers pay attention to social context and interconnections between social phenomena.
-Focus on human subjectivity.
-Focus on even
Field Research
-Data is collected through direct observation
-Conducted in the natural setting of the topic being studied.
-Can study informal groups/situations or formal groups/situations.
Participant Observation
A researcher develops a sustained and intensive relationship with people as they go about their normal activities.
Complete Observation
-Researcher does not participate in group activities and is publicly defined as a researcher.
-Often interaction with subjects of study.
-Commonly used when observing specific types of events and behaviors that occur in public.
Advantages and Disadvantages of Complete Observation
-Advantages
� Researcher can refuse to participate in illegal,
unethical or dangerous activities.
-Disadvantages
� Reactive Effects- People's actions are a reaction to
being studied.
Participation and Observation
Active participation and observation and researchers identify is known.
Advantages and Disadvantages of Participation and observation
-Advantages
� Researchers can decline to participate In unethical
or dangerous activities
� Members can keep some info hidden
-Disadvantages
� Researchers may have to prove themselves
Covert (Complete) Participation
Researcher does not reveal identity as a researcher to those who are observed.
Advantages and disadvantages of Covert Participation
- Advantages:
Reduces reactive effects
Gains entry to otherwise inaccessible settings.
- Disadvantages:
Cannot openly take notes or record conversations.
Cannot ask questions that will arouse suspicion.
Role is difficult to play successfully.
Keeping up t
Basic Steps in Field Research
1. Entering the field
2. Developing and maintaining relationships
3. Sampling people and events
4. Taking Notes
5. Managing Personal dimensions
Entering the Field
Conduct necessary background work and develop trust with at least one member of the research setting is a necessity.
Developing and maintaning relationships
� Develop a plausible and honest explanation for yourself and your study.
� Maintain support of key individuals in setting.
� Avoid being too aggressive in questioning others.
� Avoid faking similarity with your subjects.
� Avoid giving and receiving mone
Sampling people and events
Major Strategies: Snowball sampling, Theoretical Sampling and Experience sampling method
Theoretical Sampling
Researcher learns of important processes during data collection and makes certain to sample enough people to investigate fully.
Experience sampling Method
� Representative sample
� Participants carry an electric pager and fill out reports when they are beeped.
Taking notes
- Field notes are the primary method of recording participant observation data.
� Jot your notes and then write complete notes later.
- Time-consuming process
� Observed, heard, context, methodology
- Notes should also be a record of the researchers' feel
Managing Personal dimensions
� Take the time to consider how you want to relate to potential subjects as people.
� Speculate about what personal problems might arise and how you will respond to them.
� Keep in touch with other researchers and personal friends outside the research set
Systematic Observation
A standard form is used to record variation within the observed setting in terms of variables of interests (frequency of behaviors). Increases Reliability.
Systematic Observation: Case Study- Sampson and Raudenbush (1999)
Study of disorder ad crime in urban neighborhoods
� Multiple method investigations
� Systematic observation of public spaces
� Observed city blocks and recorded signs of physical and social disorder.
Intensive Interviewing
� Goal is to obtain in-depth information through open-ended questions.
� Subjects should be purposely chosen among those knowledgeable and open to discussion.
� Keep interviewing subject until the situation point is reached.
Asking Questions (Intensive Interviewing)
� Have a general outline or plan for the interview.
� Make initial questions short and direct.
� Follow-up questions should be tailored to responses from initial questions.
Focus Groups
- Groups of individuals that are formed by a researcher
� Usually 7-10 participants
-Researcher leads discussion
� Asks specific questions and guides discussions
� Encourages all members to participate
� Information gathered is qualitative and
unstructure
Process of qualitative data analysis
� Documentation of the data and the process of data collection.
� Organization or categorization of the data into concepts.
� Connection of the data to show how one concept may influence another.
� Corroboration or legitimization.
� Representing the accou
Ethical Issues in Qualitative research
� Voluntary participation
� Subject well-being
� Identity disclosure of research
� Confidentiality of subjects information
� Appropriate boundaries
� Safety of researcher
Should tape recorders be used during intensive interviews?
They are usually used but during special circumstances.
What is saturation point?
Is the point in data collection when no new or relevant information emerges with respect to the newly constructed theory.
Applied Research
Rigorous investigation of a situation or problem in order to generate new knowledge or validate existing knowledge. A form of systematic inquiry involving the practical application of science
Evaluation Research
- Purpose of research is to investigate social programs
- Often referred to as applied research
- All evaluation research is empirical and data-driven
All evaluations should have?
- Utility- Serves the practical needs of intended users.
- Feasibility- Realistic
- Propriety- Conducted legally and ethically
- Accuracy- Convey technically adequate information
Program Evaluation Includes
Inputs
Program Process
Outputs
Outcomes
Feedback
Program Stakeholders
Inputs
Resources, raw materials, clients and staff that go into a program.
Program Process
Complete treatment/service delivered by program
Outputs
Services actually delivered/new products produced
Outcomes
Impact of program on cases
Feedback
Information about program process, outputs, and outcomes that is available to any program inputs.
Program stakeholders
Individual's/groups who have a basis of interest in program.
Define the three general question types in evaluation research
What is the programs impact?
How does the program do what it does?
Design studies based on stakeholders agenda
Evaluation Focuses
o Judgement-oriented
o Improvement-oriented
o Knowledge-oriented
Judgement-oriented
Overall merit, worth, or value of program
Improvement Oriented
Uses evaluation results to improve program
Knowledge Oriented
Influences thinking and tests theory
Types of evaluations
1. Needs assessment
2. Evaluability assessment
3. Process evaluation
4. Impact evaluation
5. Efficiency evaluation
Needs Assessment
- Is a new program needed? Is an existing program still needed?
- To assess need:
Determine nature and scope of problem the
program is designed to address.
Examine target population.
Evaluability assessment
- Can the program be evaluated within the available time and resources?
- Evaluability assessment may help to clarify program goals or refine program operations.
Process evaluation
- Is program operating according to its design? Is it doing what it says it is doing?
- Typically both quantitative and qualitative data are used.
Impact evaluation
- Did program have intended consequences?
Outcome refers to the effect on clients
Impact refers to the effect on larger
system/community
- Often uses experimental or quasi-experimental designs to assess outcome and impact.
Efficiency Evaluation
- Do financial benefits of a program offset the costs of providing the program?
- Two general types of evaluations: Cost benefit and Cost effectiveness
Cost benefit
Compares program expenditures (inputs) to monetary value of outcomes/impacts.
Cost Effectiveness
Compares program expenditures (inputs) to actual outcomes/impacts.
Design Decisions
1. Black box or program theory
2. Researcher or stakeholder orientation
3. Quantitative or qualitative methods
4. Simple or complex outcomes
Black Box (program theory)
� Do we care how the program gets results?
� Addresses whether the program produced the desired outcome occur or not.
Stakeholder Approaches
What goals matter most?
� Utilization-focused evaluation
� Participatory research
� Appreciative inquiry
Utilization-focused evaluation
Task force of program stakeholders formed=utilize results.
Participatory Research
Program participants are co-researchers
Appreciative inquiry
Program participants drive program.
Qualitative and Quantitative Methods
� What methods provide the best answers?
� Effects of a treatment, law, or program typically = quantitative.
�Qualitative methods:
� Add depth, detail and exemplary case studies.
� Increase understanding as to how social programs
actually operate.
Simple or Complex Outcomes
� How complicated should the findings be?
� Single outcome measure or complex outcomes measure is an important step.
� Examination of multiple outcomes greatly complicates analysis and interpretation.
Ethical Issues in Evaluation Research
�Most issues concern outcome and impact studies
o Issues related to clients:
-Distribution of benefits
- Preserving confidentiality
o Issues related to evaluation design:
- Research designs may be shaped by politics.
o Issues related to the use of evaluat
Secondary Data Analysis
The collection or analysis of data that were originally collected by someone else at another time.
Types of secondary Research
Statistics
Theoretical work
Media
Historic events
Historical Events Research
Research in which social events of a time period in the past are studied.
Event-structure analysis
Qualitative approach-code key events or national characteristics to identify the underlying structure of action in a chronology of events.
Comparative Methods
Research comparing data from more than one time period and/or more than one nation.
Several countries (Cross-national)
Different social groups
Issues that transcend borders (transnational) how cultures differ around borders.
Allows for broader vision abou
Content Analysis
o Systematic, objective, quantitative analysis of message characteristics
o Analyze various forms of media.
o Goal: Develop inferences from text.
Methodological challenges when using secondary data
Lost documents and evidence
Biased data
Data synthesis
Ethical issues in secondary analysis
o Generally confidentiality is not an issue
- However, it may be in historical records and official agency records.
The strongest research design for assessing the impacts and outcomes of programs and other interventions is?
True Experimental Design
A researcher evaluates a prison work training program. Her evaluation is rather negative. She concludes that "this program is useless unless the barriers that prevent inmates from participating in this program are removed. If inmates cannot participate in
Improvement Oriented evaluation
A cost effectiveness analysis is an analysis in which?
a. Inputs are estimated in terms of actual impacts.
B. Inputs are estimated in monetary terms and outputs are estimated in terms of actual impacts.
c. Inputs are estimated terms of actual impacts and outputs are estimated in monetary terms.
d. Inputs and
Advantages of intensive interviewing compared with standard survey research include all but which of the following?
a. It engages the researcher more actively with subjects
b. It directly reveals the social context around actions and opinions
C. It is more easy to use random sampling
d. The researcher can listen to lengthy explanations
Once coding procedures are developed for content analysis, their reliability should be assessed by?
Comparing different coders. Codes for the same variables.
Data quality is always a concern with secondary data?
True
A politician is interested in implementing a statewide welfare reform program. By providing welfare recipients with extensive job training, she believes that the number of welfare recipients will decrease. As a side benefit, she also believes that the cri
Efficiency
In a black box evaluation, a researcher addresses process and outcome of the program?
False
Evaluations aimed at determining the overall merit, worth, or value of something are known as?
Judgement-oriented
Utility is intended to ensure that an evaluation will?
Serve the practical needs of intended users
Research projects using secondary data analysis never need to be reviewed by institutional review boards?
False
The ethical requirement of voluntary participation in qualitative research is?
A. More difficult to establish in covert participation
b. Not required unless subjects are below the age of 18
c. Not required, regardless of whatever data are being collected or not.
d. Not often a problem in focus group research.
Researchers should ask sensitive questions only of informants with whom they have a good relationship?
True
Intensive interviewing seeks in-depth information using......Questions?
a. Indexes based on many close ended
b. Close ended systematically unstructured
C. Open ended relatively unstructured
John is a field researcher who studies social interaction within motorcycle groups. He is not an accomplished rider himself, but he is able to obtain jobs at various motorcycle shops that allow him to observe members of motorcycle groups and to socialize
Complete (Covert) Participant