Define applied research
� A form of systematic inquiry involving the practical application of science.
� Accesses some part of research communities' (academia) accumulated theories, knowledge, methods, and techniques -- uses them to solve specific problems (e.g., gov't, business
Give examples of applied vs. basic research
Pure Research
Academic
Hard to apply in business settings
Done out of curiosity
Concerned w/ refuting or supporting theories
EX: Do people like to congregate in landscape settings or in hardscapes? (Just to know!)
Applied Research
Non-academic
Easier to a
List the processes of the scientific method
�1. Question formulation (often the most important part!)
�2. Hypothesis/Conjecture formulation
�3. Prediction
�4. Testing: Experimentation/Observation
Experimentation = involves some sort of manipulation; usually quantitative in nature
Observation = not
List the characteristics of applied behavioral research
�5. Scientific study has an element of creativity (clients often have novel problems needing novel analysis)
�2. Uses theories and language associated with a field of study (APs develop this language for clients)
�3. Behavior is not governed by strict law
Describe the three types of empirical inquiry
�1. Descriptive
Qualitative or quantitative info about the sample you're studying
Ex: customer/employees' attitudes, demographics, levels of satisfaction/quality
�2. Relational/Correlational
Quantitative only ? Lacks causality
Ex: relationship between emp
List the characteristics of a good applied researcher
�Enthusiasm (about what you're studying)
�Open-mindedness (can't go in thinking you know everything!)
�Common sense
�Role-taking ability
�Inventiveness
�Confidence in one's own judgment (comes from knowledge and experience)
�Consistency and care about det
Give examples of applications of psychology to business problems
� Marketing/market research
� Employee training and retention
� Employee satisfaction
� Program evaluation
� Human factors
� Legal issues
List the steps in the research process
� 1. Identify problem/area of study
� 2. Gather information (about nature of problem)
� 3. Create research objective (often the most important part!)
� 4. Formulate hypotheses
� 5. Design research study
� 6. Collect data
� 7. Analyze/interpret data
� 8. R
Compare and contrast the four difference between qualitative and quantitative data
Qualitative
Nature of Data - Richly detailed data (not quantified); Concepts illustrated through recordings (words, pictures, videos, all 3)
Contextualism - Results assumed as specific to time, place, ppl, culture studied (narrow context ? not generalizab
Describe the value of qualitative research
�Qualitative research can (1) Supplement, (2) Complement, (3) Stand on its own
�Provides unique insights into:
o HOW people behave and
o WHY people behave as they do
�Providers deeper understanding to numeric data ? Situations in which it is applicable is
List the caveats to qualitative research
� May not be the best research method in given cases
� Often seen as quick and cheap - not seen as rigorous (an erroneous view - but still present)
� Often seen as "exploratory" (customers may think that researchers are lacking in a good hypothesis)
� Qua
List the five traditions in qualitative research and what questions the ask
o 1. Phenomenology
� Seeks deep understanding of lived experiences ? Asks about nature of ppl's lived experiences
� Not generalizable - this approach doesn't even seek universal understanding!
� Not common in applied research
� Data Sources: Introspection
State the history of SPSS
� Founded in 1968 by Norman Nie (a political science student at UChicago)
� Needed a way to analyze his dissertation data (there was no good package to analyze data then) ? got help from a couple of computer scientists to help him write this
� SPSS now wi
State the importance of recoding variables
� Useful for creating new variables from existing variables, especially when creating new categories
What makes up Qualitative Research?
� Qualitative Research (the core, at least) =
o (1) Interviews (usually depth interviews) +
� Interviews consist of: (1) Depth interviews, (2) Casual interviews, (3) Group interviews
o (2) Observational Research
� Ethnography is a special type of observat
Describe how to prepare for a depth interview
� Familiarize yourself w/ topic before going into field (e.g., literature reviews, perspective of clients)
� Gain some distance from the topic, and forget what you've learned.
� Have a recorder ready (including new batteries)
� Do take notes during the in
Describe how to recruit informants
� Think about who you want to recruit prior to interviewing
� Think about the best places to find informants
o Important to ask: Who is your customer's target market??? This will affect where you go!
� Convenience samples are not ideal! (You want your sam
Describe how to conduct a depth interview
� Choose a neutral place to conduct the interview. (e.g., the conference room, not their cubicle)
� Have the discussion guide/protocol in front of you or memorize it
� Important to develop beforehand - treat it only as a guide/general outline, though!
� E
List the guidelines for conducting a depth interview
� 1. Funnel questions from general � specific � general.
o Don't ask heavy-hitting questions at first - let them get comfortable beforehand.
� 2. Do not ask "Why?"
o Ask: "How is that?" "Can you tell me more about that?" "Can you please explain that a bit
Compare and contrast the advantages and disadvantages of focus groups
� Advantages
o Relatively quick and easy (for APs)
o Allow clients to observe participants first-hand (e.g., 1-way mirrors)
o Generate fresh ideas (better than depth interviews)
o Broad spectrum of use - employees, customers, HR, etc.
o Allow easy access
Describe why projective methods might be used
� Can be used in depth interviews or focus groups - but not generally used in applied research that often
� Commonly used in academic research. They have their roots in psychotherapy.
� Advantages (Why might they be used?)
o Help informants say things the
Describe the five elements of ethnography in practice
� Applied ethnography seeks to understand the experiences of people and how their lives are shaped by their experiences; ethnography in practice:
o 1. Focus - expected to contribute to marketing strategy or tactics by revealing consumer (or employee) want
List the elements of observational interviews
� Same guidelines as depth interviews
� May have more than one researcher
� Takes place in a naturalistic setting
� Disguised vs. undisguised (whether you're aware that you're being researched)
� Researcher can observe and ask questions
� Didactic intervi
Describe alternative observational methods
� Videography
o Uses: Videotaped interview; Naturalistic observations; Video diary; Use of smartphones/pictures
o Benefits: Humanizing documentation; Captures subtle temporal, social, spatial dimensions;
o Easier now with technology!
� Other forms of obse
Describe qualitative data mining
� The process of discovering useful patterns of knowledge from sources of data such as:
(1) Databases, (2) Website, (3) Text files, (4) Images, (5) Videos.
� Elements:
o Complex naturalistic in situ situations (the internet is where it is actually happeni
Compare and contrast data mining with opinion mining
� Opinion Mining = A specific type of data mining - attempts to measure online word of mouth
� Uses natural language processing (which can be very tricky!)
o Sentiment classification (valance: whether it is a positive or negative comment)
o Feature-based
Describe qualitative social network analysis
� The use of network theory to analyze social networks (network of social relationships)
� 2 main elements In a social network
o 1. Social actors themselves ("nodes")
o 2. Relationships between them ("ties")
� Useful for: Word of mouth; Diffusion of innov
Compare and contrast ethnography with netnography
� Netnography = ethnography in the social spaces of online environments
� Key elements:
o Researcher = Participant Observer
o Lived experiences of participants
o Rich contextualization (situation/data)
o Systematic and rigorous
o More academic, less appli
Describe Netnography as Applied Research
� Brand communities (Apple)
� Consumer communities (Travel boards)
� User groups (Sawtooth)
� "Guilds" (Conjoint)
� Employee experience
Compare and contrast the six different classes of data collection aids
� Data collection aids help better elicit and capture data
o Range is from simple to sophisticated - but sometimes simpler is better!
� 1. Written notes
� 2. Audio recording
o Should contain: Place; Time; Whats happening; Why; Ppl present/interviewed (obs
Recall the key terms in qualitative data analysis
� Theory: system of ideas/statements explaining some phenomena (informal hypotheses common in AP)
� Coding: Concepts that reduce the data into meaningful chunks and assigns names to these concepts
o Emic: Analyses that draw directly on the language used b
List the seven themes to watch out for when coding
� 1. Metaphors (coding these can help you figure out how people are making sense of their own reality)
� 2. Strong emotions (provide clues as to what is important to your participants)
� 3.Phrases that are Imported/used in new context (may point to contex
Describe coding (data analysis) as a process
� Run through initial coding - adjust as needed (it's an iterative process!)
o Look for variation in codes among different groups in your data
o Review of coding/data and literature/hypotheses may dictate other approaches, different types of interviews, o
Describe how theory influences analysis and report
� Presentations should be managerially useful (i.e., give directives for action that can be taken to solve the problem under investigation)
o Presentations are based on the problem, the data, and theory - theory helps to:
� Guide data analysis (coding)
�
List the elements of managerially actionable qualitative research
� Success indicators of managerial actionability:
o Clear answer to the business problem
o Clear reasoning for the methods chosen
o Outcome and recommendations must be clearly linked to the data
� Managerial actionability based on:
o Contextualized data (
List the elements of a successful qualitative report
� Tell a story/narrative.
� Make it useful and actionable (link up need for action w/ the info you found to help direct action)
� Keep it short and tight.
� Make good use of visuals (e.g., use infographics - charts and graphs.)
� Use direct quotes (text,
Describe the elements of quality in qualitative research
� Quality is assessed qualitatively! J
o Trustworthiness (e.g., admitting bias and problems with data collection)
o Honesty (e.g., the transitional nature of research and understanding)
o Sufficiency of observations and sampling
o Congruence within data
o
Discuss the approaches to reliability and validity in qualitative research
� Reliability
o Objectivity (in terms of coding and data interpretation)
o Reporting raw data
o Field notes as a reliability check (Have we treated hypothesis/theories correctly or are we off track?)
� Validity
o Correctly naming concepts during coding
o
10) Qualitative Research: Wrap Up!
� Blend old technology and new technology
o Don't be afraid of new technologies/media - but don't get carried away w/ their great potential!
� Don't be afraid to use different qualitative methodologies!
o Develop a big methodological toolkit, and select a
Articulate what research designs are
� The "plan" for the research; includes type of research, data collection method, and the analysis method
o The design is based upon the research problem and the research objective
o Because every research question is unique, every design is unique as wel
Describe how exploratory research informs the researcher
� Often just discussions with your client in order to get more context about research problem
� Uses:
o Gain background information (context in which you're conducting research)
o Establish research priorities
o Clarify problems and hypotheses
o Define te
Discuss what questions are addressed by descriptive research
� Used to describe characteristics of a population or phenomenon being studied - not trying to intervene!
o It does not answer questions about how/when/why the characteristics occurred.
o Rather it addresses the "what" question (What are characteristics o
List the important implications of causal research
� Experimentation ? Determination of Causal Relationships (Establishes Causation!)
o Experiment = manipulating an IV to see how it affects a DV
o The variables must be related (statistically)
o Temporal precedence must exist (treatment first � then outcom
Compare and contrast the different types of research designs
� Research Objectives ? Research Designs
� 2 types of general research designs:
o 1. Non-experimental studies
� To gain background information and to develop hypotheses ? Exploratory Research
� To measure the state of a variable of interest ? Descriptive
List the threats to validity of experimental designs
� 2 types of Validity
o Internal: Concerns the extent to which the change in the DV was actually due to IV (i.e., was proper experimental design used/implemented correctly, external sources of variation controlled for?)
o External: Concerns the extent to
Explain the different experimental designs
� 3 Experiment Types
o 1. True experiments (i.e., laboratory studies) ? internal validity
� Typically done in labs, where effects of IV on DV can be isolated, extraneous variables can be controlled (through use of control group), random assignment done
o
Define secondary data
� Primary data = information that is gathered by the researcher specifically for the project at hand
� Secondary data = previously gathered by someone other than researcher and/or for some other purpose
Articulate the classifications of secondary data
� Internal Data (data collected within the firm)
o Database marketing records (process of building, maintaining, and using customer databases and other databases (products, suppliers, resellers) to contract, transact, and build customer relations)
� CRM (
Compare and contrast the advantages and disadvantages of secondary data
� Advantages
o Readily available (for almost any application)
o Can be obtained quickly (i.e., days or even hours)
o Inexpensive/cost-effective (compared to collecting primary data)
o May enhance primary data (a current look at the issues/trends affecting
Evaluate secondary data sources
� Sources
o Reference guides
o Indexes and databases
o Dictionaries and encyclopedias
o Directories
o Statistical sources (e.g., census)
� Evaluating secondary data�reliability
o Original purpose (i.e., why was it collected? What was the true purpose?)
o
Discuss the difference between standardized data and other types of information
Standardized information is a type of secondary data in which the data collection process is standardized for all users
and
is reported in the same format for all users; two kinds:
Discuss the difference between syndicated data and standardized data
o 1. Packaged Services
� Prepackaged marketing research process that is used to generate info for a specific user
� E.g., Bass's / M/A/R/C; Mystery shopping
� Advantages
� Time-tested
� Reduced costs
� Speed/efficiency
� Disadvantages
� Not customized dat
Compare and contrast the advantages and disadvantages of standardized data
Advantages
� Shared costs/cheaper
� Data quality is high
� Data is disseminated very quickly
Disadvantages
� Little or no control over the data collected
� No strategic information advantage
� Long-term contracts are often required (once a yr for several
List what areas standardized data may be applied
� Measuring consumer attitudes and opinions
� Defining market segments
� Monitoring media usage and promotion effectiveness (e.g., Nielsen ratings)
� Conducting market-tracking studies
Distinguish the different types of question formats
� 1. Open-ended
o Unaided (E.g., What brand of gasoline can you think of? � Just let consumers tell you brands)
o Aided (E.g., Have you heard of this brand, yes or no? � Let respondents choose from a set of brands.)
� 2. Categorical
o Dual choice (E.g., y
Articulate basic concepts in measurement
� Measurement: Determining the description or amount of some element of interest to research objective.
o Measurements should be helping us to solve the business problem/research objective
� Properties: Specific features, attributes, characteristics of ob
List the different types of measurement scales used in applied psychology
� Natural Scales (they give us their own answer)
o Standard natural scales (e.g., age, income, number of Target visit this week)
� Nothing predefined in these questions but we know that there is going to be a linear distribution
o Ranking (rate or rank th
Describe why the type of measurement scale is important
� The choice of the scale is important because:
o It affects how long it will take your ppl to fill it out
o It affects the type of analysis to be performed (garbage in, garbage out!)
o It affects how we display the results in reporting
4 Methods of Gathering Survey Data
Listed below...
1. Person-Administered Surveys
o Interviewer reads questions and records the answers on paper.
o Types:
� In-home (interviewer comes to respondent's home during appointment time)
� In-office (interviewer comes to respondent's place of work)
� Central telephone location (interviewer wor
2. Computer-Administered/Computer-Assisted Surveys
o Computer-administered ? computer has integral role in posing questions and recording answers
o Computer-assisted ? technology plays a support role in the interviewer's work
o Types
� CATI = computer-assisted telephone interview (interviewer uses compute
3. Self-Administered Surveys
o Respondent completes the survey on his or her own; no human administers the interview
o Types:
� Mail surveys (questionnaires mailed to prospective respondents)
� Group-administered surveys (people take the survey in a group context)
� Drop-off surveys
4. Mixed-mode
o A mixture/hybrid of two or more modes
o Advantages
� Multiple avenues to achieve data collection goal - can access the advantages of each!
� Allows for more complexity
o Disadvantages
� Survey mode may affect response
� Additional complexity (i.e., diff
Apply various factors when choosing a data collection method
� Balance quality against...
� Survey time horizon
� Survey data collection budget
� Type of respondent interaction required (e.g., if you need super detailed stuff, internet is not for you)
� Special considerations
o Incidence rate (% of population posse
Define reliability
� Consistency/stability of measurements
� Extent to which experiment, test, or any measuring procedure yields the same results on repeated trials
� Identical or very similar responses from the same respondent
Define Validity
� Appropriateness/meaningfulness of measurements
� The extent to which any measuring instrument measures what it is intended to measure
� The truthfulness of responses to a measure
Articulate the relationship between reliability and validity
� Validity is usually seen as more important than reliability in the applied field
� Must have reliability in order to have validity (i.e., Reliability is necessary - but not sufficient - for validity)
Describe the different types of reliability
� Test-retest (temporal stability)(very academic - very rarely done in applied field)
o The same test is given to the same people after a period of time. One then obtains the correlation between scores on the two administrations of the same test. It is as
Describe the different types of validity
� Criterion/Predictive Validity (some specific criteria that we're measuring our results again)
o E.g., SAT scores are "validated" by showing that high scorers really do perform better in college
� Content/Face Validity (the extent to which an empirical a
Articulate why reliability and validity are important in applied research
� Scale construction
� Test sensitivity
� Prediction
� Generalizability
Describe the process of designing a quantitative questionnaire
o 1. Question Development
� The practice of wording questions that are:
� Understandable
� Unambiguous
� Unbiased
� It also involves the selection of appropriate response formats (question scaling)
o 2. Question Evaluation (used to scrutinize possible que
List the four "do's" of questionnaire wording
(Questions should...)
� 1. Focus on one topic (e.g., no double-barreled questions)
o Ex: "Did you buy this brand of cat food because it was high-quality and cheap?"
� 2. Be brief
� 3. Be grammatically simple (e.g., no double negatives)
o Ex: "Do you feel
List the four "don'ts" of questionnaire wording
� 1. Don't lead the respondent (i.e., by using questions that have strong cues on how to answer)
o Ex: "Don't you think these new cigarettes are more healthy?"
� 2. Don't uses loaded wording or phrases (i.e., those with emotional overtones)
o Ex: "Do you
Explain the basics of questionnaire organization
� Introduction
o Not a sales pitch - a crucial period because it sets the stage for answering questions!
o 1. Sponsor Identity
� Disguised ? sponsor's name is not divulged; Undisguised ? sponsoring company is identified
o 2. Purpose/content (should be des
Demonstrate how to pre-code questionnaires
� Coding: using numbers associated with/assigned to each unique question response option in order to facilitate data analysis after survey has been conducted
� Rules:
o Set up the coding system before the questionnaire is finalized
o All closed-end questi
Describe the mechanics of piloting a questionnaire
� Piloting and pretesting are totally different!
� Pretest = a dry run of a questionnaire to find and repair difficulties that respondents encounter while taking the survey
� Piloting can be done informally
� Soft-launch
� Start and then stop
� Get feedba
Articulate the key terms in survey design
� "Look and feel" (color, layout, design, etc.)
� Single-select questions (one answer per question)
� Multiple-select questions (check all that apply)
� Grid-style questions (multiples rows of questions, columns at tops)
o Most often single-select questio
Describe the features on which to evaluate survey software
� Ease of use/accessibility (we're not programmers!)
� Look and feel (can you control this? If you're clients are picky, this will be a big consideration!)
� Survey creation (who is doing this? Who can have access to software?)
� Customization (does progr
Compare and contrast the advantages and disadvantages of including open-ended questions
� Advantages:
o The answer categories are not known
o A richer answer is desired
o It is important to have the respondent's own words (can give clients quotes!)
o To give the respondent the opportunity to discuss any topics that may not have been covered
List the situations in which coding can be applied
� Survey data
� Focus group data
� In-depth interviews/one-on-one data
� Archival data
o Public records/documents
o Newspaper/magazine articles or ads
� Sales persons or CSR comments/observations
� Customer comments
� Web comments
Discuss the considerations when coding data
� Specify the objectives of the coding assignment
� Maintain a balance between too much/too little detail (don't want to get lost in detail!)
� Codes should be exhaustive and mutually exclusive (don't want codes that overlap)
� Create enough codes so answ
Apply the process of coding open-ended questions
� Sort the codes to get like responses together
� Look for large groups of similar answers
� Create a code for each large group of responses
� Start coding the data, applying previously created codes
� Revise codes as needed, giving non-fitting responses