Research
Design in Social Research
Table of Contents
Part I WHAT IS
RESEARCH DESIGN?
1 The Context of Design
2 Tools for Research Design
3 Causation and the Logic of Research Design
Part II
EXPERIMENTAL DESIGNS
4 Types of Experimental Designs
5 Issues in Experimental Design
6 Analysing Experimental Data
Part III
LONGITUDINAL DESIGNS
7 Types of Longitudinal Designs
8 Issues in Longitudinal Designs
9 Data Analysis Issues in Longitudinal Designs
PART IV
CROSS-SECTIONAL DESIGNS
10 Cross-Sectional Design
11 Issues in Cross-Sectional Design
12 Cross-Sectional Analysis
PART V CASE STUDY
DESIGN
13 Case Study Designs
14 Issues in Case Study Design
15 Case Study Analysis
DESCRIPTION AND
EXPLANATION
Descriptive research
Explanatory research
Prediction, correlation and causation
Deterministic and probabilistic and
concepts of causation
THEORY TESTING
AND THEORY CONSTRUCTION
Theory Building
Theory Testing
WHAT IS RESEARCH
DESIGN?
Design vs method
Quantitative and qualitative research
Adopting a sceptical approach to explanations
Plausible rival hypotheses
The fallacy of affirming the
consequent
Falsification: looking for evidence
to disprove the theory
The provisional nature of support for
theories
Focussing and clarifying the research question
Focussing descriptive research
questions
Focussing explanatory research
questions
Searching for
causes or effects
Exploring a
simple causal proposition
More complex
causal models
Ideographic
and nomothetic explanations
Identifying plausible rival hypotheses
Theoretical
and substantive rivals
Technical /
methodological rivals
Operationalization
Clarifying
concepts
Nominal
definitions
Operational
definitions
CONCEPTS FOR
RESEARCH DESIGN
Internal Validity
External validity
Measurement error
Types of measurement error
Validity
Reliability
Forms of measurement error
INFERRING CAUSAL
RELATIONSHIPS
Criteria for inferring cause
Co-variation
It must make sense
Time order
Dependent variable must be capable of
change
Theoretical plausibility
Types of causal patterns
Direct and indirect causal
relationships
Types of relationships in a three
variable model
PROVIDING A FRAME
OF REFERENCE
Comparing groups
Multiple comparison groups
Comparing time points
Multiple pre and post tests
Making meaningful comparisons
Matching
Ex post facto matching
Randomisation
Matched Block designs
Statistical controls
Interventions and independent variables
Types of independent variables
Number of interventions
DIMENSIONS OF A
RESEARCH DESIGN
A RANGE OF
RESEARCH DESIGNS
Experimental design
Longitudinal design
The cross sectional design
Case studies
THE CLASSIC
EXPERIMENTAL DESIGN
Experimental contexts
Laboratory experiments
Field experiments
Natural experiments
SIMPLER
EXPERIMENTAL DESIGNS
Post test only with control group
Retrospective experimental design
MORE COMPLEX
EXPERIMENTAL DESIGNS
Multiple post-tests
Multiple groups
Solomon four-group design
Factorial designs
METHODOLOGICAL
ISSUES
The problem of explanatory narrowness
Randomisation makes it difficult to
establish the role of other factors
Randomisation can underestimate the
total causal effect
Problems with internal validity
History
Maturation
Testing
Instrument decay
Statistical regression
Selection
Mortality/drop out
Problems with external validity
Reactive effect of pre testing
Unrepresentativeness
Artificiality
PRACTICAL ISSUES
How much do you tell participants?
How many participants?
How should participants be recruited?
Gaps between tests and interventions
Which method of data collection?
Problems with randomized assignment
Limits of random allocation
Gatekeepers and threats to random
allocation
Control group resentment
Refusals to participate in the
intervention
Drop outs
Unevenness of interventions
The self fulfilling prophecy
ETHICAL MATTERS
Voluntary participation
Informed consent
No harm to participants
Anonymity and confidentiality
SELECTING THE
METHOD OF ANALYSIS
What kind of analysis is required?
Description or inference?
What sort of sample do we have?
What level of data do we have?
Interval/ratio
Ordinal variables
Nominal variables
Type of comparisons required?
Differences in central tendency
Differences in variability
Shape
Proportions
Association between variables
What type of display is required?
Tabular
Graphical/ chart
Summary statistic
How many groups are to be compared?
One group comparisons with a known
value
Two
Three or more groups
Are the comparison groups independent
samples?
Independent
samples designs
Paired
samples designs
Is the dependent variable normally
distributed?
Is the variance on the dependent
variable similar between each comparison group?
How many independent variables?
Independent impact of multiple
independent variables
Removing the effects of other
variables
SUMMARY TABLE:
SELECTING THE RIGHT MEASURE
Ten questions
OTHER ISSUES
Change scores
Truncation effects (floor and ceiling effects)
Trend analysis
PURPOSES OF
LONGITUDINAL DESIGN
Describing patterns of change and stability
Establishing temporal order
Establishing developmental (age) effects
Establishing historical (period) effects
Life course 'career' analysis
TYPES OF
LONGITUDINAL DESIGN
Prospective Panel Designs
Simple prospective panel design
Multiple point prospective panel
design
Single panel design without
replacement
Single panel design with replacement
Rotating panel design
Single cohort design
Multiple cohort design
Cohort sequential design
Retrospective Designs
Retrospective Panel
Record linkage designs
Quasi longitudinal designs
Simulated before-after design
Repeated Cross Sectional
METHODOLOGICAL
Issues of internal validity
Absence of randomized control groups
History
Maturation
Testing or Panel Conditioning
Instrumentation
Measurement error
Regression
Mortality/ drop-out
Issues of external validity
Panel attrition
Panel conditioning
Immigration and emigration
PRACTICAL
Standardization of instruments
Panel attrition
Respondent burden
Respondent Recall
Cost
Method of data collection
Number of waves
Gap between waves
Sample error
Sample size
Instrument design
Staffing
ETHICAL ISSUES
Voluntary participation
Informed Consent
Harm to participants
Confidentiality and anonymity
MISSING DATA
Sources of missing data
Item non response
Unit non-response
Why a problem?
Sample size
Bias
Identifying missing data bias
Dealing with missing data
Imputation
Weighting
Introducing statistical controls for
biasing variables
MEASURING CHANGE
Aggregate level vs individual level change
Qualitative cf quantitative change
Measuring change in Panel designs
Measuring change
Raw change scores
Residual change scores
Percentage change
Distinguishing between real change
and lack of reliability
How much change is change?
Standardisation and scaling
Adjusting for 'inflation'
Standardisation: z-scores and
percentile rank
Explaining change
DESCRIBING CHANGE
Tables
Graphical
NO TIME DIMENSION
RELIANCE ON
EXISTING DIFFERENCES
THE NATURE OF
'GROUPS' IN THE CROSS SECTIONAL DESIGN
OBTAINING A TIME
DIMENSION: REPEATED CROSS SECTIONAL STUDIES
METHODOLOGICAL
ISSUES
Internal validity
Establishing causality and
controlling confounding variables
Statistical controls
Eliminating variables as causes
Models: a priori reasoning and ad hoc
reasoning
Establishing causal direction
Adequacy at the level of meaning
External validity
Representativeness
PRACTICAL ISSUES
Method of collecting data: which method of data collection
Sample sizes
Sub group analysis
Precision and accuracy of estimates
Sufficient variation in the sample on key variables
Information for statistical controls
Length
Types of data
ETHICAL ISSUES
DESCRIPTIVE
ANALYSIS
How many?
Level of detail
Form of data
Who?
Factor structures/scale structures:
How general? How close?
EXPLANATORY
ANALYSIS
The logic of statistical controls
Multiple statistical controls
THE ELABORATION
TECHNIQUE
Basic approach
Specification
Replication
Indirect causal relationships
Spuriousness
Interpretation of patterns
Problems with the elaboration model
MULTIVARIATE
ANALYSIS
COHORT ANALYSIS
Constructing and reading cohort tables
Ageing effects
Period effects
Cohort effects
Unevenly spaced surveys
Problems with cohort analysis
WHAT IS A CASE?
Units of analysis
Holistic and embedded units of analysis
CASE STUDIES AND
THEORY
Explanatory case studies
Theory testing
Theory building case studies
Clinical case studies
Descriptive case studies
Description and theory
Typologies and ideal types
Inductive typologies
OTHER ELEMENTS OF
CASE STUDY DESIGNS
Single or multiple cases?
Parallel or sequential?
Retrospective or prospective?
TYPES OF CASE
STUDY DESIGNS
WHAT A CASE STUDY
IS NOT
The one shot case study
Not a data collection method
METHODOLOGICAL
ISSUES
Internal validity
Idiographic and nomothetic
explanations
Wholes, not just parts
History and maturation
Reactive effects
External validity
Theoretical and statistical
generalisation
Replication
Strategic selection of cases
PRACTICAL
Sampling
Method of case selection
Number of cases
Case screening
Cost and access
Number of investigators: getting consistency
When to go into the field
Presenting case studies
ETHICAL
Statistical analysis
Meaning and context
ANALYSIS IN
DESCRIPTIVE CASE STUDIES
Theoretical dimension of descriptive analysis
Ideal type analysis
Typologies
Time ordered descriptions
EXPLANATORY CASE
STUDIES
Theory testing analysis
Pattern matching
Simple patterns
More complex patterns: multiple
independent variables
More complex patterns: multiple
dependent variables
Highly complex pattern matching
Refining theories
Time series analysis
Literal and theoretical replication
ANALYSIS FOR
THEORY BUILDING: ANALYTIC INDUCTION
