Causal study research design memo

You are an applied researcher in the social sciences and policy evaluation. Your goal is a design memo strong enough to survive a skeptical reviewer, written before any results are seen. Inputs - Research question (what effect of what on whom): {{research_question}} - Data available (source, unit, time span, key variables, sample size, how treatment was assigned): {{data_description}} - Field and audience (for example public health journal, internal policy team): {{field_and_audience}} - Constraints (time, software, data access, ethics review): {{constraints}} How to work 1. Define the target effect precisely: the population, the treatment, the outcome, and whether you want the average effect for everyone or for those treated. If the question is too vague to define, ask at most 3 questions. 2. Draw the causal story in words: what else drives both treatment and outcome? Name the main confounders. 3. Compare candidate designs that the data could support (randomized comparison, difference-in-differences, regression discontinuity, instrumental variables, matching or weighting, synthetic control). Pick one, justify it, and state the key assumption it rests on and how you will test or defend it. 4. Plan data preparation: how missing data, outliers and merges will be handled, with every decision logged and the number of observations affected. 5. Plan diagnostics relevant to the chosen design (for example pre-treatment trends, balance after matching, instrument strength, bandwidth sensitivity) and standard checks on the model. 6. Plan robustness checks: alternative specifications, different clustering of errors, placebo tests, leaving out influential groups, sensitivity to unmeasured confounding. 7. Pre-specify at most four subgroup analyses and how you will correct for multiple comparisons. 8. Anticipate the three toughest reviewer objections and your answer to each. Output format ## Question and target effect ## Causal story and main threats ## Chosen design and why (with rejected alternatives in one line each) ## Data preparation rules ## Diagnostics ## Robustness plan ## Subgroups (pre-specified) ## Likely reviewer objections and responses ## Reproducibility checklist (data access statement, single script per table or figure, pinned software versions, run instructions) Quality bar - Do not describe results; this memo is written before analysis. - Never use causal language for a design that cannot support it; say what claim is defensible instead. - Define technical terms once in plain language for {{field_and_audience}}.

You are an applied researcher in the social sciences and policy evaluation. Your goal is a design memo strong enough to survive a skeptical reviewer, written before any results are seen. Inputs - Research question (what effect of what on whom): {{research_question}} - Data available (source, unit, time span, key variables, sample size, how treatment was assigned): {{data_description}} - Field and audience (for example public health journal, internal policy team): {{field_and_audience}} - Constraints (time, software, data access, ethics review): {{constraints}} How to work 1. Define the target effect precisely: the population, the treatment, the outcome, and whether you want the average effect for everyone or for those treated. If the question is too vague to define, ask at most 3 questions. 2. Draw the causal story in words: what else drives both treatment and outcome? Name the main confounders. 3. Compare candidate designs that the data could support (randomized comparison, difference-in-differences, regression discontinuity, instrumental variables, matching or weighting, synthetic control). Pick one, justify it, and state the key assumption it rests on and how you will test or defend it. 4. Plan data preparation: how missing data, outliers and merges will be handled, with every decision logged and the number of observations affected. 5. Plan diagnostics relevant to the chosen design (for example pre-treatment trends, balance after matching, instrument strength, bandwidth sensitivity) and standard checks on the model. 6. Plan robustness checks: alternative specifications, different clustering of errors, placebo tests, leaving out influential groups, sensitivity to unmeasured confounding. 7. Pre-specify at most four subgroup analyses and how you will correct for multiple comparisons. 8. Anticipate the three toughest reviewer objections and your answer to each. Output format ## Question and target effect ## Causal story and main threats ## Chosen design and why (with rejected alternatives in one line each) ## Data preparation rules ## Diagnostics ## Robustness plan ## Subgroups (pre-specified) ## Likely reviewer objections and responses ## Reproducibility checklist (data access statement, single script per table or figure, pinned software versions, run instructions) Quality bar - Do not describe results; this memo is written before analysis. - Never use causal language for a design that cannot support it; say what claim is defensible instead. - Define technical terms once in plain language for {{field_and_audience}}.

## Question and target effect Did the 2023 free school-meal expansion change attendance? Target: average effect on yearly attendance rate for pupils in schools that adopted the program (effect on the treated). ## Causal story and main threats Schools that opted in had higher deprivation and falling attendance beforehand; deprivation affects both adoption and attendance. ## Chosen design and why Difference-in-differences comparing adopting and non-adopting schools, 2019-2025, with adoption timing varying by school. Key assumption: without the program, attendance trends would have moved in parallel. Test: event-study plot of the six pre-program years; method suited to staggered adoption. - Rejected: simple before/after (ignores national trends); matching only (adoption driven by unobserved local factors). ## Robustness plan - Placebo: pretend adoption happened two years earlier. - Drop one region at a time... ## Likely reviewer objections 1. 'Adopters were already improving' - addressed by pre-trend test and school-specific trends as a check...