arxiv_id,title,authors,year,venue,doi,field,outcome,outcome_label,headline_held,headline_held_earlier_code,headline_total,claims_checked,checked,page_url,evidence_url,badge_url,log_head_hash,log_events,bundle_url,bundle_sha256,summary,attempt_id,qualifiers,qualifier_text,qualifiers_source,coverage_note,assurance_level,release_on,report_json_sha256,superseded_attempts
1609.02907,Semi-Supervised Classification with Graph Convolutional Networks,Thomas N. Kipf; Max Welling,2017,International Conference on Learning Representations (ICLR),,Machine learning,inconclusive,Inconclusive: a registered validity/precision condition was not satisfied,0,0,1,1,2026-10-20,https://openrepro.org/papers/1609.02907/,https://openrepro.org/evidence/arxiv-1609-02907-20261020T035706Z,https://openrepro.org/badge/1609.02907.svg,1ef063a357fbc913ab4a383e0b30ce717a5ce0a6051a021d0fcc90d517955917,44,,,"Prospective re-run of the GCN Citeseer settling test (Table 2, 70.3%) with the plan's precision gate registered as a gate: the authors' train.py at 193c099 (TF 1.0.1 CPU, released early stopping) on a fresh cohort of 100 initializations, seeds 201-300 (101-200 already have results), judged by the sealed interval rule. The fresh mean is 70.372% (SD 4.545, SE 0.454); the enclosing t/bootstrap interval [69.38, 71.27] overlaps the paper's rounding interval, but it extends 0.993 points from the mean, above the 0.731 precision gate, so the result is inconclusive: the cohort does not estimate the mean precisely enough to settle the disputed gap. One of the 100 runs (seed 241, 26.0%, stopped after 15 epochs) is low and stays included.",arxiv-1609-02907-20261020T035706Z,,,report,,"Sealed, bound and re-derived",2026-11-04,6abbfaac06f85da6e238625a4966dae3bc058dcfcfe84f24842c32cbda3d68af,arxiv-1609-02907-20261020T031257Z; arxiv-1609-02907-20261018T022718Z
1506.02142,Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning,Yarin Gal; Zoubin Ghahramani,2016,International Conference on Machine Learning (ICML),,Machine learning,partially_reproduced,Partially reproduced,10,0,12,30,2026-10-20,https://openrepro.org/papers/1506.02142/,https://openrepro.org/evidence/arxiv-1506-02142-20261020T024912Z,https://openrepro.org/badge/1506.02142.svg,0e96b30ad495309e93a1a1d8b07fe06f7f0b1e24e4f4938a272874a1bb4646db,117,,,"Sealed settling test of MC dropout Table 1 (Concrete, Energy, Yacht; RMSE and test LL): 360 paired fits of the authors a6259f1 net.py at batch 32 (paper) and 128 (committed code) on the 20 committed splits with three seeds each, in a pinned Keras 1.1.0/Theano 0.8.2 environment. Five of six batch-32 cells fall within the predeclared allowance of 2 archive SEs (18-way Bonferroni t intervals); Yacht LL is inconclusive because its interval is wider than the allowance, though its point estimate is 0.002 nats from the archive. Batch 128 is outside the allowance in the worse direction for Energy and Yacht and inconclusive for Concrete, and every paired contrast favours batch 32, so batch-size drift explains the earlier Energy and Yacht RMSE gaps. Dropout 0.01 for Concrete/Yacht and the original environment remain unresolved.",arxiv-1506-02142-20261020T024912Z,other_checks_failed; extra_scored_runs,"10 other checks did not hold; a scored run is not the direct source of any recorded value: it may be an upstream stage whose output a later scored job read, or a run that was not used (or more scored runs started than the sealed plan declares); the report names each one",report,,"Sealed, bound and re-derived",2026-11-04,408a9155be2f532270528cb6904c5eb0f308f28d8d5712612731bfeb203eba99,arxiv-1506-02142-20261018T043211Z
2012.07436,Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting,Haoyi Zhou; Shanghang Zhang; Jieqi Peng; Shuai Zhang; Jianxin Li; Hui Xiong; Wancai Zhang,2021,Proceedings of the AAAI Conference on Artificial Intelligence,10.1609/aaai.v35i12.17325,Machine learning,partially_reproduced,Partially reproduced,2,0,3,6,2026-10-20,https://openrepro.org/papers/2012.07436/,https://openrepro.org/evidence/arxiv-2012-07436-20261019T202050Z,https://openrepro.org/badge/2012.07436.svg,cf28ac49bd513cc6b2d4438a5e99cbbfd7f7210680dec7afd136f609a40513f6,69,https://openrepro.org/bundles/arxiv-2012-07436-20261019T202050Z.tar.gz,4582bf8d09afecff41bb76abab7d4828168ebe21ec74fc77df647fb356fc7c86,"Re-run of attempt arxiv-2012-07436-20261019T132134Z on a dedicated 4-vCPU CPU droplet, with the same pre-registered claims, protocol (authors code at c648cc6) and pinned environment (Python 3.7, torch 1.8.0+cpu, numpy 1.19.4, pandas 0.25.1, scikit-learn 0.21.3), running the authors scripts/ETTh1.sh commands with --itr 1 per iteration. ETTh1 multivariate horizon 24 (Table 2), mean of 4 unseeded runs: MSE 0.5046 and MAE 0.5151 vs paper 0.577 and 0.549 (earlier attempt, 1 run: 0.4941 / 0.5111), consistent and better than reported. ETTh1 multivariate horizon 48 (Table 2), untested in the earlier attempt, single run: MSE 0.6021 and MAE 0.5700 vs paper 0.685 and 0.625, consistent and better than reported. The univariate horizon-24 claims (Table 1, seq_len 720) remain untested: at 27.6 s per training step on this box one iteration needs at least ~7.6 h, beyond the budget. No code was modified.",arxiv-2012-07436-20261019T202050Z,bound_checks_only; untested_for_budget; plan_not_sealed; evidence_not_bound; agent_selected_extraction,"every headline check is a one-sided bound, not a match to the published value; some results were left untested for budget; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it",report,Some of the paper's results were left untested to stay within the compute budget; the summary names them.,Legacy,2026-11-05,4bd7297898803b04cf041b10859864f666f0c386cc036a6c30710475af5692d7,arxiv-2012-07436-20261019T132134Z
1905.03222,Conformalized Quantile Regression,Yaniv Romano; Evan Patterson; Emmanuel J. Candès,2019,,,Statistics,reproduced,Reproduced,5,0,5,6,2026-10-20,https://openrepro.org/papers/1905.03222/,https://openrepro.org/evidence/arxiv-1905-03222-20261019T233532Z,https://openrepro.org/badge/1905.03222.svg,1a032d5f5c4c462b52fd0bd3b2c0405dabe9029151ad90ac318c6df4d3518877,55,https://openrepro.org/bundles/arxiv-1905-03222-20261019T233532Z.tar.gz,6f53c01811f7e2608a2cf303739a99398cb0b77ed05bf21ca8a079d82c9af187,"Clean re-run of attempt arxiv-1905-03222-20261019T184203Z with the same protocol and the same six pre-registered claims, and no foreign or withdrawn claims. Ran the authors' reproducible_experiments/run_experiment.py (commit 2b9b5c7, the last before arXiv v1) unchanged except for the same one-line sklearn Imputer patch, on community, concrete and bike with the same rebuilt Python 3.8 stack, 5 splits (2 for bike). All six claims landed within tolerance: CQR quantile-forest length 1.5070 vs 1.55 (community) and 0.5584 vs 0.56 (concrete), CQR quantile-net 1.8561 vs 1.76 (community) and 0.5704 vs 0.57 (bike), split-conformal RF 1.8578 vs 1.94 (community, longer than CQR as reported), and minimum CQR coverage 89.65% against the 90% nominal level. Compared with the earlier attempt, five values are identical to 4 decimals; only the bike quantile-net cell moved (0.5704 vs 0.5695, coverage 89.65 vs 89.26), likely from a different torch thread count. The paper's Table 1 averages over all 11 datasets were not tested.",arxiv-1905-03222-20261019T233532Z,plan_not_sealed; evidence_not_bound; agent_selected_extraction,"legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it",report,,Legacy,2026-11-03,455772afc11c92f854257645c6a912530789f85e37cf4a9bd97d4f28b6b436ff,arxiv-1905-03222-20261019T184203Z
1910.13051,ROCKET: exceptionally fast and accurate time series classification using random convolutional kernels,Angus Dempster; François Petitjean; Geoffrey I. Webb,2020,Data Mining and Knowledge Discovery,10.1007/s10618-020-00701-z,Machine learning,reproduced,Reproduced,3,0,3,6,2026-10-19,https://openrepro.org/papers/1910.13051/,https://openrepro.org/evidence/arxiv-1910-13051-20261019T203424Z,https://openrepro.org/badge/1910.13051.svg,68c626f8d6f80233c3b267a4dde2358204dbe838f3ad78eb1893343a1be27175,63,https://openrepro.org/bundles/arxiv-1910-13051-20261019T203424Z.tar.gz,72ccc2d85ca1af16d393700442d359e6c4e15a64f2d5fa5938b73a6ef21be408,"Corrective re-run that supersedes arxiv-1910-13051-20261017T030206Z. The primary code version was declared first: angus924/rocket 4c24610, the first code commit, 21 hours after arXiv v1; the earlier attempt ran 2022 HEAD with the July 2020 rewrite. All claims were then registered, including the best-mean-rank tie rule taken from the paper before any result. Running the authors' 2019 bake-off code on all 85 UCR datasets (10 runs, single-core processes, dedicated box), mean accuracy is 0.8517 against 0.8510 in Table 1, with a mean absolute per-dataset difference of 0.20 points. Using the paper's fractional-rank method, ROCKET has the strictly best mean rank (571/170 vs 579/170 for TS-CHIEF), so the tie rule (a tie would count as not reproduced) was not triggered. The earlier attempt's tie appears to reflect run-to-run variation of unseeded random kernels on a margin the paper calls not significant. Single-core compute times (40.5 min total, 2.35 min ElectricDevices training, 1.81 min HandOutlines) are within the paper's figures on newer hardware.",arxiv-1910-13051-20261019T203424Z,claims_registered_after_output; plan_not_sealed; evidence_not_bound; agent_selected_extraction,"some claims were registered after command output had been observed; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it",report,,Legacy,2026-11-03,8c8f2c5ea24a62319df59979a3cbce1e9b024f469fb28b993763cd5ba1c7bfef,arxiv-1910-13051-20261017T030206Z
1803.09015,Difference-in-Differences with multiple time periods,Brantly Callaway; Pedro H. C. Sant'Anna,2020,Journal of Econometrics,10.1016/j.jeconom.2020.12.001,Economics,reproduced,Reproduced,4,0,4,7,2026-10-19,https://openrepro.org/papers/1803.09015/,https://openrepro.org/evidence/arxiv-1803-09015-20261019T161404Z,https://openrepro.org/badge/1803.09015.svg,48f311848a704e6fdadc0137322dca91d96cfc105c4e13706dc6e39dbaa4c0d7,59,https://openrepro.org/bundles/arxiv-1803-09015-20261019T161404Z.tar.gz,000a0f63f36a4772d246e0a755f9ac15047ec254ba5b968461779ac2dac43897,"The minimum-wage application (Section 5, Tables 2-3) was rerun with the did package at commit 5e32efd (the last commit before arXiv v4), R 4.0.3 and 2020-12-01 CRAN dependencies, on the 2284-county 2001-2007 panel that co-author Sant'Anna publishes in the CS_RR repository (the bundled mpdta is only a 500-county subset). All 7 pre-registered claims landed within tolerance: the unconditional simple average (-0.051581 vs -0.052), event-study average (-0.089810 vs -0.090), ATT(2004,2007) (-0.135614 vs -0.136), the conditional doubly robust simple average (-0.033117 vs -0.033), both bootstrap SEs, and the Table 2 treated population mean. A post-hoc, unscored comparison found that Table 3's calendar-time rows mix two weightings: Panel (a) matches the paper's eq. (3.8) group weights, while Panel (b) matches the unweighted mean the 2020 package computed.",arxiv-1803-09015-20261019T161404Z,claims_registered_after_output; plan_not_sealed; evidence_not_bound; agent_selected_extraction,"some claims were registered after command output had been observed; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it",report,,Legacy,2026-11-03,77a755fdb0adeeaa610ccba036687c40e2be44331b0f55ee5bfd3f183742b80d,
1708.05031,Neural Collaborative Filtering,Xiangnan He; Lizi Liao; Hanwang Zhang; Liqiang Nie; Xia Hu; Tat-Seng Chua,2017,,,Recommender systems,reproduced,Reproduced,4,0,4,6,2026-10-19,https://openrepro.org/papers/1708.05031/,https://openrepro.org/evidence/arxiv-1708-05031-20261019T135218Z,https://openrepro.org/badge/1708.05031.svg,04ac3824830007550ce23545b33c04ec2f8c290dd7107cca03355bd8ce4ddfc6,67,https://openrepro.org/bundles/arxiv-1708-05031-20261019T135218Z.tar.gz,526c1df742b41900055aa6ca2ab00d065b10fd744f3fb163204fd9df964561cd,"The authors Keras code (commit 312aff0, identical in code to the current default branch) was run on MovieLens-1M with the exact README environment (Python 2.7.15, Keras 1.0.7, Theano 0.8.0) on CPU, using the README commands with 8 predictive factors and 20 epochs. All six pre-registered Table 2/3/4 values at 8 factors landed within 0.005 of the paper: NeuMF without pre-training HR@10 0.6848 vs 0.688 and NDCG@10 0.4088 vs 0.410; NeuMF with pre-training (initialised from the authors bundled GMF/MLP weights rather than retrained ones) 0.6798 vs 0.684 and 0.4054 vs 0.403; MLP-3 0.6707 vs 0.671 and 0.3979 vs 0.399. Each value is one run reported at the best-HR epoch, which is the scripts own convention. Mid-attempt the shared cloud droplet was destroyed by another process; the environment and code were rebuilt with the same recipe and only the in-flight MLP run was repeated.",arxiv-1708-05031-20261019T135218Z,claims_registered_after_output; plan_not_sealed; evidence_not_bound; agent_selected_extraction,"some claims were registered after command output had been observed; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it",report,,Legacy,2026-10-19,9415526a0cc89ac96f5f036ec663c59c3ffed64312cb6452053aeba2ad3f724f,
1606.00931,DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network,Jared Katzman; Uri Shaham; Alexander Cloninger; Jonathan Bates; Tingting Jiang; Yuval Kluger,2018,BMC Medical Research Methodology,10.1186/s12874-018-0482-1,Medical statistics,reproduced,Reproduced,4,0,4,6,2026-10-19,https://openrepro.org/papers/1606.00931/,https://openrepro.org/evidence/arxiv-1606-00931-20261018T225505Z,https://openrepro.org/badge/1606.00931.svg,b6c148f9a6936f84ea439e56d1c64f0a7dcb49e564e06bcc17ae75d7dd7f225f,63,https://openrepro.org/bundles/arxiv-1606-00931-20261018T225505Z.tar.gz,a3ec5893321c321b834852634486c0dfe6d3c97bdcdac039cbf2375a469fbe07,"Rerunning the first author code at the commit matching arXiv v3 (079dee7, 2017-08-08), with the documented command for each experiment in a reconstructed Python 2.7 / Theano 0.9 / Lasagne environment on CPU, reproduces all six DeepSurv C-index values of Table 1 within 0.01: mean of 10 unseeded runs SUPPORT 0.6186 vs 0.6183, METABRIC 0.6435 vs 0.6434, Rotterdam & GBSG 0.6720 vs 0.6684, WHAS 0.8580 vs 0.8626 (run-to-run sd 0.009), simulated linear 0.7739 vs 0.7740, simulated nonlinear 0.6458 vs 0.6489. The WHAS model file in the repository does not match the WHAS hyper-parameters listed in the paper Table 3, and every run ends with a weight-saving error after the metrics are printed (fixed upstream in December 2017). CPH, RSF and the treatment-recommendation results were not scored.",arxiv-1606-00931-20261018T225505Z,claims_registered_after_output; plan_not_sealed; evidence_not_bound; agent_selected_extraction,"some claims were registered after command output had been observed; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it",report,,Legacy,2026-10-19,4618c43c82c309831531bc345505686abff25b5cf1658e4033b8c092f0f678d0,
1711.10561,Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations,Maziar Raissi; Paris G. Perdikaris; George Em Karniadakis,2017,,,Scientific machine learning,reproduced,Reproduced,1,0,1,2,2026-10-19,https://openrepro.org/papers/1711.10561/,https://openrepro.org/evidence/arxiv-1711-10561-20261018T234638Z,https://openrepro.org/badge/1711.10561.svg,450dad5c4edbfc2ab0c9dce568dcddd793d19978beba85e4a64f26f6724759da,26,https://openrepro.org/bundles/arxiv-1711-10561-20261018T234638Z.tar.gz,1fd033882f810ccf26d9576ce635734e5c32b49e352470ea0a27c56f6118d9b0,"Running the authors Burgers.py (the earliest version in their repository, from five months after arXiv v1) with its fixed seeds on CPU gives a relative L2 error of 2.15e-3 for the continuous-time Burgers example, against 6.7e-4 reported. That meets the pre-registered order-of-magnitude headline check (at most 6.7e-3) but not the stricter within-a-factor-of-2 check. The paper own Table 1 shows errors varying between 1e-4 and 1e-2 across neighbouring settings, and this run used a CPU rather than the reported GPU. The Table 1 sweep was not run.",arxiv-1711-10561-20261018T234638Z,bound_checks_only; broad_bound; other_checks_failed; plan_not_sealed; evidence_not_bound; agent_selected_extraction,"Burgers error 3.21× the published value; passes a registered 10× ceiling; a stricter 2× check did not hold; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it",report,,Legacy,2026-10-19,c0df97b17edb7b847d7e7b5cf3dc788c18d3f4960563ba5ff3ce186d721e7037,
1607.01759,Bag of Tricks for Efficient Text Classification,Armand Joulin; Édouard Grave; Piotr Bojanowski; Tomáš Mikolov,2017,Conference of the European Chapter of the Association for Computational Linguistics,10.18653/v1/e17-2068,Natural language processing,reproduced,Reproduced,8,0,8,16,2026-10-18,https://openrepro.org/papers/1607.01759/,https://openrepro.org/evidence/arxiv-1607-01759-20261018T223713Z,https://openrepro.org/badge/1607.01759.svg,87b5ac6197ec5f210cb74ba6aaa5412fe8e2237bdf845f1afee95be9043de61d,56,https://openrepro.org/bundles/arxiv-1607-01759-20261018T223713Z.tar.gz,31c2f0bbfa19009868e934d406a85c850c9e430dd90b79022e1977909a00cad4,"With fastText at the commit of arXiv v3 and its classification-results.sh, test accuracy with 10 hidden units and bigrams reproduces Table 1 on all eight datasets: 92.2, 96.7, 98.6, 95.7, 63.9, 72.4, 60.2 and 94.6 against 92.5, 96.8, 98.6, 95.7, 63.9, 72.3, 60.2 and 94.6. The unigram row also reproduces, within 0.2 points everywhere. The script own Google Drive download no longer works, so the datasets came from a public mirror of the same tarballs, preprocessed with the script own function. Training speed (Table 2) was not tested.",arxiv-1607-01759-20261018T223713Z,plan_not_sealed; evidence_not_bound; agent_selected_extraction,"legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it",report,,Legacy,2026-10-19,d4c54196bf7f87246ae7f1b2123994691f4b3adb69a4f9def94e8ba9c76e3b9a,
2004.01105,Inferring change points in the spread of COVID-19 reveals the effectiveness of interventions,Jonas Dehning; Johannes Zierenberg; F. Paul Spitzner; Michael Wibral; João Pinheiro Neto; Michael Wilczek; Viola Priesemann,2020,Science,10.1126/science.abb9789,Epidemiology,reproduced,Reproduced,4,0,4,7,2026-10-18,https://openrepro.org/papers/2004.01105/,https://openrepro.org/evidence/arxiv-2004-01105-20261018T033025Z,https://openrepro.org/badge/2004.01105.svg,90e4d597839451c42440893ea4dea7451ef11eaedc3887da05aa3c80dcc11e78,44,https://openrepro.org/bundles/arxiv-2004-01105-20261018T033025Z.tar.gz,133120d2279425834b961d4304c7e6d626fbf8f190ff67d6e32ec166bc6ce68f,"Fitting the authors three-change-point SIR model with weekly reporting modulation (repository state on the date of arXiv v3) to German case data until 21 April 2020, using the JHU snapshot bundled with the repository, reproduces all registered values: spreading rates 0.43, 0.25, 0.15 and 0.09 (posterior medians 0.428, 0.251, 0.154, 0.094) and change points around March 6, 15 and 23 (posterior medians March 5.8, 15.2 and 22.6). The credible intervals also match Fig. 3 closely. Only the main three-change-point model was refitted; the 0-2 change-point comparisons and forecasts were not.",arxiv-2004-01105-20261018T033025Z,plan_not_sealed; evidence_not_bound; agent_selected_extraction,"legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it",report,,Legacy,2026-10-19,4f7abfc274235e7f7089ee79e78a693b99a8e4c736292bb44e7f806cb6dfae3d,
1810.08473,From Louvain to Leiden: guaranteeing well-connected communities,V. A. Traag; L. Waltman; Nees Jan van Eck,2019,Scientific Reports,10.1038/s41598-019-41695-z,Network science,reproduced,Reproduced,4,0,4,6,2026-10-18,https://openrepro.org/papers/1810.08473/,https://openrepro.org/evidence/arxiv-1810-08473-20261018T013946Z,https://openrepro.org/badge/1810.08473.svg,578432e9062b72e8e2576db5eeed0bf5ac96e6f5b649204a2e77964f446edc6c,42,https://openrepro.org/bundles/arxiv-1810-08473-20261018T013946Z.tar.gz,44a9835769081eadd17b22c481374d07d8050ab73869c91bdc82b5ea64e9bca3,"Using the authors Java implementation at the release tag matching the paper (networkanalysis 1.0.0) on the SNAP DBLP and Amazon networks, with the paper protocol of 10 random starts of 10 iterations each, all tested Table II values reproduced within 0.0005: DBLP Louvain 0.8267 vs 0.8262 and Leiden 0.8384 vs 0.8387; Amazon Louvain 0.9298 vs 0.9301 and Leiden 0.9340 vs 0.9341. Leiden found higher modularity than Louvain on both networks, as claimed. The other four networks in Table II were out of scope (download blocked, proprietary, or too large for the sandbox), and the badly connected community percentages in Fig. 4 were not tested.",arxiv-1810-08473-20261018T013946Z,broad_bound; plan_not_sealed; evidence_not_bound; agent_selected_extraction,"DBLP Leiden gain over Louvain 0.938× the published value; passes a registered floor of 0, a sign check; Amazon Leiden gain over Louvain 1.04× the published value; passes a registered floor of 0, a sign check; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it",report,,Legacy,2026-10-19,7578c2dcc994acbddf7fe022f261bda0a1b18de84da38b67c401c0f5cee29cfd,
1812.09970,Synthetic Difference-in-Differences,Dmitry Arkhangelsky; Susan Carleton Athey; David A. Hirshberg; Guido W. Imbens; Stefan Wager,2021,American Economic Review,10.1257/aer.20190159,Economics,reproduced,Reproduced,4,0,4,10,2026-10-18,https://openrepro.org/papers/1812.09970/,https://openrepro.org/evidence/arxiv-1812-09970-20261018T010328Z,https://openrepro.org/badge/1812.09970.svg,8b90233f1aba9384bcb8d580e05cbaaf9a5947ebf64d2d62c4f0f4b157f1f7cb,59,https://openrepro.org/bundles/arxiv-1812-09970-20261018T010328Z.tar.gz,b5ba1fe35f4c450bce1c3d66546b117baf1d15a8633fec07e878292fe03fd96a,"Running the authors own Table 1 code (vignettes/paper-results.Rmd) at the synthdid commit named in their session-info, with R 4.0.5 and the pinned MCPanel and glmnet commits, reproduced every entry of Table 1 at the reported precision: SDID -15.6 (8.4), SC -19.6 (9.9), DID -27.3 (17.7), MC -20.2 (11.5), DIFP -11.1 (9.5). Point estimates differ from the paper by at most 0.05, and placebo standard errors by at most 0.05. Deviations were limited to platform (x86_64 Linux instead of an M1 Mac) and one non-estimating build dependency. The paper-era synthdid commit is no longer on any branch of the repository but is still retrievable from GitHub by its hash.",arxiv-1812-09970-20261018T010328Z,plan_not_sealed; evidence_not_bound; agent_selected_extraction,"legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it",report,,Legacy,2026-10-19,1ea4c3c995f66424b59dccd235fa8015529c6ea76038251bc7b787f6a292a9ae,