Download claims.csv from openrepro/openrepro-ledger: direct link, hf CLI and curl.
- Browser
- Download file 81.5 kB
-
https://huggingface.co/datasets/openrepro/openrepro-ledger/resolve/main/claims.csv
- Command line
-
hf download hf://datasets/openrepro/openrepro-ledger/claims.csv
-
curl -L -o claims.csv https://huggingface.co/datasets/openrepro/openrepro-ledger/resolve/main/claims.csv
81.5 kB
| arxiv_id,paper,claim_id,headline,role,statement,location,metric,unit,comparison,published,observed,tolerance,accepted_if,verdict,counted,earlier_code_observed,registered_event,first_result_event,deviation_over_tolerance,check_kind,broad_bound,ratio_to_published,attempt_id | |
| 1609.02907,GCN,gcn-citeseer-compat,True,claim,"GCN (fixed Planetoid split, mean of 100 random weight initializations) reaches 70.3% test accuracy on Citeseer; judged by whether the final uncertainty interval of a fresh 100-initialization mean overlaps the paper's rounding interval [70.25, 70.35].","Table 2, p. 7, row 'GCN (this paper)', Citeseer column; protocol on p. 7 ('report mean accuracy of 100 runs with random weight initializations')","The point of the final interval [L, H] nearest to 70.3, i.e. min(max(70.3, L), H), in percentage points. It lies within 0.05 of 70.3 exactly when [L, H] overlaps [70.25, 70.35] (the settling plan's compatibility rule, section 4). Fresh cohort: seeds 201..300 inclusive (100 fits, one fresh process and TF graph/session per seed, NumPy and TF seed = the seed, released default stopping --early_stopping 10, final model, no checkpoint restoration). Per-seed canonical accuracy a_i = 100*correct/test_n in percentage points, where correct = number of the 1,000 original test-mask nodes whose argmax logit equals argmax of the original y_test row, test_n must be 1000. mu = unweighted mean of the 100 a_i; s = sample SD (ddof=1); SE = s/10; t interval mu +/- t_99(0.975)*SE (scipy.stats.t.ppf(0.975, 99)); percentile bootstrap: numpy 1.26.4 RandomState(160902), idx = rs.randint(0, 100, size=(50000, 100)), resample means, np.percentile(means, [2.5, 97.5]) with linear interpolation; final interval [L, H] = [min(t_low, boot_low), max(t_high, boot_high)] (the enclosing interval, never the narrower one). All 100 runs are included; no survivor-only mean, no clipping, no replacement of low runs.","percent (test accuracy, percentage points)",within,70.3,70.3,0.05,within ±0.05,consistent,False,,7,27,0.0,equivalence,False,1.0,arxiv-1609-02907-20261020T035706Z | |
| 1609.02907,GCN,gcn-citeseer-precision,False,gate,Precision gate of the settling plan for the Citeseer 70.3% cell: the final uncertainty interval of the fresh 100-initialization mean extends no more than 0.731 percentage points from that mean (the size of the historical 70.3 - 69.569 gap the test is meant to resolve); otherwise the comparison is inconclusive.,"Table 2, p. 7, row 'GCN (this paper)', Citeseer column; gate = 70.3 minus our prior mean 69.569 (attempt arxiv-1609-02907-20261018T022718Z, event 29); settling plan section 4","max(mu - L, H - mu), in percentage points, for the fresh cohort mean mu and final interval [L, H]. 'paper_value' here is the plan's gate 0.731 (not a number printed in the paper). Fresh cohort: seeds 201..300 inclusive (100 fits, one fresh process and TF graph/session per seed, NumPy and TF seed = the seed, released default stopping --early_stopping 10, final model, no checkpoint restoration). Per-seed canonical accuracy a_i = 100*correct/test_n in percentage points, where correct = number of the 1,000 original test-mask nodes whose argmax logit equals argmax of the original y_test row, test_n must be 1000. mu = unweighted mean of the 100 a_i; s = sample SD (ddof=1); SE = s/10; t interval mu +/- t_99(0.975)*SE (scipy.stats.t.ppf(0.975, 99)); percentile bootstrap: numpy 1.26.4 RandomState(160902), idx = rs.randint(0, 100, size=(50000, 100)), resample means, np.percentile(means, [2.5, 97.5]) with linear interpolation; final interval [L, H] = [min(t_low, boot_low), max(t_high, boot_high)] (the enclosing interval, never the narrower one). All 100 runs are included; no survivor-only mean, no clipping, no replacement of low runs.","percentage points (precision gate, settling plan section 4)",at_most,0.731,0.992999999999995,0.0,at most 0.731,inconsistent,False,,8,28,inf,bound,False,1.358413,arxiv-1609-02907-20261020T035706Z | |
| 1610.01271,Generalized Random Forests,cgrf-p20-n800-p-ci-lower,True,claim,"Plan compatibility rule, part 1: the lower end of the literal-paper-DGP (P) interval for C. GRF MSE x 10 at p=20, n=800 is at most 0.935, the top of the rounding interval of the published 0.93.","Table 1, p. 23 (row conf.=yes, heterog.=yes, p=20, n=800; column C. GRF)","P_ci_lower = P mean - q*SE_P. C. GRF test MSE x 10, cell confounding=yes, heterogeneity=yes, p=20, n=800. Per replication r=1..60: 10*mean((prediction - tau)^2) over 1,000 test points, centered causal_forest of grf 0.9.5 with 2,000 trees. Arm P = literal paper DGP (Sec. 6.2: e=(1+dbeta(X3,2,4))/4, m=2X3-1, tau=z(X1)z(X2), Y=m+(W-0.5)tau+eps, no rescaling); arm H = 2020 aw3 helper (dgps.R at e93a79fe, sigmas=1). Interval = estimate +/- q*SE, SE = sample SD/sqrt(60) over replication values (paired differences for H-P), q = qt(1-0.05/(2*3), 59).",MSE x 10,at_most,0.93,0.9046276596195867,0.005,at most 0.935,consistent,True,,5,33,-5.0745,bound,False,0.972718,arxiv-1610-01271-20261020T030230Z | |
| 1610.01271,Generalized Random Forests,cgrf-p20-n800-p-ci-upper,True,claim,"Plan compatibility rule, part 2: the upper end of the literal-paper-DGP (P) interval for C. GRF MSE x 10 at p=20, n=800 is at least 0.925, the bottom of the rounding interval of the published 0.93.","Table 1, p. 23 (row conf.=yes, heterog.=yes, p=20, n=800; column C. GRF)","P_ci_upper = P mean + q*SE_P. C. GRF test MSE x 10, cell confounding=yes, heterogeneity=yes, p=20, n=800. Per replication r=1..60: 10*mean((prediction - tau)^2) over 1,000 test points, centered causal_forest of grf 0.9.5 with 2,000 trees. Arm P = literal paper DGP (Sec. 6.2: e=(1+dbeta(X3,2,4))/4, m=2X3-1, tau=z(X1)z(X2), Y=m+(W-0.5)tau+eps, no rescaling); arm H = 2020 aw3 helper (dgps.R at e93a79fe, sigmas=1). Interval = estimate +/- q*SE, SE = sample SD/sqrt(60) over replication values (paired differences for H-P), q = qt(1-0.05/(2*3), 59).",MSE x 10,at_least,0.93,1.123609257851831,0.005,at least 0.925,consistent,True,,6,34,38.7219,bound,False,1.208182,arxiv-1610-01271-20261020T030230Z | |
| 1610.01271,Generalized Random Forests,cgrf-p20-n800-p-precision-gate,True,claim,"Plan precision gate: the half-width q*SE_P of the P interval is at most 0.11858505, half the previously observed 1.1671701-0.93 gap. paper_value is the plan's gate threshold, not a published number; the quote is the caption's 60-replication design that the gate is about.","Table 1 caption, p. 23","P_halfwidth = q*SE_P. C. GRF test MSE x 10, cell confounding=yes, heterogeneity=yes, p=20, n=800. Per replication r=1..60: 10*mean((prediction - tau)^2) over 1,000 test points, centered causal_forest of grf 0.9.5 with 2,000 trees. Arm P = literal paper DGP (Sec. 6.2: e=(1+dbeta(X3,2,4))/4, m=2X3-1, tau=z(X1)z(X2), Y=m+(W-0.5)tau+eps, no rescaling); arm H = 2020 aw3 helper (dgps.R at e93a79fe, sigmas=1). Interval = estimate +/- q*SE, SE = sample SD/sqrt(60) over replication values (paired differences for H-P), q = qt(1-0.05/(2*3), 59).",MSE x 10,at_most,0.11858505,0.10949079911612217,0.0,at most 0.118585,consistent,True,,7,35,-inf,bound,False,0.92331,arxiv-1610-01271-20261020T030230Z | |
| 1610.01271,Generalized Random Forests,cgrf-p20-n800-hp-diff-ci-lower,False,claim,"Plan diagnostic: the paired helper excess H-P (helper DGP minus literal DGP, same seeds) has an interval whose lower end is above 0. paper_value 0 is the plan's sign threshold, not a published number.","Table 1, p. 23 (row conf.=yes, heterog.=yes, p=20, n=800; column C. GRF)","HP_ci_lower = mean(H_r - P_r) - q*SE_diff, SE_diff = sample SD of the 60 paired differences / sqrt(60). C. GRF test MSE x 10, cell confounding=yes, heterogeneity=yes, p=20, n=800. Per replication r=1..60: 10*mean((prediction - tau)^2) over 1,000 test points, centered causal_forest of grf 0.9.5 with 2,000 trees. Arm P = literal paper DGP (Sec. 6.2: e=(1+dbeta(X3,2,4))/4, m=2X3-1, tau=z(X1)z(X2), Y=m+(W-0.5)tau+eps, no rescaling); arm H = 2020 aw3 helper (dgps.R at e93a79fe, sigmas=1). Interval = estimate +/- q*SE, SE = sample SD/sqrt(60) over replication values (paired differences for H-P), q = qt(1-0.05/(2*3), 59).",MSE x 10,at_least,0.0,-0.08954917340128875,0.0,at least 0,inconsistent,True,,8,36,-inf,bound,False,,arxiv-1610-01271-20261020T030230Z | |
| 1610.01271,Generalized Random Forests,report-only-p-mean,False,claim,"Reporting only: the primary number, the mean of the 60 P-arm MSE x 10 values (literal paper DGP), for comparison with the published 0.93. Its compatibility is judged only by the two interval-end claims.","Table 1, p. 23 (row conf.=yes, heterog.=yes, p=20, n=800; column C. GRF)","P_mean_mse_x10 = mean of 60 P replication values. C. GRF test MSE x 10, cell confounding=yes, heterogeneity=yes, p=20, n=800. Per replication r=1..60: 10*mean((prediction - tau)^2) over 1,000 test points, centered causal_forest of grf 0.9.5 with 2,000 trees. Arm P = literal paper DGP (Sec. 6.2: e=(1+dbeta(X3,2,4))/4, m=2X3-1, tau=z(X1)z(X2), Y=m+(W-0.5)tau+eps, no rescaling); arm H = 2020 aw3 helper (dgps.R at e93a79fe, sigmas=1). Interval = estimate +/- q*SE, SE = sample SD/sqrt(60) over replication values (paired differences for H-P), q = qt(1-0.05/(2*3), 59).",MSE x 10,within,0.93,1.0141184587357088,1000.0,within ±1000,consistent,True,,9,37,0.0001,equivalence,True,1.09045,arxiv-1610-01271-20261020T030230Z | |
| 1610.01271,Generalized Random Forests,report-only-h-mean,False,claim,"Reporting only: mean of the 60 H-arm MSE x 10 values (2020 aw3 helper DGP, same seeds and grf 0.9.5), the control; the predecessor's helper value was 1.1671701.","Table 1, p. 23 (row conf.=yes, heterog.=yes, p=20, n=800; column C. GRF)","H_mean_mse_x10 = mean of 60 H replication values. C. GRF test MSE x 10, cell confounding=yes, heterogeneity=yes, p=20, n=800. Per replication r=1..60: 10*mean((prediction - tau)^2) over 1,000 test points, centered causal_forest of grf 0.9.5 with 2,000 trees. Arm P = literal paper DGP (Sec. 6.2: e=(1+dbeta(X3,2,4))/4, m=2X3-1, tau=z(X1)z(X2), Y=m+(W-0.5)tau+eps, no rescaling); arm H = 2020 aw3 helper (dgps.R at e93a79fe, sigmas=1). Interval = estimate +/- q*SE, SE = sample SD/sqrt(60) over replication values (paired differences for H-P), q = qt(1-0.05/(2*3), 59).",MSE x 10,within,0.93,1.1247302426955974,1000.0,within ±1000,consistent,True,,10,38,0.0002,equivalence,True,1.209387,arxiv-1610-01271-20261020T030230Z | |
| 1610.01271,Generalized Random Forests,report-only-hp-mean-diff,False,claim,"Reporting only: mean paired difference H-P of MSE x 10 over the 60 replications. paper_value 0 is a reference, not a published number.","Table 1, p. 23 (row conf.=yes, heterog.=yes, p=20, n=800; column C. GRF)","HP_mean_diff_x10 = mean of the 60 paired differences H_r - P_r. C. GRF test MSE x 10, cell confounding=yes, heterogeneity=yes, p=20, n=800. Per replication r=1..60: 10*mean((prediction - tau)^2) over 1,000 test points, centered causal_forest of grf 0.9.5 with 2,000 trees. Arm P = literal paper DGP (Sec. 6.2: e=(1+dbeta(X3,2,4))/4, m=2X3-1, tau=z(X1)z(X2), Y=m+(W-0.5)tau+eps, no rescaling); arm H = 2020 aw3 helper (dgps.R at e93a79fe, sigmas=1). Interval = estimate +/- q*SE, SE = sample SD/sqrt(60) over replication values (paired differences for H-P), q = qt(1-0.05/(2*3), 59).",MSE x 10,within,0.0,0.11061178395988853,1000.0,within ±1000,consistent,True,,11,39,0.0001,equivalence,True,,arxiv-1610-01271-20261020T030230Z | |
| 1506.02142,MC dropout,b32-concrete-rmse-lower,True,claim,"Batch-32 arm, Concrete Strength RMSE: the lower end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean 5.229903994142711 (paper cell 5.23).","Table 1, p. 7 (Dropout columns)","average test RMSE (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) - q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D - q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 32, 400 epochs, author net.py at a6259f1 otherwise unchanged.","RMSE, target units",within,5.229903994142711,5.061344861682844,0.24219763055953583,within ±0.242198,consistent,True,,9,79,-0.696,equivalence,False,0.96777,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b32-concrete-rmse-upper,True,claim,"Batch-32 arm, Concrete Strength RMSE: the upper end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean 5.229903994142711 (paper cell 5.23).","Table 1, p. 7 (Dropout columns)","average test RMSE (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) + q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D + q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 32, 400 epochs, author net.py at a6259f1 otherwise unchanged.","RMSE, target units",within,5.229903994142711,5.342854364077325,0.24219763055953583,within ±0.242198,consistent,True,,10,80,0.4664,equivalence,False,1.021597,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b128-concrete-rmse-lower,False,claim,"Batch-128 arm, Concrete Strength RMSE: the lower end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean 5.229903994142711 (paper cell 5.23).","Table 1, p. 7 (Dropout columns)","average test RMSE (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) - q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D - q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 128, 400 epochs, author net.py at a6259f1 otherwise unchanged.","RMSE, target units",within,5.229903994142711,5.362996005986037,0.24219763055953583,within ±0.242198,consistent,True,,11,81,0.5495,equivalence,False,1.025448,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b128-concrete-rmse-upper,False,claim,"Batch-128 arm, Concrete Strength RMSE: the upper end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean 5.229903994142711 (paper cell 5.23).","Table 1, p. 7 (Dropout columns)","average test RMSE (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) + q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D + q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 128, 400 epochs, author net.py at a6259f1 otherwise unchanged.","RMSE, target units",within,5.229903994142711,5.681315115746546,0.24219763055953583,within ±0.242198,inconsistent,True,,12,82,1.8638,equivalence,False,1.086313,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,contrast-concrete-rmse-upper,False,claim,Concrete Strength RMSE: the upper end of the predeclared Bonferroni-t interval for the paired batch contrast E_i = mean(B32_i) - mean(B128_i) over the 20 splits excludes zero in the improving direction (RMSE lower with batch 32).,"Section 5 (UCI regression experiments), p. 7","mean of E_i + q*SE_E over the 20 splits, SE_E = sample SD of E_i / sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18); RMSE, target units","RMSE, target units",at_most,0.0,-0.2093820114881758,0.0,at most 0,consistent,True,,13,83,-inf,bound,False,,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b32-concrete-ll-lower,True,claim,"Batch-32 arm, Concrete Strength LL: the lower end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean -3.0389055163159577 (paper cell -3.04).","Table 1, p. 7 (Dropout columns)","average test predictive log-likelihood (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) - q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D - q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 32, 400 epochs, author net.py at a6259f1 otherwise unchanged.",nats per test observation,within,-3.0389055163159577,-3.0644440025785884,0.041751464880923635,within ±0.0417515,consistent,True,,14,84,-0.6117,equivalence,False,1.008404,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b32-concrete-ll-upper,True,claim,"Batch-32 arm, Concrete Strength LL: the upper end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean -3.0389055163159577 (paper cell -3.04).","Table 1, p. 7 (Dropout columns)","average test predictive log-likelihood (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) + q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D + q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 32, 400 epochs, author net.py at a6259f1 otherwise unchanged.",nats per test observation,within,-3.0389055163159577,-3.0072436764796526,0.041751464880923635,within ±0.0417515,consistent,True,,15,85,0.7583,equivalence,False,0.989581,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b128-concrete-ll-lower,False,claim,"Batch-128 arm, Concrete Strength LL: the lower end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean -3.0389055163159577 (paper cell -3.04).","Table 1, p. 7 (Dropout columns)","average test predictive log-likelihood (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) - q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D - q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 128, 400 epochs, author net.py at a6259f1 otherwise unchanged.",nats per test observation,within,-3.0389055163159577,-3.1499155608217624,0.041751464880923635,within ±0.0417515,inconsistent,True,,16,86,-2.6588,equivalence,False,1.03653,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b128-concrete-ll-upper,False,claim,"Batch-128 arm, Concrete Strength LL: the upper end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean -3.0389055163159577 (paper cell -3.04).","Table 1, p. 7 (Dropout columns)","average test predictive log-likelihood (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) + q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D + q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 128, 400 epochs, author net.py at a6259f1 otherwise unchanged.",nats per test observation,within,-3.0389055163159577,-3.07234898391703,0.041751464880923635,within ±0.0417515,consistent,True,,17,87,-0.801,equivalence,False,1.011005,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,contrast-concrete-ll-lower,False,claim,Concrete Strength LL: the lower end of the predeclared Bonferroni-t interval for the paired batch contrast E_i = mean(B32_i) - mean(B128_i) over the 20 splits excludes zero in the improving direction (LL higher with batch 32).,"Section 5 (UCI regression experiments), p. 7","mean of E_i - q*SE_E over the 20 splits, SE_E = sample SD of E_i / sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18); nats per test observation",nats per test observation,at_least,0.0,0.053286060278996586,0.0,at least 0,consistent,True,,18,88,inf,bound,False,,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b32-energy-rmse-lower,True,claim,"Batch-32 arm, Energy Efficiency RMSE: the lower end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean 1.663405433671937 (paper cell 1.66).","Table 1, p. 7 (Dropout columns)","average test RMSE (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) - q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D - q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 32, 400 epochs, author net.py at a6259f1 otherwise unchanged.","RMSE, target units",within,1.663405433671937,1.6081951689222833,0.0887607545083069,within ±0.0887608,consistent,True,,19,89,-0.622,equivalence,False,0.966809,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b32-energy-rmse-upper,True,claim,"Batch-32 arm, Energy Efficiency RMSE: the upper end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean 1.663405433671937 (paper cell 1.66).","Table 1, p. 7 (Dropout columns)","average test RMSE (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) + q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D + q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 32, 400 epochs, author net.py at a6259f1 otherwise unchanged.","RMSE, target units",within,1.663405433671937,1.7096686138436987,0.0887607545083069,within ±0.0887608,consistent,True,,20,90,0.5212,equivalence,False,1.027812,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b128-energy-rmse-lower,False,claim,"Batch-128 arm, Energy Efficiency RMSE: the lower end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean 1.663405433671937 (paper cell 1.66).","Table 1, p. 7 (Dropout columns)","average test RMSE (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) - q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D - q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 128, 400 epochs, author net.py at a6259f1 otherwise unchanged.","RMSE, target units",within,1.663405433671937,1.9639005883030252,0.0887607545083069,within ±0.0887608,inconsistent,True,,21,91,3.3855,equivalence,False,1.180651,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b128-energy-rmse-upper,False,claim,"Batch-128 arm, Energy Efficiency RMSE: the upper end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean 1.663405433671937 (paper cell 1.66).","Table 1, p. 7 (Dropout columns)","average test RMSE (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) + q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D + q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 128, 400 epochs, author net.py at a6259f1 otherwise unchanged.","RMSE, target units",within,1.663405433671937,2.115701136179798,0.0887607545083069,within ±0.0887608,inconsistent,True,,22,92,5.0957,equivalence,False,1.271909,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,contrast-energy-rmse-upper,False,claim,Energy Efficiency RMSE: the upper end of the predeclared Bonferroni-t interval for the paired batch contrast E_i = mean(B32_i) - mean(B128_i) over the 20 splits excludes zero in the improving direction (RMSE lower with batch 32).,"Section 5 (UCI regression experiments), p. 7","mean of E_i + q*SE_E over the 20 splits, SE_E = sample SD of E_i / sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18); RMSE, target units","RMSE, target units",at_most,0.0,-0.3120570105299714,0.0,at most 0,consistent,True,,23,93,-inf,bound,False,,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b32-energy-ll-lower,True,claim,"Batch-32 arm, Energy Efficiency LL: the lower end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean -1.9852732326905922 (paper cell -1.99).","Table 1, p. 7 (Dropout columns)","average test predictive log-likelihood (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) - q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D - q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 32, 400 epochs, author net.py at a6259f1 otherwise unchanged.",nats per test observation,within,-1.9852732326905922,-2.001121564376307,0.04056204617367493,within ±0.040562,consistent,True,,24,94,-0.3907,equivalence,False,1.007983,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b32-energy-ll-upper,True,claim,"Batch-32 arm, Energy Efficiency LL: the upper end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean -1.9852732326905922 (paper cell -1.99).","Table 1, p. 7 (Dropout columns)","average test predictive log-likelihood (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) + q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D + q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 32, 400 epochs, author net.py at a6259f1 otherwise unchanged.",nats per test observation,within,-1.9852732326905922,-1.954894546782792,0.04056204617367493,within ±0.040562,consistent,True,,25,95,0.7489,equivalence,False,0.984698,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b128-energy-ll-lower,False,claim,"Batch-128 arm, Energy Efficiency LL: the lower end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean -1.9852732326905922 (paper cell -1.99).","Table 1, p. 7 (Dropout columns)","average test predictive log-likelihood (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) - q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D - q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 128, 400 epochs, author net.py at a6259f1 otherwise unchanged.",nats per test observation,within,-1.9852732326905922,-2.1988543438067762,0.04056204617367493,within ±0.040562,inconsistent,True,,26,96,-5.2655,equivalence,False,1.107583,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b128-energy-ll-upper,False,claim,"Batch-128 arm, Energy Efficiency LL: the upper end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean -1.9852732326905922 (paper cell -1.99).","Table 1, p. 7 (Dropout columns)","average test predictive log-likelihood (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) + q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D + q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 128, 400 epochs, author net.py at a6259f1 otherwise unchanged.",nats per test observation,within,-1.9852732326905922,-2.1271023810418503,0.04056204617367493,within ±0.040562,inconsistent,True,,27,97,-3.4966,equivalence,False,1.071441,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,contrast-energy-ll-lower,False,claim,Energy Efficiency LL: the lower end of the predeclared Bonferroni-t interval for the paired batch contrast E_i = mean(B32_i) - mean(B128_i) over the 20 splits excludes zero in the improving direction (LL higher with batch 32).,"Section 5 (UCI regression experiments), p. 7","mean of E_i - q*SE_E over the 20 splits, SE_E = sample SD of E_i / sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18); nats per test observation",nats per test observation,at_least,0.0,0.15199974766449087,0.0,at least 0,consistent,True,,28,98,inf,bound,False,,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b32-yacht-rmse-lower,True,claim,"Batch-32 arm, Yacht Hydrodynamics RMSE: the lower end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean 1.1092497489252557 (paper cell 1.11).","Table 1, p. 7 (Dropout columns)","average test RMSE (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) - q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D - q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 32, 400 epochs, author net.py at a6259f1 otherwise unchanged.","RMSE, target units",within,1.1092497489252557,0.9651899983004549,0.17442280116622366,within ±0.174423,consistent,True,,29,99,-0.8259,equivalence,False,0.870129,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b32-yacht-rmse-upper,True,claim,"Batch-32 arm, Yacht Hydrodynamics RMSE: the upper end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean 1.1092497489252557 (paper cell 1.11).","Table 1, p. 7 (Dropout columns)","average test RMSE (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) + q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D + q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 32, 400 epochs, author net.py at a6259f1 otherwise unchanged.","RMSE, target units",within,1.1092497489252557,1.2678071673635172,0.17442280116622366,within ±0.174423,consistent,True,,30,100,0.909,equivalence,False,1.142941,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b128-yacht-rmse-lower,False,claim,"Batch-128 arm, Yacht Hydrodynamics RMSE: the lower end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean 1.1092497489252557 (paper cell 1.11).","Table 1, p. 7 (Dropout columns)","average test RMSE (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) - q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D - q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 128, 400 epochs, author net.py at a6259f1 otherwise unchanged.","RMSE, target units",within,1.1092497489252557,1.9983550081679469,0.17442280116622366,within ±0.174423,inconsistent,True,,31,101,5.0974,equivalence,False,1.801537,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b128-yacht-rmse-upper,False,claim,"Batch-128 arm, Yacht Hydrodynamics RMSE: the upper end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean 1.1092497489252557 (paper cell 1.11).","Table 1, p. 7 (Dropout columns)","average test RMSE (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) + q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D + q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 128, 400 epochs, author net.py at a6259f1 otherwise unchanged.","RMSE, target units",within,1.1092497489252557,2.7693175213950565,0.17442280116622366,within ±0.174423,inconsistent,True,,32,102,9.5175,equivalence,False,2.496568,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,contrast-yacht-rmse-upper,False,claim,Yacht Hydrodynamics RMSE: the upper end of the predeclared Bonferroni-t interval for the paired batch contrast E_i = mean(B32_i) - mean(B128_i) over the 20 splits excludes zero in the improving direction (RMSE lower with batch 32).,"Section 5 (UCI regression experiments), p. 7","mean of E_i + q*SE_E over the 20 splits, SE_E = sample SD of E_i / sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18); RMSE, target units","RMSE, target units",at_most,0.0,-0.9067515886921714,0.0,at most 0,consistent,True,,33,103,-inf,bound,False,,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b32-yacht-ll-lower,True,claim,"Batch-32 arm, Yacht Hydrodynamics LL: the lower end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean -1.5540594381496233 (paper cell -1.55).","Table 1, p. 7 (Dropout columns)","average test predictive log-likelihood (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) - q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D - q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 32, 400 epochs, author net.py at a6259f1 otherwise unchanged.",nats per test observation,within,-1.5540594381496233,-1.6194904845928275,0.05522109500594338,within ±0.0552211,inconsistent,True,,34,104,-1.1849,equivalence,False,1.042103,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b32-yacht-ll-upper,True,claim,"Batch-32 arm, Yacht Hydrodynamics LL: the upper end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean -1.5540594381496233 (paper cell -1.55).","Table 1, p. 7 (Dropout columns)","average test predictive log-likelihood (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) + q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D + q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 32, 400 epochs, author net.py at a6259f1 otherwise unchanged.",nats per test observation,within,-1.5540594381496233,-1.4928128048110798,0.05522109500594338,within ±0.0552211,inconsistent,True,,35,105,1.1091,equivalence,False,0.960589,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b128-yacht-ll-lower,False,claim,"Batch-128 arm, Yacht Hydrodynamics LL: the lower end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean -1.5540594381496233 (paper cell -1.55).","Table 1, p. 7 (Dropout columns)","average test predictive log-likelihood (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) - q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D - q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 128, 400 epochs, author net.py at a6259f1 otherwise unchanged.",nats per test observation,within,-1.5540594381496233,-2.544950474590947,0.05522109500594338,within ±0.0552211,inconsistent,True,,36,106,-17.9441,equivalence,False,1.637615,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,b128-yacht-ll-upper,False,claim,"Batch-128 arm, Yacht Hydrodynamics LL: the upper end of the predeclared Bonferroni-t interval for the paired split difference (three-seed split mean minus archived split value), on the metric scale, lies within delta = 2 x archive SE of the unrounded archive mean -1.5540594381496233 (paper cell -1.55).","Table 1, p. 7 (Dropout columns)","average test predictive log-likelihood (MC dropout, 10,000 passes); value = mean over 20 splits of (three-seed split mean) + q*SE, where SE = sample SD of the 20 paired differences D_i = split mean - archived A_i over sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18). Equivalently archive mean + (mean D + q*SE). Fresh process per fit, seed = 10000*d+100*i+r, batch size 128, 400 epochs, author net.py at a6259f1 otherwise unchanged.",nats per test observation,within,-1.5540594381496233,-1.9947939232545668,0.05522109500594338,within ±0.0552211,inconsistent,True,,37,107,-7.9813,equivalence,False,1.283602,arxiv-1506-02142-20261020T024912Z | |
| 1506.02142,MC dropout,contrast-yacht-ll-lower,False,claim,Yacht Hydrodynamics LL: the lower end of the predeclared Bonferroni-t interval for the paired batch contrast E_i = mean(B32_i) - mean(B128_i) over the 20 splits excludes zero in the improving direction (LL higher with batch 32).,"Section 5 (UCI regression experiments), p. 7","mean of E_i - q*SE_E over the 20 splits, SE_E = sample SD of E_i / sqrt(20), q = 3.4346619924588992 = t_19 quantile at 1 - 0.05/(2*18); nats per test observation",nats per test observation,at_least,0.0,0.4411305830252684,0.0,at least 0,consistent,True,,38,108,inf,bound,False,,arxiv-1506-02142-20261020T024912Z | |
| 2205.13504,DLinear,dlinear-etth1-h96-mse,True,claim,"On ETTh1 (multivariate, look-back L=336), DLinear's test MSE at forecast horizon 96 is 0.375 (Table 2).","Table 2, p. 5, ETTh1 row T=96, DLinear MSE column (columns: horizon, improvement, Linear MSE/MAE, NLinear MSE/MAE, DLinear MSE/MAE); look-back L=336 per Appendix p. 9","Test MSE of DLinear on ETTh1, features M, seq_len 336, label_len 48, pred_len 96, shared channels (individual=False), seed 2021, one fit: the value returned by the authors' utils/metrics.py MSE over all retained test windows (test loader shuffle False, drop_last True, batch = the version's batch size) x 96 steps x 7 variables, on the standardized scale (scaler fit on train rows only), no inverse transform, no percent conversion, no averaging of per-batch errors; best-validation checkpoint via the authors' early stopping (patience 3, max 10 epochs). Judged unrounded.",MSE (standardized),within,0.375,0.3840945363044739,0.0005,within ±0.0005,consistent_earlier_code,True,0.3748247027397156,5,52,18.1891,equivalence,False,1.024252,arxiv-2205-13504-20261020T025623Z | |
| 2205.13504,DLinear,dlinear-etth1-h96-mae,True,claim,"On ETTh1 (multivariate, look-back L=336), DLinear's test MAE at forecast horizon 96 is 0.399 (Table 2).","Table 2, p. 5, ETTh1 row T=96, DLinear MAE column (columns: horizon, improvement, Linear MSE/MAE, NLinear MSE/MAE, DLinear MSE/MAE); look-back L=336 per Appendix p. 9","Test MAE of DLinear on ETTh1, features M, seq_len 336, label_len 48, pred_len 96, shared channels (individual=False), seed 2021, one fit: the value returned by the authors' utils/metrics.py MAE over all retained test windows (test loader shuffle False, drop_last True, batch = the version's batch size) x 96 steps x 7 variables, on the standardized scale (scaler fit on train rows only), no inverse transform, no percent conversion, no averaging of per-batch errors; best-validation checkpoint via the authors' early stopping (patience 3, max 10 epochs). Judged unrounded.",MAE (standardized),within,0.399,0.4046996533870697,0.0005,within ±0.0005,consistent_earlier_code,True,0.39947351813316345,6,53,11.3993,equivalence,False,1.014285,arxiv-2205-13504-20261020T025623Z | |
| 2205.13504,DLinear,dlinear-etth1-h192-mse,True,claim,"On ETTh1 (multivariate, look-back L=336), DLinear's test MSE at forecast horizon 192 is 0.405 (Table 2).","Table 2, p. 5, ETTh1 row T=192, DLinear MSE column (columns: horizon, improvement, Linear MSE/MAE, NLinear MSE/MAE, DLinear MSE/MAE); look-back L=336 per Appendix p. 9","Test MSE of DLinear on ETTh1, features M, seq_len 336, label_len 48, pred_len 192, shared channels (individual=False), seed 2021, one fit: the value returned by the authors' utils/metrics.py MSE over all retained test windows (test loader shuffle False, drop_last True, batch = the version's batch size) x 192 steps x 7 variables, on the standardized scale (scaler fit on train rows only), no inverse transform, no percent conversion, no averaging of per-batch errors; best-validation checkpoint via the authors' early stopping (patience 3, max 10 epochs). Judged unrounded.",MSE (standardized),within,0.405,0.44340065121650696,0.0005,within ±0.0005,consistent_earlier_code,True,0.40548211336135864,7,54,76.8013,equivalence,False,1.094816,arxiv-2205-13504-20261020T025623Z | |
| 2205.13504,DLinear,dlinear-etth1-h192-mae,True,claim,"On ETTh1 (multivariate, look-back L=336), DLinear's test MAE at forecast horizon 192 is 0.416 (Table 2).","Table 2, p. 5, ETTh1 row T=192, DLinear MAE column (columns: horizon, improvement, Linear MSE/MAE, NLinear MSE/MAE, DLinear MSE/MAE); look-back L=336 per Appendix p. 9","Test MAE of DLinear on ETTh1, features M, seq_len 336, label_len 48, pred_len 192, shared channels (individual=False), seed 2021, one fit: the value returned by the authors' utils/metrics.py MAE over all retained test windows (test loader shuffle False, drop_last True, batch = the version's batch size) x 192 steps x 7 variables, on the standardized scale (scaler fit on train rows only), no inverse transform, no percent conversion, no averaging of per-batch errors; best-validation checkpoint via the authors' early stopping (patience 3, max 10 epochs). Judged unrounded.",MAE (standardized),within,0.416,0.44990822672843933,0.0005,within ±0.0005,consistent_earlier_code,True,0.4158572554588318,8,55,67.8165,equivalence,False,1.08151,arxiv-2205-13504-20261020T025623Z | |
| 2205.13504,DLinear,dlinear-etth1-h336-mse,True,claim,"On ETTh1 (multivariate, look-back L=336), DLinear's test MSE at forecast horizon 336 is 0.439 (Table 2).","Table 2, p. 5, ETTh1 row T=336, DLinear MSE column (columns: horizon, improvement, Linear MSE/MAE, NLinear MSE/MAE, DLinear MSE/MAE); look-back L=336 per Appendix p. 9","Test MSE of DLinear on ETTh1, features M, seq_len 336, label_len 48, pred_len 336, shared channels (individual=False), seed 2021, one fit: the value returned by the authors' utils/metrics.py MSE over all retained test windows (test loader shuffle False, drop_last True, batch = the version's batch size) x 336 steps x 7 variables, on the standardized scale (scaler fit on train rows only), no inverse transform, no percent conversion, no averaging of per-batch errors; best-validation checkpoint via the authors' early stopping (patience 3, max 10 epochs). Judged unrounded.",MSE (standardized),within,0.439,0.44690412282943726,0.0005,within ±0.0005,consistent_earlier_code,True,0.43851277232170105,9,56,15.8082,equivalence,False,1.018005,arxiv-2205-13504-20261020T025623Z | |
| 2205.13504,DLinear,dlinear-etth1-h336-mae,True,claim,"On ETTh1 (multivariate, look-back L=336), DLinear's test MAE at forecast horizon 336 is 0.443 (Table 2).","Table 2, p. 5, ETTh1 row T=336, DLinear MAE column (columns: horizon, improvement, Linear MSE/MAE, NLinear MSE/MAE, DLinear MSE/MAE); look-back L=336 per Appendix p. 9","Test MAE of DLinear on ETTh1, features M, seq_len 336, label_len 48, pred_len 336, shared channels (individual=False), seed 2021, one fit: the value returned by the authors' utils/metrics.py MAE over all retained test windows (test loader shuffle False, drop_last True, batch = the version's batch size) x 336 steps x 7 variables, on the standardized scale (scaler fit on train rows only), no inverse transform, no percent conversion, no averaging of per-batch errors; best-validation checkpoint via the authors' early stopping (patience 3, max 10 epochs). Judged unrounded.",MAE (standardized),within,0.443,0.4481101632118225,0.0005,within ±0.0005,consistent_earlier_code,True,0.4427293539047241,10,57,10.2203,equivalence,False,1.011535,arxiv-2205-13504-20261020T025623Z | |
| 2205.13504,DLinear,dlinear-etth1-h720-mse,True,claim,"On ETTh1 (multivariate, look-back L=336), DLinear's test MSE at forecast horizon 720 is 0.472 (Table 2).","Table 2, p. 5, ETTh1 row T=720, DLinear MSE column (columns: horizon, improvement, Linear MSE/MAE, NLinear MSE/MAE, DLinear MSE/MAE); look-back L=336 per Appendix p. 9","Test MSE of DLinear on ETTh1, features M, seq_len 336, label_len 48, pred_len 720, shared channels (individual=False), seed 2021, one fit: the value returned by the authors' utils/metrics.py MSE over all retained test windows (test loader shuffle False, drop_last True, batch = the version's batch size) x 720 steps x 7 variables, on the standardized scale (scaler fit on train rows only), no inverse transform, no percent conversion, no averaging of per-batch errors; best-validation checkpoint via the authors' early stopping (patience 3, max 10 epochs). Judged unrounded.",MSE (standardized),within,0.472,0.5031593441963196,0.0005,within ±0.0005,consistent_earlier_code,True,0.4719056785106659,11,58,62.3187,equivalence,False,1.066016,arxiv-2205-13504-20261020T025623Z | |
| 2205.13504,DLinear,dlinear-etth1-h720-mae,True,claim,"On ETTh1 (multivariate, look-back L=336), DLinear's test MAE at forecast horizon 720 is 0.490 (Table 2).","Table 2, p. 5, ETTh1 row T=720, DLinear MAE column (columns: horizon, improvement, Linear MSE/MAE, NLinear MSE/MAE, DLinear MSE/MAE); look-back L=336 per Appendix p. 9","Test MAE of DLinear on ETTh1, features M, seq_len 336, label_len 48, pred_len 720, shared channels (individual=False), seed 2021, one fit: the value returned by the authors' utils/metrics.py MAE over all retained test windows (test loader shuffle False, drop_last True, batch = the version's batch size) x 720 steps x 7 variables, on the standardized scale (scaler fit on train rows only), no inverse transform, no percent conversion, no averaging of per-batch errors; best-validation checkpoint via the authors' early stopping (patience 3, max 10 epochs). Judged unrounded.",MAE (standardized),within,0.49,0.5137903690338135,0.0005,within ±0.0005,consistent_earlier_code,True,0.49024033546447754,12,59,47.5807,equivalence,False,1.048552,arxiv-2205-13504-20261020T025623Z | |
| 2012.07436,Informer,etth1-uni-24-mse,True,claim,Informer reaches test MSE 0.098 on ETTh1 univariate forecasting with prediction horizon 24 (Table 1).,"Table 1, p. 5 (Univariate, ETTh1, horizon 24, Informer column)","test MSE on ETTh1 univariate (features S, target OT), pred_len 24, normalized data, mean over the runs executed (the authors script runs 5 iterations)",MSE,at_most,0.098,,0.02,at most 0.118,untested,True,,7,,,bound,False,,arxiv-2012-07436-20261019T202050Z | |
| 2012.07436,Informer,etth1-uni-24-mae,False,claim,Informer reaches test MAE 0.247 on ETTh1 univariate forecasting with prediction horizon 24 (Table 1).,"Table 1, p. 5 (Univariate, ETTh1, horizon 24, Informer column)","test MAE on ETTh1 univariate (features S, target OT), pred_len 24, normalized data, mean over the runs executed",MAE,at_most,0.247,,0.05,at most 0.297,untested,True,,8,,,bound,False,,arxiv-2012-07436-20261019T202050Z | |
| 2012.07436,Informer,etth1-multi-24-mse,True,claim,Informer reaches test MSE 0.577 on ETTh1 multivariate forecasting with prediction horizon 24 (Table 2).,"Table 2, p. 6 (Multivariate, ETTh1, horizon 24, Informer column)","test MSE on ETTh1 multivariate (features M, all 7 variables), pred_len 24, normalized data, mean over the 5 iterations the authors script runs",MSE,at_most,0.577,0.5046,0.115,at most 0.692,consistent,True,,9,61,-0.6296,bound,False,0.874523,arxiv-2012-07436-20261019T202050Z | |
| 2012.07436,Informer,etth1-multi-24-mae,False,claim,Informer reaches test MAE 0.549 on ETTh1 multivariate forecasting with prediction horizon 24 (Table 2).,"Table 2, p. 6 (Multivariate, ETTh1, horizon 24, Informer column)","test MAE on ETTh1 multivariate (features M), pred_len 24, normalized data, mean over the 5 iterations the authors script runs",MAE,at_most,0.549,0.5151,0.11,at most 0.659,consistent,True,,10,62,-0.3082,bound,False,0.938251,arxiv-2012-07436-20261019T202050Z | |
| 2012.07436,Informer,etth1-multi-48-mse,True,claim,Informer reaches test MSE 0.685 on ETTh1 multivariate forecasting with prediction horizon 48 (Table 2).,"Table 2, p. 6 (Multivariate, ETTh1, horizon 48, Informer column)","test MSE on ETTh1 multivariate (features M), pred_len 48, normalized data, mean over the 5 iterations the authors script runs",MSE,at_most,0.685,0.6021,0.137,at most 0.822,consistent,True,,11,44,-0.6051,bound,False,0.878978,arxiv-2012-07436-20261019T202050Z | |
| 2012.07436,Informer,etth1-multi-48-mae,False,claim,Informer reaches test MAE 0.625 on ETTh1 multivariate forecasting with prediction horizon 48 (Table 2).,"Table 2, p. 6 (Multivariate, ETTh1, horizon 48, Informer column)","test MAE on ETTh1 multivariate (features M), pred_len 48, normalized data, mean over the 5 iterations the authors script runs",MAE,at_most,0.625,0.57,0.125,at most 0.75,consistent,True,,12,45,-0.44,bound,False,0.912,arxiv-2012-07436-20261019T202050Z | |
| 1905.03222,Conformalized Quantile Regression,cqr-coverage-min,True,claim,"Conformal methods, including both CQR variants, attain the nominal 90% coverage; observed value is the LOWEST split-averaged test coverage among all CQR (method, dataset) cells run in this attempt.","Section 6.2, p. 10","minimum over tested CQR (method, dataset) cells of mean test coverage (%) over random splits",%,at_least,90,89.65,2.0,at least 88,consistent,True,,7,44,-0.175,bound,False,0.996111,arxiv-1905-03222-20261019T233532Z | |
| 1905.03222,Conformalized Quantile Regression,cqr-nn-bike-length,True,claim,CQR with a quantile neural network gives average 90% interval length 0.57 on the bike sharing dataset (20 random 80/20 splits).,"Figure 4 (bike panel), p. 15","average interval length on test set, mean over random splits, response rescaled by mean |y| of training set",rescaled response units,within,0.57,0.5704,0.057,within ±0.057,consistent,True,,8,43,0.007,equivalence,False,1.000702,arxiv-1905-03222-20261019T233532Z | |
| 1905.03222,Conformalized Quantile Regression,cqr-nn-community-length,True,claim,CQR with a quantile neural network gives average 90% interval length 1.76 on the community dataset (20 random 80/20 splits).,"Figure 5 (community panel), p. 16","average interval length on test set, mean over random splits, response rescaled by mean |y| of training set",rescaled response units,within,1.76,1.8561,0.176,within ±0.176,consistent,True,,9,40,0.546,equivalence,False,1.054602,arxiv-1905-03222-20261019T233532Z | |
| 1905.03222,Conformalized Quantile Regression,cqr-rf-community-length,True,claim,CQR with quantile random forests gives average 90% interval length 1.55 on the community dataset (20 random 80/20 splits).,"Figure 5 (community panel), p. 16","average interval length on test set, mean over random splits, response rescaled by mean |y| of training set",rescaled response units,within,1.55,1.507,0.155,within ±0.155,consistent,True,,10,39,-0.2774,equivalence,False,0.972258,arxiv-1905-03222-20261019T233532Z | |
| 1905.03222,Conformalized Quantile Regression,cqr-rf-concrete-length,True,claim,CQR with quantile random forests gives average 90% interval length 0.56 on the concrete dataset (20 random 80/20 splits).,"Figure 5 (concrete panel), p. 16","average interval length on test set, mean over random splits, response rescaled by mean |y| of training set",rescaled response units,within,0.56,0.5584,0.056,within ±0.056,consistent,True,,11,42,-0.0286,equivalence,False,0.997143,arxiv-1905-03222-20261019T233532Z | |
| 1905.03222,Conformalized Quantile Regression,split-rf-community-length,False,claim,"Standard split conformal prediction with a conditional-mean random forest gives average 90% interval length 1.94 on the community dataset, longer than CQR random forests (1.55).","Figure 5 (community panel), p. 16","average interval length on test set, mean over random splits, response rescaled by mean |y| of training set",rescaled response units,within,1.94,1.8578,0.194,within ±0.194,consistent,True,,12,41,-0.4237,equivalence,False,0.957629,arxiv-1905-03222-20261019T233532Z | |
| 1910.13051,ROCKET,bakeoff-mean-accuracy,True,claim,"Averaged over the 85 bake-off datasets, ROCKET (10,000 kernels, ridge classifier, 10 runs) attains the per-dataset test accuracies reported in Table 1, whose mean is 0.8510.","Table 1, pp. 25-26 (mean of the Rocket column, computed from Table 1 at event 19; the paper does not print the mean itself)","Mean over the 85 bake-off datasets of ROCKET test accuracy (each dataset: mean of 10 runs with fresh random kernels, original train/test split), unrounded",accuracy,within,0.851002,0.851669,0.005,within ±0.005,consistent,True,,25,47,0.1334,equivalence,False,1.000784,arxiv-1910-13051-20261019T203424Z | |
| 1910.13051,ROCKET,bakeoff-best-mean-rank,True,claim,ROCKET has the strictly best (lowest) mean rank of the eight classifiers in Table 1 over the 85 bake-off datasets. A tie for the best mean rank does NOT count as reproducing this claim.,"Section 4, p. 11 (also p. 11: 'ranks slightly ahead of HIVE-COTE, TS-CHIEF, and InceptionTime'); Figure 1, p. 1; ranking method p. 11 (Demsar 2006) and p. 18 (rank figures produced with code from Ismail Fawaz et al. 2019a)","Number of the other seven Table 1 classifiers (BOSS, ST, HIVE-COTE, ResNet, Proximity Forest, TS-CHIEF, InceptionTime) whose mean rank is lower than or equal to ROCKET's. Ranking method, fixed before any result: per dataset, the 8 classifiers are ranked by accuracy (1 = most accurate) with tied classifiers sharing the average of the positions they span (fractional ranks as in Demsar 2006 and the Ismail Fawaz et al. 2019a critical-difference code the paper used, which ranks with pandas rank, method 'average'); mean rank = mean over the 85 datasets, in exact rational arithmetic. The reproduced ROCKET 10-run mean accuracy replaces the Rocket column after rounding half-up to 4 decimals (the precision at which Table 1 reports all eight classifiers); the other seven columns are Table 1 as printed. Computed by analysis/calc.py (event 18).",classifiers,at_most,0,0,0,at most 0,consistent,True,,26,48,0.0,bound,False,,arxiv-1910-13051-20261019T203424Z | |
| 1910.13051,ROCKET,bakeoff-per-dataset-agreement,False,claim,Per-dataset ROCKET accuracies in Table 1 are reproducible: the mean absolute difference between reproduced and reported accuracy across the 85 datasets is small.,"Section 4.1.1, p. 11; Table 1",Mean over the 85 datasets of |reproduced 10-run mean accuracy (unrounded) - Table 1 accuracy|,accuracy,at_most,0,0.001975,0.01,at most 0.01,consistent,True,,27,49,0.1975,bound,True,,arxiv-1910-13051-20261019T203424Z | |
| 1910.13051,ROCKET,bakeoff-total-compute-minutes,True,claim,"Total compute time (training and test, single CPU core per dataset per run, averaged over runs) for ROCKET over all 85 bake-off datasets is 1 hour 50 minutes.","Section 1, p. 2","Sum over the 85 datasets of the authors' script time_training_seconds + time_test_seconds (per-run means over 10 runs; kernel transform, ridge fit and scoring included; data loading and JIT compilation excluded), in minutes, with every process single-threaded (NUMBA_NUM_THREADS=1, BLAS threads 1) on a dedicated box",minutes,at_most,110,40.54,55,at most 165,consistent,True,,28,50,-1.2629,bound,False,0.368545,arxiv-1910-13051-20261019T203424Z | |
| 1910.13051,ROCKET,electricdevices-train-minutes,False,claim,"Training ROCKET on ElectricDevices (8,926 training examples) takes about 6 minutes on a single core.","Section 1, p. 2","time_training_seconds for ElectricDevices from the authors' script (training transform + ridge fit, mean per run over 10 runs, single-threaded), in minutes",minutes,at_most,6,2.35,3,at most 9,consistent,True,,29,51,-1.2167,bound,False,0.391667,arxiv-1910-13051-20261019T203424Z | |
| 1910.13051,ROCKET,handoutlines-train-minutes,False,claim,"Training ROCKET on HandOutlines (series length 2,709) takes 4 minutes 52 seconds on a single core.","Section 1, p. 2","time_training_seconds for HandOutlines from the authors' script (training transform + ridge fit, mean per run over 10 runs, single-threaded), in minutes",minutes,at_most,4.8667,1.81,2.4333,at most 7.3,consistent,True,,30,52,-1.2562,bound,False,0.371915,arxiv-1910-13051-20261019T203424Z | |
| 1803.09015,Callaway and Sant'Anna's DiD,cond-dr-simple-att,True,claim,"Under conditional parallel trends with the doubly robust estimator, the simple weighted average of post-treatment ATT(g,t) is -0.033.","Table 3 Panel (b), p. 30","aggte(type='simple') overall ATT, DR estimator, covariates region, pop, medinc, pop^2, medinc^2, white, hs, pov",log points,within,-0.033,-0.033117,0.0005,within ±0.0005,consistent,True,,23,44,-0.234,equivalence,False,1.003545,arxiv-1803-09015-20261019T161404Z | |
| 1803.09015,Callaway and Sant'Anna's DiD,cond-dr-simple-se,False,claim,The clustered multiplier-bootstrap standard error of the conditional DR simple weighted average is 0.007.,"Table 3 Panel (b), p. 30","bootstrap standard error of aggte(type='simple'), DR conditional PT",log points,within,0.007,0.006506,0.00155,within ±0.00155,consistent,True,,24,46,-0.3187,equivalence,False,0.929429,arxiv-1803-09015-20261019T161404Z | |
| 1803.09015,Callaway and Sant'Anna's DiD,data-treated-pop-mean,False,claim,Data identity check: treated counties have mean 2000 population of 94.32 thousand in the main dataset.,"Table 2, p. 28",mean of pop across counties with first.treat > 0 (one row per county),thousands,within,94.32,94.3213,0.005,within ±0.005,consistent,True,,25,47,0.26,equivalence,False,1.000014,arxiv-1803-09015-20261019T161404Z | |
| 1803.09015,Callaway and Sant'Anna's DiD,uncond-att-2004-2007,True,claim,"Under unconditional parallel trends, the group-time ATT for the 2004 group (Illinois) in 2007 is -0.136.","Section 5.1, p. 28 (also Table 3 Panel (a) Event Study e=3, p. 30)","att_gt ATT(g=2004, t=2007), unconditional PT",log points,within,-0.136,-0.135614,0.0005,within ±0.0005,consistent,True,,26,43,0.772,equivalence,False,0.997162,arxiv-1803-09015-20261019T161404Z | |
| 1803.09015,Callaway and Sant'Anna's DiD,uncond-event-study-overall,True,claim,"Under unconditional parallel trends, the event-study single parameter (average of e=0..3 dynamic effects) is -0.090.","Table 3 Panel (a), row 'Event Study', Single Parameters column, p. 30","aggte(type='dynamic') overall ATT, unconditional PT",log points,within,-0.09,-0.08981,0.0005,within ±0.0005,consistent,True,,27,42,0.38,equivalence,False,0.997889,arxiv-1803-09015-20261019T161404Z | |
| 1803.09015,Callaway and Sant'Anna's DiD,uncond-simple-att,True,claim,"Under unconditional parallel trends with never-treated controls, the simple (group-size weighted) average of post-treatment ATT(g,t) is -0.052 log points of teen employment.","Section 5.1 p. 28; Table 3 Panel (a) 'Simple Weighted Average', p. 30","aggte(type='simple') overall ATT, unconditional PT, never-treated comparison",log points,within,-0.052,-0.051581,0.0005,within ±0.0005,consistent,True,,28,41,0.838,equivalence,False,0.991942,arxiv-1803-09015-20261019T161404Z | |
| 1803.09015,Callaway and Sant'Anna's DiD,uncond-simple-se,False,claim,The clustered multiplier-bootstrap standard error of the unconditional simple weighted average is 0.006.,"Table 3 Panel (a), p. 30","bootstrap standard error of aggte(type='simple'), unconditional PT",log points,within,0.006,0.006595,0.0014,within ±0.0014,consistent,True,,29,45,0.425,equivalence,False,1.099167,arxiv-1803-09015-20261019T161404Z | |
| 1708.05031,Neural Collaborative Filtering,neumf-nopre-8-hr10,True,claim,NeuMF trained from scratch (without pre-training) with 8 predictive factors reaches HR@10 = 0.688 on MovieLens-1M.,"Table 2 (MovieLens rows, Factors = 8, 'Without Pre-training' HR@10 column), p. 7","HR@10 of NeuMF without pre-training, 8 predictive factors, MovieLens-1M, leave-one-out with 99 sampled negatives",ratio,within,0.688,0.6848,0.02,within ±0.02,consistent,True,,12,38,-0.16,equivalence,False,0.995349,arxiv-1708-05031-20261019T135218Z | |
| 1708.05031,Neural Collaborative Filtering,neumf-nopre-8-ndcg10,True,claim,NeuMF trained from scratch (without pre-training) with 8 predictive factors reaches NDCG@10 = 0.410 on MovieLens-1M.,"Table 2 (MovieLens rows, Factors = 8, 'Without Pre-training' NDCG@10 column), p. 7","NDCG@10 of NeuMF without pre-training, 8 predictive factors, MovieLens-1M",ratio,within,0.41,0.4088,0.02,within ±0.02,consistent,True,,13,39,-0.06,equivalence,False,0.997073,arxiv-1708-05031-20261019T135218Z | |
| 1708.05031,Neural Collaborative Filtering,neumf-pre-8-hr10,True,claim,NeuMF initialised from pre-trained GMF and MLP (alpha = 0.5) with 8 predictive factors reaches HR@10 = 0.684 on MovieLens-1M.,"Table 2 (MovieLens rows, Factors = 8, 'With Pre-training' HR@10 column), p. 7","HR@10 of NeuMF with pre-training, 8 predictive factors, MovieLens-1M",ratio,within,0.684,0.6798,0.02,within ±0.02,consistent,True,,14,42,-0.21,equivalence,False,0.99386,arxiv-1708-05031-20261019T135218Z | |
| 1708.05031,Neural Collaborative Filtering,neumf-pre-8-ndcg10,True,claim,NeuMF initialised from pre-trained GMF and MLP (alpha = 0.5) with 8 predictive factors reaches NDCG@10 = 0.403 on MovieLens-1M.,"Table 2 (MovieLens rows, Factors = 8, 'With Pre-training' NDCG@10 column), p. 7","NDCG@10 of NeuMF with pre-training, 8 predictive factors, MovieLens-1M",ratio,within,0.403,0.4054,0.02,within ±0.02,consistent,True,,15,43,0.12,equivalence,False,1.005955,arxiv-1708-05031-20261019T135218Z | |
| 1708.05031,Neural Collaborative Filtering,mlp3-8-hr10,False,claim,"MLP with three hidden layers (32->16->8, embedding 16; the README's --layers [64,32,16,8]) reaches HR@10 = 0.671 on MovieLens-1M at 8 predictive factors.","Table 3 (MovieLens rows, Factors = 8, MLP-3 column), p. 8","HR@10 of MLP-3, 8 predictive factors, MovieLens-1M",ratio,within,0.671,0.6707,0.02,within ±0.02,consistent,True,,16,61,-0.015,equivalence,False,0.999553,arxiv-1708-05031-20261019T135218Z | |
| 1708.05031,Neural Collaborative Filtering,mlp3-8-ndcg10,False,claim,MLP with three hidden layers reaches NDCG@10 = 0.399 on MovieLens-1M at 8 predictive factors.,"Table 4 (MovieLens rows, Factors = 8, MLP-3 column), p. 9","NDCG@10 of MLP-3, 8 predictive factors, MovieLens-1M",ratio,within,0.399,0.3979,0.02,within ±0.02,consistent,True,,17,62,-0.055,equivalence,False,0.997243,arxiv-1708-05031-20261019T135218Z | |
| 1606.00931,DeepSurv,support-cindex,True,claim,DeepSurv reaches test C-index 0.618308 on SUPPORT.,"Table 1, p. 5 (DeepSurv column)",test-set C-index of DeepSurv on SUPPORT,C-index,within,0.618308,0.618582,0.01,within ±0.01,consistent,True,,23,51,0.0274,equivalence,False,1.000443,arxiv-1606-00931-20261018T225505Z | |
| 1606.00931,DeepSurv,metabric-cindex,True,claim,DeepSurv reaches test C-index 0.643374 on METABRIC.,"Table 1, p. 5 (DeepSurv column)",test-set C-index of DeepSurv on METABRIC,C-index,within,0.643374,0.643492,0.01,within ±0.01,consistent,True,,24,52,0.0118,equivalence,False,1.000183,arxiv-1606-00931-20261018T225505Z | |
| 1606.00931,DeepSurv,gbsg-cindex,True,claim,DeepSurv reaches test C-index 0.668402 on Rotterdam & GBSG.,"Table 1, p. 5 (DeepSurv column)",test-set C-index of DeepSurv on Rotterdam (train) and GBSG (test),C-index,within,0.668402,0.672011,0.01,within ±0.01,consistent,True,,25,53,0.3609,equivalence,False,1.005399,arxiv-1606-00931-20261018T225505Z | |
| 1606.00931,DeepSurv,whas-cindex,True,claim,DeepSurv reaches test C-index 0.862620 on WHAS.,"Table 1, p. 5 (DeepSurv column)",test-set C-index of DeepSurv on WHAS,C-index,within,0.86262,0.858003,0.01,within ±0.01,consistent,True,,26,54,-0.4617,equivalence,False,0.994648,arxiv-1606-00931-20261018T225505Z | |
| 1606.00931,DeepSurv,sim-linear-cindex,False,claim,DeepSurv reaches test C-index 0.774019 on simulated linear data.,"Table 1, p. 5 (DeepSurv column)",test-set C-index of DeepSurv on the simulated linear survival data,C-index,within,0.774019,0.773885,0.01,within ±0.01,consistent,True,,27,55,-0.0134,equivalence,False,0.999827,arxiv-1606-00931-20261018T225505Z | |
| 1606.00931,DeepSurv,sim-nonlinear-cindex,False,claim,DeepSurv reaches test C-index 0.648902 on simulated nonlinear data.,"Table 1, p. 5 (DeepSurv column)",test-set C-index of DeepSurv on the simulated nonlinear (Gaussian) survival data,C-index,within,0.648902,0.645797,0.01,within ±0.01,consistent,True,,28,56,-0.3105,equivalence,False,0.995215,arxiv-1606-00931-20261018T225505Z | |
| 1711.10561,PINNs,burgers-l2-order,True,claim,"A physics-informed network (9 layers x 20 neurons, N_u = 100, N_f = 10,000) solves the Burgers equation with relative L2 error of order 1e-4 (reported 6.7e-4).","Section 3.1, Fig. 1 caption, p. 8","relative L2 error between predicted and exact u(t, x), printed by Burgers.py as Error u",relative L2,at_most,0.00067,0.002148728,0.00603,at most 0.0067,consistent,True,,8,18,0.2452,bound,True,3.207057,arxiv-1711-10561-20261018T234638Z | |
| 1711.10561,PINNs,burgers-l2-close,False,claim,The Burgers relative L2 error is within a factor of 2 of the reported 6.7e-4.,"Section 3.1, Fig. 1 caption, p. 8",relative L2 error printed by Burgers.py as Error u,relative L2,at_most,0.00067,0.002148728,0.00067,at most 0.00134,inconsistent,True,,9,19,2.2071,bound,True,3.207057,arxiv-1711-10561-20261018T234638Z | |
| 1607.01759,fastText,ag-bigram,True,claim,fastText with 10 hidden units and bigrams reaches 92.5% test accuracy on AG News.,"Table 1, p. 3","test accuracy [%], AG News, fastText h=10 with bigrams",percent,within,92.5,92.2,0.3,within ±0.3,consistent,True,,10,33,-1.0,equivalence,False,0.996757,arxiv-1607-01759-20261018T223713Z | |
| 1607.01759,fastText,ag-unigram,False,claim,fastText with 10 hidden units without bigrams reaches 91.5% test accuracy on AG News.,"Table 1, p. 3","test accuracy [%], AG News, fastText h=10 without bigrams",percent,within,91.5,91.4,0.3,within ±0.3,consistent,True,,11,41,-0.3333,equivalence,False,0.998907,arxiv-1607-01759-20261018T223713Z | |
| 1607.01759,fastText,sogou-bigram,True,claim,fastText with 10 hidden units and bigrams reaches 96.8% test accuracy on Sogou News.,"Table 1, p. 3","test accuracy [%], Sogou News, fastText h=10 with bigrams",percent,within,96.8,96.7,0.3,within ±0.3,consistent,True,,12,34,-0.3333,equivalence,False,0.998967,arxiv-1607-01759-20261018T223713Z | |
| 1607.01759,fastText,sogou-unigram,False,claim,fastText with 10 hidden units without bigrams reaches 93.9% test accuracy on Sogou News.,"Table 1, p. 3","test accuracy [%], Sogou News, fastText h=10 without bigrams",percent,within,93.9,93.7,0.3,within ±0.3,consistent,True,,13,42,-0.6667,equivalence,False,0.99787,arxiv-1607-01759-20261018T223713Z | |
| 1607.01759,fastText,dbpedia-bigram,True,claim,fastText with 10 hidden units and bigrams reaches 98.6% test accuracy on DBpedia.,"Table 1, p. 3","test accuracy [%], DBpedia, fastText h=10 with bigrams",percent,within,98.6,98.6,0.3,within ±0.3,consistent,True,,14,35,0.0,equivalence,False,1.0,arxiv-1607-01759-20261018T223713Z | |
| 1607.01759,fastText,dbpedia-unigram,False,claim,fastText with 10 hidden units without bigrams reaches 98.1% test accuracy on DBpedia.,"Table 1, p. 3","test accuracy [%], DBpedia, fastText h=10 without bigrams",percent,within,98.1,98.3,0.3,within ±0.3,consistent,True,,15,43,0.6667,equivalence,False,1.002039,arxiv-1607-01759-20261018T223713Z | |
| 1607.01759,fastText,yelp-p-bigram,True,claim,fastText with 10 hidden units and bigrams reaches 95.7% test accuracy on Yelp Polarity.,"Table 1, p. 3","test accuracy [%], Yelp Polarity, fastText h=10 with bigrams",percent,within,95.7,95.7,0.3,within ±0.3,consistent,True,,16,36,0.0,equivalence,False,1.0,arxiv-1607-01759-20261018T223713Z | |
| 1607.01759,fastText,yelp-p-unigram,False,claim,fastText with 10 hidden units without bigrams reaches 93.8% test accuracy on Yelp Polarity.,"Table 1, p. 3","test accuracy [%], Yelp Polarity, fastText h=10 without bigrams",percent,within,93.8,93.9,0.3,within ±0.3,consistent,True,,17,44,0.3333,equivalence,False,1.001066,arxiv-1607-01759-20261018T223713Z | |
| 1607.01759,fastText,yelp-f-bigram,True,claim,fastText with 10 hidden units and bigrams reaches 63.9% test accuracy on Yelp Full.,"Table 1, p. 3","test accuracy [%], Yelp Full, fastText h=10 with bigrams",percent,within,63.9,63.9,0.3,within ±0.3,consistent,True,,18,37,0.0,equivalence,False,1.0,arxiv-1607-01759-20261018T223713Z | |
| 1607.01759,fastText,yelp-f-unigram,False,claim,fastText with 10 hidden units without bigrams reaches 60.4% test accuracy on Yelp Full.,"Table 1, p. 3","test accuracy [%], Yelp Full, fastText h=10 without bigrams",percent,within,60.4,60.3,0.3,within ±0.3,consistent,True,,19,45,-0.3333,equivalence,False,0.998344,arxiv-1607-01759-20261018T223713Z | |
| 1607.01759,fastText,yahoo-bigram,True,claim,fastText with 10 hidden units and bigrams reaches 72.3% test accuracy on Yahoo! Answers.,"Table 1, p. 3","test accuracy [%], Yahoo! Answers, fastText h=10 with bigrams",percent,within,72.3,72.4,0.3,within ±0.3,consistent,True,,20,38,0.3333,equivalence,False,1.001383,arxiv-1607-01759-20261018T223713Z | |
| 1607.01759,fastText,yahoo-unigram,False,claim,fastText with 10 hidden units without bigrams reaches 72.0% test accuracy on Yahoo! Answers.,"Table 1, p. 3","test accuracy [%], Yahoo! Answers, fastText h=10 without bigrams",percent,within,72.0,71.8,0.3,within ±0.3,consistent,True,,21,46,-0.6667,equivalence,False,0.997222,arxiv-1607-01759-20261018T223713Z | |
| 1607.01759,fastText,amazon-f-bigram,True,claim,fastText with 10 hidden units and bigrams reaches 60.2% test accuracy on Amazon Full.,"Table 1, p. 3","test accuracy [%], Amazon Full, fastText h=10 with bigrams",percent,within,60.2,60.2,0.3,within ±0.3,consistent,True,,22,39,0.0,equivalence,False,1.0,arxiv-1607-01759-20261018T223713Z | |
| 1607.01759,fastText,amazon-f-unigram,False,claim,fastText with 10 hidden units without bigrams reaches 55.8% test accuracy on Amazon Full.,"Table 1, p. 3","test accuracy [%], Amazon Full, fastText h=10 without bigrams",percent,within,55.8,55.6,0.3,within ±0.3,consistent,True,,23,47,-0.6667,equivalence,False,0.996416,arxiv-1607-01759-20261018T223713Z | |
| 1607.01759,fastText,amazon-p-bigram,True,claim,fastText with 10 hidden units and bigrams reaches 94.6% test accuracy on Amazon Polarity.,"Table 1, p. 3","test accuracy [%], Amazon Polarity, fastText h=10 with bigrams",percent,within,94.6,94.6,0.3,within ±0.3,consistent,True,,24,40,0.0,equivalence,False,1.0,arxiv-1607-01759-20261018T223713Z | |
| 1607.01759,fastText,amazon-p-unigram,False,claim,fastText with 10 hidden units without bigrams reaches 91.2% test accuracy on Amazon Polarity.,"Table 1, p. 3","test accuracy [%], Amazon Polarity, fastText h=10 without bigrams",percent,within,91.2,91.1,0.3,within ±0.3,consistent,True,,25,48,-0.3333,equivalence,False,0.998904,arxiv-1607-01759-20261018T223713Z | |
| 2004.01105,The COVID-19 change-point model,lambda0,True,claim,Before the first change point the spreading rate in Germany was 0.43 per day.,"Abstract, p. 1","posterior median of lambda_0, three-change-point model, data until 2020-04-21",per day,within,0.43,0.4275,0.02,within ±0.02,consistent,True,,7,31,-0.125,equivalence,False,0.994186,arxiv-2004-01105-20261018T033025Z | |
| 2004.01105,The COVID-19 change-point model,lambda1,True,claim,After the first change point the spreading rate was 0.25 per day.,"Abstract, p. 1",posterior median of lambda_1,per day,within,0.25,0.2509,0.02,within ±0.02,consistent,True,,8,32,0.045,equivalence,False,1.0036,arxiv-2004-01105-20261018T033025Z | |
| 2004.01105,The COVID-19 change-point model,lambda2,True,claim,After the second change point the spreading rate was 0.15 per day.,"Abstract, p. 1",posterior median of lambda_2,per day,within,0.15,0.1544,0.02,within ±0.02,consistent,True,,9,33,0.22,equivalence,False,1.029333,arxiv-2004-01105-20261018T033025Z | |
| 2004.01105,The COVID-19 change-point model,lambda3,True,claim,After the third change point the spreading rate was 0.09 per day.,"Abstract, p. 1",posterior median of lambda_3,per day,within,0.09,0.0937,0.02,within ±0.02,consistent,True,,10,34,0.185,equivalence,False,1.041111,arxiv-2004-01105-20261018T033025Z | |
| 2004.01105,The COVID-19 change-point model,cp1-date,False,claim,"The first change point occurred around March 6, 2020.","Abstract, p. 1","posterior median date of change point 1, as day of March 2020",day of March,within,6,5.79,2,within ±2,consistent,True,,11,35,-0.105,equivalence,False,0.965,arxiv-2004-01105-20261018T033025Z | |
| 2004.01105,The COVID-19 change-point model,cp2-date,False,claim,"The second change point occurred around March 15, 2020.","Abstract, p. 1","posterior median date of change point 2, as day of March 2020",day of March,within,15,15.22,2,within ±2,consistent,True,,12,36,0.11,equivalence,False,1.014667,arxiv-2004-01105-20261018T033025Z | |
| 2004.01105,The COVID-19 change-point model,cp3-date,False,claim,"The third change point occurred around March 23, 2020.","Abstract, p. 1","posterior median date of change point 3, as day of March 2020",day of March,within,23,22.63,2,within ±2,consistent,True,,13,37,-0.185,equivalence,False,0.983913,arxiv-2004-01105-20261018T033025Z | |
| 1810.08473,The Leiden algorithm,dblp-leiden,True,claim,"On DBLP, the maximal modularity found by the Leiden algorithm over 10 replications of 10 iterations is 0.8387.","Table II, p. 6 (text p. 9)","max modularity (gamma = 1) over 10 random starts x 10 iterations, Leiden, SNAP com-DBLP",modularity,within,0.8387,0.838377,0.002,within ±0.002,consistent,True,,10,29,-0.1615,equivalence,False,0.999615,arxiv-1810-08473-20261018T013946Z | |
| 1810.08473,The Leiden algorithm,amazon-leiden,True,claim,"On Amazon, the maximal modularity found by the Leiden algorithm is 0.9341.","Table II, p. 6 (text p. 9)","max modularity (gamma = 1) over 10 random starts x 10 iterations, Leiden, SNAP com-Amazon",modularity,within,0.9341,0.934022,0.002,within ±0.002,consistent,True,,11,30,-0.039,equivalence,False,0.999916,arxiv-1810-08473-20261018T013946Z | |
| 1810.08473,The Leiden algorithm,dblp-louvain,False,claim,"On DBLP, the maximal modularity found by the Louvain algorithm is 0.8262.","Table II, p. 6 (text p. 9)","max modularity (gamma = 1) over 10 random starts x 10 iterations, Louvain, SNAP com-DBLP",modularity,within,0.8262,0.826653,0.002,within ±0.002,consistent,True,,12,31,0.2265,equivalence,False,1.000548,arxiv-1810-08473-20261018T013946Z | |
| 1810.08473,The Leiden algorithm,amazon-louvain,False,claim,"On Amazon, the maximal modularity found by the Louvain algorithm is 0.9301.","Table II, p. 6 (text p. 9)","max modularity (gamma = 1) over 10 random starts x 10 iterations, Louvain, SNAP com-Amazon",modularity,within,0.9301,0.929846,0.002,within ±0.002,consistent,True,,13,32,-0.127,equivalence,False,0.999727,arxiv-1810-08473-20261018T013946Z | |
| 1810.08473,The Leiden algorithm,dblp-leiden-gain,True,claim,"On DBLP, Leiden finds higher maximal modularity than Louvain (paper: +0.0125).","Table II, p. 6 (text p. 9)",Leiden max modularity minus Louvain max modularity on DBLP,modularity,at_least,0.0125,0.011724,0.0125,at least 0,consistent,True,,14,33,-0.0621,bound,True,0.93792,arxiv-1810-08473-20261018T013946Z | |
| 1810.08473,The Leiden algorithm,amazon-leiden-gain,True,claim,"On Amazon, Leiden finds higher maximal modularity than Louvain (paper: +0.0040).","Table II, p. 6 (text p. 9)",Leiden max modularity minus Louvain max modularity on Amazon,modularity,at_least,0.004,0.004176,0.004,at least 0,consistent,True,,15,34,0.044,bound,True,1.044,arxiv-1810-08473-20261018T013946Z | |
| 1812.09970,Synthetic Difference-in-Differences,sdid-estimate,True,claim,The SDID estimate of the average effect of Proposition 99 on California per-capita cigarette sales over the 12 post-treatment years is -15.6 packs.,"Table 1, p. 8",SDID point estimate (synthdid_estimate on the California Prop 99 panel),packs per capita,within,-15.6,-15.603828,0.1,within ±0.1,consistent,True,,9,45,-0.0383,equivalence,False,1.000245,arxiv-1812-09970-20261018T010328Z | |
| 1812.09970,Synthetic Difference-in-Differences,sdid-se,True,claim,The placebo-method standard error of the SDID estimate is 8.4.,"Table 1, p. 8 (placebo method, Section 5, Algorithm 4)",sqrt of the placebo variance estimate for SDID,packs per capita,within,8.4,8.367993,0.84,within ±0.84,consistent,True,,10,50,-0.0381,equivalence,False,0.99619,arxiv-1812-09970-20261018T010328Z | |
| 1812.09970,Synthetic Difference-in-Differences,sc-estimate,True,claim,The synthetic control estimate is -19.6.,"Table 1, p. 8",SC point estimate (sc_estimate),packs per capita,within,-19.6,-19.619663,0.1,within ±0.1,consistent,True,,11,46,-0.1966,equivalence,False,1.001003,arxiv-1812-09970-20261018T010328Z | |
| 1812.09970,Synthetic Difference-in-Differences,did-estimate,True,claim,The difference-in-differences estimate is -27.3.,"Table 1, p. 8",DID point estimate (did_estimate),packs per capita,within,-27.3,-27.349111,0.1,within ±0.1,consistent,True,,12,47,-0.4911,equivalence,False,1.001799,arxiv-1812-09970-20261018T010328Z | |
| 1812.09970,Synthetic Difference-in-Differences,mc-estimate,False,claim,The matrix completion estimate is -20.2.,"Table 1, p. 8",MC point estimate (MCPanel nuclear-norm estimator as used in the authors code),packs per capita,within,-20.2,-20.218135,0.5,within ±0.5,consistent,True,,13,48,-0.0363,equivalence,False,1.000898,arxiv-1812-09970-20261018T010328Z | |
| 1812.09970,Synthetic Difference-in-Differences,difp-estimate,False,claim,The synthetic control with intercept (DIFP) estimate is -11.1.,"Table 1, p. 8",DIFP point estimate,packs per capita,within,-11.1,-11.104645,0.1,within ±0.1,consistent,True,,14,49,-0.0465,equivalence,False,1.000418,arxiv-1812-09970-20261018T010328Z | |
| 1812.09970,Synthetic Difference-in-Differences,sc-se,False,claim,The placebo-method standard error of the SC estimate is 9.9.,"Table 1, p. 8",sqrt of the placebo variance estimate for SC,packs per capita,within,9.9,9.917426,0.99,within ±0.99,consistent,True,,15,51,0.0176,equivalence,False,1.00176,arxiv-1812-09970-20261018T010328Z | |
| 1812.09970,Synthetic Difference-in-Differences,did-se,False,claim,The placebo-method standard error of the DID estimate is 17.7.,"Table 1, p. 8",sqrt of the placebo variance estimate for DID,packs per capita,within,17.7,17.740267,1.77,within ±1.77,consistent,True,,16,52,0.0227,equivalence,False,1.002275,arxiv-1812-09970-20261018T010328Z | |
| 1812.09970,Synthetic Difference-in-Differences,mc-se,False,claim,The placebo-method standard error of the MC estimate is 11.5.,"Table 1, p. 8",sqrt of the placebo variance estimate for MC,packs per capita,within,11.5,11.509864,1.15,within ±1.15,consistent,True,,17,53,0.0086,equivalence,False,1.000858,arxiv-1812-09970-20261018T010328Z | |
| 1812.09970,Synthetic Difference-in-Differences,difp-se,False,claim,The placebo-method standard error of the DIFP estimate is 9.5.,"Table 1, p. 8",sqrt of the placebo variance estimate for DIFP,packs per capita,within,9.5,9.45566,0.95,within ±0.95,consistent,True,,18,54,-0.0467,equivalence,False,0.995333,arxiv-1812-09970-20261018T010328Z | |