{"item":"t03-block-a","batch":"confirm-t03","mode":"confirm (real engine; no search, no port)","candidate":"T03","reference":"IB","seeds":[8700000,8701023],"n_seeds":1024,"protocol":"each strategy alone vs the 30 bps VanillaStrategy: amm_sim_rs.run_batch(X, VanillaStrategy, configs, 8), one call per arm per 64-seed chunk (16 chunks, each its own command, saved before the next); configs = the real MatchRunner._build_configs body (build_base_config(seed=None), BASELINE_VARIANCE) iterated over the block's seeds, so rng seed = config seed = market seed; every seed in 8700000..8701023 used once per arm, none outside the block in this measurement","metric":"edge = LightweightSimResult.edges['submission'] (the strategy under test); diff = CAND edge minus IB edge on the same seed/config","paired_vs_IB":{"n":1024,"mean_diff":0.8004589765907124,"se":0.025905856501574405,"sd":0.828987408050381,"sum":819.6699920288895,"sumsq":1359.1383884898923,"median":0.761940647276873,"share_positive":0.8681640625,"worst5":[[8700653,-3.4831758285099568],[8700380,-1.927097338277008],[8700460,-1.567357790348069],[8700002,-1.4480290898752628],[8700435,-1.3516864205953425]],"best5":[[8700658,7.615474217148403],[8700388,5.130114092103099],[8700804,5.105919550321289],[8700661,4.427500837684647],[8700902,4.184480892642114]],"t_stat":30.898765170804726,"excluding_largest_gain_seed":{"seed":8700658,"n":1023,"mean_diff":0.7937971826116726,"se":0.025059137798260932}},"diffs":[-0.4112,0.7674,-1.448,0.0255,0.511,1.0523,0.3414,0.2158,2.3962,1.095,3.7074,0.4137,1.7045,0.7037,0.7995,0.6118,0.1828,0.3818,2.1743,-1.3289,1.5562,-0.3179,1.0685,1.2035,2.0684,1.0995,0.6192,-0.0135,1.7643,0.2487,0.2874,0.4512,1.9604,1.2776,2.0026,0.4128,0.7607,-0.8062,-0.04,0.3132,1.7967,1.2187,0.0483,1.0875,-0.4942,1.0745,-0.233,0.3796,0.6022,0.9634,0.7989,1.3585,0.1453,1.0384,0.4344,1.0132,1.3002,0.7243,1.2572,2.6966,0.9367,0.2686,-0.0636,2.0709,0.5197,0.5979,1.9204,2.4567,0.1837,-0.605,-0.0654,0.4207,1.0944,0.8462,3.4726,0.7931,0.6342,0.2038,0.6994,0.6642,-0.3609,1.1108,0.8618,0.0159,1.4092,1.3089,0.514,3.4611,0.1014,-0.1072,0.2316,1.2834,1.2013,0.917,0.8522,0.3678,1.0691,0.6132,1.5211,1.5915,0.9879,0.7474,0.7485,1.4752,1.0906,0.3722,1.7092,0.8734,1.5078,0.56,0.2379,0.6746,0.5272,0.222,1.6364,1.3691,0.8975,1.2214,0.4757,0.9485,0.9735,0.6456,0.4094,-0.3072,-0.5303,0.5504,-0.6986,0.3507,1.9168,0.5323,0.1912,1.2708,2.2768,-0.1187,1.3074,1.6547,0.3,0.7223,-0.0857,0.6867,0.3615,0.1815,1.2136,1.697,2.4025,0.2361,-0.673,0.5431,0.2512,1.4975,-0.1662,0.2422,1.2853,0.8854,2.2508,-0.11,0.0163,1.2512,0.2316,0.0052,0.5331,1.1647,0.7635,1.9758,0.4544,1.0645,3.6873,2.8446,1.0631,0.7043,-0.1735,0.572,0.4461,0.3009,0.5941,1.193,0.53,1.1796,0.4678,-0.3436,0.0944,1.7838,-0.0644,0.8753,-0.1496,1.5281,1.569,0.022,0.1845,1.1771,1.3457,0.6206,0.9485,1.8355,-0.1619,1.1499,1.1983,1.3335,-0.2145,1.2127,1.1396,0.2889,0.3883,0.853,0.1317,0.7278,1.6625,1.9438,0.4066,2.4963,1.2573,1.2024,0.5226,0.2505,0.3974,0.8027,0.8508,0.5244,1.1152,1.1634,-0.4458,0.7368,-0.6088,-0.205,1.1389,1.2491,1.5514,1.1338,1.1134,1.9608,1.7996,0.4326,0.5106,0.975,1.5572,0.8665,1.6737,1.2587,0.9975,1.6553,0.191,0.5464,0.5608,0.3076,-0.8829,-0.5427,1.1516,0.2591,0.5804,0.1239,0.075,1.1321,0.5477,1.4812,1.4817,0.7159,0.525,-0.5107,1.7861,0.5982,-0.1569,0.1876,0.1551,-0.2367,1.2358,-0.8002,1.0892,0.5292,-0.17,0.4968,0.896,0.7536,0.0731,2.1339,-0.0194,1.0628,1.0818,0.3499,-0.0488,0.2417,0.9092,0.5627,0.5463,1.0614,0.1082,1.5306,1.4733,0.8706,0.6722,1.1976,-0.1237,0.3896,0.9115,0.6232,1.0617,0.8132,0.3123,1.4868,1.3859,2.033,1.6623,-0.0297,0.6834,1.6769,1.5135,1.2066,-0.0914,0.1849,0.6139,0.9162,0.198,-0.1612,0.5501,2.2902,0.8033,0.6094,-0.2303,2.7693,1.035,1.823,1.8045,1.6748,2.3337,0.9362,-0.2042,0.6117,0.3134,1.1805,0.2902,0.5129,0.774,1.4805,0.5496,0.2687,0.3227,1.3669,0.9236,1.2598,1.9895,0.9831,1.5137,2.1469,1.3956,-0.166,-0.69,-0.2371,0.7648,1.7859,2.1086,0.8878,0.9924,1.5984,2.5735,0.5393,0.0287,0.8879,1.1283,0.3709,-0.4563,0.497,1.3591,1.1888,1.2235,1.0558,0.3757,0.6295,0.2352,0.9348,1.6359,1.1876,1.7387,1.4995,0.7618,1.2413,1.5287,-0.3825,1.3663,0.1126,0.1763,1.6489,-1.9271,1.089,0.6137,2.367,0.5365,2.5494,1.2961,0.6073,5.1301,1.3163,0.8469,-0.4452,0.0484,0.4793,0.6174,0.9034,1.1106,0.0829,-0.0933,0.6048,1.3908,0.7991,0.3618,0.2057,0.223,2.4371,1.3121,1.8191,0.3036,0.7613,1.1458,1.2649,0.8966,1.3319,0.9974,0.4242,1.6454,0.4623,1.1282,0.4707,0.9459,1.5061,0.2401,-0.5785,1.0086,1.1426,-0.5416,1.5543,1.1338,1.3723,0.9122,0.9388,0.8639,0.4975,2.368,-1.3517,1.9666,1.3246,-0.2891,-0.189,1.0633,0.6065,0.3573,0.4562,0.6227,-0.0867,0.934,0.6253,1.4995,0.9041,0.258,-0.154,0.4938,0.0613,-0.063,0.6885,0.7639,0.816,0.3988,1.1564,-1.5674,0.342,0.5417,0.9678,0.3248,1.1729,0.3572,1.5595,-0.0099,1.3415,1.3357,0.3374,0.2288,1.4766,0.2674,1.004,0.6847,0.5595,0.2812,0.7621,0.9748,0.7915,0.9889,2.2447,1.3976,0.7152,0.3473,1.5706,1.6403,1.1475,0.6299,1.3234,-0.4242,0.3767,0.5548,1.0315,1.0772,0.0012,-0.4159,0.0926,0.3186,0.5984,0.8955,0.1339,0.9519,0.7997,0.4736,2.2002,1.338,1.1361,2.3054,0.8479,1.4652,-0.0289,0.6709,0.3136,0.5706,1.0943,0.2943,0.0623,0.8086,-0.382,0.2372,0.7318,0.7714,0.7074,1.3845,-0.4994,-0.0062,1.354,0.9907,0.8905,1.0793,1.3094,1.1863,3.1935,0.0343,1.6198,0.2305,0.5104,1.3016,1.2886,0.5962,-0.2861,0.6339,1.0258,2.1811,0.8487,1.1129,1.1779,1.2972,1.0349,0.6374,0.6303,0.1379,1.1056,0.5894,0.5303,0.1147,0.5543,0.6684,0.446,1.0147,0.3288,0.4944,-0.9765,0.1197,0.6821,0.5549,0.7645,-0.0216,1.9794,0.8298,1.1329,0.234,-0.0037,1.245,1.3449,1.9537,0.3188,0.3045,0.7322,1.5202,0.2641,-0.077,-0.6088,2.6962,1.7313,0.3684,0.1335,-0.1072,-0.2951,1.8579,1.2185,0.6956,-0.1746,1.5095,-0.2282,0.6104,0.3222,0.9241,1.0134,0.8241,0.5943,0.3941,0.0714,0.0897,0.1423,1.119,1.0727,-0.5385,0.5372,0.6458,0.5252,1.0723,0.7489,-0.4888,0.9848,0.8787,0.6283,1.5754,0.9928,1.214,0.1446,1.1385,0.0193,0.8941,-0.7386,0.9461,1.3942,1.7221,0.7062,0.8747,0.6588,0.5724,2.7784,0.3331,1.6624,0.6625,0.1574,1.0442,-0.8582,1.0023,1.3866,1.4293,0.0207,1.7838,1.2807,0.7283,0.5449,-0.1761,1.7918,0.3246,-3.4832,1.1019,1.7937,0.77,0.2215,7.6155,1.2954,1.9553,4.4275,0.6863,1.0864,0.3229,0.4166,0.6345,1.5319,1.2753,0.9051,0.3495,-0.5281,-0.4036,0.7349,1.0854,0.9783,0.7813,0.4116,-0.2639,0.5589,1.0256,-1.2802,0.7629,-0.7586,0.6311,1.3679,1.6029,0.5294,-0.0159,-0.1732,1.3826,1.6574,1.0354,0.6427,1.5619,1.9576,0.8093,0.7472,2.5269,1.8671,0.6202,1.5979,-0.544,-0.0313,0.6206,1.0137,0.3555,1.6909,0.8239,1.6573,0.3858,0.9509,0.6908,0.2149,0.7215,0.1814,1.3364,0.8968,0.587,1.3103,-0.0376,-0.1492,0.8555,0.8554,0.8363,-0.6836,0.2004,1.2632,-0.8468,1.5692,0.5943,1.266,1.8632,0.9258,1.3586,1.1706,0.7845,0.731,0.8267,0.547,2.9547,0.183,1.6319,-0.2013,1.9683,-1.1752,1.3314,-0.6539,-0.213,0.1177,0.0305,0.4082,2.3364,0.5131,1.2395,1.3027,2.6965,-1.0346,1.1044,0.6245,-0.1119,0.8844,-0.3265,0.4103,-0.0991,0.0456,-0.014,0.6506,-0.4108,0.0169,-0.5128,1.2638,0.703,0.1957,0.7668,2.1413,1.5018,1.0508,0.3618,1.1769,0.7509,0.7003,0.7726,1.4234,-0.6766,-0.4062,0.9194,-0.5653,0.3368,1.3192,1.4196,1.4624,1.6651,0.1991,-0.9008,1.8823,0.1958,0.1571,1.4099,1.6236,0.9102,0.2724,0.1203,1.2665,5.1059,1.1711,1.4645,-0.6513,0.7811,1.951,0.8112,0.259,0.7122,0.3542,0.2544,0.1965,1.6508,-0.1759,0.5167,0.84,0.777,0.8865,0.5774,1.28,0.7051,0.7614,0.7562,0.9583,-1.301,0.1244,1.6226,0.4627,1.3152,0.4982,2.0126,1.2266,0.882,0.4128,0.8185,0.0442,0.5264,1.4918,1.1635,1.028,0.4661,0.8338,1.1846,-0.8603,0.5437,-0.4146,1.1981,0.9467,1.2989,0.5319,0.7651,0.9275,-0.4742,2.3831,1.3753,-0.2941,1.0762,0.6802,0.6202,1.0577,1.4616,2.1141,0.7191,1.6085,0.6929,2.1353,0.0039,0.8047,0.6802,0.9644,0.4031,0.8621,1.0113,1.001,0.4847,0.7985,0.6362,0.8917,0.7009,0.9048,1.3153,-0.3838,-0.6106,0.339,0.5804,-0.0584,-0.1802,0.8337,1.206,1.0323,0.2662,0.7561,1.3516,1.6298,-0.6133,0.7429,0.4669,0.8237,4.1845,1.3049,1.4106,0.3313,0.4681,0.1657,1.2315,2.1323,1.0443,0.0968,1.5779,0.2506,-0.9788,1.0189,1.3996,0.4286,0.5862,0.6809,0.7112,-0.0582,1.2201,0.1234,0.714,1.7705,0.3025,1.2537,2.0661,0.9617,0.4978,-0.3752,0.2338,1.0144,0.3899,0.743,1.7392,0.7962,0.6996,1.5929,1.1157,0.3138,0.3366,0.5178,1.2626,1.0652,0.3165,-0.449,0.2545,0.2511,0.6746,0.2919,0.9273,0.3834,0.8667,0.6125,0.7852,4.1154,1.2012,1.6405,1.2346,0.2886,1.2713,0.7914,0.8983,0.6063,1.4368,1.1465,0.5336,1.1036,0.8767,-0.3904,-0.5039,0.5644,0.6606,0.853,1.7133,1.0281,0.6788,0.6126,0.9286,0.0756,0.3262,1.7997,1.3409,1.9065,1.9791,0.9578,0.9979,1.5016,0.7393,1.0734,0.4856,-0.0363,2.2989,0.7519,1.3713,2.1896,0.4709,1.3529,0.3097,-0.631,0.9631,2.6056,0.7047,0.2132,1.3278,0.8993,2.2635,0.4698,0.5254,2.3049,-0.4239,0.7824,1.9697,2.2743,0.0904,1.9922,1.5037,1.1899,1.2352,0.0599,2.0997,0.586],"chunk_mean_diffs":[0.7849,0.8043,0.8167,0.8204,0.736,0.9172,0.8496,0.7321,0.7087,0.7232,0.8764,0.6871,0.7401,0.7694,0.8534,0.988],"by_quartile":{"basis":"IB's own retail_volume_y (submission side, Y units) in the IB arm; q1 = quietest 256 markets","q1":{"n":256,"mean_diff":0.886670350835344,"se":0.04025009305431673,"ib_retail_volume_y_range":[57960.22164738987,68966.1187142567]},"q2":{"n":256,"mean_diff":0.7830938256549718,"se":0.049660112858378715,"ib_retail_volume_y_range":[69023.92022776944,78272.50147967698]},"q3":{"n":256,"mean_diff":0.7531496589964053,"se":0.06037811956249678,"ib_retail_volume_y_range":[78294.26072456667,86706.37517355489]},"q4":{"n":256,"mean_diff":0.7789220708761291,"se":0.05474264125612228,"ib_retail_volume_y_range":[86715.58939134999,100441.68485308133]}},"retail_volume_y_delta":{"mean":280.9070082957871,"se":10.129110141850765,"note":"CAND minus IB, the tested strategy's retail volume (Y units) per market"},"arb_volume_y_delta":{"mean":-53.61498558413648,"se":6.565296902825818,"note":"CAND minus IB, the tested strategy's arbitrage volume (Y units) per market"},"pnl_delta":{"mean":0.793441959829039,"se":0.03271095050066171},"vanilla_edge_delta":{"mean":-1.015730641691239,"se":0.04374791090363574,"note":"the 30 bps normalizer's edge in CAND's market minus in IB's market"},"retail_arb_edge_split":null,"retail_arb_edge_split_note":"run_batch results expose edges, pnl, retail_volume_y and arb_volume_y per strategy, not a retail/arb split of edge, so only volume deltas are reported","mean_edge":{"IB":525.9683261521303,"CAND":526.7687851287205,"vanilla_in_IB_arm":350.5938738158806,"vanilla_in_CAND_arm":349.57814317418894},"mean_volume_y":{"IB_retail":78142.4784282976,"CAND_retail":78423.38543659353,"IB_arb":24361.067336457458,"CAND_arb":24307.452350873304},"gas":{"IB":{"afterSwap_peak":237141,"afterInitialize_peak":243227,"getName_peak":21735,"oog":0,"other_failures":{"revert":0,"other_halt":0,"tx_error":0,"bad_return":0},"calls":{"afterInitialize":1024,"afterSwap":11023297,"getName":1024},"ok":{"afterInitialize":1024,"afterSwap":11023297,"getName":1024},"afterSwap_mean_gas":135235.09844268917,"run_errors":[],"vanilla_normalizer":{"oog":0,"other_failures":0,"calls":{"afterInitialize":1024,"afterSwap":10977796,"getName":1024},"afterSwap_peak":22592}},"CAND":{"afterSwap_peak":237789,"afterInitialize_peak":243234,"getName_peak":21742,"oog":0,"other_failures":{"revert":0,"other_halt":0,"tx_error":0,"bad_return":0},"calls":{"afterInitialize":1024,"afterSwap":11107688,"getName":1024},"ok":{"afterInitialize":1024,"afterSwap":11107688,"getName":1024},"afterSwap_mean_gas":135574.9387330649,"run_errors":[],"vanilla_normalizer":{"oog":0,"other_failures":0,"calls":{"afterInitialize":1024,"afterSwap":10971232,"getName":1024},"afterSwap_peak":22592}},"limit_afterSwap":245000,"aggregation":"per arm: gas_stats_reset() before and gas_stats() after each 64-seed run_batch; counts summed over 16 chunks, peaks = max over chunks; role Submission = the strategy under test"},"valid":true,"valid_by_arm":{"IB":true,"CAND":true},"code_sha256":{"IB":"301941829a44bb60977bb28181a5e4bfa6544f3696763f987054789b2611d686","CAND":"cb6d945511e1e10af78baa64c2a6cb25806f0998403ce6440654bfc3c70092d9"},"bytecode_sha256":{"IB":"345cbfe273190c3b629d6e624b4f19ecae919477786d13539b8bdd1ca87ed8b0","CAND":"0db05306eccdc1971e2d42d74f8ed040e359d366cd1f4b7e461c46cb442b6c7b"},"engine_seconds":{"IB":161.8,"CAND":164.5},"engine":{"source":"benedictbrady/amm-challenge @ 6b6921f8d463314aa7c96234e1be25a25245f704 (codeload tarball of that commit) + instrumentation.patch from github.com/gtmcknight/amm-sim release sim-6b6921f8 (applied cleanly; amm_sim_rs version set to 0.1.0.post1), i.e. the source of the post1 wheel","build":"native macOS arm64 (Darwin 27.0.0): rustc 1.99.1 (the release recipe's version), cargo build --release --locked --lib --features pyo3/abi3-py310 with the crate's release profile (lto=true, codegen-units=1, opt-level=3), cdylib installed as amm_sim_rs/amm_sim_rs.abi3.so in a uv Python 3.12.16 venv","so_sha256":"cfd95e00c21633d0a2bac9ce3eb33a54dd486e10d5059cd82ca83f3f3732b925","differs_from_brief":"NOT the brief's wheel amm_sim_rs-0.1.0.post1-...-manylinux2014_x86_64 (sha256 aedb6828..., downloaded and sha256-verified, but a Linux x86_64 wheel cannot be installed or run on this macOS arm64 machine); the release README itself says a macOS build of the same recipe gives a different .so (Linux post1 .so is ff88b635...)","cargo_lock_sha256":"ca987048203502b8bb8f4e3fb6934176b02e00db66805ccb310ae7f573eb735f (generated 2026-11-09; the recipe's lock was b5efc7f1..., so at least one transitive version differs; resolved revm 18.0.0, pyo3 0.22.6, rand 0.8.8, rand_pcg 0.3.1, rand_distr 0.4.3)","maturin_not_used":"maturin 1.16.0 build failed before compiling: its cargo metadata step must unpack wasi-0.11.1's .gitmodules, which this machine's sandbox forbids writing; plain cargo build for the host target does not need that crate","python_stack":"uv 0.13.7 (aarch64-apple-darwin), CPython 3.12.16, numpy 2.5.4, py-solc-x 2.1.0, pyrevm 0.3.7, solc 0.8.24 via solcx; numba 0.68.1 and cma 4.5.0 installed per the brief but unused (no search in this item)"},"setup":{"as_written":"failed","blocked":{"step":"scratch/setup.sh line 6: uv venv --python 3.12 .venv","error":"scratch/bin/uv: cannot execute binary file (exit 126): the script installs uv-x86_64-unknown-linux-gnu and the manylinux2014_x86_64 amm_sim_rs wheel; this machine is macOS arm64. Steps before it passed: wheel download and its sha256 check printed OK"},"checkout":"git clone could not run here (sandbox refuses git writing .git/config), so the exact commit was taken as the GitHub codeload tarball of 6b6921f8d463314aa7c96234e1be25a25245f704","adaptation":"same venv layout and packages via a native uv; engine built from the same source + patch as above; amm-match validate contracts/src/StarterStrategy.sol passed"},"checks":{"artifact_sha256":"CAND artifact 2333ecf2... and IB artifact afd5183e... both match their ids","decoded_code_sha256":"CAND code_zb64 -> cb6d945511e1e10af78baa64c2a6cb25806f0998403ce6440654bfc3c70092d9, IB code_zb64 -> 301941829a44bb60977bb28181a5e4bfa6544f3696763f987054789b2611d686, both as the brief states","amm_match_validate":"StarterStrategy OK; IB validated as 'inputs_b'; CAND validated as 'censored_10'","config_builder":"the seed-block configs come from the real MatchRunner._build_configs body run over the block's seeds; identical in all 12 SimulationConfig fields to MatchRunner._build_configs() for seeds 0..39","engine_check":{"what":"native engine vs numbers published in the CAND artifact's own real-engine run (seeds 8588000..8589023, Linux wheel): the two seeds named there with their CAND-minus-IB diffs. Engine check only: these 2 seeds are outside this item's block, were not counted, and no measurement here uses them","seeds":[8588481,8588793],"published_diff":[55.663177802721634,-2.4439296329163653],"native_diff":[55.66317780264728,-2.44392963300254],"abs_gap":[7.4e-11,8.6e-11],"gas":"native afterInitialize peaks IB 243227 / CAND 243234 and getName 21735 / 21742 equal the published IB / candidate values","verdict":"agree to < 1e-10 per seed (not bit-identical, consistent with platform floating-point differences); same trade paths"}},"notes":["What ran: IB's and CAND's Solidity (as decoded, sha256 above), each alone vs the 30 bps VanillaStrategy, seeds 8700000..8701023, 16 chunks of 64 seeds, each chunk its own command (about 21 s each), results saved under scratch before the next chunk; gas_stats_reset() before and gas_stats() after every arm run, aggregated per arm (sum of counts, max of peaks). Every seed in the block used exactly once per arm; no seed outside the block in the measurement.","Engine caveat: measured on a native macOS arm64 build of the same engine source + instrumentation patch, not on the brief's Linux x86_64 wheel, which cannot run on this machine. The build reproduced two published per-seed CAND-minus-IB diffs to < 1e-10 (engine_check), but it is not byte-identical to the wheel's .so.","What did not run: no search, no port, no replica (this item is confirmation only, so replica speed was not measured); no leaderboard run; no retail/arb split of edge (the engine does not expose it), so only volume deltas are given.","by_quartile uses IB's own retail_volume_y in IB's markets (submission side) to rank markets; q1 is the 256 quietest.","All figures are from this run only. No leaderboard claim is made."]}