{ "approach": "Add distinct per-side marginal adverse-jump charges only at the incumbent ordinary bid/ask sizing call sites. Each uses the unchanged six-node transition distribution: lambda_j times the adverse payoff, times phat for bids or 1-phat for asks, times 2/(1+inferred competitor spread), times sqrt(min(1,500/steps_remaining)). The unchanged qsize applies its existing risk weight afterward. Belief, transition estimator, quote-price rules, acquisition rules, order management and inventory/cash constraints are unchanged. Realized acquisition fills and budget availability can change indirectly through the ordinary quotes. This is a state-dependent loss price, not a uniform quantity multiplier.", "tried": [ { "name": "side-jump-budget-v1", "code_sha256": "3b7236c98d52efed9617e4cf6d8183012c98b8b61b18eb59b5afae937c99c724", "parameters": { "extra_jump_charge_multiplier": 1.0, "bid_exposure": "phat", "ask_exposure": "1-phat", "spread_weight": "2/(1+self.s)", "horizon_weight": "sqrt(min(1,500/H))" }, "diagnostic_prefix": { "candidate_mean": 81.48544654305, "incumbent_mean": 81.661421308565, "mean_diff": -0.17597476551500116, "se": 0.5240319288836427, "n": 20, "seeds": { "first": 100000, "last": 100019, "inclusive": true }, "components": { "retail_edge": { "candidate_mean": 100.804435759045, "incumbent_mean": 101.37548713658501, "mean_diff": -0.571051377540002, "se": 0.46416380584321143 }, "arb_edge": { "candidate_mean": -19.318989215995, "incumbent_mean": -19.714065828065, "mean_diff": 0.3950766120700002, "se": 0.16235440273888366 } }, "failures": [] }, "diagnostic_note": "20 T seeds repeated under PyPy to check interpreter equivalence; not pooled with full T and not an independent sample. No parameter edits followed this screen.", "full_T": { "candidate_mean": 94.52561713573105, "incumbent_mean": 93.89195216102316, "mean_diff": 0.6336649747078998, "se": 0.5589493447545113, "n": 2000, "seeds": { "first": 100000, "last": 101999, "inclusive": true }, "components": { "retail_edge": { "candidate_mean": 116.4707125889687, "incumbent_mean": 118.1709640703087, "mean_diff": -1.7002514813399998, "se": 0.09587733467471832 }, "arb_edge": { "candidate_mean": -21.94509545321965, "incumbent_mean": -24.2790119092727, "mean_diff": 2.33391645605305, "se": 0.5550745225020117 } }, "failures": [] } } ], "next": "Independent measure items should evaluate the frozen candidate only if its evidence merits it; this item makes no promotion. Do not use H to alter this candidate. Reserved finalist seeds were not accessed.", "code": "\"\"\"Stale-quote defence with tick probes, a sub-tick belief and cheap YES stock.\n\nEvery step the arbitrageur removes any order on the wrong side of p, so tiny\ntwo-sided probes around the estimate locate p to a tick; a grid filter narrows\nthat further. Quotes sit one tick inside the competitor and are sized by the\nmarginal rule: one more share is worth quoting while the chance a retail order\nreaches it, times its edge, exceeds the expected arbitrage loss. Near p=0 a\nretail buy pays almost its whole notional as edge, so YES is held to cover low\nasks. That YES is bought when it is cheap: a large bid rests at tick 1 while p\nis above it, so the arbitrageur fills it on the crossing for about 0.1 tick a\nshare instead of nearly a full tick once p is near zero, and smaller bids sit on\nwhichever tick p is within 0.15 of. Below tick 1 the estimate of p decays with\nthe time spent there, since no probe can see it.\n\"\"\"\nimport math\nfrom orderbook_pm_challenge.strategy import BaseStrategy\nfrom orderbook_pm_challenge.types import CancelOrder, PlaceOrder, Side\n\nP = dict(lam_r=0.25, m=5.0, sig_n=1.2, lam_j=0.0019, sig_j=0.35, sig=0.02,\n qmax=3000.0, rmin=2, kA=0.7, kL=0.15, pL=0.05,\n ammo=1.0, ydem=0.4, W=200.0, fy=0.6, ammo_q=60.0, ammo_pmax=0.05, ammo_maxpay=0.02,\n fb=0.9, fu=0.7, xu=0.1, sticky=1.0, min_ahead=2.0, keep_hi=1.5, keep_lo=0.7, Hu=300.0, bins=8.0, filt=1.0, uni=0.01, jumpinfo=1.0, dmin=0.0, kLb=0.08, kLf=0.08, pinf=1.0, pre=0.0, H1=100.0,\n ydf=0.6, Wf=2000.0, g_hi=0.15, g_lo=0.15, qf=400.0, pmaxf=0.05, pre_gap=1.0,\n ydf1=0.6, g1=5.0, f1=0.6, a1=100.0, nb1=0.0, gs=1.0, p1t=1.0, m1=1.0, zd=1.0, pfl=0.0005)\nP.update({'W': 50, 'ammo_pmax': 0.02, 'fy': 0.7, 'fu': 0.25, 'Hu': 500})\nC = {'W': 50, 'ammo_pmax': 0.02, 'fy': 0.7, 'reserve': 200, 'scar': 0.0025, 'fu': 0.25, 'Hu': 500}\nSQ2 = math.sqrt(2.0)\nSQ2PI = math.sqrt(2.0 * math.pi)\ndef cdf(x): return 0.5 * (1.0 + math.erf(x / SQ2))\ndef pdf(x): return math.exp(-0.5 * x * x) / SQ2PI\ndef inv(p):\n p = min(max(p, 1e-9), 1 - 1e-9)\n t = math.sqrt(-2.0 * math.log(min(p, 1 - p)))\n x = t - (2.515517 + 0.802853 * t + 0.010328 * t * t) / (1 + 1.432788 * t + 0.189269 * t * t + 0.001308 * t ** 3)\n x = -x if p < 0.5 else x\n for _ in range(2):\n d = pdf(x)\n if d < 1e-12:\n break\n x -= (cdf(x) - p) / d\n return x\ndef lpm(k):\n return pdf(k) + k * cdf(k)\ndef fl(q):\n return math.floor(q * 100 + 1e-9) / 100\n\nNODES = [(-2.2, .06), (-1.3, .17), (-0.6, .27), (0.6, .27), (1.3, .17), (2.2, .06)]\nPROBE = 0.01\n\n\nclass Strategy(BaseStrategy):\n def __init__(self):\n self.n = 0\n self.ahead = set() # main orders placed strictly ahead of any competitor order at their tick\n self.res = {} # order id -> uncovered quantity at placement\n self.bc = None # belief: sub-tick grid centres (ticks)\n self.bw = None # belief: weights\n self.st = 0.5 # last one-step sd in ticks\n self.pcb = self.pca = None\n self.pb = set()\n self.pa = set()\n self.tb = 0\n self.s = None # competitor spread parameter\n\n def _oid(self, tag):\n self.n += 1\n return tag + str(self.n)\n\n def on_step(self, state):\n H = max(1, state.steps_remaining)\n cb = state.competitor_best_bid_ticks or 0\n ca = state.competitor_best_ask_ticks or 100\n if self.s is None:\n self.s = max(1, (ca - cb + 1) // 2)\n lo, hi = cb, ca\n for o in state.own_orders:\n if o.side is Side.BUY:\n lo = max(lo, o.price_ticks)\n else:\n hi = min(hi, o.price_ticks)\n if P['pinf'] > 0:\n sb = {o.price_ticks for o in state.own_orders if o.side is Side.BUY and o.remaining_quantity <= PROBE + 1e-9}\n sa = {o.price_ticks for o in state.own_orders if o.side is Side.SELL and o.remaining_quantity <= PROBE + 1e-9}\n ea = self.pa - sa\n eb = self.pb - sb\n if sa and ea:\n lo = max(lo, min(max(ea), hi))\n if eb and (sb or min(eb) <= P['p1t']):\n hi = min(hi, max(min(eb), lo))\n if P['jumpinfo'] > 0 and self.pcb is not None:\n if ca > self.pca + 1:\n lo = max(lo, ca - 1)\n if cb < self.pcb - 1:\n hi = min(hi, cb + 1)\n self.pcb, self.pca = cb, ca\n if lo > hi:\n lo, hi = hi, lo\n # sub-tick belief over p_(t-1): predict with one-step noise, truncate to [lo, hi]\n wid = max(hi - lo, 1e-6)\n nb = max(2, min(48, int(round(wid * P['bins']))))\n cs = [lo + (i + 0.5) * wid / nb for i in range(nb)]\n if self.bc is None or P['filt'] <= 0:\n ws = [1.0 / nb] * nb\n else:\n sg = max(self.st, 0.6 * wid / nb)\n k = -0.5 / (sg * sg)\n ws = []\n uni = P['uni'] * sg\n for c in cs:\n t = uni\n for ci, wi in zip(self.bc, self.bw):\n d = c - ci\n d2 = d * d * k\n if d2 > -12.5:\n t += wi * math.exp(d2)\n ws.append(t)\n tot = sum(ws)\n ws = [w / tot for w in ws]\n self.bc, self.bw = cs, ws\n mean_t = sum(c * w for c, w in zip(cs, ws))\n var_t = max(sum(c * c * w for c, w in zip(cs, ws)) - mean_t * mean_t, 0.0)\n phat = min(max(mean_t / 100.0, 0.002), 0.998)\n var = P['sig'] ** 2 + P['lam_j'] * P['sig_j'] ** 2\n sd = math.sqrt(H * var)\n self.tb = self.tb + 1 if hi <= 1 else 0\n if self.tb > 0 and P['zd'] > 0:\n # below the last tick there is no probe: the score keeps walking away from it\n phat = min(phat, max(P['pfl'], cdf(inv(0.01) - P['zd'] * 0.02 * math.sqrt(self.tb) / sd)))\n z = inv(phat)\n Z = z * sd\n sp = pdf(z) / sd * P['sig']\n self.st = sp * 100.0\n s = math.sqrt(sp * sp + var_t / 1e4 + 1e-8)\n sd1 = math.sqrt(max(1.0, H - 1.0) * var)\n jp = [(cdf((Z + n * P['sig_j']) / sd1), w) for n, w in NODES]\n lj = P['lam_j']\n\n def AB(x):\n k = (x - phat) / s\n a = s * lpm(k)\n b = a - (x - phat)\n ja = sum(w * max(0.0, x - p) for p, w in jp)\n jb = sum(w * max(0.0, p - x) for p, w in jp)\n return (1 - lj) * a + lj * ja, (1 - lj) * b + lj * jb\n\n if H == 1:\n def AB(x):\n return (1 - phat) * x, phat * (1 - x)\n rho = P['lam_r'] * 0.5\n mu = math.log(P['m']) - 0.5 * P['sig_n'] ** 2\n kA = P['kL'] if phat < P['pL'] else P['kA']\n\n def qsize(loss, gain, px):\n loss *= kA\n if gain <= 0:\n return 0.0\n tail = loss / (rho * gain)\n if tail >= 1.0:\n return 0.0\n return min(P['qmax'], math.exp(mu + P['sig_n'] * inv(1 - max(tail, 1e-6))) / px)\n\n ptk = phat * 100\n def jump_charge(t, is_bid):\n x = t / 100.0\n # Adverse jump payoff, with the existing transition nodes unchanged.\n adverse = sum(w * max(0.0, x - p if is_bid else p - x) for p, w in jp)\n # Endpoint asymmetry: buying high and selling low expose more jump loss.\n exposure = phat if is_bid else 1.0 - phat\n spread_weight = 2.0 / (1.0 + self.s)\n horizon_weight = min(1.0, 500.0 / H) ** 0.5\n return 1.0 * lj * adverse * exposure * spread_weight * horizon_weight\n\n def qbid(t):\n if not 1 <= t <= 99:\n return 0.0\n a, b = AB(t / 100.0)\n if phat * 100 - t < P['dmin']:\n return 0.0\n if P['nb1'] > 0 and self.s == 1 and t == 1 and ptk < 1.0:\n return 0.0\n if phat < P['pL']:\n ytg = P['ydem'] * min(H, P['W']) / max(ca - 1, ptk + 1) * 100.0\n a *= (P['kLb'] if state.yes_inventory < ytg else P['kLf']) / P['kL']\n return qsize(a + jump_charge(t, True), b, max(phat, 0.05))\n\n def qask(t):\n if not 1 <= t <= 99:\n return 0.0\n a, b = AB(t / 100.0)\n if t - phat * 100 < P['dmin']:\n return 0.0\n if phat < C.get('scar_p', .1):\n a = max(0.0, a - C.get('scar', 0.0) * (1 - min(1.0, state.yes_inventory / C.get('stock', 3000.0))))\n return min(C.get('askcap', P['qmax']), qsize(b + jump_charge(t, False), a, t / 100.0))\n\n bids = [o for o in state.own_orders if o.side is Side.BUY and o.remaining_quantity > PROBE + 1e-9]\n asks = [o for o in state.own_orders if o.side is Side.SELL and o.remaining_quantity > PROBE + 1e-9]\n sticky = P['sticky'] > 0\n def top(orders, comp, step):\n if sticky and sum(o.remaining_quantity for o in orders if o.price_ticks == comp and o.order_id in self.ahead) >= P['min_ahead']:\n return comp\n return comp + step\n bid_t = top(bids, cb, 1)\n ask_t = top(asks, ca, -1)\n qb = min(C.get('bidcap', 3000.0), qbid(bid_t))\n qa = qask(ask_t)\n\n Y = state.yes_inventory\n if ask_t == 1 and H > P['H1'] and P['pre'] > 0:\n qa = qa if Y >= P['a1'] > 0 else 0.0 # let the competitor's 2-tick ask erode instead of fronting it for nothing\n # ammunition: hold YES so that low asks are covered\n ptick = phat * 100\n amt = 0\n qam = 0.0\n if P['ammo'] > 0 and ask_t >= 2 and phat < max(P['ammo_pmax'], P['pmaxf']):\n wealth = state.cash + Y * phat\n x = max(ask_t, ptick + 1) / 100.0\n capw = P['fy'] * wealth / max(phat, 0.005)\n target = min(P['ydem'] * min(H, P['W']) / x, capw)\n full = min(P['ydf'] * min(H, P['Wf']) / x, capw) if phat < P['pmaxf'] else 0.0\n t = max(1, min(99, int(math.ceil(ptick + 1e-9))))\n g = t - ptick\n if Y < full and g <= P['g_hi'] and bid_t <= t < ask_t:\n amt, qam = t, min(P['qf'], full - Y) # a tick p has just crossed: nearly free\n elif Y < target and phat < P['ammo_pmax'] and bid_t <= t < ask_t and t / 100.0 - phat < P['ammo_maxpay']:\n amt, qam = t, min(P['ammo_q'], target - Y)\n elif Y < full and t >= 2 and 1.0 - g <= P['g_lo']:\n amt, qam = t - 1, min(P['qf'], full - Y) # rest just under p: filled only on the crossing, or by retail\n\n if P['pre'] > 0 and ca == 2 and phat < 0.01 and H > P['H1'] and Y < P['pre'] and not qam and 1.0 - ptick < P['pre_gap']:\n amt, qam = 1, min(P['ammo_q'], P['pre'] - Y)\n q1 = 0.0\n if P['ydf1'] > 0 and P['m1'] < ptick < P['g1'] + 1.0 and amt != 1:\n q1 = max(0.0, P['ydf1'] * min(H, P['Wf']) / (self.s / 100.0) - Y)\n # order management\n actions = []\n free = state.free_cash - 2.0\n keep_ask = 0.0\n new_b = []\n new_a = []\n def cancel(o):\n nonlocal free\n actions.append(CancelOrder(o.order_id))\n x = o.price_ticks / 100.0\n if o.side is Side.BUY:\n free += x * o.remaining_quantity\n else:\n free += (1 - x) * min(self.res.get(o.order_id, 0.0), o.remaining_quantity)\n self.ahead.discard(o.order_id)\n self.res.pop(o.order_id, None)\n for o in state.own_orders:\n if o.remaining_quantity <= PROBE + 1e-9:\n cancel(o)\n kept_ids = set()\n def manage(orders, t, q, comp):\n kept = 0.0\n for o in orders:\n if not (C.get('recover', 0) and o.side is Side.SELL and t < comp and self.res.get(o.order_id, 0.0) > 1 and state.yes_inventory > C.get('recover_yes', 1) and state.free_cash < C.get('recover_cash', 200)) and o.price_ticks == t and q > 0 and kept + o.remaining_quantity <= max(q * P['keep_hi'], 1.0):\n kept += o.remaining_quantity\n kept_ids.add(o.order_id)\n else:\n cancel(o)\n return kept\n kb = manage(bids, bid_t, qb, cb)\n ka = manage(asks, ask_t, qa, ca)\n add_b = (qb - kb) if (kb < qb * P['keep_lo'] and bid_t != cb) else 0.0\n add_a = (qa - ka) if (ka < qa * P['keep_lo'] and ask_t != ca) else 0.0\n\n budget = P['fb'] * max(free, 0.0)\n cost = add_b * bid_t / 100.0 + qam * amt / 100.0\n if cost > budget:\n f = budget / cost\n add_b *= f\n qam *= f\n cost = budget\n out = []\n q1 = min(q1, P['f1'] * max(budget - cost, 0.0) * 100.0)\n if q1 >= 0.01:\n out.append(PlaceOrder(side=Side.BUY, price_ticks=1, quantity=fl(q1)))\n free -= fl(q1) / 100.0 + 1e-6\n if amt and qam >= 0.01:\n q = fl(qam)\n out.append(PlaceOrder(side=Side.BUY, price_ticks=amt, quantity=q))\n free -= q * amt / 100.0 + 1e-6\n if bid_t >= 1 and add_b >= 0.01:\n q = fl(add_b)\n oid = self._oid('b')\n out.append(PlaceOrder(side=Side.BUY, price_ticks=bid_t, quantity=q, client_order_id=oid))\n free -= q * bid_t / 100.0 + 1e-6\n if bid_t > cb:\n self.ahead.add(oid)\n if ask_t <= 99 and add_a >= 0.01:\n x = ask_t / 100.0\n covk = sum(o.remaining_quantity - min(self.res.get(o.order_id, 0.0), o.remaining_quantity)\n for o in asks if o.order_id in kept_ids)\n cov = max(0.0, Y - covk - 0.1)\n unc = max(0.0, add_a - cov)\n if unc > 0:\n ub = max(0.0, free - (1 - P['fu']) * max(state.cash, 0.0)) if x >= P['xu'] else 0.0\n if H < P['Hu'] and phat < 0.05:\n ub = max(0.0, free * (1 - H / P['Hu']))\n ub = min(ub, max(0.0, free - C.get('reserve', 0.0) * min(1.0, H / C.get('release', 50.0))))\n unc = min(unc, ub / (1 - x))\n q = fl(min(add_a, cov + unc))\n if q >= 0.01:\n oid = self._oid('a')\n out.append(PlaceOrder(side=Side.SELL, price_ticks=ask_t, quantity=q, client_order_id=oid))\n r = max(0.0, q - cov)\n self.res[oid] = r\n free -= r * (1 - x) + 1e-6\n if ask_t < ca:\n self.ahead.add(oid)\n\n # probes: tiny orders that the arbitrageur removes when p crosses them\n f = int(math.floor(ptick + 1e-9))\n r = max(int(P['rmin']), int(math.ceil(2.0 * s * 100)))\n self.pb = set()\n self.pa = set()\n if free > 1.0:\n for t in range(f, f + r + 1):\n if cb < t < ca and 1 <= t <= 99:\n out.append(PlaceOrder(side=Side.BUY, price_ticks=t, quantity=PROBE))\n self.pb.add(t)\n for t in range(f + 1 - r, f + 2):\n if cb < t < ca and 1 <= t <= 99:\n out.append(PlaceOrder(side=Side.SELL, price_ticks=t, quantity=PROBE))\n self.pa.add(t)\n return actions + out\n", "code_sha256": "3b7236c98d52efed9617e4cf6d8183012c98b8b61b18eb59b5afae937c99c724", "incumbent_code_sha256": "baef955ec10728839f7c3357016ec9eb0550e7ec689bbd5171b1cdd5c9e37273", "tuning_score": { "candidate_mean": 94.52561713573105, "incumbent_mean": 93.89195216102316, "mean_diff": 0.6336649747078998, "se": 0.5589493447545113, "n": 2000, "seeds": { "first": 100000, "last": 101999, "inclusive": true }, "components": { "retail_edge": { "candidate_mean": 116.4707125889687, "incumbent_mean": 118.1709640703087, "mean_diff": -1.7002514813399998, "se": 0.09587733467471832 }, "arb_edge": { "candidate_mean": -21.94509545321965, "incumbent_mean": -24.2790119092727, "mean_diff": 2.33391645605305, "se": 0.5550745225020117 } }, "failures": [] }, "paired": { "candidate_mean": 99.8488767034095, "incumbent_mean": 99.53686835155, "mean_diff": 0.3120083518594995, "se": 0.43247611305069644, "n": 2000, "seeds": { "first": 200000, "last": 201999, "inclusive": true }, "components": { "retail_edge": { "candidate_mean": 127.4042373613403, "incumbent_mean": 129.10654277618445, "mean_diff": -1.7023054148441503, "se": 0.10360081294115928 }, "arb_edge": { "candidate_mean": -27.5553606579405, "incumbent_mean": -29.56967442464705, "mean_diff": 2.01431376670655, "se": 0.426323503430907 } }, "failures": [] }, "evidence": { "status": "complete_not_promoted", "simulator_commit": "b244530ee2a6247b5f930830eeca9dd950da0e7e", "simulator_url": "https://github.com/danrobinson/prediction-market-challenge/tree/b244530ee2a6247b5f930830eeca9dd950da0e7e", "incumbent_url": "https://api.imdnotdead.fun/artifacts/ecbc9087b4f238fc1b5b0d52f55af073f676c038bfb6756172b1e51480ac1741", "execution": "Serial SimulationEngine.run calls, incumbent then candidate per seed; a single process; unmodified ChallengeConfig() and ParameterVariance(), 2000 steps. No workers or sandbox. No simulator source changes.", "runtime": "PyPy 7.3.20 / Python 3.11.13 for full T and H; CPython 3.12.3 for the diagnostic 20-seed prefix. Strategy uses standard library plus challenge API; no added submission dependencies.", "commands": [ "curl -L -sS https://api.imdnotdead.fun/artifacts/ecbc9087b4f238fc1b5b0d52f55af073f676c038bfb6756172b1e51480ac1741 -o test/scratch/incumbent.json", "curl -L -sS https://codeload.github.com/danrobinson/prediction-market-challenge/tar.gz/b244530ee2a6247b5f930830eeca9dd950da0e7e -o test/scratch/sim.tar.gz", "python3 test/scratch/evaluate.py 100000 20 test/scratch/t.jsonl", "test/scratch/pypy3.11-v7.3.20-linux64/bin/pypy3 test/scratch/evaluate.py 100000 20 test/scratch/pypy-check.jsonl", "test/scratch/pypy3.11-v7.3.20-linux64/bin/pypy3 test/scratch/evaluate.py 100000 2000 test/scratch/t-full.jsonl > test/scratch/t-progress.log", "test/scratch/pypy3.11-v7.3.20-linux64/bin/pypy3 test/scratch/evaluate.py 200000 2000 test/scratch/h-full.jsonl > test/scratch/h-progress.log", "python3 test/scratch/freeze.py", "python3 test/scratch/final_check.py" ], "evaluator_code": "import sys,json,time,statistics,math,hashlib,datetime\nfrom pathlib import Path\nsys.path.insert(0,str(next(Path('test/scratch').glob('prediction-market-challenge-*'))))\nfrom orderbook_pm_challenge.config import ChallengeConfig,ParameterVariance\nfrom orderbook_pm_challenge.runner import sample_config\nfrom orderbook_pm_challenge.engine import SimulationEngine\nfrom orderbook_pm_challenge.loader import load_strategy_factory\nstart,count=int(sys.argv[1]),int(sys.argv[2])\nbase,var=ChallengeConfig(),ParameterVariance()\na=load_strategy_factory('test/scratch/incumbent.py');b=load_strategy_factory('test/scratch/candidate.py')\nout=Path(sys.argv[3]); t=time.monotonic()\nwith out.open('a') as f:\n for seed in range(start,start+count):\n cfg=sample_config(base,var,seed=seed)\n ra=SimulationEngine(cfg,a,seed=seed).run();rb=SimulationEngine(cfg,b,seed=seed).run()\n row={'seed':seed,'incumbent':{k:getattr(ra,k) for k in ('total_edge','retail_edge','arb_edge','failed','error')},'candidate':{k:getattr(rb,k) for k in ('total_edge','retail_edge','arb_edge','failed','error')}}\n f.write(json.dumps(row)+'\\n'); f.flush()\n if (seed-start+1)%20==0: print(seed+1,round(time.monotonic()-t,2),flush=True)\nprint('finished',time.monotonic()-t,flush=True)\n", "aggregation_code": "import json,statistics,math,sys\nfrom pathlib import Path\ndef summary(path):\n rows=[json.loads(x) for x in Path(path).read_text().splitlines()]\n n=len(rows)\n def stats(k):\n aa=[r['incumbent'][k] for r in rows];bb=[r['candidate'][k] for r in rows];dd=[b-a for a,b in zip(aa,bb)]\n return {'candidate_mean':statistics.mean(bb),'incumbent_mean':statistics.mean(aa),'mean_diff':statistics.mean(dd),'se':statistics.stdev(dd)/math.sqrt(n) if n>1 else None}\n result=stats('total_edge');result.update(n=n,seeds={'first':rows[0]['seed'],'last':rows[-1]['seed'],'inclusive':True},components={k:stats(k) for k in ('retail_edge','arb_edge')},failures=[{'seed':r['seed'],'strategy':s,'error':r[s]['error']} for r in rows for s in ('candidate','incumbent') if r[s]['failed']])\n return result\nif __name__=='__main__':print(json.dumps(summary(sys.argv[1]),indent=2))\n", "reproduction": "Extract this JSON.code to test/scratch/candidate.py and incumbent JSON.code to test/scratch/incumbent.py with exact UTF-8 bytes. Extract the pinned simulator tarball below test/scratch; save evaluator_code as evaluate.py and aggregation_code as summarize.py there. Run listed evaluation commands with a compatible Python interpreter. No network is used during evaluation.", "freeze": { "utc": "2026-11-06T16:39:43.491689+00:00", "code_sha256": "3b7236c98d52efed9617e4cf6d8183012c98b8b61b18eb59b5afae937c99c724", "full_T_rows_sha256": "1c8288abbc4b74a7b365e25fe4b49f5d9cdac9a6891f7aa0e615af6a4488e9b0", "T_score": { "candidate_mean": 94.52561713573105, "incumbent_mean": 93.89195216102316, "mean_diff": 0.6336649747078998, "se": 0.5589493447545113, "n": 2000, "seeds": { "first": 100000, "last": 101999, "inclusive": true }, "components": { "retail_edge": { "candidate_mean": 116.4707125889687, "incumbent_mean": 118.1709640703087, "mean_diff": -1.7002514813399998, "se": 0.09587733467471832 }, "arb_edge": { "candidate_mean": -21.94509545321965, "incumbent_mean": -24.2790119092727, "mean_diff": 2.33391645605305, "se": 0.5550745225020117 } }, "failures": [] }, "H_rows_existed": false, "selection": "Only one variant was tried. Retain its exact code for the specified H evaluation; no H information was available. No promotion by this item." }, "scoreboard": { "incumbent_code_sha256": "baef955ec10728839f7c3357016ec9eb0550e7ec689bbd5171b1cdd5c9e37273", "candidate_code_sha256": "3b7236c98d52efed9617e4cf6d8183012c98b8b61b18eb59b5afae937c99c724", "idea": "side-specific adverse jump charge at ordinary sizing call sites", "variants": [ "side-jump-budget-v1" ], "selection_data": "T only", "holdout_accessed": true, "current_best_code_sha256": "baef955ec10728839f7c3357016ec9eb0550e7ec689bbd5171b1cdd5c9e37273", "candidate_T_paired_mean_diff": 0.6336649747078998, "candidate_T_paired_se": 0.5589493447545113, "T_n": 2000, "candidate_frozen_at_utc": "2026-11-06T16:39:43.491689+00:00", "promotion": false, "holdout_accessed_after_freeze": true, "H_used_for_selection": false, "candidate_H_paired_mean_diff": 0.3120083518594995, "candidate_H_paired_se": 0.43247611305069644, "H_n": 2000, "H_completed_at_utc": "2026-11-06T17:08:05.408945+00:00" }, "statistic": "Per-seed d=candidate.total_edge-incumbent.total_edge. Report mean(d) and sample_stdev(d)/sqrt(n). Same method for signed component differences. All completed rows, including failures if present, retained; any failures make the result ineligible. No PnL statistics or independent-mean errors.", "failures": { "T": [], "H": [], "tooling": [ "python command unavailable; used python3.", "tar could not decompress bzip2 because bzip2 executable was unavailable; extracted PyPy with Python standard-library tarfile instead." ] }, "raw_run_sha256": { "t.jsonl": "a1c3b1a4a063251f5f2154333aab4556e1489d6eaf7919e9d4c83a9f706a3f19", "pypy-check.jsonl": "a1c3b1a4a063251f5f2154333aab4556e1489d6eaf7919e9d4c83a9f706a3f19", "t-full.jsonl": "1c8288abbc4b74a7b365e25fe4b49f5d9cdac9a6891f7aa0e615af6a4488e9b0", "h-full.jsonl": "f6ba61408fdb5c289c8f314ff0a84a363402fe07040392a9f47d78339032e718" }, "validation": { "incumbent_hash_matches_required": true, "source_quotes_refetched_and_verified": true, "reserved_seeds_accessed": false, "H_used_for_selection": false, "candidate_variants": 1, "simulator_python_files_verified_against_archive": 19, "simulator_archive_sha256": "38e3563f24aae3f227b0620b1859d023a033dadba6426fb9e80b4985794f5acd", "reverse_diff_restores_exact_incumbent": true, "imports_only_math_and_challenge_api": true, "diagnostic_cross_interpreter_exact_matches": 20, "full_T_and_H_exact_ranges_verified": true, "zero_simulation_failures_verified": true, "paired_statistics_independently_recomputed": true, "component_totals_verified": true, "frozen_code_and_T_data_hashes_verified": true, "standalone_code_compilation_passed": true, "final_check_passed_at_utc": "2026-11-06T17:08:28.484207+00:00" }, "timing": { "T_elapsed_seconds": 1692.3343276446685, "H_elapsed_seconds": 1689.6462203678675 }, "validation_code": "import json,hashlib,ast,math,datetime\nfrom pathlib import Path\np=Path('test/scratch');out=Path('artifacts/goal/batches/r1-deepen-pilot/items/side-jump-budget')\nr=json.loads((out/'result.json').read_text());sources=json.loads((out/'sources.json').read_text())\nassert set(r)=={'approach','tried','next','code','code_sha256','incumbent_code_sha256','tuning_score','paired','evidence'}\nassert (out/'result.json').stat().st_size<=64000\nassert len(sources['sources'])<=20 and all(len(s['quote'])<=500 for s in sources['sources'])\nassert hashlib.sha256(r['code'].encode('utf-8')).hexdigest()==r['code_sha256']==json.loads((p/'freeze.json').read_text())['code_sha256']\nassert hashlib.sha256((p/'t-full.jsonl').read_bytes()).hexdigest()==json.loads((p/'freeze.json').read_text())['full_T_rows_sha256']\nassert r['incumbent_code_sha256']=='baef955ec10728839f7c3357016ec9eb0550e7ec689bbd5171b1cdd5c9e37273'\nast.parse(r['code']);compile(r['code'],'submission.py','exec')\nassert r['evidence']['status']=='complete_not_promoted' and r['evidence']['eligible_for_ranking']\nassert r['tuning_score']['n']==r['paired']['n']==2000\nassert json.loads((p/'freeze.json').read_text())['T_score']==r['tuning_score']\nfor fname,first,label in [('t-full.jsonl',100000,'tuning_score'),('h-full.jsonl',200000,'paired')]:\n rows=[json.loads(x) for x in (p/fname).read_text().splitlines()]\n assert [x['seed'] for x in rows]==list(range(first,first+2000))\n assert not any(x[s]['failed'] for x in rows for s in ('candidate','incumbent'))\n assert all(abs(x[s]['total_edge']-x[s]['retail_edge']-x[s]['arb_edge'])<1e-7 for x in rows for s in ('candidate','incumbent'))\n ds=[x['candidate']['total_edge']-x['incumbent']['total_edge'] for x in rows]\n m=math.fsum(ds)/len(ds);se=math.sqrt(math.fsum((x-m)**2 for x in ds)/(len(ds)-1)/len(ds))\n assert math.isclose(r[label]['mean_diff'],m,rel_tol=1e-12,abs_tol=1e-12)\n assert math.isclose(r[label]['se'],se,rel_tol=1e-12,abs_tol=1e-12)\n assert hashlib.sha256((p/fname).read_bytes()).hexdigest()==r['evidence']['raw_run_sha256'][fname]\nfor fn in ('incumbent.py','candidate.py'):\n assert len((p/fn).read_text())>0\nprefix=[json.loads(x) for x in (p/'t.jsonl').read_text().splitlines()]\nassert prefix==[json.loads(x) for x in (p/'pypy-check.jsonl').read_text().splitlines()]\nassert prefix==[json.loads(x) for x in (p/'t-full.jsonl').read_text().splitlines()][:20]\nprint('PASS: full T/H ranges, no failures, paired statistics, components, exact hashes, standalone code compilation, JSON schema and byte limits')\nprint('result bytes:',(out/'result.json').stat().st_size)\n", "freeze_code": "import json,hashlib,datetime\nfrom pathlib import Path\nfrom summarize import summary\np=Path('test/scratch');rows=[json.loads(x) for x in (p/'t-full.jsonl').read_text().splitlines()]\nassert [r['seed'] for r in rows]==list(range(100000,102000))\ns=summary(p/'t-full.jsonl');assert not s['failures'];assert not (p/'h-full.jsonl').exists();assert not (p/'freeze.json').exists()\ncode=(p/'candidate.py').read_bytes();ch=hashlib.sha256(code).hexdigest()\nassert ch=='3b7236c98d52efed9617e4cf6d8183012c98b8b61b18eb59b5afae937c99c724'\nf={'utc':datetime.datetime.now(datetime.timezone.utc).isoformat(),'code_sha256':ch,'full_T_rows_sha256':hashlib.sha256((p/'t-full.jsonl').read_bytes()).hexdigest(),'T_score':s,'H_rows_existed':False,'selection':'Only one variant was tried. Retain its exact code for the specified H evaluation; no H information was available. No promotion by this item.'}\n(p/'freeze.json').write_text(json.dumps(f,indent=2)+'\\n')\nscore=json.loads((p/'scoreboard.json').read_text());score.update({'current_best_code_sha256':score['incumbent_code_sha256'],'candidate_T_paired_mean_diff':s['mean_diff'],'candidate_T_paired_se':s['se'],'T_n':s['n'],'candidate_frozen_at_utc':f['utc'],'promotion':False})\n(p/'scoreboard.json').write_text(json.dumps(score,indent=2)+'\\n');print(json.dumps(f,indent=2))\n", "eligible_for_ranking": true } }