# err-a: where NP2's pHat errs Headline: pHat is about 23 bps RMS wrong after every kind of print, and arb prints alone reveal p exactly. But pHat accuracy is not NP2's bottleneck. Giving NP2 the true price lost 5.63 ± 0.14 edge in closed loop, because the fee map is tuned to the lagged signal. The observable fixes found so far total about +0.39, far short of the +5.8 needed for top 10. ## method Patched `amm_sim_rs` to log each NP2 print and step end, reading slots 0, 1, 2, 5, 6, 9, 10 from EVM storage. Seeds 10000..10199, NP2 vs Vanilla, `_build_configs` draws, 2,114,019 prints. Checks: logged edges sum to run_batch edges (max diff 5e-12; mean 523.4836). NP2 hash 51f1c1a9… and 526.115572 on seeds 0..999 reproduced. Patch leaves 12-seed edges unchanged. Error = ln(pHat/p) in bps. CH not run: its source is not in this item. ## results Conventions: `dir` = +1 if the trade pushes spot up (AMM sells X), -1 otherwise. So `err*dir` < 0 means pHat is behind the trade and > 0 means it overshoots. `pImplied` is NP2's own value: spot·γ on AMM buys, spot/γ on AMM sells. `g` = steps with no NP2 print between the previous print's step and this one, so g = 0 means the previous step printed. ### (a) Error right after each print (bps) | kind | n | share | RMS post | mean signed post (se) | median abs | p95 abs | RMS pre-call | RMS ln(pImplied/p) | mean ln(pImplied/p)·dir | mean post err·dir | |---|---|---|---|---|---|---|---|---|---|---| | arb first-in-step | 746,735 | 35.3% | 23.09 | +0.11 (0.08) | 20.12 | 37.77 | 31.64 | 0.00 | 0.00 | -20.65 | | retail first-in-step | 622,420 | 29.4% | 22.20 | +0.04 (0.05) | 14.69 | 41.99 | 16.37 | 66.84 | +51.74 | +13.55 | | retail later | 744,864 | 35.2% | 23.41 | +0.06 (0.07) | 19.19 | 40.25 | 22.53 | 75.66 | +51.72 | +3.56 | Se is across seeds. Arb never prints later in a step in this engine, because the arbitrageur acts before retail. Patterns: - **Arb prints:** pImplied equals p exactly (max |ln(pImplied/p)| was 4e-12 bps on seed 10000, RMS 0.00 on all seeds). NP2 still keeps only alpha = 0.27 of it, so post RMS = 0.73 × 31.64 = 23.1, which is the whole arb row. - **Retail prints:** pImplied is about 52 bps too far in the trade's direction, because a retail print leaves spot inside the band, not at its edge. Alpha 0.27 on a retail first print therefore makes pHat worse: RMS goes from 16.37 to 22.20. - **Mean signed error:** about 0 in every row. The error is lag and overshoot that cancel across sides, so a constant offset fix cannot help. ### (b) End of silent steps, by silence length (bps) | silence (steps) | n | RMS pHat | mean signed | median abs | RMS ln(spot/p) | mean avg fee | mean max-side fee | |---|---|---|---|---|---|---|---| | 1 | 346,536 | 16.20 | 0.00 | 11.37 | 48.48 | 32.48 | 54.87 | | 2 | 148,238 | 16.79 | -0.10 | 10.95 | 49.27 | 33.58 | 57.05 | | 3-5 | 115,943 | 19.26 | -0.28 | 11.80 | 55.93 | 36.19 | 62.58 | | 6+ | 20,128 | 44.31 | -5.38 | 17.28 | 114.58 | 56.58 | 105.36 | | (step with a print) | 1,369,155 | 23.17 | +0.07 | 18.45 | 86.25 | 36.63 | 65.37 | 31.5% of steps are silent; the longest run was 112 steps. Error barely grows up to 5 silent steps. The 6+ row is rare (1% of steps) and has much higher error, fees and spot gap, so these runs are episodes where NP2's own fees are wide. Print steps are worse than silent steps: a print step's p has just moved, or a print has just disturbed pHat. ### (c) Arbitrageur share | measure | arb share | |---|---| | all prints (746,735 / 2,114,019) | 35.32% | | first-in-step prints (746,735 / 1,369,155) | 54.54% | | Y volume (extra) | 23.81% | ### (d) Telling arb from retail on first-in-step prints n = 1,368,990 (t > 0), arb base rate 54.54%. "Label everything arb" gives P 0.545, R 1.000, F1 0.706. Each threshold was scanned in both directions on these same prints, so these are in-sample bests. | feature | AUC (arb on the listed side) | best-F1 rule for arb | P | R | F1 | FPR | best-Youden rule | P | R | FPR | |---|---|---|---|---|---|---|---|---|---|---| | tradeRatio | 0.573 (lower) | ≤ 0.002205 | 0.571 | 0.977 | 0.721 | 0.882 | ≤ 0.001311 | 0.583 | 0.890 | 0.765 | | g | 0.662 (lower) | g = 0 | 0.651 | 0.891 | 0.752 | 0.574 | g = 0 | 0.651 | 0.891 | 0.574 | | \|pImplied-pHat\|/pHat/sigmaHat | 0.770 (lower) | ≤ 1.223 | 0.661 | 0.921 | 0.770 | 0.567 | ≤ 0.960 | 0.714 | 0.777 | 0.373 | | extra: pre-spot excess, ln(s0·γ/pHat) on the trade's side, bps | 0.772 (higher) | ≥ -28.73 | 0.664 | 0.889 | 0.760 | 0.539 | ≥ -22.68 | 0.730 | 0.732 | 0.325 | | extra: logistic on the 3 briefed features + excess + fee, fit on seeds 10000..10099, scored on 10100..10199 | 0.856 (higher) | score ≥ -0.076 | 0.766 | 0.881 | 0.819 | 0.320 | score ≥ 0.257 | 0.799 | 0.824 | 0.246 | Distributions, 5/25/50/75/95 percentiles: | measure | arb | retail | |---|---|---| | tradeRatio (bps) | 0.44 / 2.29 / 4.99 / 8.90 / 17.40 | 0.75 / 2.81 / 5.99 / 12.44 / 37.52 | | z = \|pImplied-pHat\|/pHat/sigmaHat | 0.29 / 0.54 / 0.72 / 0.93 / 1.34 | 0.50 / 0.82 / 1.12 / 1.65 / 3.60 | Size barely separates the two, and arbs are slightly smaller. P(arb | g = 0) = 65.1%. P(arb | g ≥ 1) = 23.4%, and 25.3% of first prints have g ≥ 1. No single threshold is safe. Open-loop rules that set alpha 0.6-0.9 on "arb-like" prints reached precision of about 0.65-0.88, and step-end RMS never beat NP2's 21.97 by more than 0.03 in any rule replayed (best 21.94, below). A misclassified retail print costs about 52 bps × alpha. ### (e) Per-step arb loss and retail take, by |error| bucket Totals per simulation: arb loss 30.72, retail take 554.20, net 523.48. Arb loss is -edge of arb prints and retail take is edge of retail prints, both in Y on the NP2 pool, with 10,000 steps per sim over 200 sims. Bucket = |ln(pHat at end of t-1 / p_t)|, the error the quotes carry into step t. It includes step t's price shock. | \|error\| bps | steps | share | arb loss/step | retail take/step | net/step | arb per sim | retail per sim | net per sim | mean fee bps | |---|---|---|---|---|---|---|---|---|---| | <5 | 303,186 | 15.2% | 0.00100 | 0.05648 | 0.05548 | 1.52 | 85.62 | 84.10 | 34.99 | | 5-15 | 597,378 | 29.9% | 0.00089 | 0.05568 | 0.05479 | 2.65 | 166.30 | 163.66 | 34.75 | | 15-30 | 709,066 | 35.5% | 0.00177 | 0.05540 | 0.05363 | 6.27 | 196.41 | 190.14 | 34.62 | | >30 | 390,370 | 19.5% | 0.01039 | 0.05424 | 0.04385 | 20.28 | 105.87 | 85.59 | 40.49 | Bucket = |ln(pHat at end of t-1 / p_{t-1})|, which excludes step t's shock, so it is purer estimator error. | \|error\| bps | steps | share | arb loss/step | retail take/step | net/step | arb per sim | retail per sim | net per sim | mean fee bps | |---|---|---|---|---|---|---|---|---|---| | <5 | 293,770 | 14.7% | 0.00121 | 0.05581 | 0.05460 | 1.78 | 81.97 | 80.19 | 35.48 | | 5-15 | 627,827 | 31.4% | 0.00150 | 0.05534 | 0.05385 | 4.70 | 173.73 | 169.03 | 34.91 | | 15-30 | 802,023 | 40.1% | 0.00258 | 0.05634 | 0.05375 | 10.36 | 225.91 | 215.55 | 33.92 | | >30 | 276,380 | 13.8% | 0.01004 | 0.05253 | 0.04249 | 13.88 | 72.59 | 58.71 | 44.05 | >30 bps steps carry 45-66% of arb loss, but all arb loss is only 30.72 of 554.20. Retail take per step is almost flat across buckets. These are correlations, not causal gains: large-error steps are also large-move steps. ### Extra: what better pHat would buy Both tables below go beyond the brief. They are exploratory pilots on the tuning range, not confirmed on a fresh block, and no candidate is claimed from them. Open-loop replay on seeds 10000..10099: logged prints, fees and sigmaHat are held fixed. The replay reproduced NP2's EVM pHat within 6e-12 bps on seed 10000. | rule | RMS arb | RMS retail first | RMS retail later | RMS step end | class. P | class. R | |---|---|---|---|---|---|---| | NP2 as is | 23.07 | 22.17 | 23.41 | 21.97 | n/a | n/a | | oracle kind, alpha 1 on arb | 0.00 | 21.89 | 15.15 | 17.31 | 1 | 1 | | + no update on later prints in that step | 0.00 | 21.78 | 13.70 | 16.92 | 1 | 1 | | + clamp to pre-trade band on retail first prints | 0.00 | 18.92 | 12.63 | 14.93 | 1 | 1 | | band clamp only, oracle kind | 22.10 | 20.58 | 22.69 | 20.66 | n/a | n/a | | observable: excess ≥ -10 bps, g = 0, alpha 0.6, no later updates | 21.10 | 23.42 | 22.37 | 21.94 | 0.879 | 0.363 | | observable: excess ≥ -25, g = 0, z ≤ 1, alpha 0.9, no later updates | 16.96 | 25.86 | 20.71 | 22.20 | 0.817 | 0.692 | Closed-loop paired vs NP2. "Oracle" variants get true p and/or trade kind written into a slot by the patched engine, so they are diagnostic bounds and cannot be submitted. | variant | seeds | mean diff | se | worst | |---|---|---|---|---| | ORACLE pHat = true p at every call | 10000..10299 | -5.627 | 0.138 | -13.15 | | ORACLE true p, STALE_DIR_COEF × 0.50 / 0.70 / 0.78 / 0.85 / 1.15 / 1.50 | 10000..10299 | -22.66 / -4.92 / -1.94 / -1.34 / -16.14 / -83.94 | 0.31 / 0.15 / 0.12 / 0.10 / 0.25 / 1.33 | -40.04 / -14.19 / -9.32 / -7.64 / -29.40 / -171.82 | | ORACLE kind: pHat = pImplied on arb prints | 10000..10299 | -3.943 | 0.119 | -10.06 | | same + no update on later prints in that step | 10000..10299 | -4.387 | 0.121 | -10.87 | | ORACLE true p in tox magnitude only | 10000..10299 | -8.347 | 0.150 | -16.14 | | ORACLE true p in stale/trade side sign only | 10000..10299 | -0.997 | 0.044 | -3.86 | | ORACLE both tox magnitude and sign | 10000..10299 | -6.294 | 0.149 | -14.72 | | ORACLE kind: band clamp on retail first prints | 10000..10299 | +0.576 | 0.042 | -1.33 | | band clamp, all first prints | 10000..10299 | -3.569 | 0.248 | -30.86 | | band clamp, all first prints, half weight | 10000..10299 | -1.191 | 0.098 | -19.54 | | band clamp, z > 1.00 / 1.25 / 1.50 | 10000..10299 | -0.893 / -0.269 / +0.062 | 0.090 / 0.080 / 0.074 | -16.85 / -16.84 / -16.12 | | **clamp_g: band clamp on first prints with g ≥ 1** | 10000..10299 | +0.293 | 0.031 | -1.48 | | same | 10300..10599 | +0.279 | 0.025 | -1.27 | | alpha on first prints with g ≥ 1 = 0.06 / 0.10 / 0.15 | 10000..10299 | -1.372 / -0.536 / +0.062 | 0.078 / 0.063 / 0.047 | -11.32 / -8.51 / -5.39 | | **alpha_g20: same with 0.20** | 10000..10299 | +0.270 | 0.034 | -3.69 | | same | 10300..10599 | +0.270 | 0.027 | -1.01 | | **alpha_g20 + clamp_g** | 10000..10299 | +0.390 | 0.038 | -1.52 | | same | 10300..10599 | +0.392 | 0.032 | -1.00 | Base NP2 means: 523.986 on 10000..10299, 527.503 on 10300..10599. The 0.20 value and the clamp set were picked on the first block, and the second block replicates them. One combined variant (clamp on g ≥ 1 or z > 1.5) did not compile (stack too deep) and was dropped. The oracle sign and tox rows show that NP2's edge depends on pHat lagging: with an accurate tox the fee map loses more than it gains. STALE_DIR_COEF is sharply tuned in both directions, so the map has to be refit as a whole. ## blocked null ## next 1. **Band clamp after a gap.** Pattern: 76.6% of first prints with g ≥ 1 are retail (d), and a retail first print proves no arb fired, so p lay in the pre-trade band [s0·(1-bid), s0/(1-ask)]. Clamp pHat to that band on those prints, storing the pre-trade spot in a free slot. Pilot: +0.293 ± 0.031, +0.279 ± 0.025; the bound with perfect retail detection is +0.576 ± 0.042. Test: replace the g ≥ 1 gate with a better retail detector, such as the (d) logistic, and check whether it nears the bound. Do not clamp arb prints: that lost 3.57. 2. **Lower alpha on likely-retail first prints.** Pattern: retail pImplied overshoots by about 52 bps in the trade's direction and pHat ends +13.55 bps past p (a). Set alpha to 0.20 instead of 0.27 when g ≥ 1. Combined with fix 1: +0.390 ± 0.038, +0.392 ± 0.032. Test: confirm this once on a fresh block before use. It is a small gain, not a route to +5.8. 3. **Exact arb prices need a refit, or a separate slot.** Pattern: arb prints give exact p, yet pHat stays 20.65 bps behind (a), and feeding exact values into pHat loses 3.9-5.6 under today's constants. Test first: with true p as oracle, jointly refit STALE_DIR_COEF, STALE_ATTRACT_FRAC, TOX_COEF and TOX_BLEND_DECAY. If that refit cannot exceed about +5.8, a perfect pHat cannot reach top 10 through this fee map, and further pHat work should stop. If it can, keep lagged pHat for tox and add an observable arb-price slot for the new terms. ## code Applied to simulator commit 6b6921f. `AMM_TRACE_DIR` turns on logging and `AMM_ORACLE` enables the diagnostic bound runs. `pilot.py` is shown as it ran, including unused leftovers. ### patch (`git diff`) ````diff diff --git a/amm_sim_rs/src/amm/cfmm.rs b/amm_sim_rs/src/amm/cfmm.rs index 02e6645..51a8148 100644 --- a/amm_sim_rs/src/amm/cfmm.rs +++ b/amm_sim_rs/src/amm/cfmm.rs @@ -54,8 +54,20 @@ pub struct CFMM { accumulated_fees_x: f64, /// Accumulated fees in Y (collected separately, not in reserves) accumulated_fees_y: f64, + /// TRACE: row buffer (None = tracing off), step fair price, trade kind (0 arb, 1 retail) + pub trace: Option>, + pub trace_fair: f64, + pub trace_kind: f64, + trace_spot: f64, + /// ORACLE: expose fair price (slot 13) and trade kind (slot 14: 1 arb, 2 retail) to the strategy + pub oracle: bool, } +/// TRACE row: t, fair, spot_before, spot_after, is_buy, amount_x, amount_y, kind, phat_pre, +/// phat_post, sigma_pre, sigma_post, slot0_pre, slot1_pre, slot0_post, slot1_post, last_ts_pre, +/// step_count_pre, edge, rx_post, ry_post, tox_post, fee_bid_engine_pre, fee_ask_engine_pre +pub const TRACE_COLS: usize = 24; + impl CFMM { /// Create a new CFMM with the given strategy and reserves. pub fn new(strategy: EVMStrategy, reserve_x: f64, reserve_y: f64) -> Self { @@ -69,9 +81,19 @@ impl CFMM { initialized: false, accumulated_fees_x: 0.0, accumulated_fees_y: 0.0, + trace: None, + trace_fair: 0.0, + trace_kind: 0.0, + trace_spot: reserve_y / reserve_x, + oracle: false, } } + /// TRACE: read a strategy slot. + pub fn slot_f64(&self, i: u64) -> f64 { + self.strategy.slot_f64(i) + } + /// Initialize the AMM and get starting fees from strategy. pub fn initialize(&mut self) -> Result<(), crate::evm::strategy::EVMError> { let initial_x = Wad::from_f64(self.reserve_x); @@ -291,9 +313,33 @@ impl CFMM { /// Update fees from strategy after a trade. fn update_fees(&mut self, trade_info: &TradeInfo) { + let pre = if self.trace.is_some() { + Some([0u64, 1, 2, 5, 6, 10].map(|i| self.strategy.slot_f64(i))) + } else { + None + }; + if self.oracle { + self.strategy.set_slot(13, revm::primitives::U256::from((self.trace_fair * 1e18) as u128)); + self.strategy.set_slot(14, revm::primitives::U256::from(if self.trace_kind == 0.0 { 1u64 } else { 2u64 })); + } + let (fb, fa) = (self.current_fees.bid_fee.to_f64(), self.current_fees.ask_fee.to_f64()); if let Ok((bid_fee, ask_fee)) = self.strategy.after_swap(trade_info) { self.current_fees = FeeQuote::new(bid_fee.clamp_fee(), ask_fee.clamp_fee()); } + if let Some(pre) = pre { + let spot_after = self.reserve_y / self.reserve_x; + let row = [ + trade_info.timestamp as f64, self.trace_fair, self.trace_spot, spot_after, + if trade_info.is_buy { 1.0 } else { 0.0 }, + trade_info.amount_x.to_f64(), trade_info.amount_y.to_f64(), self.trace_kind, + pre[3], self.strategy.slot_f64(5), pre[4], self.strategy.slot_f64(6), + pre[0], pre[1], self.strategy.slot_f64(0), self.strategy.slot_f64(1), + pre[2] * 1e18, pre[5] * 1e18, 0.0, self.reserve_x, self.reserve_y, + self.strategy.slot_f64(9), fb, fa, + ]; + self.trace.as_mut().unwrap().push(row); + self.trace_spot = spot_after; + } // On error, keep current fees } diff --git a/amm_sim_rs/src/evm/strategy.rs b/amm_sim_rs/src/evm/strategy.rs index 1d06af1..7db0f6d 100644 --- a/amm_sim_rs/src/evm/strategy.rs +++ b/amm_sim_rs/src/evm/strategy.rs @@ -188,6 +188,22 @@ impl EVMStrategy { .ok_or_else(|| EVMError::InvalidReturnData("Failed to decode fee pair".into())) } + /// TRACE: read `slots[index]` (storage index = array index) as f64, WAD / 1e18. + pub fn slot_f64(&self, index: u64) -> f64 { + let v = self.db.accounts.get(&STRATEGY_ADDRESS) + .and_then(|a| a.storage.get(&U256::from(index)).copied()) + .unwrap_or(U256::ZERO); + let limbs = v.as_limbs(); + (limbs[0] as f64 + limbs[1] as f64 * 18446744073709551616.0) / 1e18 + } + + /// ORACLE: write a value into a strategy slot (diagnostic bound runs only). + pub fn set_slot(&mut self, index: u64, value: U256) { + if let Some(a) = self.db.accounts.get_mut(&STRATEGY_ADDRESS) { + a.storage.insert(U256::from(index), value); + } + } + /// Reset the strategy for a new simulation. pub fn reset(&mut self) -> Result<(), EVMError> { self.deploy() diff --git a/amm_sim_rs/src/simulation/engine.rs b/amm_sim_rs/src/simulation/engine.rs index 6834c43..92a539b 100644 --- a/amm_sim_rs/src/simulation/engine.rs +++ b/amm_sim_rs/src/simulation/engine.rs @@ -133,13 +133,26 @@ impl SimulationEngine { cumulative_ask_fees.insert(name.clone(), 0.0); } + // TRACE: AMM_TRACE_DIR= logs every submission print and each step end as binary f64 + let trace_dir = std::env::var("AMM_TRACE_DIR").ok(); + if trace_dir.is_some() { + amms[0].trace = Some(Vec::with_capacity(self.config.n_steps as usize * 2)); + } + let mut step_rows: Vec<[f64; 6]> = Vec::new(); + amms[0].oracle = std::env::var("AMM_ORACLE").is_ok(); + for t in 0..self.config.n_steps { // 1. Generate new fair price let fair_price = price_process.step(); + amms[0].trace_fair = fair_price; + amms[0].trace_kind = 0.0; // 2. Arbitrageur extracts profit from each AMM for amm in amms.iter_mut() { if let Some(arb_result) = arbitrageur.execute_arb(amm, fair_price, t as u64) { + if let Some(tr) = amm.trace.as_mut() { + if let Some(last) = tr.last_mut() { last[18] = -arb_result.profit; } + } *arb_volume_y.get_mut(&arb_result.amm_name).unwrap() += arb_result.amount_y; let entry = edges.entry(arb_result.amm_name).or_insert(0.0); // AMM edge is the negative of arbitrageur profit at true price @@ -149,8 +162,19 @@ impl SimulationEngine { // 3. Retail orders arrive and get routed let orders = retail_trader.generate_orders(); + amms[0].trace_kind = 1.0; + let n_before = amms[0].trace.as_ref().map_or(0, |v| v.len()); let routed_trades = router.route_orders(&orders, &mut amms, fair_price, t as u64); + let mut trace_i = n_before; for trade in routed_trades { + if trade.amm_name == submission_name { + let e = if trade.amm_buys_x { + trade.amount_x * fair_price - trade.amount_y + } else { + trade.amount_y - trade.amount_x * fair_price + }; + if let Some(tr) = amms[0].trace.as_mut() { tr[trace_i][18] = e; trace_i += 1; } + } *retail_volume_y.get_mut(&trade.amm_name).unwrap() += trade.amount_y; let trade_edge = if trade.amm_buys_x { trade.amount_x * fair_price - trade.amount_y @@ -161,6 +185,11 @@ impl SimulationEngine { *entry += trade_edge; } + if trace_dir.is_some() { + step_rows.push([t as f64, fair_price, amms[0].spot_price(), amms[0].slot_f64(5), + amms[0].fees().bid_fee.to_f64(), amms[0].fees().ask_fee.to_f64()]); + } + // 4. Capture step result and accumulate fees let step = capture_step( t, @@ -180,6 +209,16 @@ impl SimulationEngine { steps.push(step); } + if let Some(dir) = &trace_dir { + let dump = |rows: Vec, name: String| { + let bytes: Vec = rows.iter().flat_map(|v| v.to_le_bytes()).collect(); + std::fs::write(format!("{}/{}", dir, name), bytes).expect("trace write"); + }; + let tr = amms[0].trace.take().unwrap(); + dump(tr.iter().flatten().copied().collect(), format!("trades_{}.bin", seed)); + dump(step_rows.iter().flatten().copied().collect(), format!("steps_{}.bin", seed)); + } + // Calculate final PnL (reserves + accumulated fees) let final_fair_price = price_process.current_price(); let mut pnl = HashMap::new(); ```` ### common.py ````python import numpy as np, amm_sim_rs from amm_competition.competition.config import BASELINE_VARIANCE as V, build_config from amm_competition.evm.compiler import SolidityCompiler from amm_competition.evm.baseline import get_vanilla_bytecode_and_abi def cfg(s): r=np.random.default_rng(seed=s) size=r.uniform(V.retail_mean_size_min,V.retail_mean_size_max) rate=r.uniform(V.retail_arrival_rate_min,V.retail_arrival_rate_max) sig=r.uniform(V.gbm_sigma_min,V.gbm_sigma_max) return build_config(seed=s,gbm_sigma=sig,retail_arrival_rate=rate,retail_mean_size=size), (size,rate,sig) def compile_np2(path='scratch/NP2.sol'): bc,_=SolidityCompiler().compile_and_get_bytecode(open(path).read()); return bytes(bc) def vanilla(): return bytes(get_vanilla_bytecode_and_abi()[0]) def run(code,seeds,workers=6): r=amm_sim_rs.run_batch(list(code),list(vanilla()),[cfg(s)[0] for s in seeds],workers) return np.array([x.edges['submission'] for x in r.results]),np.array([x.edges['normalizer'] for x in r.results]) ```` ### trace.py ````python import sys,os,time; sys.path.insert(0,'scratch/tools') from common import * d='scratch/trace'; os.makedirs(d,exist_ok=True); os.environ['AMM_TRACE_DIR']=d a,b=10000,10200; t=time.time(); c=compile_np2(); E=[] for s in range(a,b,60): e,v=run(c,range(s,min(b,s+60))); E.append(e); print(s,round(time.time()-t),flush=True) e=np.concatenate(E); np.save(d+'/edges.npy',e) np.save(d+'/params.npy',np.array([cfg(s)[1] for s in range(a,b)])) print('mean edge',e.mean(), 'n',len(e)) ```` ### analyze.py ````python import numpy as np, sys C='t fair s0 s1 buy ax ay kind ph0 ph1 sg0 sg1 b0 a0 b1 a1 last cnt edge rx ry tox fb fa'.split() ix={c:i for i,c in enumerate(C)} seeds=range(10000,10200) def load(): Ts=[];Ss=[] for s in seeds: T=np.fromfile(f'scratch/trace/trades_{s}.bin').reshape(-1,24); S=np.fromfile(f'scratch/trace/steps_{s}.bin').reshape(-1,6) Ts.append(T);Ss.append(S) return Ts,Ss Ts,Ss=load() BP=1e4 rows={'arb':[], 'rf':[], 'rl':[]} F=[] # per-trade derived def derive(T,sid): t=T[:,0]; first=np.r_[True,t[1:]!=t[:-1]] kind=T[:,7] cat=np.where(kind==0,0,np.where(first,1,2)) buy=T[:,4]==1 fee=np.where(buy,T[:,22],T[:,23]); g=1-fee pI=np.where(buy,T[:,3]*g,T[:,3]/g) gap=np.r_[0,t[1:]-T[:-1,16+0]*0-t[:-1]-1] gap=np.zeros(len(t)); idx=np.where(first)[0] gap[idx[1:]]=t[idx[1:]]-t[idx[:-1]]-1; gap[0]=t[0] tr=T[:,6]/T[:,20] z=np.abs(pI-T[:,8])/T[:,8]/T[:,10] up=np.where(buy,-1.0,1.0) # direction the trade pushes spot # excess of pre-trade spot beyond the band around pHat, on the trade's side (bps) exc=np.where(buy,np.log(T[:,2]*g/T[:,8]),np.log(T[:,8]/(T[:,2]/g)))*BP return dict(cat=cat,first=first,buy=buy,fee=fee,pI=pI,gap=gap,tr=tr,z=z,up=up,exc=exc,sid=np.full(len(t),sid), err1=np.log(T[:,9]/T[:,1])*BP, err0=np.log(T[:,8]/T[:,1])*BP, errI=np.log(pI/T[:,1])*BP, t=t,kind=kind,edge=T[:,18],spoterr=np.log(T[:,3]/T[:,1])*BP) Ds=[derive(T,i) for i,T in enumerate(Ts)] D={k:np.concatenate([d[k] for d in Ds]) for k in Ds[0]} names=['arb first-in-step','retail first-in-step','retail later'] def rms(x): return float(np.sqrt(np.mean(x**2))) print('## a'); print('kind|n|share|RMS err post|mean signed post|se mean|median abs|p95 abs|RMS err pre|RMS ln(pImpl/p)|mean ln(pImpl/p)*dir|mean err post*dir') for c in range(3): m=D['cat']==c; e=D['err1'][m] perseed=[d['err1'][d['cat']==c].mean() for d in Ds] print(names[c],m.sum(),round(m.mean(),4),round(rms(e),3),round(e.mean(),3),round(np.std(perseed,ddof=1)/200**.5,3),round(np.median(abs(e)),3),round(np.percentile(abs(e),95),3),round(rms(D['err0'][m]),3),round(rms(D['errI'][m]),3),round((D['errI'][m]*D['up'][m]).mean(),3),round((e*D['up'][m]).mean(),3),sep='|') print('## b') out={} allrows=[] for i,(d,S) in enumerate(zip(Ds,Ss)): pr=np.zeros(10000,bool); pr[d['t'].astype(int)]=True run=np.zeros(10000,int); c=0 for j in range(10000): c=0 if pr[j] else c+1; run[j]=c e=np.log(S[:,3]/S[:,1])*BP; se=np.log(S[:,2]/S[:,1])*BP allrows.append(np.c_[run,e,se]) A=np.concatenate(allrows) bins=[(1,1,'1'),(2,2,'2'),(3,5,'3-5'),(6,10**9,'6+')] print('silence|n|RMS|mean signed|median abs|RMS ln(spot/p)') for a,b,n in bins: m=(A[:,0]>=a)&(A[:,0]<=b); e=A[m,1] print(n,m.sum(),round(rms(e),3),round(e.mean(),3),round(np.median(abs(e)),3),round(rms(A[m,2]),3),sep='|') m=A[:,0]==0; print('step with print',m.sum(),round(rms(A[m,1]),3),round(A[m,1].mean(),3),round(np.median(abs(A[m,1])),3),round(rms(A[m,2]),3),sep='|') print('silent frac',(A[:,0]>0).mean(), 'max run',A[:,0].max()) print('## c') n=len(D['cat']); f=D['first'] print('prints',n,'arb',(D['cat']==0).sum(),'share all',(D['cat']==0).mean(),'first prints',f.sum(),'share first',(D['cat'][f]==0).mean()) ay=np.concatenate([T[:,6] for T in Ts]); print('arb share of Y volume',ay[D['cat']==0].sum()/ay.sum()) print('## d') m=D['first']&(D['t']>0) y=D['cat'][m]==0 def scan(x,y,name): res=[] for sgn,label in ((1,'>='),(-1,'<=')): v=sgn*x; o=np.argsort(-v,kind='stable'); vs=v[o]; ys=y[o] tp=np.cumsum(ys); k=np.arange(1,len(y)+1); ends=np.r_[vs[1:]!=vs[:-1],True] tp=tp[ends];k=k[ends];th=vs[ends] P=tp/k;R=tp/y.sum();F1=2*P*R/(P+R+1e-30) fp=k-tp; FPR=fp/(~y).sum(); J=R-FPR bF=F1.argmax(); bJ=J.argmax() res.append((label,sgn*th[bF],P[bF],R[bF],F1[bF],FPR[bF],sgn*th[bJ],P[bJ],R[bJ],F1[bJ],FPR[bJ],J[bJ])) # AUC from_rank=x.argsort(); r=np.empty(len(x)); import scipy.stats as st r=st.rankdata(x); auc=(r[y].sum()-y.sum()*(y.sum()+1)/2)/(y.sum()*(~y).sum()) print(name,'AUC %.4f'%auc) for q in res: print(' dir %s | bestF1 th=%.6g P=%.4f R=%.4f F1=%.4f FPR=%.4f | bestJ th=%.6g P=%.4f R=%.4f F1=%.4f FPR=%.4f J=%.4f'%q) print('base rate arb',y.mean(),'all-arb F1',2*y.mean()/(1+y.mean()),'n',len(y)) scan(D['tr'][m],y,'tradeRatio'); scan(D['gap'][m],y,'gap g'); scan(D['z'][m],y,'z'); scan(D['exc'][m],y,'excess bps') print('quantiles tradeRatio bps arb',np.percentile(D['tr'][m][y]*1e4,[5,25,50,75,95]).round(2),'retail',np.percentile(D['tr'][m][~y]*1e4,[5,25,50,75,95]).round(2)) print('quantiles z arb',np.percentile(D['z'][m][y],[5,25,50,75,95]).round(2),'retail',np.percentile(D['z'][m][~y],[5,25,50,75,95]).round(2)) print('quantiles exc arb',np.percentile(D['exc'][m][y],[5,25,50,75,95]).round(2),'retail',np.percentile(D['exc'][m][~y],[5,25,50,75,95]).round(2)) # gate miss rate for c in range(3): mm=D['cat']==c; print(names[c],'fraction with pHat unchanged (gate)',) # logistic combo X=np.c_[np.log(D['tr'][m]),np.log1p(D['gap'][m]),np.log(D['z'][m]+1e-3),D['exc'][m]/10,np.log(D['fee'][m]*1e4+1)] Xm,Xs=X.mean(0),X.std(0); Z=(X-Xm)/Xs; Z=np.c_[np.ones(len(Z)),Z] tr=D['sid'][m]<100; b=np.zeros(Z.shape[1]) for it in range(25): p=1/(1+np.exp(-Z[tr]@b)); W=p*(1-p) H=(Z[tr].T*W)@Z[tr]+1e-6*np.eye(Z.shape[1]); g=Z[tr].T@(y[tr]-p); b+=np.linalg.solve(H,g) print('logit coefs',b.round(3)) te=~tr; sc=Z[te]@b; print('held-out (seeds 10100+) logistic:'); scan(sc,y[te],'logit') print('## e') print('bucket|def|steps|share|arb loss/step|retail take/step|net/step|arb per sim|retail per sim|net per sim|mean fee bps') for label in ('stale','prev'): ers=[];al=[];rt=[];fe=[] for d,S in zip(Ds,Ss): ph=np.r_[100.0,S[:-1,3]]; pprev=np.r_[100.0,S[:-1,1]] p=pprev if label=='prev' else S[:,1] e=np.abs(np.log(ph/p))*BP tt=d['t'].astype(int) a=np.zeros(10000);r=np.zeros(10000) np.add.at(a,tt[d['kind']==0],-d['edge'][d['kind']==0]); np.add.at(r,tt[d['kind']==1],d['edge'][d['kind']==1]) ers.append(e);al.append(a);rt.append(r) fe.append(np.r_[30e-4,(S[:-1,4]+S[:-1,5])/2]*BP) e,a,r,f=map(np.concatenate,(ers,al,rt,fe)) for lo,hi,n in ((0,5,'<5'),(5,15,'5-15'),(15,30,'15-30'),(30,1e9,'>30')): m=(e>=lo)&(e=a)&(A[:,0]<=b); print(n,m.sum(),*A[m,1:].mean(0).round(2),sep='|') D=np.load('scratch/trace/derived.npz') m=D['first']&(D['t']>0); g=D['gap'][m]>=1; y=D['cat'][m]==0 print('P(arb|g>=1)',y[g].mean(),'P(arb|g=0)',y[~g].mean(),'share g>=1',g.mean()) print('RMS err post by first kind & g') for name,mm in (('arb g0',y&~g),('arb g1+',y&g),('ret g0',~y&~g),('ret g1+',~y&g)): e=D['err1'][m][mm]; print(name,mm.sum(),round(float(np.sqrt((e**2).mean())),2),round(float((e*D['up'][m][mm]).mean()),2)) ```` ### replay.py ````python """Open-loop replay of pHat rules over logged prints (fees, spots, sigmaHat as logged).""" import numpy as np, sys BP=1e4 A0,AR,AS=0.27,0.06,0.12; GMULT=7.2; MING=0.03 seeds=range(10000,10100) data=[np.fromfile(f'scratch/trace/trades_{s}.bin').reshape(-1,24) for s in seeds] steps=[np.fromfile(f'scratch/trace/steps_{s}.bin').reshape(-1,6) for s in seeds] def replay(T,rule): n=len(T); ph=100.0; out=np.zeros(n); flag=np.zeros(n,bool) tprev=-1; cnt=0; d1=0; lastT=0; arbseen=False; lastspot=100.0 Tl=T.tolist() for i,r in enumerate(Tl): t,p,s0,s1,buy,ax,ay,kind,ph0,ph1,sg0,sg1,b0,a0,b1,a1,last,_,edge,rx,ry,tox,fb,fa=r first=t!=tprev or i==0 if first: g=t-tprev-1 if i else 0; cnt=0; arbseen=False fee=fb if buy else fa; gam=1-fee pI=s1*gam if buy else s1/gam pre=ph rt=abs(pI-ph)/ph alpha=A0 if first else AR second=not first and cnt==1 if second: up=pI>=ph if d1 and ((d1==1)==up): alpha=AS if first: d1=1 if pI>=ph else 2 gate=max(sg0*GMULT,MING) rl=rule if rl.get('oracle_arb') and first and kind==0: alpha=1.0 if rl.get('rule') and first: band=(s0*gam>=ph*(1-rl['T'])) if buy else (s0/gam<=ph*(1+rl['T'])) c=(g==0) if rl.get('g0') else True cz=(rt<=rl['Z']*sg0) if 'Z' in rl else True isarb=band and c and cz flag[i]=isarb if isarb: alpha=rl['A'] elif 'R' in rl: alpha=rl['R'] arbseen=isarb if rl.get('oracle_arb') and first and kind==0: arbseen=True freeze=rl.get('freeze') and not first and arbseen if rt<=gate and not freeze: ph=ph*(1-alpha)+pI*alpha if not first and fee>30e-4 and gam>0 and not (rl.get('nonp')) and not freeze: ns=s1*gam/(1-30e-4) if buy else s1*(1-30e-4)/gam ph=.9*ph+.1*ns if rl.get('clamp') and first and kind==1 or (rl.get('clampall') and first and not flag[i]): lo,hi=s0*(1-b0),s0/(1-a0) ph=min(max(ph,lo),hi) out[i]=ph; tprev=t; cnt+=1 return out,flag def report(name,rule): E={0:[],1:[],2:[],3:[]}; se=[]; fl=[];y=[] for T,S in zip(data,steps): o,f=replay(T,rule) t=T[:,0]; first=np.r_[True,t[1:]!=t[:-1]]; k=T[:,7] cat=np.where(k==0,0,np.where(first,1,2)) e=np.log(o/T[:,1])*BP for c in range(3): E[c].append(e[cat==c]) # end-of-step error: last pHat of each step, carried to silent steps ph=np.full(10000,np.nan); ph[t.astype(int)]=o # numpy keeps last assign idx=np.where(~np.isnan(ph),np.arange(10000),0); idx=np.maximum.accumulate(idx); ph=ph[idx] ph[np.isnan(ph)]=100.0 se.append(np.log(ph/S[:,1])*BP) fl.append(f[first]);y.append(k[first]==0) r=lambda x:np.sqrt(np.mean(np.concatenate(x)**2)) sa=np.concatenate(se); fa=np.concatenate(fl); ya=np.concatenate(y) P=(fa&ya).sum()/max(fa.sum(),1); R=(fa&ya).sum()/ya.sum() print(f'{name}|{r(E[0]):.2f}|{r(E[1]):.2f}|{r(E[2]):.2f}|{np.sqrt(np.mean(sa**2)):.2f}|{P:.3f}|{R:.3f}',flush=True) T=data[0]; o,_=replay(T,{}); print('replay vs EVM max bps',np.abs(np.log(o/T[:,9])).max()*BP) print('rule|RMS arb|RMS retail first|RMS retail later|RMS step-end|cls P|cls R') report('NP2 as is',{}) report('oracle kind, alpha 1',{'oracle_arb':1}) report('oracle kind, alpha 1, freeze rest of step',{'oracle_arb':1,'freeze':1}) report('oracle + retail-first clamp',{'oracle_arb':1,'freeze':1,'clamp':1}) report('clamp only',{'clamp':1}) for T_ in (10e-4,20e-4,25e-4): for g0 in (0,1): for A in (.6,.9): report(f'rule T={T_*1e4:.0f} g0={g0} A={A}',{'rule':1,'T':T_,'g0':g0,'A':A}) report(f'rule T={T_*1e4:.0f} g0={g0} A={A} freeze',{'rule':1,'T':T_,'g0':g0,'A':A,'freeze':1}) for Z in (1.0,1.25): report(f'rule T=25 g0=1 Z={Z} A=.9 freeze',{'rule':1,'T':25e-4,'g0':1,'Z':Z,'A':.9,'freeze':1}) report(f'rule T=25 g0=1 Z={Z} A=.9 freeze R=0.1',{'rule':1,'T':25e-4,'g0':1,'Z':Z,'A':.9,'freeze':1,'R':.1}) ```` ### pilot.py ````python """Closed-loop pilot: NP2 variants paired vs NP2. usage: pilot.py start end name[,name...]""" import sys,os,time; sys.path.insert(0,'scratch/tools') from common import * base=open('scratch/NP2.sol').read() GATE="if (ret <= adaptiveGate) { pHat = wmul(pHat, WAD - alpha) + wmul(pImplied, alpha); }" NP_S=base.index('// Interior retail routing'); NP_E=base.index('uint256 tradeRatio =') NPB=base[NP_S:NP_E] assert GATE in base def rep(s,a,b): assert a in s,a return s.replace(a,b) PRESPOT="slots[13]" def clamp_block(): # pre-trade spot kept in slot 15; retail-first: no arb happened, so p lies in the pre-trade band return """ if (firstInStep) { uint256 s0 = slots[15]; if (s0 > 0 && slots[16] == 0 && pHatBandOn) {} } """ V={} V['base']=(base,False) # full oracle: pHat is the true price at every call; sigma still measured against the previous p s=rep(base,GATE,""); s=rep(s,NPB,"pHat = slots[13];\n\n ") V['oracle_p']=(s,True) for k,cs,ct in (('s.7',685,105),('s.5',485,105)): pass for name,fac in (('oracle_p_stale50',50),('oracle_p_stale150',150)): V[name]=(rep(s,"uint256 constant STALE_DIR_COEF = 6850 * BPS;",f"uint256 constant STALE_DIR_COEF = {6850*fac//100} * BPS;"),True) # kind oracle: exact pHat on arbitrage prints, NP2 rule otherwise s=rep(base,GATE,"if (slots[14] == 1) { pHat = pImplied; } else if (ret <= adaptiveGate) { pHat = wmul(pHat, WAD - alpha) + wmul(pImplied, alpha); }") V['oracle_kind']=(s,True) for name,fac in (('oracle_kind_stale50',50),('oracle_kind_stale150',150)): V[name]=(rep(s,"uint256 constant STALE_DIR_COEF = 6850 * BPS;",f"uint256 constant STALE_DIR_COEF = {6850*fac//100} * BPS;"),True) # kind oracle + freeze later prints in a step that opened with an arb s2=rep(s,"if (slots[14] == 1) { pHat = pImplied; } else if (ret <= adaptiveGate)","if (slots[14] == 1) { pHat = pImplied; slots[17] = trade.timestamp; } else if (slots[17] != trade.timestamp + 1 - 1 && ret <= adaptiveGate)") s2=rep(s2,"if (!firstInStep && feeUsed > 30 * BPS && gamma > 0) {","if (!firstInStep && feeUsed > 30 * BPS && gamma > 0 && slots[17] != trade.timestamp) {") V['oracle_kind_freeze']=(s2,True) # Observable: retail/any first print band clamp. slots[15] = spot after previous call (= pre-trade spot here). CL=""" if (firstInStep && slots[15] > 0 && CLAMPCOND) { uint256 s0 = slots[15]; uint256 loB = wmul(s0, WAD - prevBidFee); uint256 hiB = wdiv(s0, WAD - prevAskFee); if (pHat < loB) pHat = pHat + wmul(loB - pHat, CLAMPW); else if (pHat > hiB) pHat = pHat - wmul(pHat - hiB, CLAMPW); } slots[15] = spot; """ def clampv(cond,w,src=base): return rep(src,"uint256 tradeRatio = trade.reserveY",CL.replace('CLAMPCOND',cond).replace('CLAMPW',w)+"uint256 tradeRatio = trade.reserveY") V['clamp_kind']=(clampv("slots[14] == 2","WAD"),True) # oracle kind: bound for the clamp # observable proxy: pImplied far outside band means retail; retail is the print that lands far from pHat for z in (100,125,150): V[f'clamp_z{z}']=(clampv(f"wdiv(absDiff(pImplied, prePhatFirst), wmul(sigmaHat, prePhatFirst)) > {z} * WAD / 100","WAD").replace("uint256 tradeRatio = trade.reserveY","uint256 tradeRatio = trade.reserveY",1),False) V['clamp_all']=(clampv("true","WAD"),False) V['clamp_all_w50']=(clampv("true","(WAD/2)"),False) V['clamp_nogate_g']=(clampv("trade.isBuy == trade.isBuy && trade.timestamp - lastTs > 1","WAD"),False) ALPHA="uint256 alpha = firstInStep ? PHAT_ALPHA : PHAT_ALPHA_RETAIL;" def alphag(a,src=base): return rep(src,ALPHA,f"uint256 alpha = firstInStep ? (trade.timestamp - lastTs > 1 ? {a} * WAD / 100 : PHAT_ALPHA) : PHAT_ALPHA_RETAIL;") SIGN1="if (spot >= pHat) {\n bidFee = bidFee + staleShift;" SIGN2="bool tradeAligned = (trade.isBuy && spot >= pHat) || (!trade.isBuy && spot < pHat);" TOX="uint256 tox = pHat > 0 ? wdiv(absDiff(spot, pHat), pHat) : 0;" s_=rep(rep(base,SIGN1,SIGN1.replace("spot >= pHat","spot >= slots[13]")),SIGN2,SIGN2.replace("spot >= pHat","spot >= slots[13]").replace("spot < pHat","spot < slots[13]")) V['sign_oracle']=(s_,True) V['tox_oracle']=(rep(base,TOX,"uint256 tox = wdiv(absDiff(spot, slots[13]), slots[13]);"),True) V['both_oracle']=(rep(s_,TOX,"uint256 tox = wdiv(absDiff(spot, slots[13]), slots[13]);"),True) s=V['oracle_p'][0] for fac in (70,78,85,115): V[f'oracle_p_s{fac}']=(rep(s,"uint256 constant STALE_DIR_COEF = 6850 * BPS;",f"uint256 constant STALE_DIR_COEF = {6850*fac//100} * BPS;"),True) GC="trade.timestamp - lastTs > 1" V['clamp_gz']=(clampv(GC+" || wdiv(absDiff(pImplied, prePhatFirst), wmul(sigmaHat, prePhatFirst)) > 150 * WAD / 100","WAD"),False) V['clamp_g']=(clampv(GC,"WAD"),False) for a in (6,10,15,20): V[f'alpha_g{a}']=(alphag(a),False) V['alpha_g20_clamp_g']=(alphag(20,V['clamp_g'][0]),False) V['alpha_g10_clamp_g']=(alphag(10,V['clamp_g'][0]),False) names=sys.argv[3].split(',') a,b=int(sys.argv[1]),int(sys.argv[2]) os.makedirs('scratch/pilot',exist_ok=True) def go(name): src,orc=V[name] if orc: os.environ['AMM_ORACLE']='1' else: os.environ.pop('AMM_ORACLE',None) if 'prePhatFirst' in src: src=src.replace("uint256 feeUsed = trade.isBuy","uint256 prePhatFirst = pHat;\n uint256 feeUsed = trade.isBuy",1) open(f'scratch/pilot/{name}.sol','w').write(src) c=compile_np2(f'scratch/pilot/{name}.sol'); E=[] for s in range(a,b,60): E.append(run(c,range(s,min(b,s+60)))[0]) return np.concatenate(E) fn=f'scratch/pilot/base_{a}_{b}.npy' if os.path.exists(fn): e0=np.load(fn) else: e0=go('base'); np.save(fn,e0) print('base',e0.mean(),flush=True) for n in names: t=time.time(); e=go(n); d=e-e0 print(f'{n}|n={len(d)}|mean {e.mean():.3f}|diff {d.mean():+.3f}|se {d.std(ddof=1)/len(d)**.5:.3f}|worst {d.min():.2f}|{time.time()-t:.0f}s',flush=True) ````