Skip to main content

top3-min-trading-engine-testing-strategy

· 2 min read

Top 3 strategies (realism-first, no sugar)​

  • Testnet-in-the-loop E2E (nightly)

    • Use Bybit testnet + real Postgres/Redis, no HTTP mocks.
    • Drive minimal-trading-system through scenarios: TP-only, SL-only, partial fills, cancel/recreate, trailing SL, and close-by-TP vs close-by-SL.
    • Assert via public contracts:
      • GET /positions/:id/pnl realized/unrealized correctness
      • tournament PnL stats equality to position-level averages
      • entity state: remainingQty, status, exitTime, exitPrice (derived when TP-closed)
    • Gate on metrics: realized/unrealized gauges non-zero for filled scenarios; no “stuck OPEN with remainingQty=0”.

  • Recorded-trace replay harness (PR/CI fast lane)

    • Record real Bybit responses (orders, executions, position info, tick prices) from testnet/live read-only.
    • Replay deterministically through the engine with time-freeze; no network, real DB.
    • Add property-based fuzz on price paths and fill ordering; validate invariants:
      • CLOSED → unrealized=0; combined = realized
      • After each filled TP: remainingQty and realized move by percentage-weighted amounts
      • Weighted-exit derivation equals fills when exitPrice absent
      • Tournament/type PnL equals average of member positions
    • Fail on divergence between replay and original trace decisions.

  • Shadow/live parallel verification (canary)

    • Run engine shadowing live market (no order placement), consume real prices; compute TPSL decisions and hypothetical fills.
    • Compare shadow decisions vs actual exchange outcomes (from real positions or historical candles with slippage rules).
    • Export invariants to Prometheus; alert on breaches (e.g., closed-without-exitPrice, zero realized after TP).

Key invariants to assert everywhere​

  • Closure: remainingQty=0 ⇒ status=CLOSED, exitTime set, unrealized=0.
  • TP fills: sum(perc of filled TPs) drives remainingQty; realized USD% = weighted by filled sizes.
  • Exit price: if TP-closed and exitPrice missing ⇒ derived weighted average of filled prices.
  • Combined PnL: realized + unrealized equals combined within epsilon.
  • Aggregation: tournament/type realized/unrealized = average of member position values used.
  • No noise: no history entries on mere recalculation; only on real fills.

Practical tips​

  • Use real DBs; freeze time; seed RNG; only mock external payments.
  • Tiered pipeline: Replay (CI, fast) → Testnet-in-loop (nightly) → Shadow canary (continuous).
  • Persist failing traces as fixtures to prevent regressions.