CodeMarketLabs8 years of systematic trading, and these 5 mistakes I made all of them. From biased backtests and survivorship bias to ghost orders, infrastructure failures, and crash recovery — a complete breakdown of what goes wrong and exactly how to fix it.
2026-05-22
8 years in market finance and systematic trading. These 5 mistakes, I made all of them. Some cost me money. Most cost me weeks of wasted work. This article breaks them down one by one — what they are, why they happen, and exactly how to fix them.
If your backtest is wrong, everything downstream is wrong. There is no point optimizing execution, infrastructure, or risk management if the research is built on a flawed simulation. There are three classic biases that corrupt backtests — and all three are fixable.
The first is look-ahead bias. This happens when your signal is computed on data that would not have been available at the time of the trade. The most common example: you generate a buy signal on the close of a daily bar, then execute on that same bar's open — which happened hours earlier. In practice, if your signal triggers at the close of a bar, your execution can only happen at the open of the next bar. If you backtest otherwise, every signal gives you a free 1% head start that you will never replicate in live trading. Do it across hundreds of signals and your backtest looks great while your live P&L bleeds from day one.
The second is survivorship bias — the one everyone knows about and almost no one corrects, mostly out of laziness or lack of access to proper data. If you backtest a long equity strategy on the current S&P 500 composition, every company in your universe is one that survived and thrived over the last 20 years. The companies that were in the index and went bankrupt, got delisted, or were removed — they are not in your dataset. Your backtest is literally selecting winners in hindsight. To fix this correctly, you need point-in-time index composition data: the exact list of S&P 500 constituents for every month over your backtest period, including the companies that no longer exist.
The third is ignoring transaction costs and spreads. Every trade you execute in live markets has a cost: broker commissions, bid-ask spread, slippage on entry and exit. If your backtest ignores these, you are systematically overstating your edge. The fix is simple: model the worst-case scenario. Apply maximum commissions and a conservative spread estimate (e.g. 0.20% per trade) to every transaction. A backtest that looks average with realistic costs is worth far more than a spectacular backtest that collapses the moment it hits real markets.
This one is not really a technical error — it is a behavioral one. Perpetual backtesting is the pattern of running backtest after backtest, refining endlessly, and never going live. I see it constantly with people I mentor. The backtest is solid. The risk parameters are reasonable. And yet the algo never gets deployed.
There are three reasons this happens. The first is a lack of confidence in the system — either because the person did not fully understand the code (especially when AI generated it), or because the jump from simulation to real capital feels too large. The second is the fear of the first drawdown. Any live strategy will hit a losing streak at some point. If you are not mentally prepared for that — if you have not already accepted it as a normal statistical outcome built into your backtest — you will cut the strategy at exactly the wrong moment. The third is simply not doing the work of connecting the backtest to a live broker: reading the API documentation, understanding order types, testing execution properly.
The fix is a forced process. Once your backtest is validated, move immediately to paper trading on Interactive Brokers. Do not wait until everything feels perfect. Run the algo live on a paper account, watch every order, verify every execution. Once it behaves exactly as your backtest predicted, you have the confidence to flip to live capital — because you have already seen it work in a real market environment.
This is where most algos silently fail. The backtest is correct. The strategy is sound. But the live implementation does not match the simulation — and the divergence compounds over time.
The most common discrepancies: timezone mismatches between your backtest data (often UTC) and your live execution environment (often exchange local time or New York Eastern) — particularly dangerous in crypto and cross-asset strategies. Lot size errors: sending quantity in units when the broker expects contracts, or vice versa, resulting in positions 100x larger than intended. Signal computation differences: your live signal uses a slightly different data source, different bar alignment, or different normalization than your backtest. And finally, ghost orders: your algorithm sends an order, the position opens at the broker, but the algo fails to register the confirmation — on the next cycle, it recomputes the signal, finds no position in memory, and sends another order. This repeats until you have a catastrophic position size you never intended.
The mitigation is to treat paper trading as seriously as backtesting. Every order type, every lot size, every execution path needs to be verified in a real market environment before live capital is at risk. Budget a full day to watch your algo run and confirm that every action matches your intention.
Running a trading algorithm on your local machine is a liability. Power outages, internet disconnections, system updates, laptop sleeping — any of these can kill your algo mid-session. If you trade a 24/7 market like crypto, you will eventually wake up to a chaos of open positions, missed signals, and no record of what happened.
The professional solution is a cloud virtual machine — a server running 24/7 on AWS, GCP, or Azure. Cost: $10–20/month depending on usage, often free for the first months on a trial tier. Your algo runs continuously regardless of what happens to your local hardware.
Beyond the server, a proper architecture requires a database. Every trade, every signal, every position — logged persistently. This gives you two things: the ability to analyze strategy performance in real time, and the foundation for crash recovery. Your algo should be stateless: at every cycle, it reads its current state from the database and the broker, computes what position it should have, and sends only the delta. It should not rely on in-memory variables that disappear when the process crashes.
This is the mistake that cost me the most. An algo with no crash recovery system is dangerous in proportion to how frequently it crashes — and every algo crashes eventually. When it restarts without knowing its prior state, it recomputes the signal from scratch, finds no position in memory, and opens a new one. If it crashes five times in a night, you have five times the intended position. You wake up thinking you made money and discover an outsized book that has nothing to do with your strategy.
The solution is a stateless architecture with persistent state. At the start of every cycle, your algo does three things before computing any signal: it reads from the database, reads from a local state file if available, and reconciles with the broker — which is always the ultimate source of truth. From this reconciled state, it computes what position it should hold and sends only the difference. No matter how many times it crashed, no matter at what point in the cycle, the next restart is always clean.
The last element every live algo needs is a circuit breaker. A parameter — readable at each cycle from a config file or database — that can be set to OFF. When triggered, the algo immediately closes all positions, cancels all orders, and sends a report of current capital and exposure. The trigger can be anything: a volatility spike, spread explosion, a drawdown threshold, or simply a manual kill switch. On institutional desks, this is handled by a dedicated risk desk. When you are trading alone, you are the risk desk. Build the kill switch before you go live — not after.
The cleanest sources are Compustat (via WRDS, available through many universities) and Sharadar via Quandl/Nasdaq Data Link. Bloomberg and Refinitiv also provide historical index membership. For a free approximation, the S&P 500 Wikipedia page lists current constituents with addition/removal dates, which can be scraped and used to reconstruct approximate historical compositions — imperfect but far better than using the current index blindly.
A cloud VM (AWS EC2 t3.micro or equivalent, $10–15/month), a lightweight database (PostgreSQL on RDS or even SQLite for low-frequency strategies), and a monitoring layer that sends alerts (email or Telegram) on errors, position anomalies, or circuit breaker triggers. For Interactive Brokers specifically, IB Gateway running on the VM handles the API connection persistently.
The safest pattern is to always reconcile position state from the broker API at the start of every cycle — req_positions() or req_account_updates() — before computing any signal. Never rely solely on in-memory state. If the broker shows an open position, do not open another regardless of what your local variables say. This reconciliation pattern also handles the crash recovery case automatically.
Always paper trade first — but treat it with the same seriousness as live capital. The goal of paper trading is not to validate the strategy (that is what the backtest is for) but to validate the execution pipeline: order routing, position tracking, lot sizes, timezone handling, error handling. Run the full paper trading setup for at least 2–4 weeks before switching to real capital.
At minimum: a maximum drawdown threshold from the strategy's expected drawdown profile, a position size check (alert if any position exceeds X% of capital), an order rate limiter (alert if more than N orders are sent in a given period — a sign of a runaway loop), and a manual on/off flag in a config file or database table that the algo reads at every cycle. More sophisticated implementations also monitor market regime indicators like VIX spikes or spread widening as automatic triggers.
8 years of systematic trading, and these 5 mistakes I made all of them. From biased backtests and survivorship bias to ghost orders, infrastructure failures, and crash recovery — a complete breakdown of what goes wrong and exactly how to fix it.
A complete breakdown of the Python backtesting framework I built with Claude Code — and how you can backtest any strategy from a single prompt. Covers the cash-flow engine architecture, the CLAUDE.md context file, overfitting prevention, and a live demo: MA Crossing 20/50 on SPY (2000–2026), +17.0% total return, Sharpe -0.08, 138 trades — including the full AI audit of why the result is bad and what to fix.