6.1 KiB
Hourly Backtest Catastrophic Failure Analysis (2026-02-11)
Results Summary
- Period: 3 months hourly (Nov 12, 2025 - Feb 11, 2026)
- Total Return: -11.44% ($88,563 from $100k)
- CAGR: -38.48%, Sharpe: -2.53, Sortino: -0.38
- Win Rate: 10% (2/20), Profit Factor: 0.08
- All 20 trades occurred in first 8 trading days, then system went to 100% cash permanently
Trade-by-Trade Analysis
All trades between Nov 12-21 (8 trading days, ~56 hourly bars):
Buys
- Nov 12 17:00 - MU @$245.29 (89 shares = $21.8k)
- Nov 12 17:00 - AMD @$255.39 (86 shares = $21.9k)
- Nov 12 20:00 - GOOGL @$287.08 (76 shares = $21.8k)
- Nov 12 20:00 - CAT @$573.69 (38 shares = $21.8k)
- Nov 12 20:00 - SNOW @$269.80 (28 shares = $7.6k) -- total 5 positions, 95% invested
Sell Cascade (Nov 13-21)
Nov 13: GOOGL -$636 (-2.9%), MU -$694 (-3.2%), SNOW -$226 (-3.0%), CAT -$744 (-3.4%) Nov 13: Rebuy ASML, AAPL Nov 14: ASML -$705 (-3.4%), AMD -$1,352 (-6.2%!!) -- exceeded 4% max loss cap Nov 14: Rebuy MU, GOOGL, CAT, SNOW Nov 17: AAPL -$440 (-2.1%), MU -$699 (-3.3%) Nov 17: Rebuy AMD, GOOGL, ASML Nov 18: AMD -$1,582 (-7.6%!!), CAT -$493 (-2.4%), GOOGL -$426 (-2.0%) Nov 18: Rebuy AAPL, MU Nov 19: MU -$941 (-4.6%), SNOW -$39 (-0.6%) Nov 19: Rebuy AMD, ISRG, SNOW Nov 20: AMD -$1,170 (-5.8%!), ISRG -$385 (-1.9%), SNOW -$164 (-2.6%), ASML -$442 (-2.2%) Nov 20: Rebuy MU Nov 21: MU -$1,274 (-6.6%!!), AAPL +$275 (+1.4%) -- ONLY WIN
Key Observations
- AMD losses: -$1,352, -$1,582, -$1,170 = -$4,104 total (36% of all losses). ALL exceeded the 4% max loss cap.
- MU losses: -$694, -$699, -$941, -$1,274 = -$3,608 (32% of all losses).
- After Nov 21: drawdown hit 11.65%, exceeding 10% halt. System went to 100% cash PERMANENTLY.
- Equity curve shows 0 positions from Nov 21 through Feb 11 (461 bars of nothing).
- Only 2 wins out of 20: GOOGL +$701, AAPL +$275. Total wins = $976.
Root Cause #1: Absurd Indicator Period Scaling (FUNDAMENTAL)
The 7x multiplier creates these hourly indicator periods:
- RSI: 14 * 7 = 98 bars (14 trading days)
- MACD fast: 12 * 7 = 84 bars
- MACD slow: 26 * 7 = 182 bars (26 trading days)
- MACD signal: 9 * 7 = 63 bars
- EMA short: 9 * 7 = 63 bars
- EMA long: 21 * 7 = 147 bars
- EMA trend: 50 * 7 = 350 bars (50 trading days)
- ADX: 14 * 7 = 98 bars
- Bollinger: 20 * 7 = 140 bars
- Volume MA: 20 * 7 = 140 bars
- Momentum (ROC): 63 * 7 = 441 bars (63 trading days)
min_bars() = max(182+63, 98+1, 350, 98*2, 140, 441) + 5 = 446 bars
This means the system needs ~64 trading days (446/7) of WARMUP before it can even produce a valid signal. For a 3-month backtest, that eats most of the data. The indicators that DO produce values are extremely slow-moving and unresponsive to hourly price action.
Why 7x is Wrong
The daily parameters (RSI-14, MACD 12/26/9) are designed for daily price action noise. On hourly bars, there are 6.5-7 bars per day, so naively you'd think 7x preserves the "look-back in days." But this ignores that:
- Hourly bars have fundamentally different noise characteristics (mean-reverting intraday patterns)
- A 98-period RSI on hourly bars is insanely slow -- it would take a massive multi-week move to push RSI to oversold/overbought
- MACD with 182-period slow EMA cannot detect hourly momentum shifts
- The strategy's edge is supposed to be momentum + pullback detection. A 441-bar momentum period on hourly data measures ~3-month trends, not tactical momentum.
Root Cause #2: Drawdown Halt is Terminal
MAX_DRAWDOWN_HALT = 10%, MAX_DRAWDOWN_RESUME = 5%.
Once drawdown exceeds 10%, the system stops buying. It only resumes when drawdown recovers to 5%. But with 0 positions, there's no way to recover! The portfolio sits in cash forever.
This is a logical impossibility: you can't recover from drawdown without taking new positions, but the system won't take new positions until drawdown recovers.
Root Cause #3: Max Loss Cap Not Working
AMD lost -6.2%, -7.6%, -5.8% in three trades despite MAX_LOSS_PCT = 4%. MU lost -6.6% in one trade.
The check_stop_loss_take_profit function checks pnl_pct <= -MAX_LOSS_PCT but this only triggers on the NEXT bar after the loss occurs. If a stock gaps down or moves 6% in one hourly bar, the 4% cap is breached before the check runs. On hourly timeframe, overnight gaps can easily exceed 4%.
Root Cause #4: All Indicators Fire Simultaneously at Warmup Edge
When indicators first become valid (right after the warmup period), all the EMA crossovers, MACD crossovers, etc. fire at once. This creates a burst of buy signals, putting 95% of capital at risk immediately. The system went from 0 to 5 positions in 3 hours on Nov 12.
Proposed Fixes
Fix 1: Proper Hourly Indicator Periods (NOT 7x scaling)
Use empirically appropriate periods for hourly timeframe:
- RSI: 14 (same as daily -- 14 hourly bars captures 2 trading days of momentum)
- MACD: 12/26/9 (same as daily -- these already work on any timeframe)
- EMA short: 20, EMA long: 50, EMA trend: 100 (~14 trading days)
- ADX: 14
- Bollinger: 20
- Volume MA: 20
- ATR: 14
- Momentum ROC: 63 (same as daily -- 63 hourly bars = ~9 trading days)
Rationale: Most technical indicators were designed for BARS, not calendar time. RSI-14 means "14 bars of price action" regardless of timeframe. The 7x scaling was conceptually wrong -- it assumed indicators need calendar-day equivalence, but they need bar-count equivalence.
Fix 2: Remove or Redesign Drawdown Halt for Backtesting
Options: a) Remove drawdown halt entirely in backtesting (it's more appropriate for live trading where you want manual review) b) Make drawdown halt time-based: halt for N bars, then auto-resume c) Change to a gradual reduction: reduce position size by 50% instead of going to 0
Fix 3: Limit Initial Deployment Speed
Don't go from 0 to 5 positions in one bar. Add a "ramp-up" period where max positions increases gradually (e.g., 1 per day for first week).
Fix 4: Tighter Max Loss Cap with Intra-Bar Checks
For hourly mode, use tighter max loss (3%) or check at open price vs entry to catch gap losses earlier.
Fix 5: Use SIP Feed Instead of IEX
Change feed=iex to feed=sip for consolidated market data. IEX-only volume is unreliable for volume-based signals.