268 lines
13 KiB
Markdown
268 lines
13 KiB
Markdown
# Consistency Auditor Memory
|
|
|
|
## Last Audit: 2026-02-13 (Post Config Update)
|
|
|
|
### AUDIT RESULT: ⚠️ 1 CRITICAL BUG + 1 CRITICAL BEHAVIORAL DIVERGENCE
|
|
|
|
**1. CRITICAL BUG: Drawdown Peak Reset Inconsistency**
|
|
- **backtester.rs** resets `peak_portfolio_value` to current value on halt expiry (line 132)
|
|
- **bot.rs** does NOT reset peak on halt expiry (lines 365-377)
|
|
- **Impact**: Bot will re-trigger drawdown halt more frequently than backtest suggests, spending more time in cash. After a 15% drawdown triggers a 10-bar halt, a partial recovery followed by a minor dip will immediately re-trigger the halt in live trading but not in backtest.
|
|
- **Fix**: Add `self.peak_portfolio_value = portfolio_value;` to bot.rs after line 374 (inside the halt expiry block)
|
|
- **Code location**: `/home/work/Documents/rust/invest-bot/src/bot.rs:365-377`
|
|
|
|
**2. CRITICAL DIVERGENCE: PDT Enforcement Differs by Timeframe**
|
|
- **bot.rs** enforces PDT blocking on non-stop-loss sells (lines 619-628), with $25K exemption (lines 281-285)
|
|
- **backtester.rs** has PDT DISABLED entirely for backtest (lines 245-248: "informational only, not blocking")
|
|
- **Impact by timeframe**:
|
|
- **Daily mode**: No impact (buys in Phase 2, sells in Phase 1 on different bars → day trades impossible by design)
|
|
- **Hourly mode**: Potential divergence IF portfolio < $25K. Bot may skip exits to avoid PDT violation, holding positions overnight that backtest would have exited same-day. Backtester relies on late-day entry prevention (line 158-166) instead of exit blocking.
|
|
- **Portfolios >= $25K**: No divergence (PDT rule doesn't apply)
|
|
- **Verdict**: Acceptable for daily mode. Document for hourly mode. If hourly is primary deployment, verify backtest PDT day-trade count matches late-day entry prevention expectations.
|
|
|
|
---
|
|
|
|
## Config Changes Since Last Audit (2026-02-13)
|
|
|
|
User reported these config changes:
|
|
- Drawdown halt: 12% → 15% (`MAX_DRAWDOWN_HALT`)
|
|
- Drawdown cooldown: 20 bars → 10 bars (`DRAWDOWN_HALT_BARS`)
|
|
- Momentum pool: 10 stocks → 20 stocks (`TOP_MOMENTUM_COUNT`)
|
|
- Buy threshold: 4.5 → 4.0 (in signal generation)
|
|
|
|
**Verified**: All constants consistent between bot.rs and backtester.rs ✅
|
|
|
|
---
|
|
|
|
## VERIFIED CONSISTENT (2026-02-13 Audit) ✅
|
|
|
|
### Core Trading Logic ✅
|
|
- **Signal generation**: Both use shared `indicators::generate_signal()` (bot:739; bt:583,630)
|
|
- **Position sizing**: Both use shared `Strategy::calculate_position_size()` (bot:463-468; bt:199-201)
|
|
- Volatility-adjusted via ATR
|
|
- Confidence scaling: 0.7 + 0.3 * confidence
|
|
- Max position size cap: 25%
|
|
- Cash reserve: 5%
|
|
- **Stop-loss/trailing/time exit**: Both use shared `Strategy::check_stop_loss_take_profit()` (bot:473-486; bt:373-380)
|
|
- Hard max loss cap: 5%
|
|
- ATR-based stop: 3.0x ATR below entry
|
|
- Fixed fallback stop: 2.5%
|
|
- Trailing stop: 2.0x ATR after 2.0x ATR gain
|
|
- Time exit: 40 bars if below trailing activation threshold
|
|
|
|
### Portfolio Controls ✅
|
|
- **Cooldown timers**: Both implement 5-bar cooldown after stop-loss (bot:507-517,659-670; bt:169-173,294-299)
|
|
- **Ramp-up period**: Both limit to 1 new position per cycle/bar for first 15 bars (bot:543-552; bt:194-196)
|
|
- **Drawdown circuit breaker**: Both trigger at 15% with 10-bar cooldown (bot:353-362; bt:104-113)
|
|
- **BUT**: bot.rs missing peak reset on expiry (see Critical Bug #1)
|
|
- **Sector limits**: Both enforce max 2 per sector (bot:534-541; bt:184-191)
|
|
- **Max concurrent positions**: Both enforce max 7 (bot:525-532; bt:180-182)
|
|
- **Momentum ranking**: Both filter to top 20 momentum stocks (bot:818-838; bt:543-554)
|
|
- **bars_held increment**: Both increment at START of trading cycle/bar (bot:763-765; bt:539-541)
|
|
|
|
### Config Constants — ALL CONSISTENT ✅
|
|
Both files import and use identical values from config.rs:
|
|
- `ATR_STOP_MULTIPLIER`: 3.0x
|
|
- `ATR_TRAIL_MULTIPLIER`: 2.0x
|
|
- `ATR_TRAIL_ACTIVATION_MULTIPLIER`: 2.0x
|
|
- `MAX_POSITION_SIZE`: 25%
|
|
- `MAX_CONCURRENT_POSITIONS`: 7
|
|
- `MAX_SECTOR_POSITIONS`: 2
|
|
- `MAX_DRAWDOWN_HALT`: 15% (updated from 12%)
|
|
- `DRAWDOWN_HALT_BARS`: 10 (updated from 20)
|
|
- `REENTRY_COOLDOWN_BARS`: 5
|
|
- `RAMPUP_PERIOD_BARS`: 15
|
|
- `TOP_MOMENTUM_COUNT`: 20 (updated from 10)
|
|
- `TIME_EXIT_BARS`: 40
|
|
- `MIN_CASH_RESERVE`: 5%
|
|
- `MAX_LOSS_PCT`: 5%
|
|
|
|
### Warmup Requirements ✅
|
|
**Daily mode**: `max(35 MACD, 15 RSI, 50 EMA, 28 ADX, 20 BB, 63 momentum) + 5 = 68 bars`
|
|
**Hourly mode**: `max(35 MACD, 15 RSI, 200 EMA, 28 ADX, 20 BB, 63 momentum) + 5 = 205 bars`
|
|
|
|
Calculation in `config.rs:193-206` (`IndicatorParams::min_bars()`)
|
|
- RSI-2/3 warmup covered by RSI-14 requirement (15 > 3)
|
|
- MACD needs slow + signal periods (26 + 9 = 35)
|
|
- ADX needs 2x period for smoothing (14 * 2 = 28)
|
|
- Hourly EMA-200 dominates warmup requirement
|
|
|
|
Both bot.rs and backtester.rs fetch sufficient historical data and validate bar count before trading.
|
|
|
|
### Entry/Exit Flow ✅
|
|
**Both follow identical two-phase execution**:
|
|
- Phase 1: Process all sells (stop-loss, trailing, time exit, signals)
|
|
- Phase 2: Process buys for top momentum stocks only
|
|
- bot.rs lines 800-849
|
|
- backtester.rs lines 556-642
|
|
|
|
---
|
|
|
|
## INTENTIONAL DIFFERENCES (Not Bugs) ✅
|
|
|
|
### 1. Slippage Modeling
|
|
- **Backtester**: Applies 10 bps on both entry and exit (backtester.rs:71-78)
|
|
- **Live bot**: Uses actual fill prices from Alpaca API (bot.rs:567-571)
|
|
- **Verdict**: Expected difference. Backtester simulates realistic costs; live bot gets market fills.
|
|
|
|
### 2. RSI Short Period Scaling
|
|
- **Daily mode**: `rsi_short_period: 2` (Connors RSI-2 for mean reversion)
|
|
- **Hourly mode**: `rsi_short_period: 3` (adjusted for intraday noise)
|
|
- **Verdict**: Intentional design choice per comment "Slightly longer for hourly noise"
|
|
|
|
### 3. EMA Trend Period Scaling
|
|
- **Daily mode**: `ema_trend: 50` (50-day trend filter)
|
|
- **Hourly mode**: `ema_trend: 200` (200-hour ≈ 28.5-day trend filter)
|
|
- **Verdict**: Hourly uses 4x scaling (not 7x like other indicators) for longer-term trend context. Appears intentional.
|
|
|
|
### 4. Hourly Late-Day Entry Prevention
|
|
- **Backtester**: Blocks entries after 19:00 UTC in hourly mode (backtester.rs:158-166) to prevent same-day stop-loss exits
|
|
- **Bot**: Relies on PDT exit blocking instead (bot.rs:619-628)
|
|
- **Verdict**: Two different approaches to PDT prevention. See Critical Divergence #2.
|
|
|
|
---
|
|
|
|
## STRATEGY ARCHITECTURE (2026-02-12, Still Current)
|
|
|
|
### Regime-Adaptive Dual Signal
|
|
The strategy uses **ADX for regime detection** and switches between two modes:
|
|
|
|
#### RANGE-BOUND (ADX < 20): Mean Reversion
|
|
- **Entry**: Connors RSI-2 extreme oversold (RSI-2 < 10) + price above EMA trend
|
|
- **Exit**: RSI-2 extreme overbought (RSI-2 > 90) or standard exits
|
|
- **Conviction boosters**: Bollinger Band extremes, volume confirmation
|
|
|
|
#### TRENDING (ADX > 25): Momentum Pullback
|
|
- **Entry**: Pullbacks in strong trends (RSI-14 dips 25-40, price near EMA support, MACD confirming)
|
|
- **Exit**: Trend break (EMA crossover down) or standard exits
|
|
- **Conviction boosters**: Strong trend (ADX > 40), DI+/DI- alignment
|
|
|
|
#### UNIVERSAL SIGNALS (Both Regimes)
|
|
- RSI-14 extremes in trending context
|
|
- MACD crossovers
|
|
- EMA crossovers
|
|
- Volume gate (reduces scores 50% if volume < 80% of 20-period MA)
|
|
|
|
### Signal Thresholds (Updated 2026-02-13)
|
|
- **StrongBuy**: total_score >= 7.0
|
|
- **Buy**: total_score >= 4.0 (was 4.5)
|
|
- **StrongSell**: total_score <= -7.0
|
|
- **Sell**: total_score <= -4.0
|
|
- **Hold**: everything else
|
|
|
|
Confidence: `(total_score.abs() / 12.0).min(1.0)`
|
|
|
|
---
|
|
|
|
## KEY LESSONS
|
|
|
|
### 1. Shared Logic Eliminates Drift
|
|
Extracting common logic into `strategy.rs` ensures bot and backtester CANNOT diverge. All core trading logic (signal generation, position sizing, stop-loss/trailing/time exit) is now in shared modules.
|
|
|
|
### 2. Drawdown Circuit Breaker Needs Peak Reset on Resume
|
|
Without resetting the peak when halt expires, any minor dip after partial recovery will immediately re-trigger the halt. This creates cascading halts that keep the bot in cash for extended periods. Backtester had this right; bot.rs needs the fix.
|
|
|
|
### 3. PDT Protection Strategy Differs by Timeframe
|
|
- **Daily mode**: Phase separation (sells Phase 1, buys Phase 2) prevents day trades by construction. PDT enforcement not needed.
|
|
- **Hourly mode**: Late-day entry prevention (backtester) vs exit blocking (bot) are two valid approaches, but they're not identical. For portfolios < $25K, bot will hold positions overnight more often than backtest suggests.
|
|
|
|
### 4. Config Constants Must Be Audited After Every Change
|
|
Recent changes to drawdown thresholds, momentum pool size, and buy thresholds were all consistent between bot and backtester, but manual audit was required to verify. Future changes should trigger automated consistency checks.
|
|
|
|
### 5. Warmup Must Account for Longest Indicator Chain
|
|
For hourly mode, EMA-200 dominates warmup (205 bars). The `+ 5` safety margin in `min_bars()` is critical.
|
|
|
|
### 6. Data Type Consistency for PDT Tracking (FIXED)
|
|
bot.rs now uses `Vec<NaiveDate>` for day_trades (line 56), matching backtester.rs (line 42). Previous audit found this as a critical bug; it's now fixed. ✅
|
|
|
|
---
|
|
|
|
## AUDIT CHECKLIST (For Future Audits)
|
|
|
|
When new changes are made, verify:
|
|
|
|
1. **Signal generation**: Still using shared `indicators::generate_signal()`?
|
|
2. **Position sizing**: Still using shared `Strategy::calculate_position_size()`?
|
|
3. **Risk management**: Still using shared `Strategy::check_stop_loss_take_profit()`?
|
|
4. **Cooldown timers**: Identical logic in both files?
|
|
5. **Ramp-up period**: Identical logic in both files?
|
|
6. **Drawdown halt**: Identical trigger logic? Peak reset on expiry?
|
|
7. **Sector limits**: Same `MAX_SECTOR_POSITIONS` constant?
|
|
8. **Max positions**: Same `MAX_CONCURRENT_POSITIONS` constant?
|
|
9. **Momentum ranking**: Same `TOP_MOMENTUM_COUNT` constant?
|
|
10. **bars_held increment**: Both increment at START of cycle/bar?
|
|
11. **Warmup calculation**: Does `min_bars()` cover all indicators?
|
|
12. **Config propagation**: Are new constants used consistently?
|
|
13. **NaN handling**: Safe defaults for all indicator checks?
|
|
14. **ATR guards**: Checks for `> 0.0` before division?
|
|
15. **PDT protection**: Same constants, logic, and data types? Document timeframe-specific behavior.
|
|
|
|
---
|
|
|
|
## FILES AUDITED (2026-02-13)
|
|
- `/home/work/Documents/rust/invest-bot/src/bot.rs` (933 lines)
|
|
- `/home/work/Documents/rust/invest-bot/src/backtester.rs` (1002 lines)
|
|
- `/home/work/Documents/rust/invest-bot/src/config.rs` (222 lines)
|
|
- `/home/work/Documents/rust/invest-bot/src/strategy.rs` (141 lines)
|
|
- `/home/work/Documents/rust/invest-bot/src/types.rs` (234 lines)
|
|
|
|
**Total**: 2,532 lines audited
|
|
**Issues found**: 1 critical bug (drawdown peak reset), 1 critical behavioral divergence (PDT enforcement)
|
|
**Status**: ⚠️ FIX REQUIRED BEFORE PRODUCTION (drawdown peak reset)
|
|
|
|
---
|
|
|
|
## REQUIRED FIX: Drawdown Peak Reset on Halt Expiry
|
|
|
|
**File**: `/home/work/Documents/rust/invest-bot/src/bot.rs`
|
|
**Location**: Lines 365-377 (inside drawdown halt expiry check)
|
|
|
|
**Current code**:
|
|
```rust
|
|
// Auto-resume after time-based cooldown
|
|
if self.drawdown_halt {
|
|
if let Some(halt_start) = self.drawdown_halt_start {
|
|
if self.trading_cycle_count >= halt_start + DRAWDOWN_HALT_BARS {
|
|
tracing::info!(
|
|
"Drawdown halt expired after {} cycles. Resuming trading at {:.2}% drawdown.",
|
|
DRAWDOWN_HALT_BARS,
|
|
drawdown_pct * 100.0
|
|
);
|
|
self.drawdown_halt = false;
|
|
self.drawdown_halt_start = None;
|
|
// MISSING: self.peak_portfolio_value = portfolio_value;
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
**Required change**: Add after line 374:
|
|
```rust
|
|
self.peak_portfolio_value = portfolio_value;
|
|
```
|
|
|
|
**Complete fixed block**:
|
|
```rust
|
|
// Auto-resume after time-based cooldown
|
|
if self.drawdown_halt {
|
|
if let Some(halt_start) = self.drawdown_halt_start {
|
|
if self.trading_cycle_count >= halt_start + DRAWDOWN_HALT_BARS {
|
|
tracing::info!(
|
|
"Drawdown halt expired after {} cycles. Resuming trading. \
|
|
Peak reset from ${:.2} to ${:.2} (was {:.2}% drawdown).",
|
|
DRAWDOWN_HALT_BARS,
|
|
self.peak_portfolio_value,
|
|
portfolio_value,
|
|
drawdown_pct * 100.0
|
|
);
|
|
self.drawdown_halt = false;
|
|
self.drawdown_halt_start = None;
|
|
// Reset peak to current value to prevent cascading re-triggers.
|
|
self.peak_portfolio_value = portfolio_value;
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
**Why this matters**: Without this reset, after a 15% drawdown triggers a 10-bar halt, the bot measures future drawdown from the OLD peak. If the portfolio recovers to 12% drawdown and then dips to 13%, it immediately re-triggers the halt. This creates cascading halts. The backtester correctly resets the peak, so backtest results show fewer/shorter halts than live trading would experience.
|