PDT protection
This commit is contained in:
@@ -1,8 +1,18 @@
|
||||
# Consistency Auditor Memory
|
||||
|
||||
## Last Audit: 2026-02-12 (Regime-Adaptive Dual Strategy Update)
|
||||
## Last Audit: 2026-02-12 (PDT Protection)
|
||||
|
||||
### AUDIT RESULT: ✅ NO CRITICAL BUGS FOUND
|
||||
### AUDIT RESULT: ⚠️ 1 CRITICAL BUG FOUND
|
||||
|
||||
**PDT (Pattern Day Trading) protection data type mismatch:**
|
||||
- bot.rs uses `Vec<String>` for day_trades (line 56)
|
||||
- backtester.rs uses `Vec<NaiveDate>` for day_trades (line 42)
|
||||
- **Impact**: String parsing on every PDT check, silent failures on malformed dates, performance degradation
|
||||
- **Fix required**: Change bot.rs to use `Vec<NaiveDate>` internally (see detailed fix below)
|
||||
|
||||
---
|
||||
|
||||
## Previous Audit: 2026-02-12 (Regime-Adaptive Dual Strategy Update)
|
||||
|
||||
The refactor to extract shared logic into `strategy.rs` has **eliminated all previous consistency issues**. Bot and backtester now share identical implementations for all critical trading logic.
|
||||
|
||||
@@ -129,6 +139,7 @@ Confidence: `(total_score.abs() / 12.0).min(1.0)`
|
||||
- **Drawdown halt**: 12% triggers 20-bar cooldown (was 35 bars)
|
||||
- **Reentry cooldown**: 5 bars after stop-loss (was 7)
|
||||
- **Ramp-up period**: 15 bars, 1 new position per bar (was 30 bars)
|
||||
- **PDT protection**: Max 3 day trades in rolling 5-business-day window (bot:34-36; bt:279-280)
|
||||
|
||||
### Backtester
|
||||
- **Slippage**: 10 bps per trade
|
||||
@@ -153,6 +164,13 @@ When ATR is zero/unavailable (e.g., low volatility or warmup), code falls back t
|
||||
### 5. Slippage Modeling is Non-Negotiable
|
||||
The backtester applies 10 bps slippage on both sides (20 bps round-trip) to simulate realistic fills. This prevents overfitting to unrealistic backtest performance.
|
||||
|
||||
### 6. Data Type Consistency Matters for PDT Protection
|
||||
**CRITICAL**: bot.rs and backtester.rs must use the SAME data type for day_trades tracking. Using `Vec<String>` in bot.rs vs `Vec<NaiveDate>` in backtester.rs creates:
|
||||
- String parsing overhead on every PDT check
|
||||
- Silent failures if malformed dates enter the system (parse errors return false)
|
||||
- Inconsistent error handling between live and backtest
|
||||
- **Fix**: Change bot.rs to store `Vec<NaiveDate>` internally, parse once on load, serialize to JSON as strings
|
||||
|
||||
---
|
||||
|
||||
## AUDIT CHECKLIST (For Future Audits)
|
||||
@@ -173,17 +191,104 @@ When new changes are made, verify:
|
||||
12. **Config propagation**: Are new constants used consistently?
|
||||
13. **NaN handling**: Safe defaults for all indicator checks?
|
||||
14. **ATR guards**: Checks for `> 0.0` before division?
|
||||
15. **PDT protection**: Same constants, logic, and DATA TYPES in both files?
|
||||
|
||||
---
|
||||
|
||||
## FILES AUDITED (2026-02-12)
|
||||
- `/home/work/Documents/rust/invest-bot/src/bot.rs` (785 lines)
|
||||
- `/home/work/Documents/rust/invest-bot/src/backtester.rs` (880 lines)
|
||||
- `/home/work/Documents/rust/invest-bot/src/config.rs` (199 lines)
|
||||
- `/home/work/Documents/rust/invest-bot/src/indicators.rs` (651 lines)
|
||||
- `/home/work/Documents/rust/invest-bot/src/strategy.rs` (141 lines)
|
||||
## FILES AUDITED (2026-02-12 PDT Audit)
|
||||
- `/home/work/Documents/rust/invest-bot/src/bot.rs` (921 lines)
|
||||
- `/home/work/Documents/rust/invest-bot/src/backtester.rs` (907 lines)
|
||||
- `/home/work/Documents/rust/invest-bot/src/types.rs` (234 lines)
|
||||
- `/home/work/Documents/rust/invest-bot/src/paths.rs` (68 lines)
|
||||
|
||||
**Total**: 2,890 lines audited
|
||||
**Issues found**: 0 critical, 0 medium, 0 low
|
||||
**Status**: ✅ PRODUCTION READY
|
||||
**Total**: 2,130 lines audited
|
||||
**Issues found**: 1 critical (data type mismatch), 0 medium, 0 low
|
||||
**Status**: ⚠️ FIX REQUIRED BEFORE PRODUCTION
|
||||
|
||||
---
|
||||
|
||||
## CRITICAL FIX REQUIRED: PDT Data Type Mismatch
|
||||
|
||||
**Problem**: bot.rs stores day_trades as `Vec<String>`, backtester.rs stores as `Vec<NaiveDate>`
|
||||
|
||||
**Required Changes to bot.rs:**
|
||||
|
||||
1. Line 56: Change field type
|
||||
```rust
|
||||
day_trades: Vec<NaiveDate>, // was Vec<String>
|
||||
```
|
||||
|
||||
2. Lines 197-218: Load with parse-once strategy
|
||||
```rust
|
||||
fn load_day_trades(&mut self) {
|
||||
if LIVE_DAY_TRADES_FILE.exists() {
|
||||
match std::fs::read_to_string(&*LIVE_DAY_TRADES_FILE) {
|
||||
Ok(content) if !content.is_empty() => {
|
||||
match serde_json::from_str::<Vec<String>>(&content) {
|
||||
Ok(date_strings) => {
|
||||
self.day_trades = date_strings
|
||||
.iter()
|
||||
.filter_map(|s| NaiveDate::parse_from_str(s, "%Y-%m-%d").ok())
|
||||
.collect();
|
||||
self.prune_old_day_trades();
|
||||
if !self.day_trades.is_empty() {
|
||||
tracing::info!("Loaded {} day trades in rolling window.", self.day_trades.len());
|
||||
}
|
||||
}
|
||||
Err(e) => tracing::error!("Error parsing day trades file: {}", e),
|
||||
}
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
3. Lines 220-229: Serialize to JSON as strings
|
||||
```rust
|
||||
fn save_day_trades(&self) {
|
||||
let date_strings: Vec<String> = self.day_trades
|
||||
.iter()
|
||||
.map(|d| d.format("%Y-%m-%d").to_string())
|
||||
.collect();
|
||||
match serde_json::to_string_pretty(&date_strings) {
|
||||
Ok(json) => {
|
||||
if let Err(e) = std::fs::write(&*LIVE_DAY_TRADES_FILE, json) {
|
||||
tracing::error!("Error saving day trades file: {}", e);
|
||||
}
|
||||
}
|
||||
Err(e) => tracing::error!("Error serializing day trades: {}", e),
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
4. Lines 231-239: Use native date comparison
|
||||
```rust
|
||||
fn prune_old_day_trades(&mut self) {
|
||||
let cutoff = self.business_days_ago(PDT_ROLLING_BUSINESS_DAYS);
|
||||
self.day_trades.retain(|&d| d >= cutoff);
|
||||
}
|
||||
```
|
||||
|
||||
5. Lines 256-267: Use native date comparison
|
||||
```rust
|
||||
fn day_trades_in_window(&self) -> usize {
|
||||
let cutoff = self.business_days_ago(PDT_ROLLING_BUSINESS_DAYS);
|
||||
self.day_trades.iter().filter(|&&d| d >= cutoff).count()
|
||||
}
|
||||
```
|
||||
|
||||
6. Lines 284-289: Push NaiveDate
|
||||
```rust
|
||||
fn record_day_trade(&mut self) {
|
||||
let today = Utc::now().date_naive();
|
||||
self.day_trades.push(today);
|
||||
self.save_day_trades();
|
||||
}
|
||||
```
|
||||
|
||||
**Benefits of fix:**
|
||||
- Type safety: malformed dates cannot enter the vec
|
||||
- Performance: native date comparison vs string parsing on every check
|
||||
- Consistency: matches backtester implementation exactly
|
||||
- Reliability: no silent failures from parse errors
|
||||
|
||||
@@ -1,42 +1,48 @@
|
||||
# Quant-Rust-Strategist Memory
|
||||
|
||||
## Architecture Overview
|
||||
- 50-symbol universe across 9 sectors
|
||||
- Hybrid momentum + mean-reversion via composite signal scoring in `generate_signal()`
|
||||
- Backtester restricts buys to top 8 momentum stocks (TOP_MOMENTUM_COUNT=8)
|
||||
- Signal thresholds: StrongBuy>=6.0, Buy>=4.5, Sell<=-3.5, StrongSell<=-6.0
|
||||
- ~100-symbol universe across 14 sectors (expanded from original 50)
|
||||
- Hybrid momentum + mean-reversion via regime-adaptive dual signal in `generate_signal()`
|
||||
- strategy.rs: shared logic between bot.rs and backtester.rs
|
||||
- Backtester restricts buys to top 10 momentum stocks (TOP_MOMENTUM_COUNT=10)
|
||||
- Signal thresholds: StrongBuy>=7.0, Buy>=4.5, Sell<=-4.0, StrongSell<=-7.0
|
||||
|
||||
## Key Finding: Daily vs Hourly Parameter Sensitivity (2026-02-11)
|
||||
## PDT Implementation (2026-02-12)
|
||||
- Tracks day trades in rolling 5-business-day window, max 3 allowed
|
||||
- CRITICAL: Stop-loss exits must NEVER be blocked by PDT (risk mgmt > compliance)
|
||||
- Late-day entry prevention: On hourly, block buys after 19:00 UTC (~last 2 hours)
|
||||
- Prevents entries needing same-day stop-loss exits
|
||||
- Reduced hourly trades 100->86, improved PF 1.24->1.59
|
||||
- "PDT performance degradation" was mostly IEX data stochasticity, not actual PDT blocking
|
||||
|
||||
### Daily Timeframe Optimization (Successful)
|
||||
- Reduced momentum_period 252->63, ema_trend 200->50 in IndicatorParams::daily()
|
||||
- Reduced warmup from 267 bars to ~70 bars
|
||||
- Result: Sharpe 0.53->0.86 (+62%), Win rate 40%->50%, PF 1.32->1.52
|
||||
|
||||
### Hourly Timeframe: DO NOT CHANGE FROM BASELINE
|
||||
- Hourly IndicatorParams: momentum=63, ema_trend=200 (long lookbacks filter IEX noise)
|
||||
- Shorter periods (momentum=21, ema_trend=50): CATASTROPHIC -8% loss
|
||||
- ADX threshold lowered 25->20 (shared const, helps both timeframes)
|
||||
|
||||
### Failed Experiments (avoid repeating)
|
||||
1. Tighter ATR stop (2.0x): too many stop-outs on hourly. Keep 2.5x
|
||||
2. Lower buy threshold (3.5): too many weak entries. Keep 4.5
|
||||
3. More positions (8): spreads capital too thin. Keep 5
|
||||
4. Higher risk per trade (1.0-1.2%): compounds losses. Keep 0.8%
|
||||
5. Wider trail (2.5x ATR): misses profit on hourly. Keep 1.5x
|
||||
6. Lower volume threshold (0.7): bad trades on IEX. Keep 0.8
|
||||
7. Lower cash reserve (3%): marginal, not worth risk. Keep 5%
|
||||
## Backtest Results (3-month, 2026-02-12, post-PDT-fix)
|
||||
### Hourly: +12.00%, Sharpe 0.12, PF 1.59, 52% WR, 86 trades, MaxDD -9.36%
|
||||
### Daily: +11.68%, Sharpe 2.65, PF 3.07, 61% WR, 18 trades, MaxDD -5.36%
|
||||
|
||||
## Current Parameters (config.rs)
|
||||
- ATR Stop: 2.5x | Trail: 1.5x distance, 1.5x activation
|
||||
- Risk: 0.8%/trade, max 22% position, 5% cash reserve, 4% max loss
|
||||
- Max 5 positions, 2/sector | Drawdown halt: 10% (35 bars) | Time exit: 30
|
||||
- Cooldown: 7 bars | Ramp-up: 30 bars | Slippage: 10bps
|
||||
- Daily params: momentum=63, ema_trend=50
|
||||
- Hourly params: momentum=63, ema_trend=200
|
||||
- ADX: threshold=20, strong=35
|
||||
- ATR Stop: 3.0x | Trail: 2.0x distance, 2.0x activation
|
||||
- Risk: 1.2%/trade, max 25% position, 5% cash reserve, 5% max loss
|
||||
- Max 7 positions, 2/sector | Drawdown halt: 12% (20 bars) | Time exit: 40
|
||||
- Cooldown: 5 bars | Ramp-up: 15 bars | Slippage: 10bps
|
||||
- Daily: momentum=63, ema_trend=50 | Hourly: momentum=63, ema_trend=200
|
||||
- ADX: range<20, trend>25, strong>40
|
||||
|
||||
## Hourly Timeframe: DO NOT CHANGE FROM BASELINE
|
||||
- Hourly IndicatorParams: momentum=63, ema_trend=200 (long lookbacks filter IEX noise)
|
||||
- Shorter periods (momentum=21, ema_trend=50): CATASTROPHIC -8% loss
|
||||
|
||||
## Failed Experiments (avoid repeating)
|
||||
1. Tighter ATR stop (<3.0x): too many stop-outs on hourly
|
||||
2. Lower buy threshold (3.5): too many weak entries. Keep 4.5
|
||||
3. Blocking stop-loss exits for PDT: traps capital in losers, dangerous
|
||||
4. Lower volume threshold (0.7): bad trades on IEX. Keep 0.8
|
||||
5. Shorter hourly lookbacks: catastrophic losses
|
||||
|
||||
## IEX Data Stochasticity
|
||||
- Backtests have significant run-to-run variation from IEX data timing
|
||||
- Do NOT panic about minor performance swings between runs
|
||||
- Always run 2-3 times and compare ranges before concluding a change helped/hurt
|
||||
|
||||
## Build Notes
|
||||
- `cargo build --release` compiles clean (only dead_code warnings)
|
||||
- No tests exist
|
||||
- Backtests have stochastic variation from IEX data timing
|
||||
|
||||
Reference in New Issue
Block a user