Skip to content

ML Prediction — Implementation

Live

Working

  • RF model training and inference across all 3 horizons (1d, 7d, 30d)
  • Models pickled to model_cache/rf_{horizon}.pkl
  • Rule-based prediction fallback when no trained model exists
  • Volatility analyzer: realized vol (5/20/60-day), GARCH(1,1) forecast, K-means regime; feeds 3 ML features
  • Buffett scorecard with vol-adjusted safety margin
  • Accuracy + Sharpe-gated retraining: job_check_retrain() runs every 6 hours and retrains when either rolling 30-day 1d accuracy < 55% or annualized 1d Sharpe < SHARPE_RETRAIN_THRESHOLD (default: 0.0, i.e. negative Sharpe). Both gates are computed from PredictionResult rows. Insufficient samples (< ACCURACY_MIN_SAMPLES) always triggers a retrain.
  • Prediction evaluation: job_evaluate_predictions() runs every 24 hours, scores elapsed predictions against realized returns and writes to PredictionResult
  • Backtest API (api/routes/backtest.py): four endpoints — /backtest/summary, /backtest/by-ticker, /backtest/confidence-tiers, /backtest/rolling, /backtest/walk-forward
  • GET /backtest/walk-forward returns non-overlapping Sharpe windows (default 30 days each), annotated with whether each fold's Sharpe fell below the retrain threshold — useful for visualizing strategy drift over time
  • SHAP explainability: explain_prediction(ticker, market, horizon, db) in predictor.py uses shap.TreeExplainer on the loaded RF model to return per-feature SHAP contributions for the predicted class. Exposed via GET /predictions/{ticker}/explain?horizon=1d. Frontend shows a "Why?" toggle button per horizon on the Stock Detail ML Prediction card.

Key Gotchas

  • FEATURE_ORDER is order-critical (predictor.py). The vector is 29 features: 4 price/volume indicators, 4 technical indicators, 9 sentiment stats (avg/count/std × 3 windows), 3 volatility features, 4 fundamental features (eps_growth, revenue_growth, debt_equity_ratio, has_fundamentals), 2 macro features (vix_level, yield_spread_norm), and 3 earnings features (eps_surprise_pct, days_to_earnings, pre_earnings_flag). Adding or removing a feature requires updating three places in sync: build_features(), FEATURE_ORDER, and the features list in train_model(). Delete stale .pkl files in model_cache/ after any feature change — the loader does not validate feature count.

  • build_features() returns None when fewer than 10 price rows exist. Every call site must handle None explicitly — it is not an error, just insufficient data.

  • get_latest_predictions() uses a subquery to get the most recent prediction per ticker. Always use this function rather than a raw ORDER BY — it handles one-row-per-ticker grouping correctly.

  • Model cold start — new tickers need 120 days of price history before RF training is possible; rule-based fallback runs until then.

  • TrendingTopic accumulates one row per scheduler run — the compute_trends job (every 30 min) appends rather than upserts. The API deduplicates on read, keeping the newest row per sector. Daily prune keeps this table bounded (30-day retention).

  • Sharpe gate uses 1d horizon only — _compute_rolling_sharpe() in scheduler/jobs.py always evaluates the 1d model (the most frequently evaluated horizon). The accuracy gate also evaluates 1d only. Both gates trigger retraining of all three horizon models.

  • Earnings date bisect type safety — _earnings_dates and _all_earnings_dates in train_model() are normalized to date objects (not datetime) before being used with bisect.bisect_right. This guards against TypeError when SQLite returns date objects while PostgreSQL returns datetime. Both sides of each bisect comparison use the same .date() normalization. The same pattern is used by _macro_features() for MacroIndicator.date.

  • Initial load ordering — job_fetch_earnings() runs before predict_all() in run_initial_load(). This ensures days_to_earnings and pre_earnings_flag are populated before the first inference cycle.

  • shap package required for explain endpoint — explain_prediction() lazy-imports shap and returns None if not installed. The API route returns 503 in that case. shap>=0.45.0 is in requirements.txt. shap_values() return shape changed in shap 0.42: (n_samples, n_features, n_classes) ndarray instead of the older list-of-arrays. explain_prediction() detects the new format via ndim == 3 and indexes with shap_vals[0, :, best_idx]; the older list path is preserved for compatibility.

  • predict_all() is exception-isolated per ticker — each predict_ticker() call inside predict_all() is wrapped in a try/except; failures are logged as warnings and the loop continues to the next ticker. A single DB flush error or network drop does not abort the entire prediction cycle.

Adding a New ML Feature

  1. Add computation to build_features() in predictor.py
  2. Add the key to FEATURE_ORDER at the correct position
  3. Add the value in the same position to the features list in train_model()
  4. Add a human-readable label to FEATURE_LABELS in predictor.py
  5. Delete model_cache/rf_*.pkl — stale pickles will silently use wrong feature counts
  6. Update docs/spec/ml-prediction.md feature vector table

Volatility Regime Warning

Shown on Stock Detail above the ML predictions section. Only displayed when K-means regime is 2 (high). Calls compute_volatility_profile() from models/volatility_analyzer.py on each page load (120-day window).

Buffett Analysis

Tab at the bottom of Stock Detail page, sourced from models/buffett.py: - Safety score adjusted: −10 pts in high-vol regime, +5 pts in low-vol regime - Vol regime badge: color-coded (green/grey/orange), includes GARCH forecast and expanding/contracting vol indicator when vol_ratio is outside the 0.85–1.15 range - Returns None when fewer than 20 price rows available; dashboard shows info message

Adding a New Sector Keyword

Add to config.py:SECTOR_KEYWORDS with both English and Korean terms.