Tech Radar¶
Tools, APIs, and integrations worth revisiting. Not planned — just on the list.
StockTwits ⚠️ BROKEN — needs rework¶
Status: Disabled. The public unauthenticated API (api.stocktwits.com/api/2/streams/symbol/) is unreliable and has been progressively restricted.
What it was: Retail sentiment stream for US tickers — scraped message bodies into NewsArticle rows and fed them into the sentiment pipeline.
What's needed to revive it: Either find a working public endpoint, switch to an authenticated API (requires account + approval), or replace with an alternative retail sentiment source (Reddit/WallStreetBets via Pushshift, Twitter/X API, etc.).
Code: scraper/stocktwits.py — job commented out in scheduler/jobs.py.
LLM-Powered Adaptive Scraper¶
What it is: A scraping layer where a cheap LLM (DeepSeek V3 via its OpenAI-compatible API — key already in .env but unused) extracts structured data from raw HTML instead of hardcoded CSS selectors. Fetch the page with curl_cffi or Scrapling, pass cleaned HTML to DeepSeek with a prompt like "extract headlines, URLs, and timestamps as JSON." No browser control needed.
Why interesting: CSS selectors break silently when sites redesign. An LLM extractor survives layout changes without code changes. DeepSeek V3 is cheap enough (~$0.001/page) that running it on every scrape cycle is negligible cost.
Why not yet: Current NAVER and Finnhub scrapers are stable and deterministic. LLM extraction adds latency and a non-deterministic failure mode (hallucinated fields) that's harder to monitor than a broken selector. Worth revisiting if a scraper breaks repeatedly after site redesigns.
Variant worth knowing about: browser-use extends this to full Playwright browser control — the LLM navigates, clicks, and scrolls. Overkill for our current targets but relevant if we ever scrape login-walled or JS-heavy sites.
Notion CLI (WSL)¶
What it is: Official Notion terminal client — or any viable fallback — for reading Notion content from a WSL environment.
Why interesting: Could be useful if Notion ever becomes the knowledge/notes layer instead of Obsidian. Currently Obsidian is the preferred path.
What's needed: Spike to confirm whether the official client works in WSL, or identify the best-supported alternative.
Tailscale for Remote Access¶
What it is: A zero-config VPN mesh (free for personal use, up to 3 users) that lets you reach the laptop's FastAPI backend and frontend from any device — phone, desktop, anywhere — without port-forwarding or deploying to the cloud.
Why interesting: The scheduler has to run on the laptop anyway, so there's no benefit to deploying the API elsewhere. Tailscale lets you hit http://laptop-tailscale-ip:8000 and :5173 from anywhere as if you're on the local network. No cold starts, no cost, no exposing ports publicly.
What's needed: Install Tailscale on the laptop and any client devices, enable it, done. Optional: set a stable machine name via Tailscale MagicDNS so the address doesn't change.
Mobile Device as Scheduler Host (Samsung Galaxy S21 Ultra)¶
What it is: Running the APScheduler-based backend on an Android phone (Exynos 2100, 12 GB RAM) via Termux, as an alternative to the current low-spec laptop host.
Why interesting: The Exynos 2100 CPU is significantly faster than a 2nd/3rd gen Intel mobile chip, and 12 GB RAM comfortably fits Whisper + both FinBERT models simultaneously. A phone cooling accessory (clip-on Peltier cooler or fan grip) could address the main thermal concern for sustained workloads.
Why not yet: Android process management requires a persistent wakelock to keep Termux alive. No CUDA support on Mali-G78 means Whisper still runs CPU-only. The Exynos 2100 specifically has a well-documented thermal throttling problem — it runs significantly hotter than the Snapdragon S21 Ultra and throttles under sustained load. Even with Whisper removed from the scheduler, FinBERT batch scoring causes bursts of heat; the laptop's fan makes it more reliable for always-on background work.
ARM64 compatibility: All packages in the stack are ARM64-safe — PyTorch, scikit-learn, transformers, and Whisper all have ARM64 wheels; FinanceDataReader is py3-none-any (pure Python, no compiled extensions).
What's needed to revisit: Spike on Termux + PyTorch + Whisper install on Exynos ARM; test sustained scheduler run with a phone cooling accessory; consider splitting architecture so the phone handles lighter jobs (prices, news) and offloads Whisper to a separate device.
financialdatasets.ai¶
What it is: REST API for US stock fundamentals — income statements, balance sheets, cash flows, SEC filings, insider trades, institutional holdings. 27,000+ tickers, 30+ years of history.
Why interesting: We have zero fundamental signals in the ML feature vector. Adding P/E, revenue growth, earnings surprises etc. could meaningfully improve prediction quality.
How it would fit: Scheduler job (quarterly cadence) fetching fundamentals per ticker → new DB table → new features in build_features().
Pricing: No free tier. Pay-as-you-go at $0.01–$0.10/call — cheap for our ~20-ticker universe fetched quarterly.
Docs: https://docs.financialdatasets.ai/