Skip to content

🪦 Superseded

  • Part 1(当前状态)→ /status/ Done 段
  • Part 2(Phase 1 outstanding)→ superpowers/specs/phase1 + superpowers/plans/phase1
  • Part 3(Phase 2 三 initiative)→ superpowers/specs/phase2 + superpowers/plans/phase2
  • Part 4(文档重组方案)→ 由 2026-05-08-site-ia-restructure-design 取代
  • Part 5(执行 roadmap)→ /status/ Next 段

本文档保留作为历史参考,不再维护。任何状态以 /status/ 为准。


Twilight Drive — Master Plan & Product Launch Spec

Date: 2026-05-07 Status: Comprehensive review & re-plan Author: Qwen Code (with Liang)


Executive Summary

Twilight Drive is an A-share research agent with mandatory citations and deterministic verification. It runs on Hermes (Nous Research's agent framework) and currently operates as 2 profiles on a Mac Mini. The product vision is a hosted SaaS: users pay ¥198/month, scan a WeChat QR, and get a personal stock-research bot — no tokens, no keys, no config.

This document consolidates:

  1. Phase 1 outstanding items (what's left to ship)
  2. Phase 2 plans (3 major initiatives)
  3. Product launch packaging (docs reorganization)
  4. Execution roadmap

Part 1: Current State

What Exists Today

ComponentStatusLocation
Core package (twilight-drive-core 0.1.0)✅ Shipped/Users/syone/twilight-drive/core/
Citation protocol, Verifier, Competence registry✅ Shippedcore/core/
Tushare adapter (get_price, get_fundamentals)✅ Shippedcore/core/data/
Stock-research skill scripts✅ Shippedskill/stock-research/scripts/
Dual-mode client (Direct/Service)✅ Shippedskill/stock-research/scripts/_client.py
CI/CD (ruff + pytest + release)✅ Shipped.github/workflows/
Install script (install.sh)✅ Shippedroot
Documentation site (VitePress → CF Pages)✅ Shippeddocs/dev.fsagent.cc (custom) + twilight-drive.pages.dev; CF GitHub App auto-deploys on push to main
Hermes Main profile (crypto/general)⚠️ Running, exit code -9 (OOM)~/.hermes/
Hermes Stock-Research profile✅ Running, active~/.hermes/profiles/stock-research-agent/
P1 Hosted Service Spec✅ Written, not mergedbranch spec/p1-service-resolved-decisions

What Does NOT Exist Yet

ComponentStatusPriority
Hermes integration adapter❌ Stub onlyP1
P1.0 MVP Backend (FastAPI + DuckDB)❌ Spec onlyP1
Research report ingestion / PDF parsing❌ Deferred to P2P2
L1 Storage layers (Raw/Normalized/Curated/Feature)❌ PlannedP2
Hosted backend (P1 service)❌ Spec onlyP1
Payment + onboarding flow❌ Spec onlyP1.1
Multi-source data layer❌ PlannedP1.2
Web search proxy❌ PlannedP1.3
Profile replication / git-push sync❌ Not designedP2
API key pool / rotation with guardrails❌ Not implementedP2
Centralized data warehouse❌ Not implementedP2

Part 2: Phase 1 Outstanding Items

These are the items from the P1 spec that need to be implemented to ship the MVP paid product.

P1.0 — MVP Backend (Target: 2 weeks from start)

  • [ ] FastAPI service on Vultr with 4 endpoints:
    • GET /price?code=&trade_date= → cite-wrapped price data
    • GET /fundamentals?code=&period= → cite-wrapped fundamentals
    • GET /reports/search → 501 Not Implemented (placeholder)
    • GET /search?query= → web search results (501 until P1.3)
    • GET /healthz → health check
  • [ ] DuckDB on-disk cache at /var/lib/twilight/cache.duckdb
  • [ ] Bearer token auth (SHA256-hashed, per-user)
  • [ ] One funded Tushare Pro account with token in systemd EnvironmentFile
  • [ ] Hermes integration adapter — register Twilight skills as Hermes Skills + Competences
  • [ ] Cite envelope on all responses (existing core/core/citation.py schema)
  • [ ] Skill side zero changes — existing _client.py calls service mode transparently

Success criteria:

  • DuckDB returns 600519 last close in < 50ms after warmup
  • One-command deploy from local
  • 100 sequential same-code requests → 1 Tushare call total (cache works)
  • Verifier passes on responses (cite envelope intact)
  • No bearer token → 401; revoked token → 403

P1.1 — Onboarding + Payment (Target: +2 weeks)

  • [ ] Landing page with 2 pricing cards (Pro ¥198/mo selectable, Lite $49/mo disabled)
  • [ ] WeChat Pay integration (payment webhook)
  • [ ] Provisioner service (separate from FastAPI):
    • Payment webhook → creates Hermes profile → issues invisible API key → returns WeChat QR
  • [ ] Postgres for users/billing (separate from DuckDB warehouse)
  • [ ] Hermes profile provisioning flow:
    • hermes profile create user-{id} --clone-from template-stock-research
    • Plant secrets (TUSHARE_TOKEN, TWILIGHT_API_TOKEN, etc.) into profile's secret store
    • Create systemd unit for new profile's gateway
    • Generate iLink Bot QR

Success criteria:

  • Pay ¥198 → user record created → hermes profile spawned → QR returned
  • User scans QR, names bot, sends query → cited answer in WeChat
  • User never copy-pasted any token, key, or URL

P1.2 — Multi-source Data Layer (Target: +1 week)

  • [ ] pytdx3 (free TDX socket) for historical backfill
  • [ ] akshare for macro, industry classification, alternative ratios
  • [ ] Yahoo Finance for US tickers, ADRs
  • [ ] Cross-source agreement checks (log warning if sources differ > 0.01%)
  • [ ] served_by field in cite envelope reveals source(s)

P1.3 — Web Search Proxy (Target: +3 days)

Note: P1.3 uses DashScope + Google as a temporary solution. Phase 2 Initiative C replaces this with SearXNG (see below).

  • [ ] DashScope built-in web_search as default (free 1000/day for Qwen OAuth)
  • [ ] Google Custom Search as fallback
  • [ ] GET /search?query=&since=&max_results= endpoint
  • [ ] Cite envelope on search results

Part 3: Phase 2 Plans — Three Major Initiatives

Initiative A: Centralized Data Warehouse

Problem

Currently, each Hermes profile fetches data independently (Tushare, akshare, Alpha Vantage, etc.). This means:

  • Duplicate API calls across profiles (wasting quota)
  • No data quality control or cross-source verification
  • No historical data persistence (every query is fresh from upstream)
  • No unified data access layer for downstream consumers

Solution: twilight-data — Centralized Data Hub

┌─────────────────────────────────────────────────────────────┐
│                    twilight-data Hub                        │
│                                                             │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌────────────┐ │
│  │ Tushare  │  │  akshare │  │  pytdx3  │  │  Yahoo/AV  │ │
│  │  (A股)   │  │  (宏观)  │  │  (历史)  │  │  (全球)    │ │
│  └────┬─────┘  └────┬─────┘  └────┬─────┘  └─────┬──────┘ │
│       └──────────────┴──────────────┴──────────────┘        │
│                          ▼                                  │
│              ┌───────────────────────┐                      │
│              │   Data Quality Layer  │                      │
│              │  - Cross-source check │                      │
│              │  - Anomaly detection  │                      │
│              │  - Schema validation  │                      │
│              └───────────┬───────────┘                      │
│                          ▼                                  │
│              ┌───────────────────────┐                      │
│              │    DuckDB Warehouse   │                      │
│              │  Raw → Normalized →   │                      │
│              │  Curated → Feature    │                      │
│              └───────────┬───────────┘                      │
│                          ▼                                  │
│              ┌───────────────────────┐                      │
│              │   Redistribution API  │                      │
│              │   FastAPI (unified)   │                      │
│              └───────────────────────┘                      │
└─────────────────────────────────────────────────────────────┘


          ┌───────────────┼───────────────┐
          ▼               ▼               ▼
    ┌──────────┐  ┌──────────────┐  ┌──────────┐
    │ Hermes #1│  │ Hermes #2..N │  │  External│
    │ (crypto) │  │  (stocks)    │  │  Users   │
    └──────────┘  └──────────────┘  └──────────┘

Key Components

LayerTechnologyPurpose
IngestAPScheduler + adaptersScheduled batch pulls from all sources
QualityGreat Expectations or customSchema validation, cross-source agreement, anomaly detection
StorageDuckDB (4-layer: Raw/Normalized/Curated/Feature)Single source of truth
ServeFastAPI (extends P1.0 backend)Unified API for all downstream consumers
Monitorstructlog + /healthzData freshness, source health, error rates

4-Layer Data Model

Raw         → Exact upstream response, immutable, append-only
Normalized  → Cleaned, typed, deduplicated, unified schema
Curated     → Business logic applied (splits adjusted, corporate actions)
Feature     → Derived metrics (PE bands, momentum, factor scores)

Data Quality Rules

RuleCheckAction
Cross-source agreementTushare close vs pytdx3 close for same (code, date)Log warning if diff > 0.01%
FreshnessLast successful fetch per sourceAlert if > 24h stale
Schema validationAll rows match expected typesReject + quarantine bad rows
Anomaly detectionPrice change > 20% single dayFlag for manual review
CompletenessExpected columns present, no nulls in key fieldsReject incomplete rows

Implementation Plan

StepTaskEstimated Effort
A1Design DuckDB schemas for 4 layers1 day
A2Build Tushare ingest scheduler (daily batch)2 days
A3Build pytdx3 historical backfill (5 years, top 300 stocks)2 days
A4Wire akshare + Yahoo behind same API2 days
A5Implement data quality checks2 days
A6Extend FastAPI with warehouse-read endpoints1 day
A7Add data freshness monitoring1 day

Initiative B: Profile Replication & Ownership

Problem

Hermes profiles live in ~/.hermes/profiles/<name>/. There is no:

  • Version control for profile configs
  • Push/pull sync between machines (Mac Mini ↔ VPS)
  • Ownership model — anyone with filesystem access can modify any profile
  • Template system for cloning new user profiles

Currently, to update all profiles you must manually edit each one, or git push and hope the other side picks it up.

Solution: hermes-profile-manager

┌──────────────────────────────────────────────────┐
│              Profile Registry                     │
│                                                   │
│  ┌──────────┐   ┌──────────┐   ┌───────────────┐ │
│  │ Profile  │   │ Profile  │   │ Profile       │ │
│  │ #1 Main  │   │ #2 Stock │   │ #3 User-abc123│ │
│  │ (crypto) │   │ Research │   │ (paid user)   │ │
│  └────┬─────┘   └────┬─────┘   └───────┬───────┘ │
│       └──────────────┴─────────────────┘          │
│                          ▼                         │
│              ┌───────────────────────┐             │
│              │   Git-backed Storage  │             │
│              │   (bare repo)         │             │
│              └───────────┬───────────┘             │
│                          ▼                         │
│              ┌───────────────────────┐             │
│              │   Ownership System    │             │
│              │   - claim-profile     │             │
│              │   - authz per profile │             │
│              │   - audit trail       │             │
│              └───────────────────────┘             │
└──────────────────────────────────────────────────┘

Key Features

FeatureDescription
Git-backed profilesAll profile configs live in a bare git repo. Changes are committed, versioned, auditable
hermes profile claim <name>User claims ownership of a profile. Records owner identity, prevents unauthorized modifications
hermes profile pushPush local profile changes to the registry (git push under the hood)
hermes profile pullPull latest profile configs from the registry
hermes profile syncSync ALL profiles on a machine to latest
Template systemtemplate-stock-research, template-crypto, etc. — used by provisioner to clone new user profiles
Multi-host syncMac Mini ↔ VPS profile configs stay in sync via git

Ownership Model

Profile: stock-research-agent
  owner: liang (verified via Telegram ID 5838121981)
  created: 2026-04-30
  last_modified: 2026-05-07
  git_remote: origin (bare repo at /var/lib/hermes-profiles/)
  
Profile: user-abc123
  owner: user-abc123 (verified via payment)
  created: 2026-05-15
  template: template-stock-research-pro
  host: vultr-1
  weixin_bot_id: xxx

Implementation Plan

StepTaskEstimated Effort
B1Create bare git repo for profile registry0.5 day
B2Migrate existing profiles into git-backed storage1 day
B3Build hermes profile CLI (claim, push, pull, sync)2 days
B4Implement ownership model (owner identity, authz)1 day
B5Create profile templates (stock-research, crypto)1 day
B6Build multi-host sync mechanism1 day
B7Integrate with provisioner (clone from template)1 day

Initiative C: Token Management — SiliconFlow API Pool with Guardrails

Problem

Both Hermes profiles currently share API keys, causing:

  • HTTP 429 rate limits (profiles competing for same key)
  • No usage tracking or guardrails
  • No automatic fallback when a key is exhausted
  • API keys exposed in config files (not managed centrally)

Solution: LLM API Gateway with Key Pool

Based on open-source research, the recommended solution is One-API (or its actively maintained fork New-API):

ProjectURLWhy
New-APIhttps://github.com/Calcium-Ion/new-apiActive fork of One-API, web UI, channel-based key pooling, auto fallback on 429/5xx, per-channel quota/budget
LiteLLM Proxyhttps://github.com/BerriAI/litellmPython proxy, model_group load balancing, Redis rate limiting, per-key budget tracking
One-API (original)https://github.com/songquanpeng/one-apiGo-based, web UI, channel management, health checks, auto-disable exhausted keys
SearXNGhttps://github.com/searxng/searxngPrivacy-respecting metasearch engine — aggregates 70+ search engines, self-hosted, no API key needed per upstream, replaces DashScope/Google search proxy
┌──────────────────────────────────────────────────────────┐
│                    New-API Gateway                       │
│                    (Docker on Vultr)                     │
│                                                          │
│  LLM Channels:                                           │
│  ┌────────────────────────────────────────────────────┐  │
│  │ Channel 1: SiliconFlow key #1  [active]  45% used │  │
│  │ Channel 2: SiliconFlow key #2  [active]  12% used │  │
│  │ Channel 3: SiliconFlow key #3  [active]  78% used │  │
│  │ Channel 4: SiliconFlow key #N  [disabled] 100%    │  │
│  └────────────────────────────────────────────────────┘  │
│                                                          │
│  Routing: Round-robin + health check                    │
│  Guardrails: RPM/TPM limits, per-channel budget caps     │
│  Fallback: Auto-switch on 429/5xx                       │
│                                                          │
│  Output: OpenAI-compatible endpoint                      │
│  http://localhost:3000/v1                                │
└──────────────────────────────────────────────────────────┘


          ┌───────────────┼───────────────┐
          ▼               ▼               ▼
    ┌──────────┐  ┌──────────────┐  ┌──────────┐
    │ Hermes #1│  │ Hermes #2    │  │ Twilight │
    │ (crypto) │  │ (stocks)     │  │ Backend  │
    └──────────┘  └──────────────┘  └──────────┘


┌──────────────────────────────────────────────────────────┐
│                   SearXNG (Web Search)                   │
│                    (Docker on Vultr)                     │
│                                                          │
│  Upstream engines enabled:                               │
│  ┌────────────────────────────────────────────────────┐  │
│  │ Google  │ Bing  │ DuckDuckGo  │ Baidu  │ Sogou   │  │
│  │ Brave   │ Yep   │ Mojeek      │ Yandex │ 360     │  │
│  └────────────────────────────────────────────────────┘  │
│                                                          │
│  Features:                                               │
│  - No per-engine API keys (uses scraping + public APIs) │
│  - Rate limiting + request caching built-in             │
│  - JSON API for programmatic access                     │
│  - Chinese search engine support (Baidu, Sogou, 360)    │
│  - Privacy: no user tracking, no logging                │
│                                                          │
│  Output: http://localhost:8080/search?q=&format=json    │
└──────────────────────────────────────────────────────────┘


          ┌───────────────┼───────────────┐
          ▼               ▼               ▼
    ┌──────────┐  ┌──────────────┐  ┌──────────┐
    │ Hermes #1│  │ Hermes #2    │  │ Twilight │
    │ (crypto) │  │ (stocks)     │  │ Backend  │
    └──────────┘  └──────────────┘  └──────────┘
AspectDashScope + GoogleSearXNG
API keys needed2 (DashScope + Google CSE)0 (self-hosted, scrapes upstream)
Chinese search qualityGood (DashScope) + limited (Google)Excellent (Baidu + Sogou + 360 built-in)
Cost$5/1000 queries above free tierFree (only server cost)
Rate limitsDashScope 1000/day, Google 100/day freeSelf-managed, no hard cap
PrivacyQueries logged by providersNo logging, privacy-first
CustomizationLimitedFull control over engines, ranking, filters
MaintenanceDepends on upstream availabilitySelf-hosted Docker, one docker-compose up

Key Features (LLM Gateway)

FeatureImplementation
Key poolingAdd each SiliconFlow key as a "channel" in New-API. Requests distributed round-robin
GuardrailsPer-channel RPM/TPM limits, daily budget caps, auto-disable on quota exhaustion
Automatic fallbackChannel returns 429 → New-API retries next healthy channel transparently
Usage dashboardWeb UI shows per-key usage, cost, health status
OpenAI-compatibleHermes profiles point to http://localhost:3000/v1 instead of api.siliconflow.cn
Multi-model routingRoute Qwen3.5-27B to SiliconFlow, Qwen3.6-plus to DashScope, etc.

Key Features (Web Search — SearXNG)

FeatureImplementation
Multi-engine aggregation70+ search engines, enable Baidu/Sogou/360 for Chinese queries
No API key managementSelf-hosted, uses public APIs and scraping — no per-engine keys needed
JSON APIGET /search?q=600519+业绩&format=json&language=zh-CN
Rate limitingBuilt-in limiter config, configurable per-IP limits
CachingBuilt-in Redis-backed cache for repeated queries
PrivacyNo query logging, no user tracking — compliant with data privacy requirements

Guardrails Configuration

GuardrailSettingAction
RPM limit100 requests/min per channelThrottle, queue, or fallback
TPM limit500K tokens/min per channelThrottle, queue, or fallback
Daily budget¥500/day per channelAuto-disable channel when exhausted
Health checkEvery 5 minTest channel with simple request, mark unhealthy on failure
Auto-recoveryEvery 30 minRe-enable disabled channels, test if recovered
SearXNG rate limit10 requests/min per IP (configurable)Return 429, Twilight backend retries with backoff

Implementation Plan

StepTaskEstimated Effort
C1Deploy New-API via Docker on Vultr0.5 day
C2Add all SiliconFlow keys as channels0.5 day
C3Configure guardrails (RPM/TPM/budget)0.5 day
C4Deploy SearXNG via Docker on Vultr0.5 day
C5Configure SearXNG engines (Baidu, Sogou, 360, Google, Bing)0.5 day
C6Update Twilight backend /search to use SearXNG instead of DashScope0.5 day
C7Update Hermes profiles to use New-API endpoint for LLM + SearXNG for search0.5 day
C8Set up monitoring + alerting on channel health1 day
C9Document key rotation procedure0.5 day

Part 4: Product Launch Packaging

Document Reorganization

All documents should be organized into a product launch structure:

twilight-drive/
├── docs/
│   ├── product/                    # Product-facing docs
│   │   ├── README.md               # What is Twilight Drive? (elevator pitch)
│   │   ├── features.md             # Feature list (citations, verifier, multi-source)
│   │   ├── pricing.md              # Pricing tiers (Pro ¥198, Lite $49)
│   │   ├── faq.md                  # Common questions
│   │   └── changelog.md            # Release history
│   │
│   ├── getting-started/            # User onboarding
│   │   ├── quickstart.md           # 5-minute onboarding flow
│   │   ├── wechat-setup.md         # How to pair your bot
│   │   └── first-query.md          # Example: "查 600519 的 P/E"
│   │
│   ├── architecture/               # Technical deep-dive
│   │   ├── overview.md             # 5-layer architecture
│   │   ├── citation-protocol.md    # How citations work
│   │   ├── verifier.md             # Deterministic verification
│   │   ├── data-warehouse.md       # 4-layer data model
│   │   └── adr/                    # Architecture Decision Records
│   │
│   ├── developer/                  # Developer docs
│   │   ├── api-reference.md        # API endpoints
│   │   ├── skill-development.md    # How to write skills
│   │   ├── data-sources.md         # Adding new data sources
│   │   └── contributing.md         # Contribution guide
│   │
│   └── operations/                 # Ops & deployment
│       ├── deployment.md           # Deploy guide (Vultr + Mac Mini)
│       ├── monitoring.md           # Observability setup
│       ├── secrets.md              # Secret management (Linux/macOS)
│       └── troubleshooting.md      # Common issues

├── spec/                           # Specifications (internal)
│   ├── p1-service.md               # P1 hosted service spec
│   ├── phase2-data-warehouse.md    # Phase 2 data warehouse spec
│   ├── phase2-profile-manager.md   # Phase 2 profile replication spec
│   └── phase2-token-gateway.md     # Phase 2 API key pool spec

└── core/                           # Source code (unchanged)

Release Milestones

MilestoneNameTargetDeliverables
v0.1.0Core Package✅ ShippedCitation protocol, verifier, Tushare adapter, skill scripts
v0.2.0MVP BackendTBDFastAPI + DuckDB + bearer auth on Vultr
v0.3.0Paid ProductTBDLanding page + WeChat Pay + provisioning flow
v0.4.0Multi-SourceTBDpytdx3 + akshare + Yahoo + cross-source checks
v0.5.0Data WarehouseTBD4-layer model + scheduled batch pulls
v1.0.0GA ReleaseTBDAll Phase 1 + Phase 2 shipped, open to non-alpha users

Part 5: Execution Roadmap

Phase 1 (Weeks 1-5): Ship the MVP Paid Product

WeekFocusDeliverables
W1P1.0 Backend foundationFastAPI skeleton, DuckDB cache, bearer auth, Tushare adapter wired
W2P1.0 complete + Hermes integrationAll 4 endpoints working, verifier passes, Hermes skill registered
W3P1.1 Payment + onboardingLanding page, WeChat Pay, provisioner service, profile cloning
W4P1.1 testing + P1.2 multi-sourceEnd-to-end onboarding flow, pytdx3 + akshare wired in
W5P1.3 web search + polishSearch proxy, fallback to Google, docs reorganized, alpha launch

Phase 2 (Weeks 6-10): Infrastructure Hardening

WeekFocusDeliverables
W6-7Initiative A: Data Warehouse4-layer DuckDB model, scheduled ingest, data quality checks
W8Initiative B: Profile ReplicationGit-backed profiles, claim/push/pull CLI, templates, multi-host sync
W9Initiative C: Token + Search GatewayNew-API deployed (key pooling), SearXNG deployed (web search), guardrails, all profiles migrated
W10Integration + TestingAll 3 initiatives integrated, load testing, docs updated

Risks & Mitigations

RiskImpactMitigation
Hermes upstream breaking changesHighPin Hermes version, test against new releases before upgrade
WeChat iLink policy changesHighHave Telegram fallback; monitor iLink API status
Tushare rate limits at scaleMediumPhase 1.2 multi-source + Phase 2 warehouse reduce live API dependency
Vultr single point of failureMediumMac Mini as overflow host; Cloudflare Tunnel for public URL
API key pool exhaustionMediumPhase 2 token gateway with guardrails + budget alerts

Appendix: Key File Locations

ResourcePath
Twilight Drive repo/Users/syone/twilight-drive/
Hermes home/Users/syone/.hermes/
Main profile config/Users/syone/.hermes/config.yaml
Stock-research profile/Users/syone/.hermes/profiles/stock-research-agent/
Stock-research SOUL.md/Users/syone/.hermes/profiles/stock-research-agent/SOUL.md
Hermes scripts (macOS)/Users/syone/hermes-scripts-macos/
Hermes scripts (VPS)/Users/syone/hermes-scripts/
Memory index/Users/syone/.qwen/projects/-Users-syone-prd/memory/MEMORY.md

团队内部文档