W5 /quality/* HTTP Endpoints — Design Spec

Date: 2026-05-11 Status: Draft (W5 implementation reference) Owner: liang Sibling docs:

2026-05-08-tushare-integration-design.md §8	AuditLog 3 tables + helper API
2026-05-10-lifespan-integration-spec.md	app.state.audit_log lifespan wiring
2026-05-11-w4-price-route-cutover.md	sibling endpoint refactor (response shape pattern)

---

1. Background

AuditLog (PR #41) defines 3 DuckDB tables — provider_audit_log, provider_freshness, provider_diff_log — with read helpers (freshness_snapshot, recent_audit, recent_diffs). The spec §8 also defines three HTTP endpoints to surface them, but the endpoints are W5 scope; PR #41 ships just the storage + helper layer.

W5 wraps the helpers in FastAPI routes. The user-facing shape:

Endpoint	Source	Pagination
GET /quality/freshness	audit_log.freshness_snapshot()	None (small fixed-size list)
GET /quality/audit	audit_log.recent_audit(since, limit)	since + limit query params
GET /quality/diff	audit_log.recent_diffs(severity, since, limit)	+ severity filter

---

2. Goals

1. Operator observability — single HTTP surface to see provider health, recent activity, and cross-source disagreements.
2. Internal-plan auth — same require_bearer as /whoami, but restricted to plan == "internal" (not paid customers).
3. Boring shapes — each endpoint returns a JSON array of objects matching the AuditLog helper's return value, with timestamps serialized as ISO-8601 UTC.
4. No new computation — endpoints are thin adapters over existing helpers. Filter logic stays in AuditLog.

2a. Non-goals

• Real-time streaming (websockets / SSE). The data is already inert history; on-demand fetch is fine.
• Pagination beyond limit + since cursor. List sizes are bounded (small N of providers; recent N of calls).
• Rate limiting. Single-tenant operator endpoints; no public abuse surface.
• Mutation endpoints (e.g., DELETE /quality/audit?older_than=). AuditLog cleanup is a separate concern (W2 alt: retention helper).
• Per-provider drill-down endpoints. The freshness_snapshot view already includes all providers; clients filter client-side.

---

3. Endpoint specs

3.1 GET /quality/freshness

Returns one row per registered provider with its latest state.

Auth: Depends(require_bearer) + principal.plan == "internal" (403 if not).

Query params: none.

Response (200):

[
  {
    "provider_id": "tushare-daily",
    "last_success": "2026-05-11T07:38:23Z",
    "last_failure": null,
    "error_msg": null,
    "rows_today": 142,
    "updated_at": "2026-05-11T07:38:23Z"
  },
  {
    "provider_id": "akshare-spot",
    "last_success": "2026-05-11T06:50:00Z",
    "last_failure": "2026-05-11T07:30:00Z",
    "error_msg": "upstream timeout",
    "rows_today": 18,
    "updated_at": "2026-05-11T07:30:00Z"
  }
]

Timestamps: ISO-8601 UTC with trailing Z. NULL last_success / last_failure serialize as null.

Empty list when no providers have run get_quote yet (cold start).

Error (403): {"detail": "quality endpoints require internal plan"}

3.2 GET /quality/audit

Returns recent Composer get_quote calls, most-recent first.

Auth: same as freshness.

Query params:

Param	Type	Default	Notes
since	ISO-8601 datetime	None (no lower bound)	e.g. 2026-05-11T00:00:00Z
limit	int	100	max 1000

Response (200):

[
  {
    "id": 1284,
    "ts": "2026-05-11T07:38:23Z",
    "code": "600519.SH",
    "trade_date": null,
    "intent": "latest",
    "market_state": "open",
    "chain": "akshare-spot,warehouse,tushare-daily",
    "served_by": "akshare-spot",
    "freshness_secs": 15,
    "latency_ms": 42,
    "error": null
  },
  ...
]

Empty list when no calls yet.

Error (400): if limit > 1000 → {"detail": "limit max 1000"}.

3.3 GET /quality/diff

Returns recent cross-source disagreements, most-recent first.

Auth: same.

Query params:

Param	Type	Default	Notes
severity	str	None (all)	one of info, warning, critical
since	ISO-8601 datetime	None
limit	int	100	max 1000

Response (200):

[
  {
    "id": 17,
    "ts": "2026-05-11T03:00:42Z",
    "code": "600519.SH",
    "trade_date": "2026-05-08",
    "provider_a": "tushare-daily",
    "value_a": 1700.0,
    "provider_b": "warehouse",
    "value_b": 1695.5,
    "diff_pct": 0.2655,
    "severity": "info"
  },
  ...
]

Empty list when no diffs recorded (typical until TDX cross-source auditor lands per spec 2026-05-10-tdx-cross-source-validation.md).

Error (400):

• invalid severity → {"detail": "severity must be one of info/warning/critical"}
• limit > 1000 → as above

---

4. Implementation skeleton

src/service/routes/quality.py (new file):

from __future__ import annotations

from datetime import datetime
from typing import Annotated, Any

from fastapi import APIRouter, Depends, HTTPException, Request, status

from service.auth import Principal, require_bearer

router = APIRouter()

_MAX_LIMIT = 1000


def _require_internal(
    principal: Annotated[Principal, Depends(require_bearer)],
) -> Principal:
    if principal.plan != "internal":
        raise HTTPException(
            status_code=status.HTTP_403_FORBIDDEN,
            detail="quality endpoints require internal plan",
        )
    return principal


def _audit_log(request: Request):
    return request.app.state.audit_log


@router.get("/quality/freshness")
def get_freshness(
    request: Request,
    _principal: Annotated[Principal, Depends(_require_internal)],
) -> list[dict[str, Any]]:
    rows = _audit_log(request).freshness_snapshot()
    return [_to_iso(r) for r in rows]


@router.get("/quality/audit")
def get_audit(
    request: Request,
    _principal: Annotated[Principal, Depends(_require_internal)],
    since: datetime | None = None,
    limit: int = 100,
) -> list[dict[str, Any]]:
    if limit > _MAX_LIMIT:
        raise HTTPException(400, f"limit max {_MAX_LIMIT}")
    rows = _audit_log(request).recent_audit(since=since, limit=limit)
    return [_to_iso(r) for r in rows]


@router.get("/quality/diff")
def get_diff(
    request: Request,
    _principal: Annotated[Principal, Depends(_require_internal)],
    severity: str | None = None,
    since: datetime | None = None,
    limit: int = 100,
) -> list[dict[str, Any]]:
    if limit > _MAX_LIMIT:
        raise HTTPException(400, f"limit max {_MAX_LIMIT}")
    if severity not in (None, "info", "warning", "critical"):
        raise HTTPException(
            400, "severity must be one of info/warning/critical"
        )
    try:
        rows = _audit_log(request).recent_diffs(
            severity=severity, since=since, limit=limit
        )
    except ValueError as exc:
        # AuditLog's own severity validation (defense in depth)
        raise HTTPException(400, str(exc)) from exc
    return [_to_iso(r) for r in rows]


def _to_iso(row: dict[str, Any]) -> dict[str, Any]:
    """Serialize naive UTC datetimes to ISO-8601 with trailing Z."""
    out: dict[str, Any] = {}
    for key, value in row.items():
        if isinstance(value, datetime):
            # AuditLog stores naive UTC.
            out[key] = value.isoformat() + "Z"
        else:
            out[key] = value
    return out

4.1 Route registration

In src/service/main.py (per 2026-05-10-lifespan-integration-spec.md §4):

from service.routes import quality as quality_routes
...
app.include_router(quality_routes.router)

---

5. Test plan

core/tests/integration/test_service_quality_route.py (new):

Test	What it verifies
test_freshness_requires_internal_plan	non-internal bearer → 403
test_freshness_empty_initially	new audit_log → []
test_freshness_after_provider_calls	seed via AuditLog.update_freshness, expect 1 row
test_audit_empty_initially	[]
test_audit_default_limit	seed 5 rows, GET returns 5
test_audit_with_limit	seed 10, ?limit=3 returns 3
test_audit_with_since	seed old + recent, ?since=cutoff filters
test_audit_limit_exceeds_max_raises_400	?limit=1001 → 400
test_diff_empty_initially	[]
test_diff_severity_filter	seed info/warning/critical; ?severity=warning returns 1
test_diff_invalid_severity_raises_400	?severity=catastrophic → 400
test_diff_since_and_severity_combined	both filters work together
test_iso_datetime_format	response timestamps end with Z

Total: 13 tests. Use FastAPI TestClient + fixture that injects an in-memory AuditLog into app.state.

---

6. Auth approach

Principal.plan is set by require_bearer from the bearer DB. Per v0.2.0 baseline, plans are: internal (Liang), future paid tiers.

W5 keeps /quality/* restricted to internal to:

• Avoid leaking provider names, timing data, error messages to paying customers.
• Prevent abuse (an external user spamming /quality/audit could read another user's get_quote history if multi-tenant ever lands).

Implementation: _require_internal decorator (§4 code) returns 403 if plan mismatch.

---

7. Dependencies

W5 requires merged:

AuditLog (PR #41)	the data layer
Lifespan injection (T2.7)	app.state.audit_log
Composer auto-syncs freshness (#43)	so freshness_snapshot returns real data
Composer audit hook (#42)	so recent_audit returns real data
AuditLog webhook (#46)	optional; doesn't affect /quality/* shape but tested together

recent_diffs returns empty until TDX cross-source auditor lands (separate W7+ scope). W5 ships endpoint anyway; just empty list until W7+ writes diffs.

---

8. Open questions

1. Should /quality/audit filter by code or severity (for the audit's error field)? Add query params later if operators ask. v0.3.0 ship without.

2. OpenAPI tags. Use tags=["quality"] on the router for FastAPI's auto-doc grouping.

3. Pagination cursor (id-based)? v0.3.0 uses since (timestamp). For high volume later, switch to cursor; but limit + since is enough now.

4. CORS? No — operator browser tools or scripts use bearer auth; cross-origin not relevant.

5. Caching? No — freshness data changes per-request; audit/diff are append-only with since filter (cache invalidation harder than benefit).

---

9. Acceptance criteria for W5 PR

When the cron / user writes W5:

• [ ] src/service/routes/quality.py with 3 GET endpoints
• [ ] _require_internal dependency enforces plan check
• [ ] Routes wired in main.py via include_router
• [ ] 13 integration tests pass (per §5)
• [ ] /healthz, /whoami, /price, /fundamentals, /warehouse/* routes still work unchanged
• [ ] Timestamps serialize as ISO-8601 with Z
• [ ] Empty list on no-data instead of 404

---

10. Cross-references

• AuditLog tables + helpers: 2026-05-08-tushare-integration-design.md §8
• Lifespan wiring app.state.audit_log: 2026-05-10-lifespan-integration-spec.md §4 Phase 4 + §7
• Sibling endpoint refactor pattern: 2026-05-11-w4-price-route-cutover.md §5.1 (route → composer delegation)
• TDX cross-source auditor (produces diff entries): 2026-05-10-tdx-cross-source-validation.md §5
• Plan slot: 2026-05-08-deployment-toolset-rollout.md §7 W5