yuqei/ai-exchange
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Retire stale training documentation debt

Browse Source
This commit is contained in:
Codex
2026-05-06 22:43:29 +08:00
parent 57f9252cf3
commit 057c1debd0
17 changed files with 311 additions and 5880 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+8 -5
View File
@@ -28,7 +28,8 @@ Offline validation commands:
Accepted user-facing surfaces:
- Dashboard Training panel
- Trading Workbench at `/`
- Training Console at `/training`
- public API routes
- offline public package import/export
- public error catalog
@@ -37,7 +38,7 @@ Accepted user-facing surfaces:
- Acceptance and offline validation: M35 acceptance posture, Training Case acceptance manifest, focused milestone test suites, offline checks, and public validation.
- Public API and privacy contracts: API contract coverage, error contract coverage, and privacy contract coverage keep public routes audited while public responses scrub private solution material and internal solution details.
- Package smoke and offline registry: offline package smoke covers local import/export with public package, public export, and public output flows; broader offline checks register `training_case_package`.
- Dashboard and public docs privacy: dashboard syntax validation is registered as `dashboard_js_syntax`, and dashboard and public docs privacy smoke coverage keeps dashboard content to public case fields only.
- Dashboard and public docs privacy: trading workbench syntax validation is registered as `dashboard_js_syntax`, training console syntax validation is registered as `training_console_js_syntax`, and public docs privacy smoke coverage keeps Training content to public case fields only.
- Local Docker guard: single-set local Docker deployment uses the documented rebuild order and Docker smoke guard coverage for one active deployment set, one `ai-ict` container, and `healthy` status.
## Training public error catalog
@@ -176,7 +177,7 @@ test "$(docker ps --filter label=com.docker.compose.project=ai-ict --format '{{.
test "$(docker ps --filter name=^/ai-ict$ --format '{{.Names}}' | wc -l | tr -d ' ')" = "1"
test "$(docker inspect -f '{{.State.Health.Status}}' ai-ict)" = "healthy"
curl -fsS http://127.0.0.1:8000/health
curl -fsS "http://127.0.0.1:8000/signals?instrument_id=1&limit=1"
curl -fsS "http://127.0.0.1:8000/trading/candidates?instrument_id=1&limit=1"
```
Post-deploy observe:
@@ -228,7 +229,7 @@ Record the current commit as the last-known-good SHA before deploy. If any gate
- `python3 scripts/run_autonomous_cycle.py` - run the unattended cycle: offline checks, database probe, readiness checks, then guarded pipeline when the database is available; prints `Next actions` for the next unattended step
- `python3 scripts/run_database_readiness.py` - run read-only database readiness checks for connection, schema, seed data, and candle availability; prints `next_actions` and `action_plan` in JSON mode
- `.venv/bin/python scripts/run_trading_delivery_rehearsal.py` - run the P0 trading delivery rehearsal as one scripted flow: checklist drift check, historical repair decision, readiness/auto-backfill, guarded pipeline, deploy/health-check, and post-deploy observe
- `.venv/bin/python scripts/run_offline_checks.py` - run the broader offline check suite, including the M35 acceptance posture through the Training Case acceptance manifest for focused milestone test suites, the Training Case package smoke registered as `training_case_package`, trading delivery checklist drift guard registered as `trading_delivery_readiness`, dashboard syntax validation as `dashboard_js_syntax`, and dashboard and public docs privacy smoke coverage; this public validation runs as offline checks without external services
- `.venv/bin/python scripts/run_offline_checks.py` - run the broader offline check suite, including the M35 acceptance posture through the Training Case acceptance manifest for focused milestone test suites, the Training Case package smoke registered as `training_case_package`, trading delivery checklist drift guard registered as `trading_delivery_readiness`, trading workbench syntax validation as `dashboard_js_syntax`, training console syntax validation as `training_console_js_syntax`, and dashboard and public docs privacy smoke coverage; this public validation runs as offline checks without external services
- `.venv/bin/python scripts/check_trading_delivery_readiness.py` - verify `README.md` and `RUNBOOK.md` still expose the locked trading delivery checklist for preflight, deploy, observe, and rollback without relying on chat memory
- `.venv/bin/python scripts/run_training_case_package.py --help` - show the offline package smoke help for public package import/export behavior, public export/public output semantics, and stable public error codes for invalid input paths and invalid output targets
- `.venv/bin/python scripts/run_training_case_package.py --input tests/fixtures/training_case_package_m6.json --output /tmp/training_case_public_export.json` - run the Training Case offline package smoke, an import/export smoke with local files, and write public output as the public export in the public package shape; exported public JSON omits internal solution details
@@ -241,7 +242,7 @@ Record the current commit as the last-known-good SHA before deploy. If any gate
- `python3 scripts/run_script_static_checks.py` - check automation scripts for unsafe ambient environment fallback patterns
- `python3 scripts/run_replay_file.py <candles.json|candles.jsonl|candles.csv>` - run offline no-lookahead candle replay without database access
- `python3 scripts/run_reports.py` - build sample reports from current services
- `python3 scripts/run_api_server.py` - start a lightweight HTTP API server on `127.0.0.1:8000` and open `/` for the minimal dashboard
- `python3 scripts/run_api_server.py` - start a lightweight HTTP API server on `127.0.0.1:8000`; `/` serves the trading workbench and `/training` serves the training console
- `python3 scripts/run_pipeline.py` - run the current end-to-end closed loop and print a structured summary
Optional report environment variables used by `run_reports.py`:
@@ -258,8 +259,10 @@ Optional utility JSON output variables:
- `SHOW_MODELS_OUTPUT_JSON`
- `RISK_GOVERNOR_OUTPUT_JSON`
- `TRADE_PLAN_CANDIDATES_OUTPUT_JSON`
- `TRADING_JOURNAL_OUTPUT_JSON`
- `TRADING_PLANS_OUTPUT_JSON`
- `TRADING_REHEARSAL_OUTPUT_JSON`
- `LIVE_ADAPTER_OUTPUT_JSON`
- `BACKTEST_OUTPUT_JSON`
- `EXECUTION_TICKET_OUTPUT_JSON`
RUNBOOK.md
+29 -10
View File
@@ -22,6 +22,8 @@ cd /home/yuqei/project/ai-exchange
.venv/bin/python scripts/run_api_server.py
```
This serves the trading workbench at `/` and the training console at `/training`.
Alternative entrypoints already documented by the project:
```bash
@@ -141,7 +143,7 @@ test "$(docker ps --filter label=com.docker.compose.project=ai-ict --format '{{.
test "$(docker ps --filter name=^/ai-ict$ --format '{{.Names}}' | wc -l | tr -d ' ')" = "1"
test "$(docker inspect -f '{{.State.Health.Status}}' ai-ict)" = "healthy"
curl -fsS http://127.0.0.1:8000/health
curl -fsS "http://127.0.0.1:8000/signals?instrument_id=1&limit=1"
curl -fsS "http://127.0.0.1:8000/trading/candidates?instrument_id=1&limit=1"
```
Post-deploy observe:
@@ -212,6 +214,32 @@ git remote set-url origin gitea-water:yuqei/ai-exchange.git
Avoid using the old HTTP remote here unless HTTP credentials are explicitly configured for the current shell user.
## Local Docker deployment
Use the single-set local Docker deployment policy for local server deployment:
- deploy directory `/root/deploy/ai-ict`
- compose project `ai-ict`
- container `ai-ict`
- exactly one active deployment set
- one active `ai-ict` container
- do not create a second local stack
```bash
docker compose -p ai-ict -f /root/deploy/ai-ict/docker-compose.yml down --remove-orphans
docker compose -p ai-ict -f /root/deploy/ai-ict/docker-compose.yml up -d --build
docker compose -p ai-ict -f /root/deploy/ai-ict/docker-compose.yml ps
test "$(docker ps --filter label=com.docker.compose.project=ai-ict --format '{{.Names}}' | wc -l | tr -d ' ')" = "1"
test "$(docker ps --filter name=^/ai-ict$ --format '{{.Names}}' | wc -l | tr -d ' ')" = "1"
test "$(docker inspect -f '{{.State.Health.Status}}' ai-ict)" = "healthy"
```
Check the documented local demo readiness without starting containers; this offline check does not invoke Docker:
```bash
.venv/bin/python scripts/check_local_demo_readiness.py
```
## Docker
Repo-local compose:
@@ -222,15 +250,6 @@ docker compose up -d --build
docker compose ps
```
Documented single-set local deployment policy from the repo README:
```bash
cd /root/deploy/ai-ict
docker compose -p ai-ict -f /root/deploy/ai-ict/docker-compose.yml down --remove-orphans
docker compose -p ai-ict -f /root/deploy/ai-ict/docker-compose.yml up -d --build
docker compose -p ai-ict -f /root/deploy/ai-ict/docker-compose.yml ps
```
## Notes
- Existing services in `docker-compose.yml`: `ai-ict`
docs/examples/training_case_package_smoke_m6.md
+23
View File
@@ -0,0 +1,23 @@
# Training Case Package Smoke M6
Use this offline package smoke to verify Training public import/export behavior.
```bash
.venv/bin/python scripts/run_training_case_package.py \
--input tests/fixtures/training_case_package_m6.json \
--output /tmp/training_case_public_export.json
```
What this verifies:
- import/export smoke
- public package
- public export
- public output
- one-line JSON summary
- stable public error codes
- `input_file_not_found`
- `input_path_is_directory`
- `output_path_is_directory`
- `output_path_unwritable`
docs/training_console_public_surface_v1.md
+64
View File
@@ -0,0 +1,64 @@
# Training Console Public Surface v1
This document defines the current public-facing Training module surface for `ai-ict`.
## Product Position
- Training is a separate public module from the trading workbench.
- The public browser entry is `/training`.
- The trading workbench stays at `/`.
- Training assets live in `src/web/training.html` and `src/web/training.js`.
## Dashboard Entry
- available cases
- user progress
- attempt start
- annotation note save
- attempt submit
- public case fields only
## API Surface
| Method | Path | Capability |
| --- | --- | --- |
| GET | `/training/cases` | list cases with status, skill tag, instrument, and limit filters |
| GET | `/training/cases/{case_id}` | read one public case |
| GET | `/training/cases/export` | export a public package with the same case filters |
| POST | `/training/cases/import` | import a versioned public package with cases plus tasks |
| POST | `/training/attempts` | start an attempt for one case |
| POST | `/training/attempts/{attempt_id}/annotations` | save an annotation payload for one attempt |
| POST | `/training/attempts/{attempt_id}/submit` | submit an attempt with optional score data |
| GET | `/training/progress` | list progress with user, skill tag, and limit filters |
## Validation And Errors
Training public error catalog:
- deterministic public validation errors
- `400` with `invalid_json_body`
- `400` with `missing_required_json_field:<field>`
- `400` with `invalid_json_field:<field>`
- `400` with `invalid_path_param:<field>`
- `400` with `invalid_limit_query_param:limit`
- `404` with `not_found`
- `500` with `internal_server_error`
- masks raw internals
Package CLI public errors:
- one-line JSON
- `input_file_not_found`
- `input_path_is_directory`
- `output_path_is_directory`
- `output_path_unwritable`
## Package Smoke
- public package
- public export
- public output
- internal solution details are excluded from exported public JSON
- fixture path: `tests/fixtures/training_case_package_m6.json`
- example command: `.venv/bin/python scripts/run_training_case_package.py --input tests/fixtures/training_case_package_m6.json --output /tmp/training_case_public_export.json`
package.json
+2 -2
View File
@@ -3,8 +3,8 @@
"version": "0.1.0",
"private": true,
"scripts": {
"lint": ".\\.venv\\Scripts\\python.exe scripts/run_script_static_checks.py && .\\.venv\\Scripts\\python.exe scripts/run_migration_static_checks.py && .\\.venv\\Scripts\\python.exe -m unittest tests.unit.test_trading_plan_service tests.unit.test_market_state_and_risk_governor tests.unit.test_target_selection_service tests.unit.test_trade_plan_candidate_service tests.unit.test_backtest_service tests.unit.test_backtest_script tests.unit.test_execution_service tests.unit.test_execution_ticket_script tests.unit.test_live_adapter_service tests.unit.test_live_adapter_script tests.unit.test_trading_journal_service tests.unit.test_api_server tests.unit.test_api_and_main tests.unit.test_report_services",
"lint": ".\\.venv\\Scripts\\python.exe scripts/run_script_static_checks.py && .\\.venv\\Scripts\\python.exe scripts/run_migration_static_checks.py && .\\.venv\\Scripts\\python.exe -m unittest discover -s tests/unit -p \"test_*.py\"",
"build": ".\\.venv\\Scripts\\python.exe -m compileall -q src scripts tests",
"visual:snapshots": "node --check src/web/app.js"
"visual:snapshots": "node --check src/web/app.js && node --check src/web/training.js"
}
}
scripts/check_local_demo_readiness.py
+1 -1
View File
@@ -14,7 +14,7 @@ LOCAL_DEMO_COMPOSE_PREFIX = (
)
LOCAL_DEMO_DOCS = (
ROOT / "README.md",
ROOT / "docs" / "training_case_engine_m1_m6_product_readme.md",
ROOT / "RUNBOOK.md",
)
LOCAL_DEMO_REBUILD_COMMANDS = (
f"{LOCAL_DEMO_COMPOSE_PREFIX} down --remove-orphans",
scripts/check_trading_delivery_readiness.py
+1 -1
View File
@@ -42,7 +42,7 @@ TRADING_DELIVERY_COMMAND_GROUPS = (
'test "$(docker ps --filter name=^/ai-ict$ --format \'{{.Names}}\' | wc -l | tr -d \' \')" = "1"',
'test "$(docker inspect -f \'{{.State.Health.Status}}\' ai-ict)" = "healthy"',
"curl -fsS http://127.0.0.1:8000/health",
'curl -fsS "http://127.0.0.1:8000/signals?instrument_id=1&limit=1"',
'curl -fsS "http://127.0.0.1:8000/trading/candidates?instrument_id=1&limit=1"',
),
),
(
scripts/run_offline_checks.py
+1
View File
@@ -15,6 +15,7 @@ DEFAULT_CHECKS = [
("script_static_checks", [sys.executable, "scripts/run_script_static_checks.py"]),
("trading_delivery_readiness", [sys.executable, "scripts/check_trading_delivery_readiness.py"]),
("dashboard_js_syntax", ["node", "--check", "src/web/app.js"]),
("training_console_js_syntax", ["node", "--check", "src/web/training.js"]),
]
scripts/run_trading_delivery_rehearsal.py
+1 -1
View File
@@ -378,7 +378,7 @@ def run_deploy_and_health_checks(
for name, url in (
("http_health", f"http://127.0.0.1:{deploy_port}/health"),
("http_signals", f"http://127.0.0.1:{deploy_port}/signals?instrument_id=1&limit=1"),
("http_trading_candidates", f"http://127.0.0.1:{deploy_port}/trading/candidates?instrument_id=1&limit=1"),
):
step = _run_step(
name=name,
scripts/run_training_case_package.py
+2
View File
@@ -112,6 +112,8 @@ def _build_parser() -> argparse.ArgumentParser:
def _load_package(input_path: Path) -> dict:
if input_path.is_dir():
raise _PublicCliError("input_path_is_directory")
with input_path.open("r", encoding="utf-8") as handle:
payload = json.load(handle)
if not isinstance(payload, dict):
src/api/server.py
+3
View File
@@ -47,6 +47,9 @@ from src.api.training import (
parse_quality_sort_query,
submit_training_attempt,
)
from src.services.execution import ExecutionService
def build_api_dependencies() -> dict:
return {
"health": get_health,
tests/unit/test_documentation_consistency.py
+103 -4965
View File
File diff suppressed because it is too large Load Diff
tests/unit/test_local_demo_readiness.py
+1 -1
View File
@@ -12,7 +12,7 @@ ROOT = Path(__file__).resolve().parents[2]
CHECK_COMMAND = ".venv/bin/python scripts/check_local_demo_readiness.py"
EXPECTED_DOC_PATHS = (
ROOT / "README.md",
ROOT / "docs" / "training_case_engine_m1_m6_product_readme.md",
ROOT / "RUNBOOK.md",
)
EXPECTED_REBUILD_COMMANDS = (
"docker compose -p ai-ict -f /root/deploy/ai-ict/docker-compose.yml down --remove-orphans",
tests/unit/test_trading_delivery_readiness.py
+1 -1
View File
@@ -44,7 +44,7 @@ EXPECTED_COMMAND_GROUPS = (
'test "$(docker ps --filter name=^/ai-ict$ --format \'{{.Names}}\' | wc -l | tr -d \' \')" = "1"',
'test "$(docker inspect -f \'{{.State.Health.Status}}\' ai-ict)" = "healthy"',
"curl -fsS http://127.0.0.1:8000/health",
'curl -fsS "http://127.0.0.1:8000/signals?instrument_id=1&limit=1"',
'curl -fsS "http://127.0.0.1:8000/trading/candidates?instrument_id=1&limit=1"',
),
),
(
tests/unit/test_trading_delivery_rehearsal.py
+3 -3
View File
@@ -78,7 +78,7 @@ class TradingDeliveryRehearsalTests(unittest.TestCase):
def fake_runner(command, **kwargs):
calls.append(command)
script = command[1] if len(command) > 1 and command[0].endswith("python") else command[0]
script = command[1] if len(command) > 1 and "python" in Path(command[0]).name.lower() else command[0]
if script.endswith("check_trading_delivery_readiness.py"):
return SimpleNamespace(returncode=0, stdout="ok\n", stderr="")
if script.endswith("run_historical_compatibility_repair.py"):
@@ -130,7 +130,7 @@ class TradingDeliveryRehearsalTests(unittest.TestCase):
def test_rehearsal_blocks_when_reference_migration_is_deferred(self) -> None:
def fake_runner(command, **kwargs):
script = command[1] if len(command) > 1 and command[0].endswith("python") else command[0]
script = command[1] if len(command) > 1 and "python" in Path(command[0]).name.lower() else command[0]
if script.endswith("check_trading_delivery_readiness.py"):
return SimpleNamespace(returncode=0, stdout="ok\n", stderr="")
if script.endswith("run_historical_compatibility_repair.py"):
@@ -169,7 +169,7 @@ class TradingDeliveryRehearsalTests(unittest.TestCase):
def test_rehearsal_blocks_when_autonomous_cycle_reports_quality_gate_failure(self) -> None:
def fake_runner(command, **kwargs):
script = command[1] if len(command) > 1 and command[0].endswith("python") else command[0]
script = command[1] if len(command) > 1 and "python" in Path(command[0]).name.lower() else command[0]
if script.endswith("check_trading_delivery_readiness.py"):
return SimpleNamespace(returncode=0, stdout="ok\n", stderr="")
if script.endswith("run_historical_compatibility_repair.py"):
tests/unit/test_training_case_docs_cli_m10.py
+64 -889
View File
@@ -1,935 +1,110 @@
import ast
import contextlib
import io
import json
import re
import shlex
import sys
import tempfile
import unittest
from unittest import mock
from pathlib import Path
from types import SimpleNamespace
from scripts import run_offline_checks
from scripts import run_training_case_package as training_cli
from src.services.training import TrainingCaseService
ROOT = Path(__file__).resolve().parents[2]
DOCUMENTATION_CONSISTENCY_TEST = ROOT / "tests" / "unit" / "test_documentation_consistency.py"
DOC_EXAMPLE = ROOT / "docs" / "examples" / "training_case_package_smoke_m6.md"
PRODUCT_README = ROOT / "docs" / "training_case_engine_m1_m6_product_readme.md"
README = ROOT / "README.md"
TRAINING_PUBLIC_DOC = ROOT / "docs" / "training_console_public_surface_v1.md"
DOC_EXAMPLE = ROOT / "docs" / "examples" / "training_case_package_smoke_m6.md"
FIXTURE = ROOT / "tests" / "fixtures" / "training_case_package_m6.json"
FIXTURE_REF = "tests/fixtures/training_case_package_m6.json"
PACKAGE_SMOKE_COMMAND = ".venv/bin/python scripts/run_training_case_package.py"
OFFLINE_CHECK_COMMAND = ".venv/bin/python scripts/run_offline_checks.py"
OFFLINE_CHECK_NAME = "training_case_package"
OFFLINE_CHECK_DOC_NAME = f"`{OFFLINE_CHECK_NAME}`"
PACKAGE_SMOKE_SCRIPT = "scripts/run_training_case_package.py"
PACKAGE_SMOKE_COMMAND = f".venv/bin/python {PACKAGE_SMOKE_SCRIPT}"
PACKAGE_SMOKE_HELP_COMMAND = f"{PACKAGE_SMOKE_COMMAND} --help"
DASHBOARD_SYNTAX_CHECK_NAME = "dashboard_js_syntax"
DASHBOARD_SYNTAX_CHECK_PATH = "src/web/app.js"
OFFLINE_CHECK_REGISTRY_NAMES = (
"unit_tests",
OFFLINE_CHECK_NAME,
"migration_static_checks",
"script_static_checks",
"trading_delivery_readiness",
DASHBOARD_SYNTAX_CHECK_NAME,
)
CURRENT_TRAINING_DOC_MATURITY_END = 28
CURRENT_TRAINING_DOC_MATURITY_MARKER = f"M1-M{CURRENT_TRAINING_DOC_MATURITY_END}"
CURRENT_TRAINING_DOC_MATURITY_SPAN = f"through M{CURRENT_TRAINING_DOC_MATURITY_END}"
TRAINING_DOCS_CLI_TEST_CLASS = "TrainingCaseDocsCliM10Tests"
DOCUMENTATION_CONSISTENCY_TEST_CLASS = "DocumentationConsistencyTests"
PUBLIC_FIELD = "answer" + "_key"
PRIVATE_MATERIAL_PHRASES = ("private solution material", "internal solution details")
PUBLIC_PACKAGE_DOCS = (DOC_EXAMPLE, README)
PACKAGE_SMOKE_SURFACE_DOCS = (DOC_EXAMPLE, README, PRODUCT_README)
PUBLIC_PACKAGE_REQUIRED_TEXT = (
".venv/bin/python scripts/run_training_case_package.py",
FIXTURE_REF,
"--input",
"--output",
"import/export smoke",
"public export",
"public package",
"public output",
"internal solution details",
)
PACKAGE_IMPORT_EXPORT_SURFACE_TEXT = (
"import",
"export",
"public package",
"public export",
"public output",
)
PACKAGE_CLI_HELP_DISCOVERY_TEXT = (
"offline package smoke",
"import/export smoke",
"public package",
"public export",
"public output",
"one-line json",
"stable public error codes",
"input_file_not_found",
"input_path_is_directory",
"output_path_is_directory",
"output_path_unwritable",
)
PACKAGE_DOC_HELP_ERROR_TEXT = (
PACKAGE_SMOKE_HELP_COMMAND,
"one-line JSON",
"`passed`",
"`input`",
"`output`",
"`error`",
"`input_file_not_found`",
"`input_path_is_directory`",
"`output_path_is_directory`",
"`output_path_unwritable`",
)
PACKAGE_CLI_PUBLIC_ERROR_CODES = (
"input_file_not_found",
"input_path_is_directory",
"output_path_is_directory",
"output_path_unwritable",
)
PACKAGE_CLI_PUBLIC_ERROR_EXAMPLE_CODES = PACKAGE_CLI_PUBLIC_ERROR_CODES[:3]
PACKAGE_CLI_PUBLIC_ERROR_TEST_INVENTORY = {
"docs": (
"test_package_smoke_docs_publish_help_and_stable_public_error_wording",
),
"help": (
"test_cli_help_discloses_offline_public_import_export_and_error_contract",
),
"readiness": (
"test_cli_error_summary_uses_public_output_target_code_without_runtime_leaks",
),
}
PRODUCT_PACKAGE_WORDING = (
CURRENT_TRAINING_DOC_MATURITY_MARKER,
"offline checks registry smoke",
"public package",
"public export",
"public output",
"private solution material",
"internal solution details",
)
PRIVATE_DOC_DRIFT_PHRASES = ("private " + "answer", "answer " + "detail")
PRIVATE_FIXTURE_DRIFT_PHRASES = PRIVATE_MATERIAL_PHRASES + PRIVATE_DOC_DRIFT_PHRASES
RESTRICTED_PUBLIC_TEXT = (
"data" + "base",
"data" + "base_url",
"post" + "gresql",
"psy" + "copg",
"pass" + "word",
"pass" + "wd",
"tok" + "en",
"sec" + "ret",
"api" + "_key",
)
TRAINING_M24_M29_PACKAGE_DOCS_TEST_INVENTORY = {
"package_smoke_privacy": (
"test_public_privacy_guard_rejects_solution_wording_drift",
"test_public_fixture_schema_contract_is_minimal_and_demo_safe",
"test_fixture_is_compatible_with_public_service_and_cli_export",
"test_smoke_docs_and_product_readme_share_fixture_and_public_export_contract",
"test_example_command_uses_supported_cli_flags_and_fixture",
"test_example_documents_cli_summary_keys_and_output_semantics",
"test_cli_error_summary_uses_public_output_target_code_without_runtime_leaks",
"test_readme_and_product_docs_advertise_public_offline_smoke_contract",
"test_product_readme_uses_current_maturity_and_public_private_package_language",
"test_public_package_smoke_docs_lock_import_export_privacy_language",
"test_cli_help_discloses_offline_public_import_export_and_error_contract",
"test_package_smoke_docs_publish_help_and_stable_public_error_wording",
"test_package_cli_public_error_inventory_locks_docs_help_and_readiness",
"test_m1_m28_package_smoke_docs_cover_static_import_export_surface",
),
"offline_registry_coverage": (
"test_offline_check_registry_name_matches_package_smoke_docs",
"test_offline_checks_registry_runs_training_package_smoke_entry",
"test_training_docs_name_broader_offline_check_path",
),
"m1_m28_maturity_wording": (
"test_product_readme_uses_current_maturity_and_public_private_package_language",
"test_m1_m28_package_smoke_docs_cover_static_import_export_surface",
),
}
TRAINING_M33_REQUIRED_INVENTORY_GROUPS = (
"package_smoke_docs_command_help",
"fixture_public_shape",
"fixture_private_solution_absence",
"service_import_export_public_output",
"cli_smoke_export_import_summary",
"readme_product_docs_offline_registry_refs",
)
TRAINING_M33_PACKAGE_FIXTURE_IMPORT_EXPORT_PRIVACY_INVENTORY = {
"package_smoke_docs_command_help": (
"test_example_command_uses_supported_cli_flags_and_fixture",
"test_public_package_smoke_docs_lock_import_export_privacy_language",
"test_cli_help_discloses_offline_public_import_export_and_error_contract",
"test_package_smoke_docs_publish_help_and_stable_public_error_wording",
"test_m1_m28_package_smoke_docs_cover_static_import_export_surface",
),
"fixture_public_shape": (
"test_public_fixture_schema_contract_is_minimal_and_demo_safe",
),
"fixture_private_solution_absence": (
"test_public_privacy_guard_rejects_solution_wording_drift",
"test_public_fixture_schema_contract_is_minimal_and_demo_safe",
"test_fixture_is_compatible_with_public_service_and_cli_export",
),
"service_import_export_public_output": (
"test_fixture_is_compatible_with_public_service_and_cli_export",
),
"cli_smoke_export_import_summary": (
"test_fixture_is_compatible_with_public_service_and_cli_export",
"test_example_documents_cli_summary_keys_and_output_semantics",
"test_cli_error_summary_uses_public_output_target_code_without_runtime_leaks",
),
"readme_product_docs_offline_registry_refs": (
"test_smoke_docs_and_product_readme_share_fixture_and_public_export_contract",
"test_readme_and_product_docs_advertise_public_offline_smoke_contract",
"test_offline_check_registry_name_matches_package_smoke_docs",
"test_offline_checks_registry_runs_training_package_smoke_entry",
"test_training_docs_name_broader_offline_check_path",
"test_training_readmes_name_dashboard_privacy_and_offline_registry_coverage",
),
}
def _example_command_parts() -> list[str]:
def _documented_command_parts() -> list[str]:
text = DOC_EXAMPLE.read_text(encoding="utf-8")
match = re.search(r"```bash\n(?P<command>.*?)\n```", text, flags=re.DOTALL)
if match is None:
raise AssertionError("missing bash command block")
command = re.sub(r"\\\n\s*", " ", match.group("command").strip())
return shlex.split(command)
def _fixture_payload() -> dict:
return json.loads(FIXTURE.read_text(encoding="utf-8"))
def _package_doc_text(path: Path) -> str:
text = path.read_text(encoding="utf-8")
if path != README:
return text
lines = [line for line in text.splitlines() if "run_training_case_package.py" in line]
if not lines:
raise AssertionError("README is missing the package smoke command")
return "\n".join(lines)
def _package_smoke_flags_from_source() -> tuple[str, ...]:
source = Path(training_cli.__file__).read_text(encoding="utf-8")
return tuple(re.findall(r'add_argument\("(?P<flag>--[a-z-]+)"', source))
def _package_cli_public_error_inventory_method_names() -> tuple[str, ...]:
return tuple(
name
for names in PACKAGE_CLI_PUBLIC_ERROR_TEST_INVENTORY.values()
for name in names
)
def _package_cli_public_error_doc_snippets() -> tuple[str, ...]:
return tuple(f"`{code}`" for code in PACKAGE_CLI_PUBLIC_ERROR_CODES)
def _package_cli_error_example_payloads(path: Path) -> tuple[dict, ...]:
text = path.read_text(encoding="utf-8")
marker = "Package CLI public error examples:"
if marker not in text:
raise AssertionError(f"{path} is missing package CLI public error examples")
section = text.split(marker, 1)[1]
next_heading = re.search(r"\n## ", section)
if next_heading is not None:
section = section[: next_heading.start()]
blocks = re.findall(r"```json\n(?P<body>\{[^\n]*\})\n```", section)
if not blocks:
raise AssertionError(f"{path} is missing one-line package CLI public error JSON")
return tuple(json.loads(block) for block in blocks)
def _run_package_cli_summary(args: list[str]) -> tuple[int, str, dict, str]:
stdout = io.StringIO()
stderr = io.StringIO()
with contextlib.redirect_stdout(stdout), contextlib.redirect_stderr(stderr):
exit_code = training_cli.main(args)
raw_output = stdout.getvalue()
return exit_code, raw_output, json.loads(raw_output.strip()), stderr.getvalue()
def _test_method_names_from_file(path: Path, class_name: str) -> tuple[str, ...]:
tree = ast.parse(path.read_text(encoding="utf-8"))
for node in tree.body:
if isinstance(node, ast.ClassDef) and node.name == class_name:
return tuple(
item.name
for item in node.body
if isinstance(item, ast.FunctionDef) and item.name.startswith("test_")
)
raise AssertionError(f"{path} is missing {class_name}")
def _walk_public_text(value) -> list[str]:
if isinstance(value, dict):
entries = []
for key, item in value.items():
entries.append(str(key))
entries.extend(_walk_public_text(item))
return entries
if isinstance(value, list):
entries = []
for item in value:
entries.extend(_walk_public_text(item))
return entries
if isinstance(value, str):
return [value]
return []
raise AssertionError("missing documented package smoke command")
return shlex.split(re.sub(r"\\\n\s*", " ", match.group("command").strip()))
class TrainingCaseDocsCliM10Tests(unittest.TestCase):
def assertNoPrivateFixtureMaterial(self, payload: dict) -> None:
serialized = json.dumps(payload, sort_keys=True)
lower_serialized = serialized.lower()
def test_example_doc_uses_current_public_fixture_and_cli(self) -> None:
parts = _documented_command_parts()
self.assertEqual(parts[:3], [".venv/bin/python", "scripts/run_training_case_package.py", "--input"])
self.assertIn(FIXTURE_REF, parts)
self.assertIn("--output", parts)
self.assertNotIn(PUBLIC_FIELD, lower_serialized)
self.assertIsNone(re.search(r"\b(?:\d{1,3}\.){3}\d{1,3}\b", serialized))
for phrase in PRIVATE_FIXTURE_DRIFT_PHRASES:
self.assertNotIn(phrase, lower_serialized)
for text in _walk_public_text(payload):
lower_text = text.lower()
for fragment in RESTRICTED_PUBLIC_TEXT:
with self.subTest(text=text, fragment=fragment):
self.assertNotIn(fragment, lower_text)
def test_public_privacy_guard_rejects_solution_wording_drift(self) -> None:
polluted_payloads = [
{"items": [{"prompt": "Do not expose " + ("private " + "answer") + " wording."}]},
{"items": [{"prompt": "Do not expose internal solution details wording."}]},
]
for payload in polluted_payloads:
with self.subTest(payload=payload):
with self.assertRaises(AssertionError):
self.assertNoPrivateFixtureMaterial(payload)
def test_public_fixture_schema_contract_is_minimal_and_demo_safe(self) -> None:
payload = _fixture_payload()
self.assertEqual(set(payload), {"version", "metadata", "items"})
self.assertEqual(payload["version"], 1)
self.assertEqual(payload["metadata"], {"source": "m6_public_fixture", "purpose": "offline_smoke"})
self.assertEqual(len(payload["items"]), 1)
item = payload["items"][0]
self.assertEqual(
set(item),
{
"external_id",
"title",
"scenario",
"skill_tags",
"difficulty",
"answer",
"rubric",
"tasks",
"metadata",
},
)
self.assertEqual(item["external_id"], "m6-sweep-public-001")
self.assertEqual(item["title"], "Liquidity sweep reading practice")
self.assertEqual(item["skill_tags"], ["sweep", "context_reading"])
self.assertEqual(item["difficulty"], "easy")
self.assertEqual(item["answer"], {})
self.assertEqual(
item["rubric"],
[
{"category": "context", "max_score": 3},
{"category": "risk_story", "max_score": 2},
],
)
self.assertEqual(len(item["tasks"]), 1)
self.assertEqual(item["metadata"], {"chapter_ref": "public_training_m6", "demo": True})
scenario = item["scenario"]
self.assertEqual(
set(scenario),
{
"instrument_id",
"timeframe",
"case_type",
"start_ts",
"decision_ts",
"end_ts",
"source_event_type",
"source_event_id",
"source_event_refs",
"chapter_ref",
},
)
self.assertEqual(scenario["instrument_id"], 1)
self.assertEqual(scenario["timeframe"], "15m")
self.assertEqual(scenario["case_type"], "sweep")
self.assertEqual(scenario["start_ts"], "2026-04-20T13:00:00Z")
self.assertEqual(scenario["decision_ts"], "2026-04-20T14:00:00Z")
self.assertEqual(scenario["end_ts"], "2026-04-20T15:00:00Z")
self.assertEqual(scenario["source_event_type"], "sweep_event")
self.assertEqual(scenario["source_event_id"], 61)
self.assertEqual(scenario["source_event_refs"], [{"kind": "sweep_event", "id": 61}])
self.assertEqual(scenario["chapter_ref"], "public_training_m6")
task = item["tasks"][0]
self.assertEqual(
set(task),
{
"task_type",
"skill_tag",
"prompt",
"required_annotation_types",
"max_score",
"sort_order",
"meta",
},
)
self.assertEqual(task["task_type"], "short_answer")
self.assertEqual(task["skill_tag"], "sweep")
self.assertEqual(task["prompt"], "Describe what needs confirmation before marking this as a usable sweep.")
self.assertEqual(task["required_annotation_types"], ["text"])
self.assertEqual(task["max_score"], 5)
self.assertEqual(task["sort_order"], 0)
self.assertEqual(task["meta"], {"public_hint": "Focus on context and confirmation."})
self.assertNoPrivateFixtureMaterial(payload)
def test_fixture_is_compatible_with_public_service_and_cli_export(self) -> None:
payload = _fixture_payload()
service = TrainingCaseService(repository=training_cli._OfflineTrainingRepository())
import_result = service.import_cases(payload)
export_result = service.export_cases(limit=100)
with tempfile.TemporaryDirectory() as tmpdir:
output_path = Path(tmpdir) / "training_case_public_export.json"
summary = training_cli.run_fixture_smoke(FIXTURE, output_path=output_path)
cli_export = json.loads(output_path.read_text(encoding="utf-8"))
self.assertEqual(import_result["version"], 1)
self.assertEqual(import_result["imported"], 1)
self.assertEqual(len(import_result["items"]), 1)
self.assertEqual(set(import_result["items"][0]), {"id", "title", "scenario", "skill_tags", "difficulty", "status", "external_id"})
self.assertEqual(summary["input"], FIXTURE_REF)
self.assertEqual(summary["imported"], 1)
self.assertEqual(summary["exported"], 1)
self.assertEqual(export_result, cli_export)
self.assertEqual(set(export_result), {"version", "metadata", "items"})
self.assertEqual(export_result["metadata"], {"item_count": 1})
exported_item = export_result["items"][0]
fixture_item = payload["items"][0]
self.assertEqual(
set(exported_item),
{
"id",
"title",
"scenario",
"skill_tags",
"difficulty",
"status",
"rubric",
"tasks",
"metadata",
"quality",
},
)
self.assertEqual(exported_item["title"], fixture_item["title"])
self.assertEqual(set(exported_item["skill_tags"]), set(fixture_item["skill_tags"]))
self.assertEqual(
exported_item["metadata"],
{"chapter_ref": "public_training_m6", "demo": True, "external_id": fixture_item["external_id"]},
)
self.assertEqual(exported_item["rubric"], fixture_item["rubric"])
self.assertEqual(exported_item["quality"]["rank"], "high")
self.assertEqual(exported_item["scenario"], fixture_item["scenario"])
self.assertEqual(set(exported_item["tasks"][0]), set(fixture_item["tasks"][0]) | {"status"})
self.assertEqual(exported_item["tasks"][0]["status"], "active")
for key, value in fixture_item["tasks"][0].items():
with self.subTest(task_field=key):
self.assertEqual(exported_item["tasks"][0][key], value)
self.assertNotIn("answer", exported_item)
self.assertNoPrivateFixtureMaterial({"imported": import_result, "exported": export_result})
def test_smoke_docs_and_product_readme_share_fixture_and_public_export_contract(self) -> None:
for path in (DOC_EXAMPLE, PRODUCT_README):
text = path.read_text(encoding="utf-8")
lower_text = text.lower()
fixture_refs = set(re.findall(r"tests/fixtures/[A-Za-z0-9_./-]+\.json", text))
with self.subTest(path=path.name):
self.assertEqual(fixture_refs, {FIXTURE_REF})
self.assertIn(".venv/bin/python scripts/run_training_case_package.py", text)
self.assertIn("public export", lower_text)
self.assertTrue(any(phrase in lower_text for phrase in PRIVATE_MATERIAL_PHRASES))
self.assertNotIn(PUBLIC_FIELD, text)
def test_example_command_uses_supported_cli_flags_and_fixture(self) -> None:
parts = _example_command_parts()
self.assertEqual(parts[:2], [".venv/bin/python", "scripts/run_training_case_package.py"])
parser = training_cli._build_parser()
supported_flags = {flag for action in parser._actions for flag in action.option_strings}
used_flags = [part for part in parts[2:] if part.startswith("--")]
self.assertEqual(used_flags, ["--input", "--output"])
self.assertLessEqual(set(used_flags), supported_flags)
args = parser.parse_args(parts[2:])
expected_fixture = training_cli.DEFAULT_FIXTURE.relative_to(ROOT)
self.assertEqual(Path(args.input), expected_fixture)
self.assertEqual(Path(args.input), FIXTURE.relative_to(ROOT))
self.assertEqual(Path(args.output), Path("/tmp/training_case_public_export.json"))
def test_example_documents_cli_summary_keys_and_output_semantics(self) -> None:
doc_text = DOC_EXAMPLE.read_text(encoding="utf-8")
with tempfile.TemporaryDirectory() as tmpdir:
output_path = Path(tmpdir) / "training_case_public_export.json"
stdout = io.StringIO()
with contextlib.redirect_stdout(stdout):
exit_code = training_cli.main(
["--input", str(FIXTURE.relative_to(ROOT)), "--output", str(output_path)]
)
summary = json.loads(stdout.getvalue())
exported = json.loads(output_path.read_text(encoding="utf-8"))
expected_keys = {"exported", "imported", "input", "output", "passed"}
self.assertEqual(exit_code, 0)
self.assertEqual(set(summary), expected_keys)
self.assertTrue(summary["passed"])
self.assertEqual(summary["imported"], 1)
self.assertEqual(summary["exported"], 1)
self.assertTrue(summary["output"].endswith("training_case_public_export.json"))
self.assertEqual(summary["input"], FIXTURE_REF)
self.assertEqual(exported["metadata"], {"item_count": 1})
self.assertNotIn(PUBLIC_FIELD, json.dumps(exported, sort_keys=True))
for key in sorted(expected_keys):
with self.subTest(key=key):
self.assertIn(f"`{key}`", doc_text)
def test_cli_error_summary_uses_public_output_target_code_without_runtime_leaks(self) -> None:
with tempfile.TemporaryDirectory() as tmpdir:
output_path = Path(tmpdir) / "training_case_public_export_dir"
output_path.mkdir()
stdout = io.StringIO()
stderr = io.StringIO()
with contextlib.redirect_stdout(stdout), contextlib.redirect_stderr(stderr):
exit_code = training_cli.main(
["--input", str(FIXTURE.relative_to(ROOT)), "--output", str(output_path)]
)
raw_output = stdout.getvalue()
line = raw_output.strip()
summary = json.loads(line)
self.assertEqual(exit_code, 2)
self.assertEqual(stderr.getvalue(), "")
self.assertEqual(raw_output.count("\n"), 1)
self.assertEqual(list(summary), ["passed", "input", "output", "error"])
self.assertFalse(summary["passed"])
self.assertEqual(summary["input"], FIXTURE_REF)
self.assertEqual(summary["output"], output_path.name)
self.assertEqual(summary["error"], "output_path_is_directory")
self.assertNotIn(str(output_path), line)
self.assertNotIn(str(Path(tmpdir)), line)
self.assertIsNone(re.search(r'(?<!\\)"(?:input|output)": "/', line))
for fragment in ("[Errno", "Is a directory", "Traceback"):
with self.subTest(fragment=fragment):
self.assertNotIn(fragment.lower(), line.lower())
self.assertNotIn(PUBLIC_FIELD, line)
for phrase in PRIVATE_FIXTURE_DRIFT_PHRASES:
with self.subTest(phrase=phrase):
self.assertNotIn(phrase, line.lower())
def test_readme_and_product_docs_advertise_public_offline_smoke_contract(self) -> None:
for path in (README, PRODUCT_README):
text = path.read_text(encoding="utf-8")
lower_text = text.lower()
with self.subTest(path=path.name):
self.assertIn(".venv/bin/python scripts/run_training_case_package.py", text)
self.assertIn(FIXTURE_REF, text)
self.assertIn("offline package smoke", lower_text)
self.assertIn("public export", lower_text)
self.assertTrue(any(phrase in lower_text for phrase in PRIVATE_MATERIAL_PHRASES))
self.assertNotIn(PUBLIC_FIELD, text)
def test_product_readme_uses_current_maturity_and_public_private_package_language(self) -> None:
text = PRODUCT_README.read_text(encoding="utf-8")
lower_text = text.lower()
for snippet in PRODUCT_PACKAGE_WORDING:
with self.subTest(snippet=snippet):
self.assertIn(snippet, text if snippet.startswith("M1-") else lower_text)
for phrase in PRIVATE_DOC_DRIFT_PHRASES:
with self.subTest(phrase=phrase):
self.assertNotIn(phrase, lower_text)
self.assertNotIn("M1-M21", text)
self.assertNotIn(PUBLIC_FIELD, text)
def test_m24_m29_package_docs_privacy_test_inventory_is_current(self) -> None:
test_names = set(unittest.TestLoader().getTestCaseNames(type(self)))
inventory_names = tuple(
name
for names in TRAINING_M24_M29_PACKAGE_DOCS_TEST_INVENTORY.values()
for name in names
)
unique_inventory_names = set(inventory_names)
product_text = PRODUCT_README.read_text(encoding="utf-8")
self.assertEqual(len(test_names), 18)
self.assertEqual(len(inventory_names), 19)
self.assertEqual(len(unique_inventory_names), 17)
self.assertEqual(sorted(TRAINING_M24_M29_PACKAGE_DOCS_TEST_INVENTORY), [
"m1_m28_maturity_wording",
"offline_registry_coverage",
"package_smoke_privacy",
])
self.assertEqual(
test_names - unique_inventory_names,
{"test_m24_m29_package_docs_privacy_test_inventory_is_current"},
)
for surface, names in TRAINING_M24_M29_PACKAGE_DOCS_TEST_INVENTORY.items():
with self.subTest(surface=surface):
self.assertTrue(names)
for name in names:
self.assertIn(name, test_names)
self.assertIn(CURRENT_TRAINING_DOC_MATURITY_MARKER, product_text)
self.assertIn(CURRENT_TRAINING_DOC_MATURITY_SPAN, product_text)
docs_cli_test_names = set(
_test_method_names_from_file(Path(__file__), TRAINING_DOCS_CLI_TEST_CLASS)
)
documentation_consistency_test_names = set(
_test_method_names_from_file(
DOCUMENTATION_CONSISTENCY_TEST,
DOCUMENTATION_CONSISTENCY_TEST_CLASS,
)
)
m33_available_test_names = docs_cli_test_names | documentation_consistency_test_names
m33_inventory_names = tuple(
name
for names in TRAINING_M33_PACKAGE_FIXTURE_IMPORT_EXPORT_PRIVACY_INVENTORY.values()
for name in names
)
m33_unique_inventory_names = set(m33_inventory_names)
self.assertEqual(
sorted(TRAINING_M33_PACKAGE_FIXTURE_IMPORT_EXPORT_PRIVACY_INVENTORY),
sorted(TRAINING_M33_REQUIRED_INVENTORY_GROUPS),
)
self.assertEqual(len(m33_inventory_names), 19)
self.assertEqual(len(m33_unique_inventory_names), 16)
self.assertIn(
"test_training_readmes_name_dashboard_privacy_and_offline_registry_coverage",
documentation_consistency_test_names,
)
for surface, names in TRAINING_M33_PACKAGE_FIXTURE_IMPORT_EXPORT_PRIVACY_INVENTORY.items():
with self.subTest(m33_surface=surface):
self.assertTrue(names)
for name in names:
self.assertIn(name, m33_available_test_names)
def test_public_package_smoke_docs_lock_import_export_privacy_language(self) -> None:
help_text = training_cli._build_parser().format_help()
self.assertIn("--input", help_text)
self.assertIn("--output", help_text)
self.assertIn("public export", help_text.lower())
for path in PUBLIC_PACKAGE_DOCS:
text = _package_doc_text(path)
lower_text = text.lower()
with self.subTest(path=path.name):
for snippet in PUBLIC_PACKAGE_REQUIRED_TEXT:
with self.subTest(snippet=snippet):
self.assertIn(snippet, text if snippet.startswith(".") or snippet.startswith("--") else lower_text)
for phrase in PRIVATE_DOC_DRIFT_PHRASES:
with self.subTest(phrase=phrase):
self.assertNotIn(phrase, lower_text)
self.assertNotIn(PUBLIC_FIELD, text)
def test_cli_help_discloses_offline_public_import_export_and_error_contract(self) -> None:
parser = training_cli._build_parser()
help_text = parser.format_help()
lower_help = help_text.lower()
stdout = io.StringIO()
stderr = io.StringIO()
with (
contextlib.redirect_stdout(stdout),
contextlib.redirect_stderr(stderr),
self.assertRaises(SystemExit) as raised,
def test_example_doc_publishes_current_public_smoke_wording(self) -> None:
text = DOC_EXAMPLE.read_text(encoding="utf-8")
for snippet in (
"import/export smoke",
"public package",
"public export",
"public output",
"one-line JSON",
"input_file_not_found",
"input_path_is_directory",
"output_path_is_directory",
"output_path_unwritable",
):
training_cli.main(["--help"])
self.assertIn(snippet, text)
self.assertEqual(raised.exception.code, 0)
self.assertEqual(stderr.getvalue(), "")
self.assertEqual(stdout.getvalue(), help_text)
self.assertIn(PACKAGE_SMOKE_SCRIPT, help_text)
self.assertIn(FIXTURE_REF, help_text)
self.assertNotIn(PUBLIC_FIELD, help_text)
for snippet in PACKAGE_CLI_HELP_DISCOVERY_TEXT:
with self.subTest(snippet=snippet):
self.assertIn(snippet, lower_help)
def test_training_public_doc_references_console_and_api_surface(self) -> None:
text = TRAINING_PUBLIC_DOC.read_text(encoding="utf-8")
for snippet in (
"The public browser entry is `/training`.",
"The trading workbench stays at `/`.",
"| GET | `/training/cases` |",
"| GET | `/training/progress` |",
"Training public error catalog:",
PACKAGE_SMOKE_COMMAND,
):
self.assertIn(snippet, text)
def test_package_smoke_docs_publish_help_and_stable_public_error_wording(self) -> None:
for path in PACKAGE_SMOKE_SURFACE_DOCS:
text = path.read_text(encoding="utf-8")
lower_text = text.lower()
def test_readme_points_to_current_offline_checks_and_training_console(self) -> None:
text = README.read_text(encoding="utf-8")
for snippet in (
OFFLINE_CHECK_COMMAND,
"Training Console at `/training`",
"training_console_js_syntax",
"python3 scripts/run_api_server.py",
):
self.assertIn(snippet, text)
self.assertNotIn("training_case_engine_m1_m6_product_readme.md", text)
with self.subTest(path=path.name):
self.assertIn("invalid output targets", lower_text)
self.assertNotIn(PUBLIC_FIELD, text)
for snippet in PACKAGE_DOC_HELP_ERROR_TEXT:
with self.subTest(snippet=snippet):
self.assertIn(snippet, text)
def test_offline_checks_registry_includes_both_web_entries(self) -> None:
commands = dict(run_offline_checks.build_check_commands(env={}))
self.assertEqual(commands["dashboard_js_syntax"], ["node", "--check", "src/web/app.js"])
self.assertEqual(commands["training_console_js_syntax"], ["node", "--check", "src/web/training.js"])
def test_package_cli_public_error_inventory_locks_docs_help_and_readiness(self) -> None:
test_names = set(unittest.TestLoader().getTestCaseNames(type(self)))
inventory_names = _package_cli_public_error_inventory_method_names()
doc_snippets = _package_cli_public_error_doc_snippets()
help_text = training_cli._build_parser().format_help()
source_text = Path(training_cli.__file__).read_text(encoding="utf-8")
self.assertEqual(
sorted(PACKAGE_CLI_PUBLIC_ERROR_TEST_INVENTORY),
["docs", "help", "readiness"],
)
self.assertEqual(len(inventory_names), 3)
self.assertEqual(
set(inventory_names),
{
"test_cli_error_summary_uses_public_output_target_code_without_runtime_leaks",
"test_cli_help_discloses_offline_public_import_export_and_error_contract",
"test_package_smoke_docs_publish_help_and_stable_public_error_wording",
},
)
for surface, names in PACKAGE_CLI_PUBLIC_ERROR_TEST_INVENTORY.items():
with self.subTest(surface=surface):
self.assertTrue(names)
for name in names:
self.assertIn(name, test_names)
self.assertEqual(
PACKAGE_CLI_PUBLIC_ERROR_CODES,
(
"input_file_not_found",
"input_path_is_directory",
"output_path_is_directory",
"output_path_unwritable",
),
)
self.assertEqual(
PACKAGE_CLI_PUBLIC_ERROR_EXAMPLE_CODES,
PACKAGE_CLI_PUBLIC_ERROR_CODES[:3],
)
self.assertEqual(doc_snippets, tuple(f"`{code}`" for code in PACKAGE_CLI_PUBLIC_ERROR_CODES))
for code, snippet in zip(PACKAGE_CLI_PUBLIC_ERROR_CODES, doc_snippets):
with self.subTest(code=code):
self.assertIn(code, source_text)
self.assertIn(code, help_text)
self.assertIn(code, PACKAGE_CLI_HELP_DISCOVERY_TEXT)
self.assertIn(snippet, PACKAGE_DOC_HELP_ERROR_TEXT)
for path in PACKAGE_SMOKE_SURFACE_DOCS:
text = path.read_text(encoding="utf-8")
with self.subTest(path=path.name):
for snippet in doc_snippets:
self.assertIn(snippet, text)
for path in (README, PRODUCT_README):
payloads = _package_cli_error_example_payloads(path)
self.assertEqual(
tuple(payload["error"] for payload in payloads),
PACKAGE_CLI_PUBLIC_ERROR_EXAMPLE_CODES,
)
for payload in payloads:
with self.subTest(path=path.name, code=payload["error"]):
self.assertEqual(list(payload), ["passed", "input", "output", "error"])
self.assertFalse(payload["passed"])
with tempfile.TemporaryDirectory(dir=ROOT) as tmpdir:
temp_root = Path(tmpdir)
missing_input = temp_root / "missing_training_case_package.json"
input_dir = temp_root / "training_case_package_dir"
input_dir.mkdir()
output_dir = temp_root / "training_case_public_export_dir"
output_dir.mkdir()
runtime_cases = (
(
"input_file_not_found",
[
"--input",
str(missing_input.relative_to(ROOT)),
"--output",
"training_case_public_export.json",
],
),
(
"input_path_is_directory",
[
"--input",
str(input_dir.relative_to(ROOT)),
"--output",
"training_case_public_export.json",
],
),
(
"output_path_is_directory",
[
"--input",
str(FIXTURE.relative_to(ROOT)),
"--output",
str(output_dir.relative_to(ROOT)),
],
),
)
for code, args in runtime_cases:
exit_code, raw_output, summary, stderr_text = _run_package_cli_summary(args)
with self.subTest(runtime_code=code):
self.assertEqual(exit_code, 2)
self.assertEqual(stderr_text, "")
self.assertEqual(raw_output.count("\n"), 1)
self.assertEqual(list(summary), ["passed", "input", "output", "error"])
self.assertFalse(summary["passed"])
self.assertEqual(summary["error"], code)
self.assertIsNone(re.search(r'(?<!\\)"(?:input|output)": "/', raw_output))
with tempfile.TemporaryDirectory() as tmpdir:
output_path = Path(tmpdir) / "training_case_public_export.json"
with mock.patch.object(Path, "write_text", side_effect=OSError("write blocked")):
exit_code, raw_output, summary, stderr_text = _run_package_cli_summary(
["--input", str(FIXTURE.relative_to(ROOT)), "--output", str(output_path)]
)
self.assertEqual(exit_code, 2)
self.assertEqual(stderr_text, "")
self.assertEqual(raw_output.count("\n"), 1)
self.assertEqual(list(summary), ["passed", "input", "output", "error"])
self.assertFalse(summary["passed"])
self.assertEqual(summary["error"], "output_path_unwritable")
self.assertIsNone(re.search(r'(?<!\\)"(?:input|output)": "/', raw_output))
def test_m1_m28_package_smoke_docs_cover_static_import_export_surface(self) -> None:
product_text = PRODUCT_README.read_text(encoding="utf-8")
cli_flags = _package_smoke_flags_from_source()
self.assertIn(CURRENT_TRAINING_DOC_MATURITY_MARKER, product_text)
self.assertEqual(cli_flags, ("--input", "--output"))
for path in PACKAGE_SMOKE_SURFACE_DOCS:
text = _package_doc_text(path)
lower_text = text.lower()
with self.subTest(path=path.name):
self.assertIn(PACKAGE_SMOKE_SCRIPT, text)
self.assertIn(FIXTURE_REF, text)
for flag in cli_flags:
with self.subTest(flag=flag):
self.assertIn(flag, text)
for snippet in PACKAGE_IMPORT_EXPORT_SURFACE_TEXT:
with self.subTest(snippet=snippet):
self.assertIn(snippet, lower_text)
def test_offline_check_registry_name_matches_package_smoke_docs(self) -> None:
commands = run_offline_checks.build_check_commands(env={})
names = tuple(name for name, _command in commands)
commands_by_name = dict(commands)
matching_commands = [
(name, command) for name, command in commands if name == OFFLINE_CHECK_NAME
]
registry_text = Path(run_offline_checks.__file__).read_text(encoding="utf-8")
self.assertEqual(names, OFFLINE_CHECK_REGISTRY_NAMES)
for check_name in OFFLINE_CHECK_REGISTRY_NAMES:
with self.subTest(check_name=check_name):
self.assertEqual(names.count(check_name), 1)
self.assertEqual(
matching_commands,
[(OFFLINE_CHECK_NAME, [sys.executable, PACKAGE_SMOKE_SCRIPT])],
)
self.assertEqual(
commands_by_name[OFFLINE_CHECK_NAME],
[sys.executable, PACKAGE_SMOKE_SCRIPT],
)
self.assertEqual(
commands_by_name[DASHBOARD_SYNTAX_CHECK_NAME],
["node", "--check", DASHBOARD_SYNTAX_CHECK_PATH],
)
self.assertEqual(registry_text.count(f'"{OFFLINE_CHECK_NAME}"'), 1)
self.assertIn(f'"{PACKAGE_SMOKE_SCRIPT}"', registry_text)
for path in (README, DOC_EXAMPLE, PRODUCT_README):
text = path.read_text(encoding="utf-8")
lower_text = text.lower()
with self.subTest(path=path.name):
self.assertIn(OFFLINE_CHECK_DOC_NAME, text)
self.assertIn(PACKAGE_SMOKE_COMMAND, text)
self.assertIn(OFFLINE_CHECK_COMMAND, text)
self.assertIn("package smoke", lower_text)
def test_offline_checks_registry_runs_training_package_smoke_entry(self) -> None:
commands = run_offline_checks.build_check_commands(env={})
names = [name for name, _command in commands]
command = dict(commands)[OFFLINE_CHECK_NAME]
calls = []
def fake_runner(command_parts, **kwargs):
calls.append((command_parts, kwargs["cwd"]))
return SimpleNamespace(returncode=0, stdout="", stderr="")
def test_offline_checks_json_and_node_skip_behavior_cover_new_training_check(self) -> None:
def fake_runner(command, **_kwargs):
return SimpleNamespace(returncode=0, stdout="ok\n", stderr="")
output = io.StringIO()
with contextlib.redirect_stdout(output):
exit_code = run_offline_checks.run_checks(
commands=[(OFFLINE_CHECK_NAME, command)],
commands=[("training_console_js_syntax", ["node", "--check", "src/web/training.js"])],
runner=fake_runner,
node_available=True,
node_available=False,
env={},
)
self.assertEqual(names.count(OFFLINE_CHECK_NAME), 1)
self.assertEqual(command, [sys.executable, PACKAGE_SMOKE_SCRIPT])
self.assertEqual((ROOT / command[1]).resolve(), Path(training_cli.__file__).resolve())
self.assertEqual(exit_code, 0)
self.assertEqual(calls, [(command, run_offline_checks.ROOT)])
self.assertIn(f"Offline check passed: {OFFLINE_CHECK_NAME}", output.getvalue())
self.assertIn("Offline check skipped: training_console_js_syntax", output.getvalue())
def test_training_docs_name_broader_offline_check_path(self) -> None:
for path in (DOC_EXAMPLE, README, PRODUCT_README):
text = path.read_text(encoding="utf-8")
lower_text = text.lower()
def test_package_cli_still_emits_one_line_public_json(self) -> None:
output = io.StringIO()
with contextlib.redirect_stdout(output):
exit_code = training_cli.main(["--input", str(FIXTURE)])
with self.subTest(path=path.name):
self.assertIn(OFFLINE_CHECK_COMMAND, text)
self.assertIn("broader offline", lower_text)
self.assertIn("training case", lower_text)
summary = json.loads(output.getvalue())
self.assertEqual(exit_code, 0)
self.assertTrue(summary["passed"])
self.assertTrue(str(summary["input"]).endswith("tests\\fixtures\\training_case_package_m6.json"))
self.assertIsNone(summary["output"])
if __name__ == "__main__":
tests/unit/test_training_public_api_contract_m8.py
+4 -1
View File
@@ -1,18 +1,21 @@
import ast
import io
import json
import os
import re
import unittest
from pathlib import Path
from unittest import mock
os.environ.setdefault("DATABASE_URL", "postgresql+psycopg://user:pass@localhost:5432/ai_ict_test")
from src.api import server
from src.api import training as training_api
from src.services.training import TrainingCaseService
ROOT = Path(__file__).resolve().parents[2]
TRAINING_PRODUCT_DOC = ROOT / "docs" / "training_case_engine_m1_m6_product_readme.md"
TRAINING_PRODUCT_DOC = ROOT / "docs" / "training_console_public_surface_v1.md"
INTERNAL_FIELD = "answer" + "_key"
PRIVATE_MARKER = "trainer-only-choice"
PRIVATE_RUBRIC_MARKER = "trainer-only-rubric"