ci: authenticate release quality gate
Security Quality Gate / npm audit, Bandit, pip-audit, detect-secrets (pull_request) Successful in 11m32s
Security Quality Gate / npm audit, Bandit, pip-audit, detect-secrets (pull_request) Successful in 11m32s
This commit is contained in:
@@ -51,7 +51,8 @@
|
||||
- Word 交付:抽样合同下载返回 `HT-20260713-70C052.docx`,MIME 为 Office Open XML,解码后 37,502 字节且 DOCX 文件头有效。
|
||||
- 资产精确核对:模板 `30 / 30 / 30`、条款 `8 / 8 / 8`、规则 `8 / 8 / 8`、Prompt `46 / 46 / 46`,法律条文运行时 `789,867` 条。
|
||||
- 法律问答工程链路可用,但 smoke fallback 抽样的引用相关性不足,不能作为生产 AI 质量证据;必须在真实 Provider、fallback 关闭后重新验收。
|
||||
- PR 旧提交 `5f89c112` 的 CI 失败来自 Linux detect-secrets 对 SDK 测试固定假值的单个平台差异误报;已使用标准 inline allowlist 标记,安全规则本身未放宽,待当前提交触发 CI 重跑。
|
||||
- PR 旧提交 `5f89c112` 的 CI 失败来自 Linux detect-secrets 对 SDK 测试固定假值的单个平台差异误报;已使用标准 inline allowlist 标记,安全规则本身未放宽。提交 `6abca4eb` 的 npm audit、Bandit、pip-audit 和 Linux detect-secrets 已全部通过。
|
||||
- 最新发布质量门:`project-docs/qa-artifacts/v2-release-quality-gate/2026-07-13T14-03-15`。7 个场景各 60 次请求、并发 6、错误率均为 0%;合同列表 p95 `199.31 ms`,知识库混合检索 p95 `66.74 ms`。
|
||||
|
||||
## 生产阻断项
|
||||
|
||||
@@ -61,7 +62,7 @@
|
||||
- [x] 验证无代理密钥直连 Contract API / AI Platform 数据接口返回 `401`。
|
||||
- [x] 验证 Web 登录后合同、模板、文件、AI Trace、知识库均正常。
|
||||
- [x] 复跑两套 schema-check、资产精确核对和 Docker smoke。
|
||||
- [ ] 当前提交 CI 全绿后,复跑带最新候选代码的接口压测与安全质量门,作为生产审批证据。
|
||||
- [x] 当前提交 CI 全绿,并复跑带最新候选代码的接口压测与安全质量门,作为生产审批证据。
|
||||
|
||||
### P0-2 真实 AI Provider 与质量验收
|
||||
|
||||
|
||||
+20
-8
@@ -110,11 +110,19 @@ After smoke passes, the old production deployment can be replaced deliberately:
|
||||
|
||||
Use this after a smoke deploy and interface benchmark to turn the release checks into one pass/fail gate:
|
||||
|
||||
When service authentication is enabled, load the protected smoke values into process environment variables first. Do not pass the values on the command line or write them to the artifact directory.
|
||||
|
||||
```powershell
|
||||
$env:CONTRACT_API_WEB_PROXY_SHARED_SECRET = "<load from protected smoke environment>"
|
||||
$env:AI_PLATFORM_SERVICE_SHARED_SECRET = "<load from protected smoke environment>"
|
||||
```
|
||||
|
||||
```powershell
|
||||
python yuqei-infra\scripts\run-release-quality-gate.py `
|
||||
--web-base-url http://192.168.3.90:13000 `
|
||||
--contract-api-base http://192.168.3.90:18100 `
|
||||
--ai-api-base http://192.168.3.90:18101
|
||||
--ai-api-base http://192.168.3.90:18101 `
|
||||
--allow-ai-provider-warning
|
||||
```
|
||||
|
||||
The gate verifies:
|
||||
@@ -131,7 +139,8 @@ python yuqei-infra\scripts\run-release-quality-gate.py `
|
||||
--web-base-url http://192.168.3.90:13000 `
|
||||
--contract-api-base http://192.168.3.90:18100 `
|
||||
--ai-api-base http://192.168.3.90:18101 `
|
||||
--run-benchmark
|
||||
--run-benchmark `
|
||||
--allow-ai-provider-warning
|
||||
```
|
||||
|
||||
To require fresh summaries without rerunning every check:
|
||||
@@ -147,17 +156,20 @@ python yuqei-infra\scripts\run-release-quality-gate.py `
|
||||
|
||||
Artifacts are written to `project-docs\qa-artifacts\v2-release-quality-gate`. GitHub Actions also exposes a manual `Release Quality Gate` workflow for a self-hosted runner with access to `192.168.3.90`.
|
||||
|
||||
`--allow-ai-provider-warning` is only for a smoke environment that intentionally disables real Provider calls and enables fallback. Omit it for production readiness; any `provider_runtime` warning then fails the gate. Both the gate and benchmark read service keys from environment variables and record only whether authentication was configured, never the values.
|
||||
|
||||
Latest passing release gate artifact:
|
||||
|
||||
- `project-docs\qa-artifacts\v2-release-quality-gate\2026-07-12T13-28-00`
|
||||
- Run mode: `--run-benchmark`
|
||||
- Contract API schema-check, AI Platform schema-check, Docker smoke summary, and 7 interface benchmark scenarios passed.
|
||||
- `project-docs\qa-artifacts\v2-release-quality-gate\2026-07-13T14-03-15`
|
||||
- Run mode: `--run-benchmark --allow-ai-provider-warning`, with protected service keys loaded from environment variables.
|
||||
- Contract API schema-check, the expected smoke-only AI Provider warning, Docker smoke summary, and 7 interface benchmark scenarios passed.
|
||||
- Each scenario ran 60 measured requests at concurrency 6 with 0% errors. Contract list p95 was 199.31 ms and knowledge chunk hybrid search p95 was 66.74 ms.
|
||||
|
||||
Latest supporting artifacts:
|
||||
|
||||
- Interface benchmark: `project-docs\qa-artifacts\v2-interface-benchmark\2026-07-12T13-24-00`.
|
||||
- Security scan: `project-docs\qa-artifacts\v2-security-scan\2026-07-12T13-08-00`.
|
||||
- Source secret scan (frozen RC index): `project-docs\qa-artifacts\v2-source-secret-scan\2026-07-12T15-24-56` (821 tracked files, 16 reviewed candidates, 0 new findings, 0 stale baseline entries).
|
||||
- Interface benchmark: `project-docs\qa-artifacts\v2-release-quality-gate\2026-07-13T14-03-15\interface-benchmark`.
|
||||
- Security scan: `project-docs\qa-artifacts\v2-security-scan\2026-07-13-auth-hardening-final-2`.
|
||||
- Source secret scan: PR CI for commit `6abca4eb` passed npm audit, Bandit, pip-audit, and Linux detect-secrets with no unreviewed source secret.
|
||||
|
||||
## Contract Review Worker Flow Verification
|
||||
|
||||
|
||||
@@ -7,6 +7,7 @@ import argparse
|
||||
import csv
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import statistics
|
||||
import sys
|
||||
import time
|
||||
@@ -49,13 +50,20 @@ def build_url(base_url: str, path: str) -> str:
|
||||
return f"{normalize_base_url(base_url)}{path}"
|
||||
|
||||
|
||||
def fetch_once(url: str, timeout_seconds: float, scenario_key: str, iteration: int) -> dict[str, Any]:
|
||||
def fetch_once(
|
||||
url: str,
|
||||
timeout_seconds: float,
|
||||
scenario_key: str,
|
||||
iteration: int,
|
||||
headers: dict[str, str],
|
||||
) -> dict[str, Any]:
|
||||
started = time.perf_counter()
|
||||
request = Request(
|
||||
url,
|
||||
headers={
|
||||
"Accept": "application/json",
|
||||
"User-Agent": "yuqei-v2-interface-benchmark/1.0",
|
||||
**headers,
|
||||
},
|
||||
method="GET",
|
||||
)
|
||||
@@ -94,8 +102,8 @@ def fetch_once(url: str, timeout_seconds: float, scenario_key: str, iteration: i
|
||||
}
|
||||
|
||||
|
||||
def fetch_json(url: str, timeout_seconds: float) -> Any:
|
||||
request = Request(url, headers={"Accept": "application/json"}, method="GET")
|
||||
def fetch_json(url: str, timeout_seconds: float, headers: dict[str, str]) -> Any:
|
||||
request = Request(url, headers={"Accept": "application/json", **headers}, method="GET")
|
||||
with urlopen(request, timeout=timeout_seconds) as response:
|
||||
payload = response.read().decode("utf-8")
|
||||
return json.loads(payload)
|
||||
@@ -257,9 +265,24 @@ def parse_args() -> argparse.Namespace:
|
||||
parser.add_argument("--keyword", default="\u5408\u540c")
|
||||
parser.add_argument("--artifact-dir", default="")
|
||||
parser.add_argument("--include-heavy-explain", action="store_true")
|
||||
parser.add_argument("--tenant-id", default="default")
|
||||
parser.add_argument("--user-id", default="user-current-admin")
|
||||
parser.add_argument("--contract-proxy-key-env", default="CONTRACT_API_WEB_PROXY_SHARED_SECRET")
|
||||
parser.add_argument("--ai-service-key-env", default="AI_PLATFORM_SERVICE_SHARED_SECRET")
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def request_headers(tenant_id: str, user_id: str, auth_header: str, secret: str) -> dict[str, str]:
|
||||
headers: dict[str, str] = {}
|
||||
if tenant_id.strip():
|
||||
headers["X-Tenant-Id"] = tenant_id.strip()
|
||||
if user_id.strip():
|
||||
headers["X-User-Id"] = user_id.strip()
|
||||
if secret:
|
||||
headers[auth_header] = secret
|
||||
return headers
|
||||
|
||||
|
||||
def main() -> int:
|
||||
args = parse_args()
|
||||
if args.requests_per_scenario <= 0:
|
||||
@@ -271,6 +294,20 @@ def main() -> int:
|
||||
|
||||
contract_api_base = normalize_base_url(args.contract_api_base)
|
||||
ai_api_base = normalize_base_url(args.ai_api_base)
|
||||
contract_proxy_key = os.getenv(args.contract_proxy_key_env, "").strip()
|
||||
ai_service_key = os.getenv(args.ai_service_key_env, "").strip()
|
||||
contract_headers = request_headers(
|
||||
args.tenant_id,
|
||||
args.user_id,
|
||||
"X-Contract-Proxy-Key",
|
||||
contract_proxy_key,
|
||||
)
|
||||
ai_headers = request_headers(
|
||||
args.tenant_id,
|
||||
args.user_id,
|
||||
"X-AI-Platform-Service-Key",
|
||||
ai_service_key,
|
||||
)
|
||||
artifact_dir = Path(args.artifact_dir) if args.artifact_dir else DEFAULT_ARTIFACT_ROOT / now_stamp()
|
||||
if not artifact_dir.is_absolute():
|
||||
artifact_dir = REPO_ROOT / artifact_dir
|
||||
@@ -279,7 +316,7 @@ def main() -> int:
|
||||
keyword = quote(args.keyword)
|
||||
dataset: dict[str, Any] = {}
|
||||
contract_list_probe_url = build_url(contract_api_base, "/api/v1/contracts?page=1&page_size=1")
|
||||
contract_probe = fetch_json(contract_list_probe_url, args.timeout_seconds)
|
||||
contract_probe = fetch_json(contract_list_probe_url, args.timeout_seconds, contract_headers)
|
||||
contract_items = contract_probe.get("items", []) if isinstance(contract_probe, dict) else []
|
||||
if not contract_items:
|
||||
raise SystemExit("No contracts found; cannot discover contract detail scenario.")
|
||||
@@ -292,7 +329,7 @@ def main() -> int:
|
||||
f"/api/v1/search?keyword={keyword}&page=1&page_size=10",
|
||||
)
|
||||
try:
|
||||
search_probe = fetch_json(search_probe_url, args.timeout_seconds)
|
||||
search_probe = fetch_json(search_probe_url, args.timeout_seconds, contract_headers)
|
||||
if isinstance(search_probe, dict):
|
||||
dataset["unified_search_total"] = search_probe.get("total", "unknown")
|
||||
except Exception as exc:
|
||||
@@ -364,15 +401,16 @@ def main() -> int:
|
||||
scenario_summaries: list[dict[str, Any]] = []
|
||||
for scenario in scenarios:
|
||||
base_url = contract_api_base if scenario.service == "contract-api" else ai_api_base
|
||||
headers = contract_headers if scenario.service == "contract-api" else ai_headers
|
||||
url = build_url(base_url, scenario.path)
|
||||
for index in range(args.warmup):
|
||||
fetch_once(url, args.timeout_seconds, scenario.key, -index - 1)
|
||||
fetch_once(url, args.timeout_seconds, scenario.key, -index - 1, headers)
|
||||
|
||||
started = time.perf_counter()
|
||||
scenario_results: list[dict[str, Any]] = []
|
||||
with ThreadPoolExecutor(max_workers=args.concurrency) as executor:
|
||||
futures = [
|
||||
executor.submit(fetch_once, url, args.timeout_seconds, scenario.key, index + 1)
|
||||
executor.submit(fetch_once, url, args.timeout_seconds, scenario.key, index + 1, headers)
|
||||
for index in range(args.requests_per_scenario)
|
||||
]
|
||||
for future in as_completed(futures):
|
||||
@@ -394,6 +432,10 @@ def main() -> int:
|
||||
"warmup_requests": args.warmup,
|
||||
"timeout_seconds": args.timeout_seconds,
|
||||
"artifact_dir": str(artifact_dir),
|
||||
"tenant_id": args.tenant_id,
|
||||
"user_id": args.user_id,
|
||||
"contract_proxy_auth_configured": bool(contract_proxy_key),
|
||||
"ai_service_auth_configured": bool(ai_service_key),
|
||||
},
|
||||
"dataset": dataset,
|
||||
"scenarios": scenario_summaries,
|
||||
|
||||
@@ -5,6 +5,7 @@ from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
@@ -118,9 +119,13 @@ def age_status(
|
||||
return "pass", f"{label} age_hours={actual_age:.2f} is within max_age_hours={max_age_hours}.", meta
|
||||
|
||||
|
||||
def fetch_json(url: str, timeout_seconds: float) -> tuple[Any | None, str, int]:
|
||||
def fetch_json(
|
||||
url: str,
|
||||
timeout_seconds: float,
|
||||
headers: dict[str, str] | None = None,
|
||||
) -> tuple[Any | None, str, int]:
|
||||
started = time.perf_counter()
|
||||
request = Request(url, headers={"Accept": "application/json"}, method="GET")
|
||||
request = Request(url, headers={"Accept": "application/json", **(headers or {})}, method="GET")
|
||||
try:
|
||||
with urlopen(request, timeout=timeout_seconds) as response:
|
||||
payload = json.loads(response.read().decode("utf-8"))
|
||||
@@ -140,9 +145,17 @@ def fetch_json(url: str, timeout_seconds: float) -> tuple[Any | None, str, int]:
|
||||
return None, f"{type(exc).__name__}: {exc}", elapsed_ms
|
||||
|
||||
|
||||
def validate_schema_check(service: str, base_url: str, timeout_seconds: float, output_dir: Path) -> GateCheck:
|
||||
def validate_schema_check(
|
||||
service: str,
|
||||
base_url: str,
|
||||
timeout_seconds: float,
|
||||
output_dir: Path,
|
||||
*,
|
||||
headers: dict[str, str] | None = None,
|
||||
allowed_warning_checks: set[str] | None = None,
|
||||
) -> GateCheck:
|
||||
url = f"{normalize_base_url(base_url)}/api/v1/runtime/schema-check"
|
||||
payload, error, elapsed_ms = fetch_json(url, timeout_seconds)
|
||||
payload, error, elapsed_ms = fetch_json(url, timeout_seconds, headers)
|
||||
artifact = output_dir / f"{service}-schema-check.json"
|
||||
artifact.write_text(json.dumps(payload or {"error": error}, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
if payload is None:
|
||||
@@ -158,6 +171,12 @@ def validate_schema_check(service: str, base_url: str, timeout_seconds: float, o
|
||||
for item in payload.get("checks", [])
|
||||
if item.get("status") == "error"
|
||||
]
|
||||
warning_checks = {
|
||||
item.get("name", "unknown")
|
||||
for item in payload.get("checks", [])
|
||||
if item.get("status") == "warning"
|
||||
}
|
||||
allowed_warnings = allowed_warning_checks or set()
|
||||
if payload.get("status") == "ok" and not failed_checks:
|
||||
return GateCheck(
|
||||
key=f"{service}_schema_check",
|
||||
@@ -166,6 +185,24 @@ def validate_schema_check(service: str, base_url: str, timeout_seconds: float, o
|
||||
detail="Runtime schema-check returned status=ok.",
|
||||
meta={"url": url, "elapsed_ms": elapsed_ms, "artifact": str(artifact)},
|
||||
)
|
||||
if (
|
||||
payload.get("status") == "warning"
|
||||
and not failed_checks
|
||||
and warning_checks
|
||||
and warning_checks <= allowed_warnings
|
||||
):
|
||||
return GateCheck(
|
||||
key=f"{service}_schema_check",
|
||||
label=f"{service} schema-check",
|
||||
status="pass",
|
||||
detail=f"Runtime schema-check returned only allowed warnings: {','.join(sorted(warning_checks))}.",
|
||||
meta={
|
||||
"url": url,
|
||||
"elapsed_ms": elapsed_ms,
|
||||
"artifact": str(artifact),
|
||||
"allowed_warning_checks": sorted(allowed_warnings),
|
||||
},
|
||||
)
|
||||
detail = f"Runtime schema-check status={payload.get('status')}; failed checks={','.join(failed_checks) or '-'}."
|
||||
return GateCheck(
|
||||
key=f"{service}_schema_check",
|
||||
@@ -343,6 +380,14 @@ def run_benchmark(args: argparse.Namespace, artifact_dir: Path) -> tuple[Path |
|
||||
str(args.timeout_seconds),
|
||||
"--artifact-dir",
|
||||
str(benchmark_dir),
|
||||
"--tenant-id",
|
||||
args.tenant_id,
|
||||
"--user-id",
|
||||
args.user_id,
|
||||
"--contract-proxy-key-env",
|
||||
args.contract_proxy_key_env,
|
||||
"--ai-service-key-env",
|
||||
args.ai_service_key_env,
|
||||
]
|
||||
if args.include_heavy_explain:
|
||||
command.append("--include-heavy-explain")
|
||||
@@ -440,11 +485,45 @@ def parse_args() -> argparse.Namespace:
|
||||
parser.add_argument("--benchmark-concurrency", type=int, default=6)
|
||||
parser.add_argument("--benchmark-warmup", type=int, default=3)
|
||||
parser.add_argument("--include-heavy-explain", action="store_true")
|
||||
parser.add_argument("--tenant-id", default="default")
|
||||
parser.add_argument("--user-id", default="user-current-admin")
|
||||
parser.add_argument("--contract-proxy-key-env", default="CONTRACT_API_WEB_PROXY_SHARED_SECRET")
|
||||
parser.add_argument("--ai-service-key-env", default="AI_PLATFORM_SERVICE_SHARED_SECRET")
|
||||
parser.add_argument(
|
||||
"--allow-ai-provider-warning",
|
||||
action="store_true",
|
||||
help="Allow only the AI provider_runtime warning for an intentional smoke fallback environment.",
|
||||
)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def request_headers(tenant_id: str, user_id: str, auth_header: str, secret: str) -> dict[str, str]:
|
||||
headers: dict[str, str] = {}
|
||||
if tenant_id.strip():
|
||||
headers["X-Tenant-Id"] = tenant_id.strip()
|
||||
if user_id.strip():
|
||||
headers["X-User-Id"] = user_id.strip()
|
||||
if secret:
|
||||
headers[auth_header] = secret
|
||||
return headers
|
||||
|
||||
|
||||
def main() -> int:
|
||||
args = parse_args()
|
||||
contract_proxy_key = os.getenv(args.contract_proxy_key_env, "").strip()
|
||||
ai_service_key = os.getenv(args.ai_service_key_env, "").strip()
|
||||
contract_headers = request_headers(
|
||||
args.tenant_id,
|
||||
args.user_id,
|
||||
"X-Contract-Proxy-Key",
|
||||
contract_proxy_key,
|
||||
)
|
||||
ai_headers = request_headers(
|
||||
args.tenant_id,
|
||||
args.user_id,
|
||||
"X-AI-Platform-Service-Key",
|
||||
ai_service_key,
|
||||
)
|
||||
artifact_dir = resolve_path(args.artifact_dir) if args.artifact_dir else DEFAULT_ARTIFACT_ROOT / now_stamp()
|
||||
artifact_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
@@ -459,8 +538,25 @@ def main() -> int:
|
||||
)
|
||||
|
||||
if not args.skip_schema_check:
|
||||
checks.append(validate_schema_check("contract_api", args.contract_api_base, args.timeout_seconds, artifact_dir))
|
||||
checks.append(validate_schema_check("ai_platform_api", args.ai_api_base, args.timeout_seconds, artifact_dir))
|
||||
checks.append(
|
||||
validate_schema_check(
|
||||
"contract_api",
|
||||
args.contract_api_base,
|
||||
args.timeout_seconds,
|
||||
artifact_dir,
|
||||
headers=contract_headers,
|
||||
)
|
||||
)
|
||||
checks.append(
|
||||
validate_schema_check(
|
||||
"ai_platform_api",
|
||||
args.ai_api_base,
|
||||
args.timeout_seconds,
|
||||
artifact_dir,
|
||||
headers=ai_headers,
|
||||
allowed_warning_checks={"provider_runtime"} if args.allow_ai_provider_warning else set(),
|
||||
)
|
||||
)
|
||||
else:
|
||||
checks.append(
|
||||
GateCheck(
|
||||
@@ -505,6 +601,11 @@ def main() -> int:
|
||||
"run_benchmark": args.run_benchmark,
|
||||
"max_smoke_summary_age_hours": args.max_smoke_summary_age_hours,
|
||||
"max_benchmark_summary_age_hours": args.max_benchmark_summary_age_hours,
|
||||
"tenant_id": args.tenant_id,
|
||||
"user_id": args.user_id,
|
||||
"contract_proxy_auth_configured": bool(contract_proxy_key),
|
||||
"ai_service_auth_configured": bool(ai_service_key),
|
||||
"allow_ai_provider_warning": args.allow_ai_provider_warning,
|
||||
},
|
||||
"sources": {
|
||||
"smoke_summary": str(smoke_summary) if smoke_summary else None,
|
||||
|
||||
Reference in New Issue
Block a user