diff --git a/CURRENT_STATUS.md b/CURRENT_STATUS.md index f2d3427..b32243d 100644 --- a/CURRENT_STATUS.md +++ b/CURRENT_STATUS.md @@ -28,7 +28,7 @@ Rust 侧官方日服资源链路已经从实验验证推进到正式入口: 11. 官方同步会拒绝危险输出目录、路径逃逸和现有 symlink 路径组件;下载目标、`.part`、manifest、snapshot、PID、status、log 和控制锁文件不会跟随 symlink,daemon 状态类文件默认以 `0600` 权限创建。 12. `/official-version-state.json` 会明确保存当前已完成版本、正在拉取版本、上一个可用版本和失败版本;同一 app version、bundle version 和 Addressables root 的失败只保留最新一条,同一版本开始重新拉取或后续发布成功时会清理对应失败记录;`bat status` 会显示最后成功时间、下次检查时间、最后错误摘要、当前阶段、当前下载 URL 进度、版本状态摘要、最近历史失败版本和原因、结构化日志路径和轮转日志路径,人类可读输出不会把完整版本状态 JSON 内联打印。 13. 资源导入链路已支持 CAS + `ResourceRepository` 索引写入,官方同步可用 `--import-repository` / `BAT_IMPORT_REPOSITORY=1` 在已校验 release 发布后触发导入,默认 CAS 路径为 `/.cas`、SQLite 索引为 `/resources.sqlite`,也可通过 `--import-cas-root`、`--import-resource-db`、`BAT_IMPORT_CAS_ROOT`、`BAT_IMPORT_RESOURCE_DB` 覆盖;`resource.index` RPC/CLI 可按类型、hash、路径模式、官方 release ID、平台、destination、bundle path、archive entry、parse status 和 TextUnit format 分页查询现有索引,release、平台、bundle path 和常用数组 metadata 过滤已下推到 SQLite,数据库不存在时返回 `available=false` 且不会创建空库;`bat doctor cas` 可只读检查既有 CAS 根目录、对象目录、元数据库文件和对象统计,不会因诊断创建空库。`Resource` metadata 已通过 `metadata_json` 兼容迁移保存 release、平台、bundle path、parse status、TextAsset 名称和 TextUnit 数量/格式;当前/上一个/结构变化 catalog、失败 staging 复用、403/404、hash mismatch、CRC、metadata 迁移与 UnityFS 边界校验均有离线回归 fixture 或单测覆盖。 -14. 非 dry-run 官方同步在校验完成并发布后,会先对比上一完整 release 与当前 release 的 `official-download-manifest.json`,在当前 release 下写入 `official-resource-changes.json` 和 `crowdin-translation-handoff.json`;同一 destination 只有 size 或 BLAKE3 改变才算 modified,仅 URL/CDN 根变化但内容相同不会触发解析/翻译候选。随后刷新 `official-parse-cache.json` 和 `official-textunit-index.json`,并从 Added/Modified 资源、parse cache 与 TextUnit 明细索引派生 `official-textunit-tasks.json` 和 `crowdin-textunit-queue.json`;删除资源只进入差异记录,不进入 TextUnit/Crowdin 队列。`parse.text_units` 和 `parse.errors` RPC/CLI 可按 destination、archive entry、path id、class id、field path 和 format 查询当前 release 的 TextUnit 明细与解析错误;`translation.tasks` RPC/CLI 可按 release、destination、archive entry、任务状态、parse status、TextUnit format 和 reason presence 查询离线 TextUnit 翻译任务状态与跳过/失败原因;`translation.task.update` 可回写 provider worker 状态,`translation.worker.run` 可触发 Rust provider worker 独立 claim/lease/retry 并落库 TextUnit 级译文结果,`translation.proofread` 可把汉化 workflow 标记为人工校对中;TextUnit 已包含 class id、field path、字段 offset/byte size 等可追溯定位。Crowdin provider 通过 `CROWDIN_*` 环境变量接入,mock provider 支持本地 fixture;up-to-date 轮询在已有有效解析缓存、TextUnit 明细索引和 TextUnit 队列时只读取摘要,不重复解析。官方同步报告默认 `localized_release_status=not_localized`,含义是原版资源已经发布、汉化资源未发布;受支持的 UnityFS TextAsset、TypeTree string field 和 managed-reference string field 发布成功后会写带 trace 的 `localized-patch-manifest.json`,校验 hash/size/diff/rollback 后才允许 `localized.status` 返回 `status=published`、`status_code=localized.published` 和 `localized_release_status=localized`,并可用 `localized.rollback` 显式恢复上一 release。`bat` 首次启动会在二进制所在目录释放 `config.toml.example` 配置模板(`0600`),`config.toml` 存在且 Unix 权限为 `0600` 或更严格时读取并使用它;`config.toml` 不存在时仅保留模板,不自动读取 example,运行时继续使用环境变量和内置默认值。优先级为命令行参数 > 进程环境变量 > `config.toml` > 内置默认值,`BAT_SKIP_ENV_FILE` 已废弃且不再影响启动;Redis 键为预留。daemon 任务历史持久化在 `/bat-tasks.json`(版本化、`0600` 原子写),重启后任务经 `task.*` 仍可查,中断任务标记 `task_interrupted`(`BAT-ERR-700005`)。官方同步报告还分别统计当前 manifest 复用、历史 release 复用、CAS 复用、网络传输字节和复用回退诊断,下载事件状态使用 `release_reused`、`cas_reused`、`downloaded` 等稳定值。 +14. 非 dry-run 官方同步在校验完成并发布后,会先对比上一完整 release 与当前 release 的 `official-download-manifest.json`,在当前 release 下写入 `official-resource-changes.json` 和 `crowdin-translation-handoff.json`;同一 destination 只有 size 或 BLAKE3 改变才算 modified,仅 URL/CDN 根变化但内容相同不会触发解析/翻译候选。随后刷新 `official-parse-cache.json` 和 `official-textunit-index.json`,并从 Added/Modified 资源、parse cache 与 TextUnit 明细索引派生 `official-textunit-tasks.json` 和 `crowdin-textunit-queue.json`;删除资源只进入差异记录,不进入 TextUnit/Crowdin 队列。`parse.text_units` 和 `parse.errors` RPC/CLI 可按 destination、archive entry、path id、class id、field path 和 format 查询当前 release 的 TextUnit 明细与解析错误;`translation.tasks` RPC/CLI 可按 release、destination、archive entry、任务状态、parse status、TextUnit format 和 reason presence 查询离线 TextUnit 翻译任务状态与跳过/失败原因;`translation.task.update` 可回写 provider worker 状态,`translation.worker.run` 可触发 Rust provider worker 独立 claim/lease/retry 并落库 TextUnit 级译文结果,`translation.proofread` 可把汉化 workflow 标记为人工校对中;TextUnit 已包含 class id、field path、字段 offset/byte size 等可追溯定位。Crowdin provider 通过 `CROWDIN_*` 环境变量接入,mock provider 支持本地 fixture;up-to-date 轮询在已有有效解析缓存、TextUnit 明细索引和 TextUnit 队列时只读取摘要,不重复解析。官方同步报告默认 `localized_release_status=not_localized`,含义是原版资源已经发布、汉化资源未发布;受支持的 UnityFS TextAsset、TypeTree string field 和 managed-reference string field 发布成功后会写带 trace 的 `localized-patch-manifest.json`,校验 hash/size/diff/rollback 后才允许 `localized.status` 返回 `status=published`、`status_code=localized.published` 和 `localized_release_status=localized`,并可用 `localized.rollback` 显式恢复上一 release。Rust `bat` 与 Go `bat-api` 首次启动都只会在二进制所在目录释放共享 `config.toml.example` 配置模板(`0600`);存在且 Unix 权限为 `0600` 或更严格的 `config.toml` 时读取,其中 Rust 消费自身 section、Go 消费 `[api]`,不读取或生成二进制旁 `.env`;`config.toml` 不存在时运行时继续使用进程环境变量和内置默认值。优先级为命令行参数 > 进程环境变量 > `config.toml` > 内置默认值。systemd `EnvironmentFile` 和 Docker Compose `.env` 仅作为外部进程环境注入。Redis 键为预留。daemon 任务历史持久化在 `/bat-tasks.json`(版本化、`0600` 原子写),重启后任务经 `task.*` 仍可查,中断任务标记 `task_interrupted`(`BAT-ERR-700005`)。官方同步报告还分别统计当前 manifest 复用、历史 release 复用、CAS 复用、网络传输字节和复用回退诊断,下载事件状态使用 `release_reused`、`cas_reused`、`downloaded` 等稳定值。 15. Rust `bat` 已提供 `res` / `parse` / `i18n` 工作流入口:支持资源拉取、解析刷新、可再生解析缓存清理、离线翻译工作台、人工文本查看/修改/清空、工作台发布前校验、generic manifest 驱动的 Binary/JSON/Text/受支持 UnityFS 汉化发布、人工校对状态标记、既有 patch 能力的批量重打包、单次/限定次数/周期执行和版本化 schedule CRUD。`translation.worker.run` 已接入 provider worker:默认并发 8、范围 `1..=256`,每个 worker 独立 claim 下一项任务并落库 lease、失败分类、重试计划和 TextUnit 译文结果。schedule 查询现在按一级工作流过滤,删除/执行会校验作用域,单轮执行可限制计划数;schedule CRUD、翻译任务查询/交接视图、翻译任务状态回写、provider worker 触发和 `translation.proofread` 状态标记已通过 `bat.sock` 的 RPC 以及 `bat-api` 的鉴权管理接口暴露,dashboard 不维护第二套状态。`bat-api` 已提供内嵌 dashboard MVP,静态资产由 Go embed 暴露在 `/admin/dashboard/`,页面直接调用已有鉴权接口控制资源、调度、翻译、任务、日志、parse TextUnit 查询和 localized 发布/回滚。该工作流只编排已有解析和 patch 能力,不扩大解析器覆盖;完整 AssetBundle 重打包和完整 Web 协作后台仍是后续工作。真实官方网络全量拉取 smoke 已固化,真实大文件与运行报告默认在 `/tmp` 隔离目录,不纳入 Git。Go 细节见 `docs/reports/GO_STATUS.md`。 diff --git a/README.md b/README.md index ea44e85..67f2f6a 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@ - `bat-infrastructure` CAS 适配层、SQLite Resource Repository、资源导入服务、官方资源 pull/update 服务。 - `bat`:官方资源自动发现、全量拉取、原子发布到 `current -> versions/`、本地 manifest audit/repair、`.part` 断点续传、403/404/5xx 分类重试、指数退避、默认并发 8(可配置 `1..=256`,report 按 plan 顺序、进度按完成数单调上报)、已发布历史 release 与 CAS 复用、下载 quarantine 诊断、ZIP 结构校验、官方 seed `.hash` 校验、snapshot/cache、版本化 `official-launcher-bootstrap.json`、`--watch` 常驻更新、`--daemon` 后台运行,以及 Unix socket JSON-RPC live control/backend 方法(`daemon.*`、`resource.*`、`parse.*`、`translation.tasks/handoff/task.update/worker.run`、`translation.memory.*`、`translation.glossary.*`、`localized.status`、`catalog.*`、`task.*`、`patch.apply`、`unityfs.patch_*`)。 - `internal/backendrpc`:Go 侧 typed Unix socket JSON-RPC client,是 `bat-api` 调用 Rust daemon 的默认路径。 -- `cmd/bat-api`:资源 bootstrap + 分发 HTTP MVP(G-009);`/v1/bootstrap` 和 `/v1/launcher/bootstrap` 组织 `bat` 已发布 release 的启动前资源入口,launcher 形状兼容端点仅输出资源 metadata / GameMainConfig 引导,`/healthz` 暴露 RPC refresh 诊断,`/readyz` 做 release readiness,CDN path 支持 `GET`/`HEAD`/`Range`、ETag、Last-Modified 和缓存头;玩家-facing 控制面已具备 token 鉴权、限流、访问日志、反代 IP 适配、动态 JSON no-store、OpenAPI、管理控制白名单、task/log/parse/translation/TM admin 查询控制入口和无构建内嵌 dashboard;`.env` 配置端口/RPC socket/刷新周期;生产资源根和长期状态来自 RPC,不负责自动拉取。 +- `cmd/bat-api`:资源 bootstrap + 分发 HTTP MVP(G-009);`/v1/bootstrap` 和 `/v1/launcher/bootstrap` 组织 `bat` 已发布 release 的启动前资源入口,launcher 形状兼容端点仅输出资源 metadata / GameMainConfig 引导,`/healthz` 暴露 RPC refresh 诊断,`/readyz` 做 release readiness,CDN path 支持 `GET`/`HEAD`/`Range`、ETag、Last-Modified 和缓存头;玩家-facing 控制面已具备 token 鉴权、限流、访问日志、反代 IP 适配、动态 JSON no-store、OpenAPI、管理控制白名单、task/log/parse/translation/TM admin 查询控制入口和无构建内嵌 dashboard;与 Rust `bat` 共享二进制旁的 `config.toml`,`[api]` 配置端口/RPC socket/刷新周期,`BAT_API_*` 进程环境变量和 CLI 可覆盖配置;应用不会读取或生成 `.env`;生产资源根和长期状态来自 RPC,不负责自动拉取。 - Go 边界权威说明:[`docs/reports/GO_STATUS.md`](docs/reports/GO_STATUS.md)(同步 CLI = Rust `bat`)。 - 官方同步会维护 `/official-version-state.json`,明确记录当前已完成版本、正在拉取版本、上一个可用版本和失败版本。 - 资源导入链路可配置为在官方 release 发布后写入 CAS + `ResourceRepository`,资源 metadata 会记录 release、平台、bundle path、parse status、TextAsset 名称和 TextUnit 数量/格式,TextAsset/Table/Media 会按类型分类索引;`resource.index` RPC/CLI 可按类型、hash、路径模式、官方 release ID、平台、destination、bundle path、archive entry、parse status 和 TextUnit format 分页查询索引,常用 metadata 过滤会下推到 SQLite;历史 release 复用会重新校验 size、BLAKE3 和 ZIP 结构,失败时按历史 release、CAS、网络顺序回退,CAS 引用记录在 `official-cas-reuse-references.json` 中;`bat doctor cas` 可只读诊断既有 CAS 目录、对象数、对象字节数和元数据库文件状态。 diff --git a/USERGUIDE.md b/USERGUIDE.md index a831913..80aa845 100644 --- a/USERGUIDE.md +++ b/USERGUIDE.md @@ -112,7 +112,7 @@ bat-api \ 开发环境不能本地全量运行 `bat` 时,用 fixture 验证 Go 服务面即可: ```bash -BAT_API_SKIP_ENV_FILE=1 go run ./cmd/bat-api \ +go run ./cmd/bat-api \ --listen 127.0.0.1:18080 \ --public-base-url http://127.0.0.1:18080 \ --resource-root internal/api/testdata/release \ @@ -293,9 +293,9 @@ curl -i -H 'Range: bytes=0-1023' \ 由于 `config.toml` 可能包含代理凭据,Unix 下实际 `config.toml` 必须保持 `0600` 或更严格;权限过宽时程序会拒绝读取。 - 优先级:**命令行参数 > 进程环境变量 > `config.toml` > 内置默认值**。 -- `config.toml` 的字段按职责分组:`[runtime]`、`[resource]`、`[localized]`、`[repository]`、`[network]`、`[translation.worker]`。 +- `config.toml` 是 Rust `bat` 与 Go `bat-api` 的共享应用配置文件。Rust `bat` 消费 `[runtime]`、`[resource]`、`[localized]`、`[repository]`、`[network]`、`[translation.worker]`;Go `bat-api` 消费 `[api]`。每个程序忽略另一个程序的 section,但各自拥有的 section 中未知字段会直接失败。 - 现有 `BAT_*` 环境变量仍然有效,可继续覆盖 `config.toml` 中的同名配置。 -- `BAT_SKIP_ENV_FILE` 已废弃且不再影响启动。 +- `bat-api` 同样只读取二进制旁的 `config.toml`;首次启动最多生成 `config.toml.example`,不会读取或生成二进制旁 `.env`。systemd `EnvironmentFile` 和 Docker Compose `.env` 属于外部进程环境注入,不是应用 dotenv 配置。 - 支持的环境变量:`BAT_OUTPUT`、`BAT_LOCALIZED_OUTPUT`、`BAT_STATE_DIR`、`BAT_AUTO_DISCOVER`、`BAT_WATCH`、`BAT_DAEMON`、`BAT_IMPORT_REPOSITORY`、`BAT_IMPORT_CAS_ROOT`、`BAT_IMPORT_RESOURCE_DB`、`BAT_PROXY`、`BAT_NO_PROXY`、`BAT_INTERVAL_SECONDS`、`BAT_ERROR_RETRY_SECONDS`、`BAT_APP_VERSION`、`BAT_CONNECTION_GROUP`、`BAT_LAUNCHER_VERSION`、`BAT_PLATFORMS`、`BAT_CURL`、`BAT_DOWNLOAD_CONCURRENCY`、`BAT_UNZIP`、`BAT_ZIP`、`BAT_JSON`、`BAT_QUIET_UP_TO_DATE`、`BAT_TRANSLATION_PROVIDER`、`BAT_TRANSLATION_FIXTURE`、`BAT_TRANSLATION_MEMORY_PATH`、`BAT_GLOSSARY_PATH`、`BAT_TRANSLATION_CONCURRENCY`、`BAT_TRANSLATION_MAX_ATTEMPTS`、`BAT_TRANSLATION_LEASE_SECONDS`、`BAT_TRANSLATION_RETRY_BACKOFF_SECONDS`、`BAT_TRANSLATION_MAX_TASKS`、`BAT_TRANSLATION_WORKER_ID`;也可以直接写 `HTTPS_PROXY` 等通用环境变量(走现有代理自动检测)。布尔值支持 `1/0/true/false/yes/no/on/off`。 - `BAT_WATCH` / `BAT_DAEMON` 只对无子命令的 `bat` 生效(两者同时为 `1` 时 daemon 优先);命令行显式传入 `--watch` / `--daemon` / `--dry-run` 时运行模式设置让位。`status` / `verify` 等子命令不受它们影响。 - 已运行的 daemon 不会热读 `config.toml`;默认 `reload` 只唤醒后台重新发现和刷新。需要应用配置文件变更时,使用带显式启动参数的 `restart`/`reload`,或先 `stop` 再重新启动 daemon。 diff --git a/cmd/bat-api/.env.example b/cmd/bat-api/.env.example deleted file mode 100644 index 85c6721..0000000 --- a/cmd/bat-api/.env.example +++ /dev/null @@ -1,32 +0,0 @@ -# bat-api configuration example (copy to .env next to the binary or export) -# Priority: CLI flags > process environment > .env > built-in defaults. -# -# Boundary: -# - Rust bat: resource auto-discover / pull / verify / publish / daemon RPC -# - bat-api: resource bootstrap + read-only distribution (official CDN-shaped paths) -# + management APIs - -BAT_API_LISTEN=:18080 -BAT_API_PUBLIC_BASE_URL=http://127.0.0.1:18080 - -# Primary discovery: bat daemon JSON-RPC socket file -BAT_API_STATE_DIR=/tmp/bat-pid -# BAT_API_SOCKET=/tmp/bat-pid/bat.sock - -# Optional release root override (local fixtures / emergency read-only diagnostics only). -# Production obtains resource_root from BAT_API_SOCKET RPC; do not set this there. -# BAT_API_RESOURCE_ROOT= - -# BAT_API_SERVER_INFO_FILE= -BAT_API_REQUIRE_INDEXED=true -BAT_API_VERIFY_SIZE=true -BAT_API_RPC_TIMEOUT=30s -# Periodically re-read bat.sock so bat-api follows Rust bat release switches. -# Set to 0 in fixture-only local development. -BAT_API_REFRESH_INTERVAL=1m - -# Reserved for future API persistence -# BAT_API_DATABASE_URL=postgres://bat:@127.0.0.1:5432/bat?sslmode=disable -# BAT_API_DATABASE_PASSWORD= -# BAT_API_REDIS_URL=redis://127.0.0.1:6379/0 -# BAT_API_REDIS_PASSWORD= diff --git a/cmd/bat-api/config.toml.example b/cmd/bat-api/config.toml.example new file mode 100644 index 0000000..13dc0bf --- /dev/null +++ b/cmd/bat-api/config.toml.example @@ -0,0 +1,88 @@ +# BlueArchive Toolkit shared application configuration. +# Priority: CLI flags > process environment > config.toml > built-in defaults. +# This file is the application configuration. The application never reads .env. +# +# Rust bat consumes [runtime], [resource], [localized], [repository], [network] +# and [translation.worker]. Go bat-api consumes [api]. Each binary ignores the +# other application's section. + +[runtime] +state_dir = '/tmp/bat-pid' +interval_seconds = 3600 +error_retry_seconds = 60 +quiet_up_to_date = false +output_format = 'human' +banner = true +progress = true +tail_lines = 200 + +[resource] +output_root = './bat-resources' +auto_discover = true +app_version = '' +connection_group = '' +launcher_version = '1.7.2' +platforms = ['windows', 'android'] +snapshot_path = '' +dry_run = false +plan = false +force = false +audit_local = true +repair = true + +[resource.server_info] +kind = 'none' +value = '' + +[localized] +output_root = './bat-localized' + +[repository] +import_repository = false +import_cas_root = '' +import_resource_repository_path = '' + +[network] +curl_command = 'curl' +proxy = 'auto' +unzip_command = 'unzip' +zip_command = 'zip' +download_concurrency = 8 + +[translation.worker] +provider = 'mock' +fixture = '' +translation_memory_path = '' +glossary_path = '' +concurrency = 8 +max_attempts = 3 +lease_seconds = 300 +retry_backoff_seconds = 5 +max_tasks = '' +worker_id = '' + +[api] +listen = ':18080' +public_base_url = 'http://127.0.0.1:18080' +state_dir = '/tmp/bat-pid' +socket_path = '' +resource_root = '' +server_info_file = '' +require_indexed = true +verify_size = true +rpc_timeout = '30s' +refresh_interval = '1m' +auth_token = '' +auth_query_param = 'bat_token' +auth_exempt_paths = [] +trust_proxy_headers = false +access_log = false +rate_limit_rps = 0 +rate_limit_burst = 0 +max_resource_page_limit = 1000 + +# Reserved for a future API persistence layer. +database_url = '' +database_password = '' +redis_url = '' +redis_password = '' diff --git a/cmd/bat-api/main.go b/cmd/bat-api/main.go index c1702f2..b96c77c 100644 --- a/cmd/bat-api/main.go +++ b/cmd/bat-api/main.go @@ -4,8 +4,9 @@ // - bat (Rust): official resource auto-discover, pull, verify, publish, daemon RPC // - bat-api (Go): startup resource bootstrap, server-info rewrite, // read-only distribution of published resources (CDN-shaped paths), release -// inspection APIs, and normal process configuration (.env / flags for listen -// port, RPC socket, reserved database/redis settings) +// inspection APIs, and normal process configuration (config.toml / +// process environment / flags for listen port, RPC socket, reserved +// database/redis settings) // // bat-api discovers and periodically refreshes the current release through the // bat.sock JSON-RPC contract (daemon.status first, then daemon.doctor, then @@ -16,7 +17,6 @@ package main import ( "context" "flag" - "fmt" "log" "os" "os/signal" @@ -34,53 +34,19 @@ func main() { log.SetPrefix("bat-api ") cfg := api.DefaultConfig() - if os.Getenv("BAT_API_SKIP_ENV_FILE") != "1" { - envPath := envFilePath() - if err := ensureEnvTemplate(envPath); err != nil { - log.Printf("warn: env template: %v", err) - } - if err := api.LoadEnvFile(envPath); err != nil { - log.Fatalf("load .env: %v", err) - } + if err := api.LoadConfigFromCurrentExe(&cfg); err != nil { + log.Fatalf("load config.toml: %v", err) } api.ApplyEnv(&cfg) - listen := flag.String("listen", cfg.Listen, "HTTP listen address") - publicBase := flag.String("public-base-url", cfg.PublicBaseURL, "public base URL for Addressables rewrite") - stateDir := flag.String("state-dir", cfg.StateDir, "bat daemon state dir (derives default socket)") - socket := flag.String("socket", cfg.SocketPath, "path to bat.sock JSON-RPC socket (primary discovery)") - resourceRoot := flag.String("resource-root", cfg.ResourceRoot, "override published release root (tests/emergency)") - serverInfo := flag.String("server-info-file", cfg.ServerInfoFile, "optional server-info JSON path") - requireIndexed := flag.Bool("require-indexed", cfg.RequireIndexed, "only serve files present in the release index") - verifySize := flag.Bool("verify-size", cfg.VerifySize, "reject CDN files whose size differs from the index") - rpcTimeout := flag.Duration("rpc-timeout", cfg.RPCTimeout, "daemon RPC timeout") - refreshInterval := flag.Duration("refresh-interval", cfg.RefreshInterval, "periodic release discovery interval (0 disables)") - authQueryParam := flag.String("auth-query-param", cfg.AuthQueryParam, "query parameter accepted for token auth fallback") - authExemptPaths := flag.String("auth-exempt-paths", strings.Join(cfg.AuthExemptPaths, ","), "comma-separated auth-exempt exact paths or slash-prefixes") - trustProxyHeaders := flag.Bool("trust-proxy-headers", cfg.TrustProxyHeaders, "trust X-Forwarded-For and X-Real-IP from reverse proxy") - accessLog := flag.Bool("access-log", cfg.AccessLog, "enable per-request access logs without query strings") - rateLimitRPS := flag.Float64("rate-limit-rps", cfg.RateLimitRPS, "per-client request rate limit; 0 disables") - rateLimitBurst := flag.Int("rate-limit-burst", cfg.RateLimitBurst, "per-client rate limit burst") - maxResourceLimit := flag.Int("max-resource-limit", cfg.MaxResourcePageLimit, "maximum /v1/resources page size") - flag.Parse() - - cfg.Listen = *listen - cfg.PublicBaseURL = *publicBase - cfg.StateDir = *stateDir - cfg.SocketPath = *socket - cfg.ResourceRoot = *resourceRoot - cfg.ServerInfoFile = *serverInfo - cfg.RequireIndexed = *requireIndexed - cfg.VerifySize = *verifySize - cfg.RPCTimeout = *rpcTimeout - cfg.RefreshInterval = *refreshInterval - cfg.AuthQueryParam = *authQueryParam - cfg.AuthExemptPaths = splitFlagCSV(*authExemptPaths) - cfg.TrustProxyHeaders = *trustProxyHeaders - cfg.AccessLog = *accessLog - cfg.RateLimitRPS = *rateLimitRPS - cfg.RateLimitBurst = *rateLimitBurst - cfg.MaxResourcePageLimit = *maxResourceLimit + var err error + cfg, err = parseFlags(os.Args[1:], cfg) + if err != nil { + if err == flag.ErrHelp { + return + } + log.Fatalf("parse flags: %v", err) + } // If socket still empty after flags, derive from state-dir. if cfg.SocketPath == "" { cfg.SocketPath = filepath.Join(cfg.StateDir, "bat.sock") @@ -110,25 +76,51 @@ func main() { } } -func envFilePath() string { - exe, err := os.Executable() - if err != nil { - return api.EnvFileName - } - return filepath.Join(filepath.Dir(exe), api.EnvFileName) -} +func parseFlags(args []string, cfg api.Config) (api.Config, error) { + flags := flag.NewFlagSet("bat-api", flag.ContinueOnError) + flags.SetOutput(os.Stderr) -func ensureEnvTemplate(path string) error { - if _, err := os.Stat(path); err == nil { - return nil - } else if !os.IsNotExist(err) { - return err + listen := flags.String("listen", cfg.Listen, "HTTP listen address") + publicBase := flags.String("public-base-url", cfg.PublicBaseURL, "public base URL for Addressables rewrite") + stateDir := flags.String("state-dir", cfg.StateDir, "bat daemon state dir (derives default socket)") + socket := flags.String("socket", cfg.SocketPath, "path to bat.sock JSON-RPC socket (primary discovery)") + resourceRoot := flags.String("resource-root", cfg.ResourceRoot, "override published release root (tests/emergency)") + serverInfo := flags.String("server-info-file", cfg.ServerInfoFile, "optional server-info JSON path") + requireIndexed := flags.Bool("require-indexed", cfg.RequireIndexed, "only serve files present in the release index") + verifySize := flags.Bool("verify-size", cfg.VerifySize, "reject CDN files whose size differs from the index") + rpcTimeout := flags.Duration("rpc-timeout", cfg.RPCTimeout, "daemon RPC timeout") + refreshInterval := flags.Duration("refresh-interval", cfg.RefreshInterval, "periodic release discovery interval (0 disables)") + authToken := flags.String("auth-token", cfg.AuthToken, "HTTP management token") + authQueryParam := flags.String("auth-query-param", cfg.AuthQueryParam, "query parameter accepted for token auth fallback") + authExemptPaths := flags.String("auth-exempt-paths", strings.Join(cfg.AuthExemptPaths, ","), "comma-separated auth-exempt exact paths or slash-prefixes") + trustProxyHeaders := flags.Bool("trust-proxy-headers", cfg.TrustProxyHeaders, "trust X-Forwarded-For and X-Real-IP from reverse proxy") + accessLog := flags.Bool("access-log", cfg.AccessLog, "enable per-request access logs without query strings") + rateLimitRPS := flags.Float64("rate-limit-rps", cfg.RateLimitRPS, "per-client request rate limit; 0 disables") + rateLimitBurst := flags.Int("rate-limit-burst", cfg.RateLimitBurst, "per-client rate limit burst") + maxResourceLimit := flags.Int("max-resource-limit", cfg.MaxResourcePageLimit, "maximum /v1/resources page size") + if err := flags.Parse(args); err != nil { + return cfg, err } - if err := os.WriteFile(path, []byte(api.EnvTemplate), 0o600); err != nil { - return fmt.Errorf("write %s: %w", path, err) - } - log.Printf("wrote config template %s", path) - return nil + + cfg.Listen = *listen + cfg.PublicBaseURL = *publicBase + cfg.StateDir = *stateDir + cfg.SocketPath = *socket + cfg.ResourceRoot = *resourceRoot + cfg.ServerInfoFile = *serverInfo + cfg.RequireIndexed = *requireIndexed + cfg.VerifySize = *verifySize + cfg.RPCTimeout = *rpcTimeout + cfg.RefreshInterval = *refreshInterval + cfg.AuthToken = *authToken + cfg.AuthQueryParam = *authQueryParam + cfg.AuthExemptPaths = splitFlagCSV(*authExemptPaths) + cfg.TrustProxyHeaders = *trustProxyHeaders + cfg.AccessLog = *accessLog + cfg.RateLimitRPS = *rateLimitRPS + cfg.RateLimitBurst = *rateLimitBurst + cfg.MaxResourcePageLimit = *maxResourceLimit + return cfg, nil } func splitFlagCSV(raw string) []string { diff --git a/cmd/bat-api/main_test.go b/cmd/bat-api/main_test.go new file mode 100644 index 0000000..848fa68 --- /dev/null +++ b/cmd/bat-api/main_test.go @@ -0,0 +1,21 @@ +package main + +import ( + "testing" + + "bat-api/internal/api" +) + +func TestParseFlagsOverridesEnvironmentConfig(t *testing.T) { + cfg := api.DefaultConfig() + cfg.Listen = ":19080" + cfg.AuthToken = "env-token" + + got, err := parseFlags([]string{"--listen", ":19082", "--auth-token", "cli-token"}, cfg) + if err != nil { + t.Fatal(err) + } + if got.Listen != ":19082" || got.AuthToken != "cli-token" { + t.Fatalf("CLI did not override lower-priority values: %+v", got) + } +} diff --git a/deployments/systemd/bat-api.env.example b/deployments/systemd/bat-api.env.example index 7c34d84..4500ebe 100644 --- a/deployments/systemd/bat-api.env.example +++ b/deployments/systemd/bat-api.env.example @@ -1,9 +1,11 @@ -# Optional overrides for bluearchive-toolkit-bat-api.service. +# Optional process-environment overrides for bluearchive-toolkit-bat-api.service. # # Install as: # sudo install -o root -g root -m 0644 deployments/systemd/bat-api.env.example /etc/bluearchive-toolkit/bat-api.env # # Production contract: +# - systemd injects these values as process environment variables through +# EnvironmentFile; bat-api does not read this file as dotenv. # - bat-api runs in the same server/container environment as Rust bat. # - The current resource_root comes from bat.sock RPC. # - Do not set BAT_API_RESOURCE_ROOT in production; it is only for local @@ -21,7 +23,6 @@ BAT_API_REQUIRE_INDEXED=true BAT_API_VERIFY_SIZE=true BAT_API_RPC_TIMEOUT=30s BAT_API_REFRESH_INTERVAL=1m -BAT_API_SKIP_ENV_FILE=1 BAT_API_AUTH_QUERY_PARAM=bat_token # BAT_API_AUTH_TOKEN= # BAT_API_AUTH_EXEMPT_PATHS=/healthz,/readyz diff --git a/deployments/systemd/bluearchive-toolkit-bat-api.service b/deployments/systemd/bluearchive-toolkit-bat-api.service index e9f594e..024b34f 100644 --- a/deployments/systemd/bluearchive-toolkit-bat-api.service +++ b/deployments/systemd/bluearchive-toolkit-bat-api.service @@ -17,7 +17,9 @@ Environment=BAT_API_REQUIRE_INDEXED=true Environment=BAT_API_VERIFY_SIZE=true Environment=BAT_API_RPC_TIMEOUT=30s Environment=BAT_API_REFRESH_INTERVAL=1m -Environment=BAT_API_SKIP_ENV_FILE=1 +# This is an external systemd EnvironmentFile injection, not an application +# dotenv file. bat-api only reads config.toml beside its executable plus the +# process environment supplied here. EnvironmentFile=-/etc/bluearchive-toolkit/bat-api.env ExecStart=/opt/bluearchive-toolkit/bin/bat-api Restart=on-failure diff --git a/docs/architecture/official-resource-backend.md b/docs/architecture/official-resource-backend.md index b140557..62b7627 100644 --- a/docs/architecture/official-resource-backend.md +++ b/docs/architecture/official-resource-backend.md @@ -380,8 +380,9 @@ JSON-RPC 2.0 服务,是面向上层服务(Go 层)的**主要跨语言边 结果;localized 或历史分发不会绕过 Rust 完整性判断。 - 受保护的 `/admin/control/release-cleanup` 只转发 Rust `release.cleanup`; 先 dry-run 获取 `plan_id`,执行时由 Rust 重验证引用、路径和 current 保护。 - - 支持 `.env` / 环境变量配置监听端口、public base URL、RPC socket 和 RPC - 刷新周期,并预留 database/redis 键供后续 API 持久化;**不**负责资源自动拉取。 + - 支持共享 `config.toml` 的 `[api]` / 进程环境变量 / CLI 配置监听端口、public + base URL、RPC socket 和 RPC 刷新周期,并预留 database/redis 键供后续 API + 持久化;应用不会读取二进制旁 `.env`,也**不**负责资源自动拉取。 - 可选改写 server-info 中的 `AddressablesCatalogUrlRoot` 指向自身;不伪装 完整游戏业务 API。启动前资源 metadata 兼容属于资源 bootstrap;账号、登录、 Gateway、游戏业务 `ApiUrl` 和鉴权全链非本服务关闭条件。 diff --git a/docs/architecture/resource-release-layout.md b/docs/architecture/resource-release-layout.md index c6bae92..eb3fc96 100644 --- a/docs/architecture/resource-release-layout.md +++ b/docs/architecture/resource-release-layout.md @@ -400,7 +400,7 @@ Addressables 改写后客户端拼接: **不读** `bat-status.json` / `bat-tasks.json` 作为常规路径。 -生产配置:`--socket` / `BAT_API_SOCKET`,`bat-api` 与 `bat` 在同服务器、同容器或同共享文件系统环境内运行。`--resource-root` 只用于本地 fixture 或应急只读诊断,不作为生产资源根配置。`BAT_API_REFRESH_INTERVAL` 控制 bat-api 周期重读 RPC,以跟随 Rust `bat` 发布新 release。见 `cmd/bat-api/.env.example`。 +生产配置:二进制旁共享 `config.toml` 的 `[api]`,或 `--socket` / `BAT_API_SOCKET` 等 CLI/进程环境覆盖项;`bat-api` 与 `bat` 在同服务器、同容器或同共享文件系统环境内运行。`--resource-root` 只用于本地 fixture 或应急只读诊断,不作为生产资源根配置。`BAT_API_REFRESH_INTERVAL` 控制 bat-api 周期重读 RPC,以跟随 Rust `bat` 发布新 release。首次启动只生成 `config.toml.example`,应用不会读取二进制旁 `.env`;systemd `EnvironmentFile` 仍可作为外部进程环境注入。 --- diff --git a/docs/guides/deployment.md b/docs/guides/deployment.md index 370f374..70fd9d1 100644 --- a/docs/guides/deployment.md +++ b/docs/guides/deployment.md @@ -367,6 +367,18 @@ sudo ln -sfn \ 如果 Rust `bat` 和 Go `bat-api` 使用同一个 release 目录发布,也可以把二者放在同一个 `` 目录下,分别通过 `/opt/bluearchive-toolkit/bin/bat` 和 `/opt/bluearchive-toolkit/bin/bat-api` 暴露稳定 symlink。 +### 共享配置文件 + +两个二进制都从各自真实可执行文件所在目录读取同一个 `config.toml`。发布新版本时,可以从仓库模板创建并设置权限: + +```bash +sudo install -o bat -g bat -m 0600 \ + cmd/bat-api/config.toml.example \ + /opt/bluearchive-toolkit/releases/"${VERSION}"/config.toml +``` + +Rust `bat` 消费自身的 `[runtime]`、`[resource]`、`[localized]`、`[repository]`、`[network]`、`[translation.worker]`;Go `bat-api` 消费 `[api]`。每个程序忽略另一个程序的 section,但各自 section 中的未知字段会使启动失败。优先级为 `CLI > 进程环境变量 > config.toml > 内置默认值`。首次启动只生成 `config.toml.example`,应用不会读取或生成二进制旁 `.env`。 + ### bat 侧前置条件 `bat-api` 依赖 live RPC,而不是直接读取 daemon 状态文件。部署 `bat-api` 前,部署所在生产主机上应已有 socket 形态的 Rust `bat`: @@ -416,6 +428,8 @@ BAT_API_RATE_LIMIT_RPS=30 BAT_API_RATE_LIMIT_BURST=120 ``` +上面的 `bat-api.env` 是 systemd `EnvironmentFile`,systemd 将其中的 `BAT_API_*` 作为进程环境变量注入;它不是应用读取的 dotenv 文件。需要固定的共享配置应写入二进制旁 `config.toml` 的 `[api]`,secret 优先通过 systemd credential 或环境变量注入。 + 生产反向代理公开后,把 `BAT_API_PUBLIC_BASE_URL` 改成客户端实际访问的 HTTPS 根,例如: ```env @@ -466,7 +480,7 @@ curl -fsS http://127.0.0.1:18080/admin/ ```bash make test-go-api make build-go-api -BAT_API_SKIP_ENV_FILE=1 go run ./cmd/bat-api \ +go run ./cmd/bat-api \ --listen 127.0.0.1:18080 \ --public-base-url http://127.0.0.1:18080 \ --resource-root internal/api/testdata/release \ diff --git a/docs/guides/development.md b/docs/guides/development.md index ab875af..cefca83 100644 --- a/docs/guides/development.md +++ b/docs/guides/development.md @@ -167,7 +167,7 @@ Go 边界与进度以 `docs/reports/GO_STATUS.md` 为准: ```bash make test-go-api make bat-api-local-live-smoke -BAT_API_SKIP_ENV_FILE=1 go run ./cmd/bat-api \ +go run ./cmd/bat-api \ --listen 127.0.0.1:18080 \ --public-base-url http://127.0.0.1:18080 \ --resource-root internal/api/testdata/release \ diff --git a/docs/reports/GO_STATUS.md b/docs/reports/GO_STATUS.md index 0c83583..452868a 100644 --- a/docs/reports/GO_STATUS.md +++ b/docs/reports/GO_STATUS.md @@ -71,7 +71,7 @@ | ID | 约定 | |---|---| -| K | `.env` / 环境变量 / CLI:端口、public base、RPC socket、RPC 刷新周期;**预留** database/redis | +| K | 共享 `config.toml` 的 `[api]` / 进程环境变量 / CLI:端口、public base、RPC socket、RPC 刷新周期;**预留** database/redis。应用不读取二进制旁 `.env` | | L | 管理面 / bootstrap:`/healthz`、`/readyz`、`/v1/bootstrap`、`/v1/release`、`/v1/resources`、`/openapi.yaml`、`/admin/` 控制入口 | | M | CDN:`GET/HEAD /prod-clientpatch.bluearchiveyostar.com/...`,支持 Range、ETag、Last-Modified、长期缓存头 | | N | server-info 可选;**只改 AddressablesCatalogUrlRoot** | @@ -96,7 +96,7 @@ |---|---|---|---| | Module | `go.mod` → `bat-api` | 已用 | 服务层模块名 | | RPC client | `internal/backendrpc` | **完成** | Unix socket JSON-RPC transport + typed helper;typed helper 覆盖 daemon 已实现控制/查询、`resource.state/sync/verify/repair/manifest/list`、`release.attestation/status/list/distribution/cleanup`、`catalog.*`、`parse.*`、`localized.status/publish/rollback`、`task.*`、`translation.tasks`、`translation.handoff`、`translation.task.update`、`translation.worker.run`、`translation.proofread`、`translation.memory.summary/query/confirm/conflicts/resolve_conflict`、`translation.glossary.summary/query/diagnose/add/update/approve/deprecate/delete` 和文件级 UnityFS patch 调用;`resource.manifest` typed params 固定 release/publication/manifest identity 和 verification generation;`localized.publish` 的 typed params 支持 `translation_file`、`from_worker` 或 `patch_manifest` 三选一;`resource.index`、`patch.apply` 仍通过通用 `Call` 走同一 contract;fake transport 单测和 `internal/api/testdata/contract/` mirror test 固化 Rust 输出字段 | -| 资源 bootstrap/分发 | `cmd/bat-api` + `internal/api` | **MVP+生产控制面** | RPC 发现 + 周期刷新/诊断 + `/v1/bootstrap` + `/v1/launcher/bootstrap` + `/v1/releases` + `/v1/distribution` + launcher 资源 metadata 兼容 + `/readyz` + CDN Range/缓存头 + 鉴权/限流/访问日志/反代适配 + OpenAPI + 管理控制白名单 + release/localized/TM/Glossary admin forwarding + 内嵌 dashboard + `.env` | +| 资源 bootstrap/分发 | `cmd/bat-api` + `internal/api` | **MVP+生产控制面** | RPC 发现 + 周期刷新/诊断 + `/v1/bootstrap` + `/v1/launcher/bootstrap` + `/v1/releases` + `/v1/distribution` + launcher 资源 metadata 兼容 + `/readyz` + CDN Range/缓存头 + 鉴权/限流/访问日志/反代适配 + OpenAPI + 管理控制白名单 + release/localized/TM/Glossary admin forwarding + 内嵌 dashboard + 共享 `config.toml` `[api]` 配置 | | 试验 CLI | `cmd/bat` | **试验** | doctor 固定 ok;manifest/sync 走 FFI | | FFI | `internal/ffi` | **可选** | 需 `build-ffi` | | 空骨架 | `api/`、`pkg/*`、部分 `internal/*` | **空** | 见各目录 README | diff --git a/go.mod b/go.mod index 260f8cf..ced46b7 100644 --- a/go.mod +++ b/go.mod @@ -1,3 +1,5 @@ module bat-api go 1.26.4 + +require github.com/pelletier/go-toml/v2 v2.2.4 diff --git a/go.sum b/go.sum new file mode 100644 index 0000000..3cf50e1 --- /dev/null +++ b/go.sum @@ -0,0 +1,2 @@ +github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4= +github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY= diff --git a/infrastructure/src/bin/bat/app.rs b/infrastructure/src/bin/bat/app.rs index afa8454..9e4258f 100644 --- a/infrastructure/src/bin/bat/app.rs +++ b/infrastructure/src/bin/bat/app.rs @@ -7302,9 +7302,6 @@ fn parse_args_with_env( if let Some(config_file) = config_file { config_file.apply_to_options(&mut options)?; } - if env_lookup("BAT_SKIP_ENV_FILE").is_some() { - terminal_output::print_deprecated_env_file_warning(); - } // `BAT_*` 环境变量先作为默认值写入,不标记 explicit;命令行参数随后解析, // 逐字段覆盖。工具/代理的"非默认"判断以 config.toml + 环境变量后的基线为准, // 保证 status/stop/logs 在用户已配置默认值时不误判为显式传了同步参数。 diff --git a/infrastructure/src/bin/bat/config_file.rs b/infrastructure/src/bin/bat/config_file.rs index 59767e6..30265ce 100644 --- a/infrastructure/src/bin/bat/config_file.rs +++ b/infrastructure/src/bin/bat/config_file.rs @@ -75,6 +75,31 @@ lease_seconds = 300 retry_backoff_seconds = 5 max_tasks = '' worker_id = '' + +# Go bat-api consumes this shared section. Rust bat ignores it. +[api] +listen = ':18080' +public_base_url = 'http://127.0.0.1:18080' +state_dir = '/tmp/bat-pid' +socket_path = '' +resource_root = '' +server_info_file = '' +require_indexed = true +verify_size = true +rpc_timeout = '30s' +refresh_interval = '1m' +auth_token = '' +auth_query_param = 'bat_token' +auth_exempt_paths = [] +trust_proxy_headers = false +access_log = false +rate_limit_rps = 0 +rate_limit_burst = 0 +max_resource_page_limit = 1000 +database_url = '' +database_password = '' +redis_url = '' +redis_password = '' "#; #[derive(Debug, Clone, Default)] @@ -164,6 +189,7 @@ struct TranslationWorkerSection { #[derive(Debug, Clone, Copy, PartialEq, Eq)] enum SectionPath { + Api, Runtime, Resource, ResourceServerInfo, @@ -405,6 +431,9 @@ impl BatConfigFile { value: &str, line_number: usize, ) -> anyhow::Result<()> { + if section == SectionPath::Api { + return Ok(()); + } match (section, key) { (SectionPath::Runtime, "state_dir") => { self.runtime.state_dir = Some(parse_required_path( @@ -708,6 +737,7 @@ fn parse_section_header(line: &str, line_number: usize) -> anyhow::Result Ok(SectionPath::Api), ["runtime"] => Ok(SectionPath::Runtime), ["resource"] => Ok(SectionPath::Resource), ["resource", "server_info"] => Ok(SectionPath::ResourceServerInfo), @@ -1071,6 +1101,21 @@ mod tests { ); } + #[test] + fn shared_api_section_is_ignored_by_bat() { + let temp = TempDir::new().unwrap(); + write_private_config( + &temp.path().join(CONFIG_FILE_NAME), + "[api]\nunknown_api_key = 'owned-by-go'\n[resource]\noutput_root = '/srv/resources'\n", + ); + + let loaded = load_from_binary_dir(temp.path()).unwrap().unwrap(); + assert_eq!( + loaded.resource.output_root, + Some(PathBuf::from("/srv/resources")) + ); + } + #[test] fn example_config_is_never_loaded_as_actual_config() { let temp = TempDir::new().unwrap(); diff --git a/infrastructure/src/bin/bat/terminal_output.rs b/infrastructure/src/bin/bat/terminal_output.rs index 1be2f44..0765ccc 100644 --- a/infrastructure/src/bin/bat/terminal_output.rs +++ b/infrastructure/src/bin/bat/terminal_output.rs @@ -85,10 +85,6 @@ pub(super) fn print_config_template_warning(path: &Path, error: impl std::fmt::D ); } -pub(super) fn print_deprecated_env_file_warning() { - eprintln!("警告:BAT_SKIP_ENV_FILE 已废弃且不再影响启动,已忽略"); -} - #[derive(Debug, Clone)] pub(super) struct ProgressLogger { enabled: bool, diff --git a/internal/api/api_test.go b/internal/api/api_test.go index 5c91d93..597a3f6 100644 --- a/internal/api/api_test.go +++ b/internal/api/api_test.go @@ -332,7 +332,7 @@ func TestRefreshCurrentReleaseHealthTransitionsAndClearsFailure(t *testing.T) { t.Fatalf("refresh diagnostics=%v", refresh) } - if err := os.WriteFile(filepath.Join(root, "prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.hash"), []byte("1234567890"), 0o644); err != nil { + if err := os.WriteFile(filepath.Join(root, "prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.hash"), []byte("2801109426\n"), 0o644); err != nil { t.Fatal(err) } backend.releaseStatusErr = nil @@ -461,7 +461,7 @@ func TestCDNSupportsRangeHeadAndConditionalRequests(t *testing.T) { t.Fatalf("Content-Range=%q", rr.Header().Get("Content-Range")) } etag := rr.Header().Get("ETag") - if etag != `"blake3-0000000000000000000000000000000000000000000000000000000000000000"` { + if etag != `"blake3-5d5c6cc8ca0afa7d71df9b0d764c7b7f9e60c8b082dbb9fc844b10442989357f"` { t.Fatalf("ETag=%q", etag) } @@ -1802,21 +1802,6 @@ func TestDiscoverTreatsCatalogUnavailableAsNoRelease(t *testing.T) { } } -func TestLoadEnvFileDoesNotOverride(t *testing.T) { - dir := t.TempDir() - path := filepath.Join(dir, ".env") - if err := os.WriteFile(path, []byte("BAT_API_LISTEN=:9999\n"), 0o600); err != nil { - t.Fatal(err) - } - t.Setenv("BAT_API_LISTEN", ":1111") - if err := LoadEnvFile(path); err != nil { - t.Fatal(err) - } - if os.Getenv("BAT_API_LISTEN") != ":1111" { - t.Fatal(os.Getenv("BAT_API_LISTEN")) - } -} - func TestRefreshDiagnosticsAndReadyz(t *testing.T) { root := fixtureRoot(t) cfg := DefaultConfig() diff --git a/internal/api/config.go b/internal/api/config.go index 7578996..4bee288 100644 --- a/internal/api/config.go +++ b/internal/api/config.go @@ -25,14 +25,16 @@ const ( DefaultAuthSkew = 5 * time.Minute ClientPatchHost = "prod-clientpatch.bluearchiveyostar.com" ServerInfoHost = "yostar-serverinfo.bluearchiveyostar.com" - EnvFileName = ".env" + ConfigFileName = "config.toml" + ConfigExampleName = "config.toml.example" ) // Config holds bat-api process configuration. // -// Values come from (highest wins): CLI flags > process environment > .env file -// > built-in defaults. Database-related keys are reserved for future API -// persistence and are loaded but not required for the resource CDN surface. +// Values come from (highest wins): CLI flags > process environment > +// config.toml > built-in defaults. Database-related keys are reserved for +// future API persistence and are loaded but not required for the resource CDN +// surface. type Config struct { // Listen is the HTTP listen address, e.g. ":18080" or "127.0.0.1:18080". Listen string @@ -54,8 +56,6 @@ type Config struct { RPCTimeout time.Duration // RefreshInterval periodically re-discovers the release through RPC. Zero disables it. RefreshInterval time.Duration - // SkipEnvFile disables loading .env when true (BAT_API_SKIP_ENV_FILE=1). - SkipEnvFile bool // AuthToken enables HTTP token authentication when non-empty. AuthToken string // AuthQueryParam is the optional query parameter accepted for token auth. @@ -148,41 +148,6 @@ func (c *Config) Normalize() error { return nil } -// LoadEnvFile reads KEY=VALUE lines from path into the process environment -// without overriding variables that are already set. -func LoadEnvFile(path string) error { - data, err := os.ReadFile(path) - if err != nil { - if os.IsNotExist(err) { - return nil - } - return err - } - for _, line := range strings.Split(string(data), "\n") { - line = strings.TrimSpace(line) - if line == "" || strings.HasPrefix(line, "#") { - continue - } - key, value, ok := strings.Cut(line, "=") - if !ok { - continue - } - key = strings.TrimSpace(key) - value = strings.TrimSpace(value) - value = strings.Trim(value, `"'`) - if key == "" { - continue - } - if _, exists := os.LookupEnv(key); exists { - continue - } - if err := os.Setenv(key, value); err != nil { - return err - } - } - return nil -} - // ApplyEnv overlays environment variables onto cfg. func ApplyEnv(cfg *Config) { if v := os.Getenv("BAT_API_LISTEN"); v != "" { @@ -239,9 +204,6 @@ func ApplyEnv(cfg *Config) { if v := os.Getenv("BAT_API_REDIS_PASSWORD"); v != "" { cfg.RedisPassword = v } - if v := os.Getenv("BAT_API_SKIP_ENV_FILE"); v != "" { - cfg.SkipEnvFile = parseBool(v, false) - } if v := os.Getenv("BAT_API_AUTH_TOKEN"); v != "" { cfg.AuthToken = v } @@ -313,61 +275,3 @@ func normalizePathList(paths []string) []string { } return out } - -// EnvTemplate is written next to the bat-api binary on first run when missing. -const EnvTemplate = `# bat-api configuration -# Priority: CLI flags > process environment > this file > built-in defaults. -# Resource auto-pull/sync is owned by the Rust bat daemon, not bat-api. -# bat-api only bootstraps/distributes already-published resources and exposes -# a small management surface for release inspection. - -# HTTP listen address (host:port or :port) -BAT_API_LISTEN=:18080 -# Public base URL used when rewriting AddressablesCatalogUrlRoot -BAT_API_PUBLIC_BASE_URL=http://127.0.0.1:18080 - -# Rust bat daemon RPC socket (primary discovery path) -# Prefer BAT_API_SOCKET; BAT_API_STATE_DIR only derives the default socket path. -BAT_API_STATE_DIR=/tmp/bat-pid -# BAT_API_SOCKET=/tmp/bat-pid/bat.sock - -# Optional override of the published release root (fixtures / emergency only). -# Production obtains resource_root from daemon RPC (release.attestation + -# generation-bound resource.manifest). -# BAT_API_RESOURCE_ROOT= - -# Optional server-info JSON for Addressables root rewrite -# BAT_API_SERVER_INFO_FILE= - - # CDN safety - BAT_API_REQUIRE_INDEXED=true - BAT_API_VERIFY_SIZE=true - BAT_API_RPC_TIMEOUT=30s -# Periodically re-read bat.sock so bat-api follows Rust bat release switches. - # Set to 0 to disable in fixture-only local development. - BAT_API_REFRESH_INTERVAL=1m - - # Player-facing HTTP controls. - # Prefer setting BAT_API_AUTH_TOKEN through a secret manager or process - # environment. Reverse proxies may inject Authorization: Bearer or - # X-BAT-Token to authenticated upstream requests. Query token fallback uses - # BAT_API_AUTH_QUERY_PARAM and is supported for clients that cannot set headers. - # BAT_API_AUTH_TOKEN= - BAT_API_AUTH_QUERY_PARAM=bat_token - # BAT_API_AUTH_EXEMPT_PATHS= - BAT_API_TRUST_PROXY_HEADERS=false - BAT_API_ACCESS_LOG=false - # Zero disables in-process limiting; production may still enforce edge limits. - BAT_API_RATE_LIMIT_RPS=0 - BAT_API_RATE_LIMIT_BURST=0 - BAT_API_MAX_RESOURCE_LIMIT=1000 - - # Reserved for future API persistence (not required for resource CDN) -# BAT_API_DATABASE_URL=postgres://bat:@127.0.0.1:5432/bat?sslmode=disable -# BAT_API_DATABASE_PASSWORD= -# BAT_API_REDIS_URL=redis://127.0.0.1:6379/0 -# BAT_API_REDIS_PASSWORD= - -# Set to 1 to ignore this file entirely -# BAT_API_SKIP_ENV_FILE=0 -` diff --git a/internal/api/config_toml.go b/internal/api/config_toml.go new file mode 100644 index 0000000..e9f85dc --- /dev/null +++ b/internal/api/config_toml.go @@ -0,0 +1,391 @@ +package api + +import ( + "fmt" + "math" + "os" + "path/filepath" + "runtime" + "time" + + "github.com/pelletier/go-toml/v2" +) + +// ConfigTOMLTemplate is the only template generated by bat-api. Rust bat +// accepts the same file and ignores the [api] section. +const ConfigTOMLTemplate = `# BlueArchive Toolkit shared application configuration. +# Priority: CLI flags > process environment > config.toml > built-in defaults. +# This file is the application configuration. The application never reads .env. +# +# Rust bat consumes [runtime], [resource], [localized], [repository], [network] +# and [translation.worker]. Go bat-api consumes [api]. Each binary ignores the +# other application's section. + +[runtime] +state_dir = '/tmp/bat-pid' +interval_seconds = 3600 +error_retry_seconds = 60 +quiet_up_to_date = false +output_format = 'human' +banner = true +progress = true +tail_lines = 200 + +[resource] +output_root = './bat-resources' +auto_discover = true +app_version = '' +connection_group = '' +launcher_version = '1.7.2' +platforms = ['windows', 'android'] +snapshot_path = '' +dry_run = false +plan = false +force = false +audit_local = true +repair = true + +[resource.server_info] +kind = 'none' +value = '' + +[localized] +output_root = './bat-localized' + +[repository] +import_repository = false +import_cas_root = '' +import_resource_repository_path = '' + +[network] +curl_command = 'curl' +proxy = 'auto' +unzip_command = 'unzip' +zip_command = 'zip' +download_concurrency = 8 + +[translation.worker] +provider = 'mock' +fixture = '' +translation_memory_path = '' +glossary_path = '' +concurrency = 8 +max_attempts = 3 +lease_seconds = 300 +retry_backoff_seconds = 5 +max_tasks = '' +worker_id = '' + +[api] +listen = ':18080' +public_base_url = 'http://127.0.0.1:18080' +state_dir = '/tmp/bat-pid' +socket_path = '' +resource_root = '' +server_info_file = '' +require_indexed = true +verify_size = true +rpc_timeout = '30s' +refresh_interval = '1m' +auth_token = '' +auth_query_param = 'bat_token' +auth_exempt_paths = [] +trust_proxy_headers = false +access_log = false +rate_limit_rps = 0 +rate_limit_burst = 0 +max_resource_page_limit = 1000 + +# Reserved for a future API persistence layer. +database_url = '' +database_password = '' +redis_url = '' +redis_password = '' +` + +// LoadConfigFromCurrentExe loads config.toml next to the running binary. +func LoadConfigFromCurrentExe(cfg *Config) error { + exe, err := os.Executable() + if err != nil { + return fmt.Errorf("locate current executable: %w", err) + } + return LoadConfigFromBinaryDir(filepath.Dir(exe), cfg) +} + +// LoadConfigFromBinaryDir loads only config.toml from binaryDir. Missing +// config.toml causes config.toml.example to be created when possible; the +// example is never parsed as the active configuration. +func LoadConfigFromBinaryDir(binaryDir string, cfg *Config) error { + if cfg == nil { + return fmt.Errorf("config must not be nil") + } + configPath := filepath.Join(binaryDir, ConfigFileName) + examplePath := filepath.Join(binaryDir, ConfigExampleName) + info, err := os.Lstat(configPath) + if err != nil { + if !os.IsNotExist(err) { + return fmt.Errorf("stat %s: %w", configPath, err) + } + ensureConfigExample(examplePath) + return nil + } + if info.Mode()&os.ModeSymlink != 0 || !info.Mode().IsRegular() { + return fmt.Errorf("%s must be a regular file", configPath) + } + if runtime.GOOS != "windows" && info.Mode().Perm()&0o077 != 0 { + return fmt.Errorf("%s permissions must be 0600 or stricter (current %03o)", configPath, info.Mode().Perm()) + } + data, err := os.ReadFile(configPath) + if err != nil { + return fmt.Errorf("read %s: %w", configPath, err) + } + if err := applyAPITOML(data, cfg); err != nil { + return fmt.Errorf("parse %s: %w", configPath, err) + } + return nil +} + +func ensureConfigExample(path string) { + if _, err := os.Lstat(path); err == nil || !os.IsNotExist(err) { + return + } + file, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o600) + if err != nil { + return + } + if _, err := file.WriteString(ConfigTOMLTemplate); err != nil { + _ = file.Close() + return + } + _ = file.Close() +} + +func applyAPITOML(data []byte, cfg *Config) error { + var document map[string]any + if err := toml.Unmarshal(data, &document); err != nil { + return err + } + rawAPI, exists := document["api"] + if !exists { + return nil + } + apiValues, ok := rawAPI.(map[string]any) + if !ok { + return fmt.Errorf("[api] must be a table") + } + for key, value := range apiValues { + if err := applyAPIValue(cfg, key, value); err != nil { + return err + } + } + return nil +} + +func applyAPIValue(cfg *Config, key string, raw any) error { + field := fmt.Sprintf("[api].%s", key) + switch key { + case "listen": + value, err := tomlString(raw, field) + if err != nil { + return err + } + cfg.Listen = value + case "public_base_url": + value, err := tomlString(raw, field) + if err != nil { + return err + } + cfg.PublicBaseURL = value + case "state_dir": + value, err := tomlString(raw, field) + if err != nil { + return err + } + cfg.StateDir = value + case "socket_path": + value, err := tomlString(raw, field) + if err != nil { + return err + } + cfg.SocketPath = value + case "resource_root": + value, err := tomlString(raw, field) + if err != nil { + return err + } + cfg.ResourceRoot = value + case "server_info_file": + value, err := tomlString(raw, field) + if err != nil { + return err + } + cfg.ServerInfoFile = value + case "require_indexed": + value, err := tomlBool(raw, field) + if err != nil { + return err + } + cfg.RequireIndexed = value + case "verify_size": + value, err := tomlBool(raw, field) + if err != nil { + return err + } + cfg.VerifySize = value + case "rpc_timeout": + value, err := tomlDuration(raw, field) + if err != nil { + return err + } + cfg.RPCTimeout = value + case "refresh_interval": + value, err := tomlDuration(raw, field) + if err != nil { + return err + } + cfg.RefreshInterval = value + case "auth_token": + value, err := tomlString(raw, field) + if err != nil { + return err + } + cfg.AuthToken = value + case "auth_query_param": + value, err := tomlString(raw, field) + if err != nil { + return err + } + cfg.AuthQueryParam = value + case "auth_exempt_paths": + value, err := tomlStringArray(raw, field) + if err != nil { + return err + } + cfg.AuthExemptPaths = value + case "trust_proxy_headers": + value, err := tomlBool(raw, field) + if err != nil { + return err + } + cfg.TrustProxyHeaders = value + case "access_log": + value, err := tomlBool(raw, field) + if err != nil { + return err + } + cfg.AccessLog = value + case "rate_limit_rps": + value, err := tomlNonNegativeNumber(raw, field) + if err != nil { + return err + } + cfg.RateLimitRPS = value + case "rate_limit_burst": + value, err := tomlNonNegativeInt(raw, field) + if err != nil { + return err + } + cfg.RateLimitBurst = value + case "max_resource_page_limit": + value, err := tomlNonNegativeInt(raw, field) + if err != nil { + return err + } + cfg.MaxResourcePageLimit = value + case "database_url": + value, err := tomlString(raw, field) + if err != nil { + return err + } + cfg.DatabaseURL = value + case "database_password": + value, err := tomlString(raw, field) + if err != nil { + return err + } + cfg.DatabasePassword = value + case "redis_url": + value, err := tomlString(raw, field) + if err != nil { + return err + } + cfg.RedisURL = value + case "redis_password": + value, err := tomlString(raw, field) + if err != nil { + return err + } + cfg.RedisPassword = value + default: + return fmt.Errorf("unsupported [api] key %q", key) + } + return nil +} + +func tomlString(raw any, field string) (string, error) { + value, ok := raw.(string) + if !ok { + return "", fmt.Errorf("%s must be a string (got %T)", field, raw) + } + return value, nil +} + +func tomlBool(raw any, field string) (bool, error) { + value, ok := raw.(bool) + if !ok { + return false, fmt.Errorf("%s must be a boolean (got %T)", field, raw) + } + return value, nil +} + +func tomlDuration(raw any, field string) (time.Duration, error) { + value, err := tomlString(raw, field) + if err != nil { + return 0, err + } + duration, err := time.ParseDuration(value) + if err != nil || duration < 0 { + return 0, fmt.Errorf("%s must be a non-negative duration", field) + } + return duration, nil +} + +func tomlNonNegativeNumber(raw any, field string) (float64, error) { + var value float64 + switch typed := raw.(type) { + case int64: + value = float64(typed) + case float64: + value = typed + default: + return 0, fmt.Errorf("%s must be a non-negative number (got %T)", field, raw) + } + if value < 0 || math.IsNaN(value) || math.IsInf(value, 0) { + return 0, fmt.Errorf("%s must be a non-negative number", field) + } + return value, nil +} + +func tomlNonNegativeInt(raw any, field string) (int, error) { + value, ok := raw.(int64) + if !ok || value < 0 || int64(int(value)) != value { + return 0, fmt.Errorf("%s must be a non-negative integer (got %T)", field, raw) + } + return int(value), nil +} + +func tomlStringArray(raw any, field string) ([]string, error) { + values, ok := raw.([]any) + if !ok { + return nil, fmt.Errorf("%s must be an array of strings (got %T)", field, raw) + } + items := make([]string, len(values)) + for index, rawItem := range values { + item, ok := rawItem.(string) + if !ok { + return nil, fmt.Errorf("%s[%d] must be a string (got %T)", field, index, rawItem) + } + items[index] = item + } + return items, nil +} diff --git a/internal/api/config_toml_test.go b/internal/api/config_toml_test.go new file mode 100644 index 0000000..d43c581 --- /dev/null +++ b/internal/api/config_toml_test.go @@ -0,0 +1,163 @@ +package api + +import ( + "os" + "path/filepath" + "runtime" + "strings" + "testing" + "time" +) + +func writePrivateConfig(t *testing.T, dir, content string) { + t.Helper() + path := filepath.Join(dir, ConfigFileName) + if err := os.WriteFile(path, []byte(content), 0o600); err != nil { + t.Fatal(err) + } +} + +func TestLoadConfigFromBinaryDirFirstLaunchCreatesExampleOnly(t *testing.T) { + dir := t.TempDir() + cfg := DefaultConfig() + + if err := LoadConfigFromBinaryDir(dir, &cfg); err != nil { + t.Fatal(err) + } + if _, err := os.Stat(filepath.Join(dir, ConfigExampleName)); err != nil { + t.Fatalf("config.toml.example was not created: %v", err) + } + if _, err := os.Stat(filepath.Join(dir, ".env")); !os.IsNotExist(err) { + t.Fatalf("unexpected .env after first launch, err=%v", err) + } + if cfg.Listen != DefaultListen || cfg.RPCTimeout != 30*time.Second { + t.Fatalf("first launch changed defaults: %+v", cfg) + } +} + +func TestCheckedInBatAPIConfigExampleMatchesRuntimeTemplate(t *testing.T) { + _, sourceFile, _, ok := runtime.Caller(0) + if !ok { + t.Fatal("runtime.Caller failed") + } + path := filepath.Join(filepath.Dir(sourceFile), "..", "..", "cmd", "bat-api", ConfigExampleName) + data, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + if string(data) != ConfigTOMLTemplate { + t.Fatal("cmd/bat-api/config.toml.example differs from runtime template") + } +} + +func TestLoadConfigFromBinaryDirDoesNotReadDotEnv(t *testing.T) { + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, ".env"), []byte("BAT_API_LISTEN=':19999'\n"), 0o600); err != nil { + t.Fatal(err) + } + cfg := DefaultConfig() + + if err := LoadConfigFromBinaryDir(dir, &cfg); err != nil { + t.Fatal(err) + } + if cfg.Listen != DefaultListen { + t.Fatalf("dotenv value was applied: %q", cfg.Listen) + } + if _, err := os.Stat(filepath.Join(dir, ConfigExampleName)); err != nil { + t.Fatalf("config.toml.example was not created: %v", err) + } +} + +func TestLoadConfigFromBinaryDirAppliesAPISectionOnly(t *testing.T) { + dir := t.TempDir() + writePrivateConfig(t, dir, ` +[runtime] +state_dir = '/rust-state' +this_key_belongs_to_rust = 'ignored-by-go' + +[translation.worker] +provider = 'mock' + +[api] +listen = '127.0.0.1:19080' +public_base_url = 'https://assets.example.test' +state_dir = '/api-state' +socket_path = '/api-state/bat.sock' +resource_root = '/srv/releases/current' +server_info_file = '/srv/server-info.json' +require_indexed = false +verify_size = false +rpc_timeout = '2s' +refresh_interval = '0s' +auth_token = 'secret' +auth_query_param = 'token' +auth_exempt_paths = ['/healthz', '/readyz'] +trust_proxy_headers = true +access_log = true +rate_limit_rps = 12.5 +rate_limit_burst = 30 +max_resource_page_limit = 250 +database_url = 'postgres://example' +database_password = 'db-secret' +redis_url = 'redis://example' +redis_password = 'redis-secret' +`) + cfg := DefaultConfig() + + if err := LoadConfigFromBinaryDir(dir, &cfg); err != nil { + t.Fatal(err) + } + if cfg.Listen != "127.0.0.1:19080" || + cfg.PublicBaseURL != "https://assets.example.test" || + cfg.StateDir != "/api-state" || + cfg.SocketPath != "/api-state/bat.sock" || + cfg.ResourceRoot != "/srv/releases/current" || + cfg.ServerInfoFile != "/srv/server-info.json" || + cfg.RequireIndexed || + cfg.VerifySize || + cfg.RPCTimeout != 2*time.Second || + cfg.RefreshInterval != 0 || + cfg.AuthToken != "secret" || + cfg.AuthQueryParam != "token" || + cfg.TrustProxyHeaders != true || + cfg.AccessLog != true || + cfg.RateLimitRPS != 12.5 || + cfg.RateLimitBurst != 30 || + cfg.MaxResourcePageLimit != 250 || + cfg.DatabaseURL != "postgres://example" || + cfg.DatabasePassword != "db-secret" || + cfg.RedisURL != "redis://example" || + cfg.RedisPassword != "redis-secret" { + t.Fatalf("unexpected config: %+v", cfg) + } + if strings.Join(cfg.AuthExemptPaths, ",") != "/healthz,/readyz" { + t.Fatalf("auth exempt paths=%v", cfg.AuthExemptPaths) + } +} + +func TestLoadConfigFromBinaryDirRejectsUnknownAPIKey(t *testing.T) { + dir := t.TempDir() + writePrivateConfig(t, dir, "[api]\nunknown_key = 'value'\n") + cfg := DefaultConfig() + + err := LoadConfigFromBinaryDir(dir, &cfg) + if err == nil || !strings.Contains(err.Error(), "unsupported [api] key") { + t.Fatalf("unexpected error: %v", err) + } +} + +func TestConfigPrecedenceEnvironmentOverConfig(t *testing.T) { + dir := t.TempDir() + writePrivateConfig(t, dir, "[api]\nlisten = ':19080'\nauth_token = 'config-token'\n") + t.Setenv("BAT_API_LISTEN", ":19081") + t.Setenv("BAT_API_AUTH_TOKEN", "env-token") + + cfg := DefaultConfig() + if err := LoadConfigFromBinaryDir(dir, &cfg); err != nil { + t.Fatal(err) + } + ApplyEnv(&cfg) + if cfg.Listen != ":19081" || cfg.AuthToken != "env-token" { + t.Fatalf("environment did not override config: %+v", cfg) + } +} diff --git a/internal/api/testdata/release/official-download-manifest.json b/internal/api/testdata/release/official-download-manifest.json index d2b5bc1..c2694a9 100644 --- a/internal/api/testdata/release/official-download-manifest.json +++ b/internal/api/testdata/release/official-download-manifest.json @@ -5,13 +5,18 @@ "url": "https://prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.bytes", "destination": "prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.bytes", "bytes": 21, - "blake3": "0000000000000000000000000000000000000000000000000000000000000000" + "blake3": "5d5c6cc8ca0afa7d71df9b0d764c7b7f9e60c8b082dbb9fc844b10442989357f" }, "https://prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.hash": { "url": "https://prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.hash", "destination": "prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.hash", - "bytes": 10, - "blake3": "0000000000000000000000000000000000000000000000000000000000000000" + "bytes": 11, + "blake3": "dd3384e165b93bc64bf856fcf7d13883bb0a26c3d097de6622a8631b3941f6ef" } - } + }, + "destination_index": { + "prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.bytes": "https://prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.bytes", + "prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.hash": "https://prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.hash" + }, + "distribution_mapping_identity": "odm-v1-8f515f013cbe2ec7116fee902ea44d0179c435812cd29571361b91efc5ac33e6" } diff --git a/internal/api/testdata/release/prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.hash b/internal/api/testdata/release/prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.hash index 6a537b5..ee6f65a 100644 --- a/internal/api/testdata/release/prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.hash +++ b/internal/api/testdata/release/prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.hash @@ -1 +1 @@ -1234567890 \ No newline at end of file +2801109426 diff --git a/scripts/bat-api-local-live-smoke.sh b/scripts/bat-api-local-live-smoke.sh index 3ce20d6..33d770f 100755 --- a/scripts/bat-api-local-live-smoke.sh +++ b/scripts/bat-api-local-live-smoke.sh @@ -99,6 +99,20 @@ write_empty_version_state() { mv -f "${temp_path}" "${state_path}" } +write_distribution_anchor() { + local id="$1" + local version_dir="$2" + jq -n \ + --arg id "${id}" \ + '{ + version: 1, + official_release_id: $id, + mapping_identity: "odm-v1-8f515f013cbe2ec7116fee902ea44d0179c435812cd29571361b91efc5ac33e6", + manifest_identity: "54f1b8a97238367625afeef3f749878cd7913f89b74a09a3af2c0a067580f9ee", + entry_count: 2 + }' >"${version_dir}/official-distribution-publication.json" +} + switch_current() { local id="$1" local link_path="${OUTPUT_DIR}/current" @@ -118,6 +132,42 @@ stop_daemon() { fi } +verify_current_release() { + local id="$1" + printf '==> verify seeded release %s and record attestation\n' "${id}" + set +e + BAT_AUTO_DISCOVER=0 "${BAT_BIN}" verify \ + --server-info-path "${FIXTURE_ROOT}/yostar-serverinfo.bluearchiveyostar.com/r93_fixture.json" \ + --connection-group Prod \ + --app-version 1.70.0 \ + --output "${OUTPUT_DIR}" \ + --localized-output "${LOCALIZED_DIR}" \ + --state-dir "${STATE_DIR}" \ + --interval 1h \ + --error-retry 1s \ + --platforms Windows \ + --curl "${LOCAL_CURL}" \ + --json \ + --no-progress \ + --no-banner \ + >"${REPORT_DIR}/verify-${id}.json" 2>"${REPORT_DIR}/verify-${id}.log" + local verify_status="$?" + set -e + jq -e ' + .local_manifest_failure_count == 0 and + .official_hash_pair_count == 1 and + .official_hash_verified_count == .official_hash_pair_count + ' "${REPORT_DIR}/verify-${id}.json" >/dev/null || + fail "Rust verify did not validate the seeded local release: ${id}" + jq -e '.ready == true and .official_release_id == "'"${id}"'"' \ + "${OUTPUT_DIR}/versions/${id}/official-distribution-attestation.json" >/dev/null || + fail "Rust verify did not record a ready attestation: ${id}" + if [[ "${verify_status}" -ne 0 ]]; then + printf 'Rust verify reported expected remote-plan gaps for fixture %s; local release proof is ready\n' \ + "${id}" >&2 + fi +} + cleanup() { if [[ -n "${API_PID}" ]] && kill -0 "${API_PID}" 2>/dev/null; then kill "${API_PID}" 2>/dev/null || true @@ -130,23 +180,48 @@ trap cleanup EXIT mkdir -p "${OUTPUT_DIR}/versions/v-current" cp -a "${FIXTURE_ROOT}/." "${OUTPUT_DIR}/versions/v-current/" +write_distribution_anchor "v-current" "${OUTPUT_DIR}/versions/v-current" write_version_state "v-current" "${OUTPUT_DIR}/versions/v-current" switch_current "v-current" -# The daemon is real and owns the live RPC socket, but this smoke must never -# reach official endpoints. Its sync loop can fail and retry while the seeded -# published release remains available to bat-api. +# The daemon is real and owns the live RPC socket. The curl wrapper serves only +# the two local fixture hash files and rejects every other network URL. LOCAL_CURL="${SMOKE_ROOT}/curl-network-blocked" -printf '%s\n' \ - '#!/usr/bin/env bash' \ - 'printf "local live smoke blocks network access: curl %q\n" "$*" >&2' \ - 'exit 7' >"${LOCAL_CURL}" +cat >"${LOCAL_CURL}" <&2 + exit 7 + ;; +esac +EOF chmod +x "${LOCAL_CURL}" printf '==> start Rust bat daemon\n' +verify_current_release "v-current" BAT_AUTO_DISCOVER=0 "${BAT_BIN}" \ --server-info-path "${FIXTURE_ROOT}/yostar-serverinfo.bluearchiveyostar.com/r93_fixture.json" \ - --connection-group Prod \ + --connection-group SmokeUnavailable \ --app-version 1.70.0 \ --output "${OUTPUT_DIR}" \ --localized-output "${LOCALIZED_DIR}" \ @@ -168,7 +243,7 @@ done [[ -S "${SOCKET_PATH}" ]] || fail "Rust bat did not create ${SOCKET_PATH}; see ${REPORT_DIR}/daemon-start.log" printf '==> start bat-api on the same host\n' -BAT_API_SKIP_ENV_FILE=1 "${BAT_API_BIN}" \ +"${BAT_API_BIN}" \ --listen "127.0.0.1:${PORT}" \ --public-base-url "${BASE_URL}" \ --state-dir "${STATE_DIR}" \ @@ -286,8 +361,10 @@ wait_http_status \ printf '==> switch current release through the local published state\n' mkdir -p "${OUTPUT_DIR}/versions/v-next" cp -a "${FIXTURE_ROOT}/." "${OUTPUT_DIR}/versions/v-next/" +write_distribution_anchor "v-next" "${OUTPUT_DIR}/versions/v-next" write_version_state "v-next" "${OUTPUT_DIR}/versions/v-next" switch_current "v-next" +verify_current_release "v-next" wait_json "/readyz" '.ready == true and (.resource_root | endswith("/versions/v-next"))' \ "${REPORT_DIR}/ready-after-switch.json" curl -fsS "${BASE_URL}/v1/bootstrap" >"${REPORT_DIR}/bootstrap-after-switch.json" @@ -301,9 +378,9 @@ printf 'intentional size mismatch\n' >"${OUTPUT_DIR}/versions/v-broken/prod-clie write_version_state "v-broken" "${OUTPUT_DIR}/versions/v-broken" switch_current "v-broken" wait_json_status "/readyz" "503" \ - '.ready == false and .present_count == 1 and .missing_count == 1 and (.resource_root | endswith("/versions/v-broken"))' \ + '.ready == false and .resource_root == "" and .present_count == 0 and .missing_count == 0' \ "${REPORT_DIR}/ready-size-mismatch.json" -wait_http_status "${CDN_PATH}" "404" "${REPORT_DIR}/cdn-size-mismatch.response" +wait_http_status "${CDN_PATH}" "503" "${REPORT_DIR}/cdn-size-mismatch.response" printf '==> publish a manifest-missing release and require not-ready\n' mkdir -p "${OUTPUT_DIR}/versions/v-no-manifest" @@ -337,7 +414,7 @@ wait_http_status "/readyz" "503" "${REPORT_DIR}/ready-rpc-disconnected.json" BAT_AUTO_DISCOVER=0 "${BAT_BIN}" \ --server-info-path "${FIXTURE_ROOT}/yostar-serverinfo.bluearchiveyostar.com/r93_fixture.json" \ - --connection-group Prod \ + --connection-group SmokeUnavailable \ --app-version 1.70.0 \ --output "${OUTPUT_DIR}" \ --localized-output "${LOCALIZED_DIR}" \