mirror of
https://github.com/Yuyi-Oak/BlueArchiveToolkit.git
synced 2026-09-18 07:24:55 +08:00
+7
-7
@@ -28,9 +28,9 @@ Rust 侧官方日服资源链路已经从实验验证推进到正式入口:
|
|||||||
11. 官方同步会拒绝危险输出目录、路径逃逸和现有 symlink 路径组件;下载目标、`.part`、manifest、snapshot、PID、status、log 和控制锁文件不会跟随 symlink,daemon 状态类文件默认以 `0600` 权限创建。
|
11. 官方同步会拒绝危险输出目录、路径逃逸和现有 symlink 路径组件;下载目标、`.part`、manifest、snapshot、PID、status、log 和控制锁文件不会跟随 symlink,daemon 状态类文件默认以 `0600` 权限创建。
|
||||||
12. `<output>/official-version-state.json` 会明确保存当前已完成版本、正在拉取版本、上一个可用版本和失败版本;同一 app version、bundle version 和 Addressables root 的失败只保留最新一条,同一版本开始重新拉取或后续发布成功时会清理对应失败记录;`bat status` 会显示最后成功时间、下次检查时间、最后错误摘要、当前阶段、当前下载 URL 进度、版本状态摘要、最近历史失败版本和原因、结构化日志路径和轮转日志路径,人类可读输出不会把完整版本状态 JSON 内联打印。
|
12. `<output>/official-version-state.json` 会明确保存当前已完成版本、正在拉取版本、上一个可用版本和失败版本;同一 app version、bundle version 和 Addressables root 的失败只保留最新一条,同一版本开始重新拉取或后续发布成功时会清理对应失败记录;`bat status` 会显示最后成功时间、下次检查时间、最后错误摘要、当前阶段、当前下载 URL 进度、版本状态摘要、最近历史失败版本和原因、结构化日志路径和轮转日志路径,人类可读输出不会把完整版本状态 JSON 内联打印。
|
||||||
13. 资源导入链路已支持 CAS + `ResourceRepository` 索引写入,官方同步可用 `--import-repository` / `BAT_IMPORT_REPOSITORY=1` 在已校验 release 发布后触发导入,默认 CAS 路径为 `<output>/.cas`、SQLite 索引为 `<output>/resources.sqlite`,也可通过 `--import-cas-root`、`--import-resource-db`、`BAT_IMPORT_CAS_ROOT`、`BAT_IMPORT_RESOURCE_DB` 覆盖;`resource.index` RPC/CLI 可按类型、hash、路径模式、官方 release ID、平台、destination、bundle path、archive entry、parse status 和 TextUnit format 分页查询现有索引,数据库不存在时返回 `available=false` 且不会创建空库。`Resource` metadata 已通过 `metadata_json` 兼容迁移保存 release、平台、bundle path、parse status、TextAsset 名称和 TextUnit 数量/格式;当前/上一个/结构变化 catalog、失败 staging 复用、403/404、hash mismatch、CRC、metadata 迁移与 UnityFS 边界校验均有离线回归 fixture 或单测覆盖。
|
13. 资源导入链路已支持 CAS + `ResourceRepository` 索引写入,官方同步可用 `--import-repository` / `BAT_IMPORT_REPOSITORY=1` 在已校验 release 发布后触发导入,默认 CAS 路径为 `<output>/.cas`、SQLite 索引为 `<output>/resources.sqlite`,也可通过 `--import-cas-root`、`--import-resource-db`、`BAT_IMPORT_CAS_ROOT`、`BAT_IMPORT_RESOURCE_DB` 覆盖;`resource.index` RPC/CLI 可按类型、hash、路径模式、官方 release ID、平台、destination、bundle path、archive entry、parse status 和 TextUnit format 分页查询现有索引,数据库不存在时返回 `available=false` 且不会创建空库。`Resource` metadata 已通过 `metadata_json` 兼容迁移保存 release、平台、bundle path、parse status、TextAsset 名称和 TextUnit 数量/格式;当前/上一个/结构变化 catalog、失败 staging 复用、403/404、hash mismatch、CRC、metadata 迁移与 UnityFS 边界校验均有离线回归 fixture 或单测覆盖。
|
||||||
14. 非 dry-run 官方同步在校验完成并发布后,会先对比上一完整 release 与当前 release 的 `official-download-manifest.json`,在当前 release 下写入 `official-resource-changes.json` 和 `crowdin-translation-handoff.json`;同一 destination 只有 size 或 BLAKE3 改变才算 modified,仅 URL/CDN 根变化但内容相同不会触发解析/翻译候选。随后刷新 `official-parse-cache.json` 和 `official-textunit-index.json`,并从 Added/Modified 资源、parse cache 与 TextUnit 明细索引派生 `official-textunit-tasks.json` 和 `crowdin-textunit-queue.json`;删除资源只进入差异记录,不进入 TextUnit/Crowdin 队列。`parse.text_units` 和 `parse.errors` RPC/CLI 可按 destination、archive entry、path id、class id、field path 和 format 查询当前 release 的 TextUnit 明细与解析错误;`translation.tasks` RPC/CLI 可按 release、destination、archive entry、任务状态、parse status、TextUnit format 和 reason presence 查询离线 TextUnit 翻译任务状态与跳过/失败原因;`translation.task.update` 可回写 provider worker 状态,`translation.proofread` 可把汉化 workflow 标记为人工校对中;TextUnit 已包含 class id、field path、字段 offset/byte size 等可追溯定位。Crowdin 当前仅预留本地离线队列,不发网络请求;up-to-date 轮询在已有有效解析缓存、TextUnit 明细索引和 TextUnit 队列时只读取摘要,不重复解析。官方同步报告默认 `localized_release_status=not_localized`,含义是原版资源已经发布、汉化资源未发布;UnityFS TextAsset patch 发布成功后会写 `localized-patch-manifest.json`,校验 hash/size/diff/rollback 后才允许 `localized.status` 返回 `status=published`、`status_code=localized.published` 和 `localized_release_status=localized`。`bat` 首次启动会在二进制所在目录释放 `.env` 配置模板(`0600`),之后每次启动自动加载(不覆盖已存在的环境变量),支持 `BAT_OUTPUT`/`BAT_LOCALIZED_OUTPUT`/`BAT_IMPORT_REPOSITORY`/`BAT_IMPORT_CAS_ROOT`/`BAT_IMPORT_RESOURCE_DB`/`BAT_STATE_DIR`/`BAT_AUTO_DISCOVER`/`BAT_WATCH`/`BAT_DAEMON`/`BAT_PROXY` 等键,实现编辑 `.env` 后无参启动;优先级为命令行参数 > 进程环境变量 > `.env` > 内置默认值,`BAT_SKIP_ENV_FILE=1` 可整体禁用;Redis 键为预留。daemon 任务历史持久化在 `<state-dir>/bat-tasks.json`(版本化、`0600` 原子写),重启后任务经 `task.*` 仍可查,中断任务标记 `task_interrupted`(`BAT-ERR-700005`)。
|
14. 非 dry-run 官方同步在校验完成并发布后,会先对比上一完整 release 与当前 release 的 `official-download-manifest.json`,在当前 release 下写入 `official-resource-changes.json` 和 `crowdin-translation-handoff.json`;同一 destination 只有 size 或 BLAKE3 改变才算 modified,仅 URL/CDN 根变化但内容相同不会触发解析/翻译候选。随后刷新 `official-parse-cache.json` 和 `official-textunit-index.json`,并从 Added/Modified 资源、parse cache 与 TextUnit 明细索引派生 `official-textunit-tasks.json` 和 `crowdin-textunit-queue.json`;删除资源只进入差异记录,不进入 TextUnit/Crowdin 队列。`parse.text_units` 和 `parse.errors` RPC/CLI 可按 destination、archive entry、path id、class id、field path 和 format 查询当前 release 的 TextUnit 明细与解析错误;`translation.tasks` RPC/CLI 可按 release、destination、archive entry、任务状态、parse status、TextUnit format 和 reason presence 查询离线 TextUnit 翻译任务状态与跳过/失败原因;`translation.task.update` 可回写 provider worker 状态,`translation.worker.run` 可触发 Rust provider worker 独立 claim/lease/retry 并落库 TextUnit 级译文结果,`translation.proofread` 可把汉化 workflow 标记为人工校对中;TextUnit 已包含 class id、field path、字段 offset/byte size 等可追溯定位。Crowdin provider 通过 `CROWDIN_*` 环境变量接入,mock provider 支持本地 fixture;up-to-date 轮询在已有有效解析缓存、TextUnit 明细索引和 TextUnit 队列时只读取摘要,不重复解析。官方同步报告默认 `localized_release_status=not_localized`,含义是原版资源已经发布、汉化资源未发布;UnityFS TextAsset patch 发布成功后会写 `localized-patch-manifest.json`,校验 hash/size/diff/rollback 后才允许 `localized.status` 返回 `status=published`、`status_code=localized.published` 和 `localized_release_status=localized`。`bat` 首次启动会在二进制所在目录释放 `.env` 配置模板(`0600`),之后每次启动自动加载(不覆盖已存在的环境变量),支持 `BAT_OUTPUT`/`BAT_LOCALIZED_OUTPUT`/`BAT_IMPORT_REPOSITORY`/`BAT_IMPORT_CAS_ROOT`/`BAT_IMPORT_RESOURCE_DB`/`BAT_STATE_DIR`/`BAT_AUTO_DISCOVER`/`BAT_WATCH`/`BAT_DAEMON`/`BAT_PROXY` 和 `BAT_TRANSLATION_*` 等键,实现编辑 `.env` 后无参启动;优先级为命令行参数 > 进程环境变量 > `.env` > 内置默认值,`BAT_SKIP_ENV_FILE=1` 可整体禁用;Redis 键为预留。daemon 任务历史持久化在 `<state-dir>/bat-tasks.json`(版本化、`0600` 原子写),重启后任务经 `task.*` 仍可查,中断任务标记 `task_interrupted`(`BAT-ERR-700005`)。
|
||||||
|
|
||||||
15. issue 43 已补齐 Rust `bat` 的 `res` / `parse` / `i18n` 工作流入口:支持资源拉取、解析刷新、可再生解析缓存清理、离线翻译工作台、人工文本查看/修改/清空、工作台发布前校验、有限 TextAsset 汉化发布、人工校对状态标记、既有 patch 能力的批量重打包、单次/限定次数/周期执行和版本化 schedule CRUD。schedule 查询现在按一级工作流过滤,删除/执行会校验作用域,单轮执行可限制计划数;schedule CRUD、翻译任务状态回写和 `translation.proofread` 状态标记已通过 `bat.sock` 的 RPC 以及 `bat-api` 的鉴权管理接口暴露,dashboard 不维护第二套状态。该例外只编排已有解析和 patch 能力,不扩大解析器覆盖;完整 AssetBundle 重打包、真实 provider worker 和 Web 前端仍是后续工作。G-008(产品级 Go 同步 CLI)已决策关闭。真实官方网络全量拉取 smoke 已固化(G-018 已关闭);真实大文件与运行报告默认在 `/tmp` 隔离目录,不纳入 Git。Go 细节见 `docs/reports/GO_STATUS.md`。
|
15. issue 43 已补齐 Rust `bat` 的 `res` / `parse` / `i18n` 工作流入口:支持资源拉取、解析刷新、可再生解析缓存清理、离线翻译工作台、人工文本查看/修改/清空、工作台发布前校验、有限 TextAsset 汉化发布、人工校对状态标记、既有 patch 能力的批量重打包、单次/限定次数/周期执行和版本化 schedule CRUD。issue 44 已接入 `translation.worker.run` provider worker:默认并发 8、范围 `1..=256`,每个 worker 独立 claim 下一项任务并落库 lease、失败分类、重试计划和 TextUnit 译文结果。schedule 查询现在按一级工作流过滤,删除/执行会校验作用域,单轮执行可限制计划数;schedule CRUD、翻译任务查询/交接视图、翻译任务状态回写、provider worker 触发和 `translation.proofread` 状态标记已通过 `bat.sock` 的 RPC 以及 `bat-api` 的鉴权管理接口暴露,dashboard 不维护第二套状态。该例外只编排已有解析和 patch 能力,不扩大解析器覆盖;完整 AssetBundle 重打包和 Web 前端仍是后续工作。G-008(产品级 Go 同步 CLI)已决策关闭。真实官方网络全量拉取 smoke 已固化(G-018 已关闭);真实大文件与运行报告默认在 `/tmp` 隔离目录,不纳入 Git。Go 细节见 `docs/reports/GO_STATUS.md`。
|
||||||
|
|
||||||
当前翻译交接还包括 `translation-tasks.sqlite` 和版本化 `translation-handoff.json`;
|
当前翻译交接还包括 `translation-tasks.sqlite` 和版本化 `translation-handoff.json`;
|
||||||
`translation.tasks` 查询单项 worker 状态,`translation.handoff` 查询完整
|
`translation.tasks` 查询单项 worker 状态,`translation.handoff` 查询完整
|
||||||
@@ -146,7 +146,7 @@ job/unit/provider run 状态。当前下载实现使用默认 8 个独立 worker
|
|||||||
|
|
||||||
待完成:
|
待完成:
|
||||||
|
|
||||||
- 将 `crowdin-textunit-queue.json` 接入真实 Crowdin/provider worker、翻译记忆和 Patch 构建;围绕 `translation.task.update` 完成真实 worker 集成与失败原因落库验证,并继续扩展 CAS 诊断查询面。
|
- 基于已接入的 `translation.worker.run` 继续推进翻译记忆、完整 Patch 构建/rollback;继续扩展 CAS 诊断查询面。
|
||||||
- 真实线上全量下载 smoke 已固化为 `scripts/official-full-pull-smoke.sh` 和 `make official-smoke`;实际运行报告由脚本写入隔离输出目录。
|
- 真实线上全量下载 smoke 已固化为 `scripts/official-full-pull-smoke.sh` 和 `make official-smoke`;实际运行报告由脚本写入隔离输出目录。
|
||||||
- 增加更多权限和极端文件系统场景测试。
|
- 增加更多权限和极端文件系统场景测试。
|
||||||
|
|
||||||
@@ -271,7 +271,7 @@ cargo run -p bat-infrastructure --bin bat -- \
|
|||||||
--watch
|
--watch
|
||||||
```
|
```
|
||||||
|
|
||||||
资源 HTTP bootstrap / 只读分发入口是 Go `cmd/bat-api`。生产拓扑下它与 Rust `bat` 同环境运行,经 `bat.sock` RPC 获取当前 `resource_root`,不在配置里写死资源目录;本地开发不能全量跑 `bat` 时用 fixture 和 Go 门禁验证。`internal/api/testdata/contract/` 已固化来自 Rust 输出并经归一化的 `catalog.status`、`resource.manifest` 和 `official-sync-snapshot.json` contract fixture,Go mirror 测试会防止字段名、null 语义和 `game_main_config_bootstrap` 再次漂移。`bat-api` 已补 launcher 资源引导兼容端点、玩家-facing HTTP 控制面和鉴权调度管理接口(token 鉴权、限流、访问日志、反代 IP 适配、动态 JSON no-store、OpenAPI、管理控制白名单;`reload` / `refresh` / `restart` / `sync` / `verify` / `repair` / `catalog-refresh` 及 `schedule.*` 可经 Web 转发),响应只来自已发布 snapshot/RPC,不提供登录、网关、鉴权或完整 package update manifest。
|
资源 HTTP bootstrap / 只读分发入口是 Go `cmd/bat-api`。生产拓扑下它与 Rust `bat` 同环境运行,经 `bat.sock` RPC 获取当前 `resource_root`,不在配置里写死资源目录;本地开发不能全量跑 `bat` 时用 fixture 和 Go 门禁验证。`internal/api/testdata/contract/` 已固化来自 Rust 输出并经归一化的 `catalog.status`、`resource.manifest` 和 `official-sync-snapshot.json` contract fixture,Go mirror 测试会防止字段名、null 语义和 `game_main_config_bootstrap` 再次漂移。`bat-api` 已补 launcher 资源引导兼容端点、玩家-facing HTTP 控制面和鉴权调度/translation 管理接口(token 鉴权、限流、访问日志、反代 IP 适配、动态 JSON no-store、OpenAPI、管理控制白名单;`reload` / `refresh` / `restart` / `sync` / `verify` / `repair` / `catalog-refresh`、`schedule.*`、`translation.tasks` / `translation.handoff` 查询、`translation.task.update`、`translation.worker.run` 和 `translation.proofread` 可经 Web 转发),响应只来自已发布 snapshot/RPC,不提供登录、网关、鉴权或完整 package update manifest。
|
||||||
|
|
||||||
生产要求:
|
生产要求:
|
||||||
|
|
||||||
@@ -292,8 +292,8 @@ GitHub issue 状态:#1 已关闭;#17 的历史决定不代表当前下载实
|
|||||||
下一阶段必须优先完成:
|
下一阶段必须优先完成:
|
||||||
|
|
||||||
1. Issue #1(P0,主体已实现):`bat.sock` Unix socket JSON-RPC 已扩展为面向 Go 服务层的 Rust Resource Backend API。统一 envelope(`ok`、`status`、`error`、`data`、`request_id`)与 `BAT-ERR` 错误码模型已落地;`daemon.*`(status/logs/stop/restart/reload/refresh/doctor)、`resource.*`(state/sync/verify/repair/manifest/list/index)、`schedule.*`(list/add/update/remove/run)、`parse.*`(status/text_units/errors)、`translation.*`(tasks/handoff/task.update/proofread)、`localized.*`(status)、`catalog.*`(status/refresh/diff/versions)、`task.*`(status/list/cancel/logs)、文件级 `patch.apply` 与 `unityfs.patch_text_asset` / `unityfs.patch_string_field` / `unityfs.patch_field` 已实现,长任务返回 `task_id` 可轮询(任务执行器单 worker FIFO,与 watch 循环互斥;任务历史持久化于 `<state-dir>/bat-tasks.json`,daemon 重启后仍可查,中断任务标记 `task_interrupted`);错误码已接入下载、launcher/metadata、server-info/marker 与配置校验路径。剩余:发布级 `patch build` / `patch rollback`、复杂 `unityfs.*` 语义编辑、`task.create`(按设计由语义方法创建)、`daemon.clean-stable`(由 CLI 侧按进程生命周期显式执行,live RPC 内不做在线清理)、Redis 任务后端(`.env` 已预留配置键,接入时机另议)。Go 层通过 RPC 调用 Rust backend,不走 FFI(FFI 降级说明见 `docs/architecture/official-resource-backend.md` §7)。
|
1. Issue #1(P0,主体已实现):`bat.sock` Unix socket JSON-RPC 已扩展为面向 Go 服务层的 Rust Resource Backend API。统一 envelope(`ok`、`status`、`error`、`data`、`request_id`)与 `BAT-ERR` 错误码模型已落地;`daemon.*`(status/logs/stop/restart/reload/refresh/doctor)、`resource.*`(state/sync/verify/repair/manifest/list/index)、`schedule.*`(list/add/update/remove/run)、`parse.*`(status/text_units/errors)、`translation.*`(tasks/handoff/task.update/proofread)、`localized.*`(status)、`catalog.*`(status/refresh/diff/versions)、`task.*`(status/list/cancel/logs)、文件级 `patch.apply` 与 `unityfs.patch_text_asset` / `unityfs.patch_string_field` / `unityfs.patch_field` 已实现,长任务返回 `task_id` 可轮询(任务执行器单 worker FIFO,与 watch 循环互斥;任务历史持久化于 `<state-dir>/bat-tasks.json`,daemon 重启后仍可查,中断任务标记 `task_interrupted`);错误码已接入下载、launcher/metadata、server-info/marker 与配置校验路径。剩余:发布级 `patch build` / `patch rollback`、复杂 `unityfs.*` 语义编辑、`task.create`(按设计由语义方法创建)、`daemon.clean-stable`(由 CLI 侧按进程生命周期显式执行,live RPC 内不做在线清理)、Redis 任务后端(`.env` 已预留配置键,接入时机另议)。Go 层通过 RPC 调用 Rust backend,不走 FFI(FFI 降级说明见 `docs/architecture/official-resource-backend.md` §7)。
|
||||||
2. Go 侧:进度见 `docs/reports/GO_STATUS.md`。G-008 已关闭;`bat-api` 资源 bootstrap/分发 MVP 已落地,已含 `/v1/bootstrap`、`/v1/launcher/bootstrap`、launcher 资源 metadata 兼容、HTTP 鉴权/限流/日志/反代适配、动态 JSON no-store、OpenAPI、管理控制白名单、CDN Range/缓存头、RPC 周期刷新和 USERGUIDE 基础章节;仓库内 Rust/Go snapshot contract fixture 与同机 live smoke 已落地。失效 `resource_root` 或 manifest 不完整时会清空旧索引并使 `/readyz` 返回 `503`;编码 dot-segment 会在路由规范化前拒绝。refresh mtime/size 增量缓存和可选持久化另议。
|
2. Go 侧:进度见 `docs/reports/GO_STATUS.md`。G-008 已关闭;`bat-api` 资源 bootstrap/分发 MVP 已落地,已含 `/v1/bootstrap`、`/v1/launcher/bootstrap`、launcher 资源 metadata 兼容、HTTP 鉴权/限流/日志/反代适配、动态 JSON no-store、OpenAPI、管理控制白名单、translation admin 查询/控制、CDN Range/缓存头、RPC 周期刷新和 USERGUIDE 基础章节;仓库内 Rust/Go snapshot contract fixture 与同机 live smoke 已落地。失效 `resource_root` 或 manifest 不完整时会清空旧索引并使 `/readyz` 返回 `503`;编码 dot-segment 会在路由规范化前拒绝。refresh mtime/size 增量缓存和可选持久化另议。
|
||||||
3. 文本提取 / 翻译队列 / Patch 输入:`official-textunit-index.json`、`official-textunit-tasks.json` 与 `crowdin-textunit-queue.json` 已生成;TextUnit 明细、解析错误和离线翻译任务状态/失败原因已可查询,`translation.task.update` 已提供 Rust CLI、Go typed helper 和 bat-api dashboard 状态回写入口,`translation.proofread` 已提供人工校对状态标记入口。剩余为真实 Crowdin/provider worker、翻译记忆和 Patch 构建。
|
3. 文本提取 / 翻译队列 / Patch 输入:`official-textunit-index.json`、`official-textunit-tasks.json` 与 `crowdin-textunit-queue.json` 已生成;TextUnit 明细、解析错误和离线翻译任务状态/失败原因已可查询,`translation.tasks` / `translation.handoff` 已提供 Go typed helper 和 bat-api dashboard 查询入口,`translation.task.update` 已提供 Rust CLI、Go typed helper 和 bat-api dashboard 状态回写入口,`translation.worker.run` 已提供 mock/Crowdin provider worker、lease、重试和 TextUnit 译文结果落库,`translation.proofread` 已提供人工校对状态标记入口。剩余为翻译记忆、完整 Patch 构建/rollback 和复杂重打包。
|
||||||
4. Issue #3(已完成当前目标):AssetBundle UnityFS 基础解析校验已覆盖 header、block、directory、LZ4/LZMA、alignment、总大小、计数、路径安全、重复 directory、边界和隔离 UnityPy 真实样本;复杂对象解析与发布级重打包继续跟踪 G-005。
|
4. Issue #3(已完成当前目标):AssetBundle UnityFS 基础解析校验已覆盖 header、block、directory、LZ4/LZMA、alignment、总大小、计数、路径安全、重复 directory、边界和隔离 UnityPy 真实样本;复杂对象解析与发布级重打包继续跟踪 G-005。
|
||||||
5. Issue #2(已完成当前目标):Addressables JSON/compact catalog 已提取 bundle provider、name、hash/size/CRC、资源类型和依赖关系,并贯通 ResourceEntry、SQLite 与 resource.index 输出;独立二进制格式仍明确拒绝。
|
5. Issue #2(已完成当前目标):Addressables JSON/compact catalog 已提取 bundle provider、name、hash/size/CRC、资源类型和依赖关系,并贯通 ResourceEntry、SQLite 与 resource.index 输出;独立二进制格式仍明确拒绝。
|
||||||
6. 通用 Binary/JSON/Text Patch 基础已落地;复杂 AssetBundle 重打包和真实翻译系统仍应后置,UnityFS TextAsset patch 发布前置已具备回归测试。
|
6. 通用 Binary/JSON/Text Patch 基础已落地;复杂 AssetBundle 重打包和真实翻译系统仍应后置,UnityFS TextAsset patch 发布前置已具备回归测试。
|
||||||
@@ -315,4 +315,4 @@ GitHub issue 状态:#1 已关闭;#17 的历史决定不代表当前下载实
|
|||||||
|
|
||||||
- **当前总体完成度**:不再固定写单一百分比,以各模块状态、`GO_STATUS.md` 和 issue 为准。
|
- **当前总体完成度**:不再固定写单一百分比,以各模块状态、`GO_STATUS.md` 和 issue 为准。
|
||||||
- **当前基线状态**:Rust `bat` 同步闭环可用;Go `bat-api` 资源 bootstrap/分发 MVP + 玩家-facing HTTP 控制面 + launcher 资源引导兼容 + RPC 周期刷新/诊断 + readiness + `backendrpc` 可用;CAS 用户级导入、TextUnit 明细索引/查询、增量 Crowdin 离线队列、通用 Binary/JSON/Text Patch 基础和 UnityFS TextAsset patch 发布前置可用;完整 AssetBundle 重打包未完成。
|
- **当前基线状态**:Rust `bat` 同步闭环可用;Go `bat-api` 资源 bootstrap/分发 MVP + 玩家-facing HTTP 控制面 + launcher 资源引导兼容 + RPC 周期刷新/诊断 + readiness + `backendrpc` 可用;CAS 用户级导入、TextUnit 明细索引/查询、增量 Crowdin 离线队列、通用 Binary/JSON/Text Patch 基础和 UnityFS TextAsset patch 发布前置可用;完整 AssetBundle 重打包未完成。
|
||||||
- **下一工程里程碑**:真实 Crowdin/provider worker / 翻译记忆、复杂 AssetBundle 解析和重打包;`bat-api` 同机 live 联调已完成。
|
- **下一工程里程碑**:翻译记忆、完整 localized Patch 构建/rollback、复杂 AssetBundle 解析和重打包;`bat-api` 同机 live 联调已完成。
|
||||||
|
|||||||
+8
-8
@@ -36,7 +36,7 @@ BlueArchiveToolkit 不是一次性脚本,也不是演示项目。最终交付
|
|||||||
8. `bat-ffi` 已提供 Manifest inspect 和官方 sync plan 的可选无状态粗粒度 JSON C ABI helper。
|
8. `bat-ffi` 已提供 Manifest inspect 和官方 sync plan 的可选无状态粗粒度 JSON C ABI helper。
|
||||||
9. 官方原版资源默认发布到 `./bat-resources`,汉化产物默认发布到独立的 `./bat-localized`;当前官方同步报告会标记 `localized_release_status=not_localized`,表示原版资源已发布、汉化资源未发布;`translation.proofread` 可把汉化 workflow 标记为人工校对中,但不会覆盖已发布的汉化 release。
|
9. 官方原版资源默认发布到 `./bat-resources`,汉化产物默认发布到独立的 `./bat-localized`;当前官方同步报告会标记 `localized_release_status=not_localized`,表示原版资源已发布、汉化资源未发布;`translation.proofread` 可把汉化 workflow 标记为人工校对中,但不会覆盖已发布的汉化 release。
|
||||||
10. 官方同步校验完成并发布新 release 后会生成 `official-resource-changes.json`、`crowdin-translation-handoff.json`、`official-parse-cache.json`、`official-textunit-index.json`、`official-textunit-tasks.json` 和 `crowdin-textunit-queue.json`,用 Added/Modified 资源驱动后续解析/翻译增量;up-to-date 轮询在已有有效缓存、TextUnit 明细索引和队列时只读取摘要,不重复解析。
|
10. 官方同步校验完成并发布新 release 后会生成 `official-resource-changes.json`、`crowdin-translation-handoff.json`、`official-parse-cache.json`、`official-textunit-index.json`、`official-textunit-tasks.json` 和 `crowdin-textunit-queue.json`,用 Added/Modified 资源驱动后续解析/翻译增量;up-to-date 轮询在已有有效缓存、TextUnit 明细索引和队列时只读取摘要,不重复解析。
|
||||||
11. issue 43 已补齐 Rust `bat` 的 `res` / `parse` / `i18n` 工作流:单次/限定次数/周期执行、版本化 schedule CRUD 与作用域过滤、解析缓存清理、翻译工作台校验、离线翻译工作台、人工文本查看/修改/清空、翻译任务状态回写、人工校对状态标记、既有 patch 能力的批量重打包和独立汉化 release 发布;schedule CRUD、`translation.task.update` 和 `translation.proofread` 已经通过 `bat.sock` 和 `bat-api` 管理接口暴露,dashboard 不维护第二套状态;该能力不扩大冻结期解析器覆盖。
|
11. issue 43 已补齐 Rust `bat` 的 `res` / `parse` / `i18n` 工作流:单次/限定次数/周期执行、版本化 schedule CRUD 与作用域过滤、解析缓存清理、翻译工作台校验、离线翻译工作台、人工文本查看/修改/清空、翻译任务状态回写、人工校对状态标记、既有 patch 能力的批量重打包和独立汉化 release 发布;issue 44 已接入 `translation.worker.run` provider worker,默认并发 8、范围 `1..=256`,每个 worker 独立 claim 下一项任务并落库 lease、失败分类、重试计划和 TextUnit 译文结果;schedule CRUD、`translation.tasks` / `translation.handoff`、`translation.task.update`、`translation.worker.run` 和 `translation.proofread` 已经通过 `bat.sock` 和 `bat-api` 管理接口暴露,dashboard 不维护第二套状态;该能力不扩大冻结期解析器覆盖。
|
||||||
12. 文档已整理:根目录保留入口文档,历史报告进入 `docs/reports/historical/`,误嵌套的 `docs/docs` 已合并。
|
12. 文档已整理:根目录保留入口文档,历史报告进入 `docs/reports/historical/`,误嵌套的 `docs/docs` 已合并。
|
||||||
|
|
||||||
### 仍是骨架或占位
|
### 仍是骨架或占位
|
||||||
@@ -45,8 +45,8 @@ BlueArchiveToolkit 不是一次性脚本,也不是演示项目。最终交付
|
|||||||
2. `bat-patch` 已具备确定性 Binary hunk diff/apply、RFC 6902 JSON Patch apply、UTF-8 Text Patch、通用 Patch manifest、BLAKE3/size 完整性校验和 rollback 元数据;文件级 `patch.apply` RPC / `patch-apply` CLI 与 UnityFS TextAsset / TypeTree string / TypeTree 语义字段写入入口已开放,当前发布级可用的是 `bat-assetbundle` + `LocalizedPatchService` 的 UnityFS TextAsset patch 前置链路。
|
2. `bat-patch` 已具备确定性 Binary hunk diff/apply、RFC 6902 JSON Patch apply、UTF-8 Text Patch、通用 Patch manifest、BLAKE3/size 完整性校验和 rollback 元数据;文件级 `patch.apply` RPC / `patch-apply` CLI 与 UnityFS TextAsset / TypeTree string / TypeTree 语义字段写入入口已开放,当前发布级可用的是 `bat-assetbundle` + `LocalizedPatchService` 的 UnityFS TextAsset patch 前置链路。
|
||||||
3. Go 侧边界已冻结(见 `docs/reports/GO_STATUS.md`):同步/运维命令行 = Rust `bat`;资源分发 = `cmd/bat-api` MVP;`internal/backendrpc` 完成;`cmd/bat` 仅为试验(`bin/bat-go`)。完整游戏业务 API / Web / SDK 仍未完成。
|
3. Go 侧边界已冻结(见 `docs/reports/GO_STATUS.md`):同步/运维命令行 = Rust `bat`;资源分发 = `cmd/bat-api` MVP;`internal/backendrpc` 完成;`cmd/bat` 仅为试验(`bin/bat-go`)。完整游戏业务 API / Web / SDK 仍未完成。
|
||||||
4. Addressables parser 已覆盖当前真实形态 fixture/golden,但还不是完整 Unity Addressables/SBP catalog 兼容层。
|
4. Addressables parser 已覆盖当前真实形态 fixture/golden,但还不是完整 Unity Addressables/SBP catalog 兼容层。
|
||||||
5. 官方同步结果可配置为发布后自动导入 CAS + ResourceRepository,并通过 `resource.index` RPC/CLI 查询;Resource metadata 已保存 release、平台、bundle path、parse status、TextAsset 名称和 TextUnit 摘要,资源级查询已覆盖 release、平台、destination、archive entry、parse status 和 TextUnit format;单条 TextUnit 明细和解析错误已持久化到 `official-textunit-index.json`,可通过 `parse.text_units` / `parse.errors` 查询;离线 TextUnit 翻译任务状态和跳过/失败原因可通过 `translation.tasks` 查询,`translation.task.update` 已提供 worker 状态回写 contract,`translation.proofread` 已提供汉化 workflow 人工校对标记 contract,真实 provider worker 集成仍需继续推进。
|
5. 官方同步结果可配置为发布后自动导入 CAS + ResourceRepository,并通过 `resource.index` RPC/CLI 查询;Resource metadata 已保存 release、平台、bundle path、parse status、TextAsset 名称和 TextUnit 摘要,资源级查询已覆盖 release、平台、destination、archive entry、parse status 和 TextUnit format;单条 TextUnit 明细和解析错误已持久化到 `official-textunit-index.json`,可通过 `parse.text_units` / `parse.errors` 查询;离线 TextUnit 翻译任务状态和跳过/失败原因可通过 `translation.tasks` 查询,`translation.task.update` 已提供 worker 状态回写 contract,`translation.worker.run` 已提供真实 provider worker 触发、lease/retry 和结果落库 contract,`translation.proofread` 已提供汉化 workflow 人工校对标记 contract。
|
||||||
6. 汉化 Patch 发布前置已具备 UnityFS TextAsset manifest/apply/diff/rollback/完整性校验和 `localized.status` 严格校验;真实 Crowdin worker、翻译记忆到完整汉化文件集合的构建仍未完成。
|
6. 汉化 Patch 发布前置已具备 UnityFS TextAsset manifest/apply/diff/rollback/完整性校验和 `localized.status` 严格校验;真实 provider worker 已接入,翻译记忆到完整汉化文件集合的构建仍未完成。
|
||||||
7. 真实官方网络全量下载 smoke 已固化为可重复脚本和 runbook(G-018 已关闭);真实运行记录处于长期运行测试阶段,报告待后续提供。
|
7. 真实官方网络全量下载 smoke 已固化为可重复脚本和 runbook(G-018 已关闭);真实运行记录处于长期运行测试阶段,报告待后续提供。
|
||||||
8. 完整 Web、数据库迁移、插件加载机制尚未实现;`bat-api` 资源 bootstrap/分发 OpenAPI 已通过 `/openapi.yaml` 提供,完整游戏业务 API 的 OpenAPI 仍未完成。
|
8. 完整 Web、数据库迁移、插件加载机制尚未实现;`bat-api` 资源 bootstrap/分发 OpenAPI 已通过 `/openapi.yaml` 提供,完整游戏业务 API 的 OpenAPI 仍未完成。
|
||||||
9. 原 Git 历史未恢复;当前仓库以新初始化基线为准。
|
9. 原 Git 历史未恢复;当前仓库以新初始化基线为准。
|
||||||
@@ -179,7 +179,7 @@ BlueArchiveToolkit 不是一次性脚本,也不是演示项目。最终交付
|
|||||||
4. Rust 自动更新入口:**已完成当前生产入口**。`bat` 支持 snapshot、marker diff、bootstrap cache、one-shot、`--watch`、`--daemon`、默认 1 小时间隔、北京时间固定强制刷新,以及 Unix socket JSON-RPC 后台运维命令返回。
|
4. Rust 自动更新入口:**已完成当前生产入口**。`bat` 支持 snapshot、marker diff、bootstrap cache、one-shot、`--watch`、`--daemon`、默认 1 小时间隔、北京时间固定强制刷新,以及 Unix socket JSON-RPC 后台运维命令返回。
|
||||||
5. Go 入口边界:**已冻结**。同步命令行 = Rust `bat`(G-008 关闭);资源分发 = `bat-api` MVP(G-009 部分完成)。详见 `docs/reports/GO_STATUS.md`。
|
5. Go 入口边界:**已冻结**。同步命令行 = Rust `bat`(G-008 关闭);资源分发 = `bat-api` MVP(G-009 部分完成)。详见 `docs/reports/GO_STATUS.md`。
|
||||||
6. 用户级 `sync`、`manifest inspect`、`cache status`:**未完成**。Rust `bat --json` 是当前稳定进程边界;`bat-ffi` 只提供可选兼容用的 Manifest inspect 和 sync plan JSON helper。
|
6. 用户级 `sync`、`manifest inspect`、`cache status`:**未完成**。Rust `bat --json` 是当前稳定进程边界;`bat-ffi` 只提供可选兼容用的 Manifest inspect 和 sync plan JSON helper。
|
||||||
7. 下载结果写入 CAS + ResourceRepository:**部分完成**。CAS 和 SQLite ResourceRepository 已存在,官方同步入口可用 `--import-repository` / `BAT_IMPORT_REPOSITORY=1` 在已校验 release 发布后导入;`resource.index` 可按资源级 release、平台、destination、archive entry、parse status 和 TextUnit format 查询现有索引和资源 metadata;`parse.text_units` / `parse.errors` 可查询当前 release 的 TextUnit 明细与解析错误;`translation.tasks` 可查询离线 TextUnit 翻译任务状态和跳过/失败原因。剩余工作是真实 provider worker 集成、CAS 诊断入口和面向大索引的查询优化。
|
7. 下载结果写入 CAS + ResourceRepository:**部分完成**。CAS 和 SQLite ResourceRepository 已存在,官方同步入口可用 `--import-repository` / `BAT_IMPORT_REPOSITORY=1` 在已校验 release 发布后导入;`resource.index` 可按资源级 release、平台、destination、archive entry、parse status 和 TextUnit format 查询现有索引和资源 metadata;`parse.text_units` / `parse.errors` 可查询当前 release 的 TextUnit 明细与解析错误;`translation.tasks` 可查询离线 TextUnit 翻译任务状态、跳过/失败原因和 worker 结果。剩余工作是 CAS 诊断入口和面向大索引的查询优化。
|
||||||
8. Linux 生产同步不依赖已安装官方启动器:**已完成当前 Rust 入口**。`--auto-discover` 只使用官方 HTTP metadata 和临时目录解析 `GameMainConfig`。
|
8. Linux 生产同步不依赖已安装官方启动器:**已完成当前 Rust 入口**。`--auto-discover` 只使用官方 HTTP metadata 和临时目录解析 `GameMainConfig`。
|
||||||
9. 真实官方网络全量下载 smoke test:**命令已固化(G-018 已关闭)**。`scripts/official-full-pull-smoke.sh` / `make official-smoke` 已固化 dry-run、首次下载、二次 up-to-date 和本地损坏 repair 的可重复流程;真实运行处于长期运行测试阶段,报告待后续提供。
|
9. 真实官方网络全量下载 smoke test:**命令已固化(G-018 已关闭)**。`scripts/official-full-pull-smoke.sh` / `make official-smoke` 已固化 dry-run、首次下载、二次 up-to-date 和本地损坏 repair 的可重复流程;真实运行处于长期运行测试阶段,报告待后续提供。
|
||||||
10. 官方发布后的增量 handoff 与解析缓存:**已完成基础入口**。新 release 发布后先生成 `official-resource-changes.json` 和 `crowdin-translation-handoff.json`,新增+变更资源进入解析/翻译候选;`official-parse-cache.json` 基于下载 manifest 覆盖直接 UnityFS bundle、zip 内 UnityFS 条目和非候选资源记录;随后生成 `official-textunit-index.json`、`official-textunit-tasks.json` 与 `crowdin-textunit-queue.json`,本地文件未变化且缓存/索引有效时跳过重复解析。
|
10. 官方发布后的增量 handoff 与解析缓存:**已完成基础入口**。新 release 发布后先生成 `official-resource-changes.json` 和 `crowdin-translation-handoff.json`,新增+变更资源进入解析/翻译候选;`official-parse-cache.json` 基于下载 manifest 覆盖直接 UnityFS bundle、zip 内 UnityFS 条目和非候选资源记录;随后生成 `official-textunit-index.json`、`official-textunit-tasks.json` 与 `crowdin-textunit-queue.json`,本地文件未变化且缓存/索引有效时跳过重复解析。
|
||||||
@@ -404,8 +404,8 @@ BlueArchiveToolkit 不是一次性脚本,也不是演示项目。最终交付
|
|||||||
2. G-008 已决策关闭:同步/运维命令行 = 近乎全自动的 Rust `bat`。
|
2. G-008 已决策关闭:同步/运维命令行 = 近乎全自动的 Rust `bat`。
|
||||||
3. G-009 / issue #19:`bat-api` 资源分发和同机 `bat.sock` live 联调已完成;拉取仍在 Rust `bat`。
|
3. G-009 / issue #19:`bat-api` 资源分发和同机 `bat.sock` live 联调已完成;拉取仍在 Rust `bat`。
|
||||||
4. 继续 Addressables 结构变体与 UnityFS 复杂对象能力(issue #3 / G-005 的后续阶段)。
|
4. 继续 Addressables 结构变体与 UnityFS 复杂对象能力(issue #3 / G-005 的后续阶段)。
|
||||||
5. 将 `crowdin-textunit-queue.json` 接入真实 Crowdin worker、翻译记忆和 Patch 构建。
|
5. 基于 `translation.worker.run` 继续推进翻译记忆和 Patch 构建。
|
||||||
6. 继续扩展 G-011 剩余查询面:真实 provider worker 集成与状态落库验证、`doctor cas` 诊断入口和面向大索引的查询优化。
|
6. 继续扩展 G-011 剩余查询面:`doctor cas` 诊断入口和面向大索引的查询优化。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -459,9 +459,9 @@ BlueArchiveToolkit 不是一次性脚本,也不是演示项目。最终交付
|
|||||||
|
|
||||||
按最终目标计算,当前总体完成度不再固定写单一百分比,以模块状态和 issue 收敛情况为准。
|
按最终目标计算,当前总体完成度不再固定写单一百分比,以模块状态和 issue 收敛情况为准。
|
||||||
|
|
||||||
已完成的是稳定基线、架构骨架、部分接口、CAS V1、Rust 官方资源同步闭环、可配置 CAS/ResourceRepository 导入、TextUnit 明细索引/查询、增量 Crowdin 离线队列、通用 Binary/JSON/Text Patch 基础、UnityFS TextAsset patch 发布前置,以及 Go `bat-api` 资源分发和同机 live 联调。下一阶段的关键是真实 Crowdin worker、翻译记忆、复杂 AssetBundle 解析/重打包和官方资源长期运行报告。
|
已完成的是稳定基线、架构骨架、部分接口、CAS V1、Rust 官方资源同步闭环、可配置 CAS/ResourceRepository 导入、TextUnit 明细索引/查询、增量 Crowdin 离线队列、provider worker、通用 Binary/JSON/Text Patch 基础、UnityFS TextAsset patch 发布前置,以及 Go `bat-api` 资源分发和同机 live 联调。下一阶段的关键是翻译记忆、完整 localized Patch 构建/rollback、复杂 AssetBundle 解析/重打包和官方资源长期运行报告。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
- **下一份应更新文档**:真实官方网络 smoke 记录
|
- **下一份应更新文档**:真实官方网络 smoke 记录
|
||||||
- **下一项工程任务**:推进真实 Crowdin worker / 翻译记忆、复杂 AssetBundle 解析,并持续执行官方资源长期运行 smoke。
|
- **下一项工程任务**:推进翻译记忆、完整 localized Patch 构建/rollback、复杂 AssetBundle 解析,并持续执行官方资源长期运行 smoke。
|
||||||
|
|||||||
@@ -15,7 +15,7 @@
|
|||||||
- `bat-infrastructure` CAS 适配层、SQLite Resource Repository、资源导入服务、官方资源 pull/update 服务。
|
- `bat-infrastructure` CAS 适配层、SQLite Resource Repository、资源导入服务、官方资源 pull/update 服务。
|
||||||
- `bat`:官方资源自动发现、全量拉取、原子发布到 `current -> versions/<id>`、本地 manifest audit/repair、`.part` 断点续传、403/404/5xx 分类重试、指数退避、默认并发 8(可配置 `1..=256`,report 按 plan 顺序、进度按完成数单调上报)、下载 quarantine 诊断、ZIP 结构校验、官方 seed `.hash` 校验、snapshot/cache、版本化 `official-launcher-bootstrap.json`、`--watch` 常驻更新、`--daemon` 后台运行,以及 Unix socket JSON-RPC live control/backend 方法(`daemon.*`、`resource.*`、`parse.*`、`translation.tasks/handoff/task.update`、`localized.status`、`catalog.*`、`task.*`、`patch.apply`、`unityfs.patch_*`)。
|
- `bat`:官方资源自动发现、全量拉取、原子发布到 `current -> versions/<id>`、本地 manifest audit/repair、`.part` 断点续传、403/404/5xx 分类重试、指数退避、默认并发 8(可配置 `1..=256`,report 按 plan 顺序、进度按完成数单调上报)、下载 quarantine 诊断、ZIP 结构校验、官方 seed `.hash` 校验、snapshot/cache、版本化 `official-launcher-bootstrap.json`、`--watch` 常驻更新、`--daemon` 后台运行,以及 Unix socket JSON-RPC live control/backend 方法(`daemon.*`、`resource.*`、`parse.*`、`translation.tasks/handoff/task.update`、`localized.status`、`catalog.*`、`task.*`、`patch.apply`、`unityfs.patch_*`)。
|
||||||
- `internal/backendrpc`:Go 侧 typed Unix socket JSON-RPC client,是 `bat-api` 调用 Rust daemon 的默认路径。
|
- `internal/backendrpc`:Go 侧 typed Unix socket JSON-RPC client,是 `bat-api` 调用 Rust daemon 的默认路径。
|
||||||
- `cmd/bat-api`:资源 bootstrap + 分发 HTTP MVP(issue #19 / G-009);`/v1/bootstrap` 和 `/v1/launcher/bootstrap` 组织 `bat` 已发布 release 的启动前资源入口,launcher 形状兼容端点仅输出资源 metadata / GameMainConfig 引导,`/healthz` 暴露 RPC refresh 诊断,`/readyz` 做 release readiness,CDN path 支持 `GET`/`HEAD`/`Range`、ETag、Last-Modified 和缓存头;玩家-facing 控制面已具备 token 鉴权、限流、访问日志、反代 IP 适配、动态 JSON no-store、OpenAPI 和管理控制白名单;`.env` 配置端口/RPC socket/刷新周期;生产资源根来自 RPC,不负责自动拉取。
|
- `cmd/bat-api`:资源 bootstrap + 分发 HTTP MVP(issue #19 / G-009);`/v1/bootstrap` 和 `/v1/launcher/bootstrap` 组织 `bat` 已发布 release 的启动前资源入口,launcher 形状兼容端点仅输出资源 metadata / GameMainConfig 引导,`/healthz` 暴露 RPC refresh 诊断,`/readyz` 做 release readiness,CDN path 支持 `GET`/`HEAD`/`Range`、ETag、Last-Modified 和缓存头;玩家-facing 控制面已具备 token 鉴权、限流、访问日志、反代 IP 适配、动态 JSON no-store、OpenAPI、管理控制白名单和 translation admin 查询/控制入口;`.env` 配置端口/RPC socket/刷新周期;生产资源根来自 RPC,不负责自动拉取。
|
||||||
- Go 边界权威说明:[`docs/reports/GO_STATUS.md`](docs/reports/GO_STATUS.md)(G-008 已关闭:同步 CLI = Rust `bat`)。
|
- Go 边界权威说明:[`docs/reports/GO_STATUS.md`](docs/reports/GO_STATUS.md)(G-008 已关闭:同步 CLI = Rust `bat`)。
|
||||||
- 官方同步会维护 `<output>/official-version-state.json`,明确记录当前已完成版本、正在拉取版本、上一个可用版本和失败版本。
|
- 官方同步会维护 `<output>/official-version-state.json`,明确记录当前已完成版本、正在拉取版本、上一个可用版本和失败版本。
|
||||||
- 资源导入链路可配置为在官方 release 发布后写入 CAS + `ResourceRepository`,资源 metadata 会记录 release、平台、bundle path、parse status、TextAsset 名称和 TextUnit 数量/格式,TextAsset/Table/Media 会按类型分类索引;`resource.index` RPC/CLI 可按类型、hash、路径模式、官方 release ID、平台、destination、archive entry、parse status 和 TextUnit format 分页查询索引。
|
- 资源导入链路可配置为在官方 release 发布后写入 CAS + `ResourceRepository`,资源 metadata 会记录 release、平台、bundle path、parse status、TextAsset 名称和 TextUnit 数量/格式,TextAsset/Table/Media 会按类型分类索引;`resource.index` RPC/CLI 可按类型、hash、路径模式、官方 release ID、平台、destination、archive entry、parse status 和 TextUnit format 分页查询索引。
|
||||||
@@ -186,7 +186,7 @@ BlueArchiveToolkit/
|
|||||||
1. 维护并联调 Go `bat-api` 资源 bootstrap/分发入口;`cmd/bat` 仅有 `doctor`、`manifest inspect`、`sync plan` 试验能力,不应误写成完整产品 CLI。
|
1. 维护并联调 Go `bat-api` 资源 bootstrap/分发入口;`cmd/bat` 仅有 `doctor`、`manifest inspect`、`sync plan` 试验能力,不应误写成完整产品 CLI。
|
||||||
2. 补齐 AssetBundle UnityFS 引擎级解析。
|
2. 补齐 AssetBundle UnityFS 引擎级解析。
|
||||||
3. 扩展 Addressables catalog 解析覆盖,继续用真实形态 fixture/golden 锁定行为。
|
3. 扩展 Addressables catalog 解析覆盖,继续用真实形态 fixture/golden 锁定行为。
|
||||||
4. 将 `official-textunit-tasks.json` / `crowdin-textunit-queue.json` 接入真实 Crowdin worker、翻译记忆和 Patch 构建。
|
4. 基于 `translation.worker.run` provider worker 推进翻译记忆、完整 Patch 构建和发布/回滚闭环。
|
||||||
5. 按 smoke runbook 在具备网络和磁盘窗口的环境中执行真实官方全量拉取,并保留本地报告。
|
5. 按 smoke runbook 在具备网络和磁盘窗口的环境中执行真实官方全量拉取,并保留本地报告。
|
||||||
|
|
||||||
不建议在 Go 产品入口、资源解析和文本提取基础能力完成前优先开发 Web UI。
|
不建议在 Go 产品入口、资源解析和文本提取基础能力完成前优先开发 Web UI。
|
||||||
|
|||||||
+7
-1
@@ -58,6 +58,7 @@ HTTPS_PROXY=http://user:pass@127.0.0.1:7890 bat --auto-discover --daemon
|
|||||||
| `i18n handoff` | 查询当前翻译交接视图 |
|
| `i18n handoff` | 查询当前翻译交接视图 |
|
||||||
| `i18n status` | 显示当前汉化 release 状态 |
|
| `i18n status` | 显示当前汉化 release 状态 |
|
||||||
| `i18n task update` | 回写 provider worker 任务状态 |
|
| `i18n task update` | 回写 provider worker 任务状态 |
|
||||||
|
| `i18n worker run` | 运行真实 provider worker;支持单次、限定次数和周期执行 |
|
||||||
| `i18n publish` | 校验工作台并发布独立汉化 release;`--force` 使用新的手动 release ID |
|
| `i18n publish` | 校验工作台并发布独立汉化 release;`--force` 使用新的手动 release ID |
|
||||||
| `i18n schedule` | 管理翻译和汉化发布计划 |
|
| `i18n schedule` | 管理翻译和汉化发布计划 |
|
||||||
| `refresh` | 执行一次更新检查;若有 live daemon,则通过 RPC 请求其刷新 |
|
| `refresh` | 执行一次更新检查;若有 live daemon,则通过 RPC 请求其刷新 |
|
||||||
@@ -137,13 +138,15 @@ BAT_API_SKIP_ENV_FILE=1 go run ./cmd/bat-api \
|
|||||||
| `GET /openapi.yaml` | bat-api OpenAPI 文档 |
|
| `GET /openapi.yaml` | bat-api OpenAPI 文档 |
|
||||||
| `GET /admin/` | 管理控制入口与允许操作列表 |
|
| `GET /admin/` | 管理控制入口与允许操作列表 |
|
||||||
| `GET /admin/schedules?id=...&group=...&enabled=...` | 读取/过滤 Rust `bat` 调度计划;需要管理 token |
|
| `GET /admin/schedules?id=...&group=...&enabled=...` | 读取/过滤 Rust `bat` 调度计划;需要管理 token |
|
||||||
|
| `GET /admin/translation/tasks?limit=100&worker_status=failed` | 读取/过滤 Rust 翻译任务和 provider worker 状态;需要管理 token |
|
||||||
|
| `GET /admin/translation/handoff` | 读取当前 release 的完整翻译交接视图;需要管理 token |
|
||||||
| `POST /admin/control/{action}` | 经白名单转发 Rust `bat` 控制请求;见下文 |
|
| `POST /admin/control/{action}` | 经白名单转发 Rust `bat` 控制请求;见下文 |
|
||||||
|
|
||||||
launcher 兼容端点只服务启动前资源发现。它们复用 Rust `bat` snapshot 中的 `launcher_metadata` 和 `game_main_config_bootstrap`,显式标记 `scope=resource_bootstrap_only` / `package_update_manifest=false`。`bat-api` 不下载 launcher 包,不生成官方 PC package update manifest,也不仿造登录、账号、网关、鉴权或游戏业务协议。
|
launcher 兼容端点只服务启动前资源发现。它们复用 Rust `bat` snapshot 中的 `launcher_metadata` 和 `game_main_config_bootstrap`,显式标记 `scope=resource_bootstrap_only` / `package_update_manifest=false`。`bat-api` 不下载 launcher 包,不生成官方 PC package update manifest,也不仿造登录、账号、网关、鉴权或游戏业务协议。
|
||||||
|
|
||||||
生产面对玩家分发时,应启用 HTTP token 鉴权、限流和访问日志:
|
生产面对玩家分发时,应启用 HTTP token 鉴权、限流和访问日志:
|
||||||
|
|
||||||
- `BAT_API_AUTH_TOKEN`:启用 `Authorization: Bearer <token>`、`X-BAT-Token` 或 query fallback 鉴权;token 推荐由 secret manager 或进程环境提供,不建议写入提交文件。`/admin/control/*` 和 `/admin/schedules` 需要此 token。
|
- `BAT_API_AUTH_TOKEN`:启用 `Authorization: Bearer <token>`、`X-BAT-Token` 或 query fallback 鉴权;token 推荐由 secret manager 或进程环境提供,不建议写入提交文件。`/admin/control/*`、`/admin/schedules` 和 `/admin/translation/*` 需要此 token。
|
||||||
- `BAT_API_AUTH_QUERY_PARAM`:query fallback 参数名,默认 `bat_token`;兼容不能写 header 的客户端,访问日志不会记录 query。
|
- `BAT_API_AUTH_QUERY_PARAM`:query fallback 参数名,默认 `bat_token`;兼容不能写 header 的客户端,访问日志不会记录 query。
|
||||||
- `BAT_API_AUTH_EXEMPT_PATHS`:逗号分隔的免鉴权 path 或 slash-prefix,例如 `/healthz,/readyz`。
|
- `BAT_API_AUTH_EXEMPT_PATHS`:逗号分隔的免鉴权 path 或 slash-prefix,例如 `/healthz,/readyz`。
|
||||||
- `BAT_API_RATE_LIMIT_RPS` / `BAT_API_RATE_LIMIT_BURST`:按客户端 IP 的进程内 token bucket 限流;边缘反代/CDN 仍应配置独立限流。
|
- `BAT_API_RATE_LIMIT_RPS` / `BAT_API_RATE_LIMIT_BURST`:按客户端 IP 的进程内 token bucket 限流;边缘反代/CDN 仍应配置独立限流。
|
||||||
@@ -167,6 +170,8 @@ launcher 兼容端点只服务启动前资源发现。它们复用 Rust `bat` sn
|
|||||||
| `schedule-remove` | `schedule.remove` | `{ "id": "..." }` | `202` + Rust schedule report |
|
| `schedule-remove` | `schedule.remove` | `{ "id": "..." }` | `202` + Rust schedule report |
|
||||||
| `schedule-run` | `schedule.run` | 可选 `{ "id": "...", "force": true }` | `202` + 执行报告 |
|
| `schedule-run` | `schedule.run` | 可选 `{ "id": "...", "force": true }` | `202` + 执行报告 |
|
||||||
| `translation-task-update` | `translation.task.update` | `{ "task_id": "...", "status": "...", "failure_reason": "...", "provider_run_id": "..." }` | `202` + 当前任务记录 |
|
| `translation-task-update` | `translation.task.update` | `{ "task_id": "...", "status": "...", "failure_reason": "...", "provider_run_id": "..." }` | `202` + 当前任务记录 |
|
||||||
|
| `translation-worker-run` | `translation.worker.run` | `{ "provider": "mock", "concurrency": 8, "max_tasks": 2 }` | `202` + worker task |
|
||||||
|
| `translation-proofread` | `translation.proofread` | 无 | `202` + 汉化状态 |
|
||||||
|
|
||||||
`stop`、`clean-stable`、patch 和 UnityFS 写入命令不会经 HTTP 暴露。
|
`stop`、`clean-stable`、patch 和 UnityFS 写入命令不会经 HTTP 暴露。
|
||||||
|
|
||||||
@@ -420,6 +425,7 @@ curl -i -H 'Range: bytes=0-1023' \
|
|||||||
| `translation.tasks` | ✅ | 查询离线 TextUnit 翻译任务及 worker 状态 |
|
| `translation.tasks` | ✅ | 查询离线 TextUnit 翻译任务及 worker 状态 |
|
||||||
| `translation.handoff` | ✅ | 查询完整 job/unit/provider run 交接视图 |
|
| `translation.handoff` | ✅ | 查询完整 job/unit/provider run 交接视图 |
|
||||||
| `translation.task.update` | ✅ | 回写当前 release 的 provider worker 状态 |
|
| `translation.task.update` | ✅ | 回写当前 release 的 provider worker 状态 |
|
||||||
|
| `translation.worker.run` | ✅ | 触发 Rust provider worker,落库 TextUnit 译文结果、lease、失败分类和重试状态 |
|
||||||
| `translation.proofread` | ✅ | 将当前汉化 workflow 标记为人工校对中 |
|
| `translation.proofread` | ✅ | 将当前汉化 workflow 标记为人工校对中 |
|
||||||
| `localized.status` | ✅ | 查询汉化 release 与当前官方 release 的匹配状态 |
|
| `localized.status` | ✅ | 查询汉化 release 与当前官方 release 的匹配状态 |
|
||||||
| `catalog.status` | ✅ | 当前已发布版本的 catalog 概览(app/bundle 版本、addressables 根、端点与 marker 计数、launcher 元数据) |
|
| `catalog.status` | ✅ | 当前已发布版本的 catalog 概览(app/bundle 版本、addressables 根、端点与 marker 计数、launcher 元数据) |
|
||||||
|
|||||||
+168
-2
@@ -108,16 +108,114 @@ paths:
|
|||||||
responses:
|
responses:
|
||||||
"200":
|
"200":
|
||||||
description: Admin links and allowlisted control actions.
|
description: Admin links and allowlisted control actions.
|
||||||
|
/admin/schedules:
|
||||||
|
get:
|
||||||
|
summary: List Rust-owned resource workflow schedules
|
||||||
|
parameters:
|
||||||
|
- name: id
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
- name: group
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
enum: [res, parse, i18n]
|
||||||
|
- name: enabled
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: boolean
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Current schedule JSON report.
|
||||||
|
"401":
|
||||||
|
description: Missing or invalid admin token.
|
||||||
|
"503":
|
||||||
|
description: Rust bat schedule backend is unavailable.
|
||||||
|
/admin/translation/tasks:
|
||||||
|
get:
|
||||||
|
summary: List Rust-owned translation task worker status
|
||||||
|
parameters:
|
||||||
|
- name: offset
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: integer
|
||||||
|
minimum: 0
|
||||||
|
- name: limit
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: integer
|
||||||
|
minimum: 1
|
||||||
|
maximum: 1000
|
||||||
|
- name: task_id
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
- name: release_id
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
- name: destination
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
- name: archive_entry
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
- name: status
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
- name: worker_status
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
- name: parse_status
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
- name: format
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
- name: has_reason
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: boolean
|
||||||
|
- name: has_failure_reason
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: boolean
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Current translation task JSON report from Rust bat.
|
||||||
|
"400":
|
||||||
|
description: Invalid translation task query.
|
||||||
|
"401":
|
||||||
|
description: Missing or invalid admin token.
|
||||||
|
"503":
|
||||||
|
description: Rust bat translation backend is unavailable.
|
||||||
|
/admin/translation/handoff:
|
||||||
|
get:
|
||||||
|
summary: Read Rust-owned translation handoff state
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Current translation handoff JSON report from Rust bat.
|
||||||
|
"401":
|
||||||
|
description: Missing or invalid admin token.
|
||||||
|
"503":
|
||||||
|
description: Rust bat translation backend is unavailable.
|
||||||
/admin/control/{action}:
|
/admin/control/{action}:
|
||||||
post:
|
post:
|
||||||
summary: Forward an allowlisted control action to Rust bat
|
summary: Forward an allowlisted control or schedule action to Rust bat
|
||||||
parameters:
|
parameters:
|
||||||
- name: action
|
- name: action
|
||||||
in: path
|
in: path
|
||||||
required: true
|
required: true
|
||||||
schema:
|
schema:
|
||||||
type: string
|
type: string
|
||||||
enum: [reload, refresh, restart, sync, verify, repair, catalog-refresh]
|
enum: [reload, refresh, restart, sync, verify, repair, catalog-refresh, schedule-add, schedule-update, schedule-remove, schedule-run, translation-task-update, translation-worker-run, translation-proofread]
|
||||||
requestBody:
|
requestBody:
|
||||||
required: false
|
required: false
|
||||||
content:
|
content:
|
||||||
@@ -128,6 +226,74 @@ paths:
|
|||||||
properties:
|
properties:
|
||||||
force:
|
force:
|
||||||
type: boolean
|
type: boolean
|
||||||
|
id:
|
||||||
|
type: string
|
||||||
|
group:
|
||||||
|
type: string
|
||||||
|
action:
|
||||||
|
type: string
|
||||||
|
args:
|
||||||
|
type: array
|
||||||
|
items:
|
||||||
|
type: string
|
||||||
|
next_run_unix_seconds:
|
||||||
|
type: integer
|
||||||
|
format: int64
|
||||||
|
delay_seconds:
|
||||||
|
type: integer
|
||||||
|
format: int64
|
||||||
|
every_seconds:
|
||||||
|
type: integer
|
||||||
|
format: int64
|
||||||
|
count:
|
||||||
|
type: integer
|
||||||
|
format: int64
|
||||||
|
max_runs:
|
||||||
|
type: integer
|
||||||
|
format: int64
|
||||||
|
minimum: 1
|
||||||
|
clear_args:
|
||||||
|
type: boolean
|
||||||
|
clear_every:
|
||||||
|
type: boolean
|
||||||
|
enabled:
|
||||||
|
type: boolean
|
||||||
|
task_id:
|
||||||
|
type: string
|
||||||
|
status:
|
||||||
|
type: string
|
||||||
|
failure_reason:
|
||||||
|
type: string
|
||||||
|
provider_run_id:
|
||||||
|
type: string
|
||||||
|
provider:
|
||||||
|
type: string
|
||||||
|
enum: [mock, crowdin]
|
||||||
|
fixture_path:
|
||||||
|
type: string
|
||||||
|
concurrency:
|
||||||
|
type: integer
|
||||||
|
format: int64
|
||||||
|
minimum: 1
|
||||||
|
maximum: 256
|
||||||
|
max_attempts:
|
||||||
|
type: integer
|
||||||
|
format: int64
|
||||||
|
minimum: 1
|
||||||
|
lease_seconds:
|
||||||
|
type: integer
|
||||||
|
format: int64
|
||||||
|
minimum: 1
|
||||||
|
retry_backoff_seconds:
|
||||||
|
type: integer
|
||||||
|
format: int64
|
||||||
|
minimum: 0
|
||||||
|
max_tasks:
|
||||||
|
type: integer
|
||||||
|
format: int64
|
||||||
|
minimum: 1
|
||||||
|
worker_id:
|
||||||
|
type: string
|
||||||
responses:
|
responses:
|
||||||
"202":
|
"202":
|
||||||
description: Rust bat accepted the control request.
|
description: Rust bat accepted the control request.
|
||||||
|
|||||||
@@ -208,6 +208,6 @@ catalog 仍按“明确不支持”处理,不把低保真路径伪装成完整
|
|||||||
|
|
||||||
1. 完成 Addressables Windows/Android 当前版本 catalog 样本集合,关闭 G-007 当前阶段。
|
1. 完成 Addressables Windows/Android 当前版本 catalog 样本集合,关闭 G-007 当前阶段。
|
||||||
2. 完成 TypeTree 字段 reader 和 MonoBehaviour/ScriptableObject 遍历,推进 G-005。
|
2. 完成 TypeTree 字段 reader 和 MonoBehaviour/ScriptableObject 遍历,推进 G-005。
|
||||||
3. 将 `crowdin-textunit-queue.json` 接入真实 Crowdin worker、翻译记忆和 Patch 构建。
|
3. 基于 `translation.worker.run` 推进翻译记忆和 Patch 构建。
|
||||||
4. 将翻译任务状态接入 CAS/ResourceRepository 查询面,推进 G-011。
|
4. 扩展翻译任务结果在 CAS/ResourceRepository 查询面的索引,推进 G-011。
|
||||||
5. 在通用 Binary/JSON/Text Patch 基础上继续扩展复杂 AssetBundle 重打包和发布流程统一,保留当前 UnityFS TextAsset patch 发布前置链路。
|
5. 在通用 Binary/JSON/Text Patch 基础上继续扩展复杂 AssetBundle 重打包和发布流程统一,保留当前 UnityFS TextAsset patch 发布前置链路。
|
||||||
|
|||||||
@@ -179,8 +179,9 @@
|
|||||||
`resource.index` RPC / CLI 只读查询现有 SQLite 索引;索引不存在时返回
|
`resource.index` RPC / CLI 只读查询现有 SQLite 索引;索引不存在时返回
|
||||||
`available=false`,不会因为查询创建空库。发布后的 TextUnit 队列还会在当前
|
`available=false`,不会因为查询创建空库。发布后的 TextUnit 队列还会在当前
|
||||||
release 根目录写入 `translation-tasks.sqlite`,由版本化 `schema_migrations`
|
release 根目录写入 `translation-tasks.sqlite`,由版本化 `schema_migrations`
|
||||||
管理 queued/running/failed/completed/skipped 和 failure reason。`translation.tasks`
|
管理 queued/running/failed/completed/skipped、provider run、lease、失败分类、
|
||||||
优先查询这份状态库,`translation.task.update` 供 provider worker 回写状态;
|
重试计划和 TextUnit 级译文结果。`translation.tasks` 优先查询这份状态库,
|
||||||
|
`translation.worker.run` 由 Rust worker 回写状态;`translation.task.update` 仍供外部 provider 流程回写状态;
|
||||||
没有状态库的旧 release 才回退到 immutable JSON 队列。G-011 剩余工作是 CAS
|
没有状态库的旧 release 才回退到 immutable JSON 队列。G-011 剩余工作是 CAS
|
||||||
诊断入口和面向大索引的查询优化。
|
诊断入口和面向大索引的查询优化。
|
||||||
|
|
||||||
|
|||||||
@@ -92,8 +92,8 @@ CAS V1 和 Rust 官方同步闭环完成后,下一阶段优先推进:
|
|||||||
|
|
||||||
1. 继续联调 Go `bat-api` 与 Rust daemon 的资源分发路径;Go 同步 CLI 不再作为产品目标。
|
1. 继续联调 Go `bat-api` 与 Rust daemon 的资源分发路径;Go 同步 CLI 不再作为产品目标。
|
||||||
2. 按 `docs/guides/official-full-pull-smoke.md` 在隔离目录执行真实官方网络全量下载 smoke,并保留运行报告。
|
2. 按 `docs/guides/official-full-pull-smoke.md` 在隔离目录执行真实官方网络全量下载 smoke,并保留运行报告。
|
||||||
3. 将 `crowdin-textunit-queue.json` 接入真实 Crowdin worker、翻译记忆和 Patch 构建。
|
3. 基于 `translation.worker.run` 推进翻译记忆和 Patch 构建。
|
||||||
4. 扩展 ResourceRepository 查询面:真实 provider worker 集成与状态落库验证、CAS 诊断入口和更丰富 TextUnit 查询。
|
4. 扩展 ResourceRepository 查询面:CAS 诊断入口、面向大索引的查询优化和更丰富 TextUnit 查询。
|
||||||
5. 继续完善 AssetBundle 复杂对象解析、复杂对象重打包和 Patch 发布流程统一;通用 Binary/JSON/Text Patch 基础与 UnityFS TextAsset patch 发布前置链路已可用。
|
5. 继续完善 AssetBundle 复杂对象解析、复杂对象重打包和 Patch 发布流程统一;通用 Binary/JSON/Text Patch 基础与 UnityFS TextAsset patch 发布前置链路已可用。
|
||||||
|
|
||||||
优先阅读:
|
优先阅读:
|
||||||
|
|||||||
@@ -133,9 +133,30 @@ bat i18n publish \
|
|||||||
|
|
||||||
发布只接受当前实现支持的直接 TextAsset 条目;TypeTree 字段和 zip 内 bundle 使用 `parse repack` 的 spec 单独处理。`--force` 不覆盖已有目录,而是生成独立的 `<official-release>-manual-<unix-seconds>` 汉化 release ID;也可以用 `--localized-release-id` 显式指定新 ID。因此强制发布仍保留旧 release 和 rollback 信息。
|
发布只接受当前实现支持的直接 TextAsset 条目;TypeTree 字段和 zip 内 bundle 使用 `parse repack` 的 spec 单独处理。`--force` 不覆盖已有目录,而是生成独立的 `<official-release>-manual-<unix-seconds>` 汉化 release ID;也可以用 `--localized-release-id` 显式指定新 ID。因此强制发布仍保留旧 release 和 rollback 信息。
|
||||||
|
|
||||||
当前 `i18n run` 是离线工作流:刷新 TextUnit 队列,并可用 `--translation-file` 导出工作台;不会假装调用未接入的外部 provider。
|
当前 `i18n run` 是离线工作流:刷新 TextUnit 队列,并可用 `--translation-file` 导出工作台;真实 provider 由单独的 worker 命令消费 `translation-tasks.sqlite`。
|
||||||
|
|
||||||
provider worker 消费 handoff 后,可以用 `i18n task update` 回写当前 release 的任务状态:
|
运行一次 mock provider worker:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bat i18n worker run \
|
||||||
|
--output /tmp/bat-resources \
|
||||||
|
--provider mock \
|
||||||
|
--worker-concurrency 8
|
||||||
|
```
|
||||||
|
|
||||||
|
`--provider` 支持 `mock` 和 `crowdin`。`mock` 可通过 `--translation-fixture`
|
||||||
|
读取本地 JSON fixture;`crowdin` 从环境变量 `CROWDIN_PROJECT_ID`、
|
||||||
|
`CROWDIN_LANGUAGE_ID`、`CROWDIN_API_TOKEN` 读取配置。worker 默认并发为 8,
|
||||||
|
范围 `1..=256`;每个 worker 完成当前任务后立即从 SQLite 队列领取下一项,
|
||||||
|
不会等待当前一批 worker 全部结束后再重新分配。
|
||||||
|
|
||||||
|
可用参数包括 `--worker-max-attempts`、`--worker-lease-seconds`、
|
||||||
|
`--worker-retry-backoff` / `--worker-retry-backoff-seconds`、
|
||||||
|
`--worker-max-tasks` 和 `--worker-id`。worker 支持 `--run-count` 与
|
||||||
|
`--watch`,因此可以单次、限定次数或周期执行;`--run-count > 1` 时仍必须
|
||||||
|
显式指定 `--interval`。
|
||||||
|
|
||||||
|
外部 provider 或人工流程也可以用 `i18n task update` 回写当前 release 的任务状态:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
bat i18n task update \
|
bat i18n task update \
|
||||||
@@ -224,11 +245,24 @@ contract:`id`、`group`、`action`、`args`、`next_run_unix_seconds`、
|
|||||||
`enabled`;`schedule.list` 额外接受 `id`、`group`、`enabled` 过滤,
|
`enabled`;`schedule.list` 额外接受 `id`、`group`、`enabled` 过滤,
|
||||||
`schedule.run` 额外接受 `group`、`force` 和 `max_runs`。
|
`schedule.run` 额外接受 `group`、`force` 和 `max_runs`。
|
||||||
|
|
||||||
|
翻译任务状态可由已鉴权的 dashboard 通过 `GET /admin/translation/tasks`
|
||||||
|
查询,query 过滤项包括 `offset`、`limit`、`task_id`、`release_id`、
|
||||||
|
`destination`、`path_pattern`、`archive_entry`、`status`、`worker_status`、
|
||||||
|
`parse_status`、`format`、`has_reason` 和 `has_failure_reason`。完整 provider
|
||||||
|
run 交接视图通过 `GET /admin/translation/handoff` 查询。两个查询接口都只转发
|
||||||
|
Rust `translation.tasks` / `translation.handoff`,不在 Go 侧维护状态。
|
||||||
|
|
||||||
翻译任务状态也可由已鉴权的 dashboard 通过
|
翻译任务状态也可由已鉴权的 dashboard 通过
|
||||||
`POST /admin/control/translation-task-update` 回写,请求字段为
|
`POST /admin/control/translation-task-update` 回写,请求字段为
|
||||||
`task_id`、`status`,以及可选的 `failure_reason`、`provider_run_id`;
|
`task_id`、`status`,以及可选的 `failure_reason`、`provider_run_id`;
|
||||||
该接口只转发 `translation.task.update`。
|
该接口只转发 `translation.task.update`。
|
||||||
|
|
||||||
|
`POST /admin/control/translation-worker-run` 会触发 Rust 侧
|
||||||
|
`translation.worker.run`,请求字段为 `provider`、`fixture_path`、
|
||||||
|
`concurrency`、`max_attempts`、`lease_seconds`、`retry_backoff_seconds`、
|
||||||
|
`max_tasks` 和 `worker_id`。bat-api 只做鉴权、JSON 解码和基础范围校验;
|
||||||
|
任务状态、lease、重试和 provider 结果仍由 Rust 持久化。
|
||||||
|
|
||||||
`POST /admin/control/translation-proofread` 会把当前汉化 workflow 标记为人工校对中;
|
`POST /admin/control/translation-proofread` 会把当前汉化 workflow 标记为人工校对中;
|
||||||
该接口只转发 `translation.proofread`,不会改动已发布汉化 release 指针。
|
该接口只转发 `translation.proofread`,不会改动已发布汉化 release 指针。
|
||||||
|
|
||||||
|
|||||||
@@ -258,6 +258,7 @@ cargo run -p bat-infrastructure --bin bat -- parse-errors --limit 50
|
|||||||
cargo run -p bat-infrastructure --bin bat -- translation-tasks --task-status skipped_parse_failed --has-reason --limit 50
|
cargo run -p bat-infrastructure --bin bat -- translation-tasks --task-status skipped_parse_failed --has-reason --limit 50
|
||||||
cargo run -p bat-infrastructure --bin bat -- translation-tasks --worker-status failed --has-failure-reason --limit 50
|
cargo run -p bat-infrastructure --bin bat -- translation-tasks --worker-status failed --has-failure-reason --limit 50
|
||||||
cargo run -p bat-infrastructure --bin bat -- translation-handoff
|
cargo run -p bat-infrastructure --bin bat -- translation-handoff
|
||||||
|
cargo run -p bat-infrastructure --bin bat -- i18n worker run --provider mock --worker-concurrency 8 --worker-max-tasks 10
|
||||||
cargo run -p bat-infrastructure --bin bat -- localized-status
|
cargo run -p bat-infrastructure --bin bat -- localized-status
|
||||||
cargo run -p bat-infrastructure --bin bat -- resource-index --limit 50
|
cargo run -p bat-infrastructure --bin bat -- resource-index --limit 50
|
||||||
cargo run -p bat-infrastructure --bin bat -- resource-index --release-id <ID> --platform windows --archive-entry <PATH> --format json --limit 50
|
cargo run -p bat-infrastructure --bin bat -- resource-index --release-id <ID> --platform windows --archive-entry <PATH> --format json --limit 50
|
||||||
@@ -270,6 +271,9 @@ cargo run -p bat-infrastructure --bin bat -- resource-index --release-id <ID> --
|
|||||||
worker 状态、parse status、TextUnit format、队列 reason 和 provider failure reason
|
worker 状态、parse status、TextUnit format、队列 reason 和 provider failure reason
|
||||||
查询离线 TextUnit 翻译任务状态与跳过/失败原因;发布后的状态保存在当前 release
|
查询离线 TextUnit 翻译任务状态与跳过/失败原因;发布后的状态保存在当前 release
|
||||||
根目录的 `translation-tasks.sqlite`,旧 release 没有状态库时回退到 JSON 队列;
|
根目录的 `translation-tasks.sqlite`,旧 release 没有状态库时回退到 JSON 队列;
|
||||||
|
`i18n worker run` / `translation.worker.run` 会由 Rust provider worker 独立 claim
|
||||||
|
下一项任务并落库 lease、失败分类、重试计划和 TextUnit 级译文结果,默认并发为 8,
|
||||||
|
范围 `1..=256`;
|
||||||
`translation-handoff` / `translation.handoff` 会动态合并版本化
|
`translation-handoff` / `translation.handoff` 会动态合并版本化
|
||||||
`translation-handoff.json` 与 SQLite 状态,返回 job、unit、provider run 的完整交接
|
`translation-handoff.json` 与 SQLite 状态,返回 job、unit、provider run 的完整交接
|
||||||
视图;
|
视图;
|
||||||
|
|||||||
@@ -150,11 +150,12 @@ SQLite `ResourceRepository`,索引不存在时返回 `ok=true` 且
|
|||||||
本地文件未变且索引有效时复用,不重复解析。
|
本地文件未变且索引有效时复用,不重复解析。
|
||||||
- `official-textunit-tasks.json`:只由 added + modified 资源、parse cache 和
|
- `official-textunit-tasks.json`:只由 added + modified 资源、parse cache 和
|
||||||
TextUnit 明细索引派生,记录 TextUnit 任务、跳过原因和解析诊断。
|
TextUnit 明细索引派生,记录 TextUnit 任务、跳过原因和解析诊断。
|
||||||
- `crowdin-textunit-queue.json`:只包含已产生 TextUnit 的离线任务,预留给后续
|
- `crowdin-textunit-queue.json`:只包含已产生 TextUnit 的离线任务;官方同步阶段
|
||||||
Crowdin worker;当前不会发出网络请求。
|
不发出 provider 网络请求。
|
||||||
- `translation-tasks.sqlite`:当前 release 的可变 worker 状态库,记录
|
- `translation-tasks.sqlite`:当前 release 的可变 worker 状态库,记录
|
||||||
queued / running / failed / completed / skipped、attempt count、provider run
|
queued / running / failed / completed / skipped、attempt count、provider run
|
||||||
ID 和 failure reason;schema 由 `schema_migrations` 版本表管理。
|
ID、provider、TextUnit 级译文结果、lease、失败分类、可重试标记和
|
||||||
|
next attempt;schema 由 `schema_migrations` 版本表管理。
|
||||||
- `translation-handoff.json`:当前 release 的版本化 job/unit/provider run 交接
|
- `translation-handoff.json`:当前 release 的版本化 job/unit/provider run 交接
|
||||||
快照;worker 更新后的实时状态仍以 `translation-tasks.sqlite` 为准。
|
快照;worker 更新后的实时状态仍以 `translation-tasks.sqlite` 为准。
|
||||||
|
|
||||||
@@ -206,6 +207,7 @@ SQLite `ResourceRepository`,索引不存在时返回 `ok=true` 且
|
|||||||
| `translation.tasks` | 已实现 | `{ "offset": 0, "limit": 100, "task_id": "...", "release_id": "...", "destination": "...", "archive_entry": "...", "status": "skipped_parse_failed", "parse_status": "failed", "format": "json", "has_reason": true }` | 当前官方 release 的离线 TextUnit 翻译任务状态分页。 |
|
| `translation.tasks` | 已实现 | `{ "offset": 0, "limit": 100, "task_id": "...", "release_id": "...", "destination": "...", "archive_entry": "...", "status": "skipped_parse_failed", "parse_status": "failed", "format": "json", "has_reason": true }` | 当前官方 release 的离线 TextUnit 翻译任务状态分页。 |
|
||||||
| `translation.handoff` | 已实现 | `null` | 当前官方 release 的 job、unit、provider run 交接视图;动态合并队列和 SQLite worker 状态。 |
|
| `translation.handoff` | 已实现 | `null` | 当前官方 release 的 job、unit、provider run 交接视图;动态合并队列和 SQLite worker 状态。 |
|
||||||
| `translation.task.update` | 已实现 | `{ "task_id": "...", "status": "failed", "failure_reason": "...", "provider_run_id": "..." }` | 写入当前 release 的 provider worker 状态,返回可回查任务记录。 |
|
| `translation.task.update` | 已实现 | `{ "task_id": "...", "status": "failed", "failure_reason": "...", "provider_run_id": "..." }` | 写入当前 release 的 provider worker 状态,返回可回查任务记录。 |
|
||||||
|
| `translation.worker.run` | 已实现 | provider worker 参数 | 异步触发 Rust provider worker,返回 `{ "task_id": "...", "kind": "translation.worker.run", "worker": {...} }`。 |
|
||||||
| `translation.proofread` | 已实现 | `null` | 将当前汉化 workflow 标记为人工校对中,返回工作流状态报告。 |
|
| `translation.proofread` | 已实现 | `null` | 将当前汉化 workflow 标记为人工校对中,返回工作流状态报告。 |
|
||||||
|
|
||||||
`parse.status` 是只读查询;没有当前 release 或没有解析缓存时返回
|
`parse.status` 是只读查询;没有当前 release 或没有解析缓存时返回
|
||||||
@@ -215,8 +217,8 @@ SQLite `ResourceRepository`,索引不存在时返回 `ok=true` 且
|
|||||||
`textunit_index_summary`;存在 `official-textunit-tasks.json` 时,响应会包含
|
`textunit_index_summary`;存在 `official-textunit-tasks.json` 时,响应会包含
|
||||||
`textunit_queue_available=true`、`textunit_task_queue_path` 和
|
`textunit_queue_available=true`、`textunit_task_queue_path` 和
|
||||||
`textunit_task_summary`。当 TextUnit 队列存在且有离线任务时,
|
`textunit_task_summary`。当 TextUnit 队列存在且有离线任务时,
|
||||||
`translation_status_code=translation.queued_offline`;真实 Crowdin worker
|
`translation_status_code=translation.queued_offline`;provider worker 完成任务后,
|
||||||
尚未接入时不会返回翻译完成状态。
|
同一查询面会返回已落库的 worker 状态和 TextUnit 级译文结果。
|
||||||
|
|
||||||
`parse.text_units` / `parse.errors` 是只读查询;没有当前 release 或没有
|
`parse.text_units` / `parse.errors` 是只读查询;没有当前 release 或没有
|
||||||
`official-textunit-index.json` 时返回 `ok=true` 且 `data.available=false`。
|
`official-textunit-index.json` 时返回 `ok=true` 且 `data.available=false`。
|
||||||
@@ -260,6 +262,31 @@ bat-api 可通过 `translation.tasks` 查询单项任务,也可通过
|
|||||||
不会触发下载或 provider 网络请求;没有当前 release 或任务队列时返回
|
不会触发下载或 provider 网络请求;没有当前 release 或任务队列时返回
|
||||||
`data.available=false`。
|
`data.available=false`。
|
||||||
|
|
||||||
|
`translation.worker.run` 通过 daemon 任务队列异步启动 Rust provider worker。
|
||||||
|
provider worker 会先同步当前 release 的 TextUnit 队列到
|
||||||
|
`translation-tasks.sqlite`,回收过期 lease,然后由 `concurrency` 个独立 worker
|
||||||
|
循环 claim 下一项任务;任一 worker 完成当前任务后会立即领取下一项,不等待
|
||||||
|
其他 worker 完成本轮批次。默认并发为 8,范围 `1..=256`。
|
||||||
|
|
||||||
|
provider worker 参数:
|
||||||
|
|
||||||
|
| 字段 | 类型 | 默认 | 说明 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `provider` | string | `mock` | `mock` 或 `crowdin`。 |
|
||||||
|
| `fixture_path` | string/null | `null` | mock provider fixture;别名为 `translation_fixture`、`provider_fixture`、`mock_fixture`、`fixture`。 |
|
||||||
|
| `concurrency` | uint | `8` | 独立 worker 数,范围 `1..=256`;别名为 `worker_concurrency`、`translation_concurrency`。 |
|
||||||
|
| `max_attempts` | uint | `3` | 单个任务最大 claim 次数,必须大于 0。 |
|
||||||
|
| `lease_seconds` | uint | `300` | claim lease 秒数,必须大于 0。 |
|
||||||
|
| `retry_backoff_seconds` | uint | `5` | 可重试 provider 失败的 next attempt 间隔,可为 0。 |
|
||||||
|
| `max_tasks` | uint/null | `null` | 本轮最多 claim 的任务数,设置时必须大于 0。 |
|
||||||
|
| `worker_id` | string | `bat-rpc-worker` | lease 诊断用 worker ID 前缀。 |
|
||||||
|
|
||||||
|
数字字段必须是 JSON number;字符串数字、负数和越界值会返回
|
||||||
|
`BAT-ERR-700002`。`mock` provider 在没有 fixture 时把 source text 写成可诊断的
|
||||||
|
mock 译文;`crowdin` provider 从 `CROWDIN_PROJECT_ID`、`CROWDIN_LANGUAGE_ID`、
|
||||||
|
`CROWDIN_API_TOKEN` 读取配置,可选 `CROWDIN_API_BASE_URL` 和 `BAT_CURL`。
|
||||||
|
token 不会进入报告、任务记录或调试输出。
|
||||||
|
|
||||||
### localized
|
### localized
|
||||||
|
|
||||||
| 方法 | 状态 | params | data |
|
| 方法 | 状态 | params | data |
|
||||||
@@ -397,6 +424,7 @@ CLI 对应关系:
|
|||||||
| `bat translation-handoff` | `translation.handoff` |
|
| `bat translation-handoff` | `translation.handoff` |
|
||||||
| `bat i18n task list` / `bat i18n task status` | `translation.tasks` |
|
| `bat i18n task list` / `bat i18n task status` | `translation.tasks` |
|
||||||
| `bat i18n task update` | `translation.task.update` |
|
| `bat i18n task update` | `translation.task.update` |
|
||||||
|
| `bat i18n worker run` | `translation.worker.run` |
|
||||||
| `bat i18n proofread` | `translation.proofread` |
|
| `bat i18n proofread` | `translation.proofread` |
|
||||||
| `bat localized-status` | `localized.status` |
|
| `bat localized-status` | `localized.status` |
|
||||||
| `bat resource-index` | `resource.index` |
|
| `bat resource-index` | `resource.index` |
|
||||||
@@ -424,10 +452,10 @@ CLI 对应关系:
|
|||||||
- typed helper 已覆盖 daemon 已实现方法(`status/logs/stop/restart/reload/refresh/doctor`)、
|
- typed helper 已覆盖 daemon 已实现方法(`status/logs/stop/restart/reload/refresh/doctor`)、
|
||||||
`resource.state/sync/verify/repair/manifest/list`、`schedule.list/add/update/remove/run`、
|
`resource.state/sync/verify/repair/manifest/list`、`schedule.list/add/update/remove/run`、
|
||||||
`catalog.*`、`parse.*`、
|
`catalog.*`、`parse.*`、
|
||||||
`localized.status`、`translation.task.update`、`translation.proofread`、`task.*` 和三个
|
`localized.status`、`translation.tasks`、`translation.handoff`、
|
||||||
`unityfs.patch_*` 方法。
|
`translation.task.update`、`translation.worker.run`、`translation.proofread`、
|
||||||
- `resource.index`、`translation.tasks`、`translation.handoff` 和
|
`task.*` 和三个 `unityfs.patch_*` 方法。
|
||||||
`patch.apply` 当前没有专用 typed helper;需要直接使用 `Call`,并仍须遵守
|
- `resource.index` 和 `patch.apply` 当前没有专用 typed helper;需要直接使用 `Call`,并仍须遵守
|
||||||
本契约的参数和响应定义。
|
本契约的参数和响应定义。
|
||||||
|
|
||||||
`internal/api` 对 bat-api 生产路径进一步收窄接口:
|
`internal/api` 对 bat-api 生产路径进一步收窄接口:
|
||||||
@@ -437,7 +465,7 @@ CLI 对应关系:
|
|||||||
| `Backend` | `daemon.status`、`daemon.doctor`、`resource.state`、`catalog.status`、`resource.manifest` | 启动发现、周期刷新和资源分发 |
|
| `Backend` | `daemon.status`、`daemon.doctor`、`resource.state`、`catalog.status`、`resource.manifest` | 启动发现、周期刷新和资源分发 |
|
||||||
| `ControlBackend` | `daemon.restart`、`daemon.reload`、`daemon.refresh`、`resource.sync`、`resource.verify`、`resource.repair`、`catalog.refresh` | 鉴权后的管理控制白名单 |
|
| `ControlBackend` | `daemon.restart`、`daemon.reload`、`daemon.refresh`、`resource.sync`、`resource.verify`、`resource.repair`、`catalog.refresh` | 鉴权后的管理控制白名单 |
|
||||||
| `ScheduleBackend` | `schedule.list`、`schedule.add`、`schedule.update`、`schedule.remove`、`schedule.run` | 鉴权后的 dashboard 调度计划控制 |
|
| `ScheduleBackend` | `schedule.list`、`schedule.add`、`schedule.update`、`schedule.remove`、`schedule.run` | 鉴权后的 dashboard 调度计划控制 |
|
||||||
| `TranslationBackend` | `translation.task.update`、`translation.proofread` | 鉴权后的 dashboard 翻译任务状态回写与人工校对标记 |
|
| `TranslationBackend` | `translation.tasks`、`translation.handoff`、`translation.task.update`、`translation.worker.run`、`translation.proofread` | 鉴权后的 dashboard 翻译任务查询、交接视图、状态回写、provider worker 触发与人工校对标记 |
|
||||||
|
|
||||||
`daemon.stop`、`daemon.clean-stable` 和任意通用 RPC 不属于 bat-api 管理控制面。
|
`daemon.stop`、`daemon.clean-stable` 和任意通用 RPC 不属于 bat-api 管理控制面。
|
||||||
Rust dispatch、Go transport 和 bat-api 接口的权威实现位置分别是
|
Rust dispatch、Go transport 和 bat-api 接口的权威实现位置分别是
|
||||||
|
|||||||
@@ -232,8 +232,8 @@ issue 43 例外说明:新增的 `parse repack` 只编排已有 TextAsset、Typ
|
|||||||
- `cmd/bat-api`:组织 Rust `bat` 已发布 release 的启动前资源入口,并只读分发官方 CDN host/path 形态资源。
|
- `cmd/bat-api`:组织 Rust `bat` 已发布 release 的启动前资源入口,并只读分发官方 CDN host/path 形态资源。
|
||||||
- **拉取归属 Rust `bat`**;`bat-api` 不做下载器。
|
- **拉取归属 Rust `bat`**;`bat-api` 不做下载器。
|
||||||
- 发现经 `bat.sock`:先 `daemon.status`,再 `daemon.doctor`,再 `catalog.status` / `resource.manifest`。
|
- 发现经 `bat.sock`:先 `daemon.status`,再 `daemon.doctor`,再 `catalog.status` / `resource.manifest`。
|
||||||
- Rust RPC 的 `resource.state`、`catalog.status`、`parse.status` 和 `localized.status` 会返回短状态 `status` 与稳定生命周期状态码 `status_code`;`backendrpc` 已提供对应 client 能力,当前 `bat-api` 发现流程只消费 `resource.state`、`catalog.status`、`resource.manifest`,parse/localized 不由 HTTP surface 暴露;错误原因仍以 `BAT-ERR-*` 为准。
|
- Rust RPC 的 `resource.state`、`catalog.status`、`parse.status` 和 `localized.status` 会返回短状态 `status` 与稳定生命周期状态码 `status_code`;`backendrpc` 已提供对应 client 能力,当前 `bat-api` 发现流程只消费 `resource.state`、`catalog.status`、`resource.manifest`,translation task/handoff 通过受鉴权的 admin 查询端点暴露,parse/localized 不由 HTTP surface 暴露;错误原因仍以 `BAT-ERR-*` 为准。
|
||||||
- `bat-api` 可通过受限 Web 控制面转发 `reload` / `refresh` / `restart` / `sync` / `verify` / `repair` / `catalog-refresh`,这些控制动作均走 Rust live RPC;`parse.*`、`localized.status` 和文件级 `unityfs.patch_*` 属于 `backendrpc` 能力,但当前不由 HTTP surface 暴露;`daemon.clean-stable` 等危险或离线生命周期命令不经 Web 转发。
|
- `bat-api` 可通过受限 Web 控制面转发 `reload` / `refresh` / `restart` / `sync` / `verify` / `repair` / `catalog-refresh`,并通过 translation admin 端点查询/回写任务、触发 provider worker 和标记人工校对;这些动作均走 Rust live RPC;`parse.*`、`localized.status` 和文件级 `unityfs.patch_*` 属于 `backendrpc` 能力,但当前不由 HTTP surface 暴露;`daemon.clean-stable` 等危险或离线生命周期命令不经 Web 转发。
|
||||||
- 生产与 Rust `bat` 同环境运行,资源根来自 RPC 返回的 `resource_root`;`--resource-root` 仅用于 fixture 或应急只读诊断。
|
- 生产与 Rust `bat` 同环境运行,资源根来自 RPC 返回的 `resource_root`;`--resource-root` 仅用于 fixture 或应急只读诊断。
|
||||||
- `.env` 配置端口 / public base / RPC socket / RPC 刷新周期;预留 database/redis。
|
- `.env` 配置端口 / public base / RPC socket / RPC 刷新周期;预留 database/redis。
|
||||||
- `/v1/bootstrap` 返回 RPC 健康、release 摘要、server-info URL、client-patch base 和改写后的 Addressables root。
|
- `/v1/bootstrap` 返回 RPC 健康、release 摘要、server-info URL、client-patch base 和改写后的 Addressables root。
|
||||||
@@ -294,9 +294,9 @@ issue 43 例外说明:新增的 `parse repack` 只编排已有 TextAsset、Typ
|
|||||||
- 官方 release 发布后还会写入版本化 `translation-handoff.json`;`translation.handoff` RPC/CLI 动态合并该快照与 SQLite worker 状态,暴露 job、unit、provider run、attempt 和 failure reason。
|
- 官方 release 发布后还会写入版本化 `translation-handoff.json`;`translation.handoff` RPC/CLI 动态合并该快照与 SQLite worker 状态,暴露 job、unit、provider run、attempt 和 failure reason。
|
||||||
- `translation-tasks.sqlite` 由 `schema_migrations` 管理 durable task state、attempt count、provider run ID 和 failure reason;重复同步会保留已有 worker 状态。
|
- `translation-tasks.sqlite` 由 `schema_migrations` 管理 durable task state、attempt count、provider run ID 和 failure reason;重复同步会保留已有 worker 状态。
|
||||||
- `translation.tasks` RPC/CLI 优先查询 `translation-tasks.sqlite`,旧 release 没有状态库时回退到 `official-textunit-tasks.json`;返回队列 `status`、worker `task_status`、failure reason 和时间/尝试次数。
|
- `translation.tasks` RPC/CLI 优先查询 `translation-tasks.sqlite`,旧 release 没有状态库时回退到 `official-textunit-tasks.json`;返回队列 `status`、worker `task_status`、failure reason 和时间/尝试次数。
|
||||||
- `translation.task.update` 已提供 queued/running/failed/completed/skipped 状态回写契约,并已通过 `bat i18n task update`、Go typed helper 和 bat-api 鉴权控制面暴露;`translation.proofread` 已提供人工校对状态标记契约,provider worker 可在消费 handoff 后按 task_id 更新并由同一查询接口反查。
|
- `translation.tasks` / `translation.handoff` 已提供 Go typed helper 和 bat-api 鉴权查询端点;`translation.task.update` 已提供 queued/running/failed/completed/skipped 状态回写契约,并已通过 `bat i18n task update`、Go typed helper 和 bat-api 鉴权控制面暴露;`translation.worker.run` 已提供 Rust provider worker,支持 mock/Crowdin、独立 claim、lease、失败分类、重试和 TextUnit 级译文结果落库;`translation.proofread` 已提供人工校对状态标记契约。
|
||||||
- `i18n get/set/unset` 可查看、修改或清空单条工作台译文,`i18n validate` 可在发布前校验工作台 release、source text、重复 patch 目标,并区分可直接发布的 TextAsset 与必须进入 `parse repack` 的条目;`i18n proofread` 可把当前汉化 workflow 标记为人工校对中,且不会遮蔽已发布汉化 release;`parse clear-cache` 只删除可再生解析/队列 JSON,保留 `translation-tasks.sqlite` 的 worker 状态。
|
- `i18n get/set/unset` 可查看、修改或清空单条工作台译文,`i18n validate` 可在发布前校验工作台 release、source text、重复 patch 目标,并区分可直接发布的 TextAsset 与必须进入 `parse repack` 的条目;`i18n proofread` 可把当前汉化 workflow 标记为人工校对中,且不会遮蔽已发布汉化 release;`parse clear-cache` 只删除可再生解析/队列 JSON,保留 `translation-tasks.sqlite` 的 worker 状态。
|
||||||
- 真实 Crowdin 网络 worker、翻译记忆和完整 localized repack 仍属于后续翻译系统工作,不在当前 Rust 离线状态仓储范围内。
|
- 翻译记忆和完整 localized repack 仍属于后续翻译系统工作,不在当前 provider worker 状态闭环范围内。
|
||||||
|
|
||||||
验收:
|
验收:
|
||||||
|
|
||||||
|
|||||||
@@ -95,7 +95,7 @@
|
|||||||
| 组件 | 路径 | 状态 | 说明 |
|
| 组件 | 路径 | 状态 | 说明 |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| Module | `go.mod` → `bat-api` | 已用 | 服务层模块名 |
|
| Module | `go.mod` → `bat-api` | 已用 | 服务层模块名 |
|
||||||
| RPC client | `internal/backendrpc` | **完成** | Unix socket JSON-RPC transport + typed helper;typed helper 覆盖 daemon 已实现控制/查询、`resource.state/sync/verify/repair/manifest/list`、`catalog.*`、`parse.*`、`localized.status`、`task.*`、`translation.task.update`、`translation.proofread` 和文件级 UnityFS patch 调用;`resource.index`、其余 `translation.*`、`patch.apply` 仍通过通用 `Call` 走同一 contract;fake transport 单测,配合 `internal/api/testdata/contract/` 固化 Rust 输出 mirror |
|
| RPC client | `internal/backendrpc` | **完成** | Unix socket JSON-RPC transport + typed helper;typed helper 覆盖 daemon 已实现控制/查询、`resource.state/sync/verify/repair/manifest/list`、`catalog.*`、`parse.*`、`localized.status`、`task.*`、`translation.tasks`、`translation.handoff`、`translation.task.update`、`translation.worker.run`、`translation.proofread` 和文件级 UnityFS patch 调用;`resource.index`、`patch.apply` 仍通过通用 `Call` 走同一 contract;fake transport 单测,配合 `internal/api/testdata/contract/` 固化 Rust 输出 mirror |
|
||||||
| 资源 bootstrap/分发 | `cmd/bat-api` + `internal/api` | **MVP+生产控制面** | RPC 发现 + 周期刷新/诊断 + `/v1/bootstrap` + `/v1/launcher/bootstrap` + launcher 资源 metadata 兼容 + `/readyz` + CDN Range/缓存头 + 鉴权/限流/访问日志/反代适配 + OpenAPI + 管理控制白名单 + `.env` |
|
| 资源 bootstrap/分发 | `cmd/bat-api` + `internal/api` | **MVP+生产控制面** | RPC 发现 + 周期刷新/诊断 + `/v1/bootstrap` + `/v1/launcher/bootstrap` + launcher 资源 metadata 兼容 + `/readyz` + CDN Range/缓存头 + 鉴权/限流/访问日志/反代适配 + OpenAPI + 管理控制白名单 + `.env` |
|
||||||
| 试验 CLI | `cmd/bat` | **试验** | doctor 固定 ok;manifest/sync 走 FFI |
|
| 试验 CLI | `cmd/bat` | **试验** | doctor 固定 ok;manifest/sync 走 FFI |
|
||||||
| FFI | `internal/ffi` | **可选** | 需 `build-ffi` |
|
| FFI | `internal/ffi` | **可选** | 需 `build-ffi` |
|
||||||
|
|||||||
@@ -23,14 +23,16 @@ use bat_infrastructure::{
|
|||||||
OfficialUpdateService, OfficialUpdateSnapshot, OfficialUpdateStatus,
|
OfficialUpdateService, OfficialUpdateSnapshot, OfficialUpdateStatus,
|
||||||
OfficialVerificationSummary, OfficialVersionRecord, OfficialVersionState, PatchApplyKind,
|
OfficialVerificationSummary, OfficialVersionRecord, OfficialVersionState, PatchApplyKind,
|
||||||
PatchApplyParams, PatchApplyReport, ReleaseFlowStatusCode, RepackReport,
|
PatchApplyParams, PatchApplyReport, ReleaseFlowStatusCode, RepackReport,
|
||||||
SqliteResourceRepository, SqliteTranslationTaskRepository, TranslationTaskStatus,
|
SqliteResourceRepository, SqliteTranslationTaskRepository, TranslationProviderKind,
|
||||||
UnityFsFieldPatchParams, UnityFsPatchReport, UnityFsStringFieldPatchParams,
|
TranslationTaskStatus, TranslationWorkerConfig, UnityFsFieldPatchParams, UnityFsPatchReport,
|
||||||
UnityFsTextAssetPatchParams, CROWDIN_TEXTUNIT_QUEUE_FILE, LOCALIZED_CURRENT_LINK,
|
UnityFsStringFieldPatchParams, UnityFsTextAssetPatchParams, CROWDIN_TEXTUNIT_QUEUE_FILE,
|
||||||
|
DEFAULT_TRANSLATION_CONCURRENCY, DEFAULT_TRANSLATION_LEASE_SECONDS,
|
||||||
|
DEFAULT_TRANSLATION_MAX_ATTEMPTS, DEFAULT_TRANSLATION_RETRY_BACKOFF, LOCALIZED_CURRENT_LINK,
|
||||||
LOCALIZED_PATCH_MANIFEST_FILE, LOCALIZED_TRANSLATION_STATUS_MANUAL_PROOFREADING,
|
LOCALIZED_PATCH_MANIFEST_FILE, LOCALIZED_TRANSLATION_STATUS_MANUAL_PROOFREADING,
|
||||||
LOCALIZED_TRANSLATION_STATUS_MANUAL_PROOFREADING_LABEL, LOCALIZED_VERSIONS_DIR,
|
LOCALIZED_TRANSLATION_STATUS_MANUAL_PROOFREADING_LABEL, LOCALIZED_VERSIONS_DIR,
|
||||||
LOCALIZED_VERSION_STATE_FILE, MAX_DOWNLOAD_CONCURRENCY, MIN_DOWNLOAD_CONCURRENCY,
|
LOCALIZED_VERSION_STATE_FILE, MAX_DOWNLOAD_CONCURRENCY, MAX_TRANSLATION_CONCURRENCY,
|
||||||
OFFICIAL_PARSE_CACHE_FILE, OFFICIAL_TEXTUNIT_INDEX_FILE, OFFICIAL_TEXTUNIT_TASK_QUEUE_FILE,
|
MIN_DOWNLOAD_CONCURRENCY, MIN_TRANSLATION_CONCURRENCY, OFFICIAL_PARSE_CACHE_FILE,
|
||||||
PRIVATE_FILE_MODE,
|
OFFICIAL_TEXTUNIT_INDEX_FILE, OFFICIAL_TEXTUNIT_TASK_QUEUE_FILE, PRIVATE_FILE_MODE,
|
||||||
};
|
};
|
||||||
use serde::de::DeserializeOwned;
|
use serde::de::DeserializeOwned;
|
||||||
use serde::{Deserialize, Serialize};
|
use serde::{Deserialize, Serialize};
|
||||||
@@ -85,6 +87,7 @@ use workflow_commands::{
|
|||||||
run_parse_clear_cache, run_parse_once, run_publish_localized, run_repack, run_translate_once,
|
run_parse_clear_cache, run_parse_once, run_publish_localized, run_repack, run_translate_once,
|
||||||
run_translation_get, run_translation_proofread, run_translation_set,
|
run_translation_get, run_translation_proofread, run_translation_set,
|
||||||
run_translation_task_update, run_translation_unset, run_translation_validate,
|
run_translation_task_update, run_translation_unset, run_translation_validate,
|
||||||
|
run_translation_worker,
|
||||||
};
|
};
|
||||||
|
|
||||||
const EXIT_ERROR: i32 = 1;
|
const EXIT_ERROR: i32 = 1;
|
||||||
@@ -229,6 +232,10 @@ fn run() -> anyhow::Result<i32> {
|
|||||||
run_translation_proofread(&options)?;
|
run_translation_proofread(&options)?;
|
||||||
Ok(0)
|
Ok(0)
|
||||||
}
|
}
|
||||||
|
CliCommand::TranslationWorker => {
|
||||||
|
run_repeated_workflow(&options, "translation-worker", run_translation_worker)?;
|
||||||
|
Ok(0)
|
||||||
|
}
|
||||||
CliCommand::Repack => {
|
CliCommand::Repack => {
|
||||||
run_repack(&options)?;
|
run_repack(&options)?;
|
||||||
Ok(0)
|
Ok(0)
|
||||||
@@ -378,6 +385,15 @@ struct CliOptions {
|
|||||||
translation_text_file: Option<PathBuf>,
|
translation_text_file: Option<PathBuf>,
|
||||||
translation_failure_reason: Option<String>,
|
translation_failure_reason: Option<String>,
|
||||||
translation_provider_run_id: Option<String>,
|
translation_provider_run_id: Option<String>,
|
||||||
|
translation_provider: Option<String>,
|
||||||
|
translation_fixture: Option<PathBuf>,
|
||||||
|
worker_concurrency: usize,
|
||||||
|
worker_max_attempts: u32,
|
||||||
|
worker_lease_seconds: u64,
|
||||||
|
worker_retry_backoff: Duration,
|
||||||
|
worker_max_tasks: Option<usize>,
|
||||||
|
worker_id: Option<String>,
|
||||||
|
translation_worker_option_explicit: bool,
|
||||||
localized_release_id: Option<String>,
|
localized_release_id: Option<String>,
|
||||||
repack_spec: Option<PathBuf>,
|
repack_spec: Option<PathBuf>,
|
||||||
schedule_group: Option<String>,
|
schedule_group: Option<String>,
|
||||||
@@ -463,6 +479,15 @@ impl Default for CliOptions {
|
|||||||
translation_text_file: None,
|
translation_text_file: None,
|
||||||
translation_failure_reason: None,
|
translation_failure_reason: None,
|
||||||
translation_provider_run_id: None,
|
translation_provider_run_id: None,
|
||||||
|
translation_provider: None,
|
||||||
|
translation_fixture: None,
|
||||||
|
worker_concurrency: DEFAULT_TRANSLATION_CONCURRENCY,
|
||||||
|
worker_max_attempts: DEFAULT_TRANSLATION_MAX_ATTEMPTS,
|
||||||
|
worker_lease_seconds: DEFAULT_TRANSLATION_LEASE_SECONDS,
|
||||||
|
worker_retry_backoff: DEFAULT_TRANSLATION_RETRY_BACKOFF,
|
||||||
|
worker_max_tasks: None,
|
||||||
|
worker_id: None,
|
||||||
|
translation_worker_option_explicit: false,
|
||||||
localized_release_id: None,
|
localized_release_id: None,
|
||||||
repack_spec: None,
|
repack_spec: None,
|
||||||
schedule_group: None,
|
schedule_group: None,
|
||||||
@@ -546,6 +571,7 @@ enum CliCommand {
|
|||||||
TranslationGet,
|
TranslationGet,
|
||||||
TranslationUnset,
|
TranslationUnset,
|
||||||
TranslationTaskUpdate,
|
TranslationTaskUpdate,
|
||||||
|
TranslationWorker,
|
||||||
TranslationProofread,
|
TranslationProofread,
|
||||||
Repack,
|
Repack,
|
||||||
PublishLocalized,
|
PublishLocalized,
|
||||||
@@ -1025,6 +1051,7 @@ const RPC_METHOD_TRANSLATION_TASKS: &str = "translation.tasks";
|
|||||||
const RPC_METHOD_TRANSLATION_HANDOFF: &str = "translation.handoff";
|
const RPC_METHOD_TRANSLATION_HANDOFF: &str = "translation.handoff";
|
||||||
const RPC_METHOD_TRANSLATION_TASK_UPDATE: &str = "translation.task.update";
|
const RPC_METHOD_TRANSLATION_TASK_UPDATE: &str = "translation.task.update";
|
||||||
const RPC_METHOD_TRANSLATION_PROOFREAD: &str = "translation.proofread";
|
const RPC_METHOD_TRANSLATION_PROOFREAD: &str = "translation.proofread";
|
||||||
|
const RPC_METHOD_TRANSLATION_WORKER_RUN: &str = "translation.worker.run";
|
||||||
const RPC_METHOD_LOCALIZED_STATUS: &str = "localized.status";
|
const RPC_METHOD_LOCALIZED_STATUS: &str = "localized.status";
|
||||||
const RPC_METHOD_CATALOG_STATUS: &str = "catalog.status";
|
const RPC_METHOD_CATALOG_STATUS: &str = "catalog.status";
|
||||||
const RPC_METHOD_CATALOG_VERSIONS: &str = "catalog.versions";
|
const RPC_METHOD_CATALOG_VERSIONS: &str = "catalog.versions";
|
||||||
@@ -2027,6 +2054,13 @@ fn dispatch_rpc_method(
|
|||||||
mark_localized_manual_proofreading_report(state_dir, &tasks.base_config)
|
mark_localized_manual_proofreading_report(state_dir, &tasks.base_config)
|
||||||
.and_then(|report| serde_json::to_value(report).map_err(anyhow::Error::from)),
|
.and_then(|report| serde_json::to_value(report).map_err(anyhow::Error::from)),
|
||||||
),
|
),
|
||||||
|
RPC_METHOD_TRANSLATION_WORKER_RUN => {
|
||||||
|
let config = match rpc_translation_worker_config(request.params.as_ref()) {
|
||||||
|
Ok(config) => config,
|
||||||
|
Err(error) => return rpc_envelope_error(request_id, error),
|
||||||
|
};
|
||||||
|
enqueue_translation_worker_envelope(tasks, config, request_id)
|
||||||
|
}
|
||||||
RPC_METHOD_LOCALIZED_STATUS => rpc_envelope_from_result(
|
RPC_METHOD_LOCALIZED_STATUS => rpc_envelope_from_result(
|
||||||
request_id,
|
request_id,
|
||||||
"localized.status",
|
"localized.status",
|
||||||
@@ -2211,7 +2245,9 @@ fn enqueue_task_envelope(
|
|||||||
.unwrap_or_else(|| Arc::new(AtomicBool::new(false)));
|
.unwrap_or_else(|| Arc::new(AtomicBool::new(false)));
|
||||||
let job = TaskJob {
|
let job = TaskJob {
|
||||||
id: task_id.clone(),
|
id: task_id.clone(),
|
||||||
|
kind,
|
||||||
config,
|
config,
|
||||||
|
translation_worker_config: None,
|
||||||
cancel,
|
cancel,
|
||||||
};
|
};
|
||||||
if tasks.queue.send(job).is_err() {
|
if tasks.queue.send(job).is_err() {
|
||||||
@@ -2237,6 +2273,50 @@ fn enqueue_task_envelope(
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn enqueue_translation_worker_envelope(
|
||||||
|
tasks: &DaemonTaskContext,
|
||||||
|
worker_config: TranslationWorkerConfig,
|
||||||
|
request_id: String,
|
||||||
|
) -> RpcEnvelope {
|
||||||
|
let config = TaskKind::TranslationWorker.build_config(&tasks.base_config, false);
|
||||||
|
let task_id = tasks.registry.create(TaskKind::TranslationWorker);
|
||||||
|
let cancel = tasks
|
||||||
|
.registry
|
||||||
|
.cancel_flag(&task_id)
|
||||||
|
.unwrap_or_else(|| Arc::new(AtomicBool::new(false)));
|
||||||
|
let job = TaskJob {
|
||||||
|
id: task_id.clone(),
|
||||||
|
kind: TaskKind::TranslationWorker,
|
||||||
|
config,
|
||||||
|
translation_worker_config: Some(worker_config.clone()),
|
||||||
|
cancel,
|
||||||
|
};
|
||||||
|
if tasks.queue.send(job).is_err() {
|
||||||
|
tasks.registry.update(&task_id, |record| {
|
||||||
|
record.status = "failed";
|
||||||
|
record.finished_at = Some(unix_seconds_now());
|
||||||
|
record.error = Some(ApiError::new(
|
||||||
|
ErrorCode::INTERNAL,
|
||||||
|
"task.enqueue",
|
||||||
|
"任务执行器不可用",
|
||||||
|
));
|
||||||
|
});
|
||||||
|
return rpc_envelope_error(
|
||||||
|
request_id,
|
||||||
|
ApiError::new(ErrorCode::INTERNAL, "task.enqueue", "任务执行器不可用"),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
rpc_envelope_ok(
|
||||||
|
request_id,
|
||||||
|
"accepted",
|
||||||
|
serde_json::json!({
|
||||||
|
"task_id": task_id,
|
||||||
|
"kind": RPC_METHOD_TRANSLATION_WORKER_RUN,
|
||||||
|
"worker": worker_config,
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
fn rpc_ack_value(
|
fn rpc_ack_value(
|
||||||
command: &'static str,
|
command: &'static str,
|
||||||
message: &'static str,
|
message: &'static str,
|
||||||
@@ -3070,6 +3150,207 @@ fn rpc_struct_params<T: DeserializeOwned>(
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn rpc_translation_worker_config(
|
||||||
|
params: Option<&serde_json::Value>,
|
||||||
|
) -> Result<TranslationWorkerConfig, ApiError> {
|
||||||
|
let empty = serde_json::json!({});
|
||||||
|
let params = params.unwrap_or(&empty);
|
||||||
|
if !params.is_object() {
|
||||||
|
return Err(ApiError::new(
|
||||||
|
ErrorCode::RPC_INVALID_PARAMS,
|
||||||
|
RPC_METHOD_TRANSLATION_WORKER_RUN,
|
||||||
|
"params 必须是 JSON object",
|
||||||
|
));
|
||||||
|
}
|
||||||
|
let provider = rpc_translation_worker_string_param(
|
||||||
|
params,
|
||||||
|
&["provider", "translation_provider"],
|
||||||
|
"provider",
|
||||||
|
)?
|
||||||
|
.unwrap_or_else(|| TranslationProviderKind::Mock.as_str().to_string());
|
||||||
|
let provider = TranslationProviderKind::parse(&provider).ok_or_else(|| {
|
||||||
|
ApiError::new(
|
||||||
|
ErrorCode::RPC_INVALID_PARAMS,
|
||||||
|
RPC_METHOD_TRANSLATION_WORKER_RUN,
|
||||||
|
format!("不支持的 translation provider:{provider}"),
|
||||||
|
)
|
||||||
|
})?;
|
||||||
|
let concurrency = rpc_translation_worker_usize_param(
|
||||||
|
params,
|
||||||
|
&[
|
||||||
|
"concurrency",
|
||||||
|
"worker_concurrency",
|
||||||
|
"translation_concurrency",
|
||||||
|
],
|
||||||
|
"concurrency",
|
||||||
|
)?
|
||||||
|
.unwrap_or(DEFAULT_TRANSLATION_CONCURRENCY);
|
||||||
|
let max_attempts = rpc_translation_worker_u32_param(
|
||||||
|
params,
|
||||||
|
&[
|
||||||
|
"max_attempts",
|
||||||
|
"worker_max_attempts",
|
||||||
|
"translation_max_attempts",
|
||||||
|
],
|
||||||
|
"max_attempts",
|
||||||
|
)?
|
||||||
|
.unwrap_or(DEFAULT_TRANSLATION_MAX_ATTEMPTS);
|
||||||
|
let lease_seconds = rpc_translation_worker_u64_param(
|
||||||
|
params,
|
||||||
|
&[
|
||||||
|
"lease_seconds",
|
||||||
|
"worker_lease_seconds",
|
||||||
|
"translation_lease_seconds",
|
||||||
|
],
|
||||||
|
"lease_seconds",
|
||||||
|
)?
|
||||||
|
.unwrap_or(DEFAULT_TRANSLATION_LEASE_SECONDS);
|
||||||
|
let retry_backoff_seconds = rpc_translation_worker_u64_param(
|
||||||
|
params,
|
||||||
|
&[
|
||||||
|
"retry_backoff_seconds",
|
||||||
|
"worker_retry_backoff_seconds",
|
||||||
|
"translation_retry_backoff_seconds",
|
||||||
|
],
|
||||||
|
"retry_backoff_seconds",
|
||||||
|
)?
|
||||||
|
.unwrap_or(DEFAULT_TRANSLATION_RETRY_BACKOFF.as_secs());
|
||||||
|
let max_tasks = rpc_translation_worker_usize_param(
|
||||||
|
params,
|
||||||
|
&["max_tasks", "worker_max_tasks", "translation_max_tasks"],
|
||||||
|
"max_tasks",
|
||||||
|
)?;
|
||||||
|
let config = TranslationWorkerConfig {
|
||||||
|
provider,
|
||||||
|
fixture_path: rpc_translation_worker_string_param(
|
||||||
|
params,
|
||||||
|
&[
|
||||||
|
"fixture_path",
|
||||||
|
"translation_fixture",
|
||||||
|
"provider_fixture",
|
||||||
|
"mock_fixture",
|
||||||
|
"fixture",
|
||||||
|
],
|
||||||
|
"fixture_path",
|
||||||
|
)?
|
||||||
|
.map(PathBuf::from),
|
||||||
|
concurrency,
|
||||||
|
max_attempts,
|
||||||
|
lease_seconds,
|
||||||
|
retry_backoff: Duration::from_secs(retry_backoff_seconds),
|
||||||
|
max_tasks,
|
||||||
|
worker_id: rpc_translation_worker_string_param(
|
||||||
|
params,
|
||||||
|
&["worker_id", "translation_worker_id"],
|
||||||
|
"worker_id",
|
||||||
|
)?
|
||||||
|
.unwrap_or_else(|| "bat-rpc-worker".to_string()),
|
||||||
|
};
|
||||||
|
config.validate().map_err(|error| {
|
||||||
|
ApiError::new(
|
||||||
|
ErrorCode::RPC_INVALID_PARAMS,
|
||||||
|
RPC_METHOD_TRANSLATION_WORKER_RUN,
|
||||||
|
error.to_string(),
|
||||||
|
)
|
||||||
|
})?;
|
||||||
|
Ok(config)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn rpc_translation_worker_string_param(
|
||||||
|
params: &serde_json::Value,
|
||||||
|
aliases: &[&str],
|
||||||
|
label: &str,
|
||||||
|
) -> Result<Option<String>, ApiError> {
|
||||||
|
for key in aliases {
|
||||||
|
let Some(value) = params.get(*key) else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
if value.is_null() {
|
||||||
|
return Ok(None);
|
||||||
|
}
|
||||||
|
let Some(text) = value.as_str() else {
|
||||||
|
return Err(ApiError::new(
|
||||||
|
ErrorCode::RPC_INVALID_PARAMS,
|
||||||
|
RPC_METHOD_TRANSLATION_WORKER_RUN,
|
||||||
|
format!("{label} 必须是字符串"),
|
||||||
|
));
|
||||||
|
};
|
||||||
|
let text = text.trim();
|
||||||
|
if text.is_empty() {
|
||||||
|
return Ok(None);
|
||||||
|
}
|
||||||
|
return Ok(Some(text.to_string()));
|
||||||
|
}
|
||||||
|
Ok(None)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn rpc_translation_worker_u64_param(
|
||||||
|
params: &serde_json::Value,
|
||||||
|
aliases: &[&str],
|
||||||
|
label: &str,
|
||||||
|
) -> Result<Option<u64>, ApiError> {
|
||||||
|
for key in aliases {
|
||||||
|
let Some(value) = params.get(*key) else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
if value.is_null() {
|
||||||
|
return Ok(None);
|
||||||
|
}
|
||||||
|
if value.as_i64().is_some_and(|number| number < 0) {
|
||||||
|
return Err(ApiError::new(
|
||||||
|
ErrorCode::RPC_INVALID_PARAMS,
|
||||||
|
RPC_METHOD_TRANSLATION_WORKER_RUN,
|
||||||
|
format!("{label} 不能为负数"),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
let Some(number) = value.as_u64() else {
|
||||||
|
return Err(ApiError::new(
|
||||||
|
ErrorCode::RPC_INVALID_PARAMS,
|
||||||
|
RPC_METHOD_TRANSLATION_WORKER_RUN,
|
||||||
|
format!("{label} 必须是非负整数 JSON number"),
|
||||||
|
));
|
||||||
|
};
|
||||||
|
return Ok(Some(number));
|
||||||
|
}
|
||||||
|
Ok(None)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn rpc_translation_worker_usize_param(
|
||||||
|
params: &serde_json::Value,
|
||||||
|
aliases: &[&str],
|
||||||
|
label: &str,
|
||||||
|
) -> Result<Option<usize>, ApiError> {
|
||||||
|
rpc_translation_worker_u64_param(params, aliases, label)?
|
||||||
|
.map(|number| {
|
||||||
|
usize::try_from(number).map_err(|error| {
|
||||||
|
ApiError::new(
|
||||||
|
ErrorCode::RPC_INVALID_PARAMS,
|
||||||
|
RPC_METHOD_TRANSLATION_WORKER_RUN,
|
||||||
|
format!("{label} 无效:{error}"),
|
||||||
|
)
|
||||||
|
})
|
||||||
|
})
|
||||||
|
.transpose()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn rpc_translation_worker_u32_param(
|
||||||
|
params: &serde_json::Value,
|
||||||
|
aliases: &[&str],
|
||||||
|
label: &str,
|
||||||
|
) -> Result<Option<u32>, ApiError> {
|
||||||
|
rpc_translation_worker_u64_param(params, aliases, label)?
|
||||||
|
.map(|number| {
|
||||||
|
u32::try_from(number).map_err(|error| {
|
||||||
|
ApiError::new(
|
||||||
|
ErrorCode::RPC_INVALID_PARAMS,
|
||||||
|
RPC_METHOD_TRANSLATION_WORKER_RUN,
|
||||||
|
format!("{label} 无效:{error}"),
|
||||||
|
)
|
||||||
|
})
|
||||||
|
})
|
||||||
|
.transpose()
|
||||||
|
}
|
||||||
|
|
||||||
/// 分页参数:`offset` 默认 0;`limit` 默认 100,范围 1..=1000。
|
/// 分页参数:`offset` 默认 0;`limit` 默认 100,范围 1..=1000。
|
||||||
fn rpc_page_params(params: Option<&serde_json::Value>) -> anyhow::Result<(usize, usize)> {
|
fn rpc_page_params(params: Option<&serde_json::Value>) -> anyhow::Result<(usize, usize)> {
|
||||||
let offset = params
|
let offset = params
|
||||||
@@ -4221,6 +4502,32 @@ impl HumanReport for bat_infrastructure::LocalizedTranslationWorkflowReport {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
impl HumanReport for bat_infrastructure::TranslationWorkerReport {
|
||||||
|
fn print_human(&self) -> anyhow::Result<()> {
|
||||||
|
print_title("翻译 provider worker");
|
||||||
|
print_field("命令", self.command);
|
||||||
|
print_field("状态", self.status);
|
||||||
|
print_field("官方 release", &self.official_release_id);
|
||||||
|
print_field("provider", &self.provider);
|
||||||
|
print_field("回收 lease", self.recovered_lease_count);
|
||||||
|
print_field("领取任务", self.claimed_count);
|
||||||
|
print_field("完成任务", self.completed_count);
|
||||||
|
print_field("失败任务", self.failed_count);
|
||||||
|
print_field("已安排重试", self.retry_scheduled_count);
|
||||||
|
print_field("剩余任务", self.remaining_count);
|
||||||
|
for failure in &self.failures {
|
||||||
|
println!(
|
||||||
|
" - {} [{}] retryable={} {}",
|
||||||
|
failure.task_id,
|
||||||
|
failure.failure_class,
|
||||||
|
format_bool(failure.retryable),
|
||||||
|
failure.failure_reason
|
||||||
|
);
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
fn print_human_json_value(value: &serde_json::Value) -> anyhow::Result<()> {
|
fn print_human_json_value(value: &serde_json::Value) -> anyhow::Result<()> {
|
||||||
if value.get("running").is_some() && value.get("state_dir").is_some() {
|
if value.get("running").is_some() && value.get("state_dir").is_some() {
|
||||||
print_title("后台状态");
|
print_title("后台状态");
|
||||||
@@ -6321,6 +6628,17 @@ BAT_AUTO_DISCOVER=1
|
|||||||
#BAT_DOWNLOAD_CONCURRENCY=8
|
#BAT_DOWNLOAD_CONCURRENCY=8
|
||||||
#BAT_UNZIP=unzip
|
#BAT_UNZIP=unzip
|
||||||
|
|
||||||
|
# ---- 翻译 provider worker ----
|
||||||
|
# provider:mock(本地 fixture)或 crowdin(读取 CROWDIN_* 环境变量)
|
||||||
|
#BAT_TRANSLATION_PROVIDER=mock
|
||||||
|
#BAT_TRANSLATION_FIXTURE=
|
||||||
|
#BAT_TRANSLATION_CONCURRENCY=8
|
||||||
|
#BAT_TRANSLATION_MAX_ATTEMPTS=3
|
||||||
|
#BAT_TRANSLATION_LEASE_SECONDS=300
|
||||||
|
#BAT_TRANSLATION_RETRY_BACKOFF_SECONDS=5
|
||||||
|
#BAT_TRANSLATION_MAX_TASKS=
|
||||||
|
#BAT_TRANSLATION_WORKER_ID=
|
||||||
|
|
||||||
# ---- 输出 ----
|
# ---- 输出 ----
|
||||||
# 设为 1 时输出机器可读 JSON(默认人类可读)
|
# 设为 1 时输出机器可读 JSON(默认人类可读)
|
||||||
#BAT_JSON=0
|
#BAT_JSON=0
|
||||||
@@ -6504,6 +6822,38 @@ fn apply_bat_env_overrides(
|
|||||||
if let Some(v) = value("BAT_UNZIP") {
|
if let Some(v) = value("BAT_UNZIP") {
|
||||||
options.config.unzip_command = PathBuf::from(v);
|
options.config.unzip_command = PathBuf::from(v);
|
||||||
}
|
}
|
||||||
|
if let Some(v) = value("BAT_TRANSLATION_PROVIDER") {
|
||||||
|
options.translation_provider = Some(v);
|
||||||
|
}
|
||||||
|
if let Some(v) = value("BAT_TRANSLATION_FIXTURE") {
|
||||||
|
options.translation_fixture = Some(PathBuf::from(v));
|
||||||
|
}
|
||||||
|
if let Some(v) = value("BAT_TRANSLATION_CONCURRENCY") {
|
||||||
|
options.worker_concurrency =
|
||||||
|
parse_translation_worker_concurrency(&v, "环境变量 BAT_TRANSLATION_CONCURRENCY")?;
|
||||||
|
}
|
||||||
|
if let Some(v) = value("BAT_TRANSLATION_MAX_ATTEMPTS") {
|
||||||
|
options.worker_max_attempts =
|
||||||
|
parse_positive_u32(&v, "环境变量 BAT_TRANSLATION_MAX_ATTEMPTS")?;
|
||||||
|
}
|
||||||
|
if let Some(v) = value("BAT_TRANSLATION_LEASE_SECONDS") {
|
||||||
|
options.worker_lease_seconds =
|
||||||
|
parse_positive_u64(&v, "环境变量 BAT_TRANSLATION_LEASE_SECONDS")?;
|
||||||
|
}
|
||||||
|
if let Some(v) = value("BAT_TRANSLATION_RETRY_BACKOFF_SECONDS") {
|
||||||
|
options.worker_retry_backoff = Duration::from_secs(v.parse::<u64>().map_err(|error| {
|
||||||
|
anyhow::anyhow!("环境变量 BAT_TRANSLATION_RETRY_BACKOFF_SECONDS 的秒数无效:{error}")
|
||||||
|
})?);
|
||||||
|
}
|
||||||
|
if let Some(v) = value("BAT_TRANSLATION_MAX_TASKS") {
|
||||||
|
options.worker_max_tasks = Some(parse_positive_usize(
|
||||||
|
&v,
|
||||||
|
"环境变量 BAT_TRANSLATION_MAX_TASKS",
|
||||||
|
)?);
|
||||||
|
}
|
||||||
|
if let Some(v) = value("BAT_TRANSLATION_WORKER_ID") {
|
||||||
|
options.worker_id = Some(v);
|
||||||
|
}
|
||||||
if let Some(v) = value("BAT_PROXY") {
|
if let Some(v) = value("BAT_PROXY") {
|
||||||
options.config.curl_proxy = parse_proxy_config(&v)?;
|
options.config.curl_proxy = parse_proxy_config(&v)?;
|
||||||
}
|
}
|
||||||
@@ -6725,6 +7075,55 @@ fn parse_args_with_env(
|
|||||||
"--provider-run-id" => {
|
"--provider-run-id" => {
|
||||||
options.translation_provider_run_id = Some(next_option_value(&mut args, &flag)?);
|
options.translation_provider_run_id = Some(next_option_value(&mut args, &flag)?);
|
||||||
}
|
}
|
||||||
|
"--translation-provider" | "--provider" => {
|
||||||
|
options.translation_provider = Some(next_option_value(&mut args, &flag)?);
|
||||||
|
options.translation_worker_option_explicit = true;
|
||||||
|
}
|
||||||
|
"--translation-fixture" | "--provider-fixture" | "--mock-fixture" | "--fixture" => {
|
||||||
|
options.translation_fixture =
|
||||||
|
Some(PathBuf::from(next_option_value(&mut args, &flag)?));
|
||||||
|
options.translation_worker_option_explicit = true;
|
||||||
|
}
|
||||||
|
"--worker-concurrency" | "--translation-concurrency" => {
|
||||||
|
options.worker_concurrency = parse_translation_worker_concurrency(
|
||||||
|
&next_option_value(&mut args, &flag)?,
|
||||||
|
&flag,
|
||||||
|
)?;
|
||||||
|
options.translation_worker_option_explicit = true;
|
||||||
|
}
|
||||||
|
"--worker-max-attempts" | "--translation-max-attempts" => {
|
||||||
|
options.worker_max_attempts =
|
||||||
|
parse_positive_u32(&next_option_value(&mut args, &flag)?, &flag)?;
|
||||||
|
options.translation_worker_option_explicit = true;
|
||||||
|
}
|
||||||
|
"--worker-lease-seconds" | "--translation-lease-seconds" => {
|
||||||
|
options.worker_lease_seconds =
|
||||||
|
parse_positive_u64(&next_option_value(&mut args, &flag)?, &flag)?;
|
||||||
|
options.translation_worker_option_explicit = true;
|
||||||
|
}
|
||||||
|
"--worker-retry-backoff" | "--translation-retry-backoff" => {
|
||||||
|
options.worker_retry_backoff =
|
||||||
|
parse_duration(&next_option_value(&mut args, &flag)?)?;
|
||||||
|
options.translation_worker_option_explicit = true;
|
||||||
|
}
|
||||||
|
"--worker-retry-backoff-seconds" | "--translation-retry-backoff-seconds" => {
|
||||||
|
let seconds = next_option_value(&mut args, &flag)?
|
||||||
|
.parse::<u64>()
|
||||||
|
.map_err(|error| anyhow::anyhow!("{flag} 的秒数无效:{error}"))?;
|
||||||
|
options.worker_retry_backoff = Duration::from_secs(seconds);
|
||||||
|
options.translation_worker_option_explicit = true;
|
||||||
|
}
|
||||||
|
"--worker-max-tasks" | "--translation-max-tasks" => {
|
||||||
|
options.worker_max_tasks = Some(parse_positive_usize(
|
||||||
|
&next_option_value(&mut args, &flag)?,
|
||||||
|
&flag,
|
||||||
|
)?);
|
||||||
|
options.translation_worker_option_explicit = true;
|
||||||
|
}
|
||||||
|
"--worker-id" | "--translation-worker-id" => {
|
||||||
|
options.worker_id = Some(next_option_value(&mut args, &flag)?);
|
||||||
|
options.translation_worker_option_explicit = true;
|
||||||
|
}
|
||||||
"--localized-release-id" => {
|
"--localized-release-id" => {
|
||||||
options.localized_release_id = Some(next_option_value(&mut args, &flag)?);
|
options.localized_release_id = Some(next_option_value(&mut args, &flag)?);
|
||||||
}
|
}
|
||||||
@@ -7185,6 +7584,12 @@ fn parse_args_with_env(
|
|||||||
));
|
));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if options.command != CliCommand::TranslationWorker
|
||||||
|
&& options.translation_worker_option_explicit
|
||||||
|
{
|
||||||
|
return Err(anyhow::anyhow!("翻译 worker 参数只适用于 i18n worker run"));
|
||||||
|
}
|
||||||
|
|
||||||
match options.command {
|
match options.command {
|
||||||
CliCommand::Status | CliCommand::Stop | CliCommand::Logs => {
|
CliCommand::Status | CliCommand::Stop | CliCommand::Logs => {
|
||||||
if options.sync_option_explicit
|
if options.sync_option_explicit
|
||||||
@@ -7406,6 +7811,56 @@ fn parse_args_with_env(
|
|||||||
options.progress = false;
|
options.progress = false;
|
||||||
options.banner = false;
|
options.banner = false;
|
||||||
}
|
}
|
||||||
|
CliCommand::TranslationWorker => {
|
||||||
|
if options.daemon || options.daemon_child {
|
||||||
|
return Err(anyhow::anyhow!(
|
||||||
|
"i18n worker run 使用 --watch 或 --run-count,不支持 daemon"
|
||||||
|
));
|
||||||
|
}
|
||||||
|
if options.config.force
|
||||||
|
|| options.config.dry_run
|
||||||
|
|| options.schedule_option_explicit
|
||||||
|
|| options.query_option_explicit
|
||||||
|
|| options.write_patch_option_explicit
|
||||||
|
|| options.translation_file.is_some()
|
||||||
|
|| options.translation_id.is_some()
|
||||||
|
|| options.translation_text.is_some()
|
||||||
|
|| options.translation_text_file.is_some()
|
||||||
|
|| options.translation_failure_reason.is_some()
|
||||||
|
|| options.translation_provider_run_id.is_some()
|
||||||
|
|| options.localized_release_id.is_some()
|
||||||
|
|| options.repack_spec.is_some()
|
||||||
|
|| options.proxy_option_explicit
|
||||||
|
|| tools_are_non_default(&options.config, &options.env_baseline_config)
|
||||||
|
{
|
||||||
|
return Err(anyhow::anyhow!(
|
||||||
|
"i18n worker run 只接受 --output、--resource-root、--state-dir、worker 参数和轮询参数"
|
||||||
|
));
|
||||||
|
}
|
||||||
|
let provider = options
|
||||||
|
.translation_provider
|
||||||
|
.as_deref()
|
||||||
|
.unwrap_or(TranslationProviderKind::Mock.as_str());
|
||||||
|
if TranslationProviderKind::parse(provider).is_none() {
|
||||||
|
return Err(anyhow::anyhow!(
|
||||||
|
"i18n worker run 的 provider 无效:{provider}"
|
||||||
|
));
|
||||||
|
}
|
||||||
|
TranslationWorkerConfig {
|
||||||
|
provider: TranslationProviderKind::parse(provider).expect("provider 已在上方校验"),
|
||||||
|
fixture_path: options.translation_fixture.clone(),
|
||||||
|
concurrency: options.worker_concurrency,
|
||||||
|
max_attempts: options.worker_max_attempts,
|
||||||
|
lease_seconds: options.worker_lease_seconds,
|
||||||
|
retry_backoff: options.worker_retry_backoff,
|
||||||
|
max_tasks: options.worker_max_tasks,
|
||||||
|
worker_id: options
|
||||||
|
.worker_id
|
||||||
|
.clone()
|
||||||
|
.unwrap_or_else(|| "bat-worker-validation".to_string()),
|
||||||
|
}
|
||||||
|
.validate()?;
|
||||||
|
}
|
||||||
CliCommand::TranslationProofread => {
|
CliCommand::TranslationProofread => {
|
||||||
if options.watch || options.daemon || options.daemon_child {
|
if options.watch || options.daemon || options.daemon_child {
|
||||||
return Err(anyhow::anyhow!("i18n proofread 只支持单次执行或 RPC 调用"));
|
return Err(anyhow::anyhow!("i18n proofread 只支持单次执行或 RPC 调用"));
|
||||||
@@ -7506,6 +7961,7 @@ fn parse_args_with_env(
|
|||||||
CliCommand::Pull
|
CliCommand::Pull
|
||||||
| CliCommand::Parse
|
| CliCommand::Parse
|
||||||
| CliCommand::Translate
|
| CliCommand::Translate
|
||||||
|
| CliCommand::TranslationWorker
|
||||||
| CliCommand::PublishLocalized
|
| CliCommand::PublishLocalized
|
||||||
)
|
)
|
||||||
{
|
{
|
||||||
@@ -7515,7 +7971,11 @@ fn parse_args_with_env(
|
|||||||
}
|
}
|
||||||
if matches!(
|
if matches!(
|
||||||
options.command,
|
options.command,
|
||||||
CliCommand::Pull | CliCommand::Parse | CliCommand::Translate | CliCommand::PublishLocalized
|
CliCommand::Pull
|
||||||
|
| CliCommand::Parse
|
||||||
|
| CliCommand::Translate
|
||||||
|
| CliCommand::PublishLocalized
|
||||||
|
| CliCommand::TranslationWorker
|
||||||
) {
|
) {
|
||||||
if options.watch && options.run_count.is_some() {
|
if options.watch && options.run_count.is_some() {
|
||||||
return Err(anyhow::anyhow!(
|
return Err(anyhow::anyhow!(
|
||||||
@@ -7632,6 +8092,9 @@ fn parse_translation_command(
|
|||||||
if action == "workbench" || action == "wb" {
|
if action == "workbench" || action == "wb" {
|
||||||
return parse_translation_workbench_command(args, options);
|
return parse_translation_workbench_command(args, options);
|
||||||
}
|
}
|
||||||
|
if action == "worker" {
|
||||||
|
return parse_translation_worker_command(args, options);
|
||||||
|
}
|
||||||
let command = match action.as_str() {
|
let command = match action.as_str() {
|
||||||
"run" => CliCommand::Translate,
|
"run" => CliCommand::Translate,
|
||||||
"export" => CliCommand::Translate,
|
"export" => CliCommand::Translate,
|
||||||
@@ -7655,6 +8118,20 @@ fn parse_translation_command(
|
|||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn parse_translation_worker_command(
|
||||||
|
args: &mut impl Iterator<Item = String>,
|
||||||
|
options: &mut CliOptions,
|
||||||
|
) -> anyhow::Result<()> {
|
||||||
|
let action = next_option_value(args, "translation worker")?;
|
||||||
|
let command = match action.as_str() {
|
||||||
|
"run" => CliCommand::TranslationWorker,
|
||||||
|
other => return Err(anyhow::anyhow!("未知 translation worker 二级命令:{other}")),
|
||||||
|
};
|
||||||
|
ensure_command_not_set(options.command, &format!("translation worker {action}"))?;
|
||||||
|
options.command = command;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
fn parse_translation_workbench_command(
|
fn parse_translation_workbench_command(
|
||||||
args: &mut impl Iterator<Item = String>,
|
args: &mut impl Iterator<Item = String>,
|
||||||
options: &mut CliOptions,
|
options: &mut CliOptions,
|
||||||
@@ -7738,6 +8215,48 @@ fn parse_download_concurrency(value: &str, source: &str) -> anyhow::Result<usize
|
|||||||
Ok(parsed)
|
Ok(parsed)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn parse_translation_worker_concurrency(value: &str, source: &str) -> anyhow::Result<usize> {
|
||||||
|
let parsed = value
|
||||||
|
.parse::<usize>()
|
||||||
|
.map_err(|error| anyhow::anyhow!("{source} 无效:{error}"))?;
|
||||||
|
if !(MIN_TRANSLATION_CONCURRENCY..=MAX_TRANSLATION_CONCURRENCY).contains(&parsed) {
|
||||||
|
return Err(anyhow::anyhow!(
|
||||||
|
"{source} 必须在 {MIN_TRANSLATION_CONCURRENCY}..={MAX_TRANSLATION_CONCURRENCY} 范围内"
|
||||||
|
));
|
||||||
|
}
|
||||||
|
Ok(parsed)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn parse_positive_usize(value: &str, source: &str) -> anyhow::Result<usize> {
|
||||||
|
let parsed = value
|
||||||
|
.parse::<usize>()
|
||||||
|
.map_err(|error| anyhow::anyhow!("{source} 无效:{error}"))?;
|
||||||
|
if parsed == 0 {
|
||||||
|
return Err(anyhow::anyhow!("{source} 必须大于 0"));
|
||||||
|
}
|
||||||
|
Ok(parsed)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn parse_positive_u32(value: &str, source: &str) -> anyhow::Result<u32> {
|
||||||
|
let parsed = value
|
||||||
|
.parse::<u32>()
|
||||||
|
.map_err(|error| anyhow::anyhow!("{source} 无效:{error}"))?;
|
||||||
|
if parsed == 0 {
|
||||||
|
return Err(anyhow::anyhow!("{source} 必须大于 0"));
|
||||||
|
}
|
||||||
|
Ok(parsed)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn parse_positive_u64(value: &str, source: &str) -> anyhow::Result<u64> {
|
||||||
|
let parsed = value
|
||||||
|
.parse::<u64>()
|
||||||
|
.map_err(|error| anyhow::anyhow!("{source} 无效:{error}"))?;
|
||||||
|
if parsed == 0 {
|
||||||
|
return Err(anyhow::anyhow!("{source} 必须大于 0"));
|
||||||
|
}
|
||||||
|
Ok(parsed)
|
||||||
|
}
|
||||||
|
|
||||||
fn next_option_value(
|
fn next_option_value(
|
||||||
args: &mut impl Iterator<Item = String>,
|
args: &mut impl Iterator<Item = String>,
|
||||||
flag: &str,
|
flag: &str,
|
||||||
@@ -7807,6 +8326,7 @@ fn print_usage(binary: &str) {
|
|||||||
eprintln!(" i18n unset Clear one translated workbench entry");
|
eprintln!(" i18n unset Clear one translated workbench entry");
|
||||||
eprintln!(" i18n validate Validate workbench against the current official release");
|
eprintln!(" i18n validate Validate workbench against the current official release");
|
||||||
eprintln!(" i18n proofread Mark localized workflow as manual proofreading");
|
eprintln!(" i18n proofread Mark localized workflow as manual proofreading");
|
||||||
|
eprintln!(" i18n worker run Run translation provider worker once or repeatedly");
|
||||||
eprintln!(
|
eprintln!(
|
||||||
" i18n tasks / i18n task list / i18n task status Query current offline TextUnit translation task status"
|
" i18n tasks / i18n task list / i18n task status Query current offline TextUnit translation task status"
|
||||||
);
|
);
|
||||||
@@ -7854,6 +8374,9 @@ fn print_usage(binary: &str) {
|
|||||||
" {binary} i18n unset --translation-file /tmp/bat-workbench.json --translation-id unit-1"
|
" {binary} i18n unset --translation-file /tmp/bat-workbench.json --translation-id unit-1"
|
||||||
);
|
);
|
||||||
eprintln!(" {binary} i18n proofread --json");
|
eprintln!(" {binary} i18n proofread --json");
|
||||||
|
eprintln!(
|
||||||
|
" {binary} i18n worker run --provider mock --worker-concurrency 8 --run-count 2 --interval 30s"
|
||||||
|
);
|
||||||
eprintln!(" {binary} i18n tasks --json");
|
eprintln!(" {binary} i18n tasks --json");
|
||||||
eprintln!(" {binary} i18n handoff --json");
|
eprintln!(" {binary} i18n handoff --json");
|
||||||
eprintln!(" {binary} i18n status --json");
|
eprintln!(" {binary} i18n status --json");
|
||||||
@@ -7908,6 +8431,17 @@ fn print_usage(binary: &str) {
|
|||||||
eprintln!(" --translated-file <PATH> UTF-8 translation file for i18n set");
|
eprintln!(" --translated-file <PATH> UTF-8 translation file for i18n set");
|
||||||
eprintln!(" --failure-reason <TEXT> Provider failure reason for i18n task update");
|
eprintln!(" --failure-reason <TEXT> Provider failure reason for i18n task update");
|
||||||
eprintln!(" --provider-run-id <ID> Provider run ID for i18n task update");
|
eprintln!(" --provider-run-id <ID> Provider run ID for i18n task update");
|
||||||
|
eprintln!(" --translation-provider <NAME> / --provider <NAME> Provider for i18n worker run (mock/crowdin)");
|
||||||
|
eprintln!(" --translation-fixture <PATH> Mock/provider fixture for i18n worker run");
|
||||||
|
eprintln!(
|
||||||
|
" --worker-concurrency <N> Translation worker concurrency (default: 8, range 1..=256)"
|
||||||
|
);
|
||||||
|
eprintln!(" --worker-max-attempts <N> Maximum claims per translation task");
|
||||||
|
eprintln!(" --worker-lease-seconds <N> Lease seconds for one claimed task");
|
||||||
|
eprintln!(" --worker-retry-backoff <DURATION> Retry backoff after retryable failure");
|
||||||
|
eprintln!(" --worker-retry-backoff-seconds <N> Retry backoff seconds");
|
||||||
|
eprintln!(" --worker-max-tasks <N> Maximum tasks claimed in one worker run");
|
||||||
|
eprintln!(" --worker-id <ID> Worker ID prefix for lease diagnostics");
|
||||||
eprintln!(" --localized-release-id <ID> Explicit localized publication ID");
|
eprintln!(" --localized-release-id <ID> Explicit localized publication ID");
|
||||||
eprintln!(" --repack-spec <PATH> UnityFS batch repack JSON spec");
|
eprintln!(" --repack-spec <PATH> UnityFS batch repack JSON spec");
|
||||||
eprintln!();
|
eprintln!();
|
||||||
|
|||||||
@@ -140,6 +140,165 @@ fn env_invalid_values_error() {
|
|||||||
assert!(parse(&["bat", "--download-concurrency", "257"]).is_err());
|
assert!(parse(&["bat", "--download-concurrency", "257"]).is_err());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn translation_worker_command_options_are_validated() {
|
||||||
|
let options = parse(&[
|
||||||
|
"bat",
|
||||||
|
"i18n",
|
||||||
|
"worker",
|
||||||
|
"run",
|
||||||
|
"--provider",
|
||||||
|
"mock",
|
||||||
|
"--translation-fixture",
|
||||||
|
"/tmp/mock-translation.json",
|
||||||
|
"--worker-concurrency",
|
||||||
|
"8",
|
||||||
|
"--worker-max-attempts",
|
||||||
|
"4",
|
||||||
|
"--worker-lease-seconds",
|
||||||
|
"60",
|
||||||
|
"--worker-retry-backoff",
|
||||||
|
"0s",
|
||||||
|
"--worker-max-tasks",
|
||||||
|
"2",
|
||||||
|
"--worker-id",
|
||||||
|
"manual-run",
|
||||||
|
"--run-count",
|
||||||
|
"2",
|
||||||
|
"--interval",
|
||||||
|
"30s",
|
||||||
|
])
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(options.command, CliCommand::TranslationWorker);
|
||||||
|
assert_eq!(options.translation_provider.as_deref(), Some("mock"));
|
||||||
|
assert_eq!(
|
||||||
|
options.translation_fixture,
|
||||||
|
Some(PathBuf::from("/tmp/mock-translation.json"))
|
||||||
|
);
|
||||||
|
assert_eq!(options.worker_concurrency, 8);
|
||||||
|
assert_eq!(options.worker_max_attempts, 4);
|
||||||
|
assert_eq!(options.worker_lease_seconds, 60);
|
||||||
|
assert_eq!(options.worker_retry_backoff, Duration::ZERO);
|
||||||
|
assert_eq!(options.worker_max_tasks, Some(2));
|
||||||
|
assert_eq!(options.worker_id.as_deref(), Some("manual-run"));
|
||||||
|
assert_eq!(options.run_count, Some(2));
|
||||||
|
|
||||||
|
assert!(parse(&["bat", "i18n", "worker", "run"]).is_ok());
|
||||||
|
assert!(parse(&[
|
||||||
|
"bat",
|
||||||
|
"translation",
|
||||||
|
"worker",
|
||||||
|
"run",
|
||||||
|
"--translation-concurrency",
|
||||||
|
"256"
|
||||||
|
])
|
||||||
|
.is_ok());
|
||||||
|
assert!(parse(&["bat", "i18n", "worker", "run", "--worker-concurrency", "0"]).is_err());
|
||||||
|
assert!(parse(&[
|
||||||
|
"bat",
|
||||||
|
"i18n",
|
||||||
|
"worker",
|
||||||
|
"run",
|
||||||
|
"--worker-concurrency",
|
||||||
|
"257"
|
||||||
|
])
|
||||||
|
.is_err());
|
||||||
|
assert!(parse(&["bat", "i18n", "worker", "run", "--worker-max-attempts", "0"]).is_err());
|
||||||
|
assert!(parse(&[
|
||||||
|
"bat",
|
||||||
|
"i18n",
|
||||||
|
"worker",
|
||||||
|
"run",
|
||||||
|
"--worker-lease-seconds",
|
||||||
|
"0"
|
||||||
|
])
|
||||||
|
.is_err());
|
||||||
|
assert!(parse(&["bat", "i18n", "worker", "run", "--worker-max-tasks", "0"]).is_err());
|
||||||
|
assert!(parse(&["bat", "i18n", "run", "--worker-concurrency", "8"]).is_err());
|
||||||
|
assert!(parse(&["bat", "i18n", "worker", "run", "--provider", "unknown"]).is_err());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn translation_worker_env_defaults_apply() {
|
||||||
|
let options = parse_with_env(
|
||||||
|
&["bat", "i18n", "worker", "run"],
|
||||||
|
&[
|
||||||
|
("BAT_TRANSLATION_PROVIDER", "mock"),
|
||||||
|
("BAT_TRANSLATION_FIXTURE", "/tmp/fixture.json"),
|
||||||
|
("BAT_TRANSLATION_CONCURRENCY", "16"),
|
||||||
|
("BAT_TRANSLATION_MAX_ATTEMPTS", "5"),
|
||||||
|
("BAT_TRANSLATION_LEASE_SECONDS", "120"),
|
||||||
|
("BAT_TRANSLATION_RETRY_BACKOFF_SECONDS", "0"),
|
||||||
|
("BAT_TRANSLATION_MAX_TASKS", "7"),
|
||||||
|
("BAT_TRANSLATION_WORKER_ID", "env-worker"),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(options.command, CliCommand::TranslationWorker);
|
||||||
|
assert_eq!(options.translation_provider.as_deref(), Some("mock"));
|
||||||
|
assert_eq!(
|
||||||
|
options.translation_fixture,
|
||||||
|
Some(PathBuf::from("/tmp/fixture.json"))
|
||||||
|
);
|
||||||
|
assert_eq!(options.worker_concurrency, 16);
|
||||||
|
assert_eq!(options.worker_max_attempts, 5);
|
||||||
|
assert_eq!(options.worker_lease_seconds, 120);
|
||||||
|
assert_eq!(options.worker_retry_backoff, Duration::ZERO);
|
||||||
|
assert_eq!(options.worker_max_tasks, Some(7));
|
||||||
|
assert_eq!(options.worker_id.as_deref(), Some("env-worker"));
|
||||||
|
|
||||||
|
assert!(parse_with_env(
|
||||||
|
&["bat", "i18n", "worker", "run"],
|
||||||
|
&[("BAT_TRANSLATION_CONCURRENCY", "257")]
|
||||||
|
)
|
||||||
|
.is_err());
|
||||||
|
assert!(parse_with_env(
|
||||||
|
&["bat", "i18n", "worker", "run"],
|
||||||
|
&[("BAT_TRANSLATION_MAX_ATTEMPTS", "0")]
|
||||||
|
)
|
||||||
|
.is_err());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn translation_worker_rpc_params_are_strict_and_accept_aliases() {
|
||||||
|
let default_config = rpc_translation_worker_config(None).unwrap();
|
||||||
|
assert_eq!(default_config.provider, TranslationProviderKind::Mock);
|
||||||
|
assert_eq!(default_config.concurrency, DEFAULT_TRANSLATION_CONCURRENCY);
|
||||||
|
|
||||||
|
let config = rpc_translation_worker_config(Some(&serde_json::json!({
|
||||||
|
"translation_provider": "mock",
|
||||||
|
"provider_fixture": "/tmp/mock-provider.json",
|
||||||
|
"worker_concurrency": 8,
|
||||||
|
"worker_max_attempts": 4,
|
||||||
|
"worker_lease_seconds": 60,
|
||||||
|
"worker_retry_backoff_seconds": 0,
|
||||||
|
"worker_max_tasks": 2,
|
||||||
|
"translation_worker_id": "rpc-worker"
|
||||||
|
})))
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(config.provider, TranslationProviderKind::Mock);
|
||||||
|
assert_eq!(
|
||||||
|
config.fixture_path,
|
||||||
|
Some(PathBuf::from("/tmp/mock-provider.json"))
|
||||||
|
);
|
||||||
|
assert_eq!(config.concurrency, 8);
|
||||||
|
assert_eq!(config.max_attempts, 4);
|
||||||
|
assert_eq!(config.lease_seconds, 60);
|
||||||
|
assert_eq!(config.retry_backoff, Duration::ZERO);
|
||||||
|
assert_eq!(config.max_tasks, Some(2));
|
||||||
|
assert_eq!(config.worker_id, "rpc-worker");
|
||||||
|
|
||||||
|
assert!(rpc_translation_worker_config(Some(&serde_json::json!([]))).is_err());
|
||||||
|
assert!(rpc_translation_worker_config(Some(&serde_json::json!({"provider": "bad"}))).is_err());
|
||||||
|
assert!(rpc_translation_worker_config(Some(&serde_json::json!({"concurrency": "8"}))).is_err());
|
||||||
|
assert!(rpc_translation_worker_config(Some(&serde_json::json!({"concurrency": -1}))).is_err());
|
||||||
|
assert!(rpc_translation_worker_config(Some(&serde_json::json!({"concurrency": 0}))).is_err());
|
||||||
|
assert!(rpc_translation_worker_config(Some(&serde_json::json!({"concurrency": 257}))).is_err());
|
||||||
|
assert!(rpc_translation_worker_config(Some(&serde_json::json!({"max_attempts": 0}))).is_err());
|
||||||
|
assert!(rpc_translation_worker_config(Some(&serde_json::json!({"lease_seconds": 0}))).is_err());
|
||||||
|
assert!(rpc_translation_worker_config(Some(&serde_json::json!({"max_tasks": 0}))).is_err());
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn grouped_workflow_commands_use_short_top_level_aliases() {
|
fn grouped_workflow_commands_use_short_top_level_aliases() {
|
||||||
let options = parse(&[
|
let options = parse(&[
|
||||||
|
|||||||
@@ -4,12 +4,14 @@ pub(super) const MAX_RETAINED_TASKS: usize = 64;
|
|||||||
/// 每个任务保留的进度日志行数上限。
|
/// 每个任务保留的进度日志行数上限。
|
||||||
pub(super) const MAX_TASK_LOG_LINES: usize = 200;
|
pub(super) const MAX_TASK_LOG_LINES: usize = 200;
|
||||||
|
|
||||||
/// 任务类型:目前覆盖官方同步、校验与 catalog 更新检查。
|
/// 任务类型:覆盖资源同步、校验、修复、翻译 worker 与 catalog 更新检查。
|
||||||
#[derive(Debug, Clone, Copy)]
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
pub(super) enum TaskKind {
|
pub(super) enum TaskKind {
|
||||||
Sync,
|
Sync,
|
||||||
Verify,
|
Verify,
|
||||||
Repair,
|
Repair,
|
||||||
|
/// 翻译 provider worker 轮次。
|
||||||
|
TranslationWorker,
|
||||||
/// catalog 更新检查:只做发现 + 拉取计划(dry-run),不下载不审计。
|
/// catalog 更新检查:只做发现 + 拉取计划(dry-run),不下载不审计。
|
||||||
Refresh,
|
Refresh,
|
||||||
}
|
}
|
||||||
@@ -20,6 +22,7 @@ impl TaskKind {
|
|||||||
Self::Sync => RPC_METHOD_RESOURCE_SYNC,
|
Self::Sync => RPC_METHOD_RESOURCE_SYNC,
|
||||||
Self::Verify => RPC_METHOD_RESOURCE_VERIFY,
|
Self::Verify => RPC_METHOD_RESOURCE_VERIFY,
|
||||||
Self::Repair => RPC_METHOD_RESOURCE_REPAIR,
|
Self::Repair => RPC_METHOD_RESOURCE_REPAIR,
|
||||||
|
Self::TranslationWorker => RPC_METHOD_TRANSLATION_WORKER_RUN,
|
||||||
Self::Refresh => RPC_METHOD_CATALOG_REFRESH,
|
Self::Refresh => RPC_METHOD_CATALOG_REFRESH,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -56,6 +59,10 @@ impl TaskKind {
|
|||||||
config.repair = false;
|
config.repair = false;
|
||||||
config.force = force;
|
config.force = force;
|
||||||
}
|
}
|
||||||
|
Self::TranslationWorker => {
|
||||||
|
config.dry_run = false;
|
||||||
|
config.force = false;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
config
|
config
|
||||||
}
|
}
|
||||||
@@ -155,6 +162,7 @@ fn task_kind_static(kind: &str) -> Option<&'static str> {
|
|||||||
RPC_METHOD_RESOURCE_SYNC => Some(RPC_METHOD_RESOURCE_SYNC),
|
RPC_METHOD_RESOURCE_SYNC => Some(RPC_METHOD_RESOURCE_SYNC),
|
||||||
RPC_METHOD_RESOURCE_VERIFY => Some(RPC_METHOD_RESOURCE_VERIFY),
|
RPC_METHOD_RESOURCE_VERIFY => Some(RPC_METHOD_RESOURCE_VERIFY),
|
||||||
RPC_METHOD_RESOURCE_REPAIR => Some(RPC_METHOD_RESOURCE_REPAIR),
|
RPC_METHOD_RESOURCE_REPAIR => Some(RPC_METHOD_RESOURCE_REPAIR),
|
||||||
|
RPC_METHOD_TRANSLATION_WORKER_RUN => Some(RPC_METHOD_TRANSLATION_WORKER_RUN),
|
||||||
RPC_METHOD_CATALOG_REFRESH => Some(RPC_METHOD_CATALOG_REFRESH),
|
RPC_METHOD_CATALOG_REFRESH => Some(RPC_METHOD_CATALOG_REFRESH),
|
||||||
_ => None,
|
_ => None,
|
||||||
}
|
}
|
||||||
@@ -494,7 +502,9 @@ impl TaskStore {
|
|||||||
/// 提交给任务 worker 的作业(配置已按任务类型派生完毕)。
|
/// 提交给任务 worker 的作业(配置已按任务类型派生完毕)。
|
||||||
pub(super) struct TaskJob {
|
pub(super) struct TaskJob {
|
||||||
pub(super) id: String,
|
pub(super) id: String,
|
||||||
|
pub(super) kind: TaskKind,
|
||||||
pub(super) config: OfficialUpdateConfig,
|
pub(super) config: OfficialUpdateConfig,
|
||||||
|
pub(super) translation_worker_config: Option<TranslationWorkerConfig>,
|
||||||
/// 与任务记录共享的取消标志。
|
/// 与任务记录共享的取消标志。
|
||||||
pub(super) cancel: Arc<AtomicBool>,
|
pub(super) cancel: Arc<AtomicBool>,
|
||||||
}
|
}
|
||||||
@@ -528,6 +538,32 @@ pub(super) fn run_task_worker(
|
|||||||
});
|
});
|
||||||
|
|
||||||
let cancel = Arc::clone(&job.cancel);
|
let cancel = Arc::clone(&job.cancel);
|
||||||
|
let run_result = if job.kind == TaskKind::TranslationWorker {
|
||||||
|
let worker_config = job
|
||||||
|
.translation_worker_config
|
||||||
|
.as_ref()
|
||||||
|
.ok_or_else(|| anyhow::anyhow!("translation worker 任务缺少运行配置"));
|
||||||
|
worker_config.and_then(|worker_config| {
|
||||||
|
let _sync_guard = sync_lock
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(|poison| poison.into_inner());
|
||||||
|
registry.append_log(&job.id, "[translation-worker] 开始执行".to_string());
|
||||||
|
registry.update(&job.id, |record| {
|
||||||
|
record.stage = Some("translation-worker".to_string());
|
||||||
|
record.message = Some("翻译 provider worker 正在执行".to_string());
|
||||||
|
});
|
||||||
|
let resource_root = active_official_resource_root(&job.config.output_root)?;
|
||||||
|
let runtime = tokio::runtime::Builder::new_current_thread()
|
||||||
|
.enable_all()
|
||||||
|
.build()?;
|
||||||
|
runtime
|
||||||
|
.block_on(bat_infrastructure::run_translation_worker_at(
|
||||||
|
&resource_root,
|
||||||
|
worker_config,
|
||||||
|
))
|
||||||
|
.and_then(|report| serde_json::to_value(report).map_err(anyhow::Error::from))
|
||||||
|
})
|
||||||
|
} else {
|
||||||
let run_result = {
|
let run_result = {
|
||||||
let _sync_guard = sync_lock
|
let _sync_guard = sync_lock
|
||||||
.lock()
|
.lock()
|
||||||
@@ -539,8 +575,10 @@ pub(super) fn run_task_worker(
|
|||||||
service.run_with_progress_and_cancellation(
|
service.run_with_progress_and_cancellation(
|
||||||
&job.config,
|
&job.config,
|
||||||
|event| {
|
|event| {
|
||||||
progress_registry
|
progress_registry.append_log(
|
||||||
.append_log(&progress_id, format!("[{}] {}", event.stage, event.message));
|
&progress_id,
|
||||||
|
format!("[{}] {}", event.stage, event.message),
|
||||||
|
);
|
||||||
progress_registry.update(&progress_id, |record| {
|
progress_registry.update(&progress_id, |record| {
|
||||||
record.stage = Some(event.stage.to_string());
|
record.stage = Some(event.stage.to_string());
|
||||||
record.message = Some(event.message.clone());
|
record.message = Some(event.message.clone());
|
||||||
@@ -552,12 +590,16 @@ pub(super) fn run_task_worker(
|
|||||||
},
|
},
|
||||||
)
|
)
|
||||||
};
|
};
|
||||||
|
run_result
|
||||||
|
.map(|report| serde_json::to_value(&report).map_err(anyhow::Error::from))
|
||||||
|
.and_then(|result| result)
|
||||||
|
};
|
||||||
|
|
||||||
match run_result {
|
match run_result {
|
||||||
Ok(report) => registry.update(&job.id, |record| {
|
Ok(report) => registry.update(&job.id, |record| {
|
||||||
record.status = "succeeded";
|
record.status = "succeeded";
|
||||||
record.finished_at = Some(unix_seconds_now());
|
record.finished_at = Some(unix_seconds_now());
|
||||||
record.result = serde_json::to_value(&report).ok();
|
record.result = Some(report);
|
||||||
}),
|
}),
|
||||||
Err(error) => {
|
Err(error) => {
|
||||||
let cancelled = cancel.load(Ordering::Relaxed);
|
let cancelled = cancel.load(Ordering::Relaxed);
|
||||||
|
|||||||
@@ -253,6 +253,43 @@ pub(super) fn run_translation_proofread(options: &CliOptions) -> anyhow::Result<
|
|||||||
print_report(options.output_format, &report)
|
print_report(options.output_format, &report)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
pub(super) fn run_translation_worker(options: &CliOptions) -> anyhow::Result<()> {
|
||||||
|
let resource_root = options
|
||||||
|
.resource_root
|
||||||
|
.clone()
|
||||||
|
.map(|path| lexical_absolute(&path).map_err(anyhow::Error::msg))
|
||||||
|
.transpose()?
|
||||||
|
.unwrap_or(active_official_resource_root(&options.config.output_root)?);
|
||||||
|
let provider = options
|
||||||
|
.translation_provider
|
||||||
|
.as_deref()
|
||||||
|
.unwrap_or(TranslationProviderKind::Mock.as_str());
|
||||||
|
let provider = TranslationProviderKind::parse(provider)
|
||||||
|
.ok_or_else(|| anyhow::anyhow!("i18n worker run 的 provider 无效:{provider}"))?;
|
||||||
|
let config = TranslationWorkerConfig {
|
||||||
|
provider,
|
||||||
|
fixture_path: options.translation_fixture.clone(),
|
||||||
|
concurrency: options.worker_concurrency,
|
||||||
|
max_attempts: options.worker_max_attempts,
|
||||||
|
lease_seconds: options.worker_lease_seconds,
|
||||||
|
retry_backoff: options.worker_retry_backoff,
|
||||||
|
max_tasks: options.worker_max_tasks,
|
||||||
|
worker_id: options
|
||||||
|
.worker_id
|
||||||
|
.clone()
|
||||||
|
.unwrap_or_else(|| format!("bat-worker-{}", std::process::id())),
|
||||||
|
};
|
||||||
|
config.validate()?;
|
||||||
|
let runtime = tokio::runtime::Builder::new_current_thread()
|
||||||
|
.enable_all()
|
||||||
|
.build()?;
|
||||||
|
let report = runtime.block_on(bat_infrastructure::run_translation_worker_at(
|
||||||
|
&resource_root,
|
||||||
|
&config,
|
||||||
|
))?;
|
||||||
|
print_report(options.output_format, &report)
|
||||||
|
}
|
||||||
|
|
||||||
pub(super) fn run_repack(options: &CliOptions) -> anyhow::Result<()> {
|
pub(super) fn run_repack(options: &CliOptions) -> anyhow::Result<()> {
|
||||||
let spec = options
|
let spec = options
|
||||||
.repack_spec
|
.repack_spec
|
||||||
|
|||||||
@@ -156,7 +156,7 @@ mod tests {
|
|||||||
use super::*;
|
use super::*;
|
||||||
use std::sync::atomic::{AtomicUsize, Ordering};
|
use std::sync::atomic::{AtomicUsize, Ordering};
|
||||||
use std::thread;
|
use std::thread;
|
||||||
use std::time::Duration;
|
use std::time::{Duration, Instant};
|
||||||
|
|
||||||
struct TestBackend {
|
struct TestBackend {
|
||||||
active: AtomicUsize,
|
active: AtomicUsize,
|
||||||
@@ -222,7 +222,15 @@ mod tests {
|
|||||||
fn download(&self, task: usize) -> Result<Self::Output, Self::Error> {
|
fn download(&self, task: usize) -> Result<Self::Output, Self::Error> {
|
||||||
if task == 0 {
|
if task == 0 {
|
||||||
self.active.fetch_add(1, Ordering::SeqCst);
|
self.active.fetch_add(1, Ordering::SeqCst);
|
||||||
thread::sleep(Duration::from_millis(20));
|
let deadline = Instant::now() + Duration::from_millis(500);
|
||||||
|
while self
|
||||||
|
.task_two_started_while_task_zero_active
|
||||||
|
.load(Ordering::SeqCst)
|
||||||
|
== 0
|
||||||
|
&& Instant::now() < deadline
|
||||||
|
{
|
||||||
|
thread::sleep(Duration::from_millis(1));
|
||||||
|
}
|
||||||
self.active.fetch_sub(1, Ordering::SeqCst);
|
self.active.fetch_sub(1, Ordering::SeqCst);
|
||||||
} else {
|
} else {
|
||||||
if task == 1 {
|
if task == 1 {
|
||||||
|
|||||||
@@ -30,6 +30,7 @@ pub mod path_security;
|
|||||||
pub mod release_flow;
|
pub mod release_flow;
|
||||||
pub mod resources;
|
pub mod resources;
|
||||||
pub mod translation_tasks;
|
pub mod translation_tasks;
|
||||||
|
pub mod translation_worker;
|
||||||
pub mod translation_workflow;
|
pub mod translation_workflow;
|
||||||
mod zip_validation;
|
mod zip_validation;
|
||||||
|
|
||||||
@@ -138,11 +139,21 @@ pub use translation_tasks::{
|
|||||||
build_translation_handoff, read_translation_handoff_at, sync_translation_task_repository_at,
|
build_translation_handoff, read_translation_handoff_at, sync_translation_task_repository_at,
|
||||||
write_translation_handoff_at, PersistedTranslationTask, PersistedTranslationTaskState,
|
write_translation_handoff_at, PersistedTranslationTask, PersistedTranslationTaskState,
|
||||||
ProviderRun, ProviderRunStatus, SqliteTranslationTaskRepository, TranslationHandoff,
|
ProviderRun, ProviderRunStatus, SqliteTranslationTaskRepository, TranslationHandoff,
|
||||||
TranslationJob, TranslationJobStatus, TranslationTaskStatus, TranslationTaskSyncReport,
|
TranslationJob, TranslationJobStatus, TranslationTaskFailure, TranslationTaskStatus,
|
||||||
TranslationUnit, TranslationUnitStatus, TRANSLATION_HANDOFF_FILE,
|
TranslationTaskSyncReport, TranslationTaskUnitResult, TranslationUnit, TranslationUnitStatus,
|
||||||
TRANSLATION_HANDOFF_SCHEMA_VERSION, TRANSLATION_TASK_REPOSITORY_FILE,
|
TRANSLATION_HANDOFF_FILE, TRANSLATION_HANDOFF_SCHEMA_VERSION, TRANSLATION_TASK_REPOSITORY_FILE,
|
||||||
TRANSLATION_TASK_SCHEMA_VERSION,
|
TRANSLATION_TASK_SCHEMA_VERSION,
|
||||||
};
|
};
|
||||||
|
pub use translation_worker::{
|
||||||
|
run_translation_worker_at, run_translation_worker_with_provider, CrowdinProvider,
|
||||||
|
MockTranslationProvider, TranslationProvider, TranslationProviderFailureClass,
|
||||||
|
TranslationProviderKind, TranslationProviderRequest, TranslationProviderResponse,
|
||||||
|
TranslationProviderUnit, TranslationProviderUnitResult, TranslationWorkerConfig,
|
||||||
|
TranslationWorkerFailure, TranslationWorkerReport, DEFAULT_TRANSLATION_CONCURRENCY,
|
||||||
|
DEFAULT_TRANSLATION_LEASE_SECONDS, DEFAULT_TRANSLATION_MAX_ATTEMPTS,
|
||||||
|
DEFAULT_TRANSLATION_RETRY_BACKOFF, MAX_TRANSLATION_CONCURRENCY, MIN_TRANSLATION_CONCURRENCY,
|
||||||
|
MOCK_TRANSLATION_FIXTURE_VERSION,
|
||||||
|
};
|
||||||
pub use translation_workflow::{
|
pub use translation_workflow::{
|
||||||
export_translation_workbench, get_translation_entry, localized_text_asset_patches,
|
export_translation_workbench, get_translation_entry, localized_text_asset_patches,
|
||||||
read_translation_workbench, repack_bundle, set_translation, unset_translation,
|
read_translation_workbench, repack_bundle, set_translation, unset_translation,
|
||||||
|
|||||||
@@ -14,15 +14,15 @@ use crate::path_security::{
|
|||||||
};
|
};
|
||||||
use bat_core::Result;
|
use bat_core::Result;
|
||||||
use serde::{Deserialize, Serialize};
|
use serde::{Deserialize, Serialize};
|
||||||
use sqlx::sqlite::{SqliteConnectOptions, SqlitePoolOptions};
|
use sqlx::sqlite::{SqliteConnectOptions, SqliteJournalMode, SqlitePoolOptions};
|
||||||
use sqlx::{QueryBuilder, Sqlite, SqlitePool};
|
use sqlx::{QueryBuilder, Sqlite, SqlitePool};
|
||||||
use std::collections::{BTreeMap, BTreeSet};
|
use std::collections::{BTreeMap, BTreeSet};
|
||||||
use std::path::Path;
|
use std::path::Path;
|
||||||
use std::str::FromStr;
|
use std::str::FromStr;
|
||||||
use std::time::{SystemTime, UNIX_EPOCH};
|
use std::time::{Duration, SystemTime, UNIX_EPOCH};
|
||||||
|
|
||||||
/// SQLite schema version for durable translation task state.
|
/// SQLite schema version for durable translation task state.
|
||||||
pub const TRANSLATION_TASK_SCHEMA_VERSION: u32 = 1;
|
pub const TRANSLATION_TASK_SCHEMA_VERSION: u32 = 2;
|
||||||
const TRANSLATION_TASK_SCHEMA_COMPONENT: &str = "translation_tasks";
|
const TRANSLATION_TASK_SCHEMA_COMPONENT: &str = "translation_tasks";
|
||||||
/// SQLite file name stored under a published official release root.
|
/// SQLite file name stored under a published official release root.
|
||||||
pub const TRANSLATION_TASK_REPOSITORY_FILE: &str = "translation-tasks.sqlite";
|
pub const TRANSLATION_TASK_REPOSITORY_FILE: &str = "translation-tasks.sqlite";
|
||||||
@@ -147,6 +147,26 @@ pub struct TranslationUnit {
|
|||||||
pub status: TranslationUnitStatus,
|
pub status: TranslationUnitStatus,
|
||||||
/// Provider failure diagnostic, when present.
|
/// Provider failure diagnostic, when present.
|
||||||
pub failure_reason: Option<String>,
|
pub failure_reason: Option<String>,
|
||||||
|
/// TextUnit-level provider results associated with this resource task.
|
||||||
|
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||||
|
pub translation_results: Vec<TranslationTaskUnitResult>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One provider result linked to an immutable TextUnit.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct TranslationTaskUnitResult {
|
||||||
|
/// Stable TextUnit ID from `official-textunit-index.json`.
|
||||||
|
pub unit_id: String,
|
||||||
|
/// Source text submitted to the provider.
|
||||||
|
pub source_text: String,
|
||||||
|
/// Provider-produced or human-supplied translation.
|
||||||
|
pub translated_text: String,
|
||||||
|
/// Provider identifier.
|
||||||
|
pub provider: String,
|
||||||
|
/// Provider run that produced this result.
|
||||||
|
pub provider_run_id: String,
|
||||||
|
/// Result persistence time.
|
||||||
|
pub translated_unix_seconds: u64,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// One provider execution associated with one or more translation units.
|
/// One provider execution associated with one or more translation units.
|
||||||
@@ -203,7 +223,16 @@ pub fn build_translation_handoff(
|
|||||||
let state = persisted
|
let state = persisted
|
||||||
.get(task.task_id.as_str())
|
.get(task.task_id.as_str())
|
||||||
.copied()
|
.copied()
|
||||||
.map(|task| (task.task_status, task.failure_reason.clone(), task.attempt_count, task.provider_run_id.clone()))
|
.map(|task| {
|
||||||
|
(
|
||||||
|
task.task_status,
|
||||||
|
task.failure_reason.clone(),
|
||||||
|
task.attempt_count,
|
||||||
|
task.provider_run_id.clone(),
|
||||||
|
task.provider.clone(),
|
||||||
|
task.translation_results.clone(),
|
||||||
|
)
|
||||||
|
})
|
||||||
.unwrap_or_else(|| {
|
.unwrap_or_else(|| {
|
||||||
(
|
(
|
||||||
initial_task_status(task),
|
initial_task_status(task),
|
||||||
@@ -214,6 +243,8 @@ pub fn build_translation_handoff(
|
|||||||
},
|
},
|
||||||
0,
|
0,
|
||||||
None,
|
None,
|
||||||
|
None,
|
||||||
|
Vec::new(),
|
||||||
)
|
)
|
||||||
});
|
});
|
||||||
let unit_status = match state.0 {
|
let unit_status = match state.0 {
|
||||||
@@ -234,6 +265,7 @@ pub fn build_translation_handoff(
|
|||||||
text_unit_formats: task.text_unit_formats.clone(),
|
text_unit_formats: task.text_unit_formats.clone(),
|
||||||
status: unit_status,
|
status: unit_status,
|
||||||
failure_reason: state.1.clone(),
|
failure_reason: state.1.clone(),
|
||||||
|
translation_results: state.5.clone(),
|
||||||
};
|
};
|
||||||
if let Some(provider_run_id) = state.3 {
|
if let Some(provider_run_id) = state.3 {
|
||||||
let run = provider_runs
|
let run = provider_runs
|
||||||
@@ -241,7 +273,7 @@ pub fn build_translation_handoff(
|
|||||||
.or_insert_with(|| ProviderRun {
|
.or_insert_with(|| ProviderRun {
|
||||||
provider_run_id,
|
provider_run_id,
|
||||||
job_id: job_id.clone(),
|
job_id: job_id.clone(),
|
||||||
provider: "worker".to_string(),
|
provider: state.4.unwrap_or_else(|| "worker".to_string()),
|
||||||
status: ProviderRunStatus::Queued,
|
status: ProviderRunStatus::Queued,
|
||||||
unit_ids: Vec::new(),
|
unit_ids: Vec::new(),
|
||||||
attempt_count: 0,
|
attempt_count: 0,
|
||||||
@@ -403,6 +435,26 @@ pub struct PersistedTranslationTask {
|
|||||||
/// Provider-side run identifier, if known.
|
/// Provider-side run identifier, if known.
|
||||||
#[serde(skip_serializing_if = "Option::is_none")]
|
#[serde(skip_serializing_if = "Option::is_none")]
|
||||||
pub provider_run_id: Option<String>,
|
pub provider_run_id: Option<String>,
|
||||||
|
/// Provider-produced results keyed by TextUnit ID.
|
||||||
|
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||||
|
pub translation_results: Vec<TranslationTaskUnitResult>,
|
||||||
|
/// Provider identifier used by the latest run.
|
||||||
|
#[serde(skip_serializing_if = "Option::is_none")]
|
||||||
|
pub provider: Option<String>,
|
||||||
|
/// Worker currently holding the lease.
|
||||||
|
#[serde(skip_serializing_if = "Option::is_none")]
|
||||||
|
pub lease_owner: Option<String>,
|
||||||
|
/// Lease expiry as Unix seconds.
|
||||||
|
#[serde(skip_serializing_if = "Option::is_none")]
|
||||||
|
pub lease_expires_unix_seconds: Option<u64>,
|
||||||
|
/// Stable failure classification.
|
||||||
|
#[serde(skip_serializing_if = "Option::is_none")]
|
||||||
|
pub failure_class: Option<String>,
|
||||||
|
/// Whether the latest failure can be retried.
|
||||||
|
pub failure_retryable: bool,
|
||||||
|
/// Earliest Unix time at which a failed task may be claimed again.
|
||||||
|
#[serde(skip_serializing_if = "Option::is_none")]
|
||||||
|
pub next_attempt_unix_seconds: Option<u64>,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Mutable persistence metadata associated with an official TextUnit task.
|
/// Mutable persistence metadata associated with an official TextUnit task.
|
||||||
@@ -422,6 +474,39 @@ pub struct PersistedTranslationTaskState {
|
|||||||
pub completed_unix_seconds: Option<u64>,
|
pub completed_unix_seconds: Option<u64>,
|
||||||
/// Provider-side run identifier, if known.
|
/// Provider-side run identifier, if known.
|
||||||
pub provider_run_id: Option<String>,
|
pub provider_run_id: Option<String>,
|
||||||
|
/// Provider-produced results keyed by TextUnit ID.
|
||||||
|
pub translation_results: Vec<TranslationTaskUnitResult>,
|
||||||
|
/// Provider identifier used by the latest run.
|
||||||
|
pub provider: Option<String>,
|
||||||
|
/// Worker currently holding the lease.
|
||||||
|
pub lease_owner: Option<String>,
|
||||||
|
/// Lease expiry as Unix seconds.
|
||||||
|
pub lease_expires_unix_seconds: Option<u64>,
|
||||||
|
/// Stable failure classification.
|
||||||
|
pub failure_class: Option<String>,
|
||||||
|
/// Whether the latest failure can be retried.
|
||||||
|
pub failure_retryable: bool,
|
||||||
|
/// Earliest Unix time at which a failed task may be claimed again.
|
||||||
|
pub next_attempt_unix_seconds: Option<u64>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Provider failure data used to atomically release a task lease.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub struct TranslationTaskFailure {
|
||||||
|
/// Task being failed.
|
||||||
|
pub task_id: String,
|
||||||
|
/// Worker that owns the lease.
|
||||||
|
pub worker_id: String,
|
||||||
|
/// Provider run associated with the lease.
|
||||||
|
pub provider_run_id: String,
|
||||||
|
/// Stable failure classification.
|
||||||
|
pub failure_class: String,
|
||||||
|
/// Redacted diagnostic message.
|
||||||
|
pub failure_reason: String,
|
||||||
|
/// Whether a future attempt may retry the task.
|
||||||
|
pub retryable: bool,
|
||||||
|
/// Earliest retry time, when retryable.
|
||||||
|
pub next_attempt_unix_seconds: Option<u64>,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Result of synchronizing an immutable release queue into SQLite.
|
/// Result of synchronizing an immutable release queue into SQLite.
|
||||||
@@ -463,7 +548,9 @@ impl SqliteTranslationTaskRepository {
|
|||||||
|
|
||||||
let options = SqliteConnectOptions::from_str(&format!("sqlite://{}", path.display()))
|
let options = SqliteConnectOptions::from_str(&format!("sqlite://{}", path.display()))
|
||||||
.map_err(|error| bat_core::Error::Other(error.into()))?
|
.map_err(|error| bat_core::Error::Other(error.into()))?
|
||||||
.create_if_missing(create_if_missing);
|
.create_if_missing(create_if_missing)
|
||||||
|
.journal_mode(SqliteJournalMode::Wal)
|
||||||
|
.busy_timeout(Duration::from_secs(30));
|
||||||
let pool = SqlitePoolOptions::new()
|
let pool = SqlitePoolOptions::new()
|
||||||
.max_connections(1)
|
.max_connections(1)
|
||||||
.connect_with(options)
|
.connect_with(options)
|
||||||
@@ -510,7 +597,14 @@ impl SqliteTranslationTaskRepository {
|
|||||||
created_unix_seconds INTEGER NOT NULL,
|
created_unix_seconds INTEGER NOT NULL,
|
||||||
updated_unix_seconds INTEGER NOT NULL,
|
updated_unix_seconds INTEGER NOT NULL,
|
||||||
completed_unix_seconds INTEGER,
|
completed_unix_seconds INTEGER,
|
||||||
provider_run_id TEXT
|
provider_run_id TEXT,
|
||||||
|
translation_results_json TEXT NOT NULL DEFAULT '[]',
|
||||||
|
provider TEXT,
|
||||||
|
lease_owner TEXT,
|
||||||
|
lease_expires_unix_seconds INTEGER,
|
||||||
|
failure_class TEXT,
|
||||||
|
failure_retryable INTEGER NOT NULL DEFAULT 0,
|
||||||
|
next_attempt_unix_seconds INTEGER
|
||||||
)
|
)
|
||||||
"#,
|
"#,
|
||||||
)
|
)
|
||||||
@@ -573,6 +667,37 @@ impl SqliteTranslationTaskRepository {
|
|||||||
)
|
)
|
||||||
.await?;
|
.await?;
|
||||||
ensure_column(&self.pool, "translation_tasks", "provider_run_id", "TEXT").await?;
|
ensure_column(&self.pool, "translation_tasks", "provider_run_id", "TEXT").await?;
|
||||||
|
ensure_column(
|
||||||
|
&self.pool,
|
||||||
|
"translation_tasks",
|
||||||
|
"translation_results_json",
|
||||||
|
"TEXT NOT NULL DEFAULT '[]'",
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
ensure_column(&self.pool, "translation_tasks", "provider", "TEXT").await?;
|
||||||
|
ensure_column(&self.pool, "translation_tasks", "lease_owner", "TEXT").await?;
|
||||||
|
ensure_column(
|
||||||
|
&self.pool,
|
||||||
|
"translation_tasks",
|
||||||
|
"lease_expires_unix_seconds",
|
||||||
|
"INTEGER",
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
ensure_column(&self.pool, "translation_tasks", "failure_class", "TEXT").await?;
|
||||||
|
ensure_column(
|
||||||
|
&self.pool,
|
||||||
|
"translation_tasks",
|
||||||
|
"failure_retryable",
|
||||||
|
"INTEGER NOT NULL DEFAULT 0",
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
ensure_column(
|
||||||
|
&self.pool,
|
||||||
|
"translation_tasks",
|
||||||
|
"next_attempt_unix_seconds",
|
||||||
|
"INTEGER",
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
|
||||||
let current: Option<i64> =
|
let current: Option<i64> =
|
||||||
sqlx::query_scalar("SELECT version FROM schema_migrations WHERE component = ?1")
|
sqlx::query_scalar("SELECT version FROM schema_migrations WHERE component = ?1")
|
||||||
@@ -615,7 +740,10 @@ impl SqliteTranslationTaskRepository {
|
|||||||
let existing: Option<ExistingTaskRow> = sqlx::query_as(
|
let existing: Option<ExistingTaskRow> = sqlx::query_as(
|
||||||
r#"
|
r#"
|
||||||
SELECT task_json, worker_status, failure_reason, attempt_count,
|
SELECT task_json, worker_status, failure_reason, attempt_count,
|
||||||
created_unix_seconds, completed_unix_seconds, provider_run_id
|
created_unix_seconds, completed_unix_seconds, provider_run_id,
|
||||||
|
translation_results_json, provider, lease_owner,
|
||||||
|
lease_expires_unix_seconds, failure_class, failure_retryable,
|
||||||
|
next_attempt_unix_seconds
|
||||||
FROM translation_tasks
|
FROM translation_tasks
|
||||||
WHERE task_id = ?1
|
WHERE task_id = ?1
|
||||||
"#,
|
"#,
|
||||||
@@ -642,11 +770,31 @@ impl SqliteTranslationTaskRepository {
|
|||||||
created,
|
created,
|
||||||
completed,
|
completed,
|
||||||
provider_run_id,
|
provider_run_id,
|
||||||
|
translation_results_json,
|
||||||
|
provider,
|
||||||
|
lease_owner,
|
||||||
|
lease_expires,
|
||||||
|
failure_class,
|
||||||
|
failure_retryable,
|
||||||
|
next_attempt,
|
||||||
)) = existing
|
)) = existing
|
||||||
{
|
{
|
||||||
let immutable_unchanged = previous_task_json == task_json;
|
let immutable_unchanged = previous_task_json == task_json;
|
||||||
let (status, failure_reason, attempt_count, created, completed, provider_run_id) =
|
let (
|
||||||
if immutable_unchanged {
|
status,
|
||||||
|
failure_reason,
|
||||||
|
attempt_count,
|
||||||
|
created,
|
||||||
|
completed,
|
||||||
|
provider_run_id,
|
||||||
|
translation_results_json,
|
||||||
|
provider,
|
||||||
|
lease_owner,
|
||||||
|
lease_expires,
|
||||||
|
failure_class,
|
||||||
|
failure_retryable,
|
||||||
|
next_attempt,
|
||||||
|
) = if immutable_unchanged {
|
||||||
(
|
(
|
||||||
worker_status,
|
worker_status,
|
||||||
failure_reason,
|
failure_reason,
|
||||||
@@ -654,6 +802,13 @@ impl SqliteTranslationTaskRepository {
|
|||||||
created,
|
created,
|
||||||
completed,
|
completed,
|
||||||
provider_run_id,
|
provider_run_id,
|
||||||
|
translation_results_json,
|
||||||
|
provider,
|
||||||
|
lease_owner,
|
||||||
|
lease_expires,
|
||||||
|
failure_class,
|
||||||
|
failure_retryable,
|
||||||
|
next_attempt,
|
||||||
)
|
)
|
||||||
} else {
|
} else {
|
||||||
report.refreshed_count += 1;
|
report.refreshed_count += 1;
|
||||||
@@ -664,6 +819,13 @@ impl SqliteTranslationTaskRepository {
|
|||||||
now,
|
now,
|
||||||
None,
|
None,
|
||||||
None,
|
None,
|
||||||
|
"[]".to_string(),
|
||||||
|
None,
|
||||||
|
None,
|
||||||
|
None,
|
||||||
|
None,
|
||||||
|
0_i64,
|
||||||
|
None,
|
||||||
)
|
)
|
||||||
};
|
};
|
||||||
if immutable_unchanged {
|
if immutable_unchanged {
|
||||||
@@ -677,7 +839,11 @@ impl SqliteTranslationTaskRepository {
|
|||||||
text_unit_formats_json = ?8, task_json = ?9,
|
text_unit_formats_json = ?8, task_json = ?9,
|
||||||
worker_status = ?10, failure_reason = ?11, attempt_count = ?12,
|
worker_status = ?10, failure_reason = ?11, attempt_count = ?12,
|
||||||
created_unix_seconds = ?13, updated_unix_seconds = ?14,
|
created_unix_seconds = ?13, updated_unix_seconds = ?14,
|
||||||
completed_unix_seconds = ?15, provider_run_id = ?16
|
completed_unix_seconds = ?15, provider_run_id = ?16,
|
||||||
|
translation_results_json = ?17, provider = ?18,
|
||||||
|
lease_owner = ?19, lease_expires_unix_seconds = ?20,
|
||||||
|
failure_class = ?21, failure_retryable = ?22,
|
||||||
|
next_attempt_unix_seconds = ?23
|
||||||
WHERE task_id = ?1
|
WHERE task_id = ?1
|
||||||
"#,
|
"#,
|
||||||
)
|
)
|
||||||
@@ -697,6 +863,13 @@ impl SqliteTranslationTaskRepository {
|
|||||||
.bind(now)
|
.bind(now)
|
||||||
.bind(completed)
|
.bind(completed)
|
||||||
.bind(provider_run_id)
|
.bind(provider_run_id)
|
||||||
|
.bind(translation_results_json)
|
||||||
|
.bind(provider)
|
||||||
|
.bind(lease_owner)
|
||||||
|
.bind(lease_expires)
|
||||||
|
.bind(failure_class)
|
||||||
|
.bind(failure_retryable)
|
||||||
|
.bind(next_attempt)
|
||||||
.execute(&mut *transaction)
|
.execute(&mut *transaction)
|
||||||
.await
|
.await
|
||||||
.map_err(db_error)?;
|
.map_err(db_error)?;
|
||||||
@@ -770,7 +943,9 @@ impl SqliteTranslationTaskRepository {
|
|||||||
r#"
|
r#"
|
||||||
SELECT task_json, worker_status, failure_reason, attempt_count,
|
SELECT task_json, worker_status, failure_reason, attempt_count,
|
||||||
created_unix_seconds, updated_unix_seconds,
|
created_unix_seconds, updated_unix_seconds,
|
||||||
completed_unix_seconds, provider_run_id
|
completed_unix_seconds, provider_run_id, translation_results_json,
|
||||||
|
provider, lease_owner, lease_expires_unix_seconds, failure_class,
|
||||||
|
failure_retryable, next_attempt_unix_seconds
|
||||||
FROM translation_tasks
|
FROM translation_tasks
|
||||||
ORDER BY task_id
|
ORDER BY task_id
|
||||||
"#,
|
"#,
|
||||||
@@ -794,6 +969,247 @@ impl SqliteTranslationTaskRepository {
|
|||||||
Ok(self.list(query).await?.len() as u64)
|
Ok(self.list(query).await?.len() as u64)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Requeues tasks whose worker lease expired.
|
||||||
|
pub async fn recover_expired_leases(&self, now_unix_seconds: u64) -> Result<u64> {
|
||||||
|
let now = i64::try_from(now_unix_seconds).unwrap_or(i64::MAX);
|
||||||
|
let result = sqlx::query(
|
||||||
|
r#"
|
||||||
|
UPDATE translation_tasks
|
||||||
|
SET worker_status = CASE
|
||||||
|
WHEN attempt_count > 0 THEN 'failed'
|
||||||
|
ELSE 'queued'
|
||||||
|
END,
|
||||||
|
failure_reason = 'worker lease expired; task recovered',
|
||||||
|
failure_class = 'lease_expired',
|
||||||
|
failure_retryable = 1,
|
||||||
|
next_attempt_unix_seconds = ?1,
|
||||||
|
updated_unix_seconds = ?1,
|
||||||
|
lease_owner = NULL,
|
||||||
|
lease_expires_unix_seconds = NULL
|
||||||
|
WHERE worker_status = 'running'
|
||||||
|
AND lease_expires_unix_seconds IS NOT NULL
|
||||||
|
AND lease_expires_unix_seconds <= ?1
|
||||||
|
"#,
|
||||||
|
)
|
||||||
|
.bind(now)
|
||||||
|
.execute(&self.pool)
|
||||||
|
.await
|
||||||
|
.map_err(db_error)?;
|
||||||
|
Ok(result.rows_affected())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Claims one queued or retryable failed task and assigns an exclusive
|
||||||
|
/// lease to one worker.
|
||||||
|
pub async fn claim_next(
|
||||||
|
&self,
|
||||||
|
worker_id: &str,
|
||||||
|
provider: &str,
|
||||||
|
lease_seconds: u64,
|
||||||
|
max_attempts: u32,
|
||||||
|
) -> Result<Option<PersistedTranslationTask>> {
|
||||||
|
if worker_id.trim().is_empty() || provider.trim().is_empty() {
|
||||||
|
return Err(bat_core::Error::InvalidArgument(
|
||||||
|
"translation worker_id/provider 不能为空".to_string(),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
if max_attempts == 0 {
|
||||||
|
return Err(bat_core::Error::InvalidArgument(
|
||||||
|
"translation worker max_attempts 必须大于 0".to_string(),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
if lease_seconds == 0 {
|
||||||
|
return Err(bat_core::Error::InvalidArgument(
|
||||||
|
"translation worker lease_seconds 必须大于 0".to_string(),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
let now = unix_seconds_now_i64();
|
||||||
|
let lease_expires = now.saturating_add(i64::try_from(lease_seconds).unwrap_or(i64::MAX));
|
||||||
|
let mut transaction = self.pool.begin().await.map_err(db_error)?;
|
||||||
|
let row: Option<TranslationTaskRow> = sqlx::query_as(
|
||||||
|
r#"
|
||||||
|
SELECT task_json, worker_status, failure_reason, attempt_count,
|
||||||
|
created_unix_seconds, updated_unix_seconds,
|
||||||
|
completed_unix_seconds, provider_run_id, translation_results_json,
|
||||||
|
provider, lease_owner, lease_expires_unix_seconds, failure_class,
|
||||||
|
failure_retryable, next_attempt_unix_seconds
|
||||||
|
FROM translation_tasks
|
||||||
|
WHERE queue_status = 'queued_offline'
|
||||||
|
AND (
|
||||||
|
worker_status = 'queued'
|
||||||
|
OR (
|
||||||
|
worker_status = 'failed'
|
||||||
|
AND failure_retryable = 1
|
||||||
|
AND attempt_count < ?1
|
||||||
|
AND (next_attempt_unix_seconds IS NULL
|
||||||
|
OR next_attempt_unix_seconds <= ?2)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
AND (lease_expires_unix_seconds IS NULL OR lease_expires_unix_seconds <= ?2)
|
||||||
|
ORDER BY task_id
|
||||||
|
LIMIT 1
|
||||||
|
"#,
|
||||||
|
)
|
||||||
|
.bind(i64::from(max_attempts))
|
||||||
|
.bind(now)
|
||||||
|
.fetch_optional(&mut *transaction)
|
||||||
|
.await
|
||||||
|
.map_err(db_error)?;
|
||||||
|
let Some(row) = row else {
|
||||||
|
transaction.commit().await.map_err(db_error)?;
|
||||||
|
return Ok(None);
|
||||||
|
};
|
||||||
|
let current = PersistedTranslationTask::from_row(row)?;
|
||||||
|
let attempt_count = current.attempt_count.saturating_add(1);
|
||||||
|
let provider_run_id = format!(
|
||||||
|
"{provider}:{}:attempt-{attempt_count}",
|
||||||
|
current.task.task_id
|
||||||
|
);
|
||||||
|
let result = sqlx::query(
|
||||||
|
r#"
|
||||||
|
UPDATE translation_tasks
|
||||||
|
SET worker_status = 'running',
|
||||||
|
failure_reason = NULL,
|
||||||
|
attempt_count = ?2,
|
||||||
|
updated_unix_seconds = ?3,
|
||||||
|
completed_unix_seconds = NULL,
|
||||||
|
provider_run_id = ?4,
|
||||||
|
provider = ?5,
|
||||||
|
lease_owner = ?6,
|
||||||
|
lease_expires_unix_seconds = ?7,
|
||||||
|
failure_class = NULL,
|
||||||
|
failure_retryable = 0,
|
||||||
|
next_attempt_unix_seconds = NULL
|
||||||
|
WHERE task_id = ?1
|
||||||
|
AND (
|
||||||
|
worker_status = 'queued'
|
||||||
|
OR (
|
||||||
|
worker_status = 'failed'
|
||||||
|
AND failure_retryable = 1
|
||||||
|
AND attempt_count < ?8
|
||||||
|
AND (next_attempt_unix_seconds IS NULL
|
||||||
|
OR next_attempt_unix_seconds <= ?3)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
AND queue_status = 'queued_offline'
|
||||||
|
AND (lease_expires_unix_seconds IS NULL OR lease_expires_unix_seconds <= ?3)
|
||||||
|
"#,
|
||||||
|
)
|
||||||
|
.bind(¤t.task.task_id)
|
||||||
|
.bind(i64::from(attempt_count))
|
||||||
|
.bind(now)
|
||||||
|
.bind(&provider_run_id)
|
||||||
|
.bind(provider)
|
||||||
|
.bind(worker_id)
|
||||||
|
.bind(lease_expires)
|
||||||
|
.bind(i64::from(max_attempts))
|
||||||
|
.execute(&mut *transaction)
|
||||||
|
.await
|
||||||
|
.map_err(db_error)?;
|
||||||
|
if result.rows_affected() != 1 {
|
||||||
|
transaction.rollback().await.map_err(db_error)?;
|
||||||
|
return Ok(None);
|
||||||
|
}
|
||||||
|
transaction.commit().await.map_err(db_error)?;
|
||||||
|
self.find(¤t.task.task_id).await.map(Some)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Stores an idempotent provider result while the worker still owns its lease.
|
||||||
|
pub async fn complete_claim(
|
||||||
|
&self,
|
||||||
|
task_id: &str,
|
||||||
|
worker_id: &str,
|
||||||
|
provider_run_id: &str,
|
||||||
|
provider: &str,
|
||||||
|
translation_results: &[TranslationTaskUnitResult],
|
||||||
|
) -> Result<PersistedTranslationTask> {
|
||||||
|
let results_json = serde_json::to_string(translation_results)
|
||||||
|
.map_err(|error| bat_core::Error::Serialization(error.to_string()))?;
|
||||||
|
let now = unix_seconds_now_i64();
|
||||||
|
let result = sqlx::query(
|
||||||
|
r#"
|
||||||
|
UPDATE translation_tasks
|
||||||
|
SET worker_status = 'completed',
|
||||||
|
failure_reason = NULL,
|
||||||
|
updated_unix_seconds = ?2,
|
||||||
|
completed_unix_seconds = ?2,
|
||||||
|
provider_run_id = ?3,
|
||||||
|
provider = ?4,
|
||||||
|
translation_results_json = ?5,
|
||||||
|
lease_owner = NULL,
|
||||||
|
lease_expires_unix_seconds = NULL,
|
||||||
|
failure_class = NULL,
|
||||||
|
failure_retryable = 0,
|
||||||
|
next_attempt_unix_seconds = NULL
|
||||||
|
WHERE task_id = ?1
|
||||||
|
AND worker_status = 'running'
|
||||||
|
AND lease_owner = ?6
|
||||||
|
AND provider_run_id = ?3
|
||||||
|
"#,
|
||||||
|
)
|
||||||
|
.bind(task_id)
|
||||||
|
.bind(now)
|
||||||
|
.bind(provider_run_id)
|
||||||
|
.bind(provider)
|
||||||
|
.bind(results_json)
|
||||||
|
.bind(worker_id)
|
||||||
|
.execute(&self.pool)
|
||||||
|
.await
|
||||||
|
.map_err(db_error)?;
|
||||||
|
if result.rows_affected() != 1 {
|
||||||
|
return Err(bat_core::Error::InvalidArgument(format!(
|
||||||
|
"翻译任务 {} 的 lease 已失效,拒绝写入 provider 结果",
|
||||||
|
task_id
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
self.find(task_id).await
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Records one provider failure and releases the worker lease.
|
||||||
|
pub async fn fail_claim(
|
||||||
|
&self,
|
||||||
|
failure: TranslationTaskFailure,
|
||||||
|
) -> Result<PersistedTranslationTask> {
|
||||||
|
let now = unix_seconds_now_i64();
|
||||||
|
let next_attempt = failure
|
||||||
|
.next_attempt_unix_seconds
|
||||||
|
.map(|value| i64::try_from(value).unwrap_or(i64::MAX));
|
||||||
|
let result = sqlx::query(
|
||||||
|
r#"
|
||||||
|
UPDATE translation_tasks
|
||||||
|
SET worker_status = 'failed',
|
||||||
|
failure_reason = ?2,
|
||||||
|
updated_unix_seconds = ?3,
|
||||||
|
lease_owner = NULL,
|
||||||
|
lease_expires_unix_seconds = NULL,
|
||||||
|
failure_class = ?4,
|
||||||
|
failure_retryable = ?5,
|
||||||
|
next_attempt_unix_seconds = ?6
|
||||||
|
WHERE task_id = ?1
|
||||||
|
AND worker_status = 'running'
|
||||||
|
AND lease_owner = ?7
|
||||||
|
AND provider_run_id = ?8
|
||||||
|
"#,
|
||||||
|
)
|
||||||
|
.bind(&failure.task_id)
|
||||||
|
.bind(&failure.failure_reason)
|
||||||
|
.bind(now)
|
||||||
|
.bind(&failure.failure_class)
|
||||||
|
.bind(if failure.retryable { 1_i64 } else { 0_i64 })
|
||||||
|
.bind(next_attempt)
|
||||||
|
.bind(&failure.worker_id)
|
||||||
|
.bind(&failure.provider_run_id)
|
||||||
|
.execute(&self.pool)
|
||||||
|
.await
|
||||||
|
.map_err(db_error)?;
|
||||||
|
if result.rows_affected() != 1 {
|
||||||
|
return Err(bat_core::Error::InvalidArgument(format!(
|
||||||
|
"翻译任务 {} 的 lease 已失效,拒绝写入 provider 失败状态",
|
||||||
|
failure.task_id
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
self.find(&failure.task_id).await
|
||||||
|
}
|
||||||
|
|
||||||
/// Updates provider state and returns the durable task record.
|
/// Updates provider state and returns the durable task record.
|
||||||
pub async fn update_status(
|
pub async fn update_status(
|
||||||
&self,
|
&self,
|
||||||
@@ -818,7 +1234,10 @@ impl SqliteTranslationTaskRepository {
|
|||||||
UPDATE translation_tasks
|
UPDATE translation_tasks
|
||||||
SET worker_status = ?2, failure_reason = ?3, attempt_count = ?4,
|
SET worker_status = ?2, failure_reason = ?3, attempt_count = ?4,
|
||||||
updated_unix_seconds = ?5, completed_unix_seconds = ?6,
|
updated_unix_seconds = ?5, completed_unix_seconds = ?6,
|
||||||
provider_run_id = COALESCE(?7, provider_run_id)
|
provider_run_id = COALESCE(?7, provider_run_id),
|
||||||
|
lease_owner = NULL, lease_expires_unix_seconds = NULL,
|
||||||
|
failure_class = NULL, failure_retryable = 0,
|
||||||
|
next_attempt_unix_seconds = NULL
|
||||||
WHERE task_id = ?1
|
WHERE task_id = ?1
|
||||||
"#,
|
"#,
|
||||||
)
|
)
|
||||||
@@ -841,7 +1260,9 @@ impl SqliteTranslationTaskRepository {
|
|||||||
r#"
|
r#"
|
||||||
SELECT task_json, worker_status, failure_reason, attempt_count,
|
SELECT task_json, worker_status, failure_reason, attempt_count,
|
||||||
created_unix_seconds, updated_unix_seconds,
|
created_unix_seconds, updated_unix_seconds,
|
||||||
completed_unix_seconds, provider_run_id
|
completed_unix_seconds, provider_run_id, translation_results_json,
|
||||||
|
provider, lease_owner, lease_expires_unix_seconds, failure_class,
|
||||||
|
failure_retryable, next_attempt_unix_seconds
|
||||||
FROM translation_tasks
|
FROM translation_tasks
|
||||||
WHERE task_id = ?1
|
WHERE task_id = ?1
|
||||||
"#,
|
"#,
|
||||||
@@ -864,6 +1285,13 @@ type ExistingTaskRow = (
|
|||||||
i64,
|
i64,
|
||||||
Option<i64>,
|
Option<i64>,
|
||||||
Option<String>,
|
Option<String>,
|
||||||
|
String,
|
||||||
|
Option<String>,
|
||||||
|
Option<String>,
|
||||||
|
Option<i64>,
|
||||||
|
Option<String>,
|
||||||
|
i64,
|
||||||
|
Option<i64>,
|
||||||
);
|
);
|
||||||
|
|
||||||
type TranslationTaskRow = (
|
type TranslationTaskRow = (
|
||||||
@@ -875,6 +1303,13 @@ type TranslationTaskRow = (
|
|||||||
i64,
|
i64,
|
||||||
Option<i64>,
|
Option<i64>,
|
||||||
Option<String>,
|
Option<String>,
|
||||||
|
String,
|
||||||
|
Option<String>,
|
||||||
|
Option<String>,
|
||||||
|
Option<i64>,
|
||||||
|
Option<String>,
|
||||||
|
i64,
|
||||||
|
Option<i64>,
|
||||||
);
|
);
|
||||||
|
|
||||||
impl PersistedTranslationTask {
|
impl PersistedTranslationTask {
|
||||||
@@ -889,6 +1324,13 @@ impl PersistedTranslationTask {
|
|||||||
updated_unix_seconds: state.updated_unix_seconds,
|
updated_unix_seconds: state.updated_unix_seconds,
|
||||||
completed_unix_seconds: state.completed_unix_seconds,
|
completed_unix_seconds: state.completed_unix_seconds,
|
||||||
provider_run_id: state.provider_run_id,
|
provider_run_id: state.provider_run_id,
|
||||||
|
translation_results: state.translation_results,
|
||||||
|
provider: state.provider,
|
||||||
|
lease_owner: state.lease_owner,
|
||||||
|
lease_expires_unix_seconds: state.lease_expires_unix_seconds,
|
||||||
|
failure_class: state.failure_class,
|
||||||
|
failure_retryable: state.failure_retryable,
|
||||||
|
next_attempt_unix_seconds: state.next_attempt_unix_seconds,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -910,6 +1352,13 @@ impl PersistedTranslationTask {
|
|||||||
updated_unix_seconds: generated_unix_seconds,
|
updated_unix_seconds: generated_unix_seconds,
|
||||||
completed_unix_seconds: None,
|
completed_unix_seconds: None,
|
||||||
provider_run_id: None,
|
provider_run_id: None,
|
||||||
|
translation_results: Vec::new(),
|
||||||
|
provider: None,
|
||||||
|
lease_owner: None,
|
||||||
|
lease_expires_unix_seconds: None,
|
||||||
|
failure_class: None,
|
||||||
|
failure_retryable: false,
|
||||||
|
next_attempt_unix_seconds: None,
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
@@ -924,12 +1373,21 @@ impl PersistedTranslationTask {
|
|||||||
updated_unix_seconds,
|
updated_unix_seconds,
|
||||||
completed_unix_seconds,
|
completed_unix_seconds,
|
||||||
provider_run_id,
|
provider_run_id,
|
||||||
|
translation_results_json,
|
||||||
|
provider,
|
||||||
|
lease_owner,
|
||||||
|
lease_expires_unix_seconds,
|
||||||
|
failure_class,
|
||||||
|
failure_retryable,
|
||||||
|
next_attempt_unix_seconds,
|
||||||
) = row;
|
) = row;
|
||||||
let task = serde_json::from_str(&task_json)
|
let task = serde_json::from_str(&task_json)
|
||||||
.map_err(|error| bat_core::Error::Serialization(error.to_string()))?;
|
.map_err(|error| bat_core::Error::Serialization(error.to_string()))?;
|
||||||
let task_status = TranslationTaskStatus::parse(&worker_status).ok_or_else(|| {
|
let task_status = TranslationTaskStatus::parse(&worker_status).ok_or_else(|| {
|
||||||
bat_core::Error::Serialization(format!("未知翻译任务 worker 状态:{worker_status}"))
|
bat_core::Error::Serialization(format!("未知翻译任务 worker 状态:{worker_status}"))
|
||||||
})?;
|
})?;
|
||||||
|
let translation_results = serde_json::from_str(&translation_results_json)
|
||||||
|
.map_err(|error| bat_core::Error::Serialization(error.to_string()))?;
|
||||||
Ok(Self {
|
Ok(Self {
|
||||||
task,
|
task,
|
||||||
task_status,
|
task_status,
|
||||||
@@ -951,6 +1409,25 @@ impl PersistedTranslationTask {
|
|||||||
})
|
})
|
||||||
.transpose()?,
|
.transpose()?,
|
||||||
provider_run_id,
|
provider_run_id,
|
||||||
|
translation_results,
|
||||||
|
provider,
|
||||||
|
lease_owner,
|
||||||
|
lease_expires_unix_seconds: lease_expires_unix_seconds
|
||||||
|
.map(|value| {
|
||||||
|
u64::try_from(value).map_err(|_| {
|
||||||
|
bat_core::Error::Serialization("翻译任务 lease 时间无效".to_string())
|
||||||
|
})
|
||||||
|
})
|
||||||
|
.transpose()?,
|
||||||
|
failure_class,
|
||||||
|
failure_retryable: failure_retryable != 0,
|
||||||
|
next_attempt_unix_seconds: next_attempt_unix_seconds
|
||||||
|
.map(|value| {
|
||||||
|
u64::try_from(value).map_err(|_| {
|
||||||
|
bat_core::Error::Serialization("翻译任务 next_attempt 时间无效".to_string())
|
||||||
|
})
|
||||||
|
})
|
||||||
|
.transpose()?,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1219,6 +1696,140 @@ mod tests {
|
|||||||
assert_eq!(retrievable[0].attempt_count, 1);
|
assert_eq!(retrievable[0].attempt_count, 1);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn sqlite_translation_tasks_recover_expired_leases_for_retry() {
|
||||||
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
let repository =
|
||||||
|
SqliteTranslationTaskRepository::new(temp.path().join("translation-tasks.sqlite"))
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
let queue = queue(vec![task(
|
||||||
|
"task-a",
|
||||||
|
"Bundles/a.bundle",
|
||||||
|
OfficialTextUnitTaskStatus::QueuedOffline,
|
||||||
|
Some(OfficialParseStatus::Parsed),
|
||||||
|
None,
|
||||||
|
)]);
|
||||||
|
repository.sync_queue(&queue).await.unwrap();
|
||||||
|
|
||||||
|
let claimed = repository
|
||||||
|
.claim_next("worker-a", "mock", 1, 3)
|
||||||
|
.await
|
||||||
|
.unwrap()
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(claimed.task_status, TranslationTaskStatus::Running);
|
||||||
|
assert_eq!(claimed.lease_owner.as_deref(), Some("worker-a"));
|
||||||
|
force_expire_lease(&repository, "task-a").await;
|
||||||
|
|
||||||
|
let recovered = repository
|
||||||
|
.recover_expired_leases(unix_seconds_now())
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(recovered, 1);
|
||||||
|
let recovered_task = repository.find("task-a").await.unwrap();
|
||||||
|
assert_eq!(recovered_task.task_status, TranslationTaskStatus::Failed);
|
||||||
|
assert_eq!(
|
||||||
|
recovered_task.failure_reason.as_deref(),
|
||||||
|
Some("worker lease expired; task recovered")
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
recovered_task.failure_class.as_deref(),
|
||||||
|
Some("lease_expired")
|
||||||
|
);
|
||||||
|
assert!(recovered_task.failure_retryable);
|
||||||
|
assert_eq!(recovered_task.lease_owner, None);
|
||||||
|
|
||||||
|
let reclaimed = repository
|
||||||
|
.claim_next("worker-b", "mock", 30, 3)
|
||||||
|
.await
|
||||||
|
.unwrap()
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(reclaimed.task_status, TranslationTaskStatus::Running);
|
||||||
|
assert_eq!(reclaimed.attempt_count, 2);
|
||||||
|
assert_eq!(reclaimed.lease_owner.as_deref(), Some("worker-b"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn sqlite_translation_tasks_reject_stale_worker_writes() {
|
||||||
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
let repository =
|
||||||
|
SqliteTranslationTaskRepository::new(temp.path().join("translation-tasks.sqlite"))
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
let queue = queue(vec![task(
|
||||||
|
"task-a",
|
||||||
|
"Bundles/a.bundle",
|
||||||
|
OfficialTextUnitTaskStatus::QueuedOffline,
|
||||||
|
Some(OfficialParseStatus::Parsed),
|
||||||
|
None,
|
||||||
|
)]);
|
||||||
|
repository.sync_queue(&queue).await.unwrap();
|
||||||
|
|
||||||
|
let first_claim = repository
|
||||||
|
.claim_next("worker-a", "mock", 1, 3)
|
||||||
|
.await
|
||||||
|
.unwrap()
|
||||||
|
.unwrap();
|
||||||
|
let first_run = first_claim.provider_run_id.clone().unwrap();
|
||||||
|
force_expire_lease(&repository, "task-a").await;
|
||||||
|
repository
|
||||||
|
.recover_expired_leases(unix_seconds_now())
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
let second_claim = repository
|
||||||
|
.claim_next("worker-b", "mock", 30, 3)
|
||||||
|
.await
|
||||||
|
.unwrap()
|
||||||
|
.unwrap();
|
||||||
|
let second_run = second_claim.provider_run_id.clone().unwrap();
|
||||||
|
let result = TranslationTaskUnitResult {
|
||||||
|
unit_id: "unit-a".to_string(),
|
||||||
|
source_text: "source".to_string(),
|
||||||
|
translated_text: "translated".to_string(),
|
||||||
|
provider: "mock".to_string(),
|
||||||
|
provider_run_id: second_run.clone(),
|
||||||
|
translated_unix_seconds: 1,
|
||||||
|
};
|
||||||
|
|
||||||
|
assert!(repository
|
||||||
|
.complete_claim(
|
||||||
|
"task-a",
|
||||||
|
"worker-a",
|
||||||
|
&first_run,
|
||||||
|
"mock",
|
||||||
|
std::slice::from_ref(&result)
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.is_err());
|
||||||
|
let completed = repository
|
||||||
|
.complete_claim(
|
||||||
|
"task-a",
|
||||||
|
"worker-b",
|
||||||
|
&second_run,
|
||||||
|
"mock",
|
||||||
|
std::slice::from_ref(&result),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(completed.task_status, TranslationTaskStatus::Completed);
|
||||||
|
assert_eq!(completed.translation_results, vec![result.clone()]);
|
||||||
|
assert!(repository
|
||||||
|
.fail_claim(TranslationTaskFailure {
|
||||||
|
task_id: "task-a".to_string(),
|
||||||
|
worker_id: "worker-a".to_string(),
|
||||||
|
provider_run_id: first_run,
|
||||||
|
failure_class: "network".to_string(),
|
||||||
|
failure_reason: "late failure".to_string(),
|
||||||
|
retryable: true,
|
||||||
|
next_attempt_unix_seconds: Some(2),
|
||||||
|
})
|
||||||
|
.await
|
||||||
|
.is_err());
|
||||||
|
let task = repository.find("task-a").await.unwrap();
|
||||||
|
assert_eq!(task.task_status, TranslationTaskStatus::Completed);
|
||||||
|
assert_eq!(task.translation_results, vec![result]);
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn translation_handoff_maps_worker_state_and_preserves_provider_progress() {
|
fn translation_handoff_maps_worker_state_and_preserves_provider_progress() {
|
||||||
let queue = queue(vec![
|
let queue = queue(vec![
|
||||||
@@ -1254,6 +1865,13 @@ mod tests {
|
|||||||
updated_unix_seconds: 124,
|
updated_unix_seconds: 124,
|
||||||
completed_unix_seconds: None,
|
completed_unix_seconds: None,
|
||||||
provider_run_id: Some("provider-run-1".to_string()),
|
provider_run_id: Some("provider-run-1".to_string()),
|
||||||
|
translation_results: Vec::new(),
|
||||||
|
provider: Some("fixture".to_string()),
|
||||||
|
lease_owner: None,
|
||||||
|
lease_expires_unix_seconds: None,
|
||||||
|
failure_class: None,
|
||||||
|
failure_retryable: false,
|
||||||
|
next_attempt_unix_seconds: None,
|
||||||
},
|
},
|
||||||
PersistedTranslationTask {
|
PersistedTranslationTask {
|
||||||
task: queue.tasks[1].clone(),
|
task: queue.tasks[1].clone(),
|
||||||
@@ -1264,6 +1882,13 @@ mod tests {
|
|||||||
updated_unix_seconds: 125,
|
updated_unix_seconds: 125,
|
||||||
completed_unix_seconds: Some(125),
|
completed_unix_seconds: Some(125),
|
||||||
provider_run_id: Some("provider-run-1".to_string()),
|
provider_run_id: Some("provider-run-1".to_string()),
|
||||||
|
translation_results: Vec::new(),
|
||||||
|
provider: Some("fixture".to_string()),
|
||||||
|
lease_owner: None,
|
||||||
|
lease_expires_unix_seconds: None,
|
||||||
|
failure_class: None,
|
||||||
|
failure_retryable: false,
|
||||||
|
next_attempt_unix_seconds: None,
|
||||||
},
|
},
|
||||||
];
|
];
|
||||||
|
|
||||||
@@ -1286,6 +1911,20 @@ mod tests {
|
|||||||
assert_eq!(handoff.provider_runs[0].attempt_count, 2);
|
assert_eq!(handoff.provider_runs[0].attempt_count, 2);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async fn force_expire_lease(repository: &SqliteTranslationTaskRepository, task_id: &str) {
|
||||||
|
sqlx::query(
|
||||||
|
r#"
|
||||||
|
UPDATE translation_tasks
|
||||||
|
SET lease_expires_unix_seconds = 0
|
||||||
|
WHERE task_id = ?1
|
||||||
|
"#,
|
||||||
|
)
|
||||||
|
.bind(task_id)
|
||||||
|
.execute(&repository.pool)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn translation_handoff_file_round_trips_with_version_check() {
|
fn translation_handoff_file_round_trips_with_version_check() {
|
||||||
let temp = tempfile::TempDir::new().unwrap();
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
+167
-1
@@ -6,6 +6,7 @@ import (
|
|||||||
"errors"
|
"errors"
|
||||||
"io"
|
"io"
|
||||||
"net/http"
|
"net/http"
|
||||||
|
"net/url"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
@@ -36,6 +37,8 @@ func (s *Server) handleAdminIndex(w http.ResponseWriter, r *http.Request) {
|
|||||||
"/v1/resources",
|
"/v1/resources",
|
||||||
"/openapi.yaml",
|
"/openapi.yaml",
|
||||||
"/admin/schedules",
|
"/admin/schedules",
|
||||||
|
"/admin/translation/tasks",
|
||||||
|
"/admin/translation/handoff",
|
||||||
},
|
},
|
||||||
Controls: []string{
|
Controls: []string{
|
||||||
"/admin/control/reload",
|
"/admin/control/reload",
|
||||||
@@ -50,6 +53,7 @@ func (s *Server) handleAdminIndex(w http.ResponseWriter, r *http.Request) {
|
|||||||
"/admin/control/schedule-remove",
|
"/admin/control/schedule-remove",
|
||||||
"/admin/control/schedule-run",
|
"/admin/control/schedule-run",
|
||||||
"/admin/control/translation-task-update",
|
"/admin/control/translation-task-update",
|
||||||
|
"/admin/control/translation-worker-run",
|
||||||
"/admin/control/translation-proofread",
|
"/admin/control/translation-proofread",
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
@@ -82,6 +86,10 @@ func (s *Server) handleAdminControl(w http.ResponseWriter, r *http.Request) {
|
|||||||
s.handleAdminTranslationTaskUpdate(w, r)
|
s.handleAdminTranslationTaskUpdate(w, r)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
if action == "translation-worker-run" {
|
||||||
|
s.handleAdminTranslationWorkerRun(w, r)
|
||||||
|
return
|
||||||
|
}
|
||||||
if action == "translation-proofread" {
|
if action == "translation-proofread" {
|
||||||
s.handleAdminTranslationProofread(w, r)
|
s.handleAdminTranslationProofread(w, r)
|
||||||
return
|
return
|
||||||
@@ -187,6 +195,34 @@ func (s *Server) handleAdminTranslationTaskUpdate(w http.ResponseWriter, r *http
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleAdminTranslationWorkerRun(w http.ResponseWriter, r *http.Request) {
|
||||||
|
backend, ok := s.backend.(TranslationBackend)
|
||||||
|
if !ok || backend == nil {
|
||||||
|
writeErrorJSON(w, http.StatusServiceUnavailable, "translation_backend_unavailable", "Rust bat translation backend is unavailable")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
var params backendrpc.TranslationWorkerRunParams
|
||||||
|
if !decodeAdminTranslationJSON(w, r, ¶ms) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := validateTranslationWorkerRunParams(params); err != nil {
|
||||||
|
writeErrorJSON(w, http.StatusBadRequest, "invalid_translation_params", err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
result, err := backend.TranslationWorkerRun(r.Context(), params)
|
||||||
|
if err != nil {
|
||||||
|
s.writeControlBackendError(w, "translation-worker-run", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
writeNoStoreJSON(w, http.StatusAccepted, AdminControlResponse{
|
||||||
|
Service: "bat-api",
|
||||||
|
Action: "translation-worker-run",
|
||||||
|
RPCMethod: "translation.worker.run",
|
||||||
|
Status: "accepted",
|
||||||
|
Result: result,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
func (s *Server) handleAdminTranslationProofread(w http.ResponseWriter, r *http.Request) {
|
func (s *Server) handleAdminTranslationProofread(w http.ResponseWriter, r *http.Request) {
|
||||||
backend, ok := s.backend.(TranslationBackend)
|
backend, ok := s.backend.(TranslationBackend)
|
||||||
if !ok || backend == nil {
|
if !ok || backend == nil {
|
||||||
@@ -207,6 +243,63 @@ func (s *Server) handleAdminTranslationProofread(w http.ResponseWriter, r *http.
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleAdminTranslationTasks(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != http.MethodGet && r.Method != http.MethodHead {
|
||||||
|
writeErrorJSON(w, http.StatusMethodNotAllowed, "method_not_allowed", "method not allowed")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !s.requireAdminToken(w, r) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
backend, ok := s.backend.(TranslationBackend)
|
||||||
|
if !ok || backend == nil {
|
||||||
|
writeErrorJSON(w, http.StatusServiceUnavailable, "translation_backend_unavailable", "Rust bat translation backend is unavailable")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
params, err := translationTaskListParams(r)
|
||||||
|
if err != nil {
|
||||||
|
writeErrorJSON(w, http.StatusBadRequest, "invalid_translation_query", err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
result, err := backend.TranslationTasks(r.Context(), params)
|
||||||
|
if err != nil {
|
||||||
|
s.writeControlBackendError(w, "translation-tasks", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if r.Method == http.MethodHead {
|
||||||
|
w.Header().Set("Cache-Control", "no-store")
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
writeNoStoreJSON(w, http.StatusOK, result)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleAdminTranslationHandoff(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != http.MethodGet && r.Method != http.MethodHead {
|
||||||
|
writeErrorJSON(w, http.StatusMethodNotAllowed, "method_not_allowed", "method not allowed")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !s.requireAdminToken(w, r) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
backend, ok := s.backend.(TranslationBackend)
|
||||||
|
if !ok || backend == nil {
|
||||||
|
writeErrorJSON(w, http.StatusServiceUnavailable, "translation_backend_unavailable", "Rust bat translation backend is unavailable")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
result, err := backend.TranslationHandoff(r.Context())
|
||||||
|
if err != nil {
|
||||||
|
s.writeControlBackendError(w, "translation-handoff", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if r.Method == http.MethodHead {
|
||||||
|
w.Header().Set("Cache-Control", "no-store")
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
writeNoStoreJSON(w, http.StatusOK, result)
|
||||||
|
}
|
||||||
|
|
||||||
func (s *Server) handleAdminSchedules(w http.ResponseWriter, r *http.Request) {
|
func (s *Server) handleAdminSchedules(w http.ResponseWriter, r *http.Request) {
|
||||||
if r.Method != http.MethodGet && r.Method != http.MethodHead {
|
if r.Method != http.MethodGet && r.Method != http.MethodHead {
|
||||||
writeErrorJSON(w, http.StatusMethodNotAllowed, "method_not_allowed", "method not allowed")
|
writeErrorJSON(w, http.StatusMethodNotAllowed, "method_not_allowed", "method not allowed")
|
||||||
@@ -254,6 +347,63 @@ func scheduleListParams(r *http.Request) (backendrpc.ScheduleListParams, error)
|
|||||||
return params, nil
|
return params, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func translationTaskListParams(r *http.Request) (backendrpc.TranslationTaskListParams, error) {
|
||||||
|
query := r.URL.Query()
|
||||||
|
params := backendrpc.TranslationTaskListParams{
|
||||||
|
TaskID: strings.TrimSpace(query.Get("task_id")),
|
||||||
|
ReleaseID: firstTrimmedQuery(query, "release_id", "official_release_id"),
|
||||||
|
Destination: strings.TrimSpace(query.Get("destination")),
|
||||||
|
PathPattern: strings.TrimSpace(query.Get("path_pattern")),
|
||||||
|
ArchiveEntry: strings.TrimSpace(query.Get("archive_entry")),
|
||||||
|
Status: firstTrimmedQuery(query, "status", "task_status"),
|
||||||
|
WorkerStatus: strings.TrimSpace(query.Get("worker_status")),
|
||||||
|
ParseStatus: strings.TrimSpace(query.Get("parse_status")),
|
||||||
|
Format: firstTrimmedQuery(query, "format", "text_unit_format"),
|
||||||
|
}
|
||||||
|
if raw := strings.TrimSpace(query.Get("offset")); raw != "" {
|
||||||
|
offset, err := strconv.ParseUint(raw, 10, 64)
|
||||||
|
if err != nil {
|
||||||
|
return backendrpc.TranslationTaskListParams{}, errors.New("offset must be a non-negative integer")
|
||||||
|
}
|
||||||
|
params.Offset = &offset
|
||||||
|
}
|
||||||
|
if raw := strings.TrimSpace(query.Get("limit")); raw != "" {
|
||||||
|
limit, err := strconv.ParseUint(raw, 10, 64)
|
||||||
|
if err != nil || limit == 0 || limit > 1000 {
|
||||||
|
return backendrpc.TranslationTaskListParams{}, errors.New("limit must be in 1..=1000")
|
||||||
|
}
|
||||||
|
params.Limit = &limit
|
||||||
|
}
|
||||||
|
if raw := strings.TrimSpace(query.Get("has_reason")); raw != "" {
|
||||||
|
hasReason, err := strconv.ParseBool(raw)
|
||||||
|
if err != nil {
|
||||||
|
return backendrpc.TranslationTaskListParams{}, errors.New("has_reason must be a boolean")
|
||||||
|
}
|
||||||
|
params.HasReason = &hasReason
|
||||||
|
}
|
||||||
|
if raw := strings.TrimSpace(query.Get("has_failure_reason")); raw != "" {
|
||||||
|
hasFailureReason, err := strconv.ParseBool(raw)
|
||||||
|
if err != nil {
|
||||||
|
return backendrpc.TranslationTaskListParams{}, errors.New("has_failure_reason must be a boolean")
|
||||||
|
}
|
||||||
|
params.HasFailureReason = &hasFailureReason
|
||||||
|
}
|
||||||
|
return params, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func firstTrimmedQuery(query url.Values, keys ...string) string {
|
||||||
|
for _, key := range keys {
|
||||||
|
values := query[key]
|
||||||
|
if len(values) == 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if value := strings.TrimSpace(values[0]); value != "" {
|
||||||
|
return value
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
func (s *Server) handleAdminScheduleControl(w http.ResponseWriter, r *http.Request, action string) {
|
func (s *Server) handleAdminScheduleControl(w http.ResponseWriter, r *http.Request, action string) {
|
||||||
backend, ok := s.backend.(ScheduleBackend)
|
backend, ok := s.backend.(ScheduleBackend)
|
||||||
if !ok || backend == nil {
|
if !ok || backend == nil {
|
||||||
@@ -325,7 +475,23 @@ func decodeAdminScheduleJSON(w http.ResponseWriter, r *http.Request, target any)
|
|||||||
}
|
}
|
||||||
|
|
||||||
func decodeAdminTranslationJSON(w http.ResponseWriter, r *http.Request, target any) bool {
|
func decodeAdminTranslationJSON(w http.ResponseWriter, r *http.Request, target any) bool {
|
||||||
return decodeAdminJSON(w, r, target, "invalid_translation_params", "translation task request")
|
return decodeAdminJSON(w, r, target, "invalid_translation_params", "translation request")
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateTranslationWorkerRunParams(params backendrpc.TranslationWorkerRunParams) error {
|
||||||
|
if params.Concurrency != nil && (*params.Concurrency < 1 || *params.Concurrency > 256) {
|
||||||
|
return errors.New("translation worker concurrency must be in 1..=256")
|
||||||
|
}
|
||||||
|
if params.MaxAttempts != nil && *params.MaxAttempts == 0 {
|
||||||
|
return errors.New("translation worker max_attempts must be greater than 0")
|
||||||
|
}
|
||||||
|
if params.LeaseSeconds != nil && *params.LeaseSeconds == 0 {
|
||||||
|
return errors.New("translation worker lease_seconds must be greater than 0")
|
||||||
|
}
|
||||||
|
if params.MaxTasks != nil && *params.MaxTasks == 0 {
|
||||||
|
return errors.New("translation worker max_tasks must be greater than 0")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func decodeAdminJSON(w http.ResponseWriter, r *http.Request, target any, errorCode string, subject string) bool {
|
func decodeAdminJSON(w http.ResponseWriter, r *http.Request, target any, errorCode string, subject string) bool {
|
||||||
|
|||||||
@@ -589,6 +589,7 @@ func (f *fakeBackend) ResourceManifest(ctx context.Context, offset int, limit in
|
|||||||
type controlBackend struct {
|
type controlBackend struct {
|
||||||
*fakeBackend
|
*fakeBackend
|
||||||
calls []string
|
calls []string
|
||||||
|
translationTaskListParams []backendrpc.TranslationTaskListParams
|
||||||
}
|
}
|
||||||
|
|
||||||
func (b *controlBackend) DaemonReload(ctx context.Context) (*backendrpc.Ack, error) {
|
func (b *controlBackend) DaemonReload(ctx context.Context) (*backendrpc.Ack, error) {
|
||||||
@@ -631,6 +632,38 @@ func (b *controlBackend) TranslationTaskUpdate(ctx context.Context, params backe
|
|||||||
return json.RawMessage(`{"task_status":"` + params.Status + `"}`), nil
|
return json.RawMessage(`{"task_status":"` + params.Status + `"}`), nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (b *controlBackend) TranslationTasks(ctx context.Context, params backendrpc.TranslationTaskListParams) (json.RawMessage, error) {
|
||||||
|
b.calls = append(b.calls, "translation.tasks")
|
||||||
|
b.translationTaskListParams = append(b.translationTaskListParams, params)
|
||||||
|
return json.RawMessage(`{"tasks":[{"task_id":"textunit/v-current/Scenario","worker_status":"failed","failure_reason":"provider rejected payload"}]}`), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (b *controlBackend) TranslationHandoff(ctx context.Context) (json.RawMessage, error) {
|
||||||
|
b.calls = append(b.calls, "translation.handoff")
|
||||||
|
return json.RawMessage(`{"jobs":[],"provider_runs":[]}`), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (b *controlBackend) TranslationWorkerRun(ctx context.Context, params backendrpc.TranslationWorkerRunParams) (*backendrpc.TranslationWorkerRunResult, error) {
|
||||||
|
b.calls = append(b.calls, "translation.worker.run")
|
||||||
|
concurrency := uint64(8)
|
||||||
|
if params.Concurrency != nil {
|
||||||
|
concurrency = *params.Concurrency
|
||||||
|
}
|
||||||
|
return &backendrpc.TranslationWorkerRunResult{
|
||||||
|
TaskID: "task-translation-worker-1",
|
||||||
|
Kind: "translation.worker.run",
|
||||||
|
Worker: backendrpc.TranslationWorkerConfig{
|
||||||
|
Provider: params.Provider,
|
||||||
|
FixturePath: params.FixturePath,
|
||||||
|
Concurrency: concurrency,
|
||||||
|
MaxAttempts: 3,
|
||||||
|
LeaseSeconds: 300,
|
||||||
|
RetryBackoffSeconds: 5,
|
||||||
|
WorkerID: params.WorkerID,
|
||||||
|
},
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
func (b *controlBackend) TranslationProofread(ctx context.Context) (json.RawMessage, error) {
|
func (b *controlBackend) TranslationProofread(ctx context.Context) (json.RawMessage, error) {
|
||||||
b.calls = append(b.calls, "translation.proofread")
|
b.calls = append(b.calls, "translation.proofread")
|
||||||
return json.RawMessage(`{"translation_workflow_status":"manual_proofreading"}`), nil
|
return json.RawMessage(`{"translation_workflow_status":"manual_proofreading"}`), nil
|
||||||
@@ -743,6 +776,81 @@ func TestAdminScheduleEndpointsProxyAuthenticatedRequests(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestAdminTranslationQueryEndpointsProxyAuthenticatedRequests(t *testing.T) {
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
cfg.AuthToken = "translation-token"
|
||||||
|
if err := cfg.Normalize(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
backend := &controlBackend{fakeBackend: &fakeBackend{}}
|
||||||
|
s := NewServer(cfg, backend, nil)
|
||||||
|
|
||||||
|
request := httptest.NewRequest(
|
||||||
|
http.MethodGet,
|
||||||
|
"/admin/translation/tasks?offset=10&limit=25&task_id=textunit%2Fv-current%2FScenario&release_id=v-current&destination=MediaResources%2FGameData%2FScenario.zip&path_pattern=Scenario&archive_entry=ScenarioExcelTable.json&status=queued_offline&worker_status=failed&parse_status=ok&format=json&has_reason=true&has_failure_reason=false",
|
||||||
|
nil,
|
||||||
|
)
|
||||||
|
request.Header.Set("Authorization", "Bearer translation-token")
|
||||||
|
recorder := httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(recorder, request)
|
||||||
|
if recorder.Code != http.StatusOK {
|
||||||
|
t.Fatalf("tasks status=%d body=%s", recorder.Code, recorder.Body.String())
|
||||||
|
}
|
||||||
|
if !json.Valid(recorder.Body.Bytes()) || !strings.Contains(recorder.Body.String(), "failure_reason") {
|
||||||
|
t.Fatalf("tasks body=%s", recorder.Body.String())
|
||||||
|
}
|
||||||
|
if len(backend.translationTaskListParams) != 1 {
|
||||||
|
t.Fatalf("translation task query params=%#v", backend.translationTaskListParams)
|
||||||
|
}
|
||||||
|
params := backend.translationTaskListParams[0]
|
||||||
|
if params.Offset == nil || *params.Offset != 10 ||
|
||||||
|
params.Limit == nil || *params.Limit != 25 ||
|
||||||
|
params.TaskID != "textunit/v-current/Scenario" ||
|
||||||
|
params.ReleaseID != "v-current" ||
|
||||||
|
params.Destination != "MediaResources/GameData/Scenario.zip" ||
|
||||||
|
params.PathPattern != "Scenario" ||
|
||||||
|
params.ArchiveEntry != "ScenarioExcelTable.json" ||
|
||||||
|
params.Status != "queued_offline" ||
|
||||||
|
params.WorkerStatus != "failed" ||
|
||||||
|
params.ParseStatus != "ok" ||
|
||||||
|
params.Format != "json" ||
|
||||||
|
params.HasReason == nil || !*params.HasReason ||
|
||||||
|
params.HasFailureReason == nil || *params.HasFailureReason {
|
||||||
|
t.Fatalf("params=%#v", params)
|
||||||
|
}
|
||||||
|
|
||||||
|
request = httptest.NewRequest(http.MethodGet, "/admin/translation/handoff", nil)
|
||||||
|
request.Header.Set("Authorization", "Bearer translation-token")
|
||||||
|
recorder = httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(recorder, request)
|
||||||
|
if recorder.Code != http.StatusOK {
|
||||||
|
t.Fatalf("handoff status=%d body=%s", recorder.Code, recorder.Body.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(recorder.Body.String(), "provider_runs") {
|
||||||
|
t.Fatalf("handoff body=%s", recorder.Body.String())
|
||||||
|
}
|
||||||
|
if len(backend.calls) < 2 ||
|
||||||
|
backend.calls[len(backend.calls)-2] != "translation.tasks" ||
|
||||||
|
backend.calls[len(backend.calls)-1] != "translation.handoff" {
|
||||||
|
t.Fatalf("calls=%v", backend.calls)
|
||||||
|
}
|
||||||
|
|
||||||
|
request = httptest.NewRequest(http.MethodGet, "/admin/translation/tasks?limit=0", nil)
|
||||||
|
request.Header.Set("Authorization", "Bearer translation-token")
|
||||||
|
recorder = httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(recorder, request)
|
||||||
|
if recorder.Code != http.StatusBadRequest {
|
||||||
|
t.Fatalf("invalid task query status=%d body=%s", recorder.Code, recorder.Body.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
request = httptest.NewRequest(http.MethodGet, "/admin/translation/tasks", nil)
|
||||||
|
recorder = httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(recorder, request)
|
||||||
|
if recorder.Code != http.StatusUnauthorized {
|
||||||
|
t.Fatalf("unauthenticated tasks status=%d body=%s", recorder.Code, recorder.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestDiscoverCallsStatusBeforeDoctor(t *testing.T) {
|
func TestDiscoverCallsStatusBeforeDoctor(t *testing.T) {
|
||||||
root := fixtureRoot(t)
|
root := fixtureRoot(t)
|
||||||
bytes := uint64(20)
|
bytes := uint64(20)
|
||||||
@@ -1281,6 +1389,11 @@ func TestOpenAPIAndAdminReservedEndpoints(t *testing.T) {
|
|||||||
if len(admin.Controls) == 0 || admin.Controls[0] != "/admin/control/reload" {
|
if len(admin.Controls) == 0 || admin.Controls[0] != "/admin/control/reload" {
|
||||||
t.Fatalf("admin controls=%v", admin.Controls)
|
t.Fatalf("admin controls=%v", admin.Controls)
|
||||||
}
|
}
|
||||||
|
links := strings.Join(admin.Links, "\n")
|
||||||
|
if !strings.Contains(links, "/admin/translation/tasks") ||
|
||||||
|
!strings.Contains(links, "/admin/translation/handoff") {
|
||||||
|
t.Fatalf("admin links=%v", admin.Links)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestAdminControlForwardsAllowlistedActions(t *testing.T) {
|
func TestAdminControlForwardsAllowlistedActions(t *testing.T) {
|
||||||
@@ -1305,6 +1418,7 @@ func TestAdminControlForwardsAllowlistedActions(t *testing.T) {
|
|||||||
{name: "repair", action: "repair", rpcMethod: "resource.repair", call: "resource.repair"},
|
{name: "repair", action: "repair", rpcMethod: "resource.repair", call: "resource.repair"},
|
||||||
{name: "catalog refresh", action: "catalog-refresh", rpcMethod: "catalog.refresh", call: "catalog.refresh"},
|
{name: "catalog refresh", action: "catalog-refresh", rpcMethod: "catalog.refresh", call: "catalog.refresh"},
|
||||||
{name: "translation task update", action: "translation-task-update", body: `{"task_id":"textunit/v-current/Scenario","status":"failed","failure_reason":"provider rejected payload","provider_run_id":"provider-run-1"}`, rpcMethod: "translation.task.update", call: "translation.task.update"},
|
{name: "translation task update", action: "translation-task-update", body: `{"task_id":"textunit/v-current/Scenario","status":"failed","failure_reason":"provider rejected payload","provider_run_id":"provider-run-1"}`, rpcMethod: "translation.task.update", call: "translation.task.update"},
|
||||||
|
{name: "translation worker run", action: "translation-worker-run", body: `{"provider":"mock","concurrency":8,"max_tasks":2,"retry_backoff_seconds":0,"worker_id":"dashboard-worker"}`, rpcMethod: "translation.worker.run", call: "translation.worker.run"},
|
||||||
{name: "translation proofread", action: "translation-proofread", rpcMethod: "translation.proofread", call: "translation.proofread"},
|
{name: "translation proofread", action: "translation-proofread", rpcMethod: "translation.proofread", call: "translation.proofread"},
|
||||||
}
|
}
|
||||||
for _, tc := range tests {
|
for _, tc := range tests {
|
||||||
@@ -1336,6 +1450,14 @@ func TestAdminControlForwardsAllowlistedActions(t *testing.T) {
|
|||||||
if recorder.Code != http.StatusBadRequest {
|
if recorder.Code != http.StatusBadRequest {
|
||||||
t.Fatalf("invalid translation update status=%d body=%s", recorder.Code, recorder.Body.String())
|
t.Fatalf("invalid translation update status=%d body=%s", recorder.Code, recorder.Body.String())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
request = httptest.NewRequest(http.MethodPost, "/admin/control/translation-worker-run", strings.NewReader(`{"concurrency":0}`))
|
||||||
|
request.Header.Set("Authorization", "Bearer control-token")
|
||||||
|
recorder = httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(recorder, request)
|
||||||
|
if recorder.Code != http.StatusBadRequest {
|
||||||
|
t.Fatalf("invalid translation worker status=%d body=%s", recorder.Code, recorder.Body.String())
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestAdminControlRejectsUnauthenticatedDangerousAndUnsupportedActions(t *testing.T) {
|
func TestAdminControlRejectsUnauthenticatedDangerousAndUnsupportedActions(t *testing.T) {
|
||||||
|
|||||||
+103
-1
@@ -139,6 +139,80 @@ paths:
|
|||||||
description: Missing or invalid admin token.
|
description: Missing or invalid admin token.
|
||||||
"503":
|
"503":
|
||||||
description: Rust bat schedule backend is unavailable.
|
description: Rust bat schedule backend is unavailable.
|
||||||
|
/admin/translation/tasks:
|
||||||
|
get:
|
||||||
|
summary: List Rust-owned translation task worker status
|
||||||
|
parameters:
|
||||||
|
- name: offset
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: integer
|
||||||
|
minimum: 0
|
||||||
|
- name: limit
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: integer
|
||||||
|
minimum: 1
|
||||||
|
maximum: 1000
|
||||||
|
- name: task_id
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
- name: release_id
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
- name: destination
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
- name: archive_entry
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
- name: status
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
- name: worker_status
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
- name: parse_status
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
- name: format
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
- name: has_reason
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: boolean
|
||||||
|
- name: has_failure_reason
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: boolean
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Current translation task JSON report from Rust bat.
|
||||||
|
"400":
|
||||||
|
description: Invalid translation task query.
|
||||||
|
"401":
|
||||||
|
description: Missing or invalid admin token.
|
||||||
|
"503":
|
||||||
|
description: Rust bat translation backend is unavailable.
|
||||||
|
/admin/translation/handoff:
|
||||||
|
get:
|
||||||
|
summary: Read Rust-owned translation handoff state
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Current translation handoff JSON report from Rust bat.
|
||||||
|
"401":
|
||||||
|
description: Missing or invalid admin token.
|
||||||
|
"503":
|
||||||
|
description: Rust bat translation backend is unavailable.
|
||||||
/admin/control/{action}:
|
/admin/control/{action}:
|
||||||
post:
|
post:
|
||||||
summary: Forward an allowlisted control or schedule action to Rust bat
|
summary: Forward an allowlisted control or schedule action to Rust bat
|
||||||
@@ -148,7 +222,7 @@ paths:
|
|||||||
required: true
|
required: true
|
||||||
schema:
|
schema:
|
||||||
type: string
|
type: string
|
||||||
enum: [reload, refresh, restart, sync, verify, repair, catalog-refresh, schedule-add, schedule-update, schedule-remove, schedule-run, translation-task-update, translation-proofread]
|
enum: [reload, refresh, restart, sync, verify, repair, catalog-refresh, schedule-add, schedule-update, schedule-remove, schedule-run, translation-task-update, translation-worker-run, translation-proofread]
|
||||||
requestBody:
|
requestBody:
|
||||||
required: false
|
required: false
|
||||||
content:
|
content:
|
||||||
@@ -199,6 +273,34 @@ paths:
|
|||||||
type: string
|
type: string
|
||||||
provider_run_id:
|
provider_run_id:
|
||||||
type: string
|
type: string
|
||||||
|
provider:
|
||||||
|
type: string
|
||||||
|
enum: [mock, crowdin]
|
||||||
|
fixture_path:
|
||||||
|
type: string
|
||||||
|
concurrency:
|
||||||
|
type: integer
|
||||||
|
format: int64
|
||||||
|
minimum: 1
|
||||||
|
maximum: 256
|
||||||
|
max_attempts:
|
||||||
|
type: integer
|
||||||
|
format: int64
|
||||||
|
minimum: 1
|
||||||
|
lease_seconds:
|
||||||
|
type: integer
|
||||||
|
format: int64
|
||||||
|
minimum: 1
|
||||||
|
retry_backoff_seconds:
|
||||||
|
type: integer
|
||||||
|
format: int64
|
||||||
|
minimum: 0
|
||||||
|
max_tasks:
|
||||||
|
type: integer
|
||||||
|
format: int64
|
||||||
|
minimum: 1
|
||||||
|
worker_id:
|
||||||
|
type: string
|
||||||
responses:
|
responses:
|
||||||
"202":
|
"202":
|
||||||
description: Rust bat accepted the control request.
|
description: Rust bat accepted the control request.
|
||||||
|
|||||||
@@ -54,6 +54,9 @@ type ScheduleBackend interface {
|
|||||||
// by the dashboard. It does not create arbitrary translation jobs.
|
// by the dashboard. It does not create arbitrary translation jobs.
|
||||||
type TranslationBackend interface {
|
type TranslationBackend interface {
|
||||||
TranslationTaskUpdate(ctx context.Context, params backendrpc.TranslationTaskUpdateParams) (json.RawMessage, error)
|
TranslationTaskUpdate(ctx context.Context, params backendrpc.TranslationTaskUpdateParams) (json.RawMessage, error)
|
||||||
|
TranslationTasks(ctx context.Context, params backendrpc.TranslationTaskListParams) (json.RawMessage, error)
|
||||||
|
TranslationHandoff(ctx context.Context) (json.RawMessage, error)
|
||||||
|
TranslationWorkerRun(ctx context.Context, params backendrpc.TranslationWorkerRunParams) (*backendrpc.TranslationWorkerRunResult, error)
|
||||||
TranslationProofread(ctx context.Context) (json.RawMessage, error)
|
TranslationProofread(ctx context.Context) (json.RawMessage, error)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -117,6 +120,18 @@ func (r RPCClient) TranslationTaskUpdate(ctx context.Context, params backendrpc.
|
|||||||
return r.Client.TranslationTaskUpdate(ctx, params)
|
return r.Client.TranslationTaskUpdate(ctx, params)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (r RPCClient) TranslationTasks(ctx context.Context, params backendrpc.TranslationTaskListParams) (json.RawMessage, error) {
|
||||||
|
return r.Client.TranslationTasks(ctx, params)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r RPCClient) TranslationHandoff(ctx context.Context) (json.RawMessage, error) {
|
||||||
|
return r.Client.TranslationHandoff(ctx)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r RPCClient) TranslationWorkerRun(ctx context.Context, params backendrpc.TranslationWorkerRunParams) (*backendrpc.TranslationWorkerRunResult, error) {
|
||||||
|
return r.Client.TranslationWorkerRun(ctx, params)
|
||||||
|
}
|
||||||
|
|
||||||
func (r RPCClient) TranslationProofread(ctx context.Context) (json.RawMessage, error) {
|
func (r RPCClient) TranslationProofread(ctx context.Context) (json.RawMessage, error) {
|
||||||
return r.Client.TranslationProofread(ctx)
|
return r.Client.TranslationProofread(ctx)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -62,6 +62,8 @@ func (s *Server) Handler() http.Handler {
|
|||||||
mux.HandleFunc(launcherHostPath("/api/launcher/resource/bootstrap.json"), s.handleLauncherBootstrap)
|
mux.HandleFunc(launcherHostPath("/api/launcher/resource/bootstrap.json"), s.handleLauncherBootstrap)
|
||||||
mux.HandleFunc("/openapi.yaml", s.handleOpenAPI)
|
mux.HandleFunc("/openapi.yaml", s.handleOpenAPI)
|
||||||
mux.HandleFunc("/admin/schedules", s.handleAdminSchedules)
|
mux.HandleFunc("/admin/schedules", s.handleAdminSchedules)
|
||||||
|
mux.HandleFunc("/admin/translation/tasks", s.handleAdminTranslationTasks)
|
||||||
|
mux.HandleFunc("/admin/translation/handoff", s.handleAdminTranslationHandoff)
|
||||||
mux.HandleFunc("/admin/control/", s.handleAdminControl)
|
mux.HandleFunc("/admin/control/", s.handleAdminControl)
|
||||||
mux.HandleFunc("/admin/", s.handleAdminIndex)
|
mux.HandleFunc("/admin/", s.handleAdminIndex)
|
||||||
mux.HandleFunc("/"+ServerInfoHost+"/", s.handleServerInfoCDN)
|
mux.HandleFunc("/"+ServerInfoHost+"/", s.handleServerInfoCDN)
|
||||||
@@ -127,6 +129,8 @@ func (s *Server) handleRoot(w http.ResponseWriter, r *http.Request) {
|
|||||||
"/openapi.yaml",
|
"/openapi.yaml",
|
||||||
"/admin/",
|
"/admin/",
|
||||||
"/admin/schedules",
|
"/admin/schedules",
|
||||||
|
"/admin/translation/tasks",
|
||||||
|
"/admin/translation/handoff",
|
||||||
"/admin/control/{action}",
|
"/admin/control/{action}",
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -273,6 +273,58 @@ type TranslationTaskUpdateParams struct {
|
|||||||
ProviderRunID string `json:"provider_run_id,omitempty"`
|
ProviderRunID string `json:"provider_run_id,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TranslationTaskListParams filters the Rust-owned translation task queue.
|
||||||
|
// The result shape is intentionally left as Rust JSON so Go does not duplicate
|
||||||
|
// the translation task schema.
|
||||||
|
type TranslationTaskListParams struct {
|
||||||
|
Offset *uint64 `json:"offset,omitempty"`
|
||||||
|
Limit *uint64 `json:"limit,omitempty"`
|
||||||
|
TaskID string `json:"task_id,omitempty"`
|
||||||
|
ReleaseID string `json:"release_id,omitempty"`
|
||||||
|
Destination string `json:"destination,omitempty"`
|
||||||
|
PathPattern string `json:"path_pattern,omitempty"`
|
||||||
|
ArchiveEntry string `json:"archive_entry,omitempty"`
|
||||||
|
Status string `json:"status,omitempty"`
|
||||||
|
WorkerStatus string `json:"worker_status,omitempty"`
|
||||||
|
ParseStatus string `json:"parse_status,omitempty"`
|
||||||
|
Format string `json:"format,omitempty"`
|
||||||
|
HasReason *bool `json:"has_reason,omitempty"`
|
||||||
|
HasFailureReason *bool `json:"has_failure_reason,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// TranslationWorkerRunParams starts one Rust-owned provider worker task.
|
||||||
|
// Pointer numeric fields preserve explicit zeroes so Rust can reject invalid
|
||||||
|
// dashboard input instead of receiving omitted defaults.
|
||||||
|
type TranslationWorkerRunParams struct {
|
||||||
|
Provider string `json:"provider,omitempty"`
|
||||||
|
FixturePath string `json:"fixture_path,omitempty"`
|
||||||
|
Concurrency *uint64 `json:"concurrency,omitempty"`
|
||||||
|
MaxAttempts *uint64 `json:"max_attempts,omitempty"`
|
||||||
|
LeaseSeconds *uint64 `json:"lease_seconds,omitempty"`
|
||||||
|
RetryBackoffSeconds *uint64 `json:"retry_backoff_seconds,omitempty"`
|
||||||
|
MaxTasks *uint64 `json:"max_tasks,omitempty"`
|
||||||
|
WorkerID string `json:"worker_id,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// TranslationWorkerConfig mirrors the accepted worker config returned by Rust.
|
||||||
|
type TranslationWorkerConfig struct {
|
||||||
|
Provider string `json:"provider"`
|
||||||
|
FixturePath string `json:"fixture_path,omitempty"`
|
||||||
|
Concurrency uint64 `json:"concurrency"`
|
||||||
|
MaxAttempts uint64 `json:"max_attempts"`
|
||||||
|
LeaseSeconds uint64 `json:"lease_seconds"`
|
||||||
|
RetryBackoffSeconds uint64 `json:"retry_backoff_seconds"`
|
||||||
|
MaxTasks *uint64 `json:"max_tasks,omitempty"`
|
||||||
|
WorkerID string `json:"worker_id"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// TranslationWorkerRunResult is returned when translation.worker.run is queued.
|
||||||
|
type TranslationWorkerRunResult struct {
|
||||||
|
TaskID string `json:"task_id"`
|
||||||
|
Kind string `json:"kind"`
|
||||||
|
Worker TranslationWorkerConfig `json:"worker"`
|
||||||
|
}
|
||||||
|
|
||||||
// TranslationProofread marks the current localized workflow as manual
|
// TranslationProofread marks the current localized workflow as manual
|
||||||
// proofreading. Rust owns the persisted localized state.
|
// proofreading. Rust owns the persisted localized state.
|
||||||
func (c *Client) TranslationProofread(ctx context.Context) (json.RawMessage, error) {
|
func (c *Client) TranslationProofread(ctx context.Context) (json.RawMessage, error) {
|
||||||
@@ -502,6 +554,20 @@ func (c *Client) TranslationTaskUpdate(ctx context.Context, params TranslationTa
|
|||||||
return c.rawData(ctx, "translation.task.update", params)
|
return c.rawData(ctx, "translation.task.update", params)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (c *Client) TranslationTasks(ctx context.Context, params TranslationTaskListParams) (json.RawMessage, error) {
|
||||||
|
return c.rawData(ctx, "translation.tasks", params)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) TranslationHandoff(ctx context.Context) (json.RawMessage, error) {
|
||||||
|
return c.rawData(ctx, "translation.handoff", nil)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) TranslationWorkerRun(ctx context.Context, params TranslationWorkerRunParams) (*TranslationWorkerRunResult, error) {
|
||||||
|
var out TranslationWorkerRunResult
|
||||||
|
_, err := c.Call(ctx, "translation.worker.run", params, &out)
|
||||||
|
return &out, err
|
||||||
|
}
|
||||||
|
|
||||||
func (c *Client) CatalogStatus(ctx context.Context) (json.RawMessage, error) {
|
func (c *Client) CatalogStatus(ctx context.Context) (json.RawMessage, error) {
|
||||||
return c.rawData(ctx, "catalog.status", nil)
|
return c.rawData(ctx, "catalog.status", nil)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -336,6 +336,94 @@ func TestScheduleRunSendsScopeAndMaxRuns(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestTranslationTasksSendsQueryParams(t *testing.T) {
|
||||||
|
offset := uint64(10)
|
||||||
|
limit := uint64(25)
|
||||||
|
hasReason := true
|
||||||
|
hasFailureReason := false
|
||||||
|
client := newTestClient(t, func(t *testing.T, req testRequest) testResponse {
|
||||||
|
if req.Method != "translation.tasks" {
|
||||||
|
t.Fatalf("method = %s", req.Method)
|
||||||
|
}
|
||||||
|
var params TranslationTaskListParams
|
||||||
|
if err := json.Unmarshal(req.Params, ¶ms); err != nil {
|
||||||
|
t.Fatalf("decode params: %v", err)
|
||||||
|
}
|
||||||
|
if params.Offset == nil || *params.Offset != offset ||
|
||||||
|
params.Limit == nil || *params.Limit != limit ||
|
||||||
|
params.TaskID != "textunit/v-current/Scenario" ||
|
||||||
|
params.ReleaseID != "v-current" ||
|
||||||
|
params.Destination != "MediaResources/GameData/Scenario.zip" ||
|
||||||
|
params.PathPattern != "Scenario" ||
|
||||||
|
params.ArchiveEntry != "ScenarioExcelTable.json" ||
|
||||||
|
params.Status != "queued_offline" ||
|
||||||
|
params.WorkerStatus != "failed" ||
|
||||||
|
params.ParseStatus != "ok" ||
|
||||||
|
params.Format != "json" ||
|
||||||
|
params.HasReason == nil || *params.HasReason != hasReason ||
|
||||||
|
params.HasFailureReason == nil || *params.HasFailureReason != hasFailureReason {
|
||||||
|
t.Fatalf("params = %#v", params)
|
||||||
|
}
|
||||||
|
return testResponse{
|
||||||
|
Result: testEnvelope{
|
||||||
|
OK: true,
|
||||||
|
Status: "ok",
|
||||||
|
RequestID: "req-test-translation-tasks",
|
||||||
|
Data: map[string]any{"tasks": []any{}},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
raw, err := client.TranslationTasks(context.Background(), TranslationTaskListParams{
|
||||||
|
Offset: &offset,
|
||||||
|
Limit: &limit,
|
||||||
|
TaskID: "textunit/v-current/Scenario",
|
||||||
|
ReleaseID: "v-current",
|
||||||
|
Destination: "MediaResources/GameData/Scenario.zip",
|
||||||
|
PathPattern: "Scenario",
|
||||||
|
ArchiveEntry: "ScenarioExcelTable.json",
|
||||||
|
Status: "queued_offline",
|
||||||
|
WorkerStatus: "failed",
|
||||||
|
ParseStatus: "ok",
|
||||||
|
Format: "json",
|
||||||
|
HasReason: &hasReason,
|
||||||
|
HasFailureReason: &hasFailureReason,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("TranslationTasks error: %v", err)
|
||||||
|
}
|
||||||
|
if !json.Valid(raw) {
|
||||||
|
t.Fatalf("invalid raw JSON: %s", string(raw))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTranslationHandoffUsesRustMethod(t *testing.T) {
|
||||||
|
client := newTestClient(t, func(t *testing.T, req testRequest) testResponse {
|
||||||
|
if req.Method != "translation.handoff" {
|
||||||
|
t.Fatalf("method = %s", req.Method)
|
||||||
|
}
|
||||||
|
if string(req.Params) != "null" {
|
||||||
|
t.Fatalf("params = %s, want null", req.Params)
|
||||||
|
}
|
||||||
|
return testResponse{
|
||||||
|
Result: testEnvelope{
|
||||||
|
OK: true,
|
||||||
|
Status: "ok",
|
||||||
|
RequestID: "req-test-translation-handoff",
|
||||||
|
Data: map[string]any{"jobs": []any{}},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
raw, err := client.TranslationHandoff(context.Background())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("TranslationHandoff error: %v", err)
|
||||||
|
}
|
||||||
|
if !json.Valid(raw) {
|
||||||
|
t.Fatalf("invalid raw JSON: %s", string(raw))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestTranslationTaskUpdateSendsWorkerParams(t *testing.T) {
|
func TestTranslationTaskUpdateSendsWorkerParams(t *testing.T) {
|
||||||
client := newTestClient(t, func(t *testing.T, req testRequest) testResponse {
|
client := newTestClient(t, func(t *testing.T, req testRequest) testResponse {
|
||||||
if req.Method != "translation.task.update" {
|
if req.Method != "translation.task.update" {
|
||||||
@@ -375,6 +463,75 @@ func TestTranslationTaskUpdateSendsWorkerParams(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestTranslationWorkerRunSendsProviderConfig(t *testing.T) {
|
||||||
|
var concurrency uint64 = 8
|
||||||
|
var maxAttempts uint64 = 4
|
||||||
|
var leaseSeconds uint64 = 60
|
||||||
|
var retryBackoff uint64 = 0
|
||||||
|
var maxTasks uint64 = 2
|
||||||
|
client := newTestClient(t, func(t *testing.T, req testRequest) testResponse {
|
||||||
|
if req.Method != "translation.worker.run" {
|
||||||
|
t.Fatalf("method = %s", req.Method)
|
||||||
|
}
|
||||||
|
var params TranslationWorkerRunParams
|
||||||
|
if err := json.Unmarshal(req.Params, ¶ms); err != nil {
|
||||||
|
t.Fatalf("decode params: %v", err)
|
||||||
|
}
|
||||||
|
if params.Provider != "mock" ||
|
||||||
|
params.FixturePath != "/tmp/mock-provider.json" ||
|
||||||
|
params.Concurrency == nil || *params.Concurrency != concurrency ||
|
||||||
|
params.MaxAttempts == nil || *params.MaxAttempts != maxAttempts ||
|
||||||
|
params.LeaseSeconds == nil || *params.LeaseSeconds != leaseSeconds ||
|
||||||
|
params.RetryBackoffSeconds == nil || *params.RetryBackoffSeconds != retryBackoff ||
|
||||||
|
params.MaxTasks == nil || *params.MaxTasks != maxTasks ||
|
||||||
|
params.WorkerID != "dashboard-worker" {
|
||||||
|
t.Fatalf("params = %#v", params)
|
||||||
|
}
|
||||||
|
return testResponse{
|
||||||
|
Result: testEnvelope{
|
||||||
|
OK: true,
|
||||||
|
Status: "accepted",
|
||||||
|
RequestID: "req-test-translation-worker",
|
||||||
|
Data: map[string]any{
|
||||||
|
"task_id": "task-worker-1",
|
||||||
|
"kind": "translation.worker.run",
|
||||||
|
"worker": map[string]any{
|
||||||
|
"provider": "mock",
|
||||||
|
"fixture_path": "/tmp/mock-provider.json",
|
||||||
|
"concurrency": concurrency,
|
||||||
|
"max_attempts": maxAttempts,
|
||||||
|
"lease_seconds": leaseSeconds,
|
||||||
|
"retry_backoff_seconds": retryBackoff,
|
||||||
|
"max_tasks": maxTasks,
|
||||||
|
"worker_id": "dashboard-worker",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
result, err := client.TranslationWorkerRun(context.Background(), TranslationWorkerRunParams{
|
||||||
|
Provider: "mock",
|
||||||
|
FixturePath: "/tmp/mock-provider.json",
|
||||||
|
Concurrency: &concurrency,
|
||||||
|
MaxAttempts: &maxAttempts,
|
||||||
|
LeaseSeconds: &leaseSeconds,
|
||||||
|
RetryBackoffSeconds: &retryBackoff,
|
||||||
|
MaxTasks: &maxTasks,
|
||||||
|
WorkerID: "dashboard-worker",
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("TranslationWorkerRun error: %v", err)
|
||||||
|
}
|
||||||
|
if result.TaskID != "task-worker-1" ||
|
||||||
|
result.Kind != "translation.worker.run" ||
|
||||||
|
result.Worker.Concurrency != concurrency ||
|
||||||
|
result.Worker.RetryBackoffSeconds != retryBackoff ||
|
||||||
|
result.Worker.MaxTasks == nil || *result.Worker.MaxTasks != maxTasks {
|
||||||
|
t.Fatalf("unexpected result: %#v", result)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestTranslationProofreadUsesRustMethod(t *testing.T) {
|
func TestTranslationProofreadUsesRustMethod(t *testing.T) {
|
||||||
client := newTestClient(t, func(t *testing.T, req testRequest) testResponse {
|
client := newTestClient(t, func(t *testing.T, req testRequest) testResponse {
|
||||||
if req.Method != "translation.proofread" {
|
if req.Method != "translation.proofread" {
|
||||||
|
|||||||
+23
-13
@@ -1,16 +1,26 @@
|
|||||||
# bat-api Admin Panel
|
# bat-api Admin Panel
|
||||||
|
|
||||||
This directory is reserved for the future player-facing `bat-api` management
|
本目录预留给后续 `bat-api` 管理面板。当前后端已提供只读入口
|
||||||
panel. The current backend exposes a non-mutating index at `GET /admin/` that
|
`GET /admin/`,返回 health、ready、bootstrap、release、resources 和 OpenAPI
|
||||||
returns JSON links for health, readiness, bootstrap, release, resources, and
|
链接。
|
||||||
OpenAPI. `GET /admin/schedules` and mutating
|
|
||||||
`/admin/control/schedule-{add,update,remove,run}` routes are protected by the
|
|
||||||
configured admin token and forward the Rust-owned schedule contract.
|
|
||||||
`GET /admin/schedules` accepts optional `id`, `group`, and `enabled` filters.
|
|
||||||
`POST /admin/control/translation-task-update` uses the same token and forwards
|
|
||||||
the narrow `translation.task.update` provider-worker status contract.
|
|
||||||
|
|
||||||
The production panel must reuse the same HTTP authentication, rate limiting,
|
需要配置 `BAT_API_AUTH_TOKEN` 的接口:
|
||||||
access logging, reverse-proxy handling, and no-store dynamic response policy as
|
|
||||||
the resource API. Static frontend assets are not implemented yet. See
|
- `GET /admin/schedules`:转发 Rust 持有的 schedule 查询,支持 `id`、`group`
|
||||||
`docs/reports/GO_STATUS.md` for the current component boundary and status.
|
和 `enabled` 过滤。
|
||||||
|
- `POST /admin/control/schedule-{add,update,remove,run}`:转发
|
||||||
|
`schedule.*` 计划控制。
|
||||||
|
- `POST /admin/control/translation-task-update`:转发
|
||||||
|
`translation.task.update`,供外部 provider 流程回写任务状态。
|
||||||
|
- `POST /admin/control/translation-worker-run`:转发
|
||||||
|
`translation.worker.run`,触发 Rust provider worker。
|
||||||
|
- `POST /admin/control/translation-proofread`:转发
|
||||||
|
`translation.proofread`,把汉化 workflow 标记为人工校对中。
|
||||||
|
- `GET /admin/translation/tasks`:转发 `translation.tasks`,按分页、release、
|
||||||
|
TextUnit、状态和失败原因过滤查询 worker 状态。
|
||||||
|
- `GET /admin/translation/handoff`:转发 `translation.handoff`,读取当前
|
||||||
|
release 的完整翻译交接视图。
|
||||||
|
|
||||||
|
正式前端必须复用资源 API 的 HTTP 鉴权、限流、访问日志、反代处理和动态响应
|
||||||
|
`Cache-Control: no-store` 策略。静态前端资产仍为 not implemented,尚未作为
|
||||||
|
本目录的稳定交付物。当前组件边界见 `docs/reports/GO_STATUS.md`。
|
||||||
|
|||||||
Reference in New Issue
Block a user