mirror of
https://github.com/Yuyi-Oak/BlueArchiveToolkit.git
synced 2026-09-18 11:56:23 +08:00
Compare commits
5
Commits
102b49b666
...
03021ad649
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
03021ad649
|
||
|
|
3f78f8f880
|
||
|
|
2079c6a307
|
||
|
|
f4880a71bd
|
||
|
|
3e9bb20d79
|
@@ -39,6 +39,13 @@ BlueArchive Toolkit 是长期维护的开源工具链,不是 demo、一次性
|
|||||||
6. 不引入 God Object、God Class、超长函数、超长文件、硬编码、魔法数字、重复代码、临时实现或只为当前测试通过的伪实现。
|
6. 不引入 God Object、God Class、超长函数、超长文件、硬编码、魔法数字、重复代码、临时实现或只为当前测试通过的伪实现。
|
||||||
7. 不使用 `TODO`、`FIXME` 掩盖未完成设计。确实无法完成时,应在当前缺口文档中说明边界、风险和后续工作。
|
7. 不使用 `TODO`、`FIXME` 掩盖未完成设计。确实无法完成时,应在当前缺口文档中说明边界、风险和后续工作。
|
||||||
|
|
||||||
|
## 当前冻结
|
||||||
|
|
||||||
|
1. UnityFS / AssetBundle / Addressables / TypeTree 解析模块当前处于维护冻结,细则见 `docs/reports/PARSER_FREEZE.md`。
|
||||||
|
2. 冻结期不继续新增解析类型、字段族、catalog 结构覆盖、写入型解析 RPC/CLI 或合成 fixture 驱动的能力扩展。
|
||||||
|
3. 冻结期允许且优先处理编译、测试、clippy、真实运行回归、错误诊断、状态一致性、缓存复用和文档一致性问题。
|
||||||
|
4. 如果用户明确要求继续解析扩展,必须先指出冻结状态、说明风险,并获得明确解冻或例外授权。
|
||||||
|
|
||||||
## 开发流程
|
## 开发流程
|
||||||
|
|
||||||
1. 动手前先读相关文档和代码,确认当前真实状态。
|
1. 动手前先读相关文档和代码,确认当前真实状态。
|
||||||
|
|||||||
+14
-2
@@ -9,15 +9,27 @@
|
|||||||
### 新增
|
### 新增
|
||||||
- Addressables catalog 提取 `m_Crc`(bundle IEEE CRC-32):`ResourceEntry` 新增 `crc` 字段(compact/expanded 两种形态均解析),SQLite 持久化并对旧库幂等迁移补列;core 新增 `crc32_ieee` 与 `ResourceEntry::verify_downloaded_bytes`(按声明的 size/CRC 校验字节)(issue #2)
|
- Addressables catalog 提取 `m_Crc`(bundle IEEE CRC-32):`ResourceEntry` 新增 `crc` 字段(compact/expanded 两种形态均解析),SQLite 持久化并对旧库幂等迁移补列;core 新增 `crc32_ieee` 与 `ResourceEntry::verify_downloaded_bytes`(按声明的 size/CRC 校验字节)(issue #2)
|
||||||
- UnityFS 解析新增目录条目越界校验:directory 的 `offset+size` 必须落在解压数据区内,截断/损坏 bundle 的越界目录条目不再被静默接受(issue #3)
|
- UnityFS 解析新增目录条目越界校验:directory 的 `offset+size` 必须落在解压数据区内,截断/损坏 bundle 的越界目录条目不再被静默接受(issue #3)
|
||||||
- 官方资源下载回归顺序执行:manifest/quarantine 簿记与 seed `.hash` 校验保持串行,`fail-fast` 与「不发布不完整资源」不变量不变(issue #17)
|
- 官方资源下载回归顺序执行:manifest/quarantine 簿记与 seed `.hash` 校验保持串行,`fail-fast` 与「不发布不完整资源」不变量不变(issue #17,按 wontfix 关闭多线程下载目标)
|
||||||
|
- 新增 `cmd/bat-api` 资源 bootstrap/分发 HTTP 服务(issue #19 / G-009 资源面):与 Rust `bat` 同环境运行,经 `bat.sock` RPC(`daemon.status` → `daemon.doctor` → catalog/manifest)发现已发布 release 和 `resource_root`,按官方 CDN host/path 只读提供资源;管理面 `/healthz` `/v1/release` `/v1/resources`;可选 server-info 仅改写 Addressables root;`.env` 配置监听端口/RPC socket/刷新周期/预留数据库键。资源自动拉取仍由 Rust `bat` 负责
|
||||||
|
- `cmd/bat-api` 新增 `/v1/bootstrap`:把 Rust `bat` 的 RPC 健康、已发布 release 摘要、server-info URL、client-patch base 和改写后的 Addressables root 组织成启动前资源发现响应,固定 `bat` 是资源生产者、`bat-api` 是只读 bootstrap/分发层的关系
|
||||||
|
- `bat-api` CDN 分发补齐 Range / HEAD / 条件请求语义:基于 manifest BLAKE3 生成 ETag,返回 Last-Modified、Accept-Ranges 和长期缓存头,`.hash` 以 `text/plain` 返回
|
||||||
|
- `bat-api` 增加 RPC 周期刷新(`BAT_API_REFRESH_INTERVAL` / `--refresh-interval`),用于跟随远程长期运行的 Rust `bat` 发布新 release;生产不应写死 `BAT_API_RESOURCE_ROOT`
|
||||||
|
- `bat-api` 增加 refresh 诊断和 `/readyz`:`/healthz` 暴露最近一次 RPC refresh 的时间、耗时、warning 和错误,`/readyz` 在无可分发 release 时返回 `503`
|
||||||
|
- `bat-api` 增加 launcher 资源引导兼容:`/v1/launcher/bootstrap`、`/api/launcher/game/config`、`/api/launcher/game/config/json`、`/api/launcher/advanced/game/download/cdn` 及 `/api-launcher-jp.yo-star.com/...` host 形状入口,响应来自 Rust `bat` 已发布 snapshot/RPC,明确不提供登录、网关、鉴权或完整 package update manifest
|
||||||
|
- `bat-api` 增加玩家-facing HTTP 控制面:token 鉴权、进程内限流、访问日志、反代 IP 适配、安全响应头、动态 JSON `Cache-Control: no-store`、`/v1/resources` 分页上限、统一 JSON error、OpenAPI (`/openapi.yaml`) 和 `/admin/` 管理面板预留
|
||||||
|
- 新增 `deployments/systemd/bluearchive-toolkit-bat-api.service` 和 `deployments/systemd/bat-api.env.example`,固定 bat-api 通过本机 `bat.sock` 获取资源根的部署契约
|
||||||
|
- USERGUIDE 补充 `bat-api` 资源 bootstrap / 分发章节,说明与 Rust `bat` 的运行关系、接口、CDN 响应语义和不仿造业务 API 的边界
|
||||||
|
- 官方同步新增 `official-parse-cache.json`:校验发布后从下载 manifest 覆盖直接 UnityFS bundle、zip 内 UnityFS 条目和非候选资源记录;URL、相对路径、size 和 BLAKE3 未变化时跳过重复解析
|
||||||
|
- 官方原版资源和汉化产物目录分离:`BAT_OUTPUT`/`--output` 默认 `./bat-resources`,`BAT_LOCALIZED_OUTPUT`/`--localized-output` 默认 `./bat-localized`;同步报告新增 `localized_release_status=not_localized`,后续 Patch 发布完成后才切换为 `localized`
|
||||||
|
|
||||||
### 修复
|
### 修复
|
||||||
- 官方下载失败重试之间加入指数退避(网络类失败 200ms→400ms→800ms…,上限 5s)
|
- 官方下载失败重试之间加入指数退避(网络类失败 200ms→400ms→800ms…,上限 5s)
|
||||||
|
|
||||||
### 计划
|
### 计划
|
||||||
- [ ] 实现 `bat-api`:仿 BlueArchive 官方 API 的 Go HTTP 服务(含鉴权/签名验签,issue #19)
|
- [ ] `bat-api` 后续:全量 release 联调、Rust/Go snapshot contract fixture(用户审核后)、refresh mtime/size 增量缓存、完整 launcher 安装包更新链(若需要,新 issue)、API 持久化层接入预留 database/redis 配置
|
||||||
- [ ] 官方同步结果接入 CAS + ResourceRepository 的用户级工作流
|
- [ ] 官方同步结果接入 CAS + ResourceRepository 的用户级工作流
|
||||||
- [ ] 官方下载/导入路径接入 CRC/size 校验(复用 `verify_downloaded_bytes`)
|
- [ ] 官方下载/导入路径接入 CRC/size 校验(复用 `verify_downloaded_bytes`)
|
||||||
|
- [ ] 汉化 Patch 发布:维护 `localized-output/current`、`localized-version-state`、`localized` 状态切换和回滚
|
||||||
- [ ] 实现翻译系统
|
- [ ] 实现翻译系统
|
||||||
- [ ] 实现 Patch 引擎
|
- [ ] 实现 Patch 引擎
|
||||||
- [ ] 实现 Web 管理后台
|
- [ ] 实现 Web 管理后台
|
||||||
|
|||||||
+79
-64
@@ -1,35 +1,36 @@
|
|||||||
# BlueArchiveToolkit 当前工作区状态
|
# BlueArchiveToolkit 当前工作区状态
|
||||||
|
|
||||||
- **更新时间**:2026-07-20
|
- **更新时间**:2026-07-26
|
||||||
- **状态来源**:本地工作区盘点、代码验证和最新提交
|
- **状态来源**:本地工作区盘点、代码验证和最新提交
|
||||||
- **状态分支**:`experiment`
|
- **状态分支**:`experiment`
|
||||||
- **最新已推送功能提交**:以当前 `git log --oneline -1` 为准
|
- **最新已推送功能提交**:以当前 `git log --oneline -1` 为准
|
||||||
- **权威计划**:`PROJECT_PLAN.md`
|
- **权威计划**:`PROJECT_PLAN.md`
|
||||||
|
- **Go 进度权威**:`docs/reports/GO_STATUS.md`
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 1. 总体判断
|
## 1. 总体判断
|
||||||
|
|
||||||
当前项目处于 **稳定基线完成、CAS V1 已落地、Rust 官方资源同步链路已具备可持续生产运行形态、Go `bat-api` 已有 Rust daemon RPC client 但 CLI/API/Web 仍未形成产品入口** 阶段。
|
当前项目处于 **稳定基线完成、CAS V1 已落地、Rust 官方资源同步链路已具备可持续生产运行形态、Go 侧以 `bat-api` 资源 bootstrap/分发 MVP + `backendrpc` 为正式服务入口(同步/运维命令行仍为近乎全自动的 Rust `bat`)** 阶段。
|
||||||
|
|
||||||
Rust 侧官方日服资源链路已经从实验验证推进到正式入口:
|
Rust 侧官方日服资源链路已经从实验验证推进到正式入口:
|
||||||
|
|
||||||
1. 首次运行可以通过 `--auto-discover` 从官方 HTTP metadata 解析 `GameMainConfig`,自动获得 `app-version`、`connection-group` 和 `server-info`;解密出的 `GameMainConfig` JSON 会校验已知字段,避免把错误解密结果当成成功。
|
1. 首次运行可以通过 `--auto-discover` 从官方 HTTP metadata 解析 `GameMainConfig`,自动获得 `app-version`、`connection-group` 和 `server-info`;解密出的 `GameMainConfig` JSON 会校验已知字段,避免把错误解密结果当成成功。自动发现会记录 launcher metadata、launcher CDN config、remote manifest 文件列表 digest、选中的 `resources.assets` 来源和 `GameMainConfig` 摘要。
|
||||||
2. 不安装、不启动、不依赖已安装官方启动器。
|
2. 不安装、不启动、不依赖已安装官方启动器。
|
||||||
3. 默认平台为 `Windows + Android`。
|
3. 默认平台为 `Windows + Android`。
|
||||||
4. 能生成官方全量 pull plan,执行真实下载,维护 release 内的 `official-download-manifest.json`。
|
4. 能生成官方全量 pull plan,执行真实下载,维护 release 内的 `official-download-manifest.json`。
|
||||||
5. 下载后使用本地 manifest 的 size + BLAKE3 校验复用文件;所有 `.zip` 在下载验收、复用、本地 audit/verify 时做 ZIP 结构校验;官方 seed `.hash` 使用标准 `xxHash32(seed=0)` 强校验(早期实现的非标准 avalanche 常量已修正)。
|
5. 下载后使用本地 manifest 的 size + BLAKE3 校验复用文件;所有 `.zip` 在下载验收、复用、本地 audit/verify 时做 ZIP 结构校验;官方 seed `.hash` 使用标准 `xxHash32(seed=0)` 强校验(早期实现的非标准 avalanche 常量已修正)。
|
||||||
6. 支持 `.part` 断点续传、失败后 clean retry、本地 manifest audit/repair、失败 staging 恢复复用、403/404/5xx 分类重试(重试带指数退避)、下载 quarantine 诊断,以及旧 launcher 包官方 primary/backup CDN 切换。下载执行保持顺序处理;manifest/quarantine 簿记与 seed `.hash` 校验仍逐项执行,`fail-fast` 与「不发布不完整资源」不变量不变。下载进度按已完成数量单调上报,不再使用 plan 序号计算百分比。
|
6. 支持 `.part` 断点续传、失败后 clean retry、本地 manifest audit/repair、失败 staging 恢复复用、403/404/5xx 分类重试(重试带指数退避)、下载 quarantine 诊断,以及旧 launcher 包官方 primary/backup CDN 切换。启动器/server-info 先行更新但 client-patch seed marker 或必需 seed catalog 尚未开放时,会进入 `waiting_for_official_resources`,保留现有 `current`,不创建失败 staging,也不写入失败版本循环;启用 `--auto-discover` 的非 dry-run 会写入 `<output>/official-launcher-bootstrap.pending.json` 作为维护期证据。下载执行保持顺序处理;manifest/quarantine 簿记与 seed `.hash` 校验仍逐项执行,`fail-fast` 与「不发布不完整资源」不变量不变。下载进度按已完成数量单调上报,不再使用 plan 序号计算百分比。
|
||||||
7. 支持 curl 传输层本地代理:默认自动检测 `HTTPS_PROXY` / `ALL_PROXY` / `HTTP_PROXY` 及小写环境变量(带凭据的代理推荐用环境变量配置),也可用 `--proxy <URL>` 显式指定或 `--no-proxy` 强制直连;代理决策会写入 progress log、daemon log 和 `bat doctor` 诊断输出。代理凭据不落世界可读位置:日志/`status` 脱敏,传给 curl 经 `ALL_PROXY` 环境变量而非 argv,`--daemon` 下经环境变量下传后台子进程、不进子进程 argv 或 `bat-status.json`,复用凭据存于 `bat-proxy.secret`(`0600`)且 `clean-stable` 会清除。
|
7. 支持 curl 传输层本地代理:默认自动检测 `HTTPS_PROXY` / `ALL_PROXY` / `HTTP_PROXY` 及小写环境变量(带凭据的代理推荐用环境变量配置),也可用 `--proxy <URL>` 显式指定或 `--no-proxy` 强制直连;代理决策会写入 progress log、daemon log 和 `bat doctor` 诊断输出。代理凭据不落世界可读位置:日志/`status` 脱敏,传给 curl 经 `ALL_PROXY` 环境变量而非 argv,`--daemon` 下经环境变量下传后台子进程、不进子进程 argv 或 `bat-status.json`,复用凭据存于 `bat-proxy.secret`(`0600`)且 `clean-stable` 会清除。
|
||||||
8. `bat --watch` 可常驻运行,`bat --daemon` 可后台运行并用 `bat status` / `bat stop` / `bat restart` / `bat reload` / `bat logs` 管理;daemon 使用 `bat.sock` Unix socket JSON-RPC 作为 live 控制通道,PID/状态/日志文件作为快照和 fallback,`bat-events.jsonl` 记录带轮转的结构化事件日志,`bat-control.lock` 串行化控制命令;正常检查默认每 1 小时一次;远端和本地一致时默认静默,失败后默认 60 秒快速重试;CLI 默认向 stdout 输出人类可读摘要,向 stderr 输出 ASCII banner、progress log、失败分类和 quarantine 状态,需要机器输出时使用 `--json --no-progress`。
|
8. `bat --watch` 可常驻运行,`bat --daemon` 可后台运行并用 `bat status` / `bat stop` / `bat restart` / `bat reload` / `bat logs` 管理;daemon 使用 `bat.sock` Unix socket JSON-RPC 作为 live 控制通道,PID/状态/日志文件作为快照和 fallback,`bat-events.jsonl` 记录带轮转的结构化事件日志,`bat-control.lock` 串行化控制命令;正常检查默认每 1 小时一次;远端和本地一致时默认静默,失败后默认 60 秒快速重试,官方资源端尚未开放时状态为 `waiting` 并同样按错误重试间隔探测;CLI 默认向 stdout 输出人类可读摘要,向 stderr 输出 ASCII banner、progress log、失败分类和 quarantine 状态,需要机器输出时使用 `--json --no-progress`。
|
||||||
9. 远端 snapshot 未变化但输出目录为空时,会按首次运行执行全量拉取;官方 seed `.hash` 校验失败时会清理对应 manifest 条目,避免失败产物被后续本地 audit 误判为可复用。
|
9. 远端 snapshot 未变化但输出目录为空时,会按首次运行执行全量拉取;官方 seed `.hash` 校验失败时会清理对应 manifest 条目,避免失败产物被后续本地 audit 误判为可复用。
|
||||||
10. 默认资源目录是 `./bat-resources`,默认后台状态目录是 `/tmp/bat-pid`;资源目录是发布根目录,包含 `current` symlink、`versions/<id>` 和 `.staging/<id>`,非 dry-run 会先写 staging,校验完成后发布 versioned 目录并原子切换 `current`;如果上一轮同一 app version、bundle version 和 Addressables root 的 staging 失败但目录仍安全存在,下一轮会复用该 staging 并按 manifest 逐文件校验/补下载;后台状态目录包含 `bat.sock`、`bat.pid`、`bat-status.json`、`bat-daemon.log`、`bat-events.jsonl`、任务历史 `bat-tasks.json` 和短生命周期 `bat-control.lock`;非 dry-run 使用 `--output/.official-sync.lock` 防止并发写同一资源目录,live daemon 会阻止前台写命令直接修改它正在管理的同一目录。
|
10. 默认官方原版资源目录是 `./bat-resources`,默认汉化产物目录是 `./bat-localized`,默认后台状态目录是 `/tmp/bat-pid`;官方资源目录是发布根目录,包含 `current` symlink、`versions/<id>` 和 `.staging/<id>`,非 dry-run 会先写 staging,校验完成后发布 versioned 目录并原子切换 `current`;启用 `--auto-discover` 的 release 会包含 `official-launcher-bootstrap.json`,up-to-date 轮询会为旧 release 补写该产物;如果上一轮同一 app version、bundle version 和 Addressables root 的 staging 失败但目录仍安全存在,下一轮会复用该 staging 并按 manifest 逐文件校验/补下载;后台状态目录包含 `bat.sock`、`bat.pid`、`bat-status.json`、`bat-daemon.log`、`bat-events.jsonl`、任务历史 `bat-tasks.json` 和短生命周期 `bat-control.lock`;非 dry-run 使用 `--output/.official-sync.lock` 防止并发写同一资源目录,live daemon 会阻止前台写命令直接修改它正在管理的同一目录。
|
||||||
11. 官方同步会拒绝危险输出目录、路径逃逸和现有 symlink 路径组件;下载目标、`.part`、manifest、snapshot、PID、status、log 和控制锁文件不会跟随 symlink,daemon 状态类文件默认以 `0600` 权限创建。
|
11. 官方同步会拒绝危险输出目录、路径逃逸和现有 symlink 路径组件;下载目标、`.part`、manifest、snapshot、PID、status、log 和控制锁文件不会跟随 symlink,daemon 状态类文件默认以 `0600` 权限创建。
|
||||||
12. `<output>/official-version-state.json` 会明确保存当前已完成版本、正在拉取版本、上一个可用版本和失败版本;同一 app version、bundle version 和 Addressables root 的失败只保留最新一条,同一版本开始重新拉取或后续发布成功时会清理对应失败记录;`bat status` 会显示最后成功时间、下次检查时间、最后错误摘要、当前阶段、当前下载 URL 进度、版本状态摘要、最近历史失败版本和原因、结构化日志路径和轮转日志路径,人类可读输出不会把完整版本状态 JSON 内联打印。
|
12. `<output>/official-version-state.json` 会明确保存当前已完成版本、正在拉取版本、上一个可用版本和失败版本;同一 app version、bundle version 和 Addressables root 的失败只保留最新一条,同一版本开始重新拉取或后续发布成功时会清理对应失败记录;`bat status` 会显示最后成功时间、下次检查时间、最后错误摘要、当前阶段、当前下载 URL 进度、版本状态摘要、最近历史失败版本和原因、结构化日志路径和轮转日志路径,人类可读输出不会把完整版本状态 JSON 内联打印。
|
||||||
13. 资源导入链路已支持 CAS + `ResourceRepository` 索引写入,AssetBundle 导入会记录 UnityFS 摘要,TextAsset/Table/Media 会按类型分类;当前/上一个/结构变化 catalog、失败 staging 复用、403/404、hash mismatch、CRC 与 UnityFS 边界校验均有离线回归 fixture 或单测覆盖。
|
13. 资源导入链路已支持 CAS + `ResourceRepository` 索引写入,官方同步可用 `--import-repository` / `BAT_IMPORT_REPOSITORY=1` 在已校验 release 发布后触发导入,默认 CAS 路径为 `<output>/.cas`、SQLite 索引为 `<output>/resources.sqlite`,也可通过 `--import-cas-root`、`--import-resource-db`、`BAT_IMPORT_CAS_ROOT`、`BAT_IMPORT_RESOURCE_DB` 覆盖;`resource.index` RPC 可按类型、hash、路径模式分页查询现有索引,数据库不存在时返回 `available=false` 且不会创建空库。`Resource` metadata 已通过 `metadata_json` 兼容迁移保存 release、平台、bundle path、parse status、TextAsset 名称和 TextUnit 数量/格式;当前/上一个/结构变化 catalog、失败 staging 复用、403/404、hash mismatch、CRC、metadata 迁移与 UnityFS 边界校验均有离线回归 fixture 或单测覆盖。
|
||||||
14. `bat` 首次启动会在二进制所在目录释放 `.env` 配置模板(`0600`),之后每次启动自动加载(不覆盖已存在的环境变量),支持 `BAT_OUTPUT`/`BAT_STATE_DIR`/`BAT_AUTO_DISCOVER`/`BAT_WATCH`/`BAT_DAEMON`/`BAT_PROXY` 等键,实现编辑 `.env` 后无参启动;优先级为命令行参数 > 进程环境变量 > `.env` > 内置默认值,`BAT_SKIP_ENV_FILE=1` 可整体禁用;Redis 键为预留。daemon 任务历史持久化在 `<state-dir>/bat-tasks.json`(版本化、`0600` 原子写),重启后任务经 `task.*` 仍可查,中断任务标记 `task_interrupted`(`BAT-ERR-700005`)。
|
14. 非 dry-run 官方同步在校验完成并发布后,会先对比上一完整 release 与当前 release 的 `official-download-manifest.json`,在当前 release 下写入 `official-resource-changes.json` 和 `crowdin-translation-handoff.json`;同一 destination 只有 size 或 BLAKE3 改变才算 modified,仅 URL/CDN 根变化但内容相同不会触发解析/翻译候选。随后刷新 `official-parse-cache.json` 和 `official-textunit-index.json`,并从 Added/Modified 资源、parse cache 与 TextUnit 明细索引派生 `official-textunit-tasks.json` 和 `crowdin-textunit-queue.json`;删除资源只进入差异记录,不进入 TextUnit/Crowdin 队列。`parse.text_units` 和 `parse.errors` RPC/CLI 可按 destination、archive entry、path id、class id、field path 和 format 查询当前 release 的 TextUnit 明细与解析错误;TextUnit 已包含 class id、field path、字段 offset/byte size 等可追溯定位。Crowdin 当前仅预留本地离线队列,不发网络请求;up-to-date 轮询在已有有效解析缓存、TextUnit 明细索引和 TextUnit 队列时只读取摘要,不重复解析。官方同步报告默认 `localized_release_status=not_localized`,含义是原版资源已经发布、汉化资源未发布;UnityFS TextAsset patch 发布成功后会写 `localized-patch-manifest.json`,校验 hash/size/diff/rollback 后才允许 `localized.status` 返回 `localized`。`bat` 首次启动会在二进制所在目录释放 `.env` 配置模板(`0600`),之后每次启动自动加载(不覆盖已存在的环境变量),支持 `BAT_OUTPUT`/`BAT_LOCALIZED_OUTPUT`/`BAT_IMPORT_REPOSITORY`/`BAT_IMPORT_CAS_ROOT`/`BAT_IMPORT_RESOURCE_DB`/`BAT_STATE_DIR`/`BAT_AUTO_DISCOVER`/`BAT_WATCH`/`BAT_DAEMON`/`BAT_PROXY` 等键,实现编辑 `.env` 后无参启动;优先级为命令行参数 > 进程环境变量 > `.env` > 内置默认值,`BAT_SKIP_ENV_FILE=1` 可整体禁用;Redis 键为预留。daemon 任务历史持久化在 `<state-dir>/bat-tasks.json`(版本化、`0600` 原子写),重启后任务经 `task.*` 仍可查,中断任务标记 `task_interrupted`(`BAT-ERR-700005`)。
|
||||||
|
|
||||||
仍需明确:这不是完整产品完成。Go 产品入口、完整 AssetBundle 引擎解析、Patch、翻译系统、API Server 和 Web 仍是后续工作;真实官方网络全量拉取 smoke 已固化为可重复脚本和 runbook(G-018 已关闭),当前正在进行长期运行测试,运行报告将在后续提供;真实大文件产物与运行报告默认保存在 `/tmp` 隔离目录,不纳入 Git。
|
仍需明确:这不是完整产品完成。完整 AssetBundle 重打包、翻译、Web、以及 `bat-api` 的服务器联调/可选业务扩展仍是后续工作;G-008(产品级 Go 同步 CLI)已决策关闭。真实官方网络全量拉取 smoke 已固化(G-018 已关闭);真实大文件与运行报告默认在 `/tmp` 隔离目录,不纳入 Git。Go 细节见 `docs/reports/GO_STATUS.md`。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -41,6 +42,7 @@ Rust 侧官方日服资源链路已经从实验验证推进到正式入口:
|
|||||||
- `docs/guides/official-resource-test-pull.md`:官方资源拉取与自动更新用户指南。
|
- `docs/guides/official-resource-test-pull.md`:官方资源拉取与自动更新用户指南。
|
||||||
- `docs/guides/official-full-pull-smoke.md`:真实官方全量拉取 smoke runbook 和可重复命令。
|
- `docs/guides/official-full-pull-smoke.md`:真实官方全量拉取 smoke runbook 和可重复命令。
|
||||||
- `docs/architecture/official-resource-backend.md`:官方资源后端设计和审核说明。
|
- `docs/architecture/official-resource-backend.md`:官方资源后端设计和审核说明。
|
||||||
|
- `docs/architecture/assetbundle.md`:解析补全路线图,覆盖 Addressables、UnityFS、Serialized 字段级解析、文本提取、CAS 接入和 Patch 发布前置。
|
||||||
- `docs/reports/CURRENT_GAPS.md`:当前缺口和关闭顺序。
|
- `docs/reports/CURRENT_GAPS.md`:当前缺口和关闭顺序。
|
||||||
|
|
||||||
历史 Week 2/Week 3 报告只作追溯,不再代表当前状态。
|
历史 Week 2/Week 3 报告只作追溯,不再代表当前状态。
|
||||||
@@ -96,7 +98,7 @@ Rust 侧官方日服资源链路已经从实验验证推进到正式入口:
|
|||||||
|
|
||||||
待完成:
|
待完成:
|
||||||
|
|
||||||
- `crates/bat-assetbundle` 仍是占位 crate,完整 UnityFS/对象表/TypeTree 引擎未实现。
|
- `crates/bat-assetbundle` 已具备 UnityFS 容器、对象表、TypeTree 元数据、基础字段读取、TextAsset 和 TextUnit 提取;UnityFS TextAsset patch 发布前置链路已可用,真实复杂版本差异、重打包和通用 Patch 仍未实现。
|
||||||
- Addressables parser 已覆盖当前真实形态 fixture/golden 与 `m_Crc`,但仍需继续覆盖二进制/压缩字段组合和更细失败诊断。
|
- Addressables parser 已覆盖当前真实形态 fixture/golden 与 `m_Crc`,但仍需继续覆盖二进制/压缩字段组合和更细失败诊断。
|
||||||
- 客户端发现、备份、应用补丁流程尚未连接真实实现。
|
- 客户端发现、备份、应用补丁流程尚未连接真实实现。
|
||||||
|
|
||||||
@@ -138,40 +140,46 @@ Rust 侧官方日服资源链路已经从实验验证推进到正式入口:
|
|||||||
|
|
||||||
待完成:
|
待完成:
|
||||||
|
|
||||||
- 将官方同步下载结果作为用户级流程自动导入 CAS + ResourceRepository。
|
- 将 `crowdin-textunit-queue.json` 接入真实 Crowdin worker、翻译记忆和 Patch 构建;继续扩展 ResourceRepository 对翻译任务状态和 CAS 诊断的查询面。
|
||||||
- 真实线上全量下载 smoke 已固化为 `scripts/official-full-pull-smoke.sh` 和 `make official-smoke`;实际运行报告由脚本写入隔离输出目录。
|
- 真实线上全量下载 smoke 已固化为 `scripts/official-full-pull-smoke.sh` 和 `make official-smoke`;实际运行报告由脚本写入隔离输出目录。
|
||||||
- 增加更多权限和极端文件系统场景测试。
|
- 增加更多权限和极端文件系统场景测试。
|
||||||
|
|
||||||
### `bat-assetbundle`
|
### `bat-assetbundle`
|
||||||
|
|
||||||
状态:**占位**
|
状态:**UnityFS 解包、TypeTree 字段读取、TextUnit 提取和 TextAsset patch 发布前置已起步;复杂结构覆盖、重打包与 Patch 发布统一未完成**
|
||||||
|
|
||||||
当前只有:
|
冻结说明:自 2026-07-30 起,解析模块进入维护冻结。冻结期只允许修复编译、测试、clippy、崩溃、错误诊断、真实运行回归和文档不一致;不新增 TypeTree 语义类型、不扩大 UnityFS / AssetBundle / Addressables 解析覆盖、不开放新的写入型解析 RPC/CLI,也不以合成 fixture 宣称新增解析能力。冻结细则见 `docs/reports/PARSER_FREEZE.md`。
|
||||||
|
|
||||||
- Parser trait 仍是占位。
|
当前已有:
|
||||||
- AssetType 仍是占位。
|
|
||||||
- 错误类型骨架可用,但没有完整解析引擎。
|
- `UnityFsParser`、`UnityFsBundle`、`ParsedAssetBundle`、`RawAssetBundle` 等正式类型。
|
||||||
|
- UnityFS header、block info、directory 解析。
|
||||||
|
- block info at end、LZ4/LZMA block info 解压、LZ4/LZMA 数据 block 解压、directory 文件提取、压缩/解压数据区大小和 directory 越界诊断。
|
||||||
|
- Unity serialized file header、type table、TypeTree node 元数据、object table 和 TextAsset bytes 提取。
|
||||||
|
- TypeTree 基础字段 reader 支持标量、string、bytes、array、vector/staticvector 嵌套 `Array` 形态、`List<T>` / `HashSet<T>` 集合 alias、map、PPtr、enum `value__` backing field、`LayerMask` / `BitField` 的 `m_Bits` backing field、嵌套对象、常见固定 Unity 值类型(`Vector2f/3f/4f`、`Quaternionf`、`ColorRGBA`、`Rectf`、`AABB/Bounds/Ray`、`Matrix4x4f`、`Vector2Int/Vector3Int`、`RectInt`、`BoundsInt`、`RangeInt`、`GUID`、`Hash128`)的 leaf 和 direct child TypeTree 形态、unknown fixed-size raw bytes 保留、TypeTree-covered managed reference / `SerializedReference` alias、TypeTree-covered managed reference registry 记录、常见 registry 命名别名(含 `m_ManagedReferences` / `RefIds` / verbose type 字段 / `managedReference*` 与 `serializedReference*` prefixed metadata)、managed-reference payload 命名别名(含 `data` / `value` / `payload` / `object` / `managedReferencePayload` / `referencePayload` / `serializedReferencePayload` / `managedReferenceValue` / `referenceValue` / `serializedReferenceValue` / `managedReferenceObject` / `referenceObject` / `serializedReferenceObject` / `managedReferenceData` / `referenceData` / `serializedData` / `serializedReferenceData`)、managed-reference full typename 拆解和 offset/size 诊断;array/vector/List/HashSet/map 元素与 registry payload 字段会保留独立 field path、offset 和 byte size,字符串元素可作为 patch 输入定位,enum 会暴露为语义化 `{type_name, storage_type, value}`,bit field 会暴露为语义化 `{type_name, storage_type, bits}`,object 字段组合、固定 Unity 值类型、enum、bit_field、unknown fixed-size raw bytes 同长度替换与 TypeTree schema 支撑的 array/vector/List/HashSet/map 已支持整体替换、长度变化和空容器扩容,map entry 的 `first/second` 与 `key/value` 字段命名已有重建回归覆盖。
|
||||||
|
- `TextUnitExtractor` 支持 JSON/CSV/TSV/plain TextAsset 探测、TypeTree 字段字符串提取和 JSONL 输出;TextUnit 明细包含 serialized file、path id、class id、field path、字段 offset/byte size、format、asset name 和上下文。managed-reference registry 的类型名、namespace、assembly 等元数据不会进入翻译文本队列,而是写入 payload TextUnit context;未能聚合成结构化 `references` 的 fallback registry 字段也会按 `RefIds[n]` 等记录前缀或子字段推导 managed-reference metadata 并写入 payload context,避免多条 fallback record 混用类型上下文。
|
||||||
|
- `ResourceImportService` 和 `official-parse-cache.json` 已包含 TextUnit 数量、格式和诊断摘要。
|
||||||
|
- `official-textunit-index.json` 已持久化单条 TextUnit 与解析错误;`parse.text_units` / `parse.errors` RPC 和 CLI 可分页过滤查询。
|
||||||
|
- `bat-adapters` 的 Unity 2021.3 adapter 已改为版本选择薄层,复用 `bat-assetbundle`,避免两套 UnityFS parser。
|
||||||
|
|
||||||
待完成:
|
待完成:
|
||||||
|
|
||||||
- UnityFS header、block、directory、metadata、object table。
|
- 真实 MonoBehaviour、ScriptableObject 版本差异、复杂容器结构调整、unknown 字段结构语义和未见样本驱动的完整 managed reference registry / map entry 变体覆盖;TypeTree-covered managed reference 字段与 registry 记录已可结构化解码,常见 full typename 可拆解为 assembly/namespace/class,不做低保真猜测。
|
||||||
- LZ4/LZMA 解压。
|
- 复杂对象整体结构修改后的发布级 AssetBundle 重打包;UnityFS TextAsset、TypeTree string 字段、managed-reference registry payload 字符串、基础语义字段、enum、bit_field、object 字段组合和 TypeTree schema 支撑的 array/vector/map 整体替换的文件级链路已具备重建后校验,发布级 manifest/apply/diff/rollback 仍需统一。
|
||||||
- TypeTree 解析。
|
- 真实资源 fixture 覆盖对象级解析和文本提取。
|
||||||
- TextAsset、MonoBehaviour、ScriptableObject 解析入口。
|
- 详细补全顺序见 `docs/architecture/assetbundle.md`。
|
||||||
|
|
||||||
### `bat-patch`
|
### `bat-patch`
|
||||||
|
|
||||||
状态:**占位**
|
状态:**通用 Binary/JSON/Text Patch 基础可用;文件级 patch / UnityFS 写入入口已开放,发布级 Patch 仍未完成**
|
||||||
|
|
||||||
当前 Binary Patch 和 JSON Patch 函数会明确返回未实现错误,不具备真实补丁能力。
|
当前已有确定性 Binary hunk diff/apply、RFC 6902 JSON Patch apply、UTF-8 Text Patch、通用 Patch manifest、BLAKE3/size 完整性校验和 rollback 元数据。`patch.apply` RPC 与 `patch-apply` CLI 已可对显式 source/patch/target 文件执行 Binary/JSON/Text patch,并返回 size/BLAKE3 报告;`unityfs.patch_text_asset` / `unityfs.patch_string_field` / `unityfs.patch_field` RPC 和 `unityfs-patch-text-asset` / `unityfs-patch-string-field` / `unityfs-patch-field` CLI 已可对显式 UnityFS bundle 输出目标文件。`unityfs.patch_field` 支持 bool、signed/unsigned integer、float raw bits、string、bytes、enum、bit_field、固定 Unity float/int/hash 值类型的 leaf/direct-child 形态、unknown fixed-size raw bytes 同长度替换、PPtr、object 字段组合和 TypeTree schema 支撑的 array/vector/List/HashSet/map 整体替换语义 JSON 值;array/vector/List/HashSet/map 扩容会复用当前首个元素或 TypeTree data node 的编码 schema,空容器扩容已用合成 fixture 覆盖,嵌套 vector `Array`、`List<T>` 和 `HashSet<T>` 形态、enum、bit_field、unknown fixed-size raw bytes、managed-reference registry `data` 和 `managedReferenceData` payload 字符串已有重建后重解析 fixture。`bat-assetbundle` + `LocalizedPatchService` 已能对 UnityFS TextAsset 执行替换、写 patch manifest、记录 diff/rollback,并发布到独立汉化 release;`LocalizedPatchManifest` 可转换为通用 `bat_patch::PatchManifest`,但现有汉化 manifest 文件格式不强制迁移。
|
||||||
|
|
||||||
待完成:
|
待完成:
|
||||||
|
|
||||||
- Binary diff/apply。
|
- 未见样本驱动的 map entry schema 变化、unknown 字段结构语义、完整 managed reference registry 变体驱动字段修改后的语义重打包。
|
||||||
- JSON Patch apply/validate。
|
- 发布级 `patch build` / `patch rollback` / 通用 manifest 驱动发布;当前文件级写入入口不切换 release,不替代汉化发布流程。
|
||||||
- Patch manifest。
|
- `unityfs.inspect`、复杂 UnityFS 语义编辑和写入型发布工作流仍未开放。
|
||||||
- Integrity check。
|
|
||||||
- Rollback。
|
|
||||||
|
|
||||||
### `bat-ffi`
|
### `bat-ffi`
|
||||||
|
|
||||||
@@ -198,45 +206,50 @@ Rust 侧官方日服资源链路已经从实验验证推进到正式入口:
|
|||||||
|
|
||||||
### Go / API / Web
|
### Go / API / Web
|
||||||
|
|
||||||
状态:**Go 产品入口仍未完成;`bat-api` 已有 typed Rust daemon RPC client、可选 CGO 兼容包装和试验性 `cmd/bat` 骨架**
|
状态:**边界已冻结;资源分发 MVP 已落地。权威细节见 `docs/reports/GO_STATUS.md`。**
|
||||||
|
|
||||||
当前情况:
|
| 角色 | 所有者 | 状态 |
|
||||||
|
|---|---|---|
|
||||||
|
| 同步/运维命令行(近乎全自动) | Rust `bat` | 产品入口 |
|
||||||
|
| 资源 bootstrap / 分发 HTTP | Go `cmd/bat-api` | bootstrap + CDN MVP + RPC 周期刷新/诊断 + readiness |
|
||||||
|
| daemon RPC client | `internal/backendrpc` | 完成 |
|
||||||
|
| 试验 CLI | `cmd/bat` → `bin/bat-go` | 非产品 |
|
||||||
|
| FFI | `internal/ffi` | 可选 |
|
||||||
|
| 空目录 `api/` `pkg/` 等 | 占位 | 无实现 |
|
||||||
|
| Web | `web/` | 空(G-010) |
|
||||||
|
|
||||||
- `internal/backendrpc` 已提供 Go typed Unix socket JSON-RPC client,作为 `bat-api` 调用 Rust daemon 的默认路径。
|
默认 Go 门禁:`make test-go-api`、`make build-go-api`(无 FFI)。
|
||||||
- `internal/ffi/ffi.go` 已存在。
|
|
||||||
- Go CLI 的稳定集成方向仍应优先通过 Rust daemon RPC 或 Rust `bat --json` one-shot 进程边界;`cmd/bat` 目前只是试验性骨架,不代表产品级 CLI 已完成。
|
|
||||||
- `cmd/`、`pkg/`、`api/`、`web/` 仍无可用产品入口,`cmd/bat` 目前只覆盖 `doctor`、`manifest inspect`、`sync plan` 这类最小演示能力。
|
|
||||||
- `go test ./...` 目前只有空测试包结果,`go vet ./...` 可作为基础门禁。
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 4. 已验证结果
|
## 4. 已验证结果
|
||||||
|
|
||||||
本轮复核已运行并通过:
|
近期 Rust 侧复核已运行并通过:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cargo test --workspace --quiet
|
cargo fmt --check
|
||||||
cargo clippy --workspace --all-targets -- -D warnings
|
cargo test --offline --workspace --quiet
|
||||||
go test ./...
|
cargo clippy --offline --workspace --all-targets -- -D warnings
|
||||||
go vet ./...
|
cargo test --offline -p bat-patch --quiet
|
||||||
go build -o /tmp/bat-go-cli ./cmd/bat
|
cargo test --offline -p bat-assetbundle --quiet
|
||||||
target/debug/bat --help
|
cargo test --offline -p bat-infrastructure official_parse --quiet
|
||||||
git diff --check
|
cargo test --offline -p bat-infrastructure dispatch_parse --quiet
|
||||||
|
cargo test --offline -p bat-infrastructure localized_patch --quiet
|
||||||
```
|
```
|
||||||
|
|
||||||
同步确认:
|
本次 bat-api 侧复核已运行并通过:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git status --short --branch
|
env GOCACHE=/tmp/bat-go-cache GOMODCACHE=/tmp/bat-go-modcache make test-go-api
|
||||||
|
env GOCACHE=/tmp/bat-go-cache GOMODCACHE=/tmp/bat-go-modcache make build-go-api
|
||||||
|
env GOCACHE=/tmp/bat-go-cache GOMODCACHE=/tmp/bat-go-modcache go vet ./internal/api/... ./internal/backendrpc/... ./cmd/bat-api/...
|
||||||
```
|
```
|
||||||
|
|
||||||
结果:工作区干净。
|
未执行 / 后置:
|
||||||
|
|
||||||
未执行:
|
- 真实官方全量 smoke 长期运行报告(G-018 命令已固化)。
|
||||||
|
- `bat-api` 对远程长期运行 `bat` / 全量 release 的 SSH 联调(等连接信息)。
|
||||||
- 本次状态更新未执行一次性真实官方网络全量下载 smoke;该流程已由 `docs/guides/official-full-pull-smoke.md` 和 `scripts/official-full-pull-smoke.sh` 固化并关闭(G-018),当前处于长期运行测试阶段,运行报告将在后续提供。
|
- Web(G-010)。
|
||||||
- Go CLI 端到端测试,因为 Go 产品入口尚未完成;`internal/backendrpc` 已有 fake transport 单测覆盖。
|
|
||||||
- Web/API 测试,因为 Web/API 尚未实现。
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -251,6 +264,8 @@ cargo run -p bat-infrastructure --bin bat -- \
|
|||||||
--watch
|
--watch
|
||||||
```
|
```
|
||||||
|
|
||||||
|
资源 HTTP bootstrap / 只读分发入口是 Go `cmd/bat-api`。生产拓扑下它与 Rust `bat` 同环境运行,经 `bat.sock` RPC 获取当前 `resource_root`,不在配置里写死资源目录;本地开发不能全量跑 `bat` 时用 fixture 和 Go 门禁验证。`bat-api` 已补 launcher 资源引导兼容端点和玩家-facing HTTP 控制面(token 鉴权、限流、访问日志、反代 IP 适配、动态 JSON no-store、OpenAPI、管理面板预留),响应只来自已发布 snapshot/RPC,不提供登录、网关、鉴权或完整 package update manifest。
|
||||||
|
|
||||||
生产要求:
|
生产要求:
|
||||||
|
|
||||||
1. 使用独立输出目录,例如 `/var/lib/bluearchive-toolkit/official`。
|
1. 使用独立输出目录,例如 `/var/lib/bluearchive-toolkit/official`。
|
||||||
@@ -265,16 +280,16 @@ cargo run -p bat-infrastructure --bin bat -- \
|
|||||||
|
|
||||||
## 6. 当前阻塞项
|
## 6. 当前阻塞项
|
||||||
|
|
||||||
GitHub issue 状态:#1 已升为 P0;#17 的实现已合入 HEAD,但 issue 本身尚未关闭,需在验收后再同步关闭。其他 open issue 的实时标签以 GitHub 为准。
|
GitHub issue 状态:#1 已关闭;#17 已按 wontfix 关闭(多线程下载入口已移除,下载回归顺序执行并保留指数退避与单调进度上报,子 issue #20–#23 均已关闭)。其他 open issue 的实时标签以 GitHub 为准。
|
||||||
|
|
||||||
下一阶段必须优先完成:
|
下一阶段必须优先完成:
|
||||||
|
|
||||||
1. Issue #1(P0,主体已实现):`bat.sock` Unix socket JSON-RPC 已扩展为面向 Go 服务层的 Rust Resource Backend API。统一 envelope(`ok`、`status`、`error`、`data`、`request_id`)与 `BAT-ERR` 错误码模型已落地;`daemon.*`(status/logs/stop/reload/refresh/doctor)、`resource.*`(state/sync/verify/repair/manifest/list)、`catalog.*`(status/refresh/diff/versions)、`task.*`(status/list/cancel/logs)已实现,长任务返回 `task_id` 可轮询(任务执行器单 worker FIFO,与 watch 循环互斥;任务历史持久化于 `<state-dir>/bat-tasks.json`,daemon 重启后仍可查,中断任务标记 `task_interrupted`);错误码已接入下载、launcher/metadata、server-info/marker 与配置校验路径。剩余:`patch.*` / `unityfs.*`(被引擎阻塞)、`task.create`(按设计由语义方法创建)、`daemon.restart` / `daemon.clean-stable`(由 CLI 侧按进程生命周期显式执行,live RPC 内不做自重启或在线清理)、Redis 任务后端(`.env` 已预留配置键,接入时机另议)。Go 层通过 RPC 调用 Rust backend,不走 FFI(FFI 降级说明见 `docs/architecture/official-resource-backend.md` §7)。
|
1. Issue #1(P0,主体已实现):`bat.sock` Unix socket JSON-RPC 已扩展为面向 Go 服务层的 Rust Resource Backend API。统一 envelope(`ok`、`status`、`error`、`data`、`request_id`)与 `BAT-ERR` 错误码模型已落地;`daemon.*`(status/logs/stop/reload/refresh/doctor)、`resource.*`(state/sync/verify/repair/manifest/list/index)、`parse.*`(status/text_units/errors)、`catalog.*`(status/refresh/diff/versions)、`task.*`(status/list/cancel/logs)、文件级 `patch.apply` 与 `unityfs.patch_text_asset` / `unityfs.patch_string_field` / `unityfs.patch_field` 已实现,长任务返回 `task_id` 可轮询(任务执行器单 worker FIFO,与 watch 循环互斥;任务历史持久化于 `<state-dir>/bat-tasks.json`,daemon 重启后仍可查,中断任务标记 `task_interrupted`);错误码已接入下载、launcher/metadata、server-info/marker 与配置校验路径。剩余:发布级 `patch build` / `patch rollback`、复杂 `unityfs.*` 语义编辑、`task.create`(按设计由语义方法创建)、`daemon.restart` / `daemon.clean-stable`(由 CLI 侧按进程生命周期显式执行,live RPC 内不做自重启或在线清理)、Redis 任务后端(`.env` 已预留配置键,接入时机另议)。Go 层通过 RPC 调用 Rust backend,不走 FFI(FFI 降级说明见 `docs/architecture/official-resource-backend.md` §7)。
|
||||||
2. `bat-api` Go 侧:`internal/backendrpc` typed RPC client 已起步,`cmd/bat` 当前只实现 `doctor`、`manifest inspect` 和 `sync plan` 这类试验性入口,不能视作产品级 CLI;是否继续作为长期产品入口需要单独收敛。
|
2. Go 侧:进度见 `docs/reports/GO_STATUS.md`。G-008 已关闭;`bat-api` 资源 bootstrap/分发 MVP 已落地,已含 `/v1/bootstrap`、`/v1/launcher/bootstrap`、launcher 资源 metadata 兼容、HTTP 鉴权/限流/日志/反代适配、动态 JSON no-store、OpenAPI、管理面板预留、CDN Range/缓存头、RPC 周期刷新和 USERGUIDE 基础章节;剩余为远程服务器全量 release 联调、Rust/Go snapshot contract fixture(用户审核后)、refresh mtime/size 增量缓存和可选持久化。
|
||||||
3. 官方同步结果接入 CAS + ResourceRepository 的用户级工作流(G-011 剩余部分:自动导入触发、schema 迁移、CLI 查询)。
|
3. 文本提取 / 翻译队列 / Patch 输入:`official-textunit-index.json`、`official-textunit-tasks.json` 与 `crowdin-textunit-queue.json` 已生成并可查询 TextUnit 明细;剩余为真实 Crowdin worker、翻译记忆、Patch 构建和翻译任务状态查询。
|
||||||
4. Issue #3(P2):AssetBundle UnityFS 基础解析校验。
|
4. Issue #3(P1):AssetBundle UnityFS 基础解析校验已具备离线和隔离真实样本覆盖;对象级解析继续跟踪 G-005。
|
||||||
5. Issue #2(P2):继续逆向 Addressables catalog,提取 bundle hash/size/CRC 等可校验字段。
|
5. Issue #2(P1):继续逆向 Addressables catalog,提取 bundle hash/size/CRC 等可校验字段。
|
||||||
6. Patch 和翻译系统仍应后置。
|
6. 通用 Binary/JSON/Text Patch 基础已落地;复杂 AssetBundle 重打包和真实翻译系统仍应后置,UnityFS TextAsset patch 发布前置已具备回归测试。
|
||||||
|
|
||||||
非阻塞跟踪项:官方同步长期运行测试正在进行,运行报告将在后续提供。
|
非阻塞跟踪项:官方同步长期运行测试正在进行,运行报告将在后续提供。
|
||||||
|
|
||||||
@@ -284,13 +299,13 @@ GitHub issue 状态:#1 已升为 P0;#17 的实现已合入 HEAD,但 issue
|
|||||||
|
|
||||||
立即任务:
|
立即任务:
|
||||||
|
|
||||||
1. Issue #1 收尾:协议基础设施、最小方法集、`catalog.*`、`task.*`、`resource.repair`、任务持久化、错误码模型与文档(USERGUIDE §5/§6、架构文档 §7)均已完成;剩余 `patch.*`/`unityfs.*`(待引擎)以及 `task.create`、`daemon.restart`、`daemon.clean-stable` 的设计边界确认。
|
1. Issue #1 收尾:协议基础设施、最小方法集、`catalog.*`、`parse.*`、`task.*`、`resource.repair`、文件级 `patch.apply` / `unityfs.patch_text_asset` / `unityfs.patch_string_field` / `unityfs.patch_field`、任务持久化、错误码模型与文档均已完成;剩余发布级 `patch build`/`rollback`、复杂 `unityfs.*` 语义编辑以及 `task.create`、`daemon.restart`、`daemon.clean-stable` 的设计边界确认。
|
||||||
2. 明确 Go 产品入口的边界:是继续推进独立 `bat` CLI,还是保留当前 Rust `bat` 为用户 CLI、Go 只做服务层与 `bat-api`。
|
2. `bat-api` 与远程长期运行的 `bat` / 全量 release 联调(含 `/v1/bootstrap`、`/v1/launcher/bootstrap`、server-info 和 CDN path;issue #19 剩余)。
|
||||||
3. 跟进官方同步长期运行测试,收集并归档运行报告。
|
3. 跟进官方同步长期运行测试报告。
|
||||||
4. 开始 AssetBundle parser 的 UnityFS header/block/directory(issue #3),并继续扩展 Addressables catalog 可校验字段(issue #2)。
|
4. AssetBundle / Addressables(issue #3 / #2);CAS 用户级导入(G-011)。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
- **当前总体完成度**:不再固定写单一百分比,以各模块状态和 issue 为准。
|
- **当前总体完成度**:不再固定写单一百分比,以各模块状态、`GO_STATUS.md` 和 issue 为准。
|
||||||
- **当前基线状态**:Rust 官方资源同步链路已具备可运行闭环;Go `bat-api` 已有 Rust daemon RPC client,但产品级 CLI/API、CAS 用户级导入和完整 AssetBundle 引擎仍未完成。
|
- **当前基线状态**:Rust `bat` 同步闭环可用;Go `bat-api` 资源 bootstrap/分发 MVP + 玩家-facing HTTP 控制面 + launcher 资源引导兼容 + RPC 周期刷新/诊断 + readiness + `backendrpc` 可用;CAS 用户级导入、TextUnit 明细索引/查询、增量 Crowdin 离线队列、通用 Binary/JSON/Text Patch 基础和 UnityFS TextAsset patch 发布前置可用;完整 AssetBundle 重打包未完成。
|
||||||
- **下一工程里程碑**:Rust Resource Backend RPC API 收尾、Go 产品入口收敛、官方同步结果接入 CAS/ResourceRepository、AssetBundle 解析起步。
|
- **下一工程里程碑**:bat-api 联调、真实 Crowdin worker / 翻译记忆、翻译任务状态查询、复杂 AssetBundle 解析和重打包。
|
||||||
|
|||||||
Generated
+13
-9
@@ -66,14 +66,13 @@ checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06"
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "bat-adapters"
|
name = "bat-adapters"
|
||||||
version = "0.2.0"
|
version = "1.0.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"anyhow",
|
"anyhow",
|
||||||
"async-trait",
|
"async-trait",
|
||||||
"base64",
|
"base64",
|
||||||
|
"bat-assetbundle",
|
||||||
"bat-core",
|
"bat-core",
|
||||||
"lz4",
|
|
||||||
"lzma-rs",
|
|
||||||
"serde",
|
"serde",
|
||||||
"serde_json",
|
"serde_json",
|
||||||
"tempfile",
|
"tempfile",
|
||||||
@@ -83,10 +82,13 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "bat-assetbundle"
|
name = "bat-assetbundle"
|
||||||
version = "0.2.0"
|
version = "1.0.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"anyhow",
|
"anyhow",
|
||||||
"hex",
|
"hex",
|
||||||
|
"lz4",
|
||||||
|
"lzma-rs",
|
||||||
|
"md-5",
|
||||||
"serde",
|
"serde",
|
||||||
"serde_json",
|
"serde_json",
|
||||||
"thiserror",
|
"thiserror",
|
||||||
@@ -95,7 +97,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "bat-cas-engine"
|
name = "bat-cas-engine"
|
||||||
version = "0.2.0"
|
version = "1.0.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"anyhow",
|
"anyhow",
|
||||||
"async-trait",
|
"async-trait",
|
||||||
@@ -112,7 +114,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "bat-core"
|
name = "bat-core"
|
||||||
version = "0.2.0"
|
version = "1.0.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"anyhow",
|
"anyhow",
|
||||||
"async-trait",
|
"async-trait",
|
||||||
@@ -125,7 +127,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "bat-ffi"
|
name = "bat-ffi"
|
||||||
version = "0.2.0"
|
version = "1.0.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"bat-adapters",
|
"bat-adapters",
|
||||||
"bat-infrastructure",
|
"bat-infrastructure",
|
||||||
@@ -136,13 +138,15 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "bat-infrastructure"
|
name = "bat-infrastructure"
|
||||||
version = "0.2.0"
|
version = "1.0.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"anyhow",
|
"anyhow",
|
||||||
"async-trait",
|
"async-trait",
|
||||||
"bat-adapters",
|
"bat-adapters",
|
||||||
|
"bat-assetbundle",
|
||||||
"bat-cas-engine",
|
"bat-cas-engine",
|
||||||
"bat-core",
|
"bat-core",
|
||||||
|
"bat-patch",
|
||||||
"blake3",
|
"blake3",
|
||||||
"hex",
|
"hex",
|
||||||
"libc",
|
"libc",
|
||||||
@@ -157,7 +161,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "bat-patch"
|
name = "bat-patch"
|
||||||
version = "0.2.0"
|
version = "1.0.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"anyhow",
|
"anyhow",
|
||||||
"blake3",
|
"blake3",
|
||||||
|
|||||||
+1
-1
@@ -11,7 +11,7 @@ members = [
|
|||||||
]
|
]
|
||||||
|
|
||||||
[workspace.package]
|
[workspace.package]
|
||||||
version = "0.2.0"
|
version = "1.0.0"
|
||||||
edition = "2021"
|
edition = "2021"
|
||||||
authors = ["BlueArchive Toolkit Team"]
|
authors = ["BlueArchive Toolkit Team"]
|
||||||
license = "MIT"
|
license = "MIT"
|
||||||
|
|||||||
+8
-2
@@ -11,9 +11,12 @@
|
|||||||
- `PROJECT_PLAN.md`:完整开发计划和最终目标路线图。
|
- `PROJECT_PLAN.md`:完整开发计划和最终目标路线图。
|
||||||
- `CURRENT_STATUS.md`:当前工作区真实状态。
|
- `CURRENT_STATUS.md`:当前工作区真实状态。
|
||||||
- `docs/reports/CURRENT_GAPS.md`:当前实现缺口和关闭顺序。
|
- `docs/reports/CURRENT_GAPS.md`:当前实现缺口和关闭顺序。
|
||||||
|
- `docs/reports/GO_STATUS.md`:Go 侧边界、约定与组件进度(权威)。
|
||||||
- `docs/guides/official-resource-test-pull.md`:官方资源拉取与自动更新用户指南。
|
- `docs/guides/official-resource-test-pull.md`:官方资源拉取与自动更新用户指南。
|
||||||
- `docs/guides/official-full-pull-smoke.md`:真实官方全量拉取 smoke runbook 和可重复命令。
|
- `docs/guides/official-full-pull-smoke.md`:真实官方全量拉取 smoke runbook 和可重复命令。
|
||||||
- `docs/architecture/official-resource-backend.md`:官方资源后端职责、工作原理和审核说明。
|
- `docs/architecture/official-resource-backend.md`:官方资源后端职责、工作原理和审核说明。
|
||||||
|
- `docs/architecture/resource-release-layout.md`:release 布局、URL 映射、seed 规则、bat-api 分发契约(资源侧逆向权威)。
|
||||||
|
- `docs/architecture/assetbundle.md`:AssetBundle、Addressables、Serialized File、文本提取和 Patch 前置解析路线图。
|
||||||
- `docs/reference/rpc-backend-api.md`:Rust Resource Backend JSON-RPC 稳定 contract。
|
- `docs/reference/rpc-backend-api.md`:Rust Resource Backend JSON-RPC 稳定 contract。
|
||||||
- `CHANGELOG.md`:版本变更记录。
|
- `CHANGELOG.md`:版本变更记录。
|
||||||
- `AGENTS.md`:AI agent 和自动化开发助手长期规则。
|
- `AGENTS.md`:AI agent 和自动化开发助手长期规则。
|
||||||
@@ -40,7 +43,6 @@
|
|||||||
后续建议新增:
|
后续建议新增:
|
||||||
|
|
||||||
- `docs/architecture/cas.md`:CAS 生产级设计。
|
- `docs/architecture/cas.md`:CAS 生产级设计。
|
||||||
- `docs/architecture/assetbundle.md`:AssetBundle 解析设计。
|
|
||||||
- `docs/architecture/translation.md`:翻译系统设计。
|
- `docs/architecture/translation.md`:翻译系统设计。
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -101,14 +103,18 @@
|
|||||||
- CAS V1:原子写入、BLAKE3 校验、引用计数、GC、并发测试和损坏检测。
|
- CAS V1:原子写入、BLAKE3 校验、引用计数、GC、并发测试和损坏检测。
|
||||||
- 文档整理和路线图重制。
|
- 文档整理和路线图重制。
|
||||||
- Rust 官方资源同步闭环:`bat`、`--auto-discover`、`--watch`、`--daemon`、Unix socket JSON-RPC 后台控制、`status`、`stop`、`restart`、`reload`、`refresh`、`logs`、`verify`、`repair`、`doctor`、`clean-stable`、北京时间固定强制刷新、snapshot、manifest audit/repair、官方 seed `.hash` 校验。
|
- Rust 官方资源同步闭环:`bat`、`--auto-discover`、`--watch`、`--daemon`、Unix socket JSON-RPC 后台控制、`status`、`stop`、`restart`、`reload`、`refresh`、`logs`、`verify`、`repair`、`doctor`、`clean-stable`、北京时间固定强制刷新、snapshot、manifest audit/repair、官方 seed `.hash` 校验。
|
||||||
|
- 官方原版资源与汉化产物目录分离:`./bat-resources` 只承载原版 release,`./bat-localized` 承载后续汉化 release;当前官方同步报告 `not_localized`,Patch 发布完成后才进入 `localized`。
|
||||||
|
- 官方 release 会维护 `official-parse-cache.json`,用于跳过未变化资源的重复解析。
|
||||||
- `bat-api/internal/backendrpc` typed Unix socket JSON-RPC client。
|
- `bat-api/internal/backendrpc` typed Unix socket JSON-RPC client。
|
||||||
|
- `cmd/bat-api` 资源分发 HTTP MVP(进度见 `docs/reports/GO_STATUS.md`)。
|
||||||
- 真实官方网络全量拉取 smoke 已固化为 `scripts/official-full-pull-smoke.sh` 和 `make official-smoke`,默认写入 `/tmp` 隔离目录并输出本地运行报告。
|
- 真实官方网络全量拉取 smoke 已固化为 `scripts/official-full-pull-smoke.sh` 和 `make official-smoke`,默认写入 `/tmp` 隔离目录并输出本地运行报告。
|
||||||
- `bat` 运行时 progress log 已覆盖下载已完成计数、单文件下载进度和校验结果摘要。
|
- `bat` 运行时 progress log 已覆盖下载已完成计数、单文件下载进度和校验结果摘要。
|
||||||
- Addressables 当前真实形态 fixture/golden 覆盖。
|
- Addressables 当前真实形态 fixture/golden 覆盖。
|
||||||
|
- 解析补全路线图已固化到 `docs/architecture/assetbundle.md`:解析缓存、Addressables、UnityFS、Serialized 字段级解析、文本提取、CAS 接入和 Patch 发布前置。
|
||||||
- SQLite Resource Repository 和可选无状态 `bat-ffi` JSON 兼容接口。
|
- SQLite Resource Repository 和可选无状态 `bat-ffi` JSON 兼容接口。
|
||||||
|
|
||||||
优先待办:
|
优先待办:
|
||||||
|
|
||||||
- 收敛 Go 产品入口的最终形态,避免把试验性 `cmd/bat` 误当作完成品。
|
- `bat-api` 与全量 release / 服务器 daemon 联调(issue #19 剩余)。
|
||||||
- 将官方同步结果接入 CAS + ResourceRepository 的用户级流程。
|
- 将官方同步结果接入 CAS + ResourceRepository 的用户级流程。
|
||||||
- 推进 AssetBundle UnityFS 引擎级解析。
|
- 推进 AssetBundle UnityFS 引擎级解析。
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
.PHONY: help build build-ffi test clean check fmt lint install dev docker-build docker-up docker-down official-smoke
|
.PHONY: help build build-ffi test clean check fmt lint install dev docker-build docker-up docker-down official-smoke build-go build-go-api build-go-cli test-go test-go-api test-go-ffi test-go-all
|
||||||
|
|
||||||
# 默认目标
|
# 默认目标
|
||||||
.DEFAULT_GOAL := help
|
.DEFAULT_GOAL := help
|
||||||
@@ -28,21 +28,25 @@ build-ffi: ## 构建 bat-ffi release 库(cgo 链接依赖)
|
|||||||
@echo "$(BLUE)Building bat-ffi (release)...$(NC)"
|
@echo "$(BLUE)Building bat-ffi (release)...$(NC)"
|
||||||
cargo build --release -p bat-ffi
|
cargo build --release -p bat-ffi
|
||||||
|
|
||||||
build-go: build-ffi ## 构建 Go 组件
|
build-go: build-go-api ## 构建 Go 默认产物(bat-api bootstrap/分发;同步 CLI 请用 Rust bat)
|
||||||
@echo "$(BLUE)Building Go CLI...$(NC)"
|
|
||||||
|
build-go-api: ## 构建 bat-api(资源 bootstrap/分发 HTTP,无 FFI)
|
||||||
|
@echo "$(BLUE)Building bat-api (resource bootstrap + distribution)...$(NC)"
|
||||||
|
@mkdir -p bin
|
||||||
|
go build -o bin/bat-api ./cmd/bat-api
|
||||||
|
|
||||||
|
build-go-cli: build-ffi ## 构建试验性 Go CLI → bin/bat-go(禁止命名为 bat)
|
||||||
|
@echo "$(BLUE)Building experimental Go CLI as bin/bat-go...$(NC)"
|
||||||
|
@mkdir -p bin
|
||||||
@if [ -f cmd/bat/main.go ]; then \
|
@if [ -f cmd/bat/main.go ]; then \
|
||||||
go build -o bin/bat ./cmd/bat; \
|
go build -o bin/bat-go ./cmd/bat; \
|
||||||
else \
|
else \
|
||||||
echo "$(YELLOW)Go CLI entrypoint not implemented yet, skipping...$(NC)"; \
|
echo "$(YELLOW)experimental cmd/bat missing, skipping...$(NC)"; \
|
||||||
fi
|
fi
|
||||||
|
|
||||||
install: ## 安装到本地
|
install: build-go-api ## 安装 bat-api 到 GOPATH/bin(不安装名为 bat 的 Go 二进制)
|
||||||
@echo "$(BLUE)Installing bat CLI...$(NC)"
|
@echo "$(BLUE)Installing bat-api...$(NC)"
|
||||||
@if [ -f cmd/bat/main.go ]; then \
|
go install ./cmd/bat-api
|
||||||
go install ./cmd/bat; \
|
|
||||||
else \
|
|
||||||
echo "$(YELLOW)Go CLI entrypoint not implemented yet, skipping...$(NC)"; \
|
|
||||||
fi
|
|
||||||
|
|
||||||
# ============================================================================
|
# ============================================================================
|
||||||
# 测试相关
|
# 测试相关
|
||||||
@@ -54,14 +58,17 @@ test-rust: ## 运行 Rust 测试
|
|||||||
@echo "$(BLUE)Running Rust tests...$(NC)"
|
@echo "$(BLUE)Running Rust tests...$(NC)"
|
||||||
cargo test --workspace
|
cargo test --workspace
|
||||||
|
|
||||||
test-go: build-ffi ## 运行 Go 测试
|
test-go: test-go-api ## 默认 Go 门禁(无 FFI;见 GO_STATUS.md)
|
||||||
@echo "$(BLUE)Running Go tests...$(NC)"
|
|
||||||
@if [ -n "$$(go list ./... 2>/dev/null)" ]; then \
|
|
||||||
go test -v ./...; \
|
|
||||||
else \
|
|
||||||
echo "$(YELLOW)No Go packages yet, skipping...$(NC)"; \
|
|
||||||
fi
|
|
||||||
|
|
||||||
|
test-go-api: ## 纯 Go 测试:internal/api + backendrpc
|
||||||
|
@echo "$(BLUE)Running pure Go tests (api + backendrpc)...$(NC)"
|
||||||
|
go test ./internal/api/... ./internal/backendrpc/...
|
||||||
|
|
||||||
|
test-go-ffi: build-ffi ## 含 FFI/试验 CLI 的 Go 测试
|
||||||
|
@echo "$(BLUE)Running Go tests including FFI packages...$(NC)"
|
||||||
|
go test ./...
|
||||||
|
|
||||||
|
test-go-all: test-go-api test-go-ffi ## 全部 Go 测试
|
||||||
bench: ## 运行性能基准测试
|
bench: ## 运行性能基准测试
|
||||||
@echo "$(BLUE)Running benchmarks...$(NC)"
|
@echo "$(BLUE)Running benchmarks...$(NC)"
|
||||||
cargo bench --workspace
|
cargo bench --workspace
|
||||||
|
|||||||
+55
-41
@@ -13,7 +13,7 @@ BlueArchiveToolkit 不是一次性脚本,也不是演示项目。最终交付
|
|||||||
|
|
||||||
1. **CLI 工具**:面向本地用户和自动化任务,覆盖 `doctor`、`sync`、`manifest`、`bundle`、`extract`、`translate`、`patch`、`verify`、`cache`、`serve` 等命令。
|
1. **CLI 工具**:面向本地用户和自动化任务,覆盖 `doctor`、`sync`、`manifest`、`bundle`、`extract`、`translate`、`patch`、`verify`、`cache`、`serve` 等命令。
|
||||||
2. **Rust 核心引擎**:负责 CAS、AssetBundle 解析、Patch、二进制安全处理和性能敏感逻辑。
|
2. **Rust 核心引擎**:负责 CAS、AssetBundle 解析、Patch、二进制安全处理和性能敏感逻辑。
|
||||||
3. **Go 服务层**:负责 CLI 编排、资源同步、下载器、API Server、任务调度和外部集成。
|
3. **Go 服务层**:负责资源分发 API、服务编排、任务调度和外部集成;官方资源同步/运维命令行当前由 Rust `bat` 承担,Go 通过 RPC 调用。
|
||||||
4. **Web 管理后台**:负责翻译审核、术语管理、全文搜索、历史版本、Diff 和 Dashboard。
|
4. **Web 管理后台**:负责翻译审核、术语管理、全文搜索、历史版本、Diff 和 Dashboard。
|
||||||
5. **SDK/API**:提供稳定的 Go SDK、进程边界和 REST/OpenAPI 接口,方便其他工具复用;FFI 仅保留为可选兼容层。
|
5. **SDK/API**:提供稳定的 Go SDK、进程边界和 REST/OpenAPI 接口,方便其他工具复用;FFI 仅保留为可选兼容层。
|
||||||
6. **插件系统**:允许新增解析器、翻译 Provider、存储后端、Patch 算法,而不修改核心代码。
|
6. **插件系统**:允许新增解析器、翻译 Provider、存储后端、Patch 算法,而不修改核心代码。
|
||||||
@@ -32,29 +32,31 @@ BlueArchiveToolkit 不是一次性脚本,也不是演示项目。最终交付
|
|||||||
4. `bat-cas-engine` 已完成 CAS V1:原子写入、BLAKE3 Hash、SQLite 引用计数、GC、并发测试、损坏检测。
|
4. `bat-cas-engine` 已完成 CAS V1:原子写入、BLAKE3 Hash、SQLite 引用计数、GC、并发测试、损坏检测。
|
||||||
5. `bat-infrastructure` 已改为 CAS 仓储适配层,不再重复实现对象存储。
|
5. `bat-infrastructure` 已改为 CAS 仓储适配层,不再重复实现对象存储。
|
||||||
6. `bat-infrastructure` 已提供官方资源 pull/update 服务,正式入口是 Rust binary `bat`。
|
6. `bat-infrastructure` 已提供官方资源 pull/update 服务,正式入口是 Rust binary `bat`。
|
||||||
7. `bat` 支持 `--auto-discover`、`--watch`、`--daemon`、默认 1 小时间隔、本地 manifest audit/repair、官方 seed `.hash` 校验、snapshot/cache,以及基于 Unix socket JSON-RPC 的 live control/backend 方法(`daemon.status/logs/stop/reload/refresh/doctor`、`resource.sync/verify/repair/state/manifest/list`、`catalog.*`、`task.*`);`restart` 和 `clean-stable` 仍由 CLI 侧按进程生命周期显式执行。
|
7. `bat` 支持 `--auto-discover`、`--watch`、`--daemon`、默认 1 小时间隔、本地 manifest audit/repair、官方 seed `.hash` 校验、snapshot/cache,以及基于 Unix socket JSON-RPC 的 live control/backend 方法(`daemon.status/logs/stop/reload/refresh/doctor`、`resource.sync/verify/repair/state/manifest/list/index`、`parse.status/text_units/errors`、`localized.status`、`catalog.*`、`task.*`);`restart` 和 `clean-stable` 仍由 CLI 侧按进程生命周期显式执行。
|
||||||
8. `bat-ffi` 已提供 Manifest inspect 和官方 sync plan 的可选无状态粗粒度 JSON C ABI helper。
|
8. `bat-ffi` 已提供 Manifest inspect 和官方 sync plan 的可选无状态粗粒度 JSON C ABI helper。
|
||||||
9. 文档已整理:根目录保留入口文档,历史报告进入 `docs/reports/historical/`,误嵌套的 `docs/docs` 已合并。
|
9. 官方原版资源默认发布到 `./bat-resources`,汉化产物默认发布到独立的 `./bat-localized`;当前官方同步报告会标记 `localized_release_status=not_localized`,表示原版资源已发布、汉化资源未发布。
|
||||||
|
10. 官方同步校验完成并发布新 release 后会生成 `official-resource-changes.json`、`crowdin-translation-handoff.json`、`official-parse-cache.json`、`official-textunit-index.json`、`official-textunit-tasks.json` 和 `crowdin-textunit-queue.json`,用 Added/Modified 资源驱动后续解析/翻译增量;up-to-date 轮询在已有有效缓存、TextUnit 明细索引和队列时只读取摘要,不重复解析。
|
||||||
|
11. 文档已整理:根目录保留入口文档,历史报告进入 `docs/reports/historical/`,误嵌套的 `docs/docs` 已合并。
|
||||||
|
|
||||||
### 仍是骨架或占位
|
### 仍是骨架或占位
|
||||||
|
|
||||||
1. `bat-assetbundle` 仍是占位 crate;完整 UnityFS、压缩块、TypeTree 或对象表解析未完成。
|
1. `bat-assetbundle` 已具备 UnityFS 解包和 TextAsset 提取基础能力(header/block info/directory、LZ4/LZMA block info 与数据 block、directory 文件提取、serialized file object table、TypeTree node 元数据、TextAsset bytes、TypeTree-covered managed reference payload TextUnit 上下文),并已有 UnityFS TextAsset patch 前置能力;MonoBehaviour/ScriptableObject 复杂字段级解析、重打包和通用 Patch 仍未完成。
|
||||||
2. `bat-patch` 的 Binary/JSON 模块仍返回明确的未实现错误,不具备真实补丁能力。
|
2. `bat-patch` 已具备确定性 Binary hunk diff/apply、RFC 6902 JSON Patch apply、UTF-8 Text Patch、通用 Patch manifest、BLAKE3/size 完整性校验和 rollback 元数据;文件级 `patch.apply` RPC / `patch-apply` CLI 与 UnityFS TextAsset / TypeTree string / TypeTree 语义字段写入入口已开放,当前发布级可用的是 `bat-assetbundle` + `LocalizedPatchService` 的 UnityFS TextAsset patch 前置链路。
|
||||||
3. Go CLI/API/SDK 仍没有产品级入口;当前 `bat-api` module 只有 `internal/backendrpc` Rust daemon RPC client、`cmd/bat` 试验骨架与 `internal/ffi` 兼容包装。
|
3. Go 侧边界已冻结(见 `docs/reports/GO_STATUS.md`):同步/运维命令行 = Rust `bat`;资源分发 = `cmd/bat-api` MVP;`internal/backendrpc` 完成;`cmd/bat` 仅为试验(`bin/bat-go`)。完整游戏业务 API / Web / SDK 仍未完成。
|
||||||
4. Addressables parser 已覆盖当前真实形态 fixture/golden,但还不是完整 Unity Addressables/SBP catalog 兼容层。
|
4. Addressables parser 已覆盖当前真实形态 fixture/golden,但还不是完整 Unity Addressables/SBP catalog 兼容层。
|
||||||
5. 官方同步结果尚未作为用户级流程自动导入 CAS + ResourceRepository。
|
5. 官方同步结果可配置为发布后自动导入 CAS + ResourceRepository,并通过 `resource.index` RPC 查询;Resource metadata 已保存 release、平台、bundle path、parse status、TextAsset 名称和 TextUnit 摘要;单条 TextUnit 明细和解析错误已持久化到 `official-textunit-index.json`,可通过 `parse.text_units` / `parse.errors` 查询,翻译任务状态仍需继续推进。
|
||||||
6. 真实官方网络全量下载 smoke 已固化为可重复脚本和 runbook(G-018 已关闭);真实运行记录处于长期运行测试阶段,报告待后续提供。
|
6. 汉化 Patch 发布前置已具备 UnityFS TextAsset manifest/apply/diff/rollback/完整性校验和 `localized.status` 严格校验;真实 Crowdin worker、翻译记忆到完整汉化文件集合的构建仍未完成。
|
||||||
7. Web、数据库迁移、OpenAPI、插件加载机制尚未实现。
|
7. 真实官方网络全量下载 smoke 已固化为可重复脚本和 runbook(G-018 已关闭);真实运行记录处于长期运行测试阶段,报告待后续提供。
|
||||||
8. 原 Git 历史未恢复;当前仓库以新初始化基线为准。
|
8. Web、数据库迁移、OpenAPI、插件加载机制尚未实现。
|
||||||
|
9. 原 Git 历史未恢复;当前仓库以新初始化基线为准。
|
||||||
|
|
||||||
### 已验证
|
### 已验证
|
||||||
|
|
||||||
1. `cargo test --workspace --quiet` 通过。
|
1. `cargo test --workspace --quiet` 通过。
|
||||||
2. `cargo clippy --workspace --all-targets -- -D warnings` 通过。
|
2. `cargo clippy --workspace --all-targets -- -D warnings` 通过。
|
||||||
3. `go test ./...` 通过,但目前没有 Go 产品级测试覆盖。
|
3. `make test-go-api` / `make build-go-api` 覆盖 `internal/api` 与 `internal/backendrpc`。
|
||||||
4. `go vet ./...` 通过。
|
4. `go vet` 覆盖 bat-api 相关包。
|
||||||
5. `go build -o /tmp/bat-go-cli ./cmd/bat` 通过。
|
5. `target/debug/bat --help`(Rust)可用。
|
||||||
6. `target/debug/bat --help` 可用。
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -141,7 +143,7 @@ BlueArchiveToolkit 不是一次性脚本,也不是演示项目。最终交付
|
|||||||
|
|
||||||
**目标**:完成可长期使用的 Content Addressable Storage。
|
**目标**:完成可长期使用的 Content Addressable Storage。
|
||||||
|
|
||||||
**当前状态**:已完成 CAS V1。Go CLI/API 产品入口尚未完成;`bat-api` 已有 `internal/backendrpc` typed RPC client 和 `cmd/bat` 试验骨架;Rust 继续承载完整资源拉取与更新检查核心逻辑;`bat-ffi` 仅保留为可选兼容层。
|
**当前状态**:已完成 CAS V1。Rust 承载完整资源拉取与更新检查;Go 以 `bat-api` 资源分发 MVP + `backendrpc` 为服务入口(`GO_STATUS.md`);`bat-ffi` 仅可选兼容层。
|
||||||
|
|
||||||
交付物:
|
交付物:
|
||||||
|
|
||||||
@@ -174,11 +176,13 @@ BlueArchiveToolkit 不是一次性脚本,也不是演示项目。最终交付
|
|||||||
2. 资源版本、区域、渠道、远端 URL、Hash、大小、依赖关系模型:**部分完成**。`Resource` 和官方 endpoint/snapshot 模型已扩展;仍需冻结 Go CLI/API 可见模型。
|
2. 资源版本、区域、渠道、远端 URL、Hash、大小、依赖关系模型:**部分完成**。`Resource` 和官方 endpoint/snapshot 模型已扩展;仍需冻结 Go CLI/API 可见模型。
|
||||||
3. Rust 官方下载器:**已完成当前生产入口需要的核心能力**。包含官方 URL 校验、`.part` 续传、重试、本地 manifest size+BLAKE3 校验、官方 seed `.hash` 校验和 repair。
|
3. Rust 官方下载器:**已完成当前生产入口需要的核心能力**。包含官方 URL 校验、`.part` 续传、重试、本地 manifest size+BLAKE3 校验、官方 seed `.hash` 校验和 repair。
|
||||||
4. Rust 自动更新入口:**已完成当前生产入口**。`bat` 支持 snapshot、marker diff、bootstrap cache、one-shot、`--watch`、`--daemon`、默认 1 小时间隔、北京时间固定强制刷新,以及 Unix socket JSON-RPC 后台运维命令返回。
|
4. Rust 自动更新入口:**已完成当前生产入口**。`bat` 支持 snapshot、marker diff、bootstrap cache、one-shot、`--watch`、`--daemon`、默认 1 小时间隔、北京时间固定强制刷新,以及 Unix socket JSON-RPC 后台运维命令返回。
|
||||||
5. Go 产品入口:**未完成**。`internal/backendrpc` 已提供 Go 调 Rust daemon 的 typed RPC client;当前 `cmd/bat` 仅有 `doctor`、`manifest inspect`、`sync plan` 试验能力,尚不构成产品级 CLI;若要继续由 Go 承担用户入口,需要单独收敛命令集和调用边界。
|
5. Go 入口边界:**已冻结**。同步命令行 = Rust `bat`(G-008 关闭);资源分发 = `bat-api` MVP(G-009 部分完成)。详见 `docs/reports/GO_STATUS.md`。
|
||||||
6. 用户级 `sync`、`manifest inspect`、`cache status`:**未完成**。Rust `bat --json` 是当前稳定进程边界;`bat-ffi` 只提供可选兼容用的 Manifest inspect 和 sync plan JSON helper。
|
6. 用户级 `sync`、`manifest inspect`、`cache status`:**未完成**。Rust `bat --json` 是当前稳定进程边界;`bat-ffi` 只提供可选兼容用的 Manifest inspect 和 sync plan JSON helper。
|
||||||
7. 下载结果写入 CAS + ResourceRepository:**部分完成**。CAS 和 SQLite ResourceRepository 已存在,官方同步入口尚未把完整下载结果自动作为用户级流程导入。
|
7. 下载结果写入 CAS + ResourceRepository:**部分完成**。CAS 和 SQLite ResourceRepository 已存在,官方同步入口可用 `--import-repository` / `BAT_IMPORT_REPOSITORY=1` 在已校验 release 发布后导入;`resource.index` 可查询现有索引和资源 metadata;`parse.text_units` / `parse.errors` 可查询当前 release 的 TextUnit 明细与解析错误。剩余工作是翻译任务状态、CAS 诊断入口和更丰富查询。
|
||||||
8. Linux 生产同步不依赖已安装官方启动器:**已完成当前 Rust 入口**。`--auto-discover` 只使用官方 HTTP metadata 和临时目录解析 `GameMainConfig`。
|
8. Linux 生产同步不依赖已安装官方启动器:**已完成当前 Rust 入口**。`--auto-discover` 只使用官方 HTTP metadata 和临时目录解析 `GameMainConfig`。
|
||||||
9. 真实官方网络全量下载 smoke test:**命令已固化(G-018 已关闭)**。`scripts/official-full-pull-smoke.sh` / `make official-smoke` 已固化 dry-run、首次下载、二次 up-to-date 和本地损坏 repair 的可重复流程;真实运行处于长期运行测试阶段,报告待后续提供。
|
9. 真实官方网络全量下载 smoke test:**命令已固化(G-018 已关闭)**。`scripts/official-full-pull-smoke.sh` / `make official-smoke` 已固化 dry-run、首次下载、二次 up-to-date 和本地损坏 repair 的可重复流程;真实运行处于长期运行测试阶段,报告待后续提供。
|
||||||
|
10. 官方发布后的增量 handoff 与解析缓存:**已完成基础入口**。新 release 发布后先生成 `official-resource-changes.json` 和 `crowdin-translation-handoff.json`,新增+变更资源进入解析/翻译候选;`official-parse-cache.json` 基于下载 manifest 覆盖直接 UnityFS bundle、zip 内 UnityFS 条目和非候选资源记录;随后生成 `official-textunit-index.json`、`official-textunit-tasks.json` 与 `crowdin-textunit-queue.json`,本地文件未变化且缓存/索引有效时跳过重复解析。
|
||||||
|
11. 汉化发布状态:**已完成前置闭环**。官方同步默认报告 `not_localized`,表示只发布原版资源;UnityFS TextAsset patch 发布成功并通过 `localized-patch-manifest.json`、current symlink 和 release ID 校验后才切换为 `localized`。
|
||||||
|
|
||||||
验收标准:
|
验收标准:
|
||||||
|
|
||||||
@@ -190,6 +194,9 @@ BlueArchiveToolkit 不是一次性脚本,也不是演示项目。最终交付
|
|||||||
6. 自动更新入口必须做到无变化不下载,有变化下载成功后才写入新 snapshot。
|
6. 自动更新入口必须做到无变化不下载,有变化下载成功后才写入新 snapshot。
|
||||||
7. `--watch` 模式必须在 Rust 内部保持持久检查能力,外部 supervisor 只负责进程守护。
|
7. `--watch` 模式必须在 Rust 内部保持持久检查能力,外部 supervisor 只负责进程守护。
|
||||||
8. 真实官方网络 smoke 必须记录输出目录、命令、结果摘要和未纳入仓库的大文件位置。
|
8. 真实官方网络 smoke 必须记录输出目录、命令、结果摘要和未纳入仓库的大文件位置。
|
||||||
|
9. 官方原版资源目录和汉化产物目录必须物理分离,不能相同或互相嵌套。
|
||||||
|
10. 官方同步完成后必须能区分 `not_localized` 和 `localized`,不能把原版资源发布状态与汉化产物发布状态混为一谈。
|
||||||
|
11. 新 release 发布后必须能产出可审计的资源变更集,新增+变更资源进入解析/翻译 handoff,Crowdin 调用由后续翻译 worker 消费本地 handoff 决定。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -199,19 +206,25 @@ BlueArchiveToolkit 不是一次性脚本,也不是演示项目。最终交付
|
|||||||
|
|
||||||
交付物:
|
交付物:
|
||||||
|
|
||||||
1. 解析 UnityFS header、blocks、directory、metadata、objects。
|
1. **解析缓存闭环**:官方同步发布后生成 `official-parse-cache.json`,覆盖 manifest 全部条目、直接 bundle、zip 内 bundle、非候选资源和解析失败诊断;未变化文件按 URL、相对路径、size 和 BLAKE3 复用解析结果。
|
||||||
2. 支持 LZ4/LZMA 解压,记录压缩块校验。
|
2. **Addressables 完整化**:覆盖 Windows/Android JSON、compact JSON 和后续二进制 catalog 入口,解析 provider、internal id、primary key、dependency、bundle name、hash、size、CRC 和资源类型。
|
||||||
3. 实现 TypeTree/ObjectInfo 读取。
|
3. **UnityFS 容器层**:继续完善 header、block info、directory、data block、压缩、alignment、边界错误、directory 文件提取和真实样本回归。
|
||||||
4. 实现 TextAsset、MonoBehaviour、ScriptableObject 的可扩展解析入口。
|
4. **Serialized file 层**:稳定 Unity serialized file header、type table、TypeTree node、object table、path id、class id 和 raw object bytes 表示。
|
||||||
5. 增加解析器注册表和版本适配器。
|
5. **字段级解析层**:实现 TypeTree 字段 reader,支持 bool、integer、float、string、bytes、array、vector/staticvector 嵌套 `Array`、`List<T>` / `HashSet<T>` 集合 alias、map、PPtr、enum `value__` backing field、`LayerMask` / `BitField` 的 `m_Bits` backing field、常见固定 Unity float/int/hash 值类型的 leaf/direct-child 形态、unknown fixed-size raw bytes 保留和同长度替换、TypeTree-covered managed reference / `SerializedReference` alias 和 TypeTree-covered managed reference registry 记录;managed-reference full typename 可拆为 assembly/namespace/class,常见 `m_ManagedReferences` / `RefIds` / verbose type 字段命名、`managedReference*` / `serializedReference*` metadata 和 `data` / `value` / `payload` / `object` / `managedReferencePayload` / `referencePayload` / `serializedReferencePayload` / `managedReferenceValue` / `referenceValue` / `serializedReferenceValue` / `managedReferenceObject` / `referenceObject` / `serializedReferenceObject` / `managedReferenceData` / `referenceData` / `serializedData` / `serializedReferenceData` payload 命名已有回归覆盖,TextUnit 只提取 payload 字符串并按结构化 record、`RefIds[n]` 等记录前缀或子字段保留类型上下文;array/vector/List/HashSet/map 元素与 registry payload 字段保留独立 field path、offset 和 byte size,可支撑字符串元素、managed-reference registry payload 字段、基础语义字段 patch、enum/bit_field 语义 patch、固定值类型 patch、unknown fixed-size bytes patch、object 字段组合和 TypeTree schema 支撑的 array/vector/List/HashSet/map 整体变长替换,`first/second` 与 `key/value` map entry schema 已有回归覆盖;解析模块当前处于维护冻结,未见样本驱动的完整 managed reference registry / map entry 变体和 unknown 字段结构语义暂不继续扩展,除非属于冻结规则允许的稳定性修复。
|
||||||
6. 编写 `bundle inspect`、`bundle extract`。
|
6. **文本对象入口**:实现 TextAsset、MonoBehaviour、ScriptableObject 的可扩展提取入口,输出可追溯到 bundle、serialized file、path id 和 field path 的文本定位。
|
||||||
|
7. **工具与接口**:编写 `bundle inspect`、`bundle extract`、`text extract` 的最小稳定入口;CLI/RPC/API 使用解析器输出,不直接耦合解析内部结构。
|
||||||
|
8. **汉化发布前置**:解析结果必须能作为 Patch 输入;Patch 发布阶段才写入 `--localized-output` / `BAT_LOCALIZED_OUTPUT` 配置的汉化输出目录并切换 `localized` 状态。
|
||||||
|
|
||||||
验收标准:
|
验收标准:
|
||||||
|
|
||||||
1. 能解析真实样本或明确结构化测试样本。
|
1. 能解析结构化测试样本、离线回归 fixture 和隔离真实样本。
|
||||||
2. 错误报告包含 bundle 名称、偏移、字段和 Unity 版本。
|
2. 错误报告包含 URL/路径、archive entry、UnityFS directory、object path id、class id、field path、offset 和 Unity 版本。
|
||||||
3. 解析器和业务流程解耦。
|
3. 解析器和业务流程解耦;解析器不直接写 `bat-resources` 或 `bat-localized`。
|
||||||
4. 不支持的 Unity 版本返回明确错误,不做隐式猜测。
|
4. 不支持的 Unity 版本或 TypeTree 结构返回明确错误,不做隐式猜测。
|
||||||
|
5. `official-parse-cache.json` 能跳过未变化资源的重复解析,且不会影响官方原版资源发布。
|
||||||
|
6. 文本提取结果能追溯到原始资源位置,并可作为后续 Patch manifest 输入。
|
||||||
|
|
||||||
|
详细分层路线图见 `docs/architecture/assetbundle.md`。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -284,11 +297,12 @@ BlueArchiveToolkit 不是一次性脚本,也不是演示项目。最终交付
|
|||||||
|
|
||||||
交付物:
|
交付物:
|
||||||
|
|
||||||
1. 实现 Binary Patch、JSON Patch、Text Patch。
|
1. 已实现确定性 Binary hunk diff/apply、RFC 6902 JSON Patch apply 和 UTF-8 Text Patch。
|
||||||
2. 定义 Patch manifest:目标版本、文件列表、Hash、签名、回滚信息。
|
2. 已定义 Patch manifest 基础:目标版本、文件列表、BLAKE3、size 和 rollback 元数据;签名后置。
|
||||||
3. 实现客户端发现、路径校验、备份、应用、回滚。
|
3. 实现客户端发现、路径校验、备份、应用、回滚。
|
||||||
4. 实现 `patch build`、`patch apply`、`patch rollback`、`verify`。
|
4. 实现 `patch build`、`patch apply`、`patch rollback`、`verify`。
|
||||||
5. 实现 dry-run 和安全检查。
|
5. 实现 dry-run 和安全检查。
|
||||||
|
6. 将通用 Patch manifest 与汉化发布流程进一步统一。
|
||||||
|
|
||||||
验收标准:
|
验收标准:
|
||||||
|
|
||||||
@@ -296,6 +310,7 @@ BlueArchiveToolkit 不是一次性脚本,也不是演示项目。最终交付
|
|||||||
2. 任一步失败都能回滚到补丁前状态。
|
2. 任一步失败都能回滚到补丁前状态。
|
||||||
3. 不直接覆盖未经备份的客户端文件。
|
3. 不直接覆盖未经备份的客户端文件。
|
||||||
4. Patch 生成与应用有端到端测试。
|
4. Patch 生成与应用有端到端测试。
|
||||||
|
5. 汉化产物写入 `--localized-output` / `BAT_LOCALIZED_OUTPUT` 配置的独立目录,保留官方相对目录结构;只有完整 Patch 发布并通过校验后才切换为 `localized`。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -366,7 +381,7 @@ BlueArchiveToolkit 不是一次性脚本,也不是演示项目。最终交付
|
|||||||
|
|
||||||
## 5. 推荐执行顺序
|
## 5. 推荐执行顺序
|
||||||
|
|
||||||
近期不要直接跳到 Web 或 AI Provider。项目当前的真实瓶颈是 Go 产品入口边界、资源解析、同步结果进入 CAS/ResourceRepository,以及真实端到端验证。
|
近期不要直接跳到 Web 或 AI Provider。项目当前的真实瓶颈是资源解析、增量变更集进入文本提取/翻译队列、`bat-api` 与全量 release 联调,以及真实端到端验证。
|
||||||
|
|
||||||
建议顺序:
|
建议顺序:
|
||||||
|
|
||||||
@@ -380,15 +395,14 @@ BlueArchiveToolkit 不是一次性脚本,也不是演示项目。最终交付
|
|||||||
|
|
||||||
## 6. 近期具体任务
|
## 6. 近期具体任务
|
||||||
|
|
||||||
优先完善 Rust `bat` 后端,并同步收敛 Go 产品入口边界。当前事实是 Rust `bat` 已承担可用的资源同步/运维入口,`bat-api/internal/backendrpc` 已提供 Go 到 Rust daemon 的本机 RPC client,Go `cmd/bat` 仍只是试验骨架,`bat-api` HTTP 服务仍是独立目标(issue #19 / G-009):
|
优先完善 Rust 解析与资源库接入,并联调 Go 资源分发。边界见 `docs/reports/GO_STATUS.md`:
|
||||||
|
|
||||||
1. 对 issue #17 做验收并关闭或更新范围:多线程下载与指数退避实现已合入,但 GitHub issue 仍 open。
|
1. issue #17 已关闭(顺序下载 + 指数退避)。
|
||||||
2. 继续逆向 Addressables catalog,扩大 bundle hash/size/CRC 等可校验字段覆盖(issue #2)。
|
2. G-008 已决策关闭:同步/运维命令行 = 近乎全自动的 Rust `bat`。
|
||||||
3. 对 AssetBundle/UnityFS 做引擎级解析:header/block/directory/metadata/object table(issue #3 / G-005)。
|
3. G-009 / issue #19:`bat-api` 资源分发 MVP 已落地;优先服务器联调;拉取仍在 Rust `bat`。
|
||||||
4. 将官方同步下载结果接入 CAS + `SqliteResourceRepository` 的用户级流程(G-011)。
|
4. 继续 Addressables(issue #2)与 UnityFS(issue #3 / G-005)。
|
||||||
5. 收敛 Go 产品入口:明确继续推进最小 Go CLI,或把用户 CLI 固化为 Rust `bat` 并把 Go 侧集中到 `bat-api`。
|
5. 将 `crowdin-textunit-queue.json` 接入真实 Crowdin worker、翻译记忆和 Patch 构建。
|
||||||
6. 实现 `bat-api`(仿官方 API 的 Go HTTP 服务,含鉴权/签名验签,issue #19 / G-009)。
|
6. 继续扩展 G-011 剩余查询面:翻译任务状态、`doctor cas` 诊断入口和更丰富 TextUnit 查询。
|
||||||
7. 为 CAS 增加 `doctor cas` 诊断入口。
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -416,7 +430,7 @@ BlueArchiveToolkit 不是一次性脚本,也不是演示项目。最终交付
|
|||||||
处理策略:
|
处理策略:
|
||||||
|
|
||||||
1. Rust 提供稳定引擎能力,并在当前阶段承担可生产运行的官方资源同步 CLI、watch 和 daemon。
|
1. Rust 提供稳定引擎能力,并在当前阶段承担可生产运行的官方资源同步 CLI、watch 和 daemon。
|
||||||
2. Go 的长期职责包括用户命令、最小稳定 CLI、服务编排、网络和 Provider;当前 Go 产品入口尚未完成,不能把 `cmd/bat` 试验骨架视为完成。
|
2. Go 的长期职责包括资源分发 HTTP(`bat-api`)、服务编排、网络和 Provider;同步/运维命令行由近乎全自动的 Rust `bat` 承担。不能把试验性 `cmd/bat` 视为产品 CLI。
|
||||||
3. 跨边界优先进程或 SDK,FFI 只作为可选的粗粒度、无状态、安全、可测试兼容 API。
|
3. 跨边界优先进程或 SDK,FFI 只作为可选的粗粒度、无状态、安全、可测试兼容 API。
|
||||||
4. Rust 不需要被强制写成 Go 调用库;当前 `bat --watch` / `bat --daemon` 是允许长期运行的 Rust 生产任务。
|
4. Rust 不需要被强制写成 Go 调用库;当前 `bat --watch` / `bat --daemon` 是允许长期运行的 Rust 生产任务。
|
||||||
|
|
||||||
@@ -442,9 +456,9 @@ BlueArchiveToolkit 不是一次性脚本,也不是演示项目。最终交付
|
|||||||
|
|
||||||
按最终目标计算,当前总体完成度不再固定写单一百分比,以模块状态和 issue 收敛情况为准。
|
按最终目标计算,当前总体完成度不再固定写单一百分比,以模块状态和 issue 收敛情况为准。
|
||||||
|
|
||||||
已完成的是稳定基线、架构骨架、部分接口、CAS V1 和 Rust 官方资源同步闭环,不是完整产品能力。下一阶段的关键不是继续堆目录,而是把 Go 产品入口边界、官方同步端到端验证、CAS/ResourceRepository 编排和 AssetBundle 解析链路做实。
|
已完成的是稳定基线、架构骨架、部分接口、CAS V1、Rust 官方资源同步闭环、可配置 CAS/ResourceRepository 导入、TextUnit 明细索引/查询、增量 Crowdin 离线队列、通用 Binary/JSON/Text Patch 基础、UnityFS TextAsset patch 发布前置,以及 Go `bat-api` 资源分发 MVP。下一阶段的关键是真实 Crowdin worker、翻译记忆、复杂 AssetBundle 解析/重打包,以及 bat-api 与全量 release 联调。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
- **下一份应更新文档**:真实官方网络 smoke 记录
|
- **下一份应更新文档**:真实官方网络 smoke 记录
|
||||||
- **下一项工程任务**:收敛 Go 产品入口边界、执行官方同步端到端 smoke,并推进 CAS/ResourceRepository 与 AssetBundle 解析。
|
- **下一项工程任务**:执行官方同步端到端 smoke,推进真实 Crowdin worker / 翻译记忆、复杂 AssetBundle 解析和 bat-api 全量 release 联调。
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
**BlueArchiveToolkit** 是一个面向长期维护的 Blue Archive 资源管理、解析、翻译和补丁工具套件。
|
**BlueArchiveToolkit** 是一个面向长期维护的 Blue Archive 资源管理、解析、翻译和补丁工具套件。
|
||||||
|
|
||||||
当前仓库仍不是完整产品,但 Rust 侧已经具备一条可运行的官方日服资源同步链路:可以在 Linux 上通过官方 HTTP metadata 自动发现资源入口,拉取 Windows + Android 官方资源,保存同步 snapshot,校验本地下载清单,并用 `--watch` 常驻定期检查更新。Go module 名为 `bat-api`,目前包含试验性的 `cmd/bat` 骨架、`internal/backendrpc` Rust daemon RPC client 和 `internal/ffi` 兼容包装;产品级 CLI、API Server、Web、完整 AssetBundle 解析、翻译系统和 Patch 系统仍在后续阶段。
|
当前仓库仍不是完整产品,但 Rust 侧已经具备一条可运行的官方日服资源同步链路:可以在 Linux 上通过官方 HTTP metadata 自动发现资源入口,拉取 Windows + Android 官方资源,保存同步 snapshot,校验本地下载清单,并用近乎全自动的 `--watch` / `--daemon` 常驻更新。Go module 名为 `bat-api`:正式 Go 入口是资源 bootstrap + 分发服务 `cmd/bat-api`(与 Rust `bat` 同环境运行,经 `bat.sock` RPC 周期发现 release 和 `resource_root`,提供 `/v1/bootstrap`、server-info 改写和 CDN path 只读分发);`internal/backendrpc` 为 RPC client;`cmd/bat` 仅为试验骨架(产物 `bin/bat-go`,不是产品 CLI)。边界与进度见 [`docs/reports/GO_STATUS.md`](docs/reports/GO_STATUS.md)。完整游戏业务 API、Web、AssetBundle 引擎、翻译和 Patch 仍在后续阶段。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -10,27 +10,33 @@
|
|||||||
|
|
||||||
- Rust workspace 和 monorepo 结构。
|
- Rust workspace 和 monorepo 结构。
|
||||||
- `bat-core` 领域对象和仓储接口骨架。
|
- `bat-core` 领域对象和仓储接口骨架。
|
||||||
- `bat-adapters` Unity、Manifest、Client 集成框架,以及当前真实形态 Addressables catalog 解析覆盖,含 `m_Crc` 提取和 UnityFS 基础校验。
|
- `bat-adapters` Unity、Manifest、Client 集成框架,以及当前真实形态 Addressables catalog 解析覆盖,含 `m_Crc` 提取和 UnityFS 解包/TextAsset 提取基础校验。
|
||||||
- `bat-cas-engine` CAS V1:原子写入、BLAKE3 校验、引用计数、GC、并发写入测试、损坏检测。
|
- `bat-cas-engine` CAS V1:原子写入、BLAKE3 校验、引用计数、GC、并发写入测试、损坏检测。
|
||||||
- `bat-infrastructure` CAS 适配层、SQLite Resource Repository、资源导入服务、官方资源 pull/update 服务。
|
- `bat-infrastructure` CAS 适配层、SQLite Resource Repository、资源导入服务、官方资源 pull/update 服务。
|
||||||
- `bat`:官方资源自动发现、全量拉取、原子发布到 `current -> versions/<id>`、本地 manifest audit/repair、`.part` 断点续传、403/404/5xx 分类重试、指数退避、顺序下载、下载 quarantine 诊断、ZIP 结构校验、官方 seed `.hash` 校验、snapshot/cache、`--watch` 常驻更新、`--daemon` 后台运行,以及 Unix socket JSON-RPC live control/backend 方法(`daemon.status/logs/stop/reload/refresh/doctor`、`resource.sync/verify/repair/state/manifest/list`、`catalog.*`、`task.*`)。
|
- `bat`:官方资源自动发现、全量拉取、原子发布到 `current -> versions/<id>`、本地 manifest audit/repair、`.part` 断点续传、403/404/5xx 分类重试、指数退避、顺序下载、下载 quarantine 诊断、ZIP 结构校验、官方 seed `.hash` 校验、snapshot/cache、版本化 `official-launcher-bootstrap.json`、`--watch` 常驻更新、`--daemon` 后台运行,以及 Unix socket JSON-RPC live control/backend 方法(`daemon.status/logs/stop/reload/refresh/doctor`、`resource.sync/verify/repair/state/manifest/list/index`、`parse.status/text_units/errors`、`localized.status`、`catalog.*`、`task.*`)。
|
||||||
- `internal/backendrpc`:Go 侧 typed Unix socket JSON-RPC client,是 `bat-api` 调用 Rust daemon 的默认路径。
|
- `internal/backendrpc`:Go 侧 typed Unix socket JSON-RPC client,是 `bat-api` 调用 Rust daemon 的默认路径。
|
||||||
|
- `cmd/bat-api`:资源 bootstrap + 分发 HTTP MVP(issue #19 / G-009);`/v1/bootstrap` 和 `/v1/launcher/bootstrap` 组织 `bat` 已发布 release 的启动前资源入口,launcher 形状兼容端点仅输出资源 metadata / GameMainConfig 引导,`/healthz` 暴露 RPC refresh 诊断,`/readyz` 做 release readiness,CDN path 支持 `GET`/`HEAD`/`Range`、ETag、Last-Modified 和缓存头;玩家-facing 控制面已具备 token 鉴权、限流、访问日志、反代 IP 适配、动态 JSON no-store、OpenAPI 和管理面板预留;`.env` 配置端口/RPC socket/刷新周期;生产资源根来自 RPC,不负责自动拉取。
|
||||||
|
- Go 边界权威说明:[`docs/reports/GO_STATUS.md`](docs/reports/GO_STATUS.md)(G-008 已关闭:同步 CLI = Rust `bat`)。
|
||||||
- 官方同步会维护 `<output>/official-version-state.json`,明确记录当前已完成版本、正在拉取版本、上一个可用版本和失败版本。
|
- 官方同步会维护 `<output>/official-version-state.json`,明确记录当前已完成版本、正在拉取版本、上一个可用版本和失败版本。
|
||||||
- 资源导入链路可将 manifest 条目写入 CAS + `ResourceRepository`,AssetBundle 会记录 UnityFS 摘要,TextAsset/Table/Media 会按类型分类索引。
|
- 资源导入链路可配置为在官方 release 发布后写入 CAS + `ResourceRepository`,资源 metadata 会记录 release、平台、bundle path、parse status、TextAsset 名称和 TextUnit 数量/格式,TextAsset/Table/Media 会按类型分类索引;`resource.index` RPC 可按类型、hash、路径模式分页查询索引。
|
||||||
|
- 新 release 发布后会生成 `official-resource-changes.json`、`official-parse-cache.json`、`official-textunit-index.json`、`official-textunit-tasks.json`、`crowdin-translation-handoff.json` 和 `crowdin-textunit-queue.json`;其中 TextUnit/Crowdin 队列只使用 Added/Modified 资源,不调用 Crowdin 网络 API。
|
||||||
|
- `LocalizedPatchService` 已具备 UnityFS TextAsset patch 发布前置能力:在 `--localized-output` / `BAT_LOCALIZED_OUTPUT` 配置的独立汉化目录 staging 中复制官方 release、应用 TextAsset patch、写 `localized-patch-manifest.json`(hash、size、diff、rollback)、校验后发布到 `versions/<id>` 并切换 `current`。
|
||||||
|
- `bat-patch` 已具备通用 Patch 基础:确定性 Binary hunk diff/apply、RFC 6902 JSON Patch apply、UTF-8 Text Patch、Patch manifest、BLAKE3/size 完整性校验和 rollback 元数据;文件级 `patch.apply` RPC / `patch-apply` CLI 与 UnityFS TextAsset / TypeTree string / TypeTree 语义字段写入入口已开放,TypeTree 语义字段支持基础标量、固定 Unity float/int/hash 值类型的 leaf/direct-child 形态、PPtr、managed-reference registry payload 字符串、object 字段组合、unknown fixed-size raw bytes 同长度替换和 TypeTree schema 支撑的 array/vector/map 整体替换;TextUnit 提取会把 managed-reference 类型信息保留为上下文而非翻译文本,汉化发布当前仍走 UnityFS TextAsset 前置链路。
|
||||||
- `bat-ffi` 可选无状态 C ABI 兼容层:仅保留 Manifest inspect 和官方 sync plan 的粗粒度 JSON helper,不作为 Go CLI 或生产同步的主集成边界。
|
- `bat-ffi` 可选无状态 C ABI 兼容层:仅保留 Manifest inspect 和官方 sync plan 的粗粒度 JSON helper,不作为 Go CLI 或生产同步的主集成边界。
|
||||||
- 文档路线图、当前状态、缺口清单、官方资源运行指南。
|
- 文档路线图、当前状态、缺口清单、官方资源运行指南。
|
||||||
|
|
||||||
仍未完成:
|
仍未完成:
|
||||||
|
|
||||||
- Go CLI 产品入口(当前仅有试验性 `cmd/bat` 骨架)。
|
- `bat-api` 完整游戏业务 API / launcher 安装包更新全链(资源 CDN、HTTP 控制面和 launcher 资源引导兼容已可用)。
|
||||||
- 完整 UnityFS / AssetBundle 引擎解析。
|
- 完整 AssetBundle 对象级解析(UnityFS 解包、object table、TypeTree node 元数据、TextAsset bytes、MonoBehaviour/ScriptableObject 基础 TypeTree 字段级解析已起步;复杂字段覆盖、发布级重打包和 Patch 发布统一仍未完成)。
|
||||||
- 真实 Patch apply/diff。
|
- 复杂 AssetBundle 重打包和真实翻译构建 worker;当前通用 Binary/JSON/Text Patch 基础已在 crate 层可用,发布链路仍只开放 UnityFS TextAsset patch 前置能力。
|
||||||
- Translation Memory、Glossary、AI Provider。
|
- Translation Memory、Glossary、AI Provider。
|
||||||
- API Server、SDK、Web 管理后台。
|
- SDK、Web 管理后台。
|
||||||
|
|
||||||
详细状态见:
|
详细状态见:
|
||||||
|
|
||||||
- [当前状态](CURRENT_STATUS.md)
|
- [当前状态](CURRENT_STATUS.md)
|
||||||
|
- [Go 侧进度与边界](docs/reports/GO_STATUS.md)
|
||||||
- [完整开发计划](PROJECT_PLAN.md)
|
- [完整开发计划](PROJECT_PLAN.md)
|
||||||
- [文档索引](DOCS_INDEX.md)
|
- [文档索引](DOCS_INDEX.md)
|
||||||
- [当前缺口清单](docs/reports/CURRENT_GAPS.md)
|
- [当前缺口清单](docs/reports/CURRENT_GAPS.md)
|
||||||
@@ -81,7 +87,7 @@ cargo run -p bat-infrastructure --bin bat -- \
|
|||||||
--error-retry 60s
|
--error-retry 60s
|
||||||
```
|
```
|
||||||
|
|
||||||
后台自动运行可以把 `--watch` 换成 `--daemon`。默认资源目录是 `./bat-resources`,默认后台状态目录是 `/tmp/bat-pid`。daemon 会在状态目录下创建 `bat.sock` 作为 Unix socket JSON-RPC 控制通道,同时写入 `bat.pid`、`bat-status.json`、`bat-daemon.log`、`bat-events.jsonl` 和短生命周期的 `bat-control.lock`;`bat-events.jsonl` 是带轮转的结构化 JSONL 事件日志,`bat-status.json` 保存最后成功时间、下次检查时间、最后错误摘要和当前下载进度,`bat-control.lock` 用于串行化 `status/stop/restart/reload/logs/refresh/repair` 等控制命令:
|
后台自动运行可以把 `--watch` 换成 `--daemon`。默认官方原版资源目录是 `./bat-resources`,默认汉化产物目录是 `./bat-localized`,默认后台状态目录是 `/tmp/bat-pid`。daemon 会在状态目录下创建 `bat.sock` 作为 Unix socket JSON-RPC 控制通道,同时写入 `bat.pid`、`bat-status.json`、`bat-daemon.log`、`bat-events.jsonl` 和短生命周期的 `bat-control.lock`;`bat-events.jsonl` 是带轮转的结构化 JSONL 事件日志,`bat-status.json` 保存最后成功时间、下次检查时间、最后错误摘要和当前下载进度,`bat-control.lock` 用于串行化 `status/stop/restart/reload/logs/refresh/repair` 等控制命令:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cargo run -p bat-infrastructure --bin bat -- \
|
cargo run -p bat-infrastructure --bin bat -- \
|
||||||
@@ -99,7 +105,7 @@ cargo run -p bat-infrastructure --bin bat -- stop
|
|||||||
|
|
||||||
`bat` 会拒绝危险输出目录、路径逃逸和现有 symlink 路径组件;下载目标、`.part`、manifest、snapshot、PID、status、log 和控制锁文件不会跟随 symlink,daemon 状态类文件默认以 `0600` 权限创建。
|
`bat` 会拒绝危险输出目录、路径逃逸和现有 symlink 路径组件;下载目标、`.part`、manifest、snapshot、PID、status、log 和控制锁文件不会跟随 symlink,daemon 状态类文件默认以 `0600` 权限创建。
|
||||||
|
|
||||||
非 dry-run 同步不会把新文件直接写进生产可读目录。资源会先下载到 `<output>/.staging/<id>`,完成 manifest、BLAKE3、ZIP 和官方 `.hash` 校验后移动到 `<output>/versions/<id>`,再原子切换 `<output>/current` symlink;生产读取方应只读取 `<output>/current`。同步过程会更新 `<output>/official-version-state.json`:下载开始时写入 `in_progress_version`,发布成功后写入 `current_completed_version` 和 `previous_available_version`,失败或中断时写入 `failed_versions`。
|
非 dry-run 同步不会把新文件直接写进生产可读目录。官方原版资源会先下载到 `<output>/.staging/<id>`,完成 manifest、BLAKE3、ZIP 和官方 `.hash` 校验后移动到 `<output>/versions/<id>`,再原子切换 `<output>/current` symlink;生产读取方应只读取 `<output>/current`。同步过程会更新 `<output>/official-version-state.json`:下载开始时写入 `in_progress_version`,发布成功后写入 `current_completed_version` 和 `previous_available_version`,失败或中断时写入 `failed_versions`。启用 `--auto-discover` 时,已发布 release 会写入 `official-launcher-bootstrap.json`,其中包含 launcher metadata、launcher CDN config、remote manifest 文件列表、选中的 `resources.assets` 来源和 `GameMainConfig` 摘要;官方资源端尚未开放时会写 `<output>/official-launcher-bootstrap.pending.json`,但不会切换 `current`。新 release 发布后会对比上一完整 release 的 download manifest,在当前 release 下写入 `official-resource-changes.json`、`crowdin-translation-handoff.json`、`official-parse-cache.json`、`official-textunit-index.json`、`official-textunit-tasks.json` 和 `crowdin-textunit-queue.json`;新增+变更资源作为解析/翻译候选,删除资源只进入差异记录。up-to-date 轮询在已有有效解析缓存、TextUnit 明细索引和 TextUnit 队列时只读取摘要,不重复解析。官方同步报告默认 `localized_release_status=not_localized`,表示原版资源已发布、汉化资源未发布;后续 Patch/导出写入 `--localized-output` / `BAT_LOCALIZED_OUTPUT` 指定的独立目录,保留官方相对目录结构,manifest 校验通过后才切换为 `localized`。
|
||||||
|
|
||||||
资源操作命令默认输出人类可读摘要,并在没有显式 metadata 参数时默认走官方自动发现。脚本或上层程序需要稳定结构化输出时加 `--json`:
|
资源操作命令默认输出人类可读摘要,并在没有显式 metadata 参数时默认走官方自动发现。脚本或上层程序需要稳定结构化输出时加 `--json`:
|
||||||
|
|
||||||
@@ -114,7 +120,7 @@ cargo run -p bat-infrastructure --bin bat -- clean-stable
|
|||||||
|
|
||||||
`verify` 会以只读方式检查当前官方计划、`current` 指向的 active release 中 download manifest 的 size+BLAKE3、ZIP 结构,以及本地已有官方 seed `.bytes/.hash` 对的 xxHash32;发现缺失、远端变化或本地损坏会返回非 0。`repair` 会在异常资源存在时复用当前同步链路重新下载必要文件。`clean-stable` 只清理 `.part`、临时状态文件、失效或损坏的 PID/锁/socket,不删除正式资源。
|
`verify` 会以只读方式检查当前官方计划、`current` 指向的 active release 中 download manifest 的 size+BLAKE3、ZIP 结构,以及本地已有官方 seed `.bytes/.hash` 对的 xxHash32;发现缺失、远端变化或本地损坏会返回非 0。`repair` 会在异常资源存在时复用当前同步链路重新下载必要文件。`clean-stable` 只清理 `.part`、临时状态文件、失效或损坏的 PID/锁/socket,不删除正式资源。
|
||||||
|
|
||||||
下载失败会按 curl exit 和 HTTP 状态分类:403/404/普通 4xx 视为不可重试,5xx、429、DNS、连接、超时、中断和网络类错误会按尝试次数重试。某个 URL 最终失败后会写入 `<output>/current` 或 staging 下的 `official-download-quarantine.json`,stderr progress、daemon status 和 `bat-events.jsonl` 会记录失败类型、HTTP 状态、是否可重试、尝试次数和 quarantine 状态;同步会中断并阻止发布不完整资源。旧 launcher 包下载路径会在官方 primary CDN 失败后切换官方 backup CDN。
|
下载失败会按 curl exit 和 HTTP 状态分类:403/404/普通 4xx 视为不可重试,5xx、429、DNS、连接、超时、中断和网络类错误会按尝试次数重试。若官方启动器/server-info 已先行更新,但 client-patch root 下的 seed marker 或必需 seed catalog 仍返回 403/404/普通 4xx,`bat` 会返回 `update_status=waiting_for_official_resources`,保留现有 `current`,不创建失败 staging,不把维护期记为失败版本;watch/daemon 会按错误重试间隔继续探测。已进入下载阶段的单个资源 URL 最终失败后会写入 `<output>/current` 或 staging 下的 `official-download-quarantine.json`,stderr progress、daemon status 和 `bat-events.jsonl` 会记录失败类型、HTTP 状态、是否可重试、尝试次数和 quarantine 状态;同步会中断并阻止发布不完整资源。旧 launcher 包下载路径会在官方 primary CDN 失败后切换官方 backup CDN。
|
||||||
|
|
||||||
CLI 默认启动时会向 stderr 打印 `BlueArchiveToolkit` ASCII banner,并继续把阶段进度日志写到 stderr,例如自动发现、拉取 catalog、audit、下载已完成计数、单文件下载进度、校验结果摘要、snapshot 和 publish;命令结果默认以人类可读摘要写到 stdout。需要给上层程序保留稳定结构化输出时加 `--json --no-progress`,只想关闭横幅但保留日志时可加 `--no-banner`。
|
CLI 默认启动时会向 stderr 打印 `BlueArchiveToolkit` ASCII banner,并继续把阶段进度日志写到 stderr,例如自动发现、拉取 catalog、audit、下载已完成计数、单文件下载进度、校验结果摘要、snapshot 和 publish;命令结果默认以人类可读摘要写到 stdout。需要给上层程序保留稳定结构化输出时加 `--json --no-progress`,只想关闭横幅但保留日志时可加 `--no-banner`。
|
||||||
|
|
||||||
@@ -129,7 +135,7 @@ make official-smoke
|
|||||||
|
|
||||||
该 smoke 会执行 dry-run plan、首次全量拉取、二次 `up_to_date` 检查、本地文件破坏后的 `repair`、repair 后 `verify`,并在 `report/SMOKE_REPORT.md` 记录命令、输出目录、active release、文件数量、release 大小和被破坏文件。大型官方资源文件不纳入 Git。
|
该 smoke 会执行 dry-run plan、首次全量拉取、二次 `up_to_date` 检查、本地文件破坏后的 `repair`、repair 后 `verify`,并在 `report/SMOKE_REPORT.md` 记录命令、输出目录、active release、文件数量、release 大小和被破坏文件。大型官方资源文件不纳入 Git。
|
||||||
|
|
||||||
生产资源输出目录必须使用独立目录,不要指向已有客户端目录,也不要指向 `/home/wanye/D/BlueArchive` 这类人工维护或开发资源目录。需要覆盖默认位置时,用 `--output <资源目录>`;需要覆盖后台状态目录时,用 `--state-dir <状态目录>`。
|
生产官方资源输出目录和汉化产物目录都必须使用独立目录,不要指向已有客户端目录,也不要指向 `/home/wanye/D/BlueArchive` 这类人工维护或开发资源目录。需要覆盖官方原版资源位置时,用 `--output <资源目录>` 或 `.env` 中的 `BAT_OUTPUT`;需要覆盖汉化产物位置时,用 `--localized-output <目录>` 或 `.env` 中的 `BAT_LOCALIZED_OUTPUT`;需要启用官方 release 导入 CAS/索引时,用 `--import-repository`,并可用 `--import-cas-root`、`--import-resource-db` 或 `.env` 中的 `BAT_IMPORT_CAS_ROOT`、`BAT_IMPORT_RESOURCE_DB` 覆盖默认路径;需要覆盖后台状态目录时,用 `--state-dir <状态目录>`。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -179,7 +185,7 @@ BlueArchiveToolkit/
|
|||||||
1. 收敛 Go CLI 产品入口的最终形态:当前 `cmd/bat` 仅有 `doctor`、`manifest inspect`、`sync plan` 试验能力,不应误写成完整 CLI。
|
1. 收敛 Go CLI 产品入口的最终形态:当前 `cmd/bat` 仅有 `doctor`、`manifest inspect`、`sync plan` 试验能力,不应误写成完整 CLI。
|
||||||
2. 补齐 AssetBundle UnityFS 引擎级解析。
|
2. 补齐 AssetBundle UnityFS 引擎级解析。
|
||||||
3. 扩展 Addressables catalog 解析覆盖,继续用真实形态 fixture/golden 锁定行为。
|
3. 扩展 Addressables catalog 解析覆盖,继续用真实形态 fixture/golden 锁定行为。
|
||||||
4. 将官方同步结果接入 CAS + ResourceRepository 的用户级工作流。
|
4. 将 `official-textunit-tasks.json` / `crowdin-textunit-queue.json` 接入真实 Crowdin worker、翻译记忆和 Patch 构建。
|
||||||
5. 按 smoke runbook 在具备网络和磁盘窗口的环境中执行真实官方全量拉取,并保留本地报告。
|
5. 按 smoke runbook 在具备网络和磁盘窗口的环境中执行真实官方全量拉取,并保留本地报告。
|
||||||
|
|
||||||
不建议在 Go 产品入口、资源解析和文本提取基础能力完成前优先开发 Web UI。
|
不建议在 Go 产品入口、资源解析和文本提取基础能力完成前优先开发 Web UI。
|
||||||
|
|||||||
+89
-2
@@ -11,7 +11,7 @@
|
|||||||
|
|
||||||
`bat` 是 Linux 上官方日服(Yostar JP)资源同步的正式入口。它可以:
|
`bat` 是 Linux 上官方日服(Yostar JP)资源同步的正式入口。它可以:
|
||||||
|
|
||||||
- `--auto-discover` 从官方 HTTP metadata 解析 `GameMainConfig`,自动获得 app-version、连接组和 server-info,不安装、不启动官方启动器。
|
- `--auto-discover` 从官方 HTTP metadata 解析 `GameMainConfig`,自动获得 app-version、连接组和 server-info,不安装、不启动官方启动器;已发布 release 会保存 `official-launcher-bootstrap.json`。
|
||||||
- 生成官方全量 pull plan、执行真实下载,维护 release 内的下载 manifest,并做 size + BLAKE3 复用校验、官方 seed `.hash`(标准 xxHash32(seed=0))强校验、ZIP 结构校验。
|
- 生成官方全量 pull plan、执行真实下载,维护 release 内的下载 manifest,并做 size + BLAKE3 复用校验、官方 seed `.hash`(标准 xxHash32(seed=0))强校验、ZIP 结构校验。
|
||||||
- 断点续传、失败分类重试、下载 quarantine、本地 manifest audit/repair。
|
- 断点续传、失败分类重试、下载 quarantine、本地 manifest audit/repair。
|
||||||
- 原子发布:先写 `.staging/<id>`,校验通过后发布 `versions/<id>` 并原子切换 `current` symlink。
|
- 原子发布:先写 `.staging/<id>`,校验通过后发布 `versions/<id>` 并原子切换 `current` symlink。
|
||||||
@@ -57,6 +57,93 @@ HTTPS_PROXY=http://user:pass@127.0.0.1:7890 bat --auto-discover --daemon
|
|||||||
|
|
||||||
`status`/`stop`/`logs`/`reload` 和默认形态的 `refresh` 优先走 `bat.sock` JSON-RPC;socket 不可用时 `status`/`stop` 回退到 PID/状态文件兼容路径。
|
`status`/`stop`/`logs`/`reload` 和默认形态的 `refresh` 优先走 `bat.sock` JSON-RPC;socket 不可用时 `status`/`stop` 回退到 PID/状态文件兼容路径。
|
||||||
|
|
||||||
|
### bat-api 资源 bootstrap / 分发服务
|
||||||
|
|
||||||
|
`bat-api` 是 Go 侧正式服务入口,用于给客户端、补丁器或上层工具提供启动前资源入口和 CDN 形态只读分发。它不负责自动发现、下载、校验或发布资源;这些长期状态由 Rust `bat` / daemon 持有。
|
||||||
|
|
||||||
|
生产拓扑上,`bat-api` 基本应与 Rust `bat` 运行在同一台服务器、同一容器或同一共享文件系统环境。当前可读资源目录不在 `bat-api` 配置里写死,而是由 `bat.sock` RPC 的 `catalog.status` / `resource.manifest` 返回 `resource_root`。
|
||||||
|
|
||||||
|
推荐运行关系:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 先让 Rust bat 生产并维护 release
|
||||||
|
bat --auto-discover --daemon \
|
||||||
|
--output /var/lib/bluearchive-toolkit/official \
|
||||||
|
--state-dir /var/lib/bluearchive-toolkit/daemon-state
|
||||||
|
|
||||||
|
# 再启动 bat-api 读取同一个 daemon socket
|
||||||
|
bat-api \
|
||||||
|
--listen :18080 \
|
||||||
|
--public-base-url http://127.0.0.1:18080 \
|
||||||
|
--socket /var/lib/bluearchive-toolkit/daemon-state/bat.sock \
|
||||||
|
--refresh-interval 1m
|
||||||
|
```
|
||||||
|
|
||||||
|
测试、fixture 或应急只读诊断场景可用 `--resource-root <DIR>` 直接指向已发布 release 根;生产默认应通过 `--socket` / `BAT_API_SOCKET` 从 `bat.sock` 发现当前版本。`bat.sock` 不应暴露到公网;对外发布时只暴露 `bat-api` HTTP,并把 `--public-base-url` 设为客户端实际访问的 HTTPS 根。
|
||||||
|
|
||||||
|
开发环境不能本地全量运行 `bat` 时,用 fixture 验证 Go 服务面即可:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BAT_API_SKIP_ENV_FILE=1 go run ./cmd/bat-api \
|
||||||
|
--listen 127.0.0.1:18080 \
|
||||||
|
--public-base-url http://127.0.0.1:18080 \
|
||||||
|
--resource-root internal/api/testdata/release \
|
||||||
|
--refresh-interval 0
|
||||||
|
```
|
||||||
|
|
||||||
|
常用接口:
|
||||||
|
|
||||||
|
| 接口 | 说明 |
|
||||||
|
|---|---|
|
||||||
|
| `GET /healthz` | 服务存活、RPC 可用性、release ready 状态和最近一次 RPC refresh 诊断 |
|
||||||
|
| `GET/HEAD /readyz` | release 就绪检查;当前无可分发 release 时返回 `503` |
|
||||||
|
| `GET /v1/bootstrap` | 启动前资源入口:`bat` RPC 健康、release 摘要、server-info URL、client-patch base、改写后的 Addressables root |
|
||||||
|
| `GET /v1/launcher/bootstrap` | 启动器资源引导聚合视图:release、launcher metadata、GameMainConfig 摘要和资源 URL |
|
||||||
|
| `GET /api/launcher/game/config` | launcher 资源 metadata 兼容 envelope;字段来自 Rust `bat` 已发布 snapshot/RPC |
|
||||||
|
| `GET /api/launcher/game/config/json` | launcher 形状的资源引导 JSON URL;不会返回完整 PC package update manifest |
|
||||||
|
| `GET /api/launcher/advanced/game/download/cdn` | launcher 形状的 CDN 配置;返回当前 `--public-base-url`,用于资源引导 |
|
||||||
|
| `GET /api-launcher-jp.yo-star.com/api/launcher/...` | 与上面 `/api/launcher/...` 等价,便于反代或 hosts 映射保持官方 host 形状 |
|
||||||
|
| `GET /v1/release` | 当前 release 摘要 |
|
||||||
|
| `GET /v1/resources?offset=0&limit=100` | 当前 manifest 索引分页 |
|
||||||
|
| `GET /v1/server-info` | 调试用 server-info JSON,只改 `AddressablesCatalogUrlRoot` |
|
||||||
|
| `GET /yostar-serverinfo.bluearchiveyostar.com/server-info.json` | 官方 host/path 形态的 server-info |
|
||||||
|
| `GET/HEAD /prod-clientpatch.bluearchiveyostar.com/...` | 官方 CDN path 形态资源字节 |
|
||||||
|
| `GET /openapi.yaml` | bat-api OpenAPI 文档 |
|
||||||
|
| `GET /admin/` | 管理面板预留入口;当前返回 JSON 链接,未来接入 Web UI |
|
||||||
|
|
||||||
|
launcher 兼容端点只服务启动前资源发现。它们复用 Rust `bat` snapshot 中的 `launcher_metadata` 和 `game_main_config_bootstrap`,显式标记 `scope=resource_bootstrap_only` / `package_update_manifest=false`。`bat-api` 不下载 launcher 包,不生成官方 PC package update manifest,也不仿造登录、账号、网关、鉴权或游戏业务协议。
|
||||||
|
|
||||||
|
生产面对玩家分发时,应启用 HTTP token 鉴权、限流和访问日志:
|
||||||
|
|
||||||
|
- `BAT_API_AUTH_TOKEN`:启用 `Authorization: Bearer <token>`、`X-BAT-Token` 或 query fallback 鉴权;token 推荐由 secret manager 或进程环境提供,不建议写入提交文件。
|
||||||
|
- `BAT_API_AUTH_QUERY_PARAM`:query fallback 参数名,默认 `bat_token`;兼容不能写 header 的客户端,访问日志不会记录 query。
|
||||||
|
- `BAT_API_AUTH_EXEMPT_PATHS`:逗号分隔的免鉴权 path 或 slash-prefix,例如 `/healthz,/readyz`。
|
||||||
|
- `BAT_API_RATE_LIMIT_RPS` / `BAT_API_RATE_LIMIT_BURST`:按客户端 IP 的进程内 token bucket 限流;边缘反代/CDN 仍应配置独立限流。
|
||||||
|
- `BAT_API_TRUST_PROXY_HEADERS`:只有反代已经清洗并覆盖 `X-Forwarded-For` / `X-Real-IP` 时才设为 `true`。
|
||||||
|
- `BAT_API_ACCESS_LOG`:结构化访问日志,记录 method/path/status/bytes/duration/client_ip/request_id/user_agent,不记录 query string。
|
||||||
|
- `BAT_API_MAX_RESOURCE_LIMIT`:`/v1/resources` 最大分页上限,默认 `1000`。
|
||||||
|
|
||||||
|
所有动态 JSON(bootstrap、health、ready、release、resources、launcher 兼容、server-info、OpenAPI、admin placeholder 和错误响应)显式返回 `Cache-Control: no-store`。资源字节 CDN path 仍返回长期 immutable cache header。
|
||||||
|
|
||||||
|
CDN path 只服务 manifest 索引内且磁盘存在、size 匹配的文件。响应支持 `GET`、`HEAD`、`Range`、条件请求、ETag、Last-Modified、Accept-Ranges 和长期缓存头;ETag 优先使用 manifest 中的 BLAKE3。`.hash` 以 `text/plain` 返回,其它未知扩展默认为 `application/octet-stream`。
|
||||||
|
|
||||||
|
`bat-api` 只改写资源相关入口:server-info 中的 `AddressablesCatalogUrlRoot` 会指向 `--public-base-url` 下的 `prod-clientpatch...` path;`ApiUrl`、`GatewayUrl`、登录、账号、网关和游戏业务协议不会被仿造或改写。
|
||||||
|
|
||||||
|
示例:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -fsS http://127.0.0.1:18080/v1/bootstrap
|
||||||
|
curl -fsS http://127.0.0.1:18080/v1/launcher/bootstrap
|
||||||
|
curl -fsS http://127.0.0.1:18080/api-launcher-jp.yo-star.com/api/launcher/game/config
|
||||||
|
curl -fsS http://127.0.0.1:18080/openapi.yaml
|
||||||
|
|
||||||
|
curl -fsS \
|
||||||
|
http://127.0.0.1:18080/prod-clientpatch.bluearchiveyostar.com/<root_token>/TableBundles/TableCatalog.hash
|
||||||
|
|
||||||
|
curl -i -H 'Range: bytes=0-1023' \
|
||||||
|
http://127.0.0.1:18080/prod-clientpatch.bluearchiveyostar.com/<root_token>/TableBundles/TableCatalog.bytes
|
||||||
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 3. 选项
|
## 3. 选项
|
||||||
@@ -115,7 +202,7 @@ HTTPS_PROXY=http://user:pass@127.0.0.1:7890 bat --auto-discover --daemon
|
|||||||
### 默认值与运行时行为
|
### 默认值与运行时行为
|
||||||
|
|
||||||
- 平台:`Windows,Android`。
|
- 平台:`Windows,Android`。
|
||||||
- 资源输出:`./bat-resources`(`current` → `versions/<id>`、`.staging/<id>`)。
|
- 资源输出:`./bat-resources`(`current` → `versions/<id>`、`.staging/<id>`;自动发现 release 下包含 `official-launcher-bootstrap.json`,维护期 pending 证据位于发布根 `official-launcher-bootstrap.pending.json`)。
|
||||||
- 后台状态目录:`/tmp/bat-pid`(`bat.sock`、`bat.pid`、`bat-status.json`、`bat-daemon.log`、`bat-events.jsonl`、任务历史 `bat-tasks.json`、短生命周期 `bat-control.lock`;代理凭据在 `bat-proxy.secret`,`0600`)。
|
- 后台状态目录:`/tmp/bat-pid`(`bat.sock`、`bat.pid`、`bat-status.json`、`bat-daemon.log`、`bat-events.jsonl`、任务历史 `bat-tasks.json`、短生命周期 `bat-control.lock`;代理凭据在 `bat-proxy.secret`,`0600`)。
|
||||||
- 强制刷新:每天北京时间(UTC+8)`03:00`、`16:00`、`18:00` 各一次。
|
- 强制刷新:每天北京时间(UTC+8)`03:00`、`16:00`、`18:00` 各一次。
|
||||||
- 状态类文件默认 `0600` 权限,读写不跟随 symlink。
|
- 状态类文件默认 `0600` 权限,读写不跟随 symlink。
|
||||||
|
|||||||
+1
-2
@@ -6,6 +6,7 @@ authors.workspace = true
|
|||||||
license.workspace = true
|
license.workspace = true
|
||||||
|
|
||||||
[dependencies]
|
[dependencies]
|
||||||
|
bat-assetbundle = { path = "../crates/bat-assetbundle" }
|
||||||
bat-core = { path = "../core" }
|
bat-core = { path = "../core" }
|
||||||
anyhow.workspace = true
|
anyhow.workspace = true
|
||||||
thiserror.workspace = true
|
thiserror.workspace = true
|
||||||
@@ -13,8 +14,6 @@ serde.workspace = true
|
|||||||
serde_json.workspace = true
|
serde_json.workspace = true
|
||||||
async-trait.workspace = true
|
async-trait.workspace = true
|
||||||
tokio.workspace = true
|
tokio.workspace = true
|
||||||
lz4 = "1.28"
|
|
||||||
lzma-rs = "0.3"
|
|
||||||
base64 = "0.22"
|
base64 = "0.22"
|
||||||
|
|
||||||
[dev-dependencies]
|
[dev-dependencies]
|
||||||
|
|||||||
@@ -428,57 +428,87 @@ impl AddressablesCatalogDriver {
|
|||||||
resources
|
resources
|
||||||
}
|
}
|
||||||
|
|
||||||
fn compact_entry_resources(json: &Value) -> Vec<ResourceEntry> {
|
fn compact_entry_resources(json: &Value) -> Result<Vec<ResourceEntry>, String> {
|
||||||
let Some(internal_ids) = Self::string_array(json, "m_InternalIds") else {
|
let internal_ids = Self::string_array(json, "m_InternalIds")
|
||||||
return Vec::new();
|
.ok_or_else(|| "compact catalog missing string array m_InternalIds".to_string())?;
|
||||||
};
|
let provider_ids = Self::string_array(json, "m_ProviderIds")
|
||||||
let Some(provider_ids) = Self::string_array(json, "m_ProviderIds") else {
|
.ok_or_else(|| "compact catalog missing string array m_ProviderIds".to_string())?;
|
||||||
return Vec::new();
|
let key_bytes = Self::blob_bytes(json, "m_KeyDataString")
|
||||||
};
|
.ok_or_else(|| "compact catalog missing decodable m_KeyDataString".to_string())?;
|
||||||
let Some(key_bytes) = Self::blob_bytes(json, "m_KeyDataString") else {
|
let entry_records = Self::compact_entry_records(json)
|
||||||
return Vec::new();
|
.ok_or_else(|| "failed to decode m_EntryDataString compact records".to_string())?;
|
||||||
};
|
let buckets = Self::compact_buckets(json)
|
||||||
let Some(entry_records) = Self::compact_entry_records(json) else {
|
.ok_or_else(|| "failed to decode m_BucketDataString compact buckets".to_string())?;
|
||||||
return Vec::new();
|
|
||||||
};
|
|
||||||
let Some(buckets) = Self::compact_buckets(json) else {
|
|
||||||
return Vec::new();
|
|
||||||
};
|
|
||||||
|
|
||||||
let keys = Self::compact_keys(&key_bytes, &buckets);
|
let keys = Self::compact_keys(&key_bytes, &buckets);
|
||||||
if keys.is_empty() {
|
if keys.is_empty() {
|
||||||
return Vec::new();
|
return Err("compact catalog contains no decodable key buckets".to_string());
|
||||||
}
|
}
|
||||||
|
|
||||||
let internal_id_prefixes = Self::string_array(json, "m_InternalIdPrefixes")
|
let internal_id_prefixes = Self::string_array(json, "m_InternalIdPrefixes")
|
||||||
.unwrap_or_default()
|
.unwrap_or_default()
|
||||||
.into_iter()
|
.into_iter()
|
||||||
.collect::<Vec<_>>();
|
.collect::<Vec<_>>();
|
||||||
let extra_data = Self::blob_bytes(json, "m_ExtraDataString").unwrap_or_default();
|
let extra_data = if json.get("m_ExtraDataString").is_some() {
|
||||||
|
Self::blob_bytes(json, "m_ExtraDataString")
|
||||||
|
.ok_or_else(|| "compact catalog has undecodable m_ExtraDataString".to_string())?
|
||||||
|
} else {
|
||||||
|
Vec::new()
|
||||||
|
};
|
||||||
|
|
||||||
entry_records
|
let mut resources = Vec::with_capacity(entry_records.len());
|
||||||
.iter()
|
for (index, record) in entry_records.iter().enumerate() {
|
||||||
.enumerate()
|
if record.internal_id < 0 {
|
||||||
.filter_map(|(index, record)| {
|
return Err(format!("compact entry {index} has negative internal_id"));
|
||||||
let internal_id = internal_ids.get(record.internal_id as usize)?;
|
}
|
||||||
provider_ids.get(record.provider_index as usize)?;
|
if record.provider_index < 0 {
|
||||||
|
return Err(format!("compact entry {index} has negative provider_index"));
|
||||||
|
}
|
||||||
|
if record.primary_key_index < 0 {
|
||||||
|
return Err(format!(
|
||||||
|
"compact entry {index} has negative primary_key_index"
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
let internal_id = internal_ids
|
||||||
|
.get(record.internal_id as usize)
|
||||||
|
.ok_or_else(|| {
|
||||||
|
format!(
|
||||||
|
"compact entry {index} internal_id index {} out of range {}",
|
||||||
|
record.internal_id,
|
||||||
|
internal_ids.len()
|
||||||
|
)
|
||||||
|
})?;
|
||||||
|
provider_ids
|
||||||
|
.get(record.provider_index as usize)
|
||||||
|
.ok_or_else(|| {
|
||||||
|
format!(
|
||||||
|
"compact entry {index} provider_index {} out of range {}",
|
||||||
|
record.provider_index,
|
||||||
|
provider_ids.len()
|
||||||
|
)
|
||||||
|
})?;
|
||||||
let primary_key = keys
|
let primary_key = keys
|
||||||
.get(record.primary_key_index as usize)
|
.get(record.primary_key_index as usize)
|
||||||
.and_then(|key| key.as_ref())
|
.and_then(|key| key.as_ref())
|
||||||
.and_then(AddressablesObject::key_string)
|
.and_then(AddressablesObject::key_string)
|
||||||
.unwrap_or_else(|| format!("addressable_{}", index));
|
.ok_or_else(|| {
|
||||||
|
format!(
|
||||||
|
"compact entry {index} primary_key_index {} has no decodable key",
|
||||||
|
record.primary_key_index
|
||||||
|
)
|
||||||
|
})?;
|
||||||
let path = Self::normalize_internal_id(&internal_id_prefixes, internal_id);
|
let path = Self::normalize_internal_id(&internal_id_prefixes, internal_id);
|
||||||
let extra = Self::extra_data_at(&extra_data, record.data_index);
|
let extra = Self::extra_data_at(&extra_data, record.data_index);
|
||||||
let resource_type_name = Self::resource_type_name(json, record.resource_type_index);
|
let resource_type_name = Self::resource_type_name(json, record.resource_type_index);
|
||||||
let dependencies =
|
let dependencies = Self::compact_dependencies(record, &entry_records, &buckets, &keys);
|
||||||
Self::compact_dependencies(record, &entry_records, &buckets, &keys);
|
|
||||||
let hash = extra
|
let hash = extra
|
||||||
.hash
|
.hash
|
||||||
.filter(|value| !value.is_empty())
|
.filter(|value| !value.is_empty())
|
||||||
.or_else(|| extra.bundle_name.filter(|value| !value.is_empty()))
|
.or_else(|| extra.bundle_name.filter(|value| !value.is_empty()))
|
||||||
.unwrap_or_else(|| format!("addressable_{}", index));
|
.unwrap_or_else(|| format!("addressable_{}", index));
|
||||||
|
|
||||||
Some(ResourceEntry {
|
resources.push(ResourceEntry {
|
||||||
path: path.clone(),
|
path: path.clone(),
|
||||||
hash,
|
hash,
|
||||||
size: extra.bundle_size.unwrap_or_default(),
|
size: extra.bundle_size.unwrap_or_default(),
|
||||||
@@ -493,9 +523,10 @@ impl AddressablesCatalogDriver {
|
|||||||
},
|
},
|
||||||
dependencies,
|
dependencies,
|
||||||
crc: extra.crc,
|
crc: extra.crc,
|
||||||
})
|
});
|
||||||
})
|
}
|
||||||
.collect()
|
|
||||||
|
Ok(resources)
|
||||||
}
|
}
|
||||||
|
|
||||||
fn string_array(json: &Value, field: &str) -> Option<Vec<String>> {
|
fn string_array(json: &Value, field: &str) -> Option<Vec<String>> {
|
||||||
@@ -677,15 +708,30 @@ impl AddressablesCatalogDriver {
|
|||||||
format!("{prefix}{path}")
|
format!("{prefix}{path}")
|
||||||
}
|
}
|
||||||
|
|
||||||
fn resources(json: &Value) -> Vec<ResourceEntry> {
|
fn has_compact_catalog_fields(json: &Value) -> bool {
|
||||||
let entry_resources = Self::entry_resources(json);
|
[
|
||||||
if !entry_resources.is_empty() {
|
"m_ProviderIds",
|
||||||
return entry_resources;
|
"m_KeyDataString",
|
||||||
|
"m_BucketDataString",
|
||||||
|
"m_EntryDataString",
|
||||||
|
"m_ExtraDataString",
|
||||||
|
"m_resourceTypes",
|
||||||
|
]
|
||||||
|
.iter()
|
||||||
|
.any(|field| json.get(field).is_some())
|
||||||
}
|
}
|
||||||
|
|
||||||
let compact_resources = Self::compact_entry_resources(json);
|
fn resources(json: &Value) -> Result<Vec<ResourceEntry>, String> {
|
||||||
|
let entry_resources = Self::entry_resources(json);
|
||||||
|
if !entry_resources.is_empty() {
|
||||||
|
return Ok(entry_resources);
|
||||||
|
}
|
||||||
|
|
||||||
|
if Self::has_compact_catalog_fields(json) {
|
||||||
|
let compact_resources = Self::compact_entry_resources(json)?;
|
||||||
if !compact_resources.is_empty() {
|
if !compact_resources.is_empty() {
|
||||||
return compact_resources;
|
return Ok(compact_resources);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
let key_resources = Self::key_data_resources(json);
|
let key_resources = Self::key_data_resources(json);
|
||||||
@@ -694,13 +740,13 @@ impl AddressablesCatalogDriver {
|
|||||||
.map(|count| key_resources.len() >= count)
|
.map(|count| key_resources.len() >= count)
|
||||||
.unwrap_or(true)
|
.unwrap_or(true)
|
||||||
{
|
{
|
||||||
return key_resources;
|
return Ok(key_resources);
|
||||||
}
|
}
|
||||||
|
|
||||||
Self::internal_id_resources(json)
|
Ok(Self::internal_id_resources(json))
|
||||||
}
|
}
|
||||||
|
|
||||||
fn extra_metadata(json: &Value) -> HashMap<String, String> {
|
fn extra_metadata(json: &Value, resources: &[ResourceEntry]) -> HashMap<String, String> {
|
||||||
let mut extra = HashMap::new();
|
let mut extra = HashMap::new();
|
||||||
Self::insert_array_len(&mut extra, json, "m_InternalIds", "internal_id_count");
|
Self::insert_array_len(&mut extra, json, "m_InternalIds", "internal_id_count");
|
||||||
Self::insert_array_len(&mut extra, json, "m_Entries", "entry_count");
|
Self::insert_array_len(&mut extra, json, "m_Entries", "entry_count");
|
||||||
@@ -732,7 +778,7 @@ impl AddressablesCatalogDriver {
|
|||||||
"m_ExtraDataString",
|
"m_ExtraDataString",
|
||||||
"extra_data_string_len",
|
"extra_data_string_len",
|
||||||
);
|
);
|
||||||
Self::insert_dependency_count(&mut extra, json);
|
Self::insert_resource_summary(&mut extra, resources);
|
||||||
extra
|
extra
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -772,13 +818,56 @@ impl AddressablesCatalogDriver {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
fn insert_dependency_count(extra: &mut HashMap<String, String>, json: &Value) {
|
fn insert_resource_summary(extra: &mut HashMap<String, String>, resources: &[ResourceEntry]) {
|
||||||
let count = Self::entry_resources(json)
|
extra.insert("resource_count".to_string(), resources.len().to_string());
|
||||||
.into_iter()
|
|
||||||
|
let asset_bundle_count = resources
|
||||||
|
.iter()
|
||||||
|
.filter(|entry| entry.resource_type == ResourceType::AssetBundle)
|
||||||
|
.count();
|
||||||
|
if asset_bundle_count > 0 {
|
||||||
|
extra.insert(
|
||||||
|
"asset_bundle_count".to_string(),
|
||||||
|
asset_bundle_count.to_string(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
let declared_size_count = resources.iter().filter(|entry| entry.size != 0).count();
|
||||||
|
if declared_size_count > 0 {
|
||||||
|
extra.insert(
|
||||||
|
"declared_size_count".to_string(),
|
||||||
|
declared_size_count.to_string(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
let declared_crc_count = resources
|
||||||
|
.iter()
|
||||||
|
.filter(|entry| entry.declared_crc().is_some())
|
||||||
|
.count();
|
||||||
|
if declared_crc_count > 0 {
|
||||||
|
extra.insert(
|
||||||
|
"declared_crc_count".to_string(),
|
||||||
|
declared_crc_count.to_string(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
let dependency_count = resources
|
||||||
|
.iter()
|
||||||
.map(|entry| entry.dependencies.len())
|
.map(|entry| entry.dependencies.len())
|
||||||
.sum::<usize>();
|
.sum::<usize>();
|
||||||
if count > 0 {
|
if dependency_count > 0 {
|
||||||
extra.insert("dependency_count".to_string(), count.to_string());
|
extra.insert("dependency_count".to_string(), dependency_count.to_string());
|
||||||
|
}
|
||||||
|
|
||||||
|
let fallback_hash_count = resources
|
||||||
|
.iter()
|
||||||
|
.filter(|entry| entry.hash.starts_with("addressable_"))
|
||||||
|
.count();
|
||||||
|
if fallback_hash_count > 0 {
|
||||||
|
extra.insert(
|
||||||
|
"fallback_hash_count".to_string(),
|
||||||
|
fallback_hash_count.to_string(),
|
||||||
|
);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -886,16 +975,17 @@ impl ManifestDriver for AddressablesCatalogDriver {
|
|||||||
|
|
||||||
async fn parse(&self, raw_data: &[u8]) -> Result<GenericManifest, String> {
|
async fn parse(&self, raw_data: &[u8]) -> Result<GenericManifest, String> {
|
||||||
let json = Self::parse_json(raw_data)?;
|
let json = Self::parse_json(raw_data)?;
|
||||||
|
let resources = Self::resources(&json)?;
|
||||||
|
|
||||||
let metadata = ManifestMetadata {
|
let metadata = ManifestMetadata {
|
||||||
locator_id: Self::locator_id(&json),
|
locator_id: Self::locator_id(&json),
|
||||||
cdn_prefixes: Self::cdn_prefixes(&json),
|
cdn_prefixes: Self::cdn_prefixes(&json),
|
||||||
extra: Self::extra_metadata(&json),
|
extra: Self::extra_metadata(&json, &resources),
|
||||||
};
|
};
|
||||||
|
|
||||||
Ok(GenericManifest {
|
Ok(GenericManifest {
|
||||||
format: ManifestFormat::AddressablesCatalog,
|
format: ManifestFormat::AddressablesCatalog,
|
||||||
resources: Self::resources(&json),
|
resources,
|
||||||
metadata,
|
metadata,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -904,6 +994,91 @@ impl ManifestDriver for AddressablesCatalogDriver {
|
|||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
use base64::engine::general_purpose::STANDARD;
|
||||||
|
use base64::Engine;
|
||||||
|
|
||||||
|
fn push_u32_le(data: &mut Vec<u8>, value: u32) {
|
||||||
|
data.extend_from_slice(&value.to_le_bytes());
|
||||||
|
}
|
||||||
|
|
||||||
|
fn push_i32_le(data: &mut Vec<u8>, value: i32) {
|
||||||
|
data.extend_from_slice(&value.to_le_bytes());
|
||||||
|
}
|
||||||
|
|
||||||
|
fn push_serialized_string(data: &mut Vec<u8>, value: &str) -> usize {
|
||||||
|
let offset = data.len();
|
||||||
|
data.push(0);
|
||||||
|
push_u32_le(data, value.len() as u32);
|
||||||
|
data.extend_from_slice(value.as_bytes());
|
||||||
|
offset
|
||||||
|
}
|
||||||
|
|
||||||
|
fn serialized_json_object(json_text: &str) -> Vec<u8> {
|
||||||
|
let assembly_name =
|
||||||
|
"Unity.ResourceManager, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null";
|
||||||
|
let class_name =
|
||||||
|
"UnityEngine.ResourceManagement.ResourceProviders.AssetBundleRequestOptions";
|
||||||
|
let mut json_bytes = Vec::new();
|
||||||
|
for unit in json_text.encode_utf16() {
|
||||||
|
json_bytes.extend_from_slice(&unit.to_le_bytes());
|
||||||
|
}
|
||||||
|
|
||||||
|
let mut data = Vec::new();
|
||||||
|
data.push(7);
|
||||||
|
data.push(assembly_name.len() as u8);
|
||||||
|
data.extend_from_slice(assembly_name.as_bytes());
|
||||||
|
data.push(class_name.len() as u8);
|
||||||
|
data.extend_from_slice(class_name.as_bytes());
|
||||||
|
push_u32_le(&mut data, json_bytes.len() as u32);
|
||||||
|
data.extend_from_slice(&json_bytes);
|
||||||
|
data
|
||||||
|
}
|
||||||
|
|
||||||
|
fn compact_catalog_json(extra_json: &str) -> String {
|
||||||
|
let mut key_data = Vec::new();
|
||||||
|
push_u32_le(&mut key_data, 1);
|
||||||
|
let key_offset = push_serialized_string(&mut key_data, "synthetic.bundle");
|
||||||
|
|
||||||
|
let mut bucket_data = Vec::new();
|
||||||
|
push_u32_le(&mut bucket_data, 1);
|
||||||
|
push_i32_le(&mut bucket_data, key_offset as i32);
|
||||||
|
push_i32_le(&mut bucket_data, 1);
|
||||||
|
push_i32_le(&mut bucket_data, 0);
|
||||||
|
|
||||||
|
let mut entry_data = Vec::new();
|
||||||
|
push_u32_le(&mut entry_data, 1);
|
||||||
|
push_i32_le(&mut entry_data, 0); // internal_id
|
||||||
|
push_i32_le(&mut entry_data, 0); // provider_index
|
||||||
|
push_i32_le(&mut entry_data, -1); // dependency_key_index
|
||||||
|
push_i32_le(&mut entry_data, 0); // reserved/unused
|
||||||
|
push_i32_le(&mut entry_data, 0); // data_index
|
||||||
|
push_i32_le(&mut entry_data, 0); // primary_key_index
|
||||||
|
push_i32_le(&mut entry_data, 0); // resource_type_index
|
||||||
|
|
||||||
|
let extra_data = serialized_json_object(extra_json);
|
||||||
|
|
||||||
|
serde_json::json!({
|
||||||
|
"m_LocatorId": "AddressablesMainContentCatalog",
|
||||||
|
"m_InternalIdPrefixes": [],
|
||||||
|
"m_ProviderIds": [
|
||||||
|
"UnityEngine.ResourceManagement.ResourceProviders.AssetBundleProvider"
|
||||||
|
],
|
||||||
|
"m_InternalIds": [
|
||||||
|
"{PlatformUtils.AddressableLoadPath}\\synthetic.bundle"
|
||||||
|
],
|
||||||
|
"m_resourceTypes": [
|
||||||
|
{
|
||||||
|
"m_AssemblyName": "Unity.ResourceManager, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null",
|
||||||
|
"m_ClassName": "UnityEngine.ResourceManagement.ResourceProviders.IAssetBundleResource"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"m_KeyDataString": STANDARD.encode(key_data),
|
||||||
|
"m_BucketDataString": STANDARD.encode(bucket_data),
|
||||||
|
"m_EntryDataString": STANDARD.encode(entry_data),
|
||||||
|
"m_ExtraDataString": STANDARD.encode(extra_data)
|
||||||
|
})
|
||||||
|
.to_string()
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn test_can_parse_valid_catalog() {
|
fn test_can_parse_valid_catalog() {
|
||||||
@@ -1048,6 +1223,65 @@ mod tests {
|
|||||||
assert!(error.contains("Invalid JSON at line"));
|
assert!(error.contains("Invalid JSON at line"));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn test_parse_compact_catalog_extracts_verification_fields() {
|
||||||
|
let driver = AddressablesCatalogDriver::new();
|
||||||
|
let catalog_json = compact_catalog_json(
|
||||||
|
r#"{
|
||||||
|
"m_Hash":"hash-compact",
|
||||||
|
"m_Crc":305419896,
|
||||||
|
"m_BundleName":"synthetic-bundle-name",
|
||||||
|
"m_BundleSize":42
|
||||||
|
}"#,
|
||||||
|
);
|
||||||
|
|
||||||
|
let manifest = driver.parse(catalog_json.as_bytes()).await.unwrap();
|
||||||
|
|
||||||
|
assert_eq!(manifest.resources.len(), 1);
|
||||||
|
let resource = &manifest.resources[0];
|
||||||
|
assert_eq!(resource.path, "synthetic.bundle");
|
||||||
|
assert_eq!(resource.hash, "hash-compact");
|
||||||
|
assert_eq!(resource.size, 42);
|
||||||
|
assert_eq!(resource.crc, Some(0x1234_5678));
|
||||||
|
assert_eq!(resource.resource_type, ResourceType::AssetBundle);
|
||||||
|
assert_eq!(resource.address.as_deref(), Some("synthetic.bundle"));
|
||||||
|
assert_eq!(
|
||||||
|
manifest.metadata.extra.get("resource_count"),
|
||||||
|
Some(&"1".to_string())
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
manifest.metadata.extra.get("asset_bundle_count"),
|
||||||
|
Some(&"1".to_string())
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
manifest.metadata.extra.get("declared_size_count"),
|
||||||
|
Some(&"1".to_string())
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
manifest.metadata.extra.get("declared_crc_count"),
|
||||||
|
Some(&"1".to_string())
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn test_parse_compact_catalog_reports_blob_decode_failure() {
|
||||||
|
let driver = AddressablesCatalogDriver::new();
|
||||||
|
let catalog_json = r#"{
|
||||||
|
"m_LocatorId": "AddressablesMainContentCatalog",
|
||||||
|
"m_ProviderIds": ["UnityEngine.ResourceManagement.ResourceProviders.AssetBundleProvider"],
|
||||||
|
"m_InternalIds": ["synthetic.bundle"],
|
||||||
|
"m_KeyDataString": "not-base64",
|
||||||
|
"m_BucketDataString": "not-base64",
|
||||||
|
"m_EntryDataString": "not-base64",
|
||||||
|
"m_ExtraDataString": "not-base64"
|
||||||
|
}"#;
|
||||||
|
|
||||||
|
let error = driver.parse(catalog_json.as_bytes()).await.unwrap_err();
|
||||||
|
|
||||||
|
assert!(error.contains("compact catalog"), "{error}");
|
||||||
|
assert!(error.contains("m_KeyDataString"), "{error}");
|
||||||
|
}
|
||||||
|
|
||||||
#[tokio::test]
|
#[tokio::test]
|
||||||
async fn test_parse_table_bundle_resource_types() {
|
async fn test_parse_table_bundle_resource_types() {
|
||||||
let driver = AddressablesCatalogDriver::new();
|
let driver = AddressablesCatalogDriver::new();
|
||||||
|
|||||||
@@ -38,13 +38,14 @@ pub struct YostarJpGameMainConfig {
|
|||||||
impl YostarJpGameMainConfig {
|
impl YostarJpGameMainConfig {
|
||||||
/// Reads and decrypts `GameMainConfig` from a Unity serialized file.
|
/// Reads and decrypts `GameMainConfig` from a Unity serialized file.
|
||||||
pub fn from_resources_assets(path: impl AsRef<Path>) -> Result<Self, String> {
|
pub fn from_resources_assets(path: impl AsRef<Path>) -> Result<Self, String> {
|
||||||
let serialized = UnitySerializedFile::from_path(path)?;
|
let serialized = UnitySerializedFile::from_path(path).map_err(|error| error.to_string())?;
|
||||||
Self::from_serialized_file(&serialized)
|
Self::from_serialized_file(&serialized)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Reads and decrypts `GameMainConfig` from serialized file bytes.
|
/// Reads and decrypts `GameMainConfig` from serialized file bytes.
|
||||||
pub fn from_resources_assets_bytes(bytes: &[u8]) -> Result<Self, String> {
|
pub fn from_resources_assets_bytes(bytes: &[u8]) -> Result<Self, String> {
|
||||||
let serialized = UnitySerializedFile::from_slice(bytes)?;
|
let serialized =
|
||||||
|
UnitySerializedFile::from_slice(bytes).map_err(|error| error.to_string())?;
|
||||||
Self::from_serialized_file(&serialized)
|
Self::from_serialized_file(&serialized)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -9,8 +9,11 @@ pub mod unity_2021_3;
|
|||||||
|
|
||||||
pub use adapter::{
|
pub use adapter::{
|
||||||
ParsedAssetBundle, RawAssetBundle, UnityAdapter, UnityFsBlockInfo, UnityFsCompression,
|
ParsedAssetBundle, RawAssetBundle, UnityAdapter, UnityFsBlockInfo, UnityFsCompression,
|
||||||
UnityFsDirectoryInfo, UnityFsHeader, VersionRange,
|
UnityFsDirectoryInfo, UnityFsFile, UnityFsHeader, UnitySerializedParseError, VersionRange,
|
||||||
};
|
};
|
||||||
pub use registry::UnityAdapterRegistry;
|
pub use registry::UnityAdapterRegistry;
|
||||||
pub use serialized_file::{UnitySerializedFile, UnitySerializedTextAsset};
|
pub use serialized_file::{
|
||||||
|
UnitySerializedField, UnitySerializedFile, UnitySerializedObject, UnitySerializedTextAsset,
|
||||||
|
UnitySerializedType, UnitySerializedValue, UnityTypeTreeNode,
|
||||||
|
};
|
||||||
pub use unity_2021_3::Unity2021_3Adapter;
|
pub use unity_2021_3::Unity2021_3Adapter;
|
||||||
|
|||||||
@@ -1,6 +1,10 @@
|
|||||||
//! Unity Adapter 接口定义
|
//! Unity Adapter 接口定义
|
||||||
|
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
|
pub use bat_assetbundle::{
|
||||||
|
ParsedAssetBundle, RawAssetBundle, UnityFsBlockInfo, UnityFsCompression, UnityFsDirectoryInfo,
|
||||||
|
UnityFsFile, UnityFsHeader, UnitySerializedParseError,
|
||||||
|
};
|
||||||
|
|
||||||
/// Unity 版本范围
|
/// Unity 版本范围
|
||||||
#[derive(Debug, Clone)]
|
#[derive(Debug, Clone)]
|
||||||
@@ -56,92 +60,6 @@ fn parse_version_components(version: &str) -> Option<(u64, u64, u64)> {
|
|||||||
Some((major, minor, patch))
|
Some((major, minor, patch))
|
||||||
}
|
}
|
||||||
|
|
||||||
/// 原始 AssetBundle 数据
|
|
||||||
#[derive(Debug)]
|
|
||||||
pub struct RawAssetBundle {
|
|
||||||
/// 文件数据
|
|
||||||
pub data: Vec<u8>,
|
|
||||||
/// 文件路径(可选)
|
|
||||||
pub path: Option<String>,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// 解析后的 AssetBundle
|
|
||||||
#[derive(Debug)]
|
|
||||||
pub struct ParsedAssetBundle {
|
|
||||||
/// Unity 版本
|
|
||||||
pub unity_version: String,
|
|
||||||
/// 资源列表(简化表示)
|
|
||||||
pub assets: Vec<String>,
|
|
||||||
/// 原始数据(保留用于序列化)
|
|
||||||
pub raw_data: Vec<u8>,
|
|
||||||
/// UnityFS 文件头信息。
|
|
||||||
pub unityfs_header: Option<UnityFsHeader>,
|
|
||||||
/// UnityFS 压缩块信息。
|
|
||||||
pub blocks: Vec<UnityFsBlockInfo>,
|
|
||||||
/// UnityFS 目录信息。
|
|
||||||
pub directories: Vec<UnityFsDirectoryInfo>,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// UnityFS 文件头。
|
|
||||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
|
||||||
pub struct UnityFsHeader {
|
|
||||||
/// UnityFS 格式版本。
|
|
||||||
pub format_version: u32,
|
|
||||||
/// Bundle 目标版本字符串,例如 `5.x.x`。
|
|
||||||
pub target_version: String,
|
|
||||||
/// Unity 编辑器版本字符串。
|
|
||||||
pub unity_version: String,
|
|
||||||
/// 文件总大小。
|
|
||||||
pub total_size: u64,
|
|
||||||
/// 压缩后的 block info 大小。
|
|
||||||
pub compressed_blocks_info_size: u32,
|
|
||||||
/// 解压后的 block info 大小。
|
|
||||||
pub uncompressed_blocks_info_size: u32,
|
|
||||||
/// UnityFS flags 原始值。
|
|
||||||
pub flags: u32,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// UnityFS 块压缩类型。
|
|
||||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
||||||
pub enum UnityFsCompression {
|
|
||||||
/// 未压缩。
|
|
||||||
None,
|
|
||||||
/// LZMA 压缩。
|
|
||||||
Lzma,
|
|
||||||
/// LZ4 压缩。
|
|
||||||
Lz4,
|
|
||||||
/// LZ4HC 压缩。
|
|
||||||
Lz4Hc,
|
|
||||||
/// 当前版本未识别的压缩类型。
|
|
||||||
Unknown(u16),
|
|
||||||
}
|
|
||||||
|
|
||||||
/// UnityFS 压缩块信息。
|
|
||||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
|
||||||
pub struct UnityFsBlockInfo {
|
|
||||||
/// 解压后大小。
|
|
||||||
pub uncompressed_size: u32,
|
|
||||||
/// 压缩后大小。
|
|
||||||
pub compressed_size: u32,
|
|
||||||
/// 块 flags 原始值。
|
|
||||||
pub flags: u16,
|
|
||||||
/// 解析出的压缩类型。
|
|
||||||
pub compression: UnityFsCompression,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// UnityFS 目录条目。
|
|
||||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
|
||||||
pub struct UnityFsDirectoryInfo {
|
|
||||||
/// 条目在数据区中的偏移。
|
|
||||||
pub offset: u64,
|
|
||||||
/// 条目大小。
|
|
||||||
pub size: u64,
|
|
||||||
/// 条目 flags 原始值。
|
|
||||||
pub flags: u32,
|
|
||||||
/// 条目路径。
|
|
||||||
pub path: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Unity Adapter 接口
|
/// Unity Adapter 接口
|
||||||
///
|
///
|
||||||
/// 用于解析不同 Unity 版本的 AssetBundle
|
/// 用于解析不同 Unity 版本的 AssetBundle
|
||||||
@@ -172,8 +90,7 @@ pub trait UnityAdapter: Send + Sync {
|
|||||||
/// - 成功:返回解析后的 AssetBundle
|
/// - 成功:返回解析后的 AssetBundle
|
||||||
/// - 失败:返回错误
|
/// - 失败:返回错误
|
||||||
///
|
///
|
||||||
/// # 注意
|
/// 当前 UnityFS 容器解析由具体适配器委托给 `bat-assetbundle`。
|
||||||
/// Phase 1 中标记为 TODO,Phase 2 实现
|
|
||||||
async fn parse(&self, bundle: &RawAssetBundle) -> Result<ParsedAssetBundle, String>;
|
async fn parse(&self, bundle: &RawAssetBundle) -> Result<ParsedAssetBundle, String>;
|
||||||
|
|
||||||
/// 序列化 AssetBundle
|
/// 序列化 AssetBundle
|
||||||
@@ -185,8 +102,7 @@ pub trait UnityAdapter: Send + Sync {
|
|||||||
/// - 成功:返回序列化后的数据
|
/// - 成功:返回序列化后的数据
|
||||||
/// - 失败:返回错误
|
/// - 失败:返回错误
|
||||||
///
|
///
|
||||||
/// # 注意
|
/// 当前阶段只定义接口;具体序列化能力尚未进入实现范围。
|
||||||
/// Phase 1 中标记为 TODO,Phase 2 实现
|
|
||||||
async fn serialize(&self, parsed: &ParsedAssetBundle) -> Result<Vec<u8>, String>;
|
async fn serialize(&self, parsed: &ParsedAssetBundle) -> Result<Vec<u8>, String>;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1,521 +1,9 @@
|
|||||||
//! Unity serialized file reader.
|
//! Compatibility exports for Unity serialized file parsing.
|
||||||
//!
|
//!
|
||||||
//! This module is intentionally narrow: it extracts `TextAsset` payloads from
|
//! The implementation lives in `bat-assetbundle`; adapters keep this module so
|
||||||
//! Unity serialized files such as `resources.assets` and
|
//! existing call sites can continue to import through `bat_adapters::unity`.
|
||||||
//! `globalgamemanagers.assets`.
|
|
||||||
|
|
||||||
use std::fs;
|
pub use bat_assetbundle::{
|
||||||
use std::path::Path;
|
UnitySerializedField, UnitySerializedFile, UnitySerializedObject, UnitySerializedTextAsset,
|
||||||
|
UnitySerializedType, UnitySerializedValue, UnityTypeTreeNode,
|
||||||
/// One extracted Unity `TextAsset`.
|
|
||||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
|
||||||
pub struct UnitySerializedTextAsset {
|
|
||||||
/// Unity path ID of the object.
|
|
||||||
pub path_id: i64,
|
|
||||||
/// Asset name stored in the serialized object.
|
|
||||||
pub name: String,
|
|
||||||
/// Raw bytes stored by the `TextAsset`.
|
|
||||||
pub bytes: Vec<u8>,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Parsed Unity serialized file summary.
|
|
||||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
|
||||||
pub struct UnitySerializedFile {
|
|
||||||
/// Serialized file format version.
|
|
||||||
pub version: u32,
|
|
||||||
/// Unity editor version stored in the file.
|
|
||||||
pub unity_version: String,
|
|
||||||
/// Target platform value from the file header.
|
|
||||||
pub platform: i32,
|
|
||||||
text_assets: Vec<UnitySerializedTextAsset>,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl UnitySerializedFile {
|
|
||||||
/// Parses a serialized file from raw bytes.
|
|
||||||
pub fn from_slice(data: &[u8]) -> Result<Self, String> {
|
|
||||||
let mut reader = Reader::new(data);
|
|
||||||
|
|
||||||
let _metadata_size = reader.read_u32_be("metadata_size")?;
|
|
||||||
let _file_size = reader.read_u32_be("file_size")?;
|
|
||||||
let version = reader.read_u32_be("version")?;
|
|
||||||
let _data_offset = reader.read_u32_be("data_offset")?;
|
|
||||||
let endian_flag = reader.read_u8("endian_flag")?;
|
|
||||||
reader.read_bytes(3, "reserved")?;
|
|
||||||
|
|
||||||
let (metadata_size, file_size, data_offset) = if version >= 22 {
|
|
||||||
let metadata_size = reader.read_u32_be("metadata_size_2")?;
|
|
||||||
let file_size = reader.read_u64_be("file_size_2")?;
|
|
||||||
let data_offset = reader.read_u64_be("data_offset_2")? as usize;
|
|
||||||
let _unknown = reader.read_u64_be("unknown_2")?;
|
|
||||||
(metadata_size, file_size, data_offset)
|
|
||||||
} else {
|
|
||||||
(_metadata_size, _file_size as u64, _data_offset as usize)
|
|
||||||
};
|
};
|
||||||
let _ = metadata_size;
|
|
||||||
let _ = file_size;
|
|
||||||
|
|
||||||
let endian = if endian_flag == 0 {
|
|
||||||
Endian::Little
|
|
||||||
} else {
|
|
||||||
Endian::Big
|
|
||||||
};
|
|
||||||
reader.set_endian(endian);
|
|
||||||
|
|
||||||
let unity_version = reader.read_c_string("unity_version")?;
|
|
||||||
let platform = reader.read_i32("platform")?;
|
|
||||||
let enable_type_tree = reader.read_u8("enable_type_tree")?;
|
|
||||||
let type_count = reader.read_i32("type_count")?;
|
|
||||||
if type_count < 0 {
|
|
||||||
return Err(format!("Invalid Unity type count: {}", type_count));
|
|
||||||
}
|
|
||||||
|
|
||||||
let mut class_ids = Vec::with_capacity(type_count as usize);
|
|
||||||
for _ in 0..type_count {
|
|
||||||
class_ids.push(read_serialized_type(
|
|
||||||
&mut reader,
|
|
||||||
version,
|
|
||||||
enable_type_tree,
|
|
||||||
)?);
|
|
||||||
}
|
|
||||||
|
|
||||||
let big_id_enabled = if (11..14).contains(&version) {
|
|
||||||
reader.read_i32("big_id_enabled")?
|
|
||||||
} else {
|
|
||||||
0
|
|
||||||
};
|
|
||||||
|
|
||||||
let object_count = reader.read_i32("object_count")?;
|
|
||||||
if object_count < 0 {
|
|
||||||
return Err(format!("Invalid Unity object count: {}", object_count));
|
|
||||||
}
|
|
||||||
|
|
||||||
let mut text_assets = Vec::new();
|
|
||||||
for _ in 0..object_count {
|
|
||||||
if version >= 14 {
|
|
||||||
reader.align(4)?;
|
|
||||||
}
|
|
||||||
|
|
||||||
let path_id = if big_id_enabled != 0 {
|
|
||||||
reader.read_i64("path_id")?
|
|
||||||
} else if version < 14 {
|
|
||||||
reader.read_i32("path_id")? as i64
|
|
||||||
} else {
|
|
||||||
reader.read_i64("path_id")?
|
|
||||||
};
|
|
||||||
|
|
||||||
let byte_start = if version >= 22 {
|
|
||||||
reader.read_u64("byte_start")? as usize
|
|
||||||
} else {
|
|
||||||
reader.read_u32("byte_start")? as usize
|
|
||||||
};
|
|
||||||
let byte_size = reader.read_u32("byte_size")? as usize;
|
|
||||||
let type_id = reader.read_i32("type_id")?;
|
|
||||||
if version < 16 {
|
|
||||||
reader.read_u16("class_id")?;
|
|
||||||
}
|
|
||||||
if version < 11 {
|
|
||||||
reader.read_u16("is_destroyed")?;
|
|
||||||
}
|
|
||||||
if (11..17).contains(&version) {
|
|
||||||
reader.read_i16("script_type_index")?;
|
|
||||||
}
|
|
||||||
if version == 15 || version == 16 {
|
|
||||||
reader.read_u8("stripped")?;
|
|
||||||
}
|
|
||||||
|
|
||||||
let class_id = class_ids
|
|
||||||
.get(type_id as usize)
|
|
||||||
.copied()
|
|
||||||
.ok_or_else(|| format!("Invalid Unity type index: {}", type_id))?;
|
|
||||||
if class_id == 49 {
|
|
||||||
let object_start = data_offset
|
|
||||||
.checked_add(byte_start)
|
|
||||||
.ok_or_else(|| "Unity object offset overflow".to_string())?;
|
|
||||||
let object_end = object_start
|
|
||||||
.checked_add(byte_size)
|
|
||||||
.ok_or_else(|| "Unity object size overflow".to_string())?;
|
|
||||||
if object_end > data.len() {
|
|
||||||
return Err(format!(
|
|
||||||
"Unity object exceeds file size: start={}, size={}, file_size={}",
|
|
||||||
object_start,
|
|
||||||
byte_size,
|
|
||||||
data.len()
|
|
||||||
));
|
|
||||||
}
|
|
||||||
|
|
||||||
let asset = parse_text_asset(path_id, &data[object_start..object_end], endian)?;
|
|
||||||
text_assets.push(asset);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
Ok(Self {
|
|
||||||
version,
|
|
||||||
unity_version,
|
|
||||||
platform,
|
|
||||||
text_assets,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Parses a serialized file from disk.
|
|
||||||
pub fn from_path(path: impl AsRef<Path>) -> Result<Self, String> {
|
|
||||||
let path = path.as_ref();
|
|
||||||
let bytes = fs::read(path)
|
|
||||||
.map_err(|error| format!("Failed to read {}: {error}", path.display()))?;
|
|
||||||
Self::from_slice(&bytes)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Returns all extracted text assets.
|
|
||||||
pub fn text_assets(&self) -> &[UnitySerializedTextAsset] {
|
|
||||||
&self.text_assets
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Returns one extracted text asset by name.
|
|
||||||
pub fn text_asset(&self, name: &str) -> Option<&UnitySerializedTextAsset> {
|
|
||||||
self.text_assets.iter().find(|asset| asset.name == name)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn parse_text_asset(
|
|
||||||
path_id: i64,
|
|
||||||
data: &[u8],
|
|
||||||
endian: Endian,
|
|
||||||
) -> Result<UnitySerializedTextAsset, String> {
|
|
||||||
let mut reader = Reader::new(data);
|
|
||||||
reader.set_endian(endian);
|
|
||||||
let name = reader.read_len_prefixed_string("text_asset_name")?;
|
|
||||||
reader.align(4)?;
|
|
||||||
let bytes_len = reader.read_u32("text_asset_bytes_len")? as usize;
|
|
||||||
let bytes = reader.read_bytes(bytes_len, "text_asset_bytes")?.to_vec();
|
|
||||||
|
|
||||||
Ok(UnitySerializedTextAsset {
|
|
||||||
path_id,
|
|
||||||
name,
|
|
||||||
bytes,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_serialized_type(
|
|
||||||
reader: &mut Reader<'_>,
|
|
||||||
version: u32,
|
|
||||||
enable_type_tree: u8,
|
|
||||||
) -> Result<i32, String> {
|
|
||||||
let class_id = reader.read_i32("type_class_id")?;
|
|
||||||
|
|
||||||
if version >= 16 {
|
|
||||||
reader.read_u8("type_is_stripped")?;
|
|
||||||
}
|
|
||||||
if version >= 17 {
|
|
||||||
reader.read_i16("type_script_index")?;
|
|
||||||
}
|
|
||||||
if version >= 13 {
|
|
||||||
if (version < 16 && class_id < 0) || (version >= 16 && class_id == 114) {
|
|
||||||
reader.read_bytes(16, "type_script_id")?;
|
|
||||||
}
|
|
||||||
reader.read_bytes(16, "type_hash")?;
|
|
||||||
}
|
|
||||||
|
|
||||||
if enable_type_tree != 0 {
|
|
||||||
if version >= 12 || version == 10 {
|
|
||||||
let node_count = reader.read_i32("type_tree_node_count")?;
|
|
||||||
if node_count < 0 {
|
|
||||||
return Err(format!(
|
|
||||||
"Invalid Unity type tree node count: {}",
|
|
||||||
node_count
|
|
||||||
));
|
|
||||||
}
|
|
||||||
let string_buffer_size = reader.read_i32("type_tree_string_buffer_size")?;
|
|
||||||
if string_buffer_size < 0 {
|
|
||||||
return Err(format!(
|
|
||||||
"Invalid Unity type tree string buffer size: {}",
|
|
||||||
string_buffer_size
|
|
||||||
));
|
|
||||||
}
|
|
||||||
|
|
||||||
let node_size = 2 + 1 + 1 + 4 + 4 + 4 + 4 + 4 + if version >= 19 { 8 } else { 0 };
|
|
||||||
reader.read_bytes(node_count as usize * node_size, "type_tree_nodes")?;
|
|
||||||
reader.read_bytes(string_buffer_size as usize, "type_tree_strings")?;
|
|
||||||
}
|
|
||||||
|
|
||||||
if version >= 21 {
|
|
||||||
let dependency_count = reader.read_i32("type_tree_dependency_count")?;
|
|
||||||
if dependency_count < 0 {
|
|
||||||
return Err(format!(
|
|
||||||
"Invalid Unity type tree dependency count: {}",
|
|
||||||
dependency_count
|
|
||||||
));
|
|
||||||
}
|
|
||||||
reader.read_bytes(dependency_count as usize * 4, "type_tree_dependencies")?;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
Ok(class_id)
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
||||||
enum Endian {
|
|
||||||
Little,
|
|
||||||
Big,
|
|
||||||
}
|
|
||||||
|
|
||||||
struct Reader<'a> {
|
|
||||||
data: &'a [u8],
|
|
||||||
offset: usize,
|
|
||||||
endian: Endian,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl<'a> Reader<'a> {
|
|
||||||
fn new(data: &'a [u8]) -> Self {
|
|
||||||
Self {
|
|
||||||
data,
|
|
||||||
offset: 0,
|
|
||||||
endian: Endian::Big,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn set_endian(&mut self, endian: Endian) {
|
|
||||||
self.endian = endian;
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_bytes(&mut self, len: usize, field: &str) -> Result<&'a [u8], String> {
|
|
||||||
let end = self
|
|
||||||
.offset
|
|
||||||
.checked_add(len)
|
|
||||||
.ok_or_else(|| format!("{field} length overflow at offset {}", self.offset))?;
|
|
||||||
if end > self.data.len() {
|
|
||||||
return Err(format!(
|
|
||||||
"Unexpected end while reading {field} at offset {}: need {}, have {}",
|
|
||||||
self.offset,
|
|
||||||
len,
|
|
||||||
self.data.len().saturating_sub(self.offset)
|
|
||||||
));
|
|
||||||
}
|
|
||||||
|
|
||||||
let bytes = &self.data[self.offset..end];
|
|
||||||
self.offset = end;
|
|
||||||
Ok(bytes)
|
|
||||||
}
|
|
||||||
|
|
||||||
fn align(&mut self, alignment: usize) -> Result<(), String> {
|
|
||||||
if alignment == 0 {
|
|
||||||
return Ok(());
|
|
||||||
}
|
|
||||||
|
|
||||||
let remainder = self.offset % alignment;
|
|
||||||
if remainder == 0 {
|
|
||||||
return Ok(());
|
|
||||||
}
|
|
||||||
|
|
||||||
let padding = alignment - remainder;
|
|
||||||
self.read_bytes(padding, "alignment padding").map(|_| ())
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_u8(&mut self, field: &str) -> Result<u8, String> {
|
|
||||||
Ok(self.read_bytes(1, field)?[0])
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_u16(&mut self, field: &str) -> Result<u16, String> {
|
|
||||||
let bytes = self.read_bytes(2, field)?;
|
|
||||||
Ok(match self.endian {
|
|
||||||
Endian::Little => u16::from_le_bytes([bytes[0], bytes[1]]),
|
|
||||||
Endian::Big => u16::from_be_bytes([bytes[0], bytes[1]]),
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_i16(&mut self, field: &str) -> Result<i16, String> {
|
|
||||||
let bytes = self.read_bytes(2, field)?;
|
|
||||||
Ok(match self.endian {
|
|
||||||
Endian::Little => i16::from_le_bytes([bytes[0], bytes[1]]),
|
|
||||||
Endian::Big => i16::from_be_bytes([bytes[0], bytes[1]]),
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_u32(&mut self, field: &str) -> Result<u32, String> {
|
|
||||||
let bytes = self.read_bytes(4, field)?;
|
|
||||||
Ok(match self.endian {
|
|
||||||
Endian::Little => u32::from_le_bytes([bytes[0], bytes[1], bytes[2], bytes[3]]),
|
|
||||||
Endian::Big => u32::from_be_bytes([bytes[0], bytes[1], bytes[2], bytes[3]]),
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_u32_be(&mut self, field: &str) -> Result<u32, String> {
|
|
||||||
let bytes = self.read_bytes(4, field)?;
|
|
||||||
Ok(u32::from_be_bytes([bytes[0], bytes[1], bytes[2], bytes[3]]))
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_i32(&mut self, field: &str) -> Result<i32, String> {
|
|
||||||
let bytes = self.read_bytes(4, field)?;
|
|
||||||
Ok(match self.endian {
|
|
||||||
Endian::Little => i32::from_le_bytes([bytes[0], bytes[1], bytes[2], bytes[3]]),
|
|
||||||
Endian::Big => i32::from_be_bytes([bytes[0], bytes[1], bytes[2], bytes[3]]),
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_u64(&mut self, field: &str) -> Result<u64, String> {
|
|
||||||
let bytes = self.read_bytes(8, field)?;
|
|
||||||
Ok(match self.endian {
|
|
||||||
Endian::Little => u64::from_le_bytes([
|
|
||||||
bytes[0], bytes[1], bytes[2], bytes[3], bytes[4], bytes[5], bytes[6], bytes[7],
|
|
||||||
]),
|
|
||||||
Endian::Big => u64::from_be_bytes([
|
|
||||||
bytes[0], bytes[1], bytes[2], bytes[3], bytes[4], bytes[5], bytes[6], bytes[7],
|
|
||||||
]),
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_u64_be(&mut self, field: &str) -> Result<u64, String> {
|
|
||||||
let bytes = self.read_bytes(8, field)?;
|
|
||||||
Ok(u64::from_be_bytes([
|
|
||||||
bytes[0], bytes[1], bytes[2], bytes[3], bytes[4], bytes[5], bytes[6], bytes[7],
|
|
||||||
]))
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_i64(&mut self, field: &str) -> Result<i64, String> {
|
|
||||||
let bytes = self.read_bytes(8, field)?;
|
|
||||||
Ok(match self.endian {
|
|
||||||
Endian::Little => i64::from_le_bytes([
|
|
||||||
bytes[0], bytes[1], bytes[2], bytes[3], bytes[4], bytes[5], bytes[6], bytes[7],
|
|
||||||
]),
|
|
||||||
Endian::Big => i64::from_be_bytes([
|
|
||||||
bytes[0], bytes[1], bytes[2], bytes[3], bytes[4], bytes[5], bytes[6], bytes[7],
|
|
||||||
]),
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_c_string(&mut self, field: &str) -> Result<String, String> {
|
|
||||||
let remaining = &self.data[self.offset..];
|
|
||||||
let Some(length) = remaining.iter().position(|&byte| byte == 0) else {
|
|
||||||
return Err(format!(
|
|
||||||
"Missing null terminator while reading {field} at offset {}",
|
|
||||||
self.offset
|
|
||||||
));
|
|
||||||
};
|
|
||||||
let bytes = self.read_bytes(length, field)?;
|
|
||||||
self.offset += 1;
|
|
||||||
std::str::from_utf8(bytes)
|
|
||||||
.map(ToOwned::to_owned)
|
|
||||||
.map_err(|error| format!("Invalid UTF-8 in {field}: {error}"))
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_len_prefixed_string(&mut self, field: &str) -> Result<String, String> {
|
|
||||||
let len = self.read_u32(field)? as usize;
|
|
||||||
let bytes = self.read_bytes(len, field)?;
|
|
||||||
std::str::from_utf8(bytes)
|
|
||||||
.map(ToOwned::to_owned)
|
|
||||||
.map_err(|error| format!("Invalid UTF-8 in {field}: {error}"))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
fn push_i16_le(data: &mut Vec<u8>, value: i16) {
|
|
||||||
data.extend_from_slice(&value.to_le_bytes());
|
|
||||||
}
|
|
||||||
|
|
||||||
fn push_u32_le(data: &mut Vec<u8>, value: u32) {
|
|
||||||
data.extend_from_slice(&value.to_le_bytes());
|
|
||||||
}
|
|
||||||
|
|
||||||
fn push_i32_le(data: &mut Vec<u8>, value: i32) {
|
|
||||||
data.extend_from_slice(&value.to_le_bytes());
|
|
||||||
}
|
|
||||||
|
|
||||||
fn push_i64_le(data: &mut Vec<u8>, value: i64) {
|
|
||||||
data.extend_from_slice(&value.to_le_bytes());
|
|
||||||
}
|
|
||||||
|
|
||||||
fn push_u64_le(data: &mut Vec<u8>, value: u64) {
|
|
||||||
data.extend_from_slice(&value.to_le_bytes());
|
|
||||||
}
|
|
||||||
|
|
||||||
fn push_u32_be(data: &mut Vec<u8>, value: u32) {
|
|
||||||
data.extend_from_slice(&value.to_be_bytes());
|
|
||||||
}
|
|
||||||
|
|
||||||
fn push_u64_be(data: &mut Vec<u8>, value: u64) {
|
|
||||||
data.extend_from_slice(&value.to_be_bytes());
|
|
||||||
}
|
|
||||||
|
|
||||||
fn align(data: &mut Vec<u8>, alignment: usize) {
|
|
||||||
let remainder = data.len() % alignment;
|
|
||||||
if remainder != 0 {
|
|
||||||
data.resize(data.len() + alignment - remainder, 0);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn synthetic_serialized_file() -> Vec<u8> {
|
|
||||||
let mut object_data = Vec::new();
|
|
||||||
push_u32_le(&mut object_data, 14);
|
|
||||||
object_data.extend_from_slice(b"GameMainConfig");
|
|
||||||
align(&mut object_data, 4);
|
|
||||||
push_u32_le(&mut object_data, 5);
|
|
||||||
object_data.extend_from_slice(b"hello");
|
|
||||||
|
|
||||||
let mut metadata = Vec::new();
|
|
||||||
metadata.extend_from_slice(b"2021.3.56f2\0");
|
|
||||||
push_i32_le(&mut metadata, 19);
|
|
||||||
metadata.push(0);
|
|
||||||
push_i32_le(&mut metadata, 1);
|
|
||||||
push_i32_le(&mut metadata, 49);
|
|
||||||
metadata.push(0);
|
|
||||||
push_i16_le(&mut metadata, 0);
|
|
||||||
metadata.extend_from_slice(&[0; 16]);
|
|
||||||
push_i32_le(&mut metadata, 1);
|
|
||||||
align(&mut metadata, 4);
|
|
||||||
push_i64_le(&mut metadata, 1);
|
|
||||||
push_u64_le(&mut metadata, 0);
|
|
||||||
push_u32_le(&mut metadata, object_data.len() as u32);
|
|
||||||
push_i32_le(&mut metadata, 0);
|
|
||||||
|
|
||||||
let header_len = 48usize;
|
|
||||||
let data_offset = header_len + metadata.len();
|
|
||||||
let file_size = data_offset + object_data.len();
|
|
||||||
|
|
||||||
let mut file = Vec::new();
|
|
||||||
push_u32_be(&mut file, metadata.len() as u32);
|
|
||||||
push_u32_be(&mut file, file_size as u32);
|
|
||||||
push_u32_be(&mut file, 22);
|
|
||||||
push_u32_be(&mut file, 0);
|
|
||||||
file.push(0);
|
|
||||||
file.extend_from_slice(&[0, 0, 0]);
|
|
||||||
push_u32_be(&mut file, metadata.len() as u32);
|
|
||||||
push_u64_be(&mut file, file_size as u64);
|
|
||||||
push_u64_be(&mut file, data_offset as u64);
|
|
||||||
push_u64_be(&mut file, 0);
|
|
||||||
file.extend_from_slice(&metadata);
|
|
||||||
file.extend_from_slice(&object_data);
|
|
||||||
file
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn parses_synthetic_text_asset() {
|
|
||||||
let file = synthetic_serialized_file();
|
|
||||||
let parsed = UnitySerializedFile::from_slice(&file).unwrap();
|
|
||||||
|
|
||||||
assert_eq!(parsed.version, 22);
|
|
||||||
assert_eq!(parsed.unity_version, "2021.3.56f2");
|
|
||||||
assert_eq!(parsed.platform, 19);
|
|
||||||
assert_eq!(parsed.text_assets.len(), 1);
|
|
||||||
|
|
||||||
let asset = parsed.text_asset("GameMainConfig").unwrap();
|
|
||||||
assert_eq!(asset.name, "GameMainConfig");
|
|
||||||
assert_eq!(asset.bytes, b"hello");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
#[ignore = "requires BAT_REAL_RESOURCES_ASSETS pointing at a local resources.assets"]
|
|
||||||
fn reads_text_asset_from_real_resource_file() {
|
|
||||||
let path = std::env::var("BAT_REAL_RESOURCES_ASSETS")
|
|
||||||
.expect("BAT_REAL_RESOURCES_ASSETS must be set");
|
|
||||||
|
|
||||||
let parsed =
|
|
||||||
UnitySerializedFile::from_path(Path::new(&path)).expect("parse local resources.assets");
|
|
||||||
let asset = parsed
|
|
||||||
.text_asset("GameMainConfig")
|
|
||||||
.expect("GameMainConfig TextAsset present");
|
|
||||||
|
|
||||||
assert_eq!(asset.name, "GameMainConfig");
|
|
||||||
assert!(!asset.bytes.is_empty());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -1,358 +1,24 @@
|
|||||||
//! Unity 2021.3 Adapter
|
//! Unity 2021.3 adapter.
|
||||||
//!
|
//!
|
||||||
//! 支持 Unity 2021.3.x 版本的 AssetBundle
|
//! 该层只负责 Unity 版本选择;UnityFS 容器解析由 `bat-assetbundle` 引擎承担。
|
||||||
|
|
||||||
use super::adapter::{
|
use super::adapter::{ParsedAssetBundle, RawAssetBundle, UnityAdapter, VersionRange};
|
||||||
ParsedAssetBundle, RawAssetBundle, UnityAdapter, UnityFsBlockInfo, UnityFsCompression,
|
|
||||||
UnityFsDirectoryInfo, UnityFsHeader, VersionRange,
|
|
||||||
};
|
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
use std::io::Cursor;
|
use bat_assetbundle::UnityFsParser;
|
||||||
|
|
||||||
const SERIALIZE_NOT_IMPLEMENTED: &str = "serialize() 将在 Phase 2 实现";
|
const SERIALIZE_NOT_IMPLEMENTED: &str = "serialize() 尚未实现";
|
||||||
const UNITYFS_COMPRESSION_MASK: u32 = 0x3f;
|
|
||||||
const UNITYFS_BLOCK_INFO_AT_END_FLAG: u32 = 0x80;
|
|
||||||
const UNITYFS_ALIGNMENT: usize = 16;
|
|
||||||
|
|
||||||
/// Unity 2021.3 Adapter
|
/// Unity 2021.3 adapter.
|
||||||
pub struct Unity2021_3Adapter;
|
pub struct Unity2021_3Adapter;
|
||||||
|
|
||||||
impl Unity2021_3Adapter {
|
impl Unity2021_3Adapter {
|
||||||
/// 创建新的适配器实例
|
/// Creates an adapter instance.
|
||||||
pub fn new() -> Self {
|
pub fn new() -> Self {
|
||||||
Self
|
Self
|
||||||
}
|
}
|
||||||
|
|
||||||
fn unity_version_bytes(data: &[u8]) -> Option<&[u8]> {
|
|
||||||
if data.len() < 20 {
|
|
||||||
return None;
|
|
||||||
}
|
|
||||||
|
|
||||||
if &data[0..7] != b"UnityFS" {
|
|
||||||
return None;
|
|
||||||
}
|
|
||||||
|
|
||||||
let version_start = data.windows(7).position(|window| window == b"2021.3.")?;
|
|
||||||
let version_bytes = &data[version_start..];
|
|
||||||
let version_end = version_bytes
|
|
||||||
.iter()
|
|
||||||
.position(|&byte| byte == 0 || !byte.is_ascii())?;
|
|
||||||
|
|
||||||
Some(&version_bytes[..version_end])
|
|
||||||
}
|
|
||||||
|
|
||||||
/// 检测 Unity 版本(从文件头)
|
|
||||||
fn detect_unity_version(data: &[u8]) -> Option<String> {
|
fn detect_unity_version(data: &[u8]) -> Option<String> {
|
||||||
let version_bytes = Self::unity_version_bytes(data)?;
|
UnityFsParser::detect_unity_version(data)
|
||||||
std::str::from_utf8(version_bytes)
|
|
||||||
.map(ToOwned::to_owned)
|
|
||||||
.ok()
|
|
||||||
}
|
|
||||||
|
|
||||||
fn parse_unityfs(data: &[u8]) -> Result<ParsedAssetBundle, String> {
|
|
||||||
let mut reader = UnityFsReader::new(data);
|
|
||||||
let signature = reader.read_c_string("signature")?;
|
|
||||||
if signature != "UnityFS" {
|
|
||||||
return Err(format!("Unsupported AssetBundle signature: {}", signature));
|
|
||||||
}
|
|
||||||
|
|
||||||
let format_version = reader.read_u32("format_version")?;
|
|
||||||
let target_version = reader.read_c_string("target_version")?;
|
|
||||||
let unity_version = reader.read_c_string("unity_version")?;
|
|
||||||
let total_size = reader.read_u64("total_size")?;
|
|
||||||
let compressed_blocks_info_size = reader.read_u32("compressed_blocks_info_size")?;
|
|
||||||
let uncompressed_blocks_info_size = reader.read_u32("uncompressed_blocks_info_size")?;
|
|
||||||
let flags = reader.read_u32("flags")?;
|
|
||||||
|
|
||||||
let header = UnityFsHeader {
|
|
||||||
format_version,
|
|
||||||
target_version,
|
|
||||||
unity_version,
|
|
||||||
total_size,
|
|
||||||
compressed_blocks_info_size,
|
|
||||||
uncompressed_blocks_info_size,
|
|
||||||
flags,
|
|
||||||
};
|
|
||||||
|
|
||||||
if format_version >= 7 {
|
|
||||||
reader.align(UNITYFS_ALIGNMENT)?;
|
|
||||||
}
|
|
||||||
|
|
||||||
let blocks_info_bytes = read_blocks_info_bytes(data, &mut reader, &header)?;
|
|
||||||
let block_info = decompress_blocks_info(
|
|
||||||
blocks_info_bytes,
|
|
||||||
compressed_blocks_info_size,
|
|
||||||
uncompressed_blocks_info_size,
|
|
||||||
flags,
|
|
||||||
)?;
|
|
||||||
let (blocks, directories) = Self::parse_blocks_info(&block_info)?;
|
|
||||||
validate_directory_bounds(&blocks, &directories)?;
|
|
||||||
|
|
||||||
Ok(ParsedAssetBundle {
|
|
||||||
unity_version: header.unity_version.clone(),
|
|
||||||
assets: directories
|
|
||||||
.iter()
|
|
||||||
.map(|directory| directory.path.clone())
|
|
||||||
.collect(),
|
|
||||||
raw_data: data.to_vec(),
|
|
||||||
unityfs_header: Some(header),
|
|
||||||
blocks,
|
|
||||||
directories,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
fn parse_blocks_info(
|
|
||||||
data: &[u8],
|
|
||||||
) -> Result<(Vec<UnityFsBlockInfo>, Vec<UnityFsDirectoryInfo>), String> {
|
|
||||||
let mut reader = UnityFsReader::new(data);
|
|
||||||
let _hash = reader.read_bytes(16, "blocks_info_hash")?;
|
|
||||||
let block_count = reader.read_i32("block_count")?;
|
|
||||||
if block_count < 0 {
|
|
||||||
return Err(format!("Invalid UnityFS block count: {}", block_count));
|
|
||||||
}
|
|
||||||
|
|
||||||
let mut blocks = Vec::with_capacity(block_count as usize);
|
|
||||||
for _ in 0..block_count {
|
|
||||||
let uncompressed_size = reader.read_u32("block_uncompressed_size")?;
|
|
||||||
let compressed_size = reader.read_u32("block_compressed_size")?;
|
|
||||||
let flags = reader.read_u16("block_flags")?;
|
|
||||||
blocks.push(UnityFsBlockInfo {
|
|
||||||
uncompressed_size,
|
|
||||||
compressed_size,
|
|
||||||
flags,
|
|
||||||
compression: compression_from_flags(flags),
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
let directory_count = reader.read_i32("directory_count")?;
|
|
||||||
if directory_count < 0 {
|
|
||||||
return Err(format!(
|
|
||||||
"Invalid UnityFS directory count: {}",
|
|
||||||
directory_count
|
|
||||||
));
|
|
||||||
}
|
|
||||||
|
|
||||||
let mut directories = Vec::with_capacity(directory_count as usize);
|
|
||||||
for _ in 0..directory_count {
|
|
||||||
directories.push(UnityFsDirectoryInfo {
|
|
||||||
offset: reader.read_u64("directory_offset")?,
|
|
||||||
size: reader.read_u64("directory_size")?,
|
|
||||||
flags: reader.read_u32("directory_flags")?,
|
|
||||||
path: reader.read_c_string("directory_path")?,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
Ok((blocks, directories))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// 校验目录条目落在解压数据区内。
|
|
||||||
///
|
|
||||||
/// UnityFS 的 directory 是解压后(所有 block 的 uncompressed 数据依次拼接而成的)
|
|
||||||
/// 连续数据区上的 `[offset, offset + size)` 切片。解析阶段只按结构读取这些数值,
|
|
||||||
/// 并不保证它们不越界;截断或损坏的 bundle 会给出指向数据区之外的目录条目,
|
|
||||||
/// 若不校验就静默接受,后续按 offset/size 取数据时才会出错或读到错误内容。
|
|
||||||
/// 这里把每个目录条目与「各 block 解压大小之和」比对,越界即报错并带上下文。
|
|
||||||
fn validate_directory_bounds(
|
|
||||||
blocks: &[UnityFsBlockInfo],
|
|
||||||
directories: &[UnityFsDirectoryInfo],
|
|
||||||
) -> Result<(), String> {
|
|
||||||
let mut data_region_size: u64 = 0;
|
|
||||||
for (index, block) in blocks.iter().enumerate() {
|
|
||||||
data_region_size = data_region_size
|
|
||||||
.checked_add(u64::from(block.uncompressed_size))
|
|
||||||
.ok_or_else(|| {
|
|
||||||
format!(
|
|
||||||
"UnityFS 解压数据区大小溢出:累加到第 {index} 个 block(uncompressed_size={})时超过 u64",
|
|
||||||
block.uncompressed_size
|
|
||||||
)
|
|
||||||
})?;
|
|
||||||
}
|
|
||||||
|
|
||||||
for (index, directory) in directories.iter().enumerate() {
|
|
||||||
let end = directory
|
|
||||||
.offset
|
|
||||||
.checked_add(directory.size)
|
|
||||||
.ok_or_else(|| {
|
|
||||||
format!(
|
|
||||||
"UnityFS 目录条目 {} 的 offset({}) + size({}) 溢出 u64",
|
|
||||||
directory.path, directory.offset, directory.size
|
|
||||||
)
|
|
||||||
})?;
|
|
||||||
if end > data_region_size {
|
|
||||||
return Err(format!(
|
|
||||||
"UnityFS 目录条目 {}(第 {index} 项)越界:offset({}) + size({}) = {} 超过解压数据区大小 {}",
|
|
||||||
directory.path, directory.offset, directory.size, end, data_region_size
|
|
||||||
));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_blocks_info_bytes<'a>(
|
|
||||||
data: &'a [u8],
|
|
||||||
reader: &mut UnityFsReader<'a>,
|
|
||||||
header: &UnityFsHeader,
|
|
||||||
) -> Result<&'a [u8], String> {
|
|
||||||
let len = header.compressed_blocks_info_size as usize;
|
|
||||||
if blocks_info_at_end(header.flags) {
|
|
||||||
let start = data.len().checked_sub(len).ok_or_else(|| {
|
|
||||||
format!(
|
|
||||||
"UnityFS block info at end underflow: compressed size {}, file size {}",
|
|
||||||
len,
|
|
||||||
data.len()
|
|
||||||
)
|
|
||||||
})?;
|
|
||||||
return Ok(&data[start..]);
|
|
||||||
}
|
|
||||||
|
|
||||||
reader.read_bytes(len, "blocks_info")
|
|
||||||
}
|
|
||||||
|
|
||||||
fn blocks_info_at_end(flags: u32) -> bool {
|
|
||||||
flags & UNITYFS_BLOCK_INFO_AT_END_FLAG != 0
|
|
||||||
}
|
|
||||||
|
|
||||||
fn decompress_blocks_info(
|
|
||||||
data: &[u8],
|
|
||||||
compressed_size: u32,
|
|
||||||
uncompressed_size: u32,
|
|
||||||
flags: u32,
|
|
||||||
) -> Result<Vec<u8>, String> {
|
|
||||||
if data.len() != compressed_size as usize {
|
|
||||||
return Err(format!(
|
|
||||||
"UnityFS block info size mismatch: header says {}, read {}",
|
|
||||||
compressed_size,
|
|
||||||
data.len()
|
|
||||||
));
|
|
||||||
}
|
|
||||||
|
|
||||||
let compression = compression_from_flags((flags & UNITYFS_COMPRESSION_MASK) as u16);
|
|
||||||
match compression {
|
|
||||||
UnityFsCompression::None => {
|
|
||||||
if compressed_size != uncompressed_size {
|
|
||||||
return Err(format!(
|
|
||||||
"Uncompressed UnityFS block info size mismatch: compressed {} != uncompressed {}",
|
|
||||||
compressed_size, uncompressed_size
|
|
||||||
));
|
|
||||||
}
|
|
||||||
Ok(data.to_vec())
|
|
||||||
}
|
|
||||||
UnityFsCompression::Lz4 | UnityFsCompression::Lz4Hc => {
|
|
||||||
lz4::block::decompress(data, Some(uncompressed_size as i32))
|
|
||||||
.map_err(|error| format!("Failed to decompress UnityFS LZ4 block info: {}", error))
|
|
||||||
}
|
|
||||||
UnityFsCompression::Lzma => {
|
|
||||||
let mut output = Vec::with_capacity(uncompressed_size as usize);
|
|
||||||
lzma_rs::lzma_decompress(&mut Cursor::new(data), &mut output).map_err(|error| {
|
|
||||||
format!("Failed to decompress UnityFS LZMA block info: {}", error)
|
|
||||||
})?;
|
|
||||||
if output.len() != uncompressed_size as usize {
|
|
||||||
return Err(format!(
|
|
||||||
"UnityFS LZMA block info size mismatch: expected {}, got {}",
|
|
||||||
uncompressed_size,
|
|
||||||
output.len()
|
|
||||||
));
|
|
||||||
}
|
|
||||||
Ok(output)
|
|
||||||
}
|
|
||||||
UnityFsCompression::Unknown(value) => Err(format!(
|
|
||||||
"Unsupported UnityFS block info compression flag: {}",
|
|
||||||
value
|
|
||||||
)),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn compression_from_flags(flags: u16) -> UnityFsCompression {
|
|
||||||
match flags & UNITYFS_COMPRESSION_MASK as u16 {
|
|
||||||
0 => UnityFsCompression::None,
|
|
||||||
1 => UnityFsCompression::Lzma,
|
|
||||||
2 => UnityFsCompression::Lz4,
|
|
||||||
3 | 4 => UnityFsCompression::Lz4Hc,
|
|
||||||
value => UnityFsCompression::Unknown(value),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
struct UnityFsReader<'a> {
|
|
||||||
data: &'a [u8],
|
|
||||||
offset: usize,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl<'a> UnityFsReader<'a> {
|
|
||||||
fn new(data: &'a [u8]) -> Self {
|
|
||||||
Self { data, offset: 0 }
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_bytes(&mut self, len: usize, field: &str) -> Result<&'a [u8], String> {
|
|
||||||
let end = self
|
|
||||||
.offset
|
|
||||||
.checked_add(len)
|
|
||||||
.ok_or_else(|| format!("{} length overflow at offset {}", field, self.offset))?;
|
|
||||||
if end > self.data.len() {
|
|
||||||
return Err(format!(
|
|
||||||
"Unexpected end while reading {} at offset {}: need {}, have {}",
|
|
||||||
field,
|
|
||||||
self.offset,
|
|
||||||
len,
|
|
||||||
self.data.len().saturating_sub(self.offset)
|
|
||||||
));
|
|
||||||
}
|
|
||||||
|
|
||||||
let bytes = &self.data[self.offset..end];
|
|
||||||
self.offset = end;
|
|
||||||
Ok(bytes)
|
|
||||||
}
|
|
||||||
|
|
||||||
fn align(&mut self, alignment: usize) -> Result<(), String> {
|
|
||||||
if alignment == 0 {
|
|
||||||
return Ok(());
|
|
||||||
}
|
|
||||||
|
|
||||||
let remainder = self.offset % alignment;
|
|
||||||
if remainder == 0 {
|
|
||||||
return Ok(());
|
|
||||||
}
|
|
||||||
|
|
||||||
let padding = alignment - remainder;
|
|
||||||
self.read_bytes(padding, "alignment padding").map(|_| ())
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_u16(&mut self, field: &str) -> Result<u16, String> {
|
|
||||||
let bytes = self.read_bytes(2, field)?;
|
|
||||||
Ok(u16::from_be_bytes([bytes[0], bytes[1]]))
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_u32(&mut self, field: &str) -> Result<u32, String> {
|
|
||||||
let bytes = self.read_bytes(4, field)?;
|
|
||||||
Ok(u32::from_be_bytes([bytes[0], bytes[1], bytes[2], bytes[3]]))
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_i32(&mut self, field: &str) -> Result<i32, String> {
|
|
||||||
let bytes = self.read_bytes(4, field)?;
|
|
||||||
Ok(i32::from_be_bytes([bytes[0], bytes[1], bytes[2], bytes[3]]))
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_u64(&mut self, field: &str) -> Result<u64, String> {
|
|
||||||
let bytes = self.read_bytes(8, field)?;
|
|
||||||
Ok(u64::from_be_bytes([
|
|
||||||
bytes[0], bytes[1], bytes[2], bytes[3], bytes[4], bytes[5], bytes[6], bytes[7],
|
|
||||||
]))
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_c_string(&mut self, field: &str) -> Result<String, String> {
|
|
||||||
let remaining = &self.data[self.offset..];
|
|
||||||
let Some(length) = remaining.iter().position(|&byte| byte == 0) else {
|
|
||||||
return Err(format!(
|
|
||||||
"Missing null terminator while reading {} at offset {}",
|
|
||||||
field, self.offset
|
|
||||||
));
|
|
||||||
};
|
|
||||||
let bytes = self.read_bytes(length, field)?;
|
|
||||||
self.offset += 1;
|
|
||||||
std::str::from_utf8(bytes)
|
|
||||||
.map(ToOwned::to_owned)
|
|
||||||
.map_err(|error| format!("Invalid UTF-8 in {}: {}", field, error))
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -373,16 +39,15 @@ impl UnityAdapter for Unity2021_3Adapter {
|
|||||||
}
|
}
|
||||||
|
|
||||||
fn can_handle(&self, bundle: &RawAssetBundle) -> bool {
|
fn can_handle(&self, bundle: &RawAssetBundle) -> bool {
|
||||||
// 检测 Unity 版本
|
Self::detect_unity_version(&bundle.data)
|
||||||
if let Some(version) = Self::detect_unity_version(&bundle.data) {
|
.map(|version| self.supported_versions().contains(&version))
|
||||||
self.supported_versions().contains(&version)
|
.unwrap_or(false)
|
||||||
} else {
|
|
||||||
false
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn parse(&self, bundle: &RawAssetBundle) -> Result<ParsedAssetBundle, String> {
|
async fn parse(&self, bundle: &RawAssetBundle) -> Result<ParsedAssetBundle, String> {
|
||||||
let parsed = Self::parse_unityfs(&bundle.data)?;
|
let parsed = UnityFsParser::new()
|
||||||
|
.parse_asset_bundle(bundle)
|
||||||
|
.map_err(|error| error.to_string())?;
|
||||||
if !self.supported_versions().contains(&parsed.unity_version) {
|
if !self.supported_versions().contains(&parsed.unity_version) {
|
||||||
return Err(format!(
|
return Err(format!(
|
||||||
"Unsupported Unity version for {}: {}",
|
"Unsupported Unity version for {}: {}",
|
||||||
@@ -394,12 +59,6 @@ impl UnityAdapter for Unity2021_3Adapter {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async fn serialize(&self, _parsed: &ParsedAssetBundle) -> Result<Vec<u8>, String> {
|
async fn serialize(&self, _parsed: &ParsedAssetBundle) -> Result<Vec<u8>, String> {
|
||||||
// TODO: Phase 2 实现
|
|
||||||
// 需要:
|
|
||||||
// 1. 序列化 Asset 对象
|
|
||||||
// 2. 重新构建 TypeTree
|
|
||||||
// 3. 压缩数据块
|
|
||||||
// 4. 写入 UnityFS 文件头
|
|
||||||
Err(SERIALIZE_NOT_IMPLEMENTED.to_string())
|
Err(SERIALIZE_NOT_IMPLEMENTED.to_string())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -407,6 +66,9 @@ impl UnityAdapter for Unity2021_3Adapter {
|
|||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
use crate::unity::UnityFsCompression;
|
||||||
|
|
||||||
|
const UNITYFS_ALIGNMENT: usize = 16;
|
||||||
|
|
||||||
fn push_c_string(data: &mut Vec<u8>, value: &str) {
|
fn push_c_string(data: &mut Vec<u8>, value: &str) {
|
||||||
data.extend_from_slice(value.as_bytes());
|
data.extend_from_slice(value.as_bytes());
|
||||||
@@ -436,7 +98,7 @@ mod tests {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
fn synthetic_minimal_unityfs_bundle() -> Vec<u8> {
|
fn synthetic_minimal_unityfs_bundle(unity_version: &str) -> Vec<u8> {
|
||||||
let mut blocks_info = Vec::new();
|
let mut blocks_info = Vec::new();
|
||||||
blocks_info.extend_from_slice(&[0; 16]);
|
blocks_info.extend_from_slice(&[0; 16]);
|
||||||
push_i32(&mut blocks_info, 1);
|
push_i32(&mut blocks_info, 1);
|
||||||
@@ -453,7 +115,7 @@ mod tests {
|
|||||||
push_c_string(&mut data, "UnityFS");
|
push_c_string(&mut data, "UnityFS");
|
||||||
push_u32(&mut data, 8);
|
push_u32(&mut data, 8);
|
||||||
push_c_string(&mut data, "5.x.x");
|
push_c_string(&mut data, "5.x.x");
|
||||||
push_c_string(&mut data, "2021.3.56f2");
|
push_c_string(&mut data, unity_version);
|
||||||
push_u64(&mut data, 0);
|
push_u64(&mut data, 0);
|
||||||
push_u32(&mut data, blocks_info.len() as u32);
|
push_u32(&mut data, blocks_info.len() as u32);
|
||||||
push_u32(&mut data, blocks_info.len() as u32);
|
push_u32(&mut data, blocks_info.len() as u32);
|
||||||
@@ -463,7 +125,7 @@ mod tests {
|
|||||||
data.extend_from_slice(b"data");
|
data.extend_from_slice(b"data");
|
||||||
|
|
||||||
let total_size = data.len() as u64;
|
let total_size = data.len() as u64;
|
||||||
let total_size_offset = b"UnityFS\0".len() + 4 + b"5.x.x\0".len() + b"2021.3.56f2\0".len();
|
let total_size_offset = b"UnityFS\0".len() + 4 + b"5.x.x\0".len() + unity_version.len() + 1;
|
||||||
data[total_size_offset..total_size_offset + 8].copy_from_slice(&total_size.to_be_bytes());
|
data[total_size_offset..total_size_offset + 8].copy_from_slice(&total_size.to_be_bytes());
|
||||||
data
|
data
|
||||||
}
|
}
|
||||||
@@ -490,7 +152,7 @@ mod tests {
|
|||||||
let adapter = Unity2021_3Adapter::new();
|
let adapter = Unity2021_3Adapter::new();
|
||||||
|
|
||||||
let bundle = RawAssetBundle {
|
let bundle = RawAssetBundle {
|
||||||
data: synthetic_minimal_unityfs_bundle(),
|
data: synthetic_minimal_unityfs_bundle("2021.3.56f2"),
|
||||||
path: Some("synthetic-minimal.bundle".to_string()),
|
path: Some("synthetic-minimal.bundle".to_string()),
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -514,7 +176,7 @@ mod tests {
|
|||||||
let adapter = Unity2021_3Adapter::new();
|
let adapter = Unity2021_3Adapter::new();
|
||||||
|
|
||||||
let bundle = RawAssetBundle {
|
let bundle = RawAssetBundle {
|
||||||
data: synthetic_minimal_unityfs_bundle(),
|
data: synthetic_minimal_unityfs_bundle("2021.3.56f2"),
|
||||||
path: Some("synthetic-minimal.bundle".to_string()),
|
path: Some("synthetic-minimal.bundle".to_string()),
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -538,85 +200,34 @@ mod tests {
|
|||||||
path: None,
|
path: None,
|
||||||
};
|
};
|
||||||
|
|
||||||
let result = adapter.parse(&bundle).await;
|
let error = adapter.parse(&bundle).await.unwrap_err();
|
||||||
assert!(result.is_err());
|
assert!(error.contains("signature"), "{error}");
|
||||||
assert!(result.unwrap_err().contains("signature"));
|
|
||||||
}
|
|
||||||
|
|
||||||
fn block(uncompressed_size: u32) -> UnityFsBlockInfo {
|
|
||||||
UnityFsBlockInfo {
|
|
||||||
uncompressed_size,
|
|
||||||
compressed_size: uncompressed_size,
|
|
||||||
flags: 0,
|
|
||||||
compression: UnityFsCompression::None,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn directory(offset: u64, size: u64) -> UnityFsDirectoryInfo {
|
|
||||||
UnityFsDirectoryInfo {
|
|
||||||
offset,
|
|
||||||
size,
|
|
||||||
flags: 0,
|
|
||||||
path: "CAB-test".to_string(),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn directory_within_data_region_is_accepted() {
|
|
||||||
// 两个 block 共 12 字节解压数据区;目录条目正好覆盖尾部,合法。
|
|
||||||
let result =
|
|
||||||
validate_directory_bounds(&[block(8), block(4)], &[directory(0, 8), directory(8, 4)]);
|
|
||||||
assert!(result.is_ok(), "{result:?}");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn directory_past_data_region_is_rejected() {
|
|
||||||
// 解压数据区仅 4 字节,目录声称 [0, 8) 越界,应被拒绝并带上下文。
|
|
||||||
let error = validate_directory_bounds(&[block(4)], &[directory(0, 8)]).unwrap_err();
|
|
||||||
assert!(error.contains("越界"), "{error}");
|
|
||||||
assert!(error.contains("解压数据区大小 4"), "{error}");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn directory_offset_size_overflow_is_rejected() {
|
|
||||||
let error = validate_directory_bounds(&[block(4)], &[directory(u64::MAX, 1)]).unwrap_err();
|
|
||||||
assert!(error.contains("溢出"), "{error}");
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#[tokio::test]
|
#[tokio::test]
|
||||||
async fn test_parse_rejects_out_of_bounds_directory() {
|
async fn test_parse_rejects_unsupported_unity_version() {
|
||||||
// 构造一个 directory.size 超过 block 解压大小的 bundle,端到端验证被拒。
|
let adapter = Unity2021_3Adapter::new();
|
||||||
let mut blocks_info = Vec::new();
|
|
||||||
blocks_info.extend_from_slice(&[0; 16]);
|
|
||||||
push_i32(&mut blocks_info, 1);
|
|
||||||
push_u32(&mut blocks_info, 4); // block uncompressed_size = 4
|
|
||||||
push_u32(&mut blocks_info, 4);
|
|
||||||
push_u16(&mut blocks_info, 0);
|
|
||||||
push_i32(&mut blocks_info, 1);
|
|
||||||
push_u64(&mut blocks_info, 0);
|
|
||||||
push_u64(&mut blocks_info, 99); // directory size 99 远超数据区
|
|
||||||
push_u32(&mut blocks_info, 0);
|
|
||||||
push_c_string(&mut blocks_info, "CAB-test");
|
|
||||||
|
|
||||||
let mut data = Vec::new();
|
let bundle = RawAssetBundle {
|
||||||
push_c_string(&mut data, "UnityFS");
|
data: synthetic_minimal_unityfs_bundle("2022.3.1f1"),
|
||||||
push_u32(&mut data, 8);
|
path: Some("unsupported.bundle".to_string()),
|
||||||
push_c_string(&mut data, "5.x.x");
|
};
|
||||||
push_c_string(&mut data, "2021.3.56f2");
|
|
||||||
push_u64(&mut data, 0);
|
|
||||||
push_u32(&mut data, blocks_info.len() as u32);
|
|
||||||
push_u32(&mut data, blocks_info.len() as u32);
|
|
||||||
push_u32(&mut data, 0);
|
|
||||||
align(&mut data, UNITYFS_ALIGNMENT);
|
|
||||||
data.extend_from_slice(&blocks_info);
|
|
||||||
data.extend_from_slice(b"data");
|
|
||||||
|
|
||||||
|
let error = adapter.parse(&bundle).await.unwrap_err();
|
||||||
|
assert!(error.contains("Unsupported Unity version"), "{error}");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn serialize_returns_explicit_not_implemented_error() {
|
||||||
let adapter = Unity2021_3Adapter::new();
|
let adapter = Unity2021_3Adapter::new();
|
||||||
let bundle = RawAssetBundle {
|
let bundle = RawAssetBundle {
|
||||||
data,
|
data: synthetic_minimal_unityfs_bundle("2021.3.56f2"),
|
||||||
path: Some("out-of-bounds.bundle".to_string()),
|
path: Some("synthetic-minimal.bundle".to_string()),
|
||||||
};
|
};
|
||||||
let error = adapter.parse(&bundle).await.unwrap_err();
|
let parsed = adapter.parse(&bundle).await.unwrap();
|
||||||
assert!(error.contains("越界"), "{error}");
|
|
||||||
|
let error = adapter.serialize(&parsed).await.unwrap_err();
|
||||||
|
|
||||||
|
assert_eq!(error, SERIALIZE_NOT_IMPLEMENTED);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -35,7 +35,11 @@
|
|||||||
}
|
}
|
||||||
],
|
],
|
||||||
"metadata": {
|
"metadata": {
|
||||||
|
"asset_bundle_count": "3",
|
||||||
|
"declared_size_count": "3",
|
||||||
|
"dependency_count": "1",
|
||||||
"internal_id_count": "3",
|
"internal_id_count": "3",
|
||||||
|
"resource_count": "3",
|
||||||
"resource_type_count": "1",
|
"resource_type_count": "1",
|
||||||
"key_object_count": "4",
|
"key_object_count": "4",
|
||||||
"bucket_record_count": "4",
|
"bucket_record_count": "4",
|
||||||
|
|||||||
@@ -21,4 +21,6 @@ async fn parses_local_real_unityfs_bundle() {
|
|||||||
assert_eq!(parsed.unity_version, "2021.3.56f2");
|
assert_eq!(parsed.unity_version, "2021.3.56f2");
|
||||||
assert!(!parsed.blocks.is_empty());
|
assert!(!parsed.blocks.is_empty());
|
||||||
assert!(!parsed.directories.is_empty());
|
assert!(!parsed.directories.is_empty());
|
||||||
|
assert_eq!(parsed.files.len(), parsed.directories.len());
|
||||||
|
assert!(parsed.files.iter().all(|file| !file.data.is_empty()));
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,3 @@
|
|||||||
|
# Reserved empty directory
|
||||||
|
|
||||||
|
Placeholder only. **Not implemented.** See `docs/reports/GO_STATUS.md`.
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
# bat-api OpenAPI
|
||||||
|
|
||||||
|
`bat-api.yaml` describes the current resource bootstrap / read-only distribution
|
||||||
|
HTTP surface. The running service also exposes the same contract at
|
||||||
|
`GET /openapi.yaml`.
|
||||||
|
|
||||||
|
This contract covers resource bootstrap, launcher resource compatibility,
|
||||||
|
server-info rewrite, CDN-shaped resource bytes, auth schemes, and the reserved
|
||||||
|
admin panel entry. It does not describe a full game business API.
|
||||||
@@ -0,0 +1,138 @@
|
|||||||
|
openapi: 3.0.3
|
||||||
|
info:
|
||||||
|
title: BlueArchive Toolkit bat-api
|
||||||
|
version: 0.1.0
|
||||||
|
description: Resource bootstrap and read-only distribution API.
|
||||||
|
servers:
|
||||||
|
- url: http://127.0.0.1:18080
|
||||||
|
security:
|
||||||
|
- bearerAuth: []
|
||||||
|
- queryToken: []
|
||||||
|
paths:
|
||||||
|
/healthz:
|
||||||
|
get:
|
||||||
|
summary: Liveness and refresh diagnostics
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Service is alive.
|
||||||
|
/readyz:
|
||||||
|
get:
|
||||||
|
summary: Release readiness
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: A distributable release is available.
|
||||||
|
"503":
|
||||||
|
description: No distributable release is available.
|
||||||
|
/v1/bootstrap:
|
||||||
|
get:
|
||||||
|
summary: Startup resource bootstrap
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Resource bootstrap response.
|
||||||
|
"503":
|
||||||
|
description: Release is not ready.
|
||||||
|
/v1/launcher/bootstrap:
|
||||||
|
get:
|
||||||
|
summary: Launcher-shaped resource bootstrap
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Launcher bootstrap response.
|
||||||
|
"503":
|
||||||
|
description: Release is not ready.
|
||||||
|
/api/launcher/game/config:
|
||||||
|
get:
|
||||||
|
summary: Resource-only launcher game config compatibility
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Launcher envelope with resource metadata.
|
||||||
|
/api/launcher/game/config/json:
|
||||||
|
get:
|
||||||
|
summary: Resource-only launcher manifest URL compatibility
|
||||||
|
parameters:
|
||||||
|
- name: version
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
- name: file_path
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Launcher envelope pointing to resource bootstrap JSON.
|
||||||
|
/api/launcher/advanced/game/download/cdn:
|
||||||
|
get:
|
||||||
|
summary: Resource-only launcher CDN compatibility
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Launcher envelope with public base URL as CDN root.
|
||||||
|
/v1/release:
|
||||||
|
get:
|
||||||
|
summary: Current release summary
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Release summary.
|
||||||
|
/v1/resources:
|
||||||
|
get:
|
||||||
|
summary: Paginated resource manifest entries
|
||||||
|
parameters:
|
||||||
|
- name: offset
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: integer
|
||||||
|
minimum: 0
|
||||||
|
- name: limit
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: integer
|
||||||
|
minimum: 1
|
||||||
|
maximum: 1000
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Resource list page.
|
||||||
|
/v1/server-info:
|
||||||
|
get:
|
||||||
|
summary: Rewritten server-info document
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Server-info JSON with AddressablesCatalogUrlRoot rewritten.
|
||||||
|
/openapi.yaml:
|
||||||
|
get:
|
||||||
|
summary: OpenAPI document
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: OpenAPI YAML.
|
||||||
|
/admin/:
|
||||||
|
get:
|
||||||
|
summary: Reserved admin panel entry
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Admin panel placeholder and links.
|
||||||
|
/prod-clientpatch.bluearchiveyostar.com/{path}:
|
||||||
|
get:
|
||||||
|
summary: CDN-shaped resource bytes
|
||||||
|
parameters:
|
||||||
|
- name: path
|
||||||
|
in: path
|
||||||
|
required: true
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Resource bytes.
|
||||||
|
"206":
|
||||||
|
description: Partial resource bytes.
|
||||||
|
head:
|
||||||
|
summary: CDN-shaped resource metadata
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Resource headers.
|
||||||
|
components:
|
||||||
|
securitySchemes:
|
||||||
|
bearerAuth:
|
||||||
|
type: http
|
||||||
|
scheme: bearer
|
||||||
|
queryToken:
|
||||||
|
type: apiKey
|
||||||
|
in: query
|
||||||
|
name: bat_token
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
# Reserved empty directory
|
||||||
|
|
||||||
|
This path is a monorepo placeholder and is **not implemented**.
|
||||||
|
Do not treat it as a finished module. See `docs/reports/GO_STATUS.md`.
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
# bat-api configuration example (copy to .env next to the binary or export)
|
||||||
|
# Priority: CLI flags > process environment > .env > built-in defaults.
|
||||||
|
#
|
||||||
|
# Boundary:
|
||||||
|
# - Rust bat: resource auto-discover / pull / verify / publish / daemon RPC
|
||||||
|
# - bat-api: resource bootstrap + read-only distribution (official CDN-shaped paths)
|
||||||
|
# + management APIs
|
||||||
|
|
||||||
|
BAT_API_LISTEN=:18080
|
||||||
|
BAT_API_PUBLIC_BASE_URL=http://127.0.0.1:18080
|
||||||
|
|
||||||
|
# Primary discovery: bat daemon JSON-RPC socket file
|
||||||
|
BAT_API_STATE_DIR=/tmp/bat-pid
|
||||||
|
# BAT_API_SOCKET=/tmp/bat-pid/bat.sock
|
||||||
|
|
||||||
|
# Optional release root override (local fixtures / emergency read-only diagnostics only).
|
||||||
|
# Production obtains resource_root from BAT_API_SOCKET RPC; do not set this there.
|
||||||
|
# BAT_API_RESOURCE_ROOT=
|
||||||
|
|
||||||
|
# BAT_API_SERVER_INFO_FILE=
|
||||||
|
BAT_API_REQUIRE_INDEXED=true
|
||||||
|
BAT_API_VERIFY_SIZE=true
|
||||||
|
BAT_API_RPC_TIMEOUT=30s
|
||||||
|
# Periodically re-read bat.sock so bat-api follows Rust bat release switches.
|
||||||
|
# Set to 0 in fixture-only local development.
|
||||||
|
BAT_API_REFRESH_INTERVAL=1m
|
||||||
|
|
||||||
|
# Reserved for future API persistence
|
||||||
|
# BAT_API_DATABASE_URL=postgres://bat:@127.0.0.1:5432/bat?sslmode=disable
|
||||||
|
# BAT_API_DATABASE_PASSWORD=
|
||||||
|
# BAT_API_REDIS_URL=redis://127.0.0.1:6379/0
|
||||||
|
# BAT_API_REDIS_PASSWORD=
|
||||||
@@ -0,0 +1,143 @@
|
|||||||
|
// Command bat-api is the resource bootstrap and distribution HTTP service for BlueArchiveToolkit.
|
||||||
|
//
|
||||||
|
// Responsibility boundary:
|
||||||
|
// - bat (Rust): official resource auto-discover, pull, verify, publish, daemon RPC
|
||||||
|
// - bat-api (Go): startup resource bootstrap, server-info rewrite,
|
||||||
|
// read-only distribution of published resources (CDN-shaped paths), release
|
||||||
|
// inspection APIs, and normal process configuration (.env / flags for listen
|
||||||
|
// port, RPC socket, reserved database/redis settings)
|
||||||
|
//
|
||||||
|
// bat-api discovers and periodically refreshes the current release through the
|
||||||
|
// bat.sock JSON-RPC contract (daemon.status first, then daemon.doctor, then
|
||||||
|
// catalog/resource methods). The production resource root comes from RPC; the
|
||||||
|
// resource-root override is for local fixtures or emergency diagnostics.
|
||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"log"
|
||||||
|
"os"
|
||||||
|
"os/signal"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"syscall"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"bat-api/internal/api"
|
||||||
|
"bat-api/internal/backendrpc"
|
||||||
|
)
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
log.SetFlags(log.LstdFlags | log.Lmsgprefix)
|
||||||
|
log.SetPrefix("bat-api ")
|
||||||
|
|
||||||
|
cfg := api.DefaultConfig()
|
||||||
|
if os.Getenv("BAT_API_SKIP_ENV_FILE") != "1" {
|
||||||
|
envPath := envFilePath()
|
||||||
|
if err := ensureEnvTemplate(envPath); err != nil {
|
||||||
|
log.Printf("warn: env template: %v", err)
|
||||||
|
}
|
||||||
|
if err := api.LoadEnvFile(envPath); err != nil {
|
||||||
|
log.Fatalf("load .env: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
api.ApplyEnv(&cfg)
|
||||||
|
|
||||||
|
listen := flag.String("listen", cfg.Listen, "HTTP listen address")
|
||||||
|
publicBase := flag.String("public-base-url", cfg.PublicBaseURL, "public base URL for Addressables rewrite")
|
||||||
|
stateDir := flag.String("state-dir", cfg.StateDir, "bat daemon state dir (derives default socket)")
|
||||||
|
socket := flag.String("socket", cfg.SocketPath, "path to bat.sock JSON-RPC socket (primary discovery)")
|
||||||
|
resourceRoot := flag.String("resource-root", cfg.ResourceRoot, "override published release root (tests/emergency)")
|
||||||
|
serverInfo := flag.String("server-info-file", cfg.ServerInfoFile, "optional server-info JSON path")
|
||||||
|
requireIndexed := flag.Bool("require-indexed", cfg.RequireIndexed, "only serve files present in the release index")
|
||||||
|
verifySize := flag.Bool("verify-size", cfg.VerifySize, "reject CDN files whose size differs from the index")
|
||||||
|
rpcTimeout := flag.Duration("rpc-timeout", cfg.RPCTimeout, "daemon RPC timeout")
|
||||||
|
refreshInterval := flag.Duration("refresh-interval", cfg.RefreshInterval, "periodic release discovery interval (0 disables)")
|
||||||
|
authQueryParam := flag.String("auth-query-param", cfg.AuthQueryParam, "query parameter accepted for token auth fallback")
|
||||||
|
authExemptPaths := flag.String("auth-exempt-paths", strings.Join(cfg.AuthExemptPaths, ","), "comma-separated auth-exempt exact paths or slash-prefixes")
|
||||||
|
trustProxyHeaders := flag.Bool("trust-proxy-headers", cfg.TrustProxyHeaders, "trust X-Forwarded-For and X-Real-IP from reverse proxy")
|
||||||
|
accessLog := flag.Bool("access-log", cfg.AccessLog, "enable per-request access logs without query strings")
|
||||||
|
rateLimitRPS := flag.Float64("rate-limit-rps", cfg.RateLimitRPS, "per-client request rate limit; 0 disables")
|
||||||
|
rateLimitBurst := flag.Int("rate-limit-burst", cfg.RateLimitBurst, "per-client rate limit burst")
|
||||||
|
maxResourceLimit := flag.Int("max-resource-limit", cfg.MaxResourcePageLimit, "maximum /v1/resources page size")
|
||||||
|
flag.Parse()
|
||||||
|
|
||||||
|
cfg.Listen = *listen
|
||||||
|
cfg.PublicBaseURL = *publicBase
|
||||||
|
cfg.StateDir = *stateDir
|
||||||
|
cfg.SocketPath = *socket
|
||||||
|
cfg.ResourceRoot = *resourceRoot
|
||||||
|
cfg.ServerInfoFile = *serverInfo
|
||||||
|
cfg.RequireIndexed = *requireIndexed
|
||||||
|
cfg.VerifySize = *verifySize
|
||||||
|
cfg.RPCTimeout = *rpcTimeout
|
||||||
|
cfg.RefreshInterval = *refreshInterval
|
||||||
|
cfg.AuthQueryParam = *authQueryParam
|
||||||
|
cfg.AuthExemptPaths = splitFlagCSV(*authExemptPaths)
|
||||||
|
cfg.TrustProxyHeaders = *trustProxyHeaders
|
||||||
|
cfg.AccessLog = *accessLog
|
||||||
|
cfg.RateLimitRPS = *rateLimitRPS
|
||||||
|
cfg.RateLimitBurst = *rateLimitBurst
|
||||||
|
cfg.MaxResourcePageLimit = *maxResourceLimit
|
||||||
|
// If socket still empty after flags, derive from state-dir.
|
||||||
|
if cfg.SocketPath == "" {
|
||||||
|
cfg.SocketPath = filepath.Join(cfg.StateDir, "bat.sock")
|
||||||
|
}
|
||||||
|
if err := cfg.Normalize(); err != nil {
|
||||||
|
log.Fatalf("config: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var backend api.Backend
|
||||||
|
client := backendrpc.New(cfg.SocketPath)
|
||||||
|
client.Timeout = cfg.RPCTimeout
|
||||||
|
backend = api.RPCClient{Client: client}
|
||||||
|
|
||||||
|
server := api.NewServer(cfg, backend, log.Default())
|
||||||
|
ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
refreshCtx, refreshCancel := context.WithTimeout(ctx, cfg.RPCTimeout+5*time.Second)
|
||||||
|
if err := server.Refresh(refreshCtx); err != nil {
|
||||||
|
log.Printf("initial discover failed: %v (serving with empty/partial index)", err)
|
||||||
|
}
|
||||||
|
refreshCancel()
|
||||||
|
server.StartRefreshLoop(ctx)
|
||||||
|
|
||||||
|
if err := server.ListenAndServe(ctx); err != nil && err != context.Canceled {
|
||||||
|
log.Fatalf("serve: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func envFilePath() string {
|
||||||
|
exe, err := os.Executable()
|
||||||
|
if err != nil {
|
||||||
|
return api.EnvFileName
|
||||||
|
}
|
||||||
|
return filepath.Join(filepath.Dir(exe), api.EnvFileName)
|
||||||
|
}
|
||||||
|
|
||||||
|
func ensureEnvTemplate(path string) error {
|
||||||
|
if _, err := os.Stat(path); err == nil {
|
||||||
|
return nil
|
||||||
|
} else if !os.IsNotExist(err) {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(path, []byte(api.EnvTemplate), 0o600); err != nil {
|
||||||
|
return fmt.Errorf("write %s: %w", path, err)
|
||||||
|
}
|
||||||
|
log.Printf("wrote config template %s", path)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func splitFlagCSV(raw string) []string {
|
||||||
|
var out []string
|
||||||
|
for _, part := range strings.Split(raw, ",") {
|
||||||
|
part = strings.TrimSpace(part)
|
||||||
|
if part != "" {
|
||||||
|
out = append(out, part)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
+9
-4
@@ -34,10 +34,15 @@ func main() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func printUsage() {
|
func printUsage() {
|
||||||
fmt.Println("bat - BlueArchiveToolkit CLI")
|
fmt.Println("bat-go - experimental Go helper (NOT the product CLI)")
|
||||||
|
fmt.Println()
|
||||||
|
fmt.Println("Product sync/ops CLI is the Rust binary `bat` (nearly fully automatic).")
|
||||||
|
fmt.Println("Product resource HTTP service is `bat-api` (see docs/reports/GO_STATUS.md).")
|
||||||
|
fmt.Println()
|
||||||
|
fmt.Println("This binary is experimental FFI demos only. Build output must be bin/bat-go.")
|
||||||
fmt.Println()
|
fmt.Println()
|
||||||
fmt.Println("Usage:")
|
fmt.Println("Usage:")
|
||||||
fmt.Println(" bat doctor")
|
fmt.Println(" bat-go doctor")
|
||||||
fmt.Println(" bat manifest inspect <file>")
|
fmt.Println(" bat-go manifest inspect <file>")
|
||||||
fmt.Println(" bat sync plan <current-json> [previous-json]")
|
fmt.Println(" bat-go sync plan <current-json> [previous-json]")
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -7,7 +7,9 @@ pub mod translation;
|
|||||||
|
|
||||||
pub use game_client::{ClientStatus, GameClient, GameRegion};
|
pub use game_client::{ClientStatus, GameClient, GameRegion};
|
||||||
pub use game_version::{GameVersion, UnityVersion};
|
pub use game_version::{GameVersion, UnityVersion};
|
||||||
pub use resource::{crc32_ieee, IntegrityMismatch, Resource, ResourceEntry, ResourceType};
|
pub use resource::{
|
||||||
|
crc32_ieee, IntegrityMismatch, Resource, ResourceEntry, ResourceMetadata, ResourceType,
|
||||||
|
};
|
||||||
pub use translation::{
|
pub use translation::{
|
||||||
ExtractedText, SourceText, TextContext, TextMetadata, TextSource, TranslatedText,
|
ExtractedText, SourceText, TextContext, TextMetadata, TextSource, TranslatedText,
|
||||||
TranslationStatus,
|
TranslationStatus,
|
||||||
|
|||||||
@@ -43,6 +43,58 @@ pub struct ResourceEntry {
|
|||||||
pub crc: Option<u32>,
|
pub crc: Option<u32>,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// 资源解析与发布侧元数据。
|
||||||
|
///
|
||||||
|
/// 该结构默认全空,保证旧索引和只保存基础 manifest 信息的资源仍可反序列化。
|
||||||
|
/// 官方资源导入会按 release manifest 和 parse cache 填充这些字段,供
|
||||||
|
/// `resource.index` 等只读接口暴露版本、平台、bundle、TextAsset 和 TextUnit 摘要。
|
||||||
|
#[derive(Debug, Clone, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||||
|
pub struct ResourceMetadata {
|
||||||
|
/// 资源所属的官方 release ID。
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub official_release_id: Option<String>,
|
||||||
|
/// 从官方相对路径推断的平台标签,例如 `windows` 或 `android`。
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub platform: Option<String>,
|
||||||
|
/// 资源本身或所在 bundle 的官方相对路径。
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub bundle_path: Option<String>,
|
||||||
|
/// ZIP 内被解析到的 bundle entry;直接 bundle 为空。
|
||||||
|
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||||
|
pub archive_entries: Vec<String>,
|
||||||
|
/// parse cache 中出现过的解析状态标签。
|
||||||
|
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||||
|
pub parse_statuses: Vec<String>,
|
||||||
|
/// 解析到的 Unity 版本集合。
|
||||||
|
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||||
|
pub unity_versions: Vec<String>,
|
||||||
|
/// UnityFS directory file 总数。
|
||||||
|
#[serde(default, skip_serializing_if = "is_zero")]
|
||||||
|
pub unityfs_file_count: u64,
|
||||||
|
/// Unity serialized file 总数。
|
||||||
|
#[serde(default, skip_serializing_if = "is_zero")]
|
||||||
|
pub serialized_file_count: u64,
|
||||||
|
/// TextAsset 对象总数。
|
||||||
|
#[serde(default, skip_serializing_if = "is_zero")]
|
||||||
|
pub text_asset_count: u64,
|
||||||
|
/// TextAsset 名称集合。
|
||||||
|
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||||
|
pub text_assets: Vec<String>,
|
||||||
|
/// TextUnit 总数。
|
||||||
|
#[serde(default, skip_serializing_if = "is_zero")]
|
||||||
|
pub text_unit_count: u64,
|
||||||
|
/// TextUnit 格式标签集合,例如 `json`、`csv`、`tsv`、`plain`。
|
||||||
|
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||||
|
pub text_unit_formats: Vec<String>,
|
||||||
|
/// TextUnit 提取阶段的非致命诊断数量。
|
||||||
|
#[serde(default, skip_serializing_if = "is_zero")]
|
||||||
|
pub text_unit_error_count: u64,
|
||||||
|
}
|
||||||
|
|
||||||
|
fn is_zero(value: &u64) -> bool {
|
||||||
|
*value == 0
|
||||||
|
}
|
||||||
|
|
||||||
/// 已下载字节与 catalog 声明的可校验字段不一致。
|
/// 已下载字节与 catalog 声明的可校验字段不一致。
|
||||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
pub enum IntegrityMismatch {
|
pub enum IntegrityMismatch {
|
||||||
@@ -133,6 +185,9 @@ pub struct Resource {
|
|||||||
pub local_path: PathBuf,
|
pub local_path: PathBuf,
|
||||||
/// 资源条目
|
/// 资源条目
|
||||||
pub entry: ResourceEntry,
|
pub entry: ResourceEntry,
|
||||||
|
/// 解析、发布和索引侧扩展元数据。
|
||||||
|
#[serde(default)]
|
||||||
|
pub metadata: ResourceMetadata,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
|
|||||||
@@ -11,9 +11,9 @@ thiserror.workspace = true
|
|||||||
serde.workspace = true
|
serde.workspace = true
|
||||||
serde_json.workspace = true
|
serde_json.workspace = true
|
||||||
tracing.workspace = true
|
tracing.workspace = true
|
||||||
|
lz4 = "1.28"
|
||||||
# 注意:byteorder、lz4、lzma-rs 等 UnityFS 解析/解压依赖待解析器真正实现时
|
lzma-rs = "0.3"
|
||||||
# 再按需引入,避免占位阶段白增编译负担。
|
md-5 = "0.10"
|
||||||
|
|
||||||
[dev-dependencies]
|
[dev-dependencies]
|
||||||
hex = "0.4"
|
hex = "0.4"
|
||||||
|
|||||||
@@ -1,26 +1,52 @@
|
|||||||
//! AssetBundle 错误类型定义
|
//! AssetBundle error types.
|
||||||
|
|
||||||
use thiserror::Error;
|
use thiserror::Error;
|
||||||
|
|
||||||
/// AssetBundle 错误类型
|
/// AssetBundle parser error.
|
||||||
#[derive(Error, Debug)]
|
#[derive(Error, Debug)]
|
||||||
pub enum AssetBundleError {
|
pub enum AssetBundleError {
|
||||||
/// IO 错误
|
/// I/O error.
|
||||||
#[error("IO error: {0}")]
|
#[error("IO error: {0}")]
|
||||||
Io(#[from] std::io::Error),
|
Io(#[from] std::io::Error),
|
||||||
|
|
||||||
/// 解析错误
|
/// Parser reached malformed data while reading a named field.
|
||||||
|
#[error("Parse error at offset {offset} while reading {field}: {message}")]
|
||||||
|
ParseField {
|
||||||
|
/// Field or structure name being read.
|
||||||
|
field: String,
|
||||||
|
/// Byte offset where parsing failed.
|
||||||
|
offset: usize,
|
||||||
|
/// Human-readable diagnostic.
|
||||||
|
message: String,
|
||||||
|
},
|
||||||
|
|
||||||
|
/// General parser error.
|
||||||
#[error("Parse error: {0}")]
|
#[error("Parse error: {0}")]
|
||||||
Parse(String),
|
Parse(String),
|
||||||
|
|
||||||
/// 不支持的格式
|
/// Unsupported format or compression mode.
|
||||||
#[error("Unsupported format: {0}")]
|
#[error("Unsupported format: {0}")]
|
||||||
UnsupportedFormat(String),
|
UnsupportedFormat(String),
|
||||||
|
|
||||||
/// 其他错误
|
/// Other error.
|
||||||
#[error(transparent)]
|
#[error(transparent)]
|
||||||
Other(#[from] anyhow::Error),
|
Other(#[from] anyhow::Error),
|
||||||
}
|
}
|
||||||
|
|
||||||
/// AssetBundle Result 类型
|
impl AssetBundleError {
|
||||||
|
/// Creates a field-scoped parser error with byte offset context.
|
||||||
|
pub fn parse_field(
|
||||||
|
field: impl Into<String>,
|
||||||
|
offset: usize,
|
||||||
|
message: impl Into<String>,
|
||||||
|
) -> Self {
|
||||||
|
Self::ParseField {
|
||||||
|
field: field.into(),
|
||||||
|
offset,
|
||||||
|
message: message.into(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// AssetBundle result type.
|
||||||
pub type Result<T> = std::result::Result<T, AssetBundleError>;
|
pub type Result<T> = std::result::Result<T, AssetBundleError>;
|
||||||
|
|||||||
@@ -9,9 +9,32 @@
|
|||||||
|
|
||||||
pub mod error;
|
pub mod error;
|
||||||
pub mod parser;
|
pub mod parser;
|
||||||
|
pub mod patch;
|
||||||
|
pub mod serialized;
|
||||||
|
pub mod text;
|
||||||
pub mod types;
|
pub mod types;
|
||||||
|
|
||||||
pub use error::{AssetBundleError, Result};
|
pub use error::{AssetBundleError, Result};
|
||||||
|
pub use parser::{compression_from_flags, Parser, UnityFsParser};
|
||||||
|
pub use patch::{
|
||||||
|
patch_unityfs_field, patch_unityfs_string_field, patch_unityfs_text_asset, FieldPatch,
|
||||||
|
StringFieldPatch, TextAssetPatch,
|
||||||
|
};
|
||||||
|
pub use serialized::{
|
||||||
|
UnityManagedReferenceMetadata, UnityManagedReferenceRecord, UnitySerializedField,
|
||||||
|
UnitySerializedFieldReplacement, UnitySerializedFile, UnitySerializedObject,
|
||||||
|
UnitySerializedReplacementValue, UnitySerializedTextAsset, UnitySerializedType,
|
||||||
|
UnitySerializedValue, UnityTypeTreeNode,
|
||||||
|
};
|
||||||
|
pub use text::{
|
||||||
|
text_units_to_jsonl, TextUnit, TextUnitExtractionError, TextUnitExtractionReport,
|
||||||
|
TextUnitExtractor,
|
||||||
|
};
|
||||||
|
pub use types::{
|
||||||
|
AssetType, ParsedAssetBundle, RawAssetBundle, UnityFsBlockInfo, UnityFsBundle,
|
||||||
|
UnityFsCompression, UnityFsDirectoryInfo, UnityFsFile, UnityFsHeader,
|
||||||
|
UnitySerializedParseError,
|
||||||
|
};
|
||||||
|
|
||||||
/// AssetBundle 解析器版本号
|
/// AssetBundle 解析器版本号
|
||||||
pub const VERSION: &str = env!("CARGO_PKG_VERSION");
|
pub const VERSION: &str = env!("CARGO_PKG_VERSION");
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,949 @@
|
|||||||
|
//! Text extraction from parsed Unity serialized objects.
|
||||||
|
|
||||||
|
use crate::serialized::{
|
||||||
|
managed_reference_metadata_from_fields, UnityManagedReferenceMetadata,
|
||||||
|
UnityManagedReferenceRecord, UnitySerializedField, UnitySerializedValue,
|
||||||
|
};
|
||||||
|
use crate::types::ParsedAssetBundle;
|
||||||
|
use std::collections::BTreeMap;
|
||||||
|
|
||||||
|
/// One text unit used by translation, glossary and patch pipelines.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||||
|
pub struct TextUnit {
|
||||||
|
/// Original source text.
|
||||||
|
pub source_text: String,
|
||||||
|
/// Logical AssetBundle path, when provided by the caller.
|
||||||
|
pub bundle_path: Option<String>,
|
||||||
|
/// ZIP/archive entry containing the bundle, when known.
|
||||||
|
pub archive_entry: Option<String>,
|
||||||
|
/// Unity serialized file path.
|
||||||
|
pub serialized_file: Option<String>,
|
||||||
|
/// Unity object path ID.
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub path_id: Option<i64>,
|
||||||
|
/// Unity class ID, for example `49` for `TextAsset`.
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub class_id: Option<i32>,
|
||||||
|
/// TypeTree field path. `TextAsset` is used for a whole TextAsset payload.
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub field_path: Option<String>,
|
||||||
|
/// Byte offset relative to the beginning of the Unity object payload.
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub field_offset: Option<usize>,
|
||||||
|
/// Number of bytes consumed by this field, including alignment padding.
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub field_byte_size: Option<usize>,
|
||||||
|
/// Unity version associated with the source.
|
||||||
|
pub version: String,
|
||||||
|
/// Stable context for format, asset name and extraction details.
|
||||||
|
pub context: BTreeMap<String, String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Non-fatal diagnostic generated while extracting text units.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||||
|
pub struct TextUnitExtractionError {
|
||||||
|
/// Serialized file containing the failed object.
|
||||||
|
pub serialized_file: Option<String>,
|
||||||
|
/// Object path ID, when known.
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub path_id: Option<i64>,
|
||||||
|
/// Unity class ID, when known.
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub class_id: Option<i32>,
|
||||||
|
/// TypeTree field path, when known.
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub field_path: Option<String>,
|
||||||
|
/// Byte offset relative to the beginning of the Unity object payload.
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub offset: Option<usize>,
|
||||||
|
/// Human-readable error.
|
||||||
|
pub error: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Result of extracting text units from one parsed bundle.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||||
|
pub struct TextUnitExtractionReport {
|
||||||
|
/// Extracted text units in deterministic traversal order.
|
||||||
|
pub units: Vec<TextUnit>,
|
||||||
|
/// Non-fatal object-level errors.
|
||||||
|
pub errors: Vec<TextUnitExtractionError>,
|
||||||
|
/// TextAsset payloads that were binary or invalid UTF-8.
|
||||||
|
pub skipped_binary_text_assets: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Extracts translation-ready text from Unity bundle data.
|
||||||
|
#[derive(Debug, Default, Clone, Copy)]
|
||||||
|
pub struct TextUnitExtractor;
|
||||||
|
|
||||||
|
impl TextUnitExtractor {
|
||||||
|
/// Creates an extractor.
|
||||||
|
pub fn new() -> Self {
|
||||||
|
Self
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Extracts TextAsset and TypeTree string fields from a parsed bundle.
|
||||||
|
pub fn extract_bundle(
|
||||||
|
&self,
|
||||||
|
bundle: &ParsedAssetBundle,
|
||||||
|
bundle_path: Option<&str>,
|
||||||
|
) -> TextUnitExtractionReport {
|
||||||
|
self.extract_bundle_with_context(bundle, bundle_path, None)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Extracts text with both logical bundle and archive-entry context.
|
||||||
|
pub fn extract_bundle_with_context(
|
||||||
|
&self,
|
||||||
|
bundle: &ParsedAssetBundle,
|
||||||
|
bundle_path: Option<&str>,
|
||||||
|
archive_entry: Option<&str>,
|
||||||
|
) -> TextUnitExtractionReport {
|
||||||
|
let mut report = TextUnitExtractionReport {
|
||||||
|
units: Vec::new(),
|
||||||
|
errors: Vec::new(),
|
||||||
|
skipped_binary_text_assets: 0,
|
||||||
|
};
|
||||||
|
|
||||||
|
for asset in &bundle.text_assets {
|
||||||
|
if let Some((format, text)) = decode_text_payload(&asset.bytes) {
|
||||||
|
let mut context = BTreeMap::new();
|
||||||
|
context.insert("format".to_string(), format.to_string());
|
||||||
|
context.insert("asset_name".to_string(), asset.name.clone());
|
||||||
|
context.insert("source_kind".to_string(), "TextAsset".to_string());
|
||||||
|
report.units.push(TextUnit {
|
||||||
|
source_text: text,
|
||||||
|
bundle_path: bundle_path.map(ToOwned::to_owned),
|
||||||
|
archive_entry: archive_entry.map(ToOwned::to_owned),
|
||||||
|
serialized_file: asset.source_path.clone(),
|
||||||
|
path_id: Some(asset.path_id),
|
||||||
|
class_id: Some(49),
|
||||||
|
field_path: Some("TextAsset".to_string()),
|
||||||
|
field_offset: None,
|
||||||
|
field_byte_size: None,
|
||||||
|
version: bundle.unity_version.clone(),
|
||||||
|
context,
|
||||||
|
});
|
||||||
|
} else {
|
||||||
|
report.skipped_binary_text_assets += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for serialized_file in &bundle.serialized_files {
|
||||||
|
for object in &serialized_file.objects {
|
||||||
|
if object.class_id == 49 || !serialized_file.object_has_type_tree(object) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let fields = match serialized_file.fields_for_object_entry(object) {
|
||||||
|
Ok(fields) => fields,
|
||||||
|
Err(error) => {
|
||||||
|
report.errors.push(TextUnitExtractionError {
|
||||||
|
serialized_file: serialized_file.source_path.clone(),
|
||||||
|
path_id: Some(object.path_id),
|
||||||
|
class_id: Some(object.class_id),
|
||||||
|
field_path: None,
|
||||||
|
offset: None,
|
||||||
|
error: error.to_string(),
|
||||||
|
});
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let context = FieldTextContext {
|
||||||
|
serialized_file_path: serialized_file.source_path.as_deref(),
|
||||||
|
path_id: object.path_id,
|
||||||
|
class_id: object.class_id,
|
||||||
|
version: &bundle.unity_version,
|
||||||
|
bundle_path,
|
||||||
|
archive_entry,
|
||||||
|
managed_reference: None,
|
||||||
|
};
|
||||||
|
for field in fields {
|
||||||
|
collect_field_text(&mut report.units, &context, &field);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
report
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Serializes text units as one stable JSON object per line.
|
||||||
|
pub fn text_units_to_jsonl(units: &[TextUnit]) -> Result<String, serde_json::Error> {
|
||||||
|
let mut output = String::new();
|
||||||
|
for unit in units {
|
||||||
|
output.push_str(&serde_json::to_string(unit)?);
|
||||||
|
output.push('\n');
|
||||||
|
}
|
||||||
|
Ok(output)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Clone)]
|
||||||
|
struct FieldTextContext<'a> {
|
||||||
|
serialized_file_path: Option<&'a str>,
|
||||||
|
path_id: i64,
|
||||||
|
class_id: i32,
|
||||||
|
version: &'a str,
|
||||||
|
bundle_path: Option<&'a str>,
|
||||||
|
archive_entry: Option<&'a str>,
|
||||||
|
managed_reference: Option<UnityManagedReferenceMetadata>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<'a> FieldTextContext<'a> {
|
||||||
|
fn with_managed_reference(&self, metadata: Option<&UnityManagedReferenceMetadata>) -> Self {
|
||||||
|
Self {
|
||||||
|
serialized_file_path: self.serialized_file_path,
|
||||||
|
path_id: self.path_id,
|
||||||
|
class_id: self.class_id,
|
||||||
|
version: self.version,
|
||||||
|
bundle_path: self.bundle_path,
|
||||||
|
archive_entry: self.archive_entry,
|
||||||
|
managed_reference: metadata.cloned().or_else(|| self.managed_reference.clone()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn collect_field_text<'a>(
|
||||||
|
units: &mut Vec<TextUnit>,
|
||||||
|
context: &FieldTextContext<'a>,
|
||||||
|
field: &'a UnitySerializedField,
|
||||||
|
) {
|
||||||
|
match &field.value {
|
||||||
|
UnitySerializedValue::String(text) if !text.is_empty() => {
|
||||||
|
let mut unit_context = BTreeMap::new();
|
||||||
|
unit_context.insert("format".to_string(), "plain".to_string());
|
||||||
|
unit_context.insert(
|
||||||
|
"source_kind".to_string(),
|
||||||
|
if context.managed_reference.is_some() {
|
||||||
|
"ManagedReferenceField"
|
||||||
|
} else {
|
||||||
|
"TypeTreeField"
|
||||||
|
}
|
||||||
|
.to_string(),
|
||||||
|
);
|
||||||
|
unit_context.insert("type_name".to_string(), field.type_name.clone());
|
||||||
|
if let Some(metadata) = &context.managed_reference {
|
||||||
|
insert_managed_reference_context(&mut unit_context, metadata);
|
||||||
|
}
|
||||||
|
units.push(TextUnit {
|
||||||
|
source_text: text.clone(),
|
||||||
|
bundle_path: context.bundle_path.map(ToOwned::to_owned),
|
||||||
|
archive_entry: context.archive_entry.map(ToOwned::to_owned),
|
||||||
|
serialized_file: context.serialized_file_path.map(ToOwned::to_owned),
|
||||||
|
path_id: Some(context.path_id),
|
||||||
|
class_id: Some(context.class_id),
|
||||||
|
field_path: Some(field.path.clone()),
|
||||||
|
field_offset: Some(field.offset),
|
||||||
|
field_byte_size: Some(field.byte_size),
|
||||||
|
version: context.version.to_string(),
|
||||||
|
context: unit_context,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
UnitySerializedValue::Object(fields) => {
|
||||||
|
for field in fields {
|
||||||
|
collect_field_text(units, context, field);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
UnitySerializedValue::ManagedReference {
|
||||||
|
metadata, fields, ..
|
||||||
|
} => {
|
||||||
|
let managed_context = context.with_managed_reference(metadata.as_ref());
|
||||||
|
for field in fields {
|
||||||
|
collect_managed_reference_child_text(units, &managed_context, field);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
UnitySerializedValue::ManagedReferenceRegistry { references, fields } => {
|
||||||
|
if references.is_empty() {
|
||||||
|
let fallback_metadata = managed_reference_metadata_from_fields(fields);
|
||||||
|
for field in fields {
|
||||||
|
let sibling_metadata =
|
||||||
|
managed_reference_metadata_from_sibling_fields(fields, field);
|
||||||
|
let managed_context = context.with_managed_reference(
|
||||||
|
sibling_metadata.as_ref().or(fallback_metadata.as_ref()),
|
||||||
|
);
|
||||||
|
collect_managed_reference_child_text(units, &managed_context, field);
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
for reference in references {
|
||||||
|
collect_managed_reference_record_text(units, context, reference);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
UnitySerializedValue::Array(values) | UnitySerializedValue::Map(values) => {
|
||||||
|
for field in values {
|
||||||
|
collect_field_text(units, context, field);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
_ => {}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn collect_managed_reference_record_text<'a>(
|
||||||
|
units: &mut Vec<TextUnit>,
|
||||||
|
context: &FieldTextContext<'a>,
|
||||||
|
reference: &'a UnityManagedReferenceRecord,
|
||||||
|
) {
|
||||||
|
let managed_context = context.with_managed_reference(Some(&reference.metadata));
|
||||||
|
for field in &reference.fields {
|
||||||
|
collect_field_text(units, &managed_context, field);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn collect_managed_reference_child_text<'a>(
|
||||||
|
units: &mut Vec<TextUnit>,
|
||||||
|
context: &FieldTextContext<'a>,
|
||||||
|
field: &'a UnitySerializedField,
|
||||||
|
) {
|
||||||
|
let key = normalized_text_metadata_key(&field.name);
|
||||||
|
if is_managed_reference_metadata_text_key(&key) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if is_managed_reference_payload_text_key(&key) {
|
||||||
|
if let Some(children) = serialized_field_children(field) {
|
||||||
|
for child in children {
|
||||||
|
collect_field_text(units, context, child);
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
collect_field_text(units, context, field);
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if let Some(children) = serialized_field_children(field) {
|
||||||
|
if let Some(metadata) = managed_reference_metadata_from_fields(children) {
|
||||||
|
let managed_context = context.with_managed_reference(Some(&metadata));
|
||||||
|
for child in children {
|
||||||
|
collect_managed_reference_child_text(units, &managed_context, child);
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
collect_field_text(units, context, field);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn managed_reference_metadata_from_sibling_fields(
|
||||||
|
fields: &[UnitySerializedField],
|
||||||
|
field: &UnitySerializedField,
|
||||||
|
) -> Option<UnityManagedReferenceMetadata> {
|
||||||
|
let record_prefix = managed_reference_record_path_prefix(&field.path)?;
|
||||||
|
let grouped_fields = fields
|
||||||
|
.iter()
|
||||||
|
.filter(|candidate| {
|
||||||
|
candidate.path == record_prefix
|
||||||
|
|| candidate
|
||||||
|
.path
|
||||||
|
.strip_prefix(record_prefix)
|
||||||
|
.is_some_and(|suffix| suffix.starts_with('.'))
|
||||||
|
})
|
||||||
|
.cloned()
|
||||||
|
.collect::<Vec<_>>();
|
||||||
|
managed_reference_metadata_from_fields(&grouped_fields)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn managed_reference_record_path_prefix(path: &str) -> Option<&str> {
|
||||||
|
if let Some(index_end) = path.rfind(']') {
|
||||||
|
return Some(&path[..=index_end]);
|
||||||
|
}
|
||||||
|
path.rsplit_once('.')
|
||||||
|
.map(|(parent, _)| parent)
|
||||||
|
.filter(|parent| !parent.is_empty())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn insert_managed_reference_context(
|
||||||
|
context: &mut BTreeMap<String, String>,
|
||||||
|
metadata: &UnityManagedReferenceMetadata,
|
||||||
|
) {
|
||||||
|
if let Some(reference_id) = metadata.reference_id {
|
||||||
|
context.insert("managed_reference_id".to_string(), reference_id.to_string());
|
||||||
|
}
|
||||||
|
if let Some(value) = &metadata.full_type_name {
|
||||||
|
context.insert(
|
||||||
|
"managed_reference_full_type_name".to_string(),
|
||||||
|
value.clone(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if let Some(value) = &metadata.type_name {
|
||||||
|
context.insert("managed_reference_type".to_string(), value.clone());
|
||||||
|
}
|
||||||
|
if let Some(value) = &metadata.namespace {
|
||||||
|
context.insert("managed_reference_namespace".to_string(), value.clone());
|
||||||
|
}
|
||||||
|
if let Some(value) = &metadata.assembly_name {
|
||||||
|
context.insert("managed_reference_assembly".to_string(), value.clone());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn serialized_field_children(field: &UnitySerializedField) -> Option<&[UnitySerializedField]> {
|
||||||
|
match &field.value {
|
||||||
|
UnitySerializedValue::Object(fields)
|
||||||
|
| UnitySerializedValue::Array(fields)
|
||||||
|
| UnitySerializedValue::Map(fields)
|
||||||
|
| UnitySerializedValue::ManagedReference { fields, .. }
|
||||||
|
| UnitySerializedValue::ManagedReferenceRegistry { fields, .. } => Some(fields),
|
||||||
|
_ => None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn normalized_text_metadata_key(name: &str) -> String {
|
||||||
|
name.strip_prefix("m_")
|
||||||
|
.unwrap_or(name)
|
||||||
|
.chars()
|
||||||
|
.filter(|character| character.is_ascii_alphanumeric())
|
||||||
|
.flat_map(char::to_lowercase)
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn is_managed_reference_metadata_text_key(key: &str) -> bool {
|
||||||
|
matches!(
|
||||||
|
key,
|
||||||
|
"rid"
|
||||||
|
| "id"
|
||||||
|
| "identifier"
|
||||||
|
| "refid"
|
||||||
|
| "referenceid"
|
||||||
|
| "managedreferenceid"
|
||||||
|
| "managedreferenceids"
|
||||||
|
| "managedreferencesid"
|
||||||
|
| "managedreferencesids"
|
||||||
|
| "serializedreferenceid"
|
||||||
|
| "serializedreferenceids"
|
||||||
|
| "refids"
|
||||||
|
| "type"
|
||||||
|
| "typeid"
|
||||||
|
| "typeinfo"
|
||||||
|
| "typename"
|
||||||
|
| "fullname"
|
||||||
|
| "fulltypename"
|
||||||
|
| "class"
|
||||||
|
| "classname"
|
||||||
|
| "managedreferenceclassname"
|
||||||
|
| "serializedreferenceclassname"
|
||||||
|
| "klass"
|
||||||
|
| "managedtype"
|
||||||
|
| "managedreferencetype"
|
||||||
|
| "managedreferencefullname"
|
||||||
|
| "managedreferencefulltypename"
|
||||||
|
| "serializedreferencetype"
|
||||||
|
| "serializedreferencefullname"
|
||||||
|
| "serializedreferencefulltypename"
|
||||||
|
| "assemblyqualifiedname"
|
||||||
|
| "ns"
|
||||||
|
| "namespace"
|
||||||
|
| "namespacename"
|
||||||
|
| "managedreferencenamespace"
|
||||||
|
| "managedreferencenamespacename"
|
||||||
|
| "serializedreferencenamespace"
|
||||||
|
| "serializedreferencenamespacename"
|
||||||
|
| "asm"
|
||||||
|
| "asmname"
|
||||||
|
| "assembly"
|
||||||
|
| "assemblyname"
|
||||||
|
| "managedreferenceassembly"
|
||||||
|
| "managedreferenceassemblyname"
|
||||||
|
| "serializedreferenceassembly"
|
||||||
|
| "serializedreferenceassemblyname"
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn is_managed_reference_payload_text_key(key: &str) -> bool {
|
||||||
|
matches!(
|
||||||
|
key,
|
||||||
|
"data"
|
||||||
|
| "payload"
|
||||||
|
| "value"
|
||||||
|
| "object"
|
||||||
|
| "instance"
|
||||||
|
| "managedreferencepayload"
|
||||||
|
| "referencepayload"
|
||||||
|
| "serializedreferencepayload"
|
||||||
|
| "managedreferencevalue"
|
||||||
|
| "referencevalue"
|
||||||
|
| "serializedreferencevalue"
|
||||||
|
| "managedreferenceobject"
|
||||||
|
| "referenceobject"
|
||||||
|
| "serializedreferenceobject"
|
||||||
|
| "managedreferencedata"
|
||||||
|
| "referencedata"
|
||||||
|
| "serializeddata"
|
||||||
|
| "serializedreferencedata"
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn decode_text_payload(bytes: &[u8]) -> Option<(&'static str, String)> {
|
||||||
|
let bytes = bytes.strip_prefix(&[0xef, 0xbb, 0xbf]).unwrap_or(bytes);
|
||||||
|
if bytes.contains(&0) {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
let text = std::str::from_utf8(bytes).ok()?.to_string();
|
||||||
|
if text.trim().is_empty()
|
||||||
|
|| text
|
||||||
|
.chars()
|
||||||
|
.any(|character| character.is_control() && !matches!(character, '\n' | '\r' | '\t'))
|
||||||
|
{
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
|
||||||
|
let trimmed = text.trim();
|
||||||
|
if serde_json::from_str::<serde_json::Value>(trimmed).is_ok() {
|
||||||
|
return Some(("json", text));
|
||||||
|
}
|
||||||
|
if text.lines().any(|line| line.contains('\t')) {
|
||||||
|
return Some(("tsv", text));
|
||||||
|
}
|
||||||
|
if text.lines().count() > 1 && text.lines().any(|line| line.contains(',')) {
|
||||||
|
return Some(("csv", text));
|
||||||
|
}
|
||||||
|
Some(("plain", text))
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use crate::serialized::UnitySerializedTextAsset;
|
||||||
|
use crate::types::{
|
||||||
|
ParsedAssetBundle, UnityFsBlockInfo, UnityFsDirectoryInfo, UnityFsHeader,
|
||||||
|
UnitySerializedParseError,
|
||||||
|
};
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn extracts_textasset_and_writes_jsonl() {
|
||||||
|
let bundle = ParsedAssetBundle {
|
||||||
|
unity_version: "2021.3.56f2".to_string(),
|
||||||
|
assets: vec!["CAB-test".to_string()],
|
||||||
|
raw_data: Vec::new(),
|
||||||
|
unityfs_header: Some(UnityFsHeader {
|
||||||
|
format_version: 8,
|
||||||
|
target_version: "5.x.x".to_string(),
|
||||||
|
unity_version: "2021.3.56f2".to_string(),
|
||||||
|
total_size: 0,
|
||||||
|
compressed_blocks_info_size: 0,
|
||||||
|
uncompressed_blocks_info_size: 0,
|
||||||
|
flags: 0,
|
||||||
|
}),
|
||||||
|
blocks: vec![UnityFsBlockInfo {
|
||||||
|
uncompressed_size: 0,
|
||||||
|
compressed_size: 0,
|
||||||
|
flags: 0,
|
||||||
|
compression: crate::types::UnityFsCompression::None,
|
||||||
|
}],
|
||||||
|
directories: vec![UnityFsDirectoryInfo {
|
||||||
|
offset: 0,
|
||||||
|
size: 0,
|
||||||
|
flags: 0,
|
||||||
|
path: "CAB-test".to_string(),
|
||||||
|
}],
|
||||||
|
files: Vec::new(),
|
||||||
|
serialized_files: Vec::new(),
|
||||||
|
text_assets: vec![UnitySerializedTextAsset {
|
||||||
|
source_path: Some("CAB-test".to_string()),
|
||||||
|
path_id: 1,
|
||||||
|
name: "dialogue.json".to_string(),
|
||||||
|
bytes: br#"{"text":"hello"}"#.to_vec(),
|
||||||
|
}],
|
||||||
|
serialized_parse_errors: Vec::<UnitySerializedParseError>::new(),
|
||||||
|
};
|
||||||
|
|
||||||
|
let report = TextUnitExtractor::new().extract_bundle_with_context(
|
||||||
|
&bundle,
|
||||||
|
Some("dialogue.bundle"),
|
||||||
|
Some("assets/dialogue.bundle"),
|
||||||
|
);
|
||||||
|
|
||||||
|
assert_eq!(report.units.len(), 1);
|
||||||
|
assert_eq!(
|
||||||
|
report.units[0].archive_entry.as_deref(),
|
||||||
|
Some("assets/dialogue.bundle")
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
report.units[0].context.get("format"),
|
||||||
|
Some(&"json".to_string())
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
text_units_to_jsonl(&report.units).unwrap(),
|
||||||
|
format!("{}\n", serde_json::to_string(&report.units[0]).unwrap())
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn extracts_managed_reference_payload_without_metadata_strings() {
|
||||||
|
let payload_field = string_field(
|
||||||
|
"m_ManagedReferences.references[0].data.message",
|
||||||
|
"message",
|
||||||
|
"こんにちは",
|
||||||
|
);
|
||||||
|
let metadata_field = string_field(
|
||||||
|
"m_ManagedReferences.references[0].managedReferenceFullTypeName",
|
||||||
|
"managedReferenceFullTypeName",
|
||||||
|
"Game BA.Text.ScenarioLine",
|
||||||
|
);
|
||||||
|
let registry_field = UnitySerializedField {
|
||||||
|
path: "m_ManagedReferences".to_string(),
|
||||||
|
name: "m_ManagedReferences".to_string(),
|
||||||
|
type_name: "ManagedReferenceRegistry".to_string(),
|
||||||
|
offset: 0,
|
||||||
|
byte_size: 64,
|
||||||
|
type_tree_node_index: None,
|
||||||
|
value: UnitySerializedValue::ManagedReferenceRegistry {
|
||||||
|
references: vec![UnityManagedReferenceRecord {
|
||||||
|
metadata: UnityManagedReferenceMetadata {
|
||||||
|
reference_id: Some(42),
|
||||||
|
full_type_name: Some("Game BA.Text.ScenarioLine".to_string()),
|
||||||
|
type_name: Some("ScenarioLine".to_string()),
|
||||||
|
namespace: Some("BA.Text".to_string()),
|
||||||
|
assembly_name: Some("Game".to_string()),
|
||||||
|
},
|
||||||
|
fields: vec![payload_field.clone()],
|
||||||
|
}],
|
||||||
|
fields: vec![metadata_field],
|
||||||
|
},
|
||||||
|
};
|
||||||
|
let context = FieldTextContext {
|
||||||
|
serialized_file_path: Some("CAB-test"),
|
||||||
|
path_id: 7,
|
||||||
|
class_id: 114,
|
||||||
|
version: "2021.3.56f2",
|
||||||
|
bundle_path: Some("scenario.bundle"),
|
||||||
|
archive_entry: None,
|
||||||
|
managed_reference: None,
|
||||||
|
};
|
||||||
|
let mut units = Vec::new();
|
||||||
|
|
||||||
|
collect_field_text(&mut units, &context, ®istry_field);
|
||||||
|
|
||||||
|
assert_eq!(units.len(), 1);
|
||||||
|
assert_eq!(units[0].source_text, "こんにちは");
|
||||||
|
assert_eq!(
|
||||||
|
units[0].field_path.as_deref(),
|
||||||
|
Some("m_ManagedReferences.references[0].data.message")
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
units[0].context.get("source_kind"),
|
||||||
|
Some(&"ManagedReferenceField".to_string())
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
units[0].context.get("managed_reference_id"),
|
||||||
|
Some(&"42".to_string())
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
units[0].context.get("managed_reference_full_type_name"),
|
||||||
|
Some(&"Game BA.Text.ScenarioLine".to_string())
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
units[0].context.get("managed_reference_type"),
|
||||||
|
Some(&"ScenarioLine".to_string())
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
units[0].context.get("managed_reference_namespace"),
|
||||||
|
Some(&"BA.Text".to_string())
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
units[0].context.get("managed_reference_assembly"),
|
||||||
|
Some(&"Game".to_string())
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn extracts_fallback_managed_reference_payload_alias_without_metadata_strings() {
|
||||||
|
let payload_field = string_field(
|
||||||
|
"m_ManagedReferences.RefIds[0].managedReferenceData.message",
|
||||||
|
"message",
|
||||||
|
"こんにちは",
|
||||||
|
);
|
||||||
|
let registry_field = UnitySerializedField {
|
||||||
|
path: "m_ManagedReferences".to_string(),
|
||||||
|
name: "m_ManagedReferences".to_string(),
|
||||||
|
type_name: "ManagedReferencesRegistry".to_string(),
|
||||||
|
offset: 0,
|
||||||
|
byte_size: 96,
|
||||||
|
type_tree_node_index: None,
|
||||||
|
value: UnitySerializedValue::ManagedReferenceRegistry {
|
||||||
|
references: Vec::new(),
|
||||||
|
fields: vec![
|
||||||
|
string_field(
|
||||||
|
"m_ManagedReferences.RefIds[0].managedReferenceClassName",
|
||||||
|
"managedReferenceClassName",
|
||||||
|
"ScenarioLine",
|
||||||
|
),
|
||||||
|
string_field(
|
||||||
|
"m_ManagedReferences.RefIds[0].managedReferenceNamespaceName",
|
||||||
|
"managedReferenceNamespaceName",
|
||||||
|
"BA.Text",
|
||||||
|
),
|
||||||
|
string_field(
|
||||||
|
"m_ManagedReferences.RefIds[0].managedReferenceAssemblyName",
|
||||||
|
"managedReferenceAssemblyName",
|
||||||
|
"Game",
|
||||||
|
),
|
||||||
|
string_field(
|
||||||
|
"m_ManagedReferences.RefIds[0].serializedReferenceFullTypeName",
|
||||||
|
"serializedReferenceFullTypeName",
|
||||||
|
"Game BA.Text.ScenarioLine",
|
||||||
|
),
|
||||||
|
string_field(
|
||||||
|
"m_ManagedReferences.RefIds[0].typeInfo",
|
||||||
|
"typeInfo",
|
||||||
|
"Game BA.Text.ScenarioLine",
|
||||||
|
),
|
||||||
|
string_field(
|
||||||
|
"m_ManagedReferences.RefIds[0].typeID.className",
|
||||||
|
"className",
|
||||||
|
"ScenarioLine",
|
||||||
|
),
|
||||||
|
string_field(
|
||||||
|
"m_ManagedReferences.RefIds[0].typeID.namespaceName",
|
||||||
|
"namespaceName",
|
||||||
|
"BA.Text",
|
||||||
|
),
|
||||||
|
string_field(
|
||||||
|
"m_ManagedReferences.RefIds[0].typeID.asmName",
|
||||||
|
"asmName",
|
||||||
|
"Game",
|
||||||
|
),
|
||||||
|
UnitySerializedField {
|
||||||
|
path: "m_ManagedReferences.RefIds[0].managedReferenceData".to_string(),
|
||||||
|
name: "managedReferenceData".to_string(),
|
||||||
|
type_name: "managedReference".to_string(),
|
||||||
|
offset: 64,
|
||||||
|
byte_size: 24,
|
||||||
|
type_tree_node_index: None,
|
||||||
|
value: UnitySerializedValue::Object(vec![payload_field]),
|
||||||
|
},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
};
|
||||||
|
let context = FieldTextContext {
|
||||||
|
serialized_file_path: Some("CAB-test"),
|
||||||
|
path_id: 7,
|
||||||
|
class_id: 114,
|
||||||
|
version: "2021.3.56f2",
|
||||||
|
bundle_path: Some("scenario.bundle"),
|
||||||
|
archive_entry: None,
|
||||||
|
managed_reference: None,
|
||||||
|
};
|
||||||
|
let mut units = Vec::new();
|
||||||
|
|
||||||
|
collect_field_text(&mut units, &context, ®istry_field);
|
||||||
|
|
||||||
|
assert_eq!(units.len(), 1);
|
||||||
|
assert_eq!(units[0].source_text, "こんにちは");
|
||||||
|
assert_eq!(
|
||||||
|
units[0].field_path.as_deref(),
|
||||||
|
Some("m_ManagedReferences.RefIds[0].managedReferenceData.message")
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
units[0].context.get("source_kind"),
|
||||||
|
Some(&"ManagedReferenceField".to_string())
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
units[0].context.get("managed_reference_type"),
|
||||||
|
Some(&"ScenarioLine".to_string())
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
units[0].context.get("managed_reference_namespace"),
|
||||||
|
Some(&"BA.Text".to_string())
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
units[0].context.get("managed_reference_assembly"),
|
||||||
|
Some(&"Game".to_string())
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
units[0].context.get("managed_reference_full_type_name"),
|
||||||
|
Some(&"Game BA.Text.ScenarioLine".to_string())
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn extracts_fallback_managed_reference_payload_family_alias_with_context() {
|
||||||
|
let payload_field = string_field(
|
||||||
|
"m_ManagedReferences.RefIds[0].serializedReferencePayload.message",
|
||||||
|
"message",
|
||||||
|
"こんにちは",
|
||||||
|
);
|
||||||
|
let registry_field = UnitySerializedField {
|
||||||
|
path: "m_ManagedReferences".to_string(),
|
||||||
|
name: "m_ManagedReferences".to_string(),
|
||||||
|
type_name: "ManagedReferencesRegistry".to_string(),
|
||||||
|
offset: 0,
|
||||||
|
byte_size: 96,
|
||||||
|
type_tree_node_index: None,
|
||||||
|
value: UnitySerializedValue::ManagedReferenceRegistry {
|
||||||
|
references: Vec::new(),
|
||||||
|
fields: vec![
|
||||||
|
string_field(
|
||||||
|
"m_ManagedReferences.RefIds[0].managedReferenceClassName",
|
||||||
|
"managedReferenceClassName",
|
||||||
|
"ScenarioLine",
|
||||||
|
),
|
||||||
|
string_field(
|
||||||
|
"m_ManagedReferences.RefIds[0].managedReferenceNamespaceName",
|
||||||
|
"managedReferenceNamespaceName",
|
||||||
|
"BA.Text",
|
||||||
|
),
|
||||||
|
string_field(
|
||||||
|
"m_ManagedReferences.RefIds[0].managedReferenceAssemblyName",
|
||||||
|
"managedReferenceAssemblyName",
|
||||||
|
"Game",
|
||||||
|
),
|
||||||
|
UnitySerializedField {
|
||||||
|
path: "m_ManagedReferences.RefIds[0].serializedReferencePayload"
|
||||||
|
.to_string(),
|
||||||
|
name: "serializedReferencePayload".to_string(),
|
||||||
|
type_name: "managedReference".to_string(),
|
||||||
|
offset: 64,
|
||||||
|
byte_size: 24,
|
||||||
|
type_tree_node_index: None,
|
||||||
|
value: UnitySerializedValue::Object(vec![payload_field]),
|
||||||
|
},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
};
|
||||||
|
let context = FieldTextContext {
|
||||||
|
serialized_file_path: Some("CAB-test"),
|
||||||
|
path_id: 7,
|
||||||
|
class_id: 114,
|
||||||
|
version: "2021.3.56f2",
|
||||||
|
bundle_path: Some("scenario.bundle"),
|
||||||
|
archive_entry: None,
|
||||||
|
managed_reference: None,
|
||||||
|
};
|
||||||
|
let mut units = Vec::new();
|
||||||
|
|
||||||
|
collect_field_text(&mut units, &context, ®istry_field);
|
||||||
|
|
||||||
|
assert_eq!(units.len(), 1);
|
||||||
|
assert_eq!(units[0].source_text, "こんにちは");
|
||||||
|
assert_eq!(
|
||||||
|
units[0].field_path.as_deref(),
|
||||||
|
Some("m_ManagedReferences.RefIds[0].serializedReferencePayload.message")
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
units[0].context.get("source_kind"),
|
||||||
|
Some(&"ManagedReferenceField".to_string())
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
units[0].context.get("managed_reference_type"),
|
||||||
|
Some(&"ScenarioLine".to_string())
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
units[0].context.get("managed_reference_namespace"),
|
||||||
|
Some(&"BA.Text".to_string())
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
units[0].context.get("managed_reference_assembly"),
|
||||||
|
Some(&"Game".to_string())
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn extracts_fallback_managed_reference_sibling_records_with_separate_context() {
|
||||||
|
let registry_field = UnitySerializedField {
|
||||||
|
path: "m_ManagedReferences".to_string(),
|
||||||
|
name: "m_ManagedReferences".to_string(),
|
||||||
|
type_name: "ManagedReferencesRegistry".to_string(),
|
||||||
|
offset: 0,
|
||||||
|
byte_size: 160,
|
||||||
|
type_tree_node_index: None,
|
||||||
|
value: UnitySerializedValue::ManagedReferenceRegistry {
|
||||||
|
references: Vec::new(),
|
||||||
|
fields: vec![
|
||||||
|
string_field(
|
||||||
|
"m_ManagedReferences.RefIds[0].managedReferenceClassName",
|
||||||
|
"managedReferenceClassName",
|
||||||
|
"ScenarioLine",
|
||||||
|
),
|
||||||
|
string_field(
|
||||||
|
"m_ManagedReferences.RefIds[0].managedReferenceNamespaceName",
|
||||||
|
"managedReferenceNamespaceName",
|
||||||
|
"BA.Text",
|
||||||
|
),
|
||||||
|
string_field(
|
||||||
|
"m_ManagedReferences.RefIds[0].managedReferenceAssemblyName",
|
||||||
|
"managedReferenceAssemblyName",
|
||||||
|
"Game",
|
||||||
|
),
|
||||||
|
UnitySerializedField {
|
||||||
|
path: "m_ManagedReferences.RefIds[0].managedReferenceData".to_string(),
|
||||||
|
name: "managedReferenceData".to_string(),
|
||||||
|
type_name: "managedReference".to_string(),
|
||||||
|
offset: 64,
|
||||||
|
byte_size: 24,
|
||||||
|
type_tree_node_index: None,
|
||||||
|
value: UnitySerializedValue::Object(vec![string_field(
|
||||||
|
"m_ManagedReferences.RefIds[0].managedReferenceData.message",
|
||||||
|
"message",
|
||||||
|
"こんにちは",
|
||||||
|
)]),
|
||||||
|
},
|
||||||
|
string_field(
|
||||||
|
"m_ManagedReferences.RefIds[1].managedReferenceClassName",
|
||||||
|
"managedReferenceClassName",
|
||||||
|
"ChoiceLine",
|
||||||
|
),
|
||||||
|
string_field(
|
||||||
|
"m_ManagedReferences.RefIds[1].managedReferenceNamespaceName",
|
||||||
|
"managedReferenceNamespaceName",
|
||||||
|
"BA.Text",
|
||||||
|
),
|
||||||
|
string_field(
|
||||||
|
"m_ManagedReferences.RefIds[1].managedReferenceAssemblyName",
|
||||||
|
"managedReferenceAssemblyName",
|
||||||
|
"Game",
|
||||||
|
),
|
||||||
|
UnitySerializedField {
|
||||||
|
path: "m_ManagedReferences.RefIds[1].referencePayload".to_string(),
|
||||||
|
name: "referencePayload".to_string(),
|
||||||
|
type_name: "managedReference".to_string(),
|
||||||
|
offset: 120,
|
||||||
|
byte_size: 24,
|
||||||
|
type_tree_node_index: None,
|
||||||
|
value: UnitySerializedValue::Object(vec![string_field(
|
||||||
|
"m_ManagedReferences.RefIds[1].referencePayload.message",
|
||||||
|
"message",
|
||||||
|
"選択肢",
|
||||||
|
)]),
|
||||||
|
},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
};
|
||||||
|
let context = FieldTextContext {
|
||||||
|
serialized_file_path: Some("CAB-test"),
|
||||||
|
path_id: 7,
|
||||||
|
class_id: 114,
|
||||||
|
version: "2021.3.56f2",
|
||||||
|
bundle_path: Some("scenario.bundle"),
|
||||||
|
archive_entry: None,
|
||||||
|
managed_reference: None,
|
||||||
|
};
|
||||||
|
let mut units = Vec::new();
|
||||||
|
|
||||||
|
collect_field_text(&mut units, &context, ®istry_field);
|
||||||
|
|
||||||
|
assert_eq!(units.len(), 2);
|
||||||
|
assert_eq!(
|
||||||
|
units[0].field_path.as_deref(),
|
||||||
|
Some("m_ManagedReferences.RefIds[0].managedReferenceData.message")
|
||||||
|
);
|
||||||
|
assert_eq!(units[0].source_text, "こんにちは");
|
||||||
|
assert_eq!(
|
||||||
|
units[0].context.get("managed_reference_type"),
|
||||||
|
Some(&"ScenarioLine".to_string())
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
units[1].field_path.as_deref(),
|
||||||
|
Some("m_ManagedReferences.RefIds[1].referencePayload.message")
|
||||||
|
);
|
||||||
|
assert_eq!(units[1].source_text, "選択肢");
|
||||||
|
assert_eq!(
|
||||||
|
units[1].context.get("managed_reference_type"),
|
||||||
|
Some(&"ChoiceLine".to_string())
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn string_field(path: &str, name: &str, value: &str) -> UnitySerializedField {
|
||||||
|
UnitySerializedField {
|
||||||
|
path: path.to_string(),
|
||||||
|
name: name.to_string(),
|
||||||
|
type_name: "string".to_string(),
|
||||||
|
offset: 0,
|
||||||
|
byte_size: value.len() + 4,
|
||||||
|
type_tree_node_index: None,
|
||||||
|
value: UnitySerializedValue::String(value.to_string()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,7 +1,184 @@
|
|||||||
//! AssetBundle 类型定义占位
|
//! AssetBundle and UnityFS public types.
|
||||||
|
|
||||||
/// Asset 类型(待实现)
|
use crate::serialized::{UnitySerializedFile, UnitySerializedTextAsset};
|
||||||
|
|
||||||
|
/// Asset type extracted from a Unity bundle.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||||
pub enum AssetType {
|
pub enum AssetType {
|
||||||
/// 文本资源
|
/// Unity TextAsset.
|
||||||
TextAsset,
|
TextAsset,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Raw AssetBundle bytes with optional source path.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub struct RawAssetBundle {
|
||||||
|
/// File bytes.
|
||||||
|
pub data: Vec<u8>,
|
||||||
|
/// Source path or logical name, when known.
|
||||||
|
pub path: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Parsed AssetBundle summary used by higher layers.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub struct ParsedAssetBundle {
|
||||||
|
/// Unity editor version declared by the bundle.
|
||||||
|
pub unity_version: String,
|
||||||
|
/// Directory paths exposed by the UnityFS container.
|
||||||
|
pub assets: Vec<String>,
|
||||||
|
/// Original bytes retained for future serialization.
|
||||||
|
pub raw_data: Vec<u8>,
|
||||||
|
/// UnityFS header information.
|
||||||
|
pub unityfs_header: Option<UnityFsHeader>,
|
||||||
|
/// UnityFS compressed block entries.
|
||||||
|
pub blocks: Vec<UnityFsBlockInfo>,
|
||||||
|
/// UnityFS directory entries.
|
||||||
|
pub directories: Vec<UnityFsDirectoryInfo>,
|
||||||
|
/// Files extracted from the UnityFS uncompressed data region.
|
||||||
|
pub files: Vec<UnityFsFile>,
|
||||||
|
/// Serialized files parsed from UnityFS directory files.
|
||||||
|
pub serialized_files: Vec<UnitySerializedFile>,
|
||||||
|
/// TextAsset objects extracted from serialized files.
|
||||||
|
pub text_assets: Vec<UnitySerializedTextAsset>,
|
||||||
|
/// Non-fatal serialized-file parse diagnostics for extracted files.
|
||||||
|
pub serialized_parse_errors: Vec<UnitySerializedParseError>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Parsed UnityFS container.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub struct UnityFsBundle {
|
||||||
|
/// UnityFS header.
|
||||||
|
pub header: UnityFsHeader,
|
||||||
|
/// 16-byte block info hash stored before block entries.
|
||||||
|
pub blocks_info_hash: [u8; 16],
|
||||||
|
/// UnityFS compressed block entries.
|
||||||
|
pub blocks: Vec<UnityFsBlockInfo>,
|
||||||
|
/// UnityFS directory entries.
|
||||||
|
pub directories: Vec<UnityFsDirectoryInfo>,
|
||||||
|
/// Offset where compressed block payload bytes begin.
|
||||||
|
pub data_start_offset: u64,
|
||||||
|
/// Total compressed payload bytes declared by block entries.
|
||||||
|
pub compressed_data_size: u64,
|
||||||
|
/// Total uncompressed payload bytes declared by block entries.
|
||||||
|
pub uncompressed_data_size: u64,
|
||||||
|
/// Original bytes retained for future extraction/serialization.
|
||||||
|
pub raw_data: Vec<u8>,
|
||||||
|
/// Files extracted from the UnityFS uncompressed data region.
|
||||||
|
pub files: Vec<UnityFsFile>,
|
||||||
|
/// Serialized files parsed from UnityFS directory files.
|
||||||
|
pub serialized_files: Vec<UnitySerializedFile>,
|
||||||
|
/// TextAsset objects extracted from serialized files.
|
||||||
|
pub text_assets: Vec<UnitySerializedTextAsset>,
|
||||||
|
/// Non-fatal serialized-file parse diagnostics for extracted files.
|
||||||
|
pub serialized_parse_errors: Vec<UnitySerializedParseError>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl UnityFsBundle {
|
||||||
|
/// Returns directory paths in stable order.
|
||||||
|
pub fn asset_paths(&self) -> Vec<String> {
|
||||||
|
self.directories
|
||||||
|
.iter()
|
||||||
|
.map(|directory| directory.path.clone())
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl From<UnityFsBundle> for ParsedAssetBundle {
|
||||||
|
fn from(bundle: UnityFsBundle) -> Self {
|
||||||
|
Self {
|
||||||
|
unity_version: bundle.header.unity_version.clone(),
|
||||||
|
assets: bundle.asset_paths(),
|
||||||
|
raw_data: bundle.raw_data,
|
||||||
|
unityfs_header: Some(bundle.header),
|
||||||
|
blocks: bundle.blocks,
|
||||||
|
directories: bundle.directories,
|
||||||
|
files: bundle.files,
|
||||||
|
serialized_files: bundle.serialized_files,
|
||||||
|
text_assets: bundle.text_assets,
|
||||||
|
serialized_parse_errors: bundle.serialized_parse_errors,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// UnityFS header.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||||
|
pub struct UnityFsHeader {
|
||||||
|
/// UnityFS format version.
|
||||||
|
pub format_version: u32,
|
||||||
|
/// Bundle target version, for example `5.x.x`.
|
||||||
|
pub target_version: String,
|
||||||
|
/// Unity editor version, for example `2021.3.56f2`.
|
||||||
|
pub unity_version: String,
|
||||||
|
/// Total file size declared by the header.
|
||||||
|
pub total_size: u64,
|
||||||
|
/// Compressed block info byte size.
|
||||||
|
pub compressed_blocks_info_size: u32,
|
||||||
|
/// Uncompressed block info byte size.
|
||||||
|
pub uncompressed_blocks_info_size: u32,
|
||||||
|
/// Raw UnityFS flags.
|
||||||
|
pub flags: u32,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// UnityFS compression mode.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||||
|
pub enum UnityFsCompression {
|
||||||
|
/// Uncompressed.
|
||||||
|
None,
|
||||||
|
/// LZMA compression.
|
||||||
|
Lzma,
|
||||||
|
/// LZ4 compression.
|
||||||
|
Lz4,
|
||||||
|
/// LZ4HC compression.
|
||||||
|
Lz4Hc,
|
||||||
|
/// Unknown compression mode.
|
||||||
|
Unknown(u16),
|
||||||
|
}
|
||||||
|
|
||||||
|
/// UnityFS compressed block entry.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||||
|
pub struct UnityFsBlockInfo {
|
||||||
|
/// Uncompressed block size.
|
||||||
|
pub uncompressed_size: u32,
|
||||||
|
/// Compressed block size.
|
||||||
|
pub compressed_size: u32,
|
||||||
|
/// Raw block flags.
|
||||||
|
pub flags: u16,
|
||||||
|
/// Compression mode decoded from `flags`.
|
||||||
|
pub compression: UnityFsCompression,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// UnityFS directory entry.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||||
|
pub struct UnityFsDirectoryInfo {
|
||||||
|
/// Entry offset in the uncompressed data region.
|
||||||
|
pub offset: u64,
|
||||||
|
/// Entry byte size.
|
||||||
|
pub size: u64,
|
||||||
|
/// Raw directory flags.
|
||||||
|
pub flags: u32,
|
||||||
|
/// Entry path.
|
||||||
|
pub path: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// File extracted from a UnityFS directory entry.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub struct UnityFsFile {
|
||||||
|
/// Entry path from the UnityFS directory table.
|
||||||
|
pub path: String,
|
||||||
|
/// Offset in the uncompressed UnityFS data region.
|
||||||
|
pub offset: u64,
|
||||||
|
/// File byte size.
|
||||||
|
pub size: u64,
|
||||||
|
/// Raw directory flags.
|
||||||
|
pub flags: u32,
|
||||||
|
/// Extracted file bytes.
|
||||||
|
pub data: Vec<u8>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Non-fatal parse error for an extracted UnityFS file.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||||
|
pub struct UnitySerializedParseError {
|
||||||
|
/// UnityFS directory path that failed serialized-file parsing.
|
||||||
|
pub path: String,
|
||||||
|
/// Human-readable parser error.
|
||||||
|
pub error: String,
|
||||||
|
}
|
||||||
|
|||||||
@@ -1,13 +1,145 @@
|
|||||||
//! Binary Patch 模块占位
|
//! Deterministic binary hunk patch.
|
||||||
|
|
||||||
/// Binary Patch 应用(尚未实现)。
|
use crate::PatchError;
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
|
||||||
|
/// Current binary patch schema version.
|
||||||
|
pub const BINARY_PATCH_VERSION: u32 = 1;
|
||||||
|
|
||||||
|
/// Binary patch made of deterministic copy/insert hunks.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct BinaryPatch {
|
||||||
|
/// Patch schema version.
|
||||||
|
pub version: u32,
|
||||||
|
/// Expected BLAKE3 hash of the source bytes.
|
||||||
|
pub source_blake3: String,
|
||||||
|
/// Expected BLAKE3 hash of the target bytes.
|
||||||
|
pub target_blake3: String,
|
||||||
|
/// Source byte length.
|
||||||
|
pub source_size: u64,
|
||||||
|
/// Target byte length.
|
||||||
|
pub target_size: u64,
|
||||||
|
/// Ordered hunks.
|
||||||
|
pub hunks: Vec<BinaryPatchHunk>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One binary patch hunk.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
#[serde(tag = "kind", rename_all = "snake_case")]
|
||||||
|
pub enum BinaryPatchHunk {
|
||||||
|
/// Copy a byte range from the source.
|
||||||
|
Copy {
|
||||||
|
/// Source offset.
|
||||||
|
offset: u64,
|
||||||
|
/// Number of bytes to copy.
|
||||||
|
length: u64,
|
||||||
|
},
|
||||||
|
/// Insert literal bytes.
|
||||||
|
Insert {
|
||||||
|
/// Literal bytes.
|
||||||
|
bytes: Vec<u8>,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Creates a deterministic hunk patch.
|
||||||
///
|
///
|
||||||
/// 返回 [`crate::PatchError::ApplyFailed`] 而非空结果,避免调用方把未实现的
|
/// The first implementation optimizes for correctness and stable output. It
|
||||||
/// 占位当成一次成功的补丁应用。
|
/// emits copy hunks for equal runs and insert hunks for changed runs; more
|
||||||
pub fn apply_patch(_old: &[u8], _patch: &[u8]) -> crate::Result<Vec<u8>> {
|
/// compact suffix/prefix matching can be added later without changing the
|
||||||
Err(crate::PatchError::ApplyFailed(
|
/// manifest/integrity contract.
|
||||||
"binary patch 尚未实现".to_string(),
|
pub fn diff(old: &[u8], new: &[u8]) -> BinaryPatch {
|
||||||
|
let mut hunks = Vec::new();
|
||||||
|
let mut index = 0usize;
|
||||||
|
while index < new.len() {
|
||||||
|
if index < old.len() && old[index] == new[index] {
|
||||||
|
let start = index;
|
||||||
|
while index < new.len() && index < old.len() && old[index] == new[index] {
|
||||||
|
index += 1;
|
||||||
|
}
|
||||||
|
hunks.push(BinaryPatchHunk::Copy {
|
||||||
|
offset: start as u64,
|
||||||
|
length: (index - start) as u64,
|
||||||
|
});
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
let start = index;
|
||||||
|
while index < new.len() && (index >= old.len() || old[index] != new[index]) {
|
||||||
|
index += 1;
|
||||||
|
}
|
||||||
|
hunks.push(BinaryPatchHunk::Insert {
|
||||||
|
bytes: new[start..index].to_vec(),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
BinaryPatch {
|
||||||
|
version: BINARY_PATCH_VERSION,
|
||||||
|
source_blake3: blake3_hex(old),
|
||||||
|
target_blake3: blake3_hex(new),
|
||||||
|
source_size: old.len() as u64,
|
||||||
|
target_size: new.len() as u64,
|
||||||
|
hunks,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Applies a structured binary patch.
|
||||||
|
pub fn apply_binary_patch(old: &[u8], patch: &BinaryPatch) -> crate::Result<Vec<u8>> {
|
||||||
|
if patch.version != BINARY_PATCH_VERSION {
|
||||||
|
return Err(PatchError::ApplyFailed(format!(
|
||||||
|
"unsupported binary patch version {}",
|
||||||
|
patch.version
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
if patch.source_size != old.len() as u64 || patch.source_blake3 != blake3_hex(old) {
|
||||||
|
return Err(PatchError::ApplyFailed(
|
||||||
|
"binary patch source integrity mismatch".to_string(),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
let target_capacity = usize::try_from(patch.target_size)
|
||||||
|
.map_err(|_| PatchError::ApplyFailed("binary patch target too large".to_string()))?;
|
||||||
|
let mut output = Vec::with_capacity(target_capacity);
|
||||||
|
for hunk in &patch.hunks {
|
||||||
|
match hunk {
|
||||||
|
BinaryPatchHunk::Copy { offset, length } => {
|
||||||
|
let start = usize::try_from(*offset).map_err(|_| {
|
||||||
|
PatchError::ApplyFailed("binary patch copy offset overflow".to_string())
|
||||||
|
})?;
|
||||||
|
let length = usize::try_from(*length).map_err(|_| {
|
||||||
|
PatchError::ApplyFailed("binary patch copy length overflow".to_string())
|
||||||
|
})?;
|
||||||
|
let end = start.checked_add(length).ok_or_else(|| {
|
||||||
|
PatchError::ApplyFailed("binary patch copy range overflow".to_string())
|
||||||
|
})?;
|
||||||
|
let bytes = old.get(start..end).ok_or_else(|| {
|
||||||
|
PatchError::ApplyFailed(format!(
|
||||||
|
"binary patch copy range {start}..{end} exceeds source {}",
|
||||||
|
old.len()
|
||||||
))
|
))
|
||||||
|
})?;
|
||||||
|
output.extend_from_slice(bytes);
|
||||||
|
}
|
||||||
|
BinaryPatchHunk::Insert { bytes } => output.extend_from_slice(bytes),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if output.len() as u64 != patch.target_size || blake3_hex(&output) != patch.target_blake3 {
|
||||||
|
return Err(PatchError::ApplyFailed(
|
||||||
|
"binary patch target integrity mismatch".to_string(),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
Ok(output)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Serializes and applies a binary patch.
|
||||||
|
pub fn apply_patch(old: &[u8], patch: &[u8]) -> crate::Result<Vec<u8>> {
|
||||||
|
let patch: BinaryPatch = serde_json::from_slice(patch)
|
||||||
|
.map_err(|error| PatchError::ApplyFailed(format!("invalid binary patch JSON: {error}")))?;
|
||||||
|
apply_binary_patch(old, &patch)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn blake3_hex(bytes: &[u8]) -> String {
|
||||||
|
blake3::hash(bytes).to_hex().to_string()
|
||||||
}
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
@@ -15,8 +147,24 @@ mod tests {
|
|||||||
use super::*;
|
use super::*;
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn apply_patch_reports_not_implemented() {
|
fn binary_patch_round_trips_changed_bytes() {
|
||||||
let error = apply_patch(b"old", b"patch").unwrap_err();
|
let old = b"abcdef012345";
|
||||||
|
let new = b"abcXYZ012345!";
|
||||||
|
let patch = diff(old, new);
|
||||||
|
let patch_json = serde_json::to_vec(&patch).unwrap();
|
||||||
|
|
||||||
|
assert_eq!(apply_binary_patch(old, &patch).unwrap(), new);
|
||||||
|
assert_eq!(apply_patch(old, &patch_json).unwrap(), new);
|
||||||
|
assert!(patch
|
||||||
|
.hunks
|
||||||
|
.iter()
|
||||||
|
.any(|hunk| matches!(hunk, BinaryPatchHunk::Insert { .. })));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn binary_patch_rejects_wrong_source() {
|
||||||
|
let patch = diff(b"old", b"new");
|
||||||
|
let error = apply_binary_patch(b"bad", &patch).unwrap_err();
|
||||||
assert!(matches!(error, crate::PatchError::ApplyFailed(_)));
|
assert!(matches!(error, crate::PatchError::ApplyFailed(_)));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+354
-10
@@ -1,22 +1,366 @@
|
|||||||
//! JSON Patch 模块占位
|
//! RFC 6902 JSON Patch support.
|
||||||
|
|
||||||
/// JSON Patch 应用(尚未实现)。
|
use crate::PatchError;
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use serde_json::Value;
|
||||||
|
|
||||||
|
/// One RFC 6902 JSON Patch operation.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||||
|
#[serde(tag = "op", rename_all = "lowercase")]
|
||||||
|
pub enum JsonPatchOperation {
|
||||||
|
/// Add a value at the target JSON Pointer.
|
||||||
|
Add {
|
||||||
|
/// Target JSON Pointer.
|
||||||
|
path: String,
|
||||||
|
/// Value to insert.
|
||||||
|
value: Value,
|
||||||
|
},
|
||||||
|
/// Remove the value at the target JSON Pointer.
|
||||||
|
Remove {
|
||||||
|
/// Target JSON Pointer.
|
||||||
|
path: String,
|
||||||
|
},
|
||||||
|
/// Replace the value at the target JSON Pointer.
|
||||||
|
Replace {
|
||||||
|
/// Target JSON Pointer.
|
||||||
|
path: String,
|
||||||
|
/// Replacement value.
|
||||||
|
value: Value,
|
||||||
|
},
|
||||||
|
/// Move a value from one JSON Pointer to another.
|
||||||
|
Move {
|
||||||
|
/// Source JSON Pointer.
|
||||||
|
from: String,
|
||||||
|
/// Target JSON Pointer.
|
||||||
|
path: String,
|
||||||
|
},
|
||||||
|
/// Copy a value from one JSON Pointer to another.
|
||||||
|
Copy {
|
||||||
|
/// Source JSON Pointer.
|
||||||
|
from: String,
|
||||||
|
/// Target JSON Pointer.
|
||||||
|
path: String,
|
||||||
|
},
|
||||||
|
/// Assert that a JSON Pointer currently contains a value.
|
||||||
|
Test {
|
||||||
|
/// Target JSON Pointer.
|
||||||
|
path: String,
|
||||||
|
/// Expected value.
|
||||||
|
value: Value,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Applies an RFC 6902 JSON Patch document to a JSON document string.
|
||||||
|
pub fn apply_json_patch(doc: &str, patch: &str) -> crate::Result<String> {
|
||||||
|
let mut document: Value = serde_json::from_str(doc)
|
||||||
|
.map_err(|error| PatchError::ApplyFailed(format!("invalid JSON document: {error}")))?;
|
||||||
|
let operations: Vec<JsonPatchOperation> = serde_json::from_str(patch)
|
||||||
|
.map_err(|error| PatchError::ApplyFailed(format!("invalid JSON patch: {error}")))?;
|
||||||
|
apply_json_patch_value(&mut document, &operations)?;
|
||||||
|
serde_json::to_string(&document)
|
||||||
|
.map_err(|error| PatchError::ApplyFailed(format!("failed to serialize JSON: {error}")))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Applies parsed JSON Patch operations to a JSON value.
|
||||||
///
|
///
|
||||||
/// 返回 [`crate::PatchError::ApplyFailed`] 而非空字符串,避免调用方把未实现的
|
/// Each operation is applied atomically: when one operation fails, the document
|
||||||
/// 占位当成一次成功的补丁应用。
|
/// remains at the state produced by the previous successful operation.
|
||||||
pub fn apply_json_patch(_doc: &str, _patch: &str) -> crate::Result<String> {
|
pub fn apply_json_patch_value(
|
||||||
Err(crate::PatchError::ApplyFailed(
|
document: &mut Value,
|
||||||
"json patch 尚未实现".to_string(),
|
operations: &[JsonPatchOperation],
|
||||||
))
|
) -> crate::Result<()> {
|
||||||
|
for operation in operations {
|
||||||
|
let mut next = document.clone();
|
||||||
|
apply_operation(&mut next, operation)?;
|
||||||
|
*document = next;
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn apply_operation(document: &mut Value, operation: &JsonPatchOperation) -> crate::Result<()> {
|
||||||
|
match operation {
|
||||||
|
JsonPatchOperation::Add { path, value } => add_value(document, path, value.clone()),
|
||||||
|
JsonPatchOperation::Remove { path } => remove_value(document, path).map(drop),
|
||||||
|
JsonPatchOperation::Replace { path, value } => replace_value(document, path, value.clone()),
|
||||||
|
JsonPatchOperation::Move { from, path } => {
|
||||||
|
if from == path {
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
let value = get_value(document, from)?.clone();
|
||||||
|
remove_value(document, from)?;
|
||||||
|
add_value(document, path, value)
|
||||||
|
}
|
||||||
|
JsonPatchOperation::Copy { from, path } => {
|
||||||
|
let value = get_value(document, from)?.clone();
|
||||||
|
add_value(document, path, value)
|
||||||
|
}
|
||||||
|
JsonPatchOperation::Test { path, value } => {
|
||||||
|
let actual = get_value(document, path)?;
|
||||||
|
if actual == value {
|
||||||
|
Ok(())
|
||||||
|
} else {
|
||||||
|
Err(failed(format!(
|
||||||
|
"JSON patch test failed at {path}: expected {value}, actual {actual}"
|
||||||
|
)))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn add_value(document: &mut Value, path: &str, value: Value) -> crate::Result<()> {
|
||||||
|
let tokens = parse_json_pointer(path)?;
|
||||||
|
if tokens.is_empty() {
|
||||||
|
*document = value;
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
|
||||||
|
let key = tokens.last().expect("checked non-empty").clone();
|
||||||
|
let parent = get_mut_by_tokens(document, &tokens[..tokens.len() - 1])?;
|
||||||
|
match parent {
|
||||||
|
Value::Object(map) => {
|
||||||
|
map.insert(key, value);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
Value::Array(items) => {
|
||||||
|
if key == "-" {
|
||||||
|
items.push(value);
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
let index = parse_array_index(&key, items.len(), true)?;
|
||||||
|
items.insert(index, value);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
other => Err(failed(format!(
|
||||||
|
"cannot add JSON patch value below non-container value {other}"
|
||||||
|
))),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn remove_value(document: &mut Value, path: &str) -> crate::Result<Value> {
|
||||||
|
let tokens = parse_json_pointer(path)?;
|
||||||
|
if tokens.is_empty() {
|
||||||
|
return Ok(std::mem::take(document));
|
||||||
|
}
|
||||||
|
|
||||||
|
let key = tokens.last().expect("checked non-empty").clone();
|
||||||
|
let parent = get_mut_by_tokens(document, &tokens[..tokens.len() - 1])?;
|
||||||
|
match parent {
|
||||||
|
Value::Object(map) => map
|
||||||
|
.remove(&key)
|
||||||
|
.ok_or_else(|| failed(format!("JSON patch path {path} does not exist"))),
|
||||||
|
Value::Array(items) => {
|
||||||
|
let index = parse_array_index(&key, items.len(), false)?;
|
||||||
|
Ok(items.remove(index))
|
||||||
|
}
|
||||||
|
other => Err(failed(format!(
|
||||||
|
"cannot remove JSON patch value below non-container value {other}"
|
||||||
|
))),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn replace_value(document: &mut Value, path: &str, value: Value) -> crate::Result<()> {
|
||||||
|
let tokens = parse_json_pointer(path)?;
|
||||||
|
if tokens.is_empty() {
|
||||||
|
*document = value;
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
|
||||||
|
let key = tokens.last().expect("checked non-empty").clone();
|
||||||
|
let parent = get_mut_by_tokens(document, &tokens[..tokens.len() - 1])?;
|
||||||
|
match parent {
|
||||||
|
Value::Object(map) => {
|
||||||
|
let slot = map
|
||||||
|
.get_mut(&key)
|
||||||
|
.ok_or_else(|| failed(format!("JSON patch path {path} does not exist")))?;
|
||||||
|
*slot = value;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
Value::Array(items) => {
|
||||||
|
let index = parse_array_index(&key, items.len(), false)?;
|
||||||
|
items[index] = value;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
other => Err(failed(format!(
|
||||||
|
"cannot replace JSON patch value below non-container value {other}"
|
||||||
|
))),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn get_value<'a>(document: &'a Value, path: &str) -> crate::Result<&'a Value> {
|
||||||
|
let tokens = parse_json_pointer(path)?;
|
||||||
|
let mut current = document;
|
||||||
|
for token in &tokens {
|
||||||
|
current = match current {
|
||||||
|
Value::Object(map) => map
|
||||||
|
.get(token)
|
||||||
|
.ok_or_else(|| failed(format!("JSON patch path {path} does not exist")))?,
|
||||||
|
Value::Array(items) => {
|
||||||
|
let index = parse_array_index(token, items.len(), false)?;
|
||||||
|
items
|
||||||
|
.get(index)
|
||||||
|
.ok_or_else(|| failed(format!("JSON patch path {path} does not exist")))?
|
||||||
|
}
|
||||||
|
other => {
|
||||||
|
return Err(failed(format!(
|
||||||
|
"cannot traverse JSON patch path {path} through non-container value {other}"
|
||||||
|
)))
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
Ok(current)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn get_mut_by_tokens<'a>(
|
||||||
|
document: &'a mut Value,
|
||||||
|
tokens: &[String],
|
||||||
|
) -> crate::Result<&'a mut Value> {
|
||||||
|
let mut current = document;
|
||||||
|
for token in tokens {
|
||||||
|
current = match current {
|
||||||
|
Value::Object(map) => map
|
||||||
|
.get_mut(token)
|
||||||
|
.ok_or_else(|| failed(format!("JSON patch path segment {token} does not exist")))?,
|
||||||
|
Value::Array(items) => {
|
||||||
|
let index = parse_array_index(token, items.len(), false)?;
|
||||||
|
items.get_mut(index).ok_or_else(|| {
|
||||||
|
failed(format!("JSON patch path segment {token} does not exist"))
|
||||||
|
})?
|
||||||
|
}
|
||||||
|
other => {
|
||||||
|
return Err(failed(format!(
|
||||||
|
"cannot traverse JSON patch path through non-container value {other}"
|
||||||
|
)))
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
Ok(current)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn parse_array_index(token: &str, len: usize, allow_end: bool) -> crate::Result<usize> {
|
||||||
|
if token.is_empty() || token == "-" {
|
||||||
|
return Err(failed(format!("invalid JSON patch array index {token}")));
|
||||||
|
}
|
||||||
|
let index = token
|
||||||
|
.parse::<usize>()
|
||||||
|
.map_err(|_| failed(format!("invalid JSON patch array index {token}")))?;
|
||||||
|
let max = if allow_end {
|
||||||
|
len
|
||||||
|
} else {
|
||||||
|
len.checked_sub(1)
|
||||||
|
.ok_or_else(|| failed("JSON patch array index exceeds empty array".to_string()))?
|
||||||
|
};
|
||||||
|
if index > max {
|
||||||
|
return Err(failed(format!(
|
||||||
|
"JSON patch array index {index} exceeds length {len}"
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
Ok(index)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn parse_json_pointer(pointer: &str) -> crate::Result<Vec<String>> {
|
||||||
|
if pointer.is_empty() {
|
||||||
|
return Ok(Vec::new());
|
||||||
|
}
|
||||||
|
if !pointer.starts_with('/') {
|
||||||
|
return Err(failed(format!(
|
||||||
|
"JSON patch pointer must be empty or start with '/': {pointer}"
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
pointer[1..]
|
||||||
|
.split('/')
|
||||||
|
.map(decode_json_pointer_token)
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn decode_json_pointer_token(token: &str) -> crate::Result<String> {
|
||||||
|
let mut decoded = String::with_capacity(token.len());
|
||||||
|
let mut chars = token.chars();
|
||||||
|
while let Some(character) = chars.next() {
|
||||||
|
if character != '~' {
|
||||||
|
decoded.push(character);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
match chars.next() {
|
||||||
|
Some('0') => decoded.push('~'),
|
||||||
|
Some('1') => decoded.push('/'),
|
||||||
|
Some(other) => {
|
||||||
|
return Err(failed(format!(
|
||||||
|
"invalid JSON patch pointer escape ~{other}"
|
||||||
|
)))
|
||||||
|
}
|
||||||
|
None => return Err(failed("invalid JSON patch pointer escape ~".to_string())),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(decoded)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn failed(message: String) -> PatchError {
|
||||||
|
PatchError::ApplyFailed(message)
|
||||||
}
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
use serde_json::json;
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn apply_json_patch_reports_not_implemented() {
|
fn apply_json_patch_handles_all_core_operations() {
|
||||||
let error = apply_json_patch("{}", "[]").unwrap_err();
|
let document = r#"{"name":"alice","items":["a","b"],"meta":{"keep":true}}"#;
|
||||||
|
let patch = r#"[
|
||||||
|
{"op":"test","path":"/meta/keep","value":true},
|
||||||
|
{"op":"add","path":"/items/-","value":"c"},
|
||||||
|
{"op":"replace","path":"/name","value":"bob"},
|
||||||
|
{"op":"copy","from":"/meta","path":"/copied"},
|
||||||
|
{"op":"move","from":"/items/0","path":"/first"},
|
||||||
|
{"op":"remove","path":"/meta/keep"}
|
||||||
|
]"#;
|
||||||
|
|
||||||
|
let output = apply_json_patch(document, patch).unwrap();
|
||||||
|
let value: Value = serde_json::from_str(&output).unwrap();
|
||||||
|
|
||||||
|
assert_eq!(value["name"], json!("bob"));
|
||||||
|
assert_eq!(value["items"], json!(["b", "c"]));
|
||||||
|
assert_eq!(value["first"], json!("a"));
|
||||||
|
assert_eq!(value["copied"], json!({"keep": true}));
|
||||||
|
assert_eq!(value["meta"], json!({}));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn apply_json_patch_supports_pointer_escapes() {
|
||||||
|
let document = r#"{"a/b":{"tilde~key":1}}"#;
|
||||||
|
let patch = r#"[{"op":"replace","path":"/a~1b/tilde~0key","value":2}]"#;
|
||||||
|
|
||||||
|
let output = apply_json_patch(document, patch).unwrap();
|
||||||
|
let value: Value = serde_json::from_str(&output).unwrap();
|
||||||
|
|
||||||
|
assert_eq!(value["a/b"]["tilde~key"], json!(2));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn apply_json_patch_rejects_failed_test_without_mutating_value() {
|
||||||
|
let mut value = json!({"enabled": true});
|
||||||
|
let operations = vec![
|
||||||
|
JsonPatchOperation::Add {
|
||||||
|
path: "/count".to_string(),
|
||||||
|
value: json!(1),
|
||||||
|
},
|
||||||
|
JsonPatchOperation::Test {
|
||||||
|
path: "/enabled".to_string(),
|
||||||
|
value: json!(false),
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
let error = apply_json_patch_value(&mut value, &operations).unwrap_err();
|
||||||
|
|
||||||
|
assert!(matches!(error, crate::PatchError::ApplyFailed(_)));
|
||||||
|
assert_eq!(value, json!({"enabled": true, "count": 1}));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn apply_json_patch_rejects_missing_remove_path() {
|
||||||
|
let error = apply_json_patch(r#"{"items":[]}"#, r#"[{"op":"remove","path":"/missing"}]"#)
|
||||||
|
.unwrap_err();
|
||||||
|
|
||||||
assert!(matches!(error, crate::PatchError::ApplyFailed(_)));
|
assert!(matches!(error, crate::PatchError::ApplyFailed(_)));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -14,8 +14,14 @@
|
|||||||
pub mod binary;
|
pub mod binary;
|
||||||
pub mod error;
|
pub mod error;
|
||||||
pub mod json;
|
pub mod json;
|
||||||
|
pub mod manifest;
|
||||||
|
pub mod text;
|
||||||
|
|
||||||
pub use error::{PatchError, Result};
|
pub use error::{PatchError, Result};
|
||||||
|
pub use manifest::{
|
||||||
|
PatchIntegrity, PatchKind, PatchManifest, PatchManifestFile, PatchRollback,
|
||||||
|
PATCH_MANIFEST_VERSION,
|
||||||
|
};
|
||||||
|
|
||||||
/// Patch 引擎版本号
|
/// Patch 引擎版本号
|
||||||
pub const VERSION: &str = env!("CARGO_PKG_VERSION");
|
pub const VERSION: &str = env!("CARGO_PKG_VERSION");
|
||||||
|
|||||||
@@ -0,0 +1,261 @@
|
|||||||
|
//! Patch manifest, integrity and rollback primitives.
|
||||||
|
|
||||||
|
use crate::PatchError;
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use std::fs;
|
||||||
|
use std::path::{Component, Path, PathBuf};
|
||||||
|
|
||||||
|
/// Current patch manifest schema version.
|
||||||
|
pub const PATCH_MANIFEST_VERSION: u32 = 1;
|
||||||
|
|
||||||
|
/// Persisted manifest for a generated patch set.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct PatchManifest {
|
||||||
|
/// Manifest schema version.
|
||||||
|
#[serde(default = "default_patch_manifest_version")]
|
||||||
|
pub version: u32,
|
||||||
|
/// Stable patch identifier.
|
||||||
|
pub patch_id: String,
|
||||||
|
/// Source resource version identifier.
|
||||||
|
pub source_version: String,
|
||||||
|
/// Target resource version identifier.
|
||||||
|
pub target_version: String,
|
||||||
|
/// Files covered by this patch set.
|
||||||
|
pub files: Vec<PatchManifestFile>,
|
||||||
|
/// Rollback metadata for the publication layer.
|
||||||
|
pub rollback: PatchRollback,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl PatchManifest {
|
||||||
|
/// Builds a manifest-level integrity summary from recorded file metadata.
|
||||||
|
pub fn integrity_summary(&self) -> PatchIntegrity {
|
||||||
|
PatchIntegrity {
|
||||||
|
file_count: self.files.len(),
|
||||||
|
source_bytes: self.files.iter().map(|file| file.source_size).sum(),
|
||||||
|
target_bytes: self.files.iter().map(|file| file.target_size).sum(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One release-relative file entry in a patch manifest.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct PatchManifestFile {
|
||||||
|
/// Release-relative path.
|
||||||
|
pub path: PathBuf,
|
||||||
|
/// Patch algorithm used to produce the target bytes.
|
||||||
|
pub patch_kind: PatchKind,
|
||||||
|
/// Expected BLAKE3 hash of the source bytes.
|
||||||
|
pub source_blake3: String,
|
||||||
|
/// Expected BLAKE3 hash of the target bytes.
|
||||||
|
pub target_blake3: String,
|
||||||
|
/// Expected source byte length.
|
||||||
|
pub source_size: u64,
|
||||||
|
/// Expected target byte length.
|
||||||
|
pub target_size: u64,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Patch algorithm family used by one manifest file.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
#[serde(rename_all = "snake_case")]
|
||||||
|
pub enum PatchKind {
|
||||||
|
/// Deterministic binary hunk patch.
|
||||||
|
Binary,
|
||||||
|
/// RFC 6902 JSON Patch.
|
||||||
|
Json,
|
||||||
|
/// UTF-8 text patch.
|
||||||
|
Text,
|
||||||
|
/// UnityFS TextAsset replacement patch.
|
||||||
|
UnityFsTextAsset,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Rollback metadata owned by higher-level publication code.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct PatchRollback {
|
||||||
|
/// Previous `current` pointer target before publication.
|
||||||
|
pub previous_current_target: Option<PathBuf>,
|
||||||
|
/// Published target path that can be removed on rollback.
|
||||||
|
pub remove_target_path: Option<PathBuf>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Manifest-level integrity summary.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct PatchIntegrity {
|
||||||
|
/// Number of manifest files verified or summarized.
|
||||||
|
pub file_count: usize,
|
||||||
|
/// Total source bytes.
|
||||||
|
pub source_bytes: u64,
|
||||||
|
/// Total target bytes.
|
||||||
|
pub target_bytes: u64,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Verifies all manifest files against source and target roots.
|
||||||
|
pub fn verify_patch_manifest_files(
|
||||||
|
source_root: &Path,
|
||||||
|
target_root: &Path,
|
||||||
|
manifest: &PatchManifest,
|
||||||
|
) -> crate::Result<PatchIntegrity> {
|
||||||
|
if manifest.version != PATCH_MANIFEST_VERSION {
|
||||||
|
return Err(PatchError::ApplyFailed(format!(
|
||||||
|
"unsupported patch manifest version {}",
|
||||||
|
manifest.version
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
|
||||||
|
let mut integrity = PatchIntegrity {
|
||||||
|
file_count: 0,
|
||||||
|
source_bytes: 0,
|
||||||
|
target_bytes: 0,
|
||||||
|
};
|
||||||
|
for file in &manifest.files {
|
||||||
|
let source_path = resolve_manifest_path(source_root, &file.path)?;
|
||||||
|
let target_path = resolve_manifest_path(target_root, &file.path)?;
|
||||||
|
let source = read_manifest_file(&source_path, "source")?;
|
||||||
|
let target = read_manifest_file(&target_path, "target")?;
|
||||||
|
verify_patch_file_bytes(&source, &target, file)?;
|
||||||
|
integrity.file_count += 1;
|
||||||
|
integrity.source_bytes += source.len() as u64;
|
||||||
|
integrity.target_bytes += target.len() as u64;
|
||||||
|
}
|
||||||
|
Ok(integrity)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Verifies one manifest file entry against source and target bytes.
|
||||||
|
pub fn verify_patch_file_bytes(
|
||||||
|
source: &[u8],
|
||||||
|
target: &[u8],
|
||||||
|
file: &PatchManifestFile,
|
||||||
|
) -> crate::Result<()> {
|
||||||
|
let source_hash = blake3_hex(source);
|
||||||
|
let target_hash = blake3_hex(target);
|
||||||
|
if source_hash != file.source_blake3 || source.len() as u64 != file.source_size {
|
||||||
|
return Err(PatchError::ApplyFailed(format!(
|
||||||
|
"patch source integrity mismatch {}: expected hash={} bytes={}, actual hash={} bytes={}",
|
||||||
|
file.path.display(),
|
||||||
|
file.source_blake3,
|
||||||
|
file.source_size,
|
||||||
|
source_hash,
|
||||||
|
source.len()
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
if target_hash != file.target_blake3 || target.len() as u64 != file.target_size {
|
||||||
|
return Err(PatchError::ApplyFailed(format!(
|
||||||
|
"patch target integrity mismatch {}: expected hash={} bytes={}, actual hash={} bytes={}",
|
||||||
|
file.path.display(),
|
||||||
|
file.target_blake3,
|
||||||
|
file.target_size,
|
||||||
|
target_hash,
|
||||||
|
target.len()
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn resolve_manifest_path(root: &Path, relative: &Path) -> crate::Result<PathBuf> {
|
||||||
|
if relative.is_absolute() {
|
||||||
|
return Err(PatchError::ApplyFailed(format!(
|
||||||
|
"patch manifest path must be relative: {}",
|
||||||
|
relative.display()
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
for component in relative.components() {
|
||||||
|
match component {
|
||||||
|
Component::Normal(_) | Component::CurDir => {}
|
||||||
|
_ => {
|
||||||
|
return Err(PatchError::ApplyFailed(format!(
|
||||||
|
"patch manifest path escapes release root: {}",
|
||||||
|
relative.display()
|
||||||
|
)))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(root.join(relative))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn read_manifest_file(path: &Path, label: &str) -> crate::Result<Vec<u8>> {
|
||||||
|
fs::read(path).map_err(|error| {
|
||||||
|
PatchError::ApplyFailed(format!(
|
||||||
|
"failed to read patch {label} file {}: {error}",
|
||||||
|
path.display()
|
||||||
|
))
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
fn blake3_hex(bytes: &[u8]) -> String {
|
||||||
|
blake3::hash(bytes).to_hex().to_string()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn default_patch_manifest_version() -> u32 {
|
||||||
|
PATCH_MANIFEST_VERSION
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn verify_patch_manifest_files_accepts_matching_roots() {
|
||||||
|
let temp = tempfile::tempdir().unwrap();
|
||||||
|
let source_root = temp.path().join("source");
|
||||||
|
let target_root = temp.path().join("target");
|
||||||
|
fs::create_dir_all(source_root.join("TableBundles")).unwrap();
|
||||||
|
fs::create_dir_all(target_root.join("TableBundles")).unwrap();
|
||||||
|
let source = b"before";
|
||||||
|
let target = b"after";
|
||||||
|
fs::write(source_root.join("TableBundles/file.bytes"), source).unwrap();
|
||||||
|
fs::write(target_root.join("TableBundles/file.bytes"), target).unwrap();
|
||||||
|
|
||||||
|
let manifest = manifest_for("TableBundles/file.bytes", source, target);
|
||||||
|
let integrity = verify_patch_manifest_files(&source_root, &target_root, &manifest).unwrap();
|
||||||
|
|
||||||
|
assert_eq!(
|
||||||
|
integrity,
|
||||||
|
PatchIntegrity {
|
||||||
|
file_count: 1,
|
||||||
|
source_bytes: source.len() as u64,
|
||||||
|
target_bytes: target.len() as u64,
|
||||||
|
}
|
||||||
|
);
|
||||||
|
assert_eq!(manifest.integrity_summary(), integrity);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn verify_patch_manifest_files_rejects_path_escape() {
|
||||||
|
let temp = tempfile::tempdir().unwrap();
|
||||||
|
let manifest = manifest_for("../escape", b"source", b"target");
|
||||||
|
|
||||||
|
let error = verify_patch_manifest_files(temp.path(), temp.path(), &manifest).unwrap_err();
|
||||||
|
|
||||||
|
assert!(matches!(error, PatchError::ApplyFailed(_)));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn verify_patch_file_bytes_rejects_hash_mismatch() {
|
||||||
|
let mut manifest = manifest_for("file.bin", b"source", b"target");
|
||||||
|
manifest.files[0].target_blake3 = blake3_hex(b"other");
|
||||||
|
|
||||||
|
let error = verify_patch_file_bytes(b"source", b"target", &manifest.files[0]).unwrap_err();
|
||||||
|
|
||||||
|
assert!(matches!(error, PatchError::ApplyFailed(_)));
|
||||||
|
}
|
||||||
|
|
||||||
|
fn manifest_for(path: &str, source: &[u8], target: &[u8]) -> PatchManifest {
|
||||||
|
PatchManifest {
|
||||||
|
version: PATCH_MANIFEST_VERSION,
|
||||||
|
patch_id: "patch-id".to_string(),
|
||||||
|
source_version: "source-version".to_string(),
|
||||||
|
target_version: "target-version".to_string(),
|
||||||
|
files: vec![PatchManifestFile {
|
||||||
|
path: PathBuf::from(path),
|
||||||
|
patch_kind: PatchKind::Binary,
|
||||||
|
source_blake3: blake3_hex(source),
|
||||||
|
target_blake3: blake3_hex(target),
|
||||||
|
source_size: source.len() as u64,
|
||||||
|
target_size: target.len() as u64,
|
||||||
|
}],
|
||||||
|
rollback: PatchRollback {
|
||||||
|
previous_current_target: None,
|
||||||
|
remove_target_path: None,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,305 @@
|
|||||||
|
//! Deterministic UTF-8 text patch support.
|
||||||
|
|
||||||
|
use crate::PatchError;
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
|
||||||
|
/// Current text patch schema version.
|
||||||
|
pub const TEXT_PATCH_VERSION: u32 = 1;
|
||||||
|
|
||||||
|
/// UTF-8 text patch made of source-relative replacement ranges.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct TextPatch {
|
||||||
|
/// Patch schema version.
|
||||||
|
pub version: u32,
|
||||||
|
/// Expected BLAKE3 hash of the source UTF-8 bytes.
|
||||||
|
pub source_blake3: String,
|
||||||
|
/// Expected BLAKE3 hash of the target UTF-8 bytes.
|
||||||
|
pub target_blake3: String,
|
||||||
|
/// Source byte length.
|
||||||
|
pub source_size: u64,
|
||||||
|
/// Target byte length.
|
||||||
|
pub target_size: u64,
|
||||||
|
/// Ordered source-relative operations.
|
||||||
|
pub operations: Vec<TextPatchOperation>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One source-relative text patch operation.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
#[serde(tag = "kind", rename_all = "snake_case")]
|
||||||
|
pub enum TextPatchOperation {
|
||||||
|
/// Replaces a UTF-8 byte range in the original source text.
|
||||||
|
ReplaceRange {
|
||||||
|
/// Byte offset in the original source text.
|
||||||
|
offset: u64,
|
||||||
|
/// Number of source bytes to replace.
|
||||||
|
length: u64,
|
||||||
|
/// Optional text that must exactly match the source range.
|
||||||
|
expected: Option<String>,
|
||||||
|
/// Replacement text.
|
||||||
|
replacement: String,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Builds a deterministic one-hunk text patch from source and target text.
|
||||||
|
pub fn diff(source: &str, target: &str) -> TextPatch {
|
||||||
|
if source == target {
|
||||||
|
return TextPatch {
|
||||||
|
version: TEXT_PATCH_VERSION,
|
||||||
|
source_blake3: blake3_hex(source.as_bytes()),
|
||||||
|
target_blake3: blake3_hex(target.as_bytes()),
|
||||||
|
source_size: source.len() as u64,
|
||||||
|
target_size: target.len() as u64,
|
||||||
|
operations: Vec::new(),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
let prefix = common_prefix_boundary(source, target);
|
||||||
|
let (source_suffix, target_suffix) = common_suffix_boundaries(source, target, prefix);
|
||||||
|
let operation = TextPatchOperation::ReplaceRange {
|
||||||
|
offset: prefix as u64,
|
||||||
|
length: (source_suffix - prefix) as u64,
|
||||||
|
expected: Some(source[prefix..source_suffix].to_string()),
|
||||||
|
replacement: target[prefix..target_suffix].to_string(),
|
||||||
|
};
|
||||||
|
TextPatch {
|
||||||
|
version: TEXT_PATCH_VERSION,
|
||||||
|
source_blake3: blake3_hex(source.as_bytes()),
|
||||||
|
target_blake3: blake3_hex(target.as_bytes()),
|
||||||
|
source_size: source.len() as u64,
|
||||||
|
target_size: target.len() as u64,
|
||||||
|
operations: vec![operation],
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Builds a text patch from caller-provided source-relative operations.
|
||||||
|
pub fn from_operations(
|
||||||
|
source: &str,
|
||||||
|
operations: Vec<TextPatchOperation>,
|
||||||
|
) -> crate::Result<TextPatch> {
|
||||||
|
let target = apply_operations(source, &operations)?;
|
||||||
|
Ok(TextPatch {
|
||||||
|
version: TEXT_PATCH_VERSION,
|
||||||
|
source_blake3: blake3_hex(source.as_bytes()),
|
||||||
|
target_blake3: blake3_hex(target.as_bytes()),
|
||||||
|
source_size: source.len() as u64,
|
||||||
|
target_size: target.len() as u64,
|
||||||
|
operations,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Applies a structured text patch.
|
||||||
|
pub fn apply_text_patch(source: &str, patch: &TextPatch) -> crate::Result<String> {
|
||||||
|
if patch.version != TEXT_PATCH_VERSION {
|
||||||
|
return Err(PatchError::ApplyFailed(format!(
|
||||||
|
"unsupported text patch version {}",
|
||||||
|
patch.version
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
if patch.source_size != source.len() as u64
|
||||||
|
|| patch.source_blake3 != blake3_hex(source.as_bytes())
|
||||||
|
{
|
||||||
|
return Err(PatchError::ApplyFailed(
|
||||||
|
"text patch source integrity mismatch".to_string(),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
let output = apply_operations(source, &patch.operations)?;
|
||||||
|
if output.len() as u64 != patch.target_size
|
||||||
|
|| patch.target_blake3 != blake3_hex(output.as_bytes())
|
||||||
|
{
|
||||||
|
return Err(PatchError::ApplyFailed(
|
||||||
|
"text patch target integrity mismatch".to_string(),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
Ok(output)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Parses and applies a JSON-encoded text patch to a UTF-8 string.
|
||||||
|
pub fn apply_patch(source: &str, patch: &str) -> crate::Result<String> {
|
||||||
|
let patch: TextPatch = serde_json::from_str(patch)
|
||||||
|
.map_err(|error| PatchError::ApplyFailed(format!("invalid text patch JSON: {error}")))?;
|
||||||
|
apply_text_patch(source, &patch)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Parses and applies a JSON-encoded text patch to UTF-8 bytes.
|
||||||
|
pub fn apply_patch_bytes(source: &[u8], patch: &[u8]) -> crate::Result<Vec<u8>> {
|
||||||
|
let source = std::str::from_utf8(source)
|
||||||
|
.map_err(|error| PatchError::ApplyFailed(format!("source is not UTF-8: {error}")))?;
|
||||||
|
let patch = std::str::from_utf8(patch)
|
||||||
|
.map_err(|error| PatchError::ApplyFailed(format!("patch is not UTF-8: {error}")))?;
|
||||||
|
Ok(apply_patch(source, patch)?.into_bytes())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn apply_operations(source: &str, operations: &[TextPatchOperation]) -> crate::Result<String> {
|
||||||
|
let mut output = String::with_capacity(source.len());
|
||||||
|
let mut cursor = 0usize;
|
||||||
|
for operation in operations {
|
||||||
|
let (offset, length, expected, replacement) = match operation {
|
||||||
|
TextPatchOperation::ReplaceRange {
|
||||||
|
offset,
|
||||||
|
length,
|
||||||
|
expected,
|
||||||
|
replacement,
|
||||||
|
} => (*offset, *length, expected, replacement),
|
||||||
|
};
|
||||||
|
let start = usize::try_from(offset)
|
||||||
|
.map_err(|_| PatchError::ApplyFailed("text patch offset overflow".to_string()))?;
|
||||||
|
let length = usize::try_from(length)
|
||||||
|
.map_err(|_| PatchError::ApplyFailed("text patch length overflow".to_string()))?;
|
||||||
|
if start < cursor {
|
||||||
|
return Err(PatchError::ApplyFailed(format!(
|
||||||
|
"text patch operation at {start} overlaps previous range ending at {cursor}"
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
let end = start
|
||||||
|
.checked_add(length)
|
||||||
|
.ok_or_else(|| PatchError::ApplyFailed("text patch range overflow".to_string()))?;
|
||||||
|
let replaced = source.get(start..end).ok_or_else(|| {
|
||||||
|
PatchError::ApplyFailed(format!(
|
||||||
|
"text patch range {start}..{end} is outside the source or not UTF-8 aligned"
|
||||||
|
))
|
||||||
|
})?;
|
||||||
|
if let Some(expected) = expected {
|
||||||
|
if replaced != expected {
|
||||||
|
return Err(PatchError::ApplyFailed(format!(
|
||||||
|
"text patch expected mismatch at {start}..{end}"
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
output.push_str(&source[cursor..start]);
|
||||||
|
output.push_str(replacement);
|
||||||
|
cursor = end;
|
||||||
|
}
|
||||||
|
output.push_str(&source[cursor..]);
|
||||||
|
Ok(output)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn common_prefix_boundary(source: &str, target: &str) -> usize {
|
||||||
|
let mut prefix = 0usize;
|
||||||
|
for ((source_index, source_char), (target_index, target_char)) in
|
||||||
|
source.char_indices().zip(target.char_indices())
|
||||||
|
{
|
||||||
|
if source_index != target_index || source_char != target_char {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
prefix = source_index + source_char.len_utf8();
|
||||||
|
}
|
||||||
|
prefix
|
||||||
|
}
|
||||||
|
|
||||||
|
fn common_suffix_boundaries(source: &str, target: &str, prefix: usize) -> (usize, usize) {
|
||||||
|
let mut source_suffix = source.len();
|
||||||
|
let mut target_suffix = target.len();
|
||||||
|
let mut source_chars = source[prefix..].char_indices().rev();
|
||||||
|
let mut target_chars = target[prefix..].char_indices().rev();
|
||||||
|
while let (Some((source_index, source_char)), Some((target_index, target_char))) =
|
||||||
|
(source_chars.next(), target_chars.next())
|
||||||
|
{
|
||||||
|
if source_char != target_char {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
source_suffix = prefix + source_index;
|
||||||
|
target_suffix = prefix + target_index;
|
||||||
|
}
|
||||||
|
(source_suffix, target_suffix)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn blake3_hex(bytes: &[u8]) -> String {
|
||||||
|
blake3::hash(bytes).to_hex().to_string()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn text_patch_round_trips_unicode_change() {
|
||||||
|
let source = "先生、こんにちは\nAbydos";
|
||||||
|
let target = "老师、你好\nAbydos";
|
||||||
|
let patch = diff(source, target);
|
||||||
|
let patch_json = serde_json::to_string(&patch).unwrap();
|
||||||
|
|
||||||
|
assert_eq!(apply_text_patch(source, &patch).unwrap(), target);
|
||||||
|
assert_eq!(apply_patch(source, &patch_json).unwrap(), target);
|
||||||
|
assert_eq!(
|
||||||
|
apply_patch_bytes(source.as_bytes(), patch_json.as_bytes()).unwrap(),
|
||||||
|
target.as_bytes()
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn text_patch_applies_multiple_source_relative_ranges() {
|
||||||
|
let source = "alpha beta gamma";
|
||||||
|
let patch = from_operations(
|
||||||
|
source,
|
||||||
|
vec![
|
||||||
|
TextPatchOperation::ReplaceRange {
|
||||||
|
offset: 0,
|
||||||
|
length: 5,
|
||||||
|
expected: Some("alpha".to_string()),
|
||||||
|
replacement: "one".to_string(),
|
||||||
|
},
|
||||||
|
TextPatchOperation::ReplaceRange {
|
||||||
|
offset: 11,
|
||||||
|
length: 5,
|
||||||
|
expected: Some("gamma".to_string()),
|
||||||
|
replacement: "three".to_string(),
|
||||||
|
},
|
||||||
|
],
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
assert_eq!(apply_text_patch(source, &patch).unwrap(), "one beta three");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn text_patch_rejects_expected_mismatch() {
|
||||||
|
let source = "alpha beta";
|
||||||
|
let operation = TextPatchOperation::ReplaceRange {
|
||||||
|
offset: 0,
|
||||||
|
length: 5,
|
||||||
|
expected: Some("wrong".to_string()),
|
||||||
|
replacement: "one".to_string(),
|
||||||
|
};
|
||||||
|
|
||||||
|
let error = from_operations(source, vec![operation]).unwrap_err();
|
||||||
|
|
||||||
|
assert!(matches!(error, PatchError::ApplyFailed(_)));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn text_patch_rejects_overlapping_ranges() {
|
||||||
|
let source = "alpha beta";
|
||||||
|
let operation_a = TextPatchOperation::ReplaceRange {
|
||||||
|
offset: 0,
|
||||||
|
length: 5,
|
||||||
|
expected: None,
|
||||||
|
replacement: "one".to_string(),
|
||||||
|
};
|
||||||
|
let operation_b = TextPatchOperation::ReplaceRange {
|
||||||
|
offset: 3,
|
||||||
|
length: 2,
|
||||||
|
expected: None,
|
||||||
|
replacement: "two".to_string(),
|
||||||
|
};
|
||||||
|
|
||||||
|
let error = from_operations(source, vec![operation_a, operation_b]).unwrap_err();
|
||||||
|
|
||||||
|
assert!(matches!(error, PatchError::ApplyFailed(_)));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn text_patch_rejects_non_boundary_range() {
|
||||||
|
let source = "éclair";
|
||||||
|
let operation = TextPatchOperation::ReplaceRange {
|
||||||
|
offset: 1,
|
||||||
|
length: 1,
|
||||||
|
expected: None,
|
||||||
|
replacement: "e".to_string(),
|
||||||
|
};
|
||||||
|
|
||||||
|
let error = from_operations(source, vec![operation]).unwrap_err();
|
||||||
|
|
||||||
|
assert!(matches!(error, PatchError::ApplyFailed(_)));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,41 @@
|
|||||||
|
# Optional overrides for bluearchive-toolkit-bat-api.service.
|
||||||
|
#
|
||||||
|
# Install as:
|
||||||
|
# sudo install -o root -g root -m 0644 deployments/systemd/bat-api.env.example /etc/bluearchive-toolkit/bat-api.env
|
||||||
|
#
|
||||||
|
# Production contract:
|
||||||
|
# - bat-api runs in the same server/container environment as Rust bat.
|
||||||
|
# - The current resource_root comes from bat.sock RPC.
|
||||||
|
# - Do not set BAT_API_RESOURCE_ROOT in production; it is only for local
|
||||||
|
# fixtures or emergency read-only diagnostics when RPC is unavailable.
|
||||||
|
# - Publish HTTP through a reverse proxy/TLS if exposed publicly; never expose
|
||||||
|
# bat.sock outside the host.
|
||||||
|
# - Player-facing deployments should set BAT_API_AUTH_TOKEN through a secret
|
||||||
|
# manager or process environment, not in a committed file.
|
||||||
|
|
||||||
|
BAT_API_LISTEN=127.0.0.1:18080
|
||||||
|
BAT_API_PUBLIC_BASE_URL=http://127.0.0.1:18080
|
||||||
|
BAT_API_STATE_DIR=/var/lib/bluearchive-toolkit/daemon-state
|
||||||
|
BAT_API_SOCKET=/var/lib/bluearchive-toolkit/daemon-state/bat.sock
|
||||||
|
BAT_API_REQUIRE_INDEXED=true
|
||||||
|
BAT_API_VERIFY_SIZE=true
|
||||||
|
BAT_API_RPC_TIMEOUT=30s
|
||||||
|
BAT_API_REFRESH_INTERVAL=1m
|
||||||
|
BAT_API_SKIP_ENV_FILE=1
|
||||||
|
BAT_API_AUTH_QUERY_PARAM=bat_token
|
||||||
|
# BAT_API_AUTH_TOKEN=
|
||||||
|
# BAT_API_AUTH_EXEMPT_PATHS=/healthz,/readyz
|
||||||
|
BAT_API_TRUST_PROXY_HEADERS=false
|
||||||
|
BAT_API_ACCESS_LOG=true
|
||||||
|
BAT_API_RATE_LIMIT_RPS=30
|
||||||
|
BAT_API_RATE_LIMIT_BURST=120
|
||||||
|
BAT_API_MAX_RESOURCE_LIMIT=1000
|
||||||
|
|
||||||
|
# Local fixture / emergency only:
|
||||||
|
# BAT_API_RESOURCE_ROOT=/var/lib/bluearchive-toolkit/official/current
|
||||||
|
|
||||||
|
# Reserved for future API persistence:
|
||||||
|
# BAT_API_DATABASE_URL=postgres://bat:@127.0.0.1:5432/bat?sslmode=disable
|
||||||
|
# BAT_API_DATABASE_PASSWORD=
|
||||||
|
# BAT_API_REDIS_URL=redis://127.0.0.1:6379/0
|
||||||
|
# BAT_API_REDIS_PASSWORD=
|
||||||
@@ -0,0 +1,43 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=BlueArchiveToolkit bat-api resource bootstrap and distribution
|
||||||
|
Documentation=https://github.com/Yuyi-Oak/BlueArchiveToolkit
|
||||||
|
Wants=network-online.target
|
||||||
|
After=network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
User=bat
|
||||||
|
Group=bat
|
||||||
|
WorkingDirectory=/var/lib/bluearchive-toolkit
|
||||||
|
Environment=BAT_API_LISTEN=127.0.0.1:18080
|
||||||
|
Environment=BAT_API_PUBLIC_BASE_URL=http://127.0.0.1:18080
|
||||||
|
Environment=BAT_API_STATE_DIR=/var/lib/bluearchive-toolkit/daemon-state
|
||||||
|
Environment=BAT_API_SOCKET=/var/lib/bluearchive-toolkit/daemon-state/bat.sock
|
||||||
|
Environment=BAT_API_REQUIRE_INDEXED=true
|
||||||
|
Environment=BAT_API_VERIFY_SIZE=true
|
||||||
|
Environment=BAT_API_RPC_TIMEOUT=30s
|
||||||
|
Environment=BAT_API_REFRESH_INTERVAL=1m
|
||||||
|
Environment=BAT_API_SKIP_ENV_FILE=1
|
||||||
|
EnvironmentFile=-/etc/bluearchive-toolkit/bat-api.env
|
||||||
|
ExecStart=/opt/bluearchive-toolkit/bin/bat-api
|
||||||
|
Restart=on-failure
|
||||||
|
RestartSec=10
|
||||||
|
TimeoutStopSec=30
|
||||||
|
KillSignal=SIGTERM
|
||||||
|
StandardOutput=journal
|
||||||
|
StandardError=journal
|
||||||
|
RuntimeDirectory=bluearchive-toolkit-bat-api
|
||||||
|
RuntimeDirectoryMode=0750
|
||||||
|
LogsDirectory=bluearchive-toolkit
|
||||||
|
LogsDirectoryMode=0750
|
||||||
|
NoNewPrivileges=true
|
||||||
|
PrivateTmp=true
|
||||||
|
ProtectHome=true
|
||||||
|
ProtectSystem=strict
|
||||||
|
ReadOnlyPaths=/var/lib/bluearchive-toolkit
|
||||||
|
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
|
||||||
|
LockPersonality=true
|
||||||
|
MemoryDenyWriteExecute=true
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
@@ -10,10 +10,11 @@ User=bat
|
|||||||
Group=bat
|
Group=bat
|
||||||
WorkingDirectory=/var/lib/bluearchive-toolkit
|
WorkingDirectory=/var/lib/bluearchive-toolkit
|
||||||
Environment=BAT_OUTPUT_ROOT=/var/lib/bluearchive-toolkit/official
|
Environment=BAT_OUTPUT_ROOT=/var/lib/bluearchive-toolkit/official
|
||||||
|
Environment=BAT_LOCALIZED_OUTPUT_ROOT=/var/lib/bluearchive-toolkit/localized
|
||||||
Environment=BAT_INTERVAL=1h
|
Environment=BAT_INTERVAL=1h
|
||||||
Environment=BAT_ERROR_RETRY=60s
|
Environment=BAT_ERROR_RETRY=60s
|
||||||
EnvironmentFile=-/etc/bluearchive-toolkit/official-sync.env
|
EnvironmentFile=-/etc/bluearchive-toolkit/official-sync.env
|
||||||
ExecStart=/opt/bluearchive-toolkit/bin/bat --auto-discover --output ${BAT_OUTPUT_ROOT} --watch --interval ${BAT_INTERVAL} --error-retry ${BAT_ERROR_RETRY} --no-banner
|
ExecStart=/opt/bluearchive-toolkit/bin/bat --auto-discover --output ${BAT_OUTPUT_ROOT} --localized-output ${BAT_LOCALIZED_OUTPUT_ROOT} --watch --interval ${BAT_INTERVAL} --error-retry ${BAT_ERROR_RETRY} --no-banner
|
||||||
Restart=on-failure
|
Restart=on-failure
|
||||||
RestartSec=30
|
RestartSec=30
|
||||||
TimeoutStopSec=60
|
TimeoutStopSec=60
|
||||||
|
|||||||
@@ -4,9 +4,11 @@
|
|||||||
# sudo install -o root -g root -m 0644 deployments/systemd/official-sync.env.example /etc/bluearchive-toolkit/official-sync.env
|
# sudo install -o root -g root -m 0644 deployments/systemd/official-sync.env.example /etc/bluearchive-toolkit/official-sync.env
|
||||||
#
|
#
|
||||||
# Paths are intentionally independent from any official launcher or game client
|
# Paths are intentionally independent from any official launcher or game client
|
||||||
# install directory. Do not point BAT_OUTPUT_ROOT at an existing game directory.
|
# install directory. Do not point either root at an existing game directory, and
|
||||||
|
# keep the official and localized roots separate.
|
||||||
|
|
||||||
BAT_OUTPUT_ROOT=/var/lib/bluearchive-toolkit/official
|
BAT_OUTPUT_ROOT=/var/lib/bluearchive-toolkit/official
|
||||||
|
BAT_LOCALIZED_OUTPUT_ROOT=/var/lib/bluearchive-toolkit/localized
|
||||||
BAT_INTERVAL=1h
|
BAT_INTERVAL=1h
|
||||||
BAT_ERROR_RETRY=60s
|
BAT_ERROR_RETRY=60s
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,206 @@
|
|||||||
|
# AssetBundle 与资源解析路线图
|
||||||
|
|
||||||
|
- **更新时间**:2026-07-26
|
||||||
|
- **适用范围**:Rust 解析引擎、官方同步后的解析缓存、CAS/ResourceRepository 接入、后续文本提取和 Patch 发布。
|
||||||
|
- **权威关联**:`PROJECT_PLAN.md` Milestone 3/4/5/8,`docs/reports/CURRENT_GAPS.md` G-005/G-007/G-011/G-011D。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. 目标边界
|
||||||
|
|
||||||
|
解析系统的目标不是把下载流程写成一次性脚本,而是建立可长期维护的资源理解层:
|
||||||
|
|
||||||
|
1. 官方资源同步负责拉取、校验和发布原版资源。
|
||||||
|
2. 解析器只读取已发布或 staging 中已校验的资源,不修改原始文件。
|
||||||
|
3. 解析结果写入派生缓存、CAS 索引或后续文本提取索引。
|
||||||
|
4. 汉化产物只能由 Patch/发布阶段写入 `--localized-output` / `BAT_LOCALIZED_OUTPUT` 指定的汉化发布根,不能写回官方资源目录。
|
||||||
|
5. 解析器必须与 CLI、daemon、Go API、Patch 业务流程解耦。
|
||||||
|
|
||||||
|
当前官方同步在新 release 发布后会先维护 `official-resource-changes.json` 和 `crowdin-translation-handoff.json`,再维护 `official-parse-cache.json`、`official-textunit-index.json`、`official-textunit-tasks.json` 和 `crowdin-textunit-queue.json`。这些都是官方 release 的派生索引,不是汉化产物;up-to-date 轮询在已有有效解析缓存、TextUnit 明细索引和 TextUnit 队列时只读取摘要,不重复解析。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. 分层模型
|
||||||
|
|
||||||
|
解析能力按从外到内分层:
|
||||||
|
|
||||||
|
| 层级 | 输入 | 输出 | 当前状态 |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| 官方 seed manifest | `TableCatalog.bytes`、`BundlePackingInfo.bytes`、`MediaCatalog.bytes` | 完整下载 URL、相对路径、hash 校验边界 | 已用于下载计划,仍需沉淀更多结构化字段 |
|
||||||
|
| Addressables catalog | `catalog_*.zip` 内 JSON/bin catalog、`catalog_*.hash` | asset path、provider、dependencies、size、CRC、bundle name | JSON/compact 当前样本已覆盖,仍需更多真实结构变体 |
|
||||||
|
| UnityFS container | `.bundle`、zip 内 bundle | header、block、directory、解压文件、基础摘要 | 已支持基础解包、LZ4/LZMA、边界校验 |
|
||||||
|
| Serialized file | UnityFS directory 文件 | header、type table、TypeTree node、object table、TextAsset bytes | 已支持基础表结构和 TextAsset bytes |
|
||||||
|
| Unity 对象字段 | TextAsset、MonoBehaviour、ScriptableObject | 可翻译文本单元、上下文、资源定位 | TypeTree 基础字段读取、`SerializedReference` / prefixed managed-reference metadata alias、payload 提取和字符串提取已落地,真实结构覆盖继续扩大 |
|
||||||
|
| Patch 发布前解析 | 已翻译文本、中间格式、原版资源 | 可验证 patch manifest、汉化 release 目录 | UnityFS TextAsset 前置已落地,通用 Binary/JSON/Text Patch 与文件级 patch / UnityFS 写入入口可用,发布级 build/rollback 未开放 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. 当前已落地能力
|
||||||
|
|
||||||
|
`crates/bat-assetbundle` 已经承担解析核心:
|
||||||
|
|
||||||
|
1. `UnityFsParser` 解析 UnityFS header、block info、directory。
|
||||||
|
2. 支持 LZ4/LZMA block info 和数据 block 解压。
|
||||||
|
3. 支持 block info at end 和官方样本中出现的 block data alignment。
|
||||||
|
4. 能从 UnityFS directory 提取文件 bytes。
|
||||||
|
5. `serialized` 模块能读取 Unity serialized file header、type table、TypeTree node 元数据、object table。
|
||||||
|
6. 能提取 TextAsset 的 name 和原始 bytes。
|
||||||
|
7. TypeTree field reader 已支持基础标量、string、bytes、array、vector/staticvector 嵌套 `Array` 形态、`List<T>` / `HashSet<T>` 集合 alias、map、PPtr、enum `value__` backing field、`LayerMask` / `BitField` 的 `m_Bits` backing field、嵌套对象、常见固定 Unity float/int/hash 值类型的 leaf 和 direct child TypeTree 形态、unknown fixed-size raw bytes 保留和同长度替换、TypeTree-covered managed reference、TypeTree-covered managed reference registry 记录、`m_ManagedReferences` / `RefIds` / `m_RefIds` / verbose type 字段等 registry 命名变体、`id` / `typeInfo` 等 metadata 命名变体、`data` / `value` / `payload` / `object` / `managedReferencePayload` / `referencePayload` / `serializedReferencePayload` / `managedReferenceValue` / `referenceValue` / `serializedReferenceValue` / `managedReferenceObject` / `referenceObject` / `serializedReferenceObject` / `managedReferenceData` / `referenceData` / `serializedData` 等 payload 命名变体、managed-reference full typename 拆解和字段 offset/size 诊断。
|
||||||
|
8. `TextUnitExtractor` 已把 JSON/CSV/TSV/plain TextAsset、TypeTree 字符串字段和 TypeTree-covered managed reference payload 字符串输出为可序列化 TextUnit/JSONL;zip 场景保留 archive entry,TextUnit 明细包含 serialized file、path id、class id、field path、字段 offset/byte size、format、asset name 和上下文。managed-reference 类型元数据保留为 payload context,不进入翻译文本队列;即使 registry 暂时只能走 fallback 字段遍历,`RefIds`、`className`、`namespaceName`、`asmName` 等元数据别名也会被跳过,payload/value/object 家族和 `managedReferenceData` / `referenceData` / `serializedData` 仍按 payload 处理,并按 `RefIds[n]` 等记录前缀或子字段推导 metadata,避免多条 fallback record 混用 managed-reference context。
|
||||||
|
9. `ResourceImportService` 能把 AssetBundle 摘要、TextAsset/Table/Media 分类和 TextUnit 摘要写入导入报告。
|
||||||
|
10. 官方同步后 `OfficialParseCacheService` 能从 `official-download-manifest.json` 遍历所有资源,解析直接 bundle 和 zip 内条目,非候选资源记录为 unsupported,并缓存 TextUnit 数量/格式/诊断摘要,同时写出 `official-textunit-index.json` 供 `parse.text_units` / `parse.errors` 查询。
|
||||||
|
|
||||||
|
当前还不能宣称完整:
|
||||||
|
|
||||||
|
1. TypeTree-covered managed reference 字段和 registry 记录已可结构化解码并参与文本提取,常见 registry 命名别名(含 `m_ManagedReferences`、`RefIds`、`m_RefIds`、verbose type 字段)、metadata 命名别名(含 `id`、`typeInfo`)、payload 命名别名(含 `data`、`value`、`payload`、`object`、`managedReferencePayload`、`referencePayload`、`serializedReferencePayload`、`managedReferenceValue`、`referenceValue`、`serializedReferenceValue`、`managedReferenceObject`、`referenceObject`、`serializedReferenceObject`、`managedReferenceData`、`referenceData`、`serializedData`)、full typename 拆解和 payload-only TextUnit 提取已有回归覆盖,多记录 registry 聚合也已有单元回归;fallback 字段遍历会跳过常见 registry 元数据字符串,避免误入翻译队列,并按记录前缀或子字段可推导 metadata 保留 managed-reference TextUnit context。enum `value__` backing field 和 `LayerMask` / `BitField` 的 `m_Bits` backing field 已可语义化解码和替换;`Vector2f/3f/4f`、`Quaternionf`、`ColorRGBA`、`Rectf`、`AABB/Bounds/Ray`、`Matrix4x4f`、`Vector2Int/Vector3Int`、`RectInt`、`BoundsInt`、`RangeInt`、`GUID`、`Hash128` 等固定 Unity 值类型的 leaf 和 direct child TypeTree 形态已可结构化解码和语义替换;array/vector/staticvector/List/HashSet/map 元素与 registry payload 字段已保留独立 field path、offset 和 byte size,可用于字符串元素 patch,managed-reference registry payload 字符串、enum、bit_field、unknown fixed-size raw bytes、object 字段组合和 TypeTree schema 支撑的 array/vector/List/HashSet/map 已可整体变长替换,`first/second` 与 `key/value` map entry schema 已有 serialized 和 UnityFS 重建回归,ScriptableObject `key/value` map 解析、变长替换和 UnityFS 重建已有专门回归,且嵌套 vector `Array`、`List<T>` / `HashSet<T>` 集合 alias、enum、bit_field、unknown fixed-size raw bytes 与 managed-reference payload 字段已有重建回归覆盖;解析模块当前处于维护冻结,未见样本驱动的完整 managed reference registry / map entry 变体、unknown 字段结构语义和版本差异冻结后再推进。
|
||||||
|
2. Addressables bin/compact catalog 结构变体仍需真实样本驱动补齐。
|
||||||
|
3. 官方 release 已可配置导入 CAS + ResourceRepository,并可通过 `resource.index` 查询现有资源索引;Resource metadata 已记录 release、平台、bundle path、parse status、TextAsset 名称和 TextUnit 摘要。
|
||||||
|
4. 不能完成复杂对象字段重打包,也不能从真实 Crowdin 结果自动生成完整汉化文件集合。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. 补全顺序
|
||||||
|
|
||||||
|
### P0:解析缓存和样本闭环
|
||||||
|
|
||||||
|
目标:让官方同步后的解析结果可复用、可诊断、可回归。
|
||||||
|
|
||||||
|
交付:
|
||||||
|
|
||||||
|
1. `official-resource-changes.json` 记录当前 release 相对上一完整 release 的新增、变更、删除资源,以及解析/翻译候选计数。
|
||||||
|
2. `crowdin-translation-handoff.json` 只包含新增+变更资源,作为后续 Crowdin worker 的本地队列输入;当前解析阶段不直接调用 Crowdin API。
|
||||||
|
3. `official-parse-cache.json` 记录 manifest entry、zip entry、解析状态、Unity 版本、文件数、TextAsset 数、TextUnit 数/格式、错误摘要和缓存复用状态。
|
||||||
|
4. `official-textunit-index.json` 持久化单条 TextUnit 和解析错误,保留 destination、archive entry、serialized file、path id、class id、field path、offset 和 format 等定位信息。
|
||||||
|
5. `official-textunit-tasks.json` 只从 Added/Modified 资源、parse cache 和 TextUnit 明细索引派生,记录可翻译 TextUnit 任务和跳过原因。
|
||||||
|
6. `crowdin-textunit-queue.json` 只包含已经产生 TextUnit 的离线任务,当前不调用 Crowdin 网络 API。
|
||||||
|
7. 解析直接 `.bundle` / `.unity3d` 和 zip 内全部文件条目,不能只假设 `FullPatch_*.zip`。
|
||||||
|
8. 非候选资源记录为 unsupported,不影响官方同步发布。
|
||||||
|
9. 缺失、损坏或无法解析的 bundle 记录 failed,但不回滚已经完成校验的官方原版 release。
|
||||||
|
10. 用合成 fixture、隔离真实样本和回归 fixture 覆盖资源变更集、Crowdin handoff、TextUnit 队列、缓存复用、zip 内条目、非候选资源、解析失败。
|
||||||
|
|
||||||
|
验收:
|
||||||
|
|
||||||
|
1. 新 release 发布时能生成资源变更集,新增+变更资源进入解析/翻译候选,删除资源不进入翻译队列。
|
||||||
|
2. 第二次 up-to-date 轮询不会重复解析已有有效缓存和 TextUnit 明细索引。
|
||||||
|
3. 修改任意 manifest entry 的 size/BLAKE3 后,变更集能标记对应资源并让后续解析/翻译只消费候选。
|
||||||
|
4. 解析缓存、handoff 和 TextUnit 队列不会写入汉化发布根。
|
||||||
|
|
||||||
|
### P1:Addressables catalog 完整化
|
||||||
|
|
||||||
|
目标:把“能列出资源”推进到“能稳定定位 bundle、依赖、校验字段和资源类型”。
|
||||||
|
|
||||||
|
交付:
|
||||||
|
|
||||||
|
1. 覆盖 JSON catalog、compact JSON、可能的二进制 catalog 入口。
|
||||||
|
2. 解析 provider id、internal id、primary key、dependency key、resource type、bundle name、hash、size、CRC。
|
||||||
|
3. 明确 `catalog_*.hash` 只作为 Addressables remote catalog marker,不套用 seed `.hash` 的 xxHash32 规则。
|
||||||
|
4. 将 Windows/Android catalog 样本拆成可复现 fixture,不把大文件纳入 Git。
|
||||||
|
5. 对未知结构返回明确错误或保真 raw metadata,不静默丢字段。
|
||||||
|
|
||||||
|
验收:
|
||||||
|
|
||||||
|
1. 当前目标版本 Windows/Android catalog 样本集合解析通过。
|
||||||
|
2. 解析结果能反查 bundle 文件和依赖链。
|
||||||
|
3. size/CRC/hash 字段能参与本地文件验证或至少进入诊断报告。
|
||||||
|
|
||||||
|
### P2:Unity Serialized 字段级解析
|
||||||
|
|
||||||
|
目标:把 Unity object table 推进到可提取文本字段。
|
||||||
|
|
||||||
|
交付:
|
||||||
|
|
||||||
|
1. TypeTree schema 内部表示稳定化:node path、type、name、size、flags、array 信息。
|
||||||
|
2. 基础字段 reader 已支持 bool、integer、float、string、bytes、array、vector/staticvector 嵌套 `Array`、`List<T>` / `HashSet<T>` 集合 alias、map、PPtr、enum `value__` backing field、`LayerMask` / `BitField` 的 `m_Bits` backing field、常见固定 Unity 值类型的 leaf/direct-child 形态,以及 unknown fixed-size raw bytes 保留和同长度替换。
|
||||||
|
3. TextAsset 已有专用 name/bytes 读取,避免和字段级遍历重复报错。
|
||||||
|
4. MonoBehaviour 和 ScriptableObject 的 TypeTree 字段遍历入口已落地,复杂版本差异继续补 fixture。
|
||||||
|
5. 对缺 TypeTree 或 stripped 类型返回可诊断结果,保留 raw object bytes 作为后备。
|
||||||
|
|
||||||
|
验收:
|
||||||
|
|
||||||
|
1. 合成 fixture 覆盖标量、数组、嵌套结构、string alignment。
|
||||||
|
2. 隔离真实样本能输出稳定 JSON field tree。
|
||||||
|
3. 解析错误包含 file path、object path id、class id、字段路径和偏移。
|
||||||
|
|
||||||
|
### P3:文本提取中间层
|
||||||
|
|
||||||
|
目标:为日语汉化提供稳定、可回写定位的文本单元。
|
||||||
|
|
||||||
|
交付:
|
||||||
|
|
||||||
|
1. 已定义 `TextUnit`:source text、bundle path、archive entry、serialized file、object path id、class id、field path、字段 offset/byte size、版本和上下文;managed-reference payload 会额外写入 reference id、full type name、assembly、namespace 和 class 上下文。
|
||||||
|
2. TextAsset 已支持 JSON/CSV/TSV/plain text 探测,二进制 payload 单独计数。
|
||||||
|
3. MonoBehaviour/ScriptableObject 已按字段路径提取字符串。
|
||||||
|
4. 保留重复文本和上下文,不在解析阶段做会丢定位的合并。
|
||||||
|
5. 已提供 JSONL 第一稳定格式,CSV/XLIFF 可后置。
|
||||||
|
|
||||||
|
验收:
|
||||||
|
|
||||||
|
1. 提取不会修改官方资源。
|
||||||
|
2. 每条文本能追溯回原 bundle、serialized file、path id 和字段路径。
|
||||||
|
3. 同一文本在不同上下文中保持可区分。
|
||||||
|
|
||||||
|
### P4:CAS/Repository 用户级接入
|
||||||
|
|
||||||
|
目标:让解析结果进入可查询资源库,而不是只停留在文件系统缓存。
|
||||||
|
|
||||||
|
交付:
|
||||||
|
|
||||||
|
1. 官方同步完成后可配置触发导入 CAS + ResourceRepository(已具备 `--import-repository` / `BAT_IMPORT_REPOSITORY=1`)。
|
||||||
|
2. ResourceRepository 已保存官方 manifest 资源的类型、路径、hash、size 和 metadata;metadata 包含 release、平台、bundle path、parse status、TextAsset 名称、TextUnit 数量/格式。
|
||||||
|
3. 支持 RPC/CLI 查询资源、bundle、TextAsset、解析错误和缓存状态;当前 `resource.index` 会返回资源 metadata,`parse-status` 会返回 TextUnit 索引和队列摘要,`parse-text-units` / `parse-errors` 会按当前 release 查询明细,`localized-status` 会校验 patch manifest。
|
||||||
|
4. schema 迁移可重复执行;当前 SQLite 已有 `crc` 和 `metadata_json` 兼容迁移。
|
||||||
|
|
||||||
|
验收:
|
||||||
|
|
||||||
|
1. 可以按版本、路径、类型、hash 查询,并在结果 metadata 中看到 TextAsset / TextUnit 摘要。
|
||||||
|
2. 解析缓存、资源变更集和 repository 数据能从同一 manifest fingerprint 追溯。
|
||||||
|
3. CAS 对象跨版本复用,不重复存储相同文件。
|
||||||
|
|
||||||
|
### P5:Patch 发布前置解析
|
||||||
|
|
||||||
|
目标:让解析结果成为可生成汉化 patch 的输入。
|
||||||
|
|
||||||
|
交付:
|
||||||
|
|
||||||
|
1. 已定义 `localized-patch-manifest.json`:目标官方版本、localized release、输出文件、hash、size、byte delta、TextAsset 操作和回滚信息。
|
||||||
|
2. 已支持 UnityFS TextAsset raw bytes 替换的最小 patch 路径。
|
||||||
|
3. MonoBehaviour/ScriptableObject 字段替换必须依赖 P2 字段级解析结果。
|
||||||
|
4. Patch 产物写入配置化汉化发布根下的 `.staging/<id>`,校验通过后发布到 `versions/<id>` 并切换 `current`。
|
||||||
|
5. 成功后发布状态从 `not_localized` 切到 `localized`;`localized.status` 要求 state、current symlink 和 patch manifest 同时匹配当前官方 release。
|
||||||
|
|
||||||
|
验收:
|
||||||
|
|
||||||
|
1. Patch 失败不影响 `bat-resources/current`。
|
||||||
|
2. 汉化 release 保留官方相对目录结构。
|
||||||
|
3. `localized` 状态能证明原版和汉化两套资源都已发布,且 patch manifest 可验证。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. 解析器接口原则
|
||||||
|
|
||||||
|
1. 解析器输入只接受 bytes、逻辑路径和可选上下文,不直接访问下载器状态。
|
||||||
|
2. 解析器输出必须可序列化,供 CLI/RPC/API、缓存和测试 golden 使用。
|
||||||
|
3. 错误必须带位置:URL 或路径、archive entry、UnityFS directory、object path id、field path、offset。
|
||||||
|
4. 未识别结构优先保留 raw metadata,不做低保真猜测。
|
||||||
|
5. 解析器不写 `bat-resources` 和 `bat-localized`,写文件由上层缓存、导入或 Patch 发布流程负责。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Fixture 策略
|
||||||
|
|
||||||
|
1. 合成 fixture 放入代码仓库,覆盖边界和回归。
|
||||||
|
2. 真实小样本可放入仓库前必须确认体积、许可和可复现性。
|
||||||
|
3. 大型真实官方资源只允许放在 `/tmp`、隔离测试目录或用户显式提供的远端测试目录,不纳入 Git。
|
||||||
|
4. 每个新增 fixture 必须说明覆盖的真实风险:字段变体、压缩模式、越界、hash mismatch、zip 内路径、TypeTree 结构等。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. 近期关闭路径
|
||||||
|
|
||||||
|
优先顺序:
|
||||||
|
|
||||||
|
1. 完成 Addressables Windows/Android 当前版本 catalog 样本集合,关闭 G-007 当前阶段。
|
||||||
|
2. 完成 TypeTree 字段 reader 和 MonoBehaviour/ScriptableObject 遍历,推进 G-005。
|
||||||
|
3. 将 `crowdin-textunit-queue.json` 接入真实 Crowdin worker、翻译记忆和 Patch 构建。
|
||||||
|
4. 将翻译任务状态接入 CAS/ResourceRepository 查询面,推进 G-011。
|
||||||
|
5. 在通用 Binary/JSON/Text Patch 基础上继续扩展复杂 AssetBundle 重打包和发布流程统一,保留当前 UnityFS TextAsset patch 发布前置链路。
|
||||||
@@ -6,6 +6,12 @@
|
|||||||
|
|
||||||
这个后端只处理 **日服官方资源**,只接受官方 `.jp/.com` 域名下的资源链路。
|
这个后端只处理 **日服官方资源**,只接受官方 `.jp/.com` 域名下的资源链路。
|
||||||
|
|
||||||
|
**Release 布局、URL→磁盘映射、seed 模板与 bat-api 分发 path 的冻结契约**见:
|
||||||
|
|
||||||
|
- `docs/architecture/resource-release-layout.md`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
明确排除:
|
明确排除:
|
||||||
|
|
||||||
- `bluearchive.cafe`
|
- `bluearchive.cafe`
|
||||||
@@ -31,7 +37,7 @@
|
|||||||
| 清单层 | 解析 `BundlePackingInfo.bytes`、`TableCatalog.bytes`、`MediaCatalog.bytes` | 得到完整文件清单 |
|
| 清单层 | 解析 `BundlePackingInfo.bytes`、`TableCatalog.bytes`、`MediaCatalog.bytes` | 得到完整文件清单 |
|
||||||
| 计划层 | 合并 discovery + inventory,去重并保序 | 得到全量 pull plan |
|
| 计划层 | 合并 discovery + inventory,去重并保序 | 得到全量 pull plan |
|
||||||
| 下载层 | 校验官方 URL,调用下载器,落盘并记录字节数 | 得到本地资源副本 |
|
| 下载层 | 校验官方 URL,调用下载器,落盘并记录字节数 | 得到本地资源副本 |
|
||||||
| 导入层 | 将 bundle 写入 CAS 和 ResourceRepository | 得到可查询的资源索引 |
|
| 导入层 | 可配置将已校验官方 release 写入 CAS 和 ResourceRepository | 得到可查询的资源索引 |
|
||||||
| 同步层 | 比较当前快照和历史快照 | 决定下载、校验、发布 |
|
| 同步层 | 比较当前快照和历史快照 | 决定下载、校验、发布 |
|
||||||
| 更新层 | 保存上次官方 snapshot,定期执行 discovery + diff + pull | 形成自动更新闭环 |
|
| 更新层 | 保存上次官方 snapshot,定期执行 discovery + diff + pull | 形成自动更新闭环 |
|
||||||
|
|
||||||
@@ -48,7 +54,7 @@
|
|||||||
3. 不要求把生产环境当作客户端安装目录。
|
3. 不要求把生产环境当作客户端安装目录。
|
||||||
4. 可以显式执行 official metadata discovery 自动发现 `server-info` URL、`connection-group` 和 `app-version`。
|
4. 可以显式执行 official metadata discovery 自动发现 `server-info` URL、`connection-group` 和 `app-version`。
|
||||||
5. 也可以通过配置、调度状态或已审计 metadata snapshot 显式提供这些值。
|
5. 也可以通过配置、调度状态或已审计 metadata snapshot 显式提供这些值。
|
||||||
6. `--auto-discover` 只允许通过官方 HTTP metadata 和临时目录解析 `GameMainConfig`;launcher metadata 未变时必须复用缓存,metadata 变化时才按 manifest 重新下载必要 `resources.assets` 或旧版官方 game zip。
|
6. `--auto-discover` 只允许通过官方 HTTP metadata 和临时目录解析 `GameMainConfig`;launcher metadata 与 remote manifest 文件列表 digest 均未变时必须复用缓存,任一变化时才按 manifest 重新下载必要 `resources.assets` 或旧版官方 game zip。
|
||||||
|
|
||||||
### 3.1 发现官方资源根
|
### 3.1 发现官方资源根
|
||||||
|
|
||||||
@@ -129,16 +135,17 @@
|
|||||||
6. `TableCatalog.bytes`、`BundlePackingInfo.bytes`、`MediaCatalog.bytes` 总是刷新并用官方 `.hash` 强校验;该 `.hash` 是 `xxHash32(seed=0)` 的十进制文本。
|
6. `TableCatalog.bytes`、`BundlePackingInfo.bytes`、`MediaCatalog.bytes` 总是刷新并用官方 `.hash` 强校验;该 `.hash` 是 `xxHash32(seed=0)` 的十进制文本。
|
||||||
7. `catalog_*.hash` 当前只作为 Addressables catalog 变更标记,不作为 zip/JSON 内容校验算法;Unity Addressables/SBP builder 对 JSON/bin catalog 使用 `HashingMethods.Calculate` 生成 `Hash128` 文本,运行时用它判断 remote catalog cache 是否过期,它不能套用 seed catalog 的 `xxHash32` 规则。
|
7. `catalog_*.hash` 当前只作为 Addressables catalog 变更标记,不作为 zip/JSON 内容校验算法;Unity Addressables/SBP builder 对 JSON/bin catalog 使用 `HashingMethods.Calculate` 生成 `Hash128` 文本,运行时用它判断 remote catalog cache 是否过期,它不能套用 seed catalog 的 `xxHash32` 规则。
|
||||||
8. 官方 seed `.hash` 校验失败会让当前下载失败,并移除对应 data/hash URL 的本地 manifest 条目,避免失败产物在下一轮被本地 BLAKE3 audit 误判为健康缓存。
|
8. 官方 seed `.hash` 校验失败会让当前下载失败,并移除对应 data/hash URL 的本地 manifest 条目,避免失败产物在下一轮被本地 BLAKE3 audit 误判为健康缓存。
|
||||||
9. 存在 `.part` 临时文件时通过 `curl --continue-at -` 尝试断点续传。
|
9. 官方启动器/server-info 先于 client-patch CDN 开放是合法上游状态。若 seed marker 或必需 seed catalog 在进入 staging 前返回 403/404/普通 4xx,更新服务返回 `waiting_for_official_resources` 和 `unavailable_endpoints`,保留现有 `current`,不创建失败 staging,不写入 `failed_versions`;watch/daemon 使用 `waiting` 状态按错误重试间隔继续探测。
|
||||||
10. 新下载写入 `.part`,成功并通过必要校验后原子 rename 到 staging 内最终路径;断点续传后的 `.zip` 如果结构无效,会删除 `.part` 并重新全量下载。
|
10. 存在 `.part` 临时文件时通过 `curl --continue-at -` 尝试断点续传。
|
||||||
11. 成功下载后更新本地下载清单。
|
11. 新下载写入 `.part`,成功并通过必要校验后原子 rename 到 staging 内最终路径;断点续传后的 `.zip` 如果结构无效,会删除 `.part` 并重新全量下载。
|
||||||
12. 上一轮失败或中断留下的 staging 只有在 `official-version-state.json` 中存在同一 app version、bundle version 和 Addressables root 的失败记录,且 `<output>/.staging/<id>` 仍安全存在、`versions/<id>` 尚未发布时才会复用;复用后仍按 manifest、BLAKE3、ZIP 结构和官方 `.hash` 逐 URL 校验,不信任散落文件。
|
12. 成功下载后更新本地下载清单。
|
||||||
13. curl 默认自动检测 `HTTPS_PROXY` / `ALL_PROXY` / `HTTP_PROXY` 及小写环境变量,保留 `NO_PROXY`;带凭据的代理推荐用这些环境变量配置。CLI 也可用 `--proxy <URL>` 显式指定代理,或用 `--no-proxy` 强制直连。代理决策会进入 progress log、daemon log 和 `doctor` 诊断输出。代理凭据全程不落世界可读位置:日志与 `status` 输出脱敏;传给 curl 子进程时经 `ALL_PROXY` 环境变量而非 `--proxy` 参数,不进 curl 的 `/proc/<pid>/cmdline`;`--daemon` 模式下经环境变量下传后台子进程,不进子进程 argv 或 `bat-status.json`,复用凭据单独存于 `bat-proxy.secret`(`0600`),`clean-stable` 会在后台停止后清除。
|
13. 上一轮失败或中断留下的 staging 只有在 `official-version-state.json` 中存在同一 app version、bundle version 和 Addressables root 的失败记录,且 `<output>/.staging/<id>` 仍安全存在、`versions/<id>` 尚未发布时才会复用;复用后仍按 manifest、BLAKE3、ZIP 结构和官方 `.hash` 逐 URL 校验,不信任散落文件。
|
||||||
14. curl 失败按 HTTP/网络类型分类:403/404/普通 4xx 不重试,5xx、429、DNS、连接、超时、中断和网络类错误按尝试次数重试。
|
14. curl 默认自动检测 `HTTPS_PROXY` / `ALL_PROXY` / `HTTP_PROXY` 及小写环境变量,保留 `NO_PROXY`;带凭据的代理推荐用这些环境变量配置。CLI 也可用 `--proxy <URL>` 显式指定代理,或用 `--no-proxy` 强制直连。代理决策会进入 progress log、daemon log 和 `doctor` 诊断输出。代理凭据全程不落世界可读位置:日志与 `status` 输出脱敏;传给 curl 子进程时经 `ALL_PROXY` 环境变量而非 `--proxy` 参数,不进 curl 的 `/proc/<pid>/cmdline`;`--daemon` 模式下经环境变量下传后台子进程,不进子进程 argv 或 `bat-status.json`,复用凭据单独存于 `bat-proxy.secret`(`0600`),`clean-stable` 会在后台停止后清除。
|
||||||
15. 单个 URL 最终失败时写入 `official-download-quarantine.json`,发出 Failed progress,并阻止发布不完整资源。
|
15. curl 失败按 HTTP/网络类型分类:403/404/普通 4xx 不重试,5xx、429、DNS、连接、超时、中断和网络类错误按尝试次数重试。
|
||||||
16. 旧 launcher 包或 `resources.assets` 下载使用官方 launcher CDN 配置,primary CDN 失败后切换 official backup CDN;资源 patch host 不猜测非官方镜像。
|
16. 单个 URL 最终失败时写入 `official-download-quarantine.json`,发出 Failed progress,并阻止发布不完整资源。
|
||||||
17. 记录最终文件大小、本次传输字节数、官方 hash 校验数和执行状态。
|
17. 旧 launcher 包或 `resources.assets` 下载使用官方 launcher CDN 配置,primary CDN 失败后切换 official backup CDN;资源 patch host 不猜测非官方镜像。
|
||||||
18. 非官方 URL 直接拒绝。
|
18. 记录最终文件大小、本次传输字节数、官方 hash 校验数和执行状态。
|
||||||
|
19. 非官方 URL 直接拒绝。
|
||||||
|
|
||||||
路径映射时会做分段清理,并在写入前做输出目录安全校验、相对路径归属校验和现有路径组件 symlink 检查,避免把不安全路径写进输出目录或通过 symlink 跳出输出目录。
|
路径映射时会做分段清理,并在写入前做输出目录安全校验、相对路径归属校验和现有路径组件 symlink 检查,避免把不安全路径写进输出目录或通过 symlink 跳出输出目录。
|
||||||
|
|
||||||
@@ -148,14 +155,24 @@
|
|||||||
|
|
||||||
### 3.5 导入到 CAS 和资源仓储
|
### 3.5 导入到 CAS 和资源仓储
|
||||||
|
|
||||||
当前实现已经提供资源导入能力,但官方同步下载完成后尚未自动作为用户级流程触发导入。手动或上层流程调用导入层时,它会:
|
官方同步下载、校验并发布 release 后,可以通过 `--import-repository` 或
|
||||||
|
`.env` 中 `BAT_IMPORT_REPOSITORY=1` 自动触发 CAS + `ResourceRepository`
|
||||||
|
导入:
|
||||||
|
|
||||||
1. 把 bundle 原始字节写入 CAS。
|
1. 读取已发布 release 下的 `official-download-manifest.json`。
|
||||||
2. 解析 UnityFS 基础摘要。
|
2. 逐条按 manifest 的相对路径、size 和 BLAKE3 重新校验本地文件。
|
||||||
3. 把资源条目写入 `ResourceRepository`。
|
3. 把已校验字节写入 CAS;默认 CAS 根目录是 `<output>/.cas`,也可用
|
||||||
4. 记录资源路径、hash、大小和解析摘要。
|
`--import-cas-root` / `BAT_IMPORT_CAS_ROOT` 覆盖。
|
||||||
|
4. 将资源条目写入 SQLite `ResourceRepository`;默认索引路径是
|
||||||
|
`<output>/resources.sqlite`,也可用 `--import-resource-db` /
|
||||||
|
`BAT_IMPORT_RESOURCE_DB` 覆盖。
|
||||||
|
5. AssetBundle、TextAsset、TableBundle、Media、Manifest/Other 会按资源类型分类;资源 metadata 会通过 `metadata_json` 保存 release、平台、bundle path、parse status、TextAsset 名称和 TextUnit 数量/格式。
|
||||||
|
6. 当前 release 的单条 TextUnit 明细和解析错误会写入 `official-textunit-index.json`,可通过 `parse.text_units` / `parse.errors` RPC 和 `parse-text-units` / `parse-errors` CLI 只读查询。
|
||||||
|
|
||||||
这层的意义是把“下载到磁盘的文件”变成“可查询、可复用、可去重”的资源对象。把官方同步结果自动接入 CAS + `ResourceRepository` 仍属于 G-011 剩余工作。
|
这层的意义是把“下载到磁盘的文件”变成“可查询、可复用、可去重”的资源对象。
|
||||||
|
`resource.index` RPC / CLI 只读查询现有 SQLite 索引;索引不存在时返回
|
||||||
|
`available=false`,不会因为查询创建空库。G-011 剩余工作是翻译任务状态、
|
||||||
|
CAS 诊断入口和更丰富查询。
|
||||||
|
|
||||||
对应实现主要在:
|
对应实现主要在:
|
||||||
|
|
||||||
@@ -196,8 +213,8 @@
|
|||||||
流程是:
|
流程是:
|
||||||
|
|
||||||
1. 显式执行 `--auto-discover` 或读取已审计 `server-info` 输入。
|
1. 显式执行 `--auto-discover` 或读取已审计 `server-info` 输入。
|
||||||
2. `--auto-discover` 先抓官方 launcher metadata;metadata 未变时复用 `official-bootstrap-cache.json` 中的 `GameMainConfig` 摘要,metadata 变化时按 manifest 临时下载 `resources.assets` 或旧版官方 game zip 并重新解析。
|
2. `--auto-discover` 先抓官方 launcher metadata、launcher CDN config 和 remote manifest;metadata 与 remote manifest 文件列表 digest 均未变时复用 `official-bootstrap-cache.json` 中的 `GameMainConfig` 摘要,任一变化时按 manifest 临时下载 `resources.assets` 或旧版官方 game zip 并重新解析。
|
||||||
3. 生成当前 v2 snapshot,记录 `app_version`、`connection_group`、`bundle_version`、`addressables_root`、endpoint URL、seed `.hash` 内容、`catalog_*.hash` marker、launcher metadata 摘要和 `GameMainConfig` 摘要。
|
3. 生成当前 v2 snapshot,记录 `app_version`、`connection_group`、`bundle_version`、`addressables_root`、endpoint URL、seed `.hash` 内容、`catalog_*.hash` marker、launcher metadata 摘要、remote manifest 文件列表 digest 和 `GameMainConfig` 摘要。
|
||||||
4. 读取上一次成功同步写出的 snapshot。
|
4. 读取上一次成功同步写出的 snapshot。
|
||||||
5. 使用 `OfficialSyncPlan` 和扩展 snapshot diff 判断是否需要下载;URL 未变但 `.hash` / marker 内容变化也会触发更新。
|
5. 使用 `OfficialSyncPlan` 和扩展 snapshot diff 判断是否需要下载;URL 未变但 `.hash` / marker 内容变化也会触发更新。
|
||||||
6. 每轮都会基于最新 seed catalog 构建当前 pull plan,并检查输出目录是否已有当前 plan 的 manifest 条目或目标文件。
|
6. 每轮都会基于最新 seed catalog 构建当前 pull plan,并检查输出目录是否已有当前 plan 的 manifest 条目或目标文件。
|
||||||
@@ -205,10 +222,16 @@
|
|||||||
8. 远端无变化且本地已有资源时执行 download manifest audit,检查路径、size、BLAKE3 和 ZIP 结构。
|
8. 远端无变化且本地已有资源时执行 download manifest audit,检查路径、size、BLAKE3 和 ZIP 结构。
|
||||||
9. 远端变化、本地 audit 发现 repair_needed,首次空目录运行,或缺少 `current` 原子发布指针时,进入下载/发布流程。
|
9. 远端变化、本地 audit 发现 repair_needed,首次空目录运行,或缺少 `current` 原子发布指针时,进入下载/发布流程。
|
||||||
10. 下载先写入 `<output>/.staging/<id>`;若已有 active release,会先 seed staging 以复用已验证文件;若 version-state 中存在同一版本的失败 staging,则优先复用该 staging 并跳过 active seed,避免旧 active 覆盖已下载的新文件。
|
10. 下载先写入 `<output>/.staging/<id>`;若已有 active release,会先 seed staging 以复用已验证文件;若 version-state 中存在同一版本的失败 staging,则优先复用该 staging 并跳过 active seed,避免旧 active 覆盖已下载的新文件。
|
||||||
11. 下载、manifest、本地 BLAKE3、ZIP 和官方 `.hash` 校验完成后写入新的 snapshot。
|
11. 下载、manifest、本地 BLAKE3、ZIP 和官方 `.hash` 校验完成后写入新的 snapshot,并在 staging 中写入 `official-launcher-bootstrap.json`(若本轮启用 `--auto-discover`)。
|
||||||
12. 将 staging rename 为 `<output>/versions/<id>`,再原子替换 `<output>/current` symlink 指向该 versioned 目录。
|
12. 将 staging rename 为 `<output>/versions/<id>`,再原子替换 `<output>/current` symlink 指向该 versioned 目录。
|
||||||
|
13. 发布完成后先对比上一完整 release 和当前 release 的 `official-download-manifest.json`,写出 `official-resource-changes.json` 和 `crowdin-translation-handoff.json`。同一 destination 只有 size 或 BLAKE3 变化才算 modified;新增+变更资源进入解析/翻译 handoff,删除资源只进入差异记录。当前只预留 Crowdin 本地 handoff,不发外部 API 请求。
|
||||||
|
14. 随后刷新 active release 下的 `official-parse-cache.json` 和 `official-textunit-index.json`,并从 Added/Modified 资源、parse cache 与 TextUnit 明细索引派生 `official-textunit-tasks.json` 和 `crowdin-textunit-queue.json`;up-to-date 轮询在已有有效解析缓存、TextUnit 明细索引和 TextUnit 队列时只读取摘要,不重复解析。
|
||||||
|
15. 若启用 `--import-repository`,已校验 release 会被导入 CAS + `ResourceRepository`,并可经 `resource.index` 查询。
|
||||||
|
16. 官方同步报告默认给出 `localized_release_status=not_localized`,表示原版资源已发布、汉化资源未发布;UnityFS TextAsset patch 发布成功并通过 `localized-patch-manifest.json`、current symlink 和 release ID 校验后,`localized.status` 才返回 `localized`,表示原版和汉化两套资源都已发布。
|
||||||
|
|
||||||
该入口不安装、不执行官方启动器,也不读取生产外的本地客户端目录。Rust 正式 binary `bat` 支持单次运行、`--watch` 常驻模式、`--daemon` 后台模式,以及 `status`、`stop`、`restart`、`reload`、`logs`、`refresh`、`verify`、`repair`、`doctor`、`clean-stable` 管理命令。`--daemon` 会在后台状态目录下创建 `bat.sock`,使用 Unix socket JSON-RPC 作为 live control plane;`bat.pid`、`bat-status.json` 和 `bat-daemon.log` 是快照、诊断和兼容 fallback;`bat-events.jsonl` 是带轮转的结构化 JSONL 事件日志;`bat-control.lock` 串行化控制命令,并在 stale/corrupt 时由下一次控制命令或 `clean-stable` 恢复。`bat-status.json` 和 `status` 子命令包含最后成功时间、下次检查时间、最后错误摘要、当前阶段和当前下载 URL 进度。PID、status、log 和控制锁文件创建时使用私有权限,读取和写入时不跟随 symlink。`status`、`stop`、`logs`、`reload`、默认形态的 `refresh` 和默认形态的 `repair` 优先走 RPC;`reload` 会唤醒或排队 watch 循环重新自动发现并强制刷新,默认 `repair` 会通过 `resource.repair` 入队本地 manifest 审计+修复任务,`restart` 才负责重启进程或替换启动参数;显式 `--proxy` / `--no-proxy` 会作为启动参数保存并在后台重启时复用。后台 daemon 管理某个资源目录时,前台 `run/watch/refresh/repair` 不允许直接写入同一目录;默认形态 `refresh` 会通过 RPC 触发后台刷新,默认形态 `repair` 会通过 RPC 入队任务。正常情况下默认每 1 小时执行一次检查;每天北京时间(UTC+8)`03:00`、`16:00`、`18:00` 会中断普通 sleep 并强制执行一次自动刷新,该轮注入 `force=true`。远端和本地一致时静默等待下次检查,不一致时自动下载或 repair。下载、发现或校验失败时不等待完整正常周期,默认 60 秒后重试;如果固定时间强制刷新失败,会保留 pending force 并按失败重试周期继续重试,可用 `--error-retry` 或 `--error-retry-seconds` 调整。默认资源输出目录是 `./bat-resources`,默认后台状态目录是 `/tmp/bat-pid`,二者通过 `--output` 和 `--state-dir` 分别配置。单次运行仍保留为核心幂等路径,systemd service、容器或 Go 进程可以只负责守护该常驻进程;cron/systemd timer 调单次模式只是可选集成方式。项目是否热更新、热重载或重启进程,由上层业务集成决定。
|
维护期特殊分支:如果官方 launcher/server-info 已经指向新资源根,但 client-patch seed marker 或必需 seed catalog 仍返回 403/404 等未开放状态,`bat` 返回 `waiting_for_official_resources`,保留现有 `current`,不创建失败 staging;若本轮启用 `--auto-discover`,会在 `<output>/official-launcher-bootstrap.pending.json` 写入待处理 launcher bootstrap 证据,供后续排障和自研客户端开发使用。
|
||||||
|
|
||||||
|
该入口不安装、不执行官方启动器,也不读取生产外的本地客户端目录。Rust 正式 binary `bat` 支持单次运行、`--watch` 常驻模式、`--daemon` 后台模式,以及 `status`、`stop`、`restart`、`reload`、`logs`、`refresh`、`verify`、`repair`、`doctor`、`clean-stable` 管理命令。`--daemon` 会在后台状态目录下创建 `bat.sock`,使用 Unix socket JSON-RPC 作为 live control plane;`bat.pid`、`bat-status.json` 和 `bat-daemon.log` 是快照、诊断和兼容 fallback;`bat-events.jsonl` 是带轮转的结构化 JSONL 事件日志;`bat-control.lock` 串行化控制命令,并在 stale/corrupt 时由下一次控制命令或 `clean-stable` 恢复。`bat-status.json` 和 `status` 子命令包含最后成功时间、下次检查时间、最后错误摘要、当前阶段和当前下载 URL 进度。PID、status、log 和控制锁文件创建时使用私有权限,读取和写入时不跟随 symlink。`status`、`stop`、`logs`、`reload`、默认形态的 `refresh` 和默认形态的 `repair` 优先走 RPC;`reload` 会唤醒或排队 watch 循环重新自动发现并强制刷新,默认 `repair` 会通过 `resource.repair` 入队本地 manifest 审计+修复任务,`restart` 才负责重启进程或替换启动参数;显式 `--proxy` / `--no-proxy` 会作为启动参数保存并在后台重启时复用。后台 daemon 管理某个资源目录时,前台 `run/watch/refresh/repair` 不允许直接写入同一目录;默认形态 `refresh` 会通过 RPC 触发后台刷新,默认形态 `repair` 会通过 RPC 入队任务。正常情况下默认每 1 小时执行一次检查;每天北京时间(UTC+8)`03:00`、`16:00`、`18:00` 会中断普通 sleep 并强制执行一次自动刷新,该轮注入 `force=true`。远端和本地一致时静默等待下次检查,不一致时自动下载或 repair。下载、发现或校验失败时不等待完整正常周期,默认 60 秒后重试;如果固定时间强制刷新失败,会保留 pending force 并按失败重试周期继续重试,可用 `--error-retry` 或 `--error-retry-seconds` 调整。默认官方原版资源输出目录是 `./bat-resources`,默认汉化产物目录是 `./bat-localized`,默认后台状态目录是 `/tmp/bat-pid`,三者分别通过 `--output`、`--localized-output` 和 `--state-dir` 配置;官方目录和汉化目录不能相同或互相嵌套。单次运行仍保留为核心幂等路径,systemd service、容器或 Go 进程可以只负责守护该常驻进程;cron/systemd timer 调单次模式只是可选集成方式。项目是否热更新、热重载或重启进程,由上层业务集成决定。
|
||||||
|
|
||||||
对应实现主要在:
|
对应实现主要在:
|
||||||
|
|
||||||
@@ -246,10 +269,11 @@ Linux 生产路径:
|
|||||||
- pull plan 会同时包含 discovery URLs 和 content URLs
|
- pull plan 会同时包含 discovery URLs 和 content URLs
|
||||||
- 全量样本下是 `2` 个 discovery URL + `5` 个内容 URL = `7` 个 URL
|
- 全量样本下是 `2` 个 discovery URL + `5` 个内容 URL = `7` 个 URL
|
||||||
- `OfficialUpdateService` 能持久化 v2 snapshot,并在远端 marker 内容变化时触发下载决策
|
- `OfficialUpdateService` 能持久化 v2 snapshot,并在远端 marker 内容变化时触发下载决策
|
||||||
- `bat` 默认向 stderr 输出 `BlueArchiveToolkit` ASCII banner 和 progress log,stdout 默认输出人类可读摘要;progress log 覆盖代理决策、下载已完成计数、单文件开始/完成状态、下载中断失败分类和校验结果摘要;支持 `--proxy` / `--no-proxy` 控制 curl 传输代理,支持 `--json` 输出稳定 JSON,支持 `--no-progress` 关闭进度日志,支持 `--no-banner` 只关闭横幅,支持 `--watch --interval 1h --error-retry 60s` 常驻运行,支持 `--daemon` Unix socket JSON-RPC live control/backend(`daemon.status/logs/stop/reload/refresh/doctor`、`resource.sync/verify/repair/state/manifest/list`、`catalog.*`、`task.*`);`restart` 与 `clean-stable` 仍由 CLI 侧按进程生命周期显式执行,非 dry-run 使用 `.official-sync.lock` 防止并发写资源目录,控制命令使用 `bat-control.lock` 防止并发状态修改,资源发布使用 `.staging`、`versions` 和 `current` 原子切换,daemon 写 `bat-events.jsonl` 结构化日志并在 `status` 中暴露下载进度、失败类型、HTTP 状态和调度状态
|
- `bat` 默认向 stderr 输出 `BlueArchiveToolkit` ASCII banner 和 progress log,stdout 默认输出人类可读摘要;progress log 覆盖代理决策、下载已完成计数、单文件开始/完成状态、下载中断失败分类和校验结果摘要;支持 `--proxy` / `--no-proxy` 控制 curl 传输代理,支持 `--json` 输出稳定 JSON,支持 `--no-progress` 关闭进度日志,支持 `--no-banner` 只关闭横幅,支持 `--watch --interval 1h --error-retry 60s` 常驻运行,支持 `--daemon` Unix socket JSON-RPC live control/backend(`daemon.status/logs/stop/reload/refresh/doctor`、`resource.sync/verify/repair/state/manifest/list/index`、`parse.status/text_units/errors`、`localized.status`、`catalog.*`、`task.*`);`restart` 与 `clean-stable` 仍由 CLI 侧按进程生命周期显式执行,非 dry-run 使用 `.official-sync.lock` 防止并发写资源目录,控制命令使用 `bat-control.lock` 防止并发状态修改,资源发布使用 `.staging`、`versions` 和 `current` 原子切换,daemon 写 `bat-events.jsonl` 结构化日志并在 `status` 中暴露下载进度、失败类型、HTTP 状态和调度状态
|
||||||
- curl 失败分类和重试策略已覆盖 404 不重试、5xx 重试耗尽后 quarantine、launcher primary CDN 失败后切换 official backup CDN
|
- curl 失败分类和重试策略已覆盖 404 不重试、5xx 重试耗尽后 quarantine、launcher primary CDN 失败后切换 official backup CDN
|
||||||
- `official-version-state.json` 已覆盖当前完成版本、正在拉取版本、上一个可用版本和失败版本;同一 app version、bundle version 和 Addressables root 的失败只保留最新一条,重新拉取或成功发布后清理同版本失败记录,同版本失败 staging 会在路径安全且未发布时复用,`bat status` 会暴露版本状态摘要和最近历史失败原因
|
- `official-version-state.json` 已覆盖当前完成版本、正在拉取版本、上一个可用版本和失败版本;同一 app version、bundle version 和 Addressables root 的失败只保留最新一条,重新拉取或成功发布后清理同版本失败记录,同版本失败 staging 会在路径安全且未发布时复用,`bat status` 会暴露版本状态摘要和最近历史失败原因
|
||||||
- 资源导入链路已覆盖 CAS 写入、`ResourceRepository` 索引、AssetBundle UnityFS 摘要,以及 TextAsset/Table/Media 分类
|
- 资源导入链路已覆盖可配置 CAS 写入、`ResourceRepository` 索引、`metadata_json` release/平台/bundle/TextAsset/TextUnit 摘要,以及 TextAsset/Table/Media 分类;`resource.index` 可只读查询现有索引
|
||||||
|
- 官方 release 发布后会生成 `official-resource-changes.json`、`crowdin-translation-handoff.json`、`official-parse-cache.json`、`official-textunit-index.json`、`official-textunit-tasks.json` 和 `crowdin-textunit-queue.json`,为后续增量解析和 Crowdin worker 预留稳定输入
|
||||||
- 离线回归样本已覆盖当前 catalog、上一个版本 catalog、catalog 结构变化、403、404 和 seed hash mismatch
|
- 离线回归样本已覆盖当前 catalog、上一个版本 catalog、catalog 结构变化、403、404 和 seed hash mismatch
|
||||||
- `OfficialUpdateService` 能读写 `official-bootstrap-cache.json`,并支持默认开启的 `audit_local` / `repair` CLI 行为
|
- `OfficialUpdateService` 能读写 `official-bootstrap-cache.json`,并支持默认开启的 `audit_local` / `repair` CLI 行为
|
||||||
- 下载层能在本地文件 size/BLAKE3/path、ZIP 结构或 manifest 不匹配时重新下载
|
- 下载层能在本地文件 size/BLAKE3/path、ZIP 结构或 manifest 不匹配时重新下载
|
||||||
@@ -296,22 +320,42 @@ JSON-RPC 2.0 服务,是面向上层服务(Go 层)的**主要跨语言边
|
|||||||
watch 循环经进程内锁互斥。任务历史持久化于 `<state-dir>/bat-tasks.json`
|
watch 循环经进程内锁互斥。任务历史持久化于 `<state-dir>/bat-tasks.json`
|
||||||
(版本化、`0600` 原子写,生命周期转换时落盘),daemon 重启后历史任务
|
(版本化、`0600` 原子写,生命周期转换时落盘),daemon 重启后历史任务
|
||||||
仍可经 `task.*` 查询,中断任务标记 `task_interrupted`(700005)。
|
仍可经 `task.*` 查询,中断任务标记 `task_interrupted`(700005)。
|
||||||
- 方法命名空间与实现状态、请求/响应示例见 `USERGUIDE.md` §6:
|
- 方法命名空间与实现状态、请求/响应示例见
|
||||||
`daemon.status/logs/stop/reload/refresh/doctor`、`resource.state/sync/verify/repair/manifest/list`、
|
`docs/reference/rpc-backend-api.md`:`daemon.status/logs/stop/reload/refresh/doctor`、
|
||||||
`catalog.*` 与 `task.status/list/cancel/logs` 已实现;`patch.*` / `unityfs.*`
|
`resource.state/sync/verify/repair/manifest/list/index`、`parse.status/text_units/errors`、
|
||||||
待引擎;`task.create` 按设计暂不开放通用任务入口;`daemon.restart` /
|
`localized.status`、`catalog.*` 与 `task.status/list/cancel/logs` 已实现;
|
||||||
`daemon.clean-stable` 仍由 CLI 侧按进程生命周期显式执行。
|
`patch.*` / `unityfs.*` 待引擎;`task.create` 按设计暂不开放通用任务入口;
|
||||||
|
`daemon.restart` / `daemon.clean-stable` 仍由 CLI 侧按进程生命周期显式执行。
|
||||||
|
|
||||||
### 7.2 Go 层职责边界
|
### 7.2 Go 层职责边界
|
||||||
|
|
||||||
- Go 层负责:BlueArchive 客户端请求处理、HTTP API、鉴权、内容分发,
|
- Rust `bat` / daemon 是资源生产者和状态拥有者;Go `bat-api` 是资源读侧、
|
||||||
以及通过 `internal/backendrpc` 作为 RPC client 调用本机 daemon
|
bootstrap 和 HTTP 分发入口。二者之间的稳定边界是 `bat.sock` RPC 和
|
||||||
(连接 `bat.sock`,每行一个 JSON-RPC 请求/响应)。当前 Go 产品入口
|
`resource_root` 中已发布的只读文件。
|
||||||
尚未完成,`cmd/bat` 仍是试验骨架。
|
- Go 层负责:资源 bootstrap、资源内容分发(`cmd/bat-api`)、HTTP API 进程配置、
|
||||||
- Rust daemon 负责:官方资源自动拉取与校验、catalog 更新检查、
|
以及通过 `internal/backendrpc` 作为 RPC client 调用本机 daemon(连接
|
||||||
|
`bat.sock`,每行一个 JSON-RPC 请求/响应)。`cmd/bat` 仍是试验骨架,不是产品级用户 CLI。
|
||||||
|
- **`bat-api`(资源分发,issue #19)**:
|
||||||
|
- 提供 `/v1/bootstrap`,把 `bat` 的 RPC 健康、release 摘要、server-info URL、
|
||||||
|
client-patch base 和改写后的 Addressables root 组织成启动前资源发现响应。
|
||||||
|
- 提供 `/healthz` 作为 liveness + 最近一次 RPC refresh 诊断,提供 `/readyz`
|
||||||
|
作为 release readiness;当前无可分发 release 时 `/readyz` 返回 `503`。
|
||||||
|
- 只读提供 Rust `bat` 已发布 release 中的资源字节(官方 CDN host/path 形态)。
|
||||||
|
- CDN path 支持 `GET` / `HEAD` / Range / 条件请求;ETag 优先使用 download
|
||||||
|
manifest 中的 BLAKE3,响应包含 Last-Modified、Accept-Ranges 和长期缓存头。
|
||||||
|
- 版本/清单发现优先走 RPC:先 `daemon.status`,再 `daemon.doctor`,再
|
||||||
|
`catalog.status` / `resource.manifest`(可用 `--socket` 指定 socket 文件)。
|
||||||
|
- 支持 `.env` / 环境变量配置监听端口、public base URL、RPC socket 和 RPC
|
||||||
|
刷新周期,并预留 database/redis 键供后续 API 持久化;**不**负责资源自动拉取。
|
||||||
|
- 可选改写 server-info 中的 `AddressablesCatalogUrlRoot` 指向自身;不伪装
|
||||||
|
完整游戏业务 API。启动前资源 metadata 兼容属于资源 bootstrap;账号、登录、
|
||||||
|
Gateway、游戏业务 `ApiUrl` 和鉴权全链非本服务关闭条件。
|
||||||
|
- Rust `bat` / daemon 负责:官方资源自动发现与拉取、校验、catalog 更新检查、
|
||||||
版本状态与发布、任务队列/日志/错误/进度管理等长期状态型工作。
|
版本状态与发布、任务队列/日志/错误/进度管理等长期状态型工作。
|
||||||
- Go 层**不**直接嵌入 Rust FFI,不直接读写 daemon 的状态文件与资源
|
- Go 层**不**直接嵌入 Rust FFI,不直接读写 daemon 的状态文件;跨语言控制面
|
||||||
目录内部结构;跨语言交互只经 RPC 契约。
|
只经 RPC 契约。生产文件字节从 RPC 给出的 `resource_root` 读取,`bat-api`
|
||||||
|
与 daemon 同服务器、同容器或同一共享文件系统部署;显式 `--resource-root`
|
||||||
|
只用于 fixture、本地开发或 RPC 不可用时的应急只读诊断。
|
||||||
|
|
||||||
### 7.3 FFI 的定位(降级说明)
|
### 7.3 FFI 的定位(降级说明)
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,359 @@
|
|||||||
|
# 官方资源 Release 布局与资源侧契约
|
||||||
|
|
||||||
|
- **更新时间**:2026-07-27
|
||||||
|
- **用途**:冻结日服官方资源在本地发布根上的布局、URL 映射、seed 规则、`bat`/`bat-api` 关系,以及 `bat-api` 分发 path 的 1:1 对应关系。
|
||||||
|
- **范围**:资源发现 / 清单 / 落盘 / 只读分发(**不是**完整游戏业务 API)。
|
||||||
|
- **权威代码**:
|
||||||
|
- URL / 平台 / seed:`adapters/src/official/yostar_jp.rs`
|
||||||
|
- inventory 抽取:`adapters/src/official/inventory.rs`
|
||||||
|
- 落盘与 manifest:`infrastructure/src/official_download.rs`(`destination_for_url`)
|
||||||
|
- 发布布局:`infrastructure/src/official_update.rs`
|
||||||
|
- 分发:`cmd/bat-api` + `internal/api`(见 `docs/reports/GO_STATUS.md`)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. 产品边界(资源侧)
|
||||||
|
|
||||||
|
| 角色 | 组件 | 职责 |
|
||||||
|
|---|---|---|
|
||||||
|
| 同步 / 运维(近乎全自动) | Rust `bat` | auto-discover、拉取、校验、发布、watch/daemon、RPC 后端 |
|
||||||
|
| 资源 bootstrap / 只读分发 | Go `bat-api` | 同环境经 `bat.sock` 发现已发布版本和 `resource_root`,提供 `/v1/bootstrap`、server-info 改写和官方 CDN path 字节 |
|
||||||
|
| 试验 CLI | Go `cmd/bat` → `bin/bat-go` | 非产品;禁止与 Rust `bat` 重名 |
|
||||||
|
|
||||||
|
**禁止**:把已安装客户端目录或 `/home/wanye/D/BlueArchive` 当作生产输入;真实全量样本优先服务器 release 或 `/tmp` 隔离目录。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. 发布根布局(L1)
|
||||||
|
|
||||||
|
```text
|
||||||
|
<output>/ # 官方原版资源发布根(--output / BAT_OUTPUT)
|
||||||
|
current -> versions/<id> # 原子 symlink,生产读侧
|
||||||
|
versions/<id>/ # 已发布 versioned release(= resource_root)
|
||||||
|
official-download-manifest.json
|
||||||
|
official-parse-cache.json # 校验后派生解析缓存,不是汉化产物
|
||||||
|
official-textunit-index.json # TextUnit 明细与解析错误索引,不是汉化产物
|
||||||
|
official-textunit-tasks.json # 翻译任务候选派生队列,不发 Crowdin 网络请求
|
||||||
|
crowdin-textunit-queue.json # Crowdin worker 离线输入队列
|
||||||
|
official-sync-snapshot.json # 常在 active root / current 下
|
||||||
|
official-launcher-bootstrap.json # 官方 launcher 引导链版本化产物
|
||||||
|
prod-clientpatch.bluearchiveyostar.com/
|
||||||
|
<root_token>/
|
||||||
|
TableBundles/
|
||||||
|
TableCatalog.bytes
|
||||||
|
TableCatalog.hash
|
||||||
|
<table files...> # e.g. ExcelDB.db, Excel.zip
|
||||||
|
Windows_PatchPack/
|
||||||
|
BundlePackingInfo.bytes
|
||||||
|
BundlePackingInfo.hash
|
||||||
|
catalog_StandaloneWindows64.zip
|
||||||
|
catalog_StandaloneWindows64.hash
|
||||||
|
FullPatch_NNN.zip
|
||||||
|
Android_PatchPack/
|
||||||
|
BundlePackingInfo.bytes
|
||||||
|
BundlePackingInfo.hash
|
||||||
|
catalog_Android.zip
|
||||||
|
catalog_Android.hash
|
||||||
|
FullPatch_NNN.zip
|
||||||
|
MediaResources-Windows/
|
||||||
|
Catalog/MediaCatalog.bytes
|
||||||
|
Catalog/MediaCatalog.hash
|
||||||
|
GameData/...
|
||||||
|
Prologue/...
|
||||||
|
MediaResources/ # Android
|
||||||
|
Catalog/MediaCatalog.bytes
|
||||||
|
Catalog/MediaCatalog.hash
|
||||||
|
...
|
||||||
|
yostar-serverinfo.bluearchiveyostar.com/ # 若曾下载 server-info
|
||||||
|
<name>.json
|
||||||
|
.staging/<id>/ # 未发布写侧(失败可复用)
|
||||||
|
official-version-state.json # 发布根级版本状态
|
||||||
|
official-bootstrap-cache.json # auto-discover 缓存
|
||||||
|
official-launcher-bootstrap.pending.json # 维护期 launcher 已前进但资源未开放时的待处理证据
|
||||||
|
|
||||||
|
<localized-output>/ # 汉化产物发布根(--localized-output / BAT_LOCALIZED_OUTPUT)
|
||||||
|
current -> versions/<id> # 已汉化后才切换;未汉化状态不发布
|
||||||
|
versions/<id>/ # 与官方相对路径一致的汉化资源
|
||||||
|
localized-version-state.json # 预留:后续 Patch 发布阶段维护,官方同步阶段不写入
|
||||||
|
```
|
||||||
|
|
||||||
|
官方资源发布和汉化发布是两个独立状态:
|
||||||
|
|
||||||
|
- `not_localized`:官方原版资源已经完成下载、校验和发布,汉化资源尚未发布;这是官方同步完成后的默认状态。
|
||||||
|
- `localized`:同一官方版本的原版资源和汉化资源都已发布,生产侧可以同时提供两套资源。
|
||||||
|
|
||||||
|
### 2.1 读侧 vs 写侧
|
||||||
|
|
||||||
|
| 阶段 | 根目录 |
|
||||||
|
|---|---|
|
||||||
|
| 下载写入 | `<output>/.staging/<id>` |
|
||||||
|
| 发布完成 | rename 到 `versions/<id>`,再切换 `current` |
|
||||||
|
| 生产读取 / bat-api | RPC 给出的 `version.resource_root`;通常等价于 `current` 解析后的 versioned 目录 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. URL → 磁盘映射(核心不变量)
|
||||||
|
|
||||||
|
实现:`OfficialResourcePullService::destination_for_url`。
|
||||||
|
|
||||||
|
```text
|
||||||
|
https://{host}/{path...} → <resource_root>/{host}/{path...}
|
||||||
|
```
|
||||||
|
|
||||||
|
规则:
|
||||||
|
|
||||||
|
1. 仅 `https://`
|
||||||
|
2. host 必须是官方 JP 资源 host(见下节)
|
||||||
|
3. path 分段不得为 `.` / `..`
|
||||||
|
4. **禁止** query / fragment(否则直接拒绝,避免同路径覆盖)
|
||||||
|
5. 分段经 sanitize 后 join;结果必须在 `resource_root` 内
|
||||||
|
|
||||||
|
### 3.1 官方 host
|
||||||
|
|
||||||
|
| Host | 用途 |
|
||||||
|
|---|---|
|
||||||
|
| `prod-clientpatch.bluearchiveyostar.com` | Addressables / Table / Media / PatchPack 内容 |
|
||||||
|
| `yostar-serverinfo.bluearchiveyostar.com` | server-info JSON |
|
||||||
|
|
||||||
|
(launcher 包 CDN 属于启动器链,**不是**默认资源 release 主体。Rust `bat` 会把启动器链中与资源发现相关的 launcher metadata、CDN config、remote manifest 文件列表、选中的 `resources.assets` 来源和 `GameMainConfig` 摘要写入 `official-launcher-bootstrap.json`,供后续 `bat-api` / 自研客户端在 Rust 侧完成前继续以 versioned release 为权威来源。)
|
||||||
|
|
||||||
|
### 3.2 bat-api 对外 path(1:1)
|
||||||
|
|
||||||
|
```text
|
||||||
|
GET {public-base-url}/prod-clientpatch.bluearchiveyostar.com/<root_token>/...
|
||||||
|
≡ 磁盘 <resource_root>/prod-clientpatch.bluearchiveyostar.com/<root_token>/...
|
||||||
|
```
|
||||||
|
|
||||||
|
默认仅服务 **download manifest 索引内且 Present + size 匹配** 的文件。
|
||||||
|
|
||||||
|
### 3.3 launcher 资源引导兼容
|
||||||
|
|
||||||
|
只读分析本机样本时可见两类启动器形态:
|
||||||
|
|
||||||
|
| 目录形态 | 说明 |
|
||||||
|
|---|---|
|
||||||
|
| `AllResources/YostarGames/BlueArchive_JP_Gamelauncher` | 官方 Electron 启动器目录 |
|
||||||
|
| `Localized` / `Localized_Official` | 汉化或改造启动器目录 |
|
||||||
|
| `AllResources/YostarGames/BlueArchive_JP` | 官方安装后的游戏客户端目录,含 `game-launcher-config.json`、`manifest.json` 和 `BlueArchive_Data/StreamingAssets/catalog_Remote.*` |
|
||||||
|
|
||||||
|
这些目录只作为开发期样本;生产链路不得依赖 `/home/wanye/D/BlueArchive` 或任何已安装客户端目录。
|
||||||
|
|
||||||
|
官方启动器样本中与资源发现相关的 HTTP path:
|
||||||
|
|
||||||
|
| Host / path | 资源侧意义 |
|
||||||
|
|---|---|
|
||||||
|
| `https://api-launcher-jp.yo-star.com/api/launcher/game/config` | 返回 launcher 观察到的最新客户端版本和包路径 |
|
||||||
|
| `https://api-launcher-jp.yo-star.com/api/launcher/game/config/json?version=...&file_path=...` | 返回远端 package manifest URL |
|
||||||
|
| `https://api-launcher-jp.yo-star.com/api/launcher/advanced/game/download/cdn` | 返回 launcher package CDN primary / backup |
|
||||||
|
|
||||||
|
`bat-api` 的兼容范围是**资源引导**,不是完整启动器更新服务:
|
||||||
|
|
||||||
|
| bat-api path | 行为 |
|
||||||
|
|---|---|
|
||||||
|
| `/v1/launcher/bootstrap` | 返回资源引导聚合视图:已发布 release、launcher metadata、GameMainConfig 摘要、server-info URL、client-patch base、改写后的 Addressables root |
|
||||||
|
| `/api/launcher/game/config` | 返回 `{code,message,data}` envelope,字段来自 Rust `bat` snapshot/RPC 中的 `launcher_metadata`,并附带 `resource_bootstrap_url` |
|
||||||
|
| `/api/launcher/game/config/json` | 返回指向 `/api-launcher-jp.yo-star.com/api/launcher/resource/bootstrap.json` 的资源引导 JSON URL,显式标记 `package_update_manifest=false` |
|
||||||
|
| `/api/launcher/advanced/game/download/cdn` | 返回 `public-base-url` 作为资源引导 CDN 根,显式标记 `package_update_manifest=false` |
|
||||||
|
| `/api-launcher-jp.yo-star.com/...` | 与上面裸 path 等价,便于反向代理或 hosts 映射保持官方 host 形状 |
|
||||||
|
|
||||||
|
数据来源只能是 Rust `bat` 已发布状态;当前 Go `bat-api` 仍主要消费 snapshot/RPC 摘要,后续字段统一与联调时应把 versioned launcher artifact 纳入 contract fixture:
|
||||||
|
|
||||||
|
1. `catalog.status` / `official-sync-snapshot.json` 中的 `launcher_metadata`。
|
||||||
|
2. `catalog.status` / `official-sync-snapshot.json` 中的 `game_main_config_bootstrap`。
|
||||||
|
3. `official-launcher-bootstrap.json` 中的官方 launcher bootstrap versioned artifact。
|
||||||
|
4. `resource.manifest` 和磁盘 Present/size 检查得到的当前 release 索引。
|
||||||
|
|
||||||
|
`bat-api` 不下载 launcher 包、不生成官方 PC package update manifest、不执行启动器签名/鉴权链、不仿造登录、账号、网关或游戏业务 API。需要真实资源拉取时,仍由 Rust `bat --auto-discover` 在隔离 staging 中通过官方 HTTP metadata 完成,并把已发布结果通过 RPC 暴露给 `bat-api`。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. `official-download-manifest.json`
|
||||||
|
|
||||||
|
| 字段 | 类型 | 说明 |
|
||||||
|
|---|---|---|
|
||||||
|
| `version` | u32 | 当前为 `1` |
|
||||||
|
| `entries` | map URL → entry | 按完整官方 URL 为键(有序 BTreeMap) |
|
||||||
|
|
||||||
|
每条 entry:
|
||||||
|
|
||||||
|
| 字段 | 说明 |
|
||||||
|
|---|---|
|
||||||
|
| `url` | 官方 https URL |
|
||||||
|
| `destination` | 相对 resource_root 的路径(`host/path...`) |
|
||||||
|
| `bytes` | 文件大小 |
|
||||||
|
| `blake3` | 本地 BLAKE3 hex |
|
||||||
|
|
||||||
|
**权威清单**:拉取闭环写入的 manifest;`bat-api` / RPC `resource.manifest` 以此为应有集合,再以磁盘校验 Present。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. 发现与 seed URL 规则(L2)
|
||||||
|
|
||||||
|
常量根:
|
||||||
|
|
||||||
|
- server-info:`https://yostar-serverinfo.bluearchiveyostar.com`
|
||||||
|
- client-patch:`https://prod-clientpatch.bluearchiveyostar.com`
|
||||||
|
|
||||||
|
`AddressablesCatalogUrlRoot` 形如:
|
||||||
|
|
||||||
|
```text
|
||||||
|
https://prod-clientpatch.bluearchiveyostar.com/<root_token>
|
||||||
|
```
|
||||||
|
|
||||||
|
默认平台:`Windows` + `Android`。
|
||||||
|
|
||||||
|
### 5.1 平台目录名
|
||||||
|
|
||||||
|
| 平台 | Patch 目录 | Media 目录 | Addressables catalog zip |
|
||||||
|
|---|---|---|---|
|
||||||
|
| Windows | `Windows_PatchPack` | `MediaResources-Windows` | `catalog_StandaloneWindows64.zip` |
|
||||||
|
| Android | `Android_PatchPack` | `MediaResources` | `catalog_Android.zip` |
|
||||||
|
|
||||||
|
### 5.2 Seed 端点模板
|
||||||
|
|
||||||
|
共享(非平台):
|
||||||
|
|
||||||
|
```text
|
||||||
|
{CLIENT_PATCH}/{token}/TableBundles/TableCatalog.bytes
|
||||||
|
{CLIENT_PATCH}/{token}/TableBundles/TableCatalog.hash
|
||||||
|
```
|
||||||
|
|
||||||
|
每平台:
|
||||||
|
|
||||||
|
```text
|
||||||
|
{CLIENT_PATCH}/{token}/{PatchDir}/BundlePackingInfo.bytes
|
||||||
|
{CLIENT_PATCH}/{token}/{PatchDir}/BundlePackingInfo.hash
|
||||||
|
{CLIENT_PATCH}/{token}/{PatchDir}/{catalog_zip}
|
||||||
|
{CLIENT_PATCH}/{token}/{PatchDir}/{catalog_base}.hash
|
||||||
|
{CLIENT_PATCH}/{token}/{MediaDir}/Catalog/MediaCatalog.bytes
|
||||||
|
{CLIENT_PATCH}/{token}/{MediaDir}/Catalog/MediaCatalog.hash
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.3 Content URL 模板
|
||||||
|
|
||||||
|
| 类型 | 模板 |
|
||||||
|
|---|---|
|
||||||
|
| Table 文件 | `{CLIENT_PATCH}/{token}/TableBundles/{Name}` |
|
||||||
|
| Patch pack | `{CLIENT_PATCH}/{token}/{PatchDir}/{FullPatch_NNN.zip}` |
|
||||||
|
| Media 文件 | `{CLIENT_PATCH}/{token}/{MediaDir}/{relative_path}` |
|
||||||
|
|
||||||
|
`relative_path` 示例:`GameData/Audio/VOC_JP/JP_Airi.zip`、`Prologue/Scenario/Event/10000_Title_Sound.ogg`。
|
||||||
|
|
||||||
|
### 5.4 校验分层
|
||||||
|
|
||||||
|
| 对象 | 算法 / 规则 |
|
||||||
|
|---|---|
|
||||||
|
| seed `.bytes` + `.hash` | 官方 `.hash` 为 **xxHash32(seed=0)** 的十进制文本;强校验 |
|
||||||
|
| 一般已下载文件 | 本地 manifest **size + BLAKE3** |
|
||||||
|
| `.zip` | 另加 ZIP central/local 结构校验 |
|
||||||
|
| `catalog_*.hash` | Addressables/SBP **Hash128 文本标记**,**不是** seed 的 xxHash32 规则 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Inventory 抽取规则(L3,当前实现)
|
||||||
|
|
||||||
|
实现:`adapters/src/official/inventory.rs`(**可打印串启发式**,非完整 schema 反序列化)。
|
||||||
|
|
||||||
|
| Catalog | 抽取逻辑 | 风险 |
|
||||||
|
|---|---|---|
|
||||||
|
| `BundlePackingInfo.bytes` | 可打印串中扩展名为 `zip` 且匹配 `FullPatch_NNN.zip`(总长 17,中间 3 位数字) | 漏抽非 FullPatch 包名(当前有意只 FullPatch) |
|
||||||
|
| `TableCatalog.bytes` | 可打印串中 `.db`/`.zip` 文件名;**出现次数 ≥ 2** 才收录 | 依赖「双份列表」启发式;形态变化会漏/多 |
|
||||||
|
| `MediaCatalog.bytes` | 可打印串中相对路径,扩展名 zip/mp4/png/jpg/jpeg/ogg/wav | 路径须 `is_plausible_relative_path` |
|
||||||
|
|
||||||
|
**R2 待真机核对**:用服务器全量 seed 字节跑抽取,与 manifest 中 content URL 集合 diff;有未解释差异再改 inventory + fixture。
|
||||||
|
|
||||||
|
仓库内已有:`adapters/tests/fixtures`、`infrastructure/tests/fixtures/official_regression`;**不能替代**全量 release 实勘。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. 客户端资源请求假设(R3,服务 bat-api)
|
||||||
|
|
||||||
|
| 面 | 假设(当前工程) | bat-api 行为 |
|
||||||
|
|---|---|---|
|
||||||
|
| 启动前资源发现 | 客户端/补丁器需要知道当前资源版本、server-info 和 client-patch 根 | `GET /v1/bootstrap` 返回 `bat` RPC 健康、release 摘要、server-info URL、client-patch base 和改写后的 Addressables root |
|
||||||
|
| 服务就绪 | 运维需要区分进程存活和 release 是否可分发 | `GET /healthz` 返回 liveness + RPC refresh 诊断;`GET/HEAD /readyz` 无可分发 release 时返回 `503` |
|
||||||
|
| client-patch 内容 | GET 官方 path;无业务鉴权头(资源 CDN) | `GET/HEAD /prod-clientpatch.../...` 原样字节,支持 Range |
|
||||||
|
| server-info | GET JSON;字段 PascalCase(`ConnectionGroups` 等) | 可选加载并**只改** `AddressablesCatalogUrlRoot` 指向 `{public-base}/prod-clientpatch.../{token}` |
|
||||||
|
| launcher bootstrap | 启动器链会先查 launcher metadata,再找到 server-info / Addressables root | `/v1/launcher/bootstrap` 与 `/api/launcher/...` 只输出资源引导兼容信息,来源是 Rust snapshot/RPC |
|
||||||
|
| seed `.hash` | 纯文本十进制(可含空白) | 原样分发 |
|
||||||
|
| Range / 断点 | 官方客户端下载器用 Range;bat 用 curl `.part` | `ServeContent` 支持 Range / `206` / `416` / `If-Range` |
|
||||||
|
| 缓存 / 条件请求 | 资源位于 versioned root;manifest 有 BLAKE3 | ETag 优先使用 manifest BLAKE3;返回 Last-Modified、Accept-Ranges、长期 Cache-Control |
|
||||||
|
| 业务 ApiUrl/Gateway | 游戏协议 | **不改写、不仿造** |
|
||||||
|
|
||||||
|
Addressables 改写后客户端拼接:
|
||||||
|
|
||||||
|
```text
|
||||||
|
{rewritten_root}/TableBundles/TableCatalog.bytes
|
||||||
|
≡ {public-base}/prod-clientpatch.../{token}/TableBundles/TableCatalog.bytes
|
||||||
|
```
|
||||||
|
|
||||||
|
与磁盘映射一致。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. RPC 与分发发现顺序
|
||||||
|
|
||||||
|
`bat-api`(及任何 Go 服务层)发现当前 release,并由 `/v1/bootstrap` 组织为启动前资源入口:
|
||||||
|
|
||||||
|
1. `daemon.status`
|
||||||
|
2. `daemon.doctor`
|
||||||
|
3. `catalog.status`(`version.resource_root`、`addressables_root`、app/bundle)
|
||||||
|
4. `resource.manifest` 分页(url / destination / bytes / blake3)
|
||||||
|
5. 在 `resource_root` 上 Lstat 校验 Present / size
|
||||||
|
|
||||||
|
**不读** `bat-status.json` / `bat-tasks.json` 作为常规路径。
|
||||||
|
|
||||||
|
生产配置:`--socket` / `BAT_API_SOCKET`,`bat-api` 与 `bat` 在同服务器、同容器或同共享文件系统环境内运行。`--resource-root` 只用于本地 fixture 或应急只读诊断,不作为生产资源根配置。`BAT_API_REFRESH_INTERVAL` 控制 bat-api 周期重读 RPC,以跟随 Rust `bat` 发布新 release。见 `cmd/bat-api/.env.example`。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. issue #2 / #3 样本索引(R4/R5 预置)
|
||||||
|
|
||||||
|
在服务器 release 上优先采集到 `/tmp` 隔离目录(**不入库大文件**):
|
||||||
|
|
||||||
|
| 用途 | 建议路径模式 |
|
||||||
|
|---|---|
|
||||||
|
| Addressables(#2) | `{PatchDir}/catalog_*.zip` 解压后的 JSON/bin + 旁路 `.hash` |
|
||||||
|
| UnityFS(#3) | `FullPatch_*.zip` 内抽样 `.bundle`,或已解包 bundle |
|
||||||
|
| seed 加固(R2) | 各平台 `TableCatalog` / `BundlePackingInfo` / `MediaCatalog` 的 `.bytes`+`.hash` |
|
||||||
|
|
||||||
|
字段目标(#2,已有 `m_Crc` 部分):继续扩大 hash/size/CRC/依赖等可校验字段覆盖。
|
||||||
|
结构目标(#3):header / block / directory / metadata / object table 引擎级解析。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. 服务器实勘清单(R1,等 SSH)
|
||||||
|
|
||||||
|
连接信息到位后只读执行:
|
||||||
|
|
||||||
|
1. `readlink current` → version id
|
||||||
|
2. 顶层是否仅有官方 host 目录 + manifest/snapshot
|
||||||
|
3. manifest 条目数 vs 磁盘抽样 size
|
||||||
|
4. RPC 四步(status → doctor → catalog.status → manifest 首页)
|
||||||
|
5. 将结论写入 `docs/reports/resource-server-survey-YYYYMMDD.md`(无凭据)
|
||||||
|
|
||||||
|
所需:
|
||||||
|
|
||||||
|
```text
|
||||||
|
SSH: user@host -p PORT
|
||||||
|
资源目录: .../official
|
||||||
|
bat.sock 或 state-dir: ...
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. 相关文档
|
||||||
|
|
||||||
|
- `docs/reports/GO_STATUS.md` — Go 边界与进度
|
||||||
|
- `docs/architecture/official-resource-backend.md` — 拉取后端总览
|
||||||
|
- `docs/reference/rpc-backend-api.md` — RPC 契约
|
||||||
|
- `docs/guides/official-resource-test-pull.md` — 用户向运行说明
|
||||||
|
- `docs/reports/CURRENT_GAPS.md` — G-009 / #2 / #3
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 12. 变更纪律
|
||||||
|
|
||||||
|
1. 改 URL 模板或落盘规则 → **必须**同步本文 + 相关单测。
|
||||||
|
2. 改 inventory 启发式 → 说明覆盖的真实风险并补 fixture。
|
||||||
|
3. 真机实勘若发现与本文冲突 → **以真机为准** 修代码与本文,禁止静默分叉。
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
# 稳定工程基线指南
|
# 稳定工程基线指南
|
||||||
|
|
||||||
- **更新时间**:2026-07-20
|
- **更新时间**:2026-07-26
|
||||||
- **目标**:让工作区处于可继续开发核心功能的可信状态。
|
- **目标**:让工作区处于可继续开发核心功能的可信状态。
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -90,10 +90,11 @@ git check-ignore -v Cargo.lock CLAUDE.md AGENTS.md CONTRIBUTING.md
|
|||||||
|
|
||||||
CAS V1 和 Rust 官方同步闭环完成后,下一阶段优先推进:
|
CAS V1 和 Rust 官方同步闭环完成后,下一阶段优先推进:
|
||||||
|
|
||||||
1. 收敛 Go 产品入口:当前 `cmd/bat` 仅是试验骨架,不能视为完成。
|
1. 继续联调 Go `bat-api` 与 Rust daemon 的资源分发路径;Go 同步 CLI 不再作为产品目标。
|
||||||
2. 按 `docs/guides/official-full-pull-smoke.md` 执行真实官方网络全量下载 smoke,并保留隔离目录报告。
|
2. 按 `docs/guides/official-full-pull-smoke.md` 在隔离目录执行真实官方网络全量下载 smoke,并保留运行报告。
|
||||||
3. 官方同步结果接入 CAS + ResourceRepository。
|
3. 将 `crowdin-textunit-queue.json` 接入真实 Crowdin worker、翻译记忆和 Patch 构建。
|
||||||
4. AssetBundle UnityFS 引擎级解析。
|
4. 扩展 ResourceRepository 查询面:翻译任务状态、CAS 诊断入口和更丰富 TextUnit 查询。
|
||||||
|
5. 继续完善 AssetBundle 复杂对象解析、复杂对象重打包和 Patch 发布流程统一;通用 Binary/JSON/Text Patch 基础与 UnityFS TextAsset patch 发布前置链路已可用。
|
||||||
|
|
||||||
优先阅读:
|
优先阅读:
|
||||||
|
|
||||||
|
|||||||
+161
-12
@@ -5,8 +5,9 @@
|
|||||||
BlueArchive Toolkit 的部署文档分为当前可用模式和目标模式:
|
BlueArchive Toolkit 的部署文档分为当前可用模式和目标模式:
|
||||||
|
|
||||||
1. **本地开发模式**:代码在本地,连接本地或远程数据库。
|
1. **本地开发模式**:代码在本地,连接本地或远程数据库。
|
||||||
2. **官方资源同步生产任务**:当前可用,运行 Rust `bat --watch`。
|
2. **官方资源同步生产任务**:当前可用,运行 Rust `bat --watch` 或 RPC/daemon 模式。
|
||||||
3. **完整单机/分布式部署**:尚未提供。API Server、数据库迁移和 Web 未实现前,不把它作为可执行部署方案。
|
3. **bat-api 资源 bootstrap / 分发服务**:当前可用,和 Rust `bat` 在同一服务器/容器环境运行,经 `bat.sock` RPC 获取当前 `resource_root`。
|
||||||
|
4. **完整单机/分布式部署**:尚未提供。完整游戏业务 API、数据库迁移和 Web 未实现前,不把它作为可执行部署方案。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -100,7 +101,7 @@ REDIS_PORT=6379
|
|||||||
|
|
||||||
## 模式 3:官方资源同步生产任务
|
## 模式 3:官方资源同步生产任务
|
||||||
|
|
||||||
当前可部署的生产任务是 Rust 官方资源同步 binary。API Server 和 Web 尚未实现,不能按完整服务端产品部署。
|
当前可部署的生产同步任务是 Rust 官方资源同步 binary。`bat-api` 资源 bootstrap / 分发服务见模式 4;完整游戏业务 API 和 Web 尚未实现,不能按完整服务端产品部署。
|
||||||
|
|
||||||
### 构建 release binary
|
### 构建 release binary
|
||||||
|
|
||||||
@@ -198,14 +199,14 @@ sudo -u bat /opt/bluearchive-toolkit/bin/bat \
|
|||||||
--no-progress
|
--no-progress
|
||||||
```
|
```
|
||||||
|
|
||||||
### 推荐模式:systemd 托管 `--watch`
|
### 推荐模式:纯同步时 systemd 托管 `--watch`
|
||||||
|
|
||||||
生产推荐让 systemd 直接托管前台 `--watch` 进程,而不是在 systemd 里再启动 `--daemon`。原因:
|
只需要远程长期同步资源、暂不部署 `bat-api` 时,推荐让 systemd 直接托管前台 `--watch` 进程,而不是在 systemd 里再启动 `--daemon`。原因:
|
||||||
|
|
||||||
- systemd 能直接追踪主进程、退出码、重启次数和 stop 信号。
|
- systemd 能直接追踪主进程、退出码、重启次数和 stop 信号。
|
||||||
- 日志进入 journald,用 `journalctl` 管理,不依赖 `bat-daemon.log`。
|
- 日志进入 journald,用 `journalctl` 管理,不依赖 `bat-daemon.log`。
|
||||||
- Rust 内部已经负责 1 小时间隔、北京时间固定强制刷新和失败快速重试,systemd 不需要 timer。
|
- Rust 内部已经负责 1 小时间隔、北京时间固定强制刷新和失败快速重试,systemd 不需要 timer。
|
||||||
- `bat --daemon` 的 Unix socket RPC 适合没有进程管理器的 shell/container 场景;systemd 场景下用 `systemctl`、`journalctl`、`bat verify/doctor` 运维即可。
|
- `bat --daemon` 的 Unix socket RPC 适合 shell/container 场景,也适合给同环境运行的 `bat-api` 提供 release 发现;纯同步 systemd 场景下用 `systemctl`、`journalctl`、`bat verify/doctor` 运维即可。
|
||||||
|
|
||||||
安装 unit 和可选环境文件:
|
安装 unit 和可选环境文件:
|
||||||
|
|
||||||
@@ -227,10 +228,11 @@ systemctl status bluearchive-toolkit-official-sync.service
|
|||||||
journalctl -u bluearchive-toolkit-official-sync.service -f
|
journalctl -u bluearchive-toolkit-official-sync.service -f
|
||||||
sudo -u bat /opt/bluearchive-toolkit/bin/bat doctor \
|
sudo -u bat /opt/bluearchive-toolkit/bin/bat doctor \
|
||||||
--output /var/lib/bluearchive-toolkit/official \
|
--output /var/lib/bluearchive-toolkit/official \
|
||||||
|
--localized-output /var/lib/bluearchive-toolkit/localized \
|
||||||
--state-dir /run/bluearchive-toolkit
|
--state-dir /run/bluearchive-toolkit
|
||||||
```
|
```
|
||||||
|
|
||||||
`--watch` 是 Rust 内部持久检查模式,正常情况下默认每 1 小时执行一次检查,并且每天北京时间(UTC+8)`03:00`、`16:00`、`18:00` 会强制执行一次自动刷新。固定时间刷新会中断普通 interval 的 sleep,该轮注入 `force=true`;如果失败,会按失败重试周期继续重试。远端和本地一致时默认静默;有远端变化或本地文件损坏时自动下载或 repair,并输出人类可读摘要。下载、发现或校验失败时默认 60 秒后重试,可显式加 `BAT_ERROR_RETRY=60s` 或调整 service `ExecStart`。默认平台是 `Windows,Android`,无需显式传 `--platforms`;需要覆盖时用 systemd drop-in 重写 `ExecStart`。默认资源目录是 `./bat-resources`,生产 service 显式使用 `/var/lib/bluearchive-toolkit/official`。生产读取方应读取 `/var/lib/bluearchive-toolkit/official/current`;同步中的文件只会进入 `.staging/<id>`,校验完成后才发布为 `versions/<id>` 并切换 `current`。
|
`--watch` 是 Rust 内部持久检查模式,正常情况下默认每 1 小时执行一次检查,并且每天北京时间(UTC+8)`03:00`、`16:00`、`18:00` 会强制执行一次自动刷新。固定时间刷新会中断普通 interval 的 sleep,该轮注入 `force=true`;如果失败,会按失败重试周期继续重试。远端和本地一致时默认静默;有远端变化或本地文件损坏时自动下载或 repair,并输出人类可读摘要。下载、发现或校验失败时默认 60 秒后重试,可显式加 `BAT_ERROR_RETRY=60s` 或调整 service `ExecStart`。默认平台是 `Windows,Android`,无需显式传 `--platforms`;需要覆盖时用 systemd drop-in 重写 `ExecStart`。默认官方原版资源目录是 `./bat-resources`,默认汉化产物目录是 `./bat-localized`;生产 service 显式使用 `/var/lib/bluearchive-toolkit/official` 和 `/var/lib/bluearchive-toolkit/localized`,两者不能相同或互相嵌套。生产读取方应读取 `/var/lib/bluearchive-toolkit/official/current`;同步中的原版文件只会进入 `.staging/<id>`,校验完成后才发布为 `versions/<id>` 并切换 `current`。官方同步报告 `localized_release_status=not_localized` 表示汉化资源尚未发布;后续 Patch 发布才切换 `/var/lib/bluearchive-toolkit/localized/current`。
|
||||||
|
|
||||||
### 可选模式:CLI 自托管 `--daemon`
|
### 可选模式:CLI 自托管 `--daemon`
|
||||||
|
|
||||||
@@ -240,6 +242,7 @@ sudo -u bat /opt/bluearchive-toolkit/bin/bat doctor \
|
|||||||
sudo -u bat /opt/bluearchive-toolkit/bin/bat \
|
sudo -u bat /opt/bluearchive-toolkit/bin/bat \
|
||||||
--auto-discover \
|
--auto-discover \
|
||||||
--output /var/lib/bluearchive-toolkit/official \
|
--output /var/lib/bluearchive-toolkit/official \
|
||||||
|
--localized-output /var/lib/bluearchive-toolkit/localized \
|
||||||
--state-dir /var/lib/bluearchive-toolkit/daemon-state \
|
--state-dir /var/lib/bluearchive-toolkit/daemon-state \
|
||||||
--daemon
|
--daemon
|
||||||
|
|
||||||
@@ -254,6 +257,8 @@ sudo -u bat /opt/bluearchive-toolkit/bin/bat stop --state-dir /var/lib/bluearchi
|
|||||||
|
|
||||||
不要同时运行 systemd `--watch` 和 standalone `--daemon` 指向同一个 `--output`。二者都会被资源锁和 live daemon 互斥保护,但生产运维上应保持单一 owner。
|
不要同时运行 systemd `--watch` 和 standalone `--daemon` 指向同一个 `--output`。二者都会被资源锁和 live daemon 互斥保护,但生产运维上应保持单一 owner。
|
||||||
|
|
||||||
|
如果同一台服务器还要运行 `bat-api`,必须让 Rust `bat` 以能提供 `bat.sock` 的 RPC 形态运行,并让 `bat-api` 通过该 socket 获取当前 `resource_root`。这种部署见模式 4;不要把 `BAT_API_RESOURCE_ROOT` 当作生产主配置。
|
||||||
|
|
||||||
### 日志和状态路径
|
### 日志和状态路径
|
||||||
|
|
||||||
systemd 模式:
|
systemd 模式:
|
||||||
@@ -262,9 +267,11 @@ systemd 模式:
|
|||||||
- 当前可读 release:`/var/lib/bluearchive-toolkit/official/current`
|
- 当前可读 release:`/var/lib/bluearchive-toolkit/official/current`
|
||||||
- 资源状态:`/var/lib/bluearchive-toolkit/official/current/official-sync-snapshot.json`
|
- 资源状态:`/var/lib/bluearchive-toolkit/official/current/official-sync-snapshot.json`
|
||||||
- 下载 manifest:`/var/lib/bluearchive-toolkit/official/current/official-download-manifest.json`
|
- 下载 manifest:`/var/lib/bluearchive-toolkit/official/current/official-download-manifest.json`
|
||||||
|
- 解析缓存:`/var/lib/bluearchive-toolkit/official/current/official-parse-cache.json`
|
||||||
- 历史 release:`/var/lib/bluearchive-toolkit/official/versions/<id>`
|
- 历史 release:`/var/lib/bluearchive-toolkit/official/versions/<id>`
|
||||||
- 同步 staging:`/var/lib/bluearchive-toolkit/official/.staging/<id>`
|
- 同步 staging:`/var/lib/bluearchive-toolkit/official/.staging/<id>`
|
||||||
- 资源写锁:`/var/lib/bluearchive-toolkit/official/.official-sync.lock`
|
- 资源写锁:`/var/lib/bluearchive-toolkit/official/.official-sync.lock`
|
||||||
|
- 汉化 release(Patch 发布后):`/var/lib/bluearchive-toolkit/localized/current`
|
||||||
- 运行期目录:`/run/bluearchive-toolkit/`
|
- 运行期目录:`/run/bluearchive-toolkit/`
|
||||||
|
|
||||||
standalone `--daemon` 模式:
|
standalone `--daemon` 模式:
|
||||||
@@ -279,10 +286,10 @@ standalone `--daemon` 模式:
|
|||||||
### 生产维护命令
|
### 生产维护命令
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
sudo -u bat /opt/bluearchive-toolkit/bin/bat refresh --output /var/lib/bluearchive-toolkit/official
|
sudo -u bat /opt/bluearchive-toolkit/bin/bat refresh --output /var/lib/bluearchive-toolkit/official --localized-output /var/lib/bluearchive-toolkit/localized
|
||||||
sudo -u bat /opt/bluearchive-toolkit/bin/bat refresh --force --output /var/lib/bluearchive-toolkit/official
|
sudo -u bat /opt/bluearchive-toolkit/bin/bat refresh --force --output /var/lib/bluearchive-toolkit/official --localized-output /var/lib/bluearchive-toolkit/localized
|
||||||
sudo -u bat /opt/bluearchive-toolkit/bin/bat verify --output /var/lib/bluearchive-toolkit/official
|
sudo -u bat /opt/bluearchive-toolkit/bin/bat verify --output /var/lib/bluearchive-toolkit/official --localized-output /var/lib/bluearchive-toolkit/localized
|
||||||
sudo -u bat /opt/bluearchive-toolkit/bin/bat repair --output /var/lib/bluearchive-toolkit/official
|
sudo -u bat /opt/bluearchive-toolkit/bin/bat repair --output /var/lib/bluearchive-toolkit/official --localized-output /var/lib/bluearchive-toolkit/localized
|
||||||
sudo -u bat /opt/bluearchive-toolkit/bin/bat doctor --output /var/lib/bluearchive-toolkit/official --state-dir /run/bluearchive-toolkit
|
sudo -u bat /opt/bluearchive-toolkit/bin/bat doctor --output /var/lib/bluearchive-toolkit/official --state-dir /run/bluearchive-toolkit
|
||||||
sudo -u bat /opt/bluearchive-toolkit/bin/bat clean-stable --output /var/lib/bluearchive-toolkit/official --state-dir /run/bluearchive-toolkit
|
sudo -u bat /opt/bluearchive-toolkit/bin/bat clean-stable --output /var/lib/bluearchive-toolkit/official --state-dir /run/bluearchive-toolkit
|
||||||
```
|
```
|
||||||
@@ -347,7 +354,149 @@ sudo -u bat tar -C /var/lib/bluearchive-toolkit/official \
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 模式 4:完整生产环境部署
|
## 模式 4:bat-api 资源 bootstrap / 分发服务
|
||||||
|
|
||||||
|
适用场景:真实 Rust `bat` 长期运行在远程服务器,并且同一服务器/容器环境内运行 Go `bat-api`,给客户端、补丁器或上层工具提供启动前资源入口和 CDN path 只读分发。
|
||||||
|
|
||||||
|
核心约束:
|
||||||
|
|
||||||
|
1. `bat-api` 与 Rust `bat` 同环境部署,至少要能访问同一个 Unix socket 和同一个已发布资源文件系统。
|
||||||
|
2. 当前资源目录由 `bat.sock` RPC 返回的 `resource_root` 决定;生产不要在 `bat-api` 配置里写死 `BAT_API_RESOURCE_ROOT`。
|
||||||
|
3. `BAT_API_RESOURCE_ROOT` 只用于本地 fixture、临时只读诊断或 RPC 不可用时的应急验证。
|
||||||
|
4. `bat.sock` 只在服务器本机使用,不通过公网暴露;对外只发布 HTTP `bat-api`,生产建议放在反向代理和 TLS 后面。
|
||||||
|
5. 本地开发环境不需要、也不应全量运行 `bat`;使用 Go 单测、fixture release 或远程服务器联调。
|
||||||
|
|
||||||
|
### 构建和安装 bat-api
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make build-go-api
|
||||||
|
|
||||||
|
VERSION="$(git rev-parse --short HEAD)"
|
||||||
|
sudo install -d -o root -g root -m 0755 \
|
||||||
|
/opt/bluearchive-toolkit/releases/"${VERSION}" \
|
||||||
|
/opt/bluearchive-toolkit/bin
|
||||||
|
sudo install -o root -g root -m 0755 \
|
||||||
|
bin/bat-api \
|
||||||
|
/opt/bluearchive-toolkit/releases/"${VERSION}"/bat-api
|
||||||
|
sudo ln -sfn \
|
||||||
|
/opt/bluearchive-toolkit/releases/"${VERSION}"/bat-api \
|
||||||
|
/opt/bluearchive-toolkit/bin/bat-api
|
||||||
|
/opt/bluearchive-toolkit/bin/bat-api --help
|
||||||
|
```
|
||||||
|
|
||||||
|
如果 Rust `bat` 和 Go `bat-api` 使用同一个 release 目录发布,也可以把二者放在同一个 `<version-or-git-sha>` 目录下,分别通过 `/opt/bluearchive-toolkit/bin/bat` 和 `/opt/bluearchive-toolkit/bin/bat-api` 暴露稳定 symlink。
|
||||||
|
|
||||||
|
### bat 侧前置条件
|
||||||
|
|
||||||
|
`bat-api` 依赖 live RPC,而不是直接读取 daemon 状态文件。部署 `bat-api` 前,远程服务器上应已有 socket 形态的 Rust `bat`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo -u bat /opt/bluearchive-toolkit/bin/bat \
|
||||||
|
--auto-discover \
|
||||||
|
--output /var/lib/bluearchive-toolkit/official \
|
||||||
|
--localized-output /var/lib/bluearchive-toolkit/localized \
|
||||||
|
--state-dir /var/lib/bluearchive-toolkit/daemon-state \
|
||||||
|
--daemon
|
||||||
|
|
||||||
|
sudo -u bat /opt/bluearchive-toolkit/bin/bat status \
|
||||||
|
--state-dir /var/lib/bluearchive-toolkit/daemon-state
|
||||||
|
```
|
||||||
|
|
||||||
|
确认 socket 存在:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo -u bat test -S /var/lib/bluearchive-toolkit/daemon-state/bat.sock
|
||||||
|
```
|
||||||
|
|
||||||
|
不要同时再运行一个 `--watch` service 指向 `/var/lib/bluearchive-toolkit/official`。如果当前服务器已经部署了 `bluearchive-toolkit-official-sync.service` 的纯同步 `--watch` 模式,需要先切换为 socket/RPC 形态,再启用 `bat-api`。
|
||||||
|
|
||||||
|
### 安装 bat-api systemd unit
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo install -o root -g root -m 0644 \
|
||||||
|
deployments/systemd/bluearchive-toolkit-bat-api.service \
|
||||||
|
/etc/systemd/system/bluearchive-toolkit-bat-api.service
|
||||||
|
sudo install -o root -g root -m 0644 \
|
||||||
|
deployments/systemd/bat-api.env.example \
|
||||||
|
/etc/bluearchive-toolkit/bat-api.env
|
||||||
|
sudo systemctl daemon-reload
|
||||||
|
sudo systemctl enable --now bluearchive-toolkit-bat-api.service
|
||||||
|
```
|
||||||
|
|
||||||
|
默认配置只监听本机:
|
||||||
|
|
||||||
|
```env
|
||||||
|
BAT_API_LISTEN=127.0.0.1:18080
|
||||||
|
BAT_API_PUBLIC_BASE_URL=http://127.0.0.1:18080
|
||||||
|
BAT_API_SOCKET=/var/lib/bluearchive-toolkit/daemon-state/bat.sock
|
||||||
|
BAT_API_REFRESH_INTERVAL=1m
|
||||||
|
BAT_API_ACCESS_LOG=true
|
||||||
|
BAT_API_RATE_LIMIT_RPS=30
|
||||||
|
BAT_API_RATE_LIMIT_BURST=120
|
||||||
|
```
|
||||||
|
|
||||||
|
生产反向代理公开后,把 `BAT_API_PUBLIC_BASE_URL` 改成客户端实际访问的 HTTPS 根,例如:
|
||||||
|
|
||||||
|
```env
|
||||||
|
BAT_API_PUBLIC_BASE_URL=https://assets.example.com
|
||||||
|
```
|
||||||
|
|
||||||
|
面对玩家分发时还应通过 secret manager 或 systemd credential 注入:
|
||||||
|
|
||||||
|
```env
|
||||||
|
BAT_API_AUTH_TOKEN=<secret>
|
||||||
|
BAT_API_AUTH_QUERY_PARAM=bat_token
|
||||||
|
BAT_API_AUTH_EXEMPT_PATHS=/healthz,/readyz
|
||||||
|
BAT_API_MAX_RESOURCE_LIMIT=1000
|
||||||
|
```
|
||||||
|
|
||||||
|
反代必须强制 HTTPS,并在转发到 `bat-api` 前清洗客户端提交的 `X-Forwarded-For` / `X-Real-IP`。只有确认反代会覆盖这些 header 时,才设置:
|
||||||
|
|
||||||
|
```env
|
||||||
|
BAT_API_TRUST_PROXY_HEADERS=true
|
||||||
|
```
|
||||||
|
|
||||||
|
否则保持默认 `false`,`bat-api` 会按 TCP peer IP 做限流和日志归因。应用层访问日志只记录 path,不记录 query string,避免 query token 进入日志。动态 JSON 响应使用 `Cache-Control: no-store`;CDN 字节路径仍使用长期 immutable 缓存。
|
||||||
|
|
||||||
|
不要在生产 env 里设置 `BAT_API_RESOURCE_ROOT`。`bat-api` 会按 `BAT_API_REFRESH_INTERVAL` 周期通过 RPC 重新读取 `catalog.status` / `resource.manifest`,从而跟随 Rust `bat` 切换 `current -> versions/<id>`。
|
||||||
|
|
||||||
|
### 健康检查
|
||||||
|
|
||||||
|
```bash
|
||||||
|
systemctl status bluearchive-toolkit-bat-api.service
|
||||||
|
journalctl -u bluearchive-toolkit-bat-api.service -f
|
||||||
|
curl -fsS http://127.0.0.1:18080/healthz
|
||||||
|
curl -fsS http://127.0.0.1:18080/readyz
|
||||||
|
curl -fsS http://127.0.0.1:18080/v1/bootstrap
|
||||||
|
curl -fsS http://127.0.0.1:18080/v1/launcher/bootstrap
|
||||||
|
curl -fsS http://127.0.0.1:18080/api-launcher-jp.yo-star.com/api/launcher/game/config
|
||||||
|
curl -fsS http://127.0.0.1:18080/openapi.yaml
|
||||||
|
curl -fsS http://127.0.0.1:18080/admin/
|
||||||
|
```
|
||||||
|
|
||||||
|
`/healthz` 是 liveness,固定返回服务存活状态,并包含最近一次 RPC refresh 的开始时间、成功时间、耗时、warning 和错误摘要。`/readyz` 是 readiness,当前没有可分发 release 时返回 `503`。`rpc_available=true` 且 `ready=true` 表示 `bat-api` 已经通过 RPC 发现可分发 release;`ready=false` 时,先检查 `bat.sock`、Rust `bat status`、`resource_root` 是否存在,以及 `official-download-manifest.json` 中的文件是否仍在磁盘上。
|
||||||
|
|
||||||
|
`/v1/launcher/bootstrap` 和 `/api-launcher-jp.yo-star.com/api/launcher/...` 只用于 launcher 资源 metadata / GameMainConfig 引导兼容。它们从 Rust `bat` 的已发布 snapshot/RPC 派生响应,显式标记不是完整 package update manifest;生产排障时应确认这些响应中的 `scope=resource_bootstrap_only`、`resource_bootstrap_url`、server-info URL 和 client-patch base 是否指向当前 `BAT_API_PUBLIC_BASE_URL`。
|
||||||
|
|
||||||
|
### 本地开发限制
|
||||||
|
|
||||||
|
开发机不能本地全量运行 `bat` 时,不需要伪造生产资源目录。Go 侧改动用单测和 fixture 验证:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make test-go-api
|
||||||
|
make build-go-api
|
||||||
|
BAT_API_SKIP_ENV_FILE=1 go run ./cmd/bat-api \
|
||||||
|
--listen 127.0.0.1:18080 \
|
||||||
|
--public-base-url http://127.0.0.1:18080 \
|
||||||
|
--resource-root internal/api/testdata/release \
|
||||||
|
--refresh-interval 0
|
||||||
|
```
|
||||||
|
|
||||||
|
这条本地命令只验证 HTTP 形态、server-info 改写、CDN path、Range/缓存语义和管理接口;真实全量 release 联调应在远程长期运行的 `bat` 环境里执行。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 模式 5:完整生产环境部署
|
||||||
|
|
||||||
当前不可用。API Server、数据库迁移、Web 管理后台和发布编排尚未实现;不要按完整服务端产品部署本仓库。
|
当前不可用。API Server、数据库迁移、Web 管理后台和发布编排尚未实现;不要按完整服务端产品部署本仓库。
|
||||||
|
|
||||||
|
|||||||
+136
-5
@@ -117,6 +117,10 @@ git push origin feature/your-feature-name
|
|||||||
|
|
||||||
禁止使用 demo、临时实现、硬编码路径或只为当前测试通过的伪实现。确实未完成的能力应写入当前缺口文档,而不是用 `TODO` 或 `FIXME` 隐藏。
|
禁止使用 demo、临时实现、硬编码路径或只为当前测试通过的伪实现。确实未完成的能力应写入当前缺口文档,而不是用 `TODO` 或 `FIXME` 隐藏。
|
||||||
|
|
||||||
|
### 解析模块冻结
|
||||||
|
|
||||||
|
UnityFS / AssetBundle / Addressables / TypeTree 解析当前处于维护冻结。冻结期不得新增解析类型、扩大解析覆盖、开放新的写入型解析 RPC/CLI,或用合成 fixture 宣称新增能力。允许变更仅限编译、测试、clippy、真实运行回归、诊断和文档一致性修复。细则见 `docs/reports/PARSER_FREEZE.md`。
|
||||||
|
|
||||||
### Go
|
### Go
|
||||||
- 遵循 [Effective Go](https://golang.org/doc/effective_go)
|
- 遵循 [Effective Go](https://golang.org/doc/effective_go)
|
||||||
- 使用 `gofmt` 格式化
|
- 使用 `gofmt` 格式化
|
||||||
@@ -141,11 +145,31 @@ git push origin feature/your-feature-name
|
|||||||
cargo fmt --check
|
cargo fmt --check
|
||||||
cargo test --workspace
|
cargo test --workspace
|
||||||
cargo clippy --workspace --all-targets -- -D warnings
|
cargo clippy --workspace --all-targets -- -D warnings
|
||||||
go test ./...
|
make test-go-api
|
||||||
go vet ./...
|
make build-go-api
|
||||||
|
go vet ./internal/api/... ./internal/backendrpc/... ./cmd/bat-api/...
|
||||||
```
|
```
|
||||||
|
|
||||||
Go 产品入口尚未完成,但仓库已有 `internal/backendrpc` Rust daemon RPC client、`cmd/bat` 试验骨架与 `internal/ffi` 兼容包装。提交前应运行 `go test ./...` 和 `go vet ./...`;`internal/backendrpc` 使用 fake transport 覆盖 JSON-RPC envelope、错误和 typed helper,不能把这些测试误认为 Go 产品级 CLI/API 已完成。
|
Go 边界与进度以 `docs/reports/GO_STATUS.md` 为准:
|
||||||
|
|
||||||
|
- **同步/运维命令行** = Rust `bat`(近乎全自动)
|
||||||
|
- **资源 bootstrap/分发服务** = `cmd/bat-api`(`make build-go-api`)
|
||||||
|
- **默认 Go 门禁** = `make test-go-api`(无 FFI)
|
||||||
|
- 试验 CLI 产物为 `bin/bat-go`(`make build-go-cli`),**禁止**与 Rust `bat` 重名
|
||||||
|
- 修改 FFI 时再跑 `make test-go-ffi`
|
||||||
|
|
||||||
|
开发环境不能本地全量运行 Rust `bat` 时,`bat-api` 不需要真实生产资源目录。用 fixture 或 mock RPC 验证服务面;生产联调再连接远程服务器上同环境运行的 `bat.sock`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make test-go-api
|
||||||
|
BAT_API_SKIP_ENV_FILE=1 go run ./cmd/bat-api \
|
||||||
|
--listen 127.0.0.1:18080 \
|
||||||
|
--public-base-url http://127.0.0.1:18080 \
|
||||||
|
--resource-root internal/api/testdata/release \
|
||||||
|
--refresh-interval 0
|
||||||
|
```
|
||||||
|
|
||||||
|
生产默认路径仍是 `--socket` / `BAT_API_SOCKET`,资源根由 Rust `bat` RPC 返回;`--resource-root` 只用于上述 fixture 或应急只读诊断。
|
||||||
|
|
||||||
### 常用聚焦命令
|
### 常用聚焦命令
|
||||||
|
|
||||||
@@ -153,6 +177,7 @@ Go 产品入口尚未完成,但仓库已有 `internal/backendrpc` Rust daemon
|
|||||||
cargo test -p bat-core -- --nocapture
|
cargo test -p bat-core -- --nocapture
|
||||||
cargo test -p bat-adapters -- --nocapture
|
cargo test -p bat-adapters -- --nocapture
|
||||||
cargo test -p bat-ffi -- --nocapture
|
cargo test -p bat-ffi -- --nocapture
|
||||||
|
cargo test -p bat-patch -- --nocapture
|
||||||
cargo test -p bat-infrastructure -- --nocapture
|
cargo test -p bat-infrastructure -- --nocapture
|
||||||
cargo test -p bat-infrastructure --bin bat -- --nocapture
|
cargo test -p bat-infrastructure --bin bat -- --nocapture
|
||||||
cargo clippy -p bat-core -p bat-adapters -p bat-infrastructure --all-targets -- -D warnings
|
cargo clippy -p bat-core -p bat-adapters -p bat-infrastructure --all-targets -- -D warnings
|
||||||
@@ -160,7 +185,7 @@ cargo clippy -p bat-core -p bat-adapters -p bat-infrastructure --all-targets --
|
|||||||
|
|
||||||
官方资源同步、下载、daemon、status、verify 或 repair 相关改动必须至少覆盖 `bat-infrastructure` 和 `bat` 二进制测试。
|
官方资源同步、下载、daemon、status、verify 或 repair 相关改动必须至少覆盖 `bat-infrastructure` 和 `bat` 二进制测试。
|
||||||
|
|
||||||
`bat-ffi` 只是可选无状态 C ABI 兼容层。修改 FFI 导出、JSON schema、错误返回或 `internal/ffi` CGO 包装时必须运行 `cargo test -p bat-ffi -- --nocapture`;未来 Go 产品入口和生产同步默认应通过 `internal/backendrpc` 调用 daemon RPC,或在 one-shot/fallback 场景使用 Rust `bat --json` 进程边界。
|
`bat-ffi` 只是可选无状态 C ABI 兼容层。修改 FFI 时必须运行 `cargo test -p bat-ffi -- --nocapture`。Go 服务层默认经 `internal/backendrpc` 调 daemon;同步任务由 Rust `bat` 执行,不由 Go 试验 CLI 承担。
|
||||||
|
|
||||||
### 集成测试
|
### 集成测试
|
||||||
|
|
||||||
@@ -186,7 +211,113 @@ cargo run -p bat-infrastructure --bin bat -- \
|
|||||||
--dry-run
|
--dry-run
|
||||||
```
|
```
|
||||||
|
|
||||||
开发环境真实下载默认写入 `./bat-resources`;如果要覆盖,必须使用 `/tmp` 或其他隔离目录,不要写入现有资源目录。
|
开发环境真实官方资源下载默认写入 `./bat-resources`;汉化产物默认写入独立的 `./bat-localized`。如果要覆盖,官方原版资源使用 `--output` / `BAT_OUTPUT`,汉化产物使用 `--localized-output` / `BAT_LOCALIZED_OUTPUT`。两者都必须使用 `/tmp` 或其他隔离目录,不要写入现有资源目录,也不要把汉化输出覆盖到官方原版资源目录。
|
||||||
|
|
||||||
|
官方 release 拉取并校验完成后会在当前 release 根目录维护
|
||||||
|
`official-resource-changes.json`、`crowdin-translation-handoff.json`、
|
||||||
|
`official-parse-cache.json` 和 `official-textunit-index.json`,随后从
|
||||||
|
Added/Modified 资源、parse cache 与 TextUnit 明细索引派生
|
||||||
|
`official-textunit-tasks.json` 和 `crowdin-textunit-queue.json`。本地已有旧完整
|
||||||
|
版本时,新版本发布后会先按 manifest destination 对比旧/新 release,只把新增和
|
||||||
|
内容变更的资源写入解析与 Crowdin handoff;删除资源只记录差异,不进入翻译队列。
|
||||||
|
up-to-date 轮询发现本地文件、解析缓存、TextUnit 明细索引和 TextUnit 队列未变时不会重复解析。
|
||||||
|
Crowdin 队列当前只落本地文件,不发网络请求。
|
||||||
|
|
||||||
|
需要把已校验官方 release 导入 CAS + `ResourceRepository` 时,显式启用:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo run -p bat-infrastructure --bin bat -- \
|
||||||
|
--auto-discover \
|
||||||
|
--import-repository \
|
||||||
|
--import-cas-root /tmp/bat-test.cas \
|
||||||
|
--import-resource-db /tmp/bat-test-resources.sqlite
|
||||||
|
```
|
||||||
|
|
||||||
|
对应 `.env` / 环境变量键为 `BAT_IMPORT_REPOSITORY`、
|
||||||
|
`BAT_IMPORT_CAS_ROOT` 和 `BAT_IMPORT_RESOURCE_DB`。只读查询命令:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo run -p bat-infrastructure --bin bat -- parse-status
|
||||||
|
cargo run -p bat-infrastructure --bin bat -- parse-text-units --limit 50
|
||||||
|
cargo run -p bat-infrastructure --bin bat -- parse-errors --limit 50
|
||||||
|
cargo run -p bat-infrastructure --bin bat -- localized-status
|
||||||
|
cargo run -p bat-infrastructure --bin bat -- resource-index --limit 50
|
||||||
|
```
|
||||||
|
|
||||||
|
`parse-status` 会额外显示 TextUnit 明细索引和队列摘要;`parse-text-units` /
|
||||||
|
`parse-errors` 可按 destination、archive entry、path id、class id、field path
|
||||||
|
和 format 分页查询当前官方 release 的 TextUnit 明细与解析错误;
|
||||||
|
`resource-index` 返回的资源 JSON 包含 release、平台、bundle path、TextAsset 和 TextUnit metadata;
|
||||||
|
`localized-status` 只有在 `localized-version-state.json`、`current` symlink 和
|
||||||
|
`localized-patch-manifest.json` 都匹配当前官方 release 时才返回 `localized`。
|
||||||
|
|
||||||
|
文件级写入命令只处理显式输入/输出文件,不切换官方或汉化 release:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo run -p bat-infrastructure --bin bat -- patch-apply \
|
||||||
|
--patch-kind text \
|
||||||
|
--source-file /tmp/bat-source.txt \
|
||||||
|
--patch-file /tmp/bat-source.text-patch.json \
|
||||||
|
--target-file /tmp/bat-target.txt
|
||||||
|
|
||||||
|
cargo run -p bat-infrastructure --bin bat -- unityfs-patch-text-asset \
|
||||||
|
--bundle-file /tmp/source.bundle \
|
||||||
|
--serialized-file CAB-Example \
|
||||||
|
--object-path-id 1 \
|
||||||
|
--replacement-file /tmp/replacement.bytes \
|
||||||
|
--target-file /tmp/target.bundle
|
||||||
|
|
||||||
|
cargo run -p bat-infrastructure --bin bat -- unityfs-patch-string-field \
|
||||||
|
--bundle-file /tmp/source.bundle \
|
||||||
|
--serialized-file CAB-Example \
|
||||||
|
--object-path-id 1 \
|
||||||
|
--string-field-path message \
|
||||||
|
--replacement-text "老师" \
|
||||||
|
--target-file /tmp/target.bundle
|
||||||
|
|
||||||
|
cargo run -p bat-infrastructure --bin bat -- unityfs-patch-field \
|
||||||
|
--bundle-file /tmp/source.bundle \
|
||||||
|
--serialized-file CAB-Example \
|
||||||
|
--object-path-id 1 \
|
||||||
|
--field-path scores[1] \
|
||||||
|
--expected-json '{"kind":"signed","value":20}' \
|
||||||
|
--replacement-json '{"kind":"signed","value":42}' \
|
||||||
|
--target-file /tmp/target.bundle
|
||||||
|
|
||||||
|
cargo run -p bat-infrastructure --bin bat -- unityfs-patch-field \
|
||||||
|
--bundle-file /tmp/source.bundle \
|
||||||
|
--serialized-file CAB-Example \
|
||||||
|
--object-path-id 1 \
|
||||||
|
--field-path difficulty \
|
||||||
|
--expected-json '{"kind":"enum","value":{"type_name":"ScenarioDifficulty","storage_type":"int","value":2}}' \
|
||||||
|
--replacement-json '{"kind":"enum","value":{"type_name":"ScenarioDifficulty","storage_type":"int","value":3}}' \
|
||||||
|
--target-file /tmp/target.bundle
|
||||||
|
|
||||||
|
cargo run -p bat-infrastructure --bin bat -- unityfs-patch-field \
|
||||||
|
--bundle-file /tmp/source.bundle \
|
||||||
|
--serialized-file CAB-Example \
|
||||||
|
--object-path-id 1 \
|
||||||
|
--field-path target_layers \
|
||||||
|
--expected-json '{"kind":"bit_field","value":{"type_name":"LayerMask","storage_type":"UInt32","bits":5}}' \
|
||||||
|
--replacement-json '{"kind":"bit_field","value":{"type_name":"LayerMask","storage_type":"UInt32","bits":9}}' \
|
||||||
|
--target-file /tmp/target.bundle
|
||||||
|
|
||||||
|
cargo run -p bat-infrastructure --bin bat -- unityfs-patch-field \
|
||||||
|
--bundle-file /tmp/source.bundle \
|
||||||
|
--serialized-file CAB-Example \
|
||||||
|
--object-path-id 1 \
|
||||||
|
--field-path messages \
|
||||||
|
--replacement-json '{"kind":"array","value":[{"kind":"string","value":"你好"},{"kind":"string","value":"老师"}]}' \
|
||||||
|
--target-file /tmp/target.bundle
|
||||||
|
|
||||||
|
cargo run -p bat-infrastructure --bin bat -- unityfs-patch-field \
|
||||||
|
--bundle-file /tmp/source.bundle \
|
||||||
|
--serialized-file CAB-Example \
|
||||||
|
--object-path-id 1 \
|
||||||
|
--field-path texts \
|
||||||
|
--replacement-json '{"kind":"map","value":[{"kind":"object","value":[{"name":"first","value":{"kind":"string","value":"jp"}},{"name":"second","value":{"kind":"string","value":"你好"}}]}]}' \
|
||||||
|
--target-file /tmp/target.bundle
|
||||||
|
```
|
||||||
|
|
||||||
生产或 CI 环境不得依赖安装官方启动器。需要启动器信息时,只能分析启动器资源、官方 manifest 或公开更新数据,并将解析结果固化为可验证流程。
|
生产或 CI 环境不得依赖安装官方启动器。需要启动器信息时,只能分析启动器资源、官方 manifest 或公开更新数据,并将解析结果固化为可验证流程。
|
||||||
|
|
||||||
|
|||||||
@@ -45,7 +45,7 @@ target/release/bat \
|
|||||||
--watch
|
--watch
|
||||||
```
|
```
|
||||||
|
|
||||||
默认资源输出目录是 `./bat-resources`,默认后台状态目录是 `/tmp/bat-pid`。资源输出目录是发布根目录:非 dry-run 同步先写 `<output>/.staging/<id>`,校验完成后移动到 `<output>/versions/<id>`,再原子切换 `<output>/current` symlink;生产读取方应读取 `current`。后台状态目录会保存 `bat.sock`、`bat.pid`、`bat-status.json`、`bat-daemon.log`、`bat-events.jsonl` 和短生命周期的 `bat-control.lock`;其中 `bat.sock` 是 live daemon 的 Unix socket JSON-RPC 控制通道,`bat-events.jsonl` 是带轮转的结构化 JSONL 事件日志,`bat-status.json` 保存最后成功时间、下次检查时间、最后错误摘要和当前下载进度,`bat-control.lock` 串行化 `status/stop/restart/reload/logs/refresh/repair` 等控制命令。生产资源输出目录必须是独立目录;需要覆盖时用 `--output <资源目录>`,不要使用已有游戏客户端目录、官方启动器安装目录、人工维护资源目录,或开发机上的 `/home/wanye/D/BlueArchive`。
|
默认官方原版资源输出目录是 `./bat-resources`,默认汉化产物目录是 `./bat-localized`,默认后台状态目录是 `/tmp/bat-pid`。官方资源输出目录是发布根目录:非 dry-run 同步先写 `<output>/.staging/<id>`,校验完成后移动到 `<output>/versions/<id>`,再原子切换 `<output>/current` symlink;生产读取方应读取 `current`。后台状态目录会保存 `bat.sock`、`bat.pid`、`bat-status.json`、`bat-daemon.log`、`bat-events.jsonl` 和短生命周期的 `bat-control.lock`;其中 `bat.sock` 是 live daemon 的 Unix socket JSON-RPC 控制通道,`bat-events.jsonl` 是带轮转的结构化 JSONL 事件日志,`bat-status.json` 保存最后成功时间、下次检查时间、最后错误摘要和当前下载进度,`bat-control.lock` 串行化 `status/stop/restart/reload/logs/refresh/repair` 等控制命令。生产官方资源目录和汉化产物目录都必须是独立目录;需要覆盖官方目录时用 `--output <资源目录>`,需要覆盖汉化目录时用 `--localized-output <目录>` 或 `BAT_LOCALIZED_OUTPUT`,不要使用已有游戏客户端目录、官方启动器安装目录、人工维护资源目录,或开发机上的 `/home/wanye/D/BlueArchive`。
|
||||||
|
|
||||||
同步流程会拒绝危险输出目录、路径逃逸和现有 symlink 路径组件;下载目标、`.part`、manifest、snapshot、PID、status、log 和控制锁文件不会跟随 symlink,daemon 状态类文件默认以 `0600` 权限创建。
|
同步流程会拒绝危险输出目录、路径逃逸和现有 symlink 路径组件;下载目标、`.part`、manifest、snapshot、PID、status、log 和控制锁文件不会跟随 symlink,daemon 状态类文件默认以 `0600` 权限创建。
|
||||||
|
|
||||||
@@ -97,9 +97,9 @@ Linux 生产运行时链路只走官方日服 HTTP 资源,不安装、不启
|
|||||||
2. 请求官方 `server-info`。
|
2. 请求官方 `server-info`。
|
||||||
3. 生成 Windows + Android 的官方资源 discovery 端点。
|
3. 生成 Windows + Android 的官方资源 discovery 端点。
|
||||||
4. 拉取 seed catalog,生成完整官方 pull plan。
|
4. 拉取 seed catalog,生成完整官方 pull plan。
|
||||||
5. dry-run 只输出 URL;非 dry-run 下载全部官方 URL 到 staging,验收完成后原子发布到 `current`。
|
5. dry-run 只输出 URL;非 dry-run 下载全部官方 URL 到 staging,验收完成后原子发布到 `current`,并在 release 中写入 `official-launcher-bootstrap.json`。
|
||||||
|
|
||||||
`--auto-discover` 会下载官方 metadata,并按官方 manifest 临时获取 `resources.assets` 解析 `GameMainConfig`;旧 ZIP manifest 才会下载临时 game zip。该流程不会安装官方启动器,也不会执行官方启动器进程。`--launcher-bootstrap` 只是旧命名兼容别名,新流程不要再推荐使用。
|
`--auto-discover` 会下载官方 metadata,记录 launcher API 返回的 game config、CDN config、remote manifest 文件列表和选中的 `resources.assets` 来源,并按官方 manifest 临时获取 `resources.assets` 解析 `GameMainConfig`;旧 ZIP manifest 才会下载临时 game zip。该流程不会安装官方启动器,也不会执行官方启动器进程。`--launcher-bootstrap` 只是旧命名兼容别名,新流程不要再推荐使用。
|
||||||
|
|
||||||
## 2. 可选 metadata 审计
|
## 2. 可选 metadata 审计
|
||||||
|
|
||||||
@@ -198,11 +198,13 @@ cargo run -p bat-infrastructure --example official_pull_plan -- \
|
|||||||
- 官方 seed `.hash` 校验失败会让本轮失败,并清理对应本地 manifest 条目;下一轮会继续把这类文件视为需要 repair,而不是把失败产物当作健康缓存复用。
|
- 官方 seed `.hash` 校验失败会让本轮失败,并清理对应本地 manifest 条目;下一轮会继续把这类文件视为需要 repair,而不是把失败产物当作健康缓存复用。
|
||||||
- curl 默认自动检测本地代理环境;也可以用 `--proxy <URL>` 显式指定代理,或用 `--no-proxy` 强制直连。代理决策会进入 progress log、daemon log 和 `doctor` 诊断输出。
|
- curl 默认自动检测本地代理环境;也可以用 `--proxy <URL>` 显式指定代理,或用 `--no-proxy` 强制直连。代理决策会进入 progress log、daemon log 和 `doctor` 诊断输出。
|
||||||
- curl 失败会按类型分类:403/404/普通 4xx 不重试,5xx、429、DNS、连接、超时、中断和网络类错误按尝试次数重试。
|
- curl 失败会按类型分类:403/404/普通 4xx 不重试,5xx、429、DNS、连接、超时、中断和网络类错误按尝试次数重试。
|
||||||
|
- 官方维护或大版本发布窗口可能出现启动器/server-info 已经给出新版本和新 `AddressablesCatalogUrlRoot`,但 client-patch CDN 的 seed marker 或必需 seed catalog 尚未开放的状态。此时单次运行会输出 `update_status=waiting_for_official_resources`、`waiting_for_official_resources=true` 和 `unavailable_endpoints`;不会进入 staging、不会写入 `failed_versions`、不会切换 `current`。watch/daemon 会把状态置为 `waiting`,按 `--error-retry` / `BAT_ERROR_RETRY_SECONDS`(默认 60 秒)继续探测。
|
||||||
- 单个 URL 最终失败后会写入 `official-download-quarantine.json`,progress log、daemon status 和 `bat-events.jsonl` 会记录失败类型、HTTP 状态、是否可重试、尝试次数和 quarantine 状态。
|
- 单个 URL 最终失败后会写入 `official-download-quarantine.json`,progress log、daemon status 和 `bat-events.jsonl` 会记录失败类型、HTTP 状态、是否可重试、尝试次数和 quarantine 状态。
|
||||||
- quarantine 项会跳过本轮发布并让同步失败,避免把不完整 staging 发布到 `current`;下一轮 repair/refresh 成功后会清理对应 quarantine 条目。
|
- quarantine 项会跳过本轮发布并让同步失败,避免把不完整 staging 发布到 `current`;下一轮 repair/refresh 成功后会清理对应 quarantine 条目。
|
||||||
- 失败或中断后的 staging 不会无条件丢弃:如果 version-state 记录的失败版本和本轮远端元数据匹配,且 staging 目录仍安全存在,下一轮会复用该 staging;已通过 manifest 校验的文件会跳过,缺失、损坏、无 manifest 或官方 seed `.hash` 需要刷新的 URL 会重新下载。
|
- 失败或中断后的 staging 不会无条件丢弃:如果 version-state 记录的失败版本和本轮远端元数据匹配,且 staging 目录仍安全存在,下一轮会复用该 staging;已通过 manifest 校验的文件会跳过,缺失、损坏、无 manifest 或官方 seed `.hash` 需要刷新的 URL 会重新下载。
|
||||||
- 旧 launcher 包或 `resources.assets` 下载路径使用官方 launcher CDN 配置,primary CDN 失败后会切换官方 backup CDN;资源 patch host 当前只使用 server-info 返回的官方 client-patch host,不猜测非官方镜像。
|
- 旧 launcher 包或 `resources.assets` 下载路径使用官方 launcher CDN 配置,primary CDN 失败后会切换官方 backup CDN;资源 patch host 当前只使用 server-info 返回的官方 client-patch host,不猜测非官方镜像。
|
||||||
- 远端和本地都一致:单次模式输出 `update_status=up_to_date`,watch 模式默认静默并等待下次检查。
|
- 远端和本地都一致:单次模式输出 `update_status=up_to_date`,watch 模式默认静默并等待下次检查。
|
||||||
|
- 远端 metadata 已更新但资源端尚未开放:单次模式输出 `update_status=waiting_for_official_resources`,watch/daemon 模式保留现有资源并短间隔重试。
|
||||||
- 有远端变化或本地 repair:生成 pull plan,下载完整官方资源到 staging,成功后更新 snapshot 并原子发布到 `current`。
|
- 有远端变化或本地 repair:生成 pull plan,下载完整官方资源到 staging,成功后更新 snapshot 并原子发布到 `current`。
|
||||||
- 非 dry-run 会维护 `<output>/official-version-state.json`:开始下载后写入 `in_progress_version`,发布成功后写入 `current_completed_version` 和 `previous_available_version`,失败或中断后写入 `failed_versions`。同一 app version、bundle version 和 Addressables root 的失败只保留最新一条;同一版本开始重新拉取或后续发布成功时会清理对应失败记录。重新拉取同一失败版本时会复用安全存在的失败 staging,不会因为 `publish_id` 变化从空目录重新开始。
|
- 非 dry-run 会维护 `<output>/official-version-state.json`:开始下载后写入 `in_progress_version`,发布成功后写入 `current_completed_version` 和 `previous_available_version`,失败或中断后写入 `failed_versions`。同一 app version、bundle version 和 Addressables root 的失败只保留最新一条;同一版本开始重新拉取或后续发布成功时会清理对应失败记录。重新拉取同一失败版本时会复用安全存在的失败 staging,不会因为 `publish_id` 变化从空目录重新开始。
|
||||||
- `--dry-run`:只报告本次是否会下载,不写 snapshot;如果 cache miss,也不会写入新的 bootstrap cache。
|
- `--dry-run`:只报告本次是否会下载,不写 snapshot;如果 cache miss,也不会写入新的 bootstrap cache。
|
||||||
@@ -210,13 +212,23 @@ cargo run -p bat-infrastructure --example official_pull_plan -- \
|
|||||||
- 真实更新会输出 `downloaded_count`、`resumed_count`、`skipped_count`、`transferred_bytes`、`official_seed_hash_verified_count`。
|
- 真实更新会输出 `downloaded_count`、`resumed_count`、`skipped_count`、`transferred_bytes`、`official_seed_hash_verified_count`。
|
||||||
- 校验报告分层输出 `official_seed_hash_verified_count`、`local_manifest_verified_count`、`addressables_marker_checked_count`、`unverified_marker_count`。
|
- 校验报告分层输出 `official_seed_hash_verified_count`、`local_manifest_verified_count`、`addressables_marker_checked_count`、`unverified_marker_count`。
|
||||||
- 下载阶段复用同一套本地清单、ZIP 结构校验和 `.part` 续传逻辑;没有清单或校验不匹配的文件会重新下载。
|
- 下载阶段复用同一套本地清单、ZIP 结构校验和 `.part` 续传逻辑;没有清单或校验不匹配的文件会重新下载。
|
||||||
|
- 非 dry-run 且启用 `--auto-discover` 时,成功发布的 release 会包含 `official-launcher-bootstrap.json`;up-to-date 轮询发现当前 release 缺少该文件时会补写。官方 launcher/server-info 已更新但 client-patch 资源尚未开放时,不切换 `current`,只在输出根写入 `official-launcher-bootstrap.pending.json` 作为维护期证据。
|
||||||
|
- 校验和发布完成后会先对比上一完整 release 与当前 release 的 `official-download-manifest.json`,写出 `<output>/current/official-resource-changes.json` 和 `<output>/current/crowdin-translation-handoff.json`。同一 destination 只有 size 或 BLAKE3 改变才算 modified;仅 URL/CDN 根变化但内容一致不会触发解析/翻译候选。新增+变更资源进入解析和 Crowdin 翻译 handoff,删除资源只进入差异记录;当前不会直接调用 Crowdin API。
|
||||||
|
- 随后会刷新 `<output>/current/official-parse-cache.json`。解析缓存从 `official-download-manifest.json` 的全部条目出发,处理直接 UnityFS bundle 和 zip 内 UnityFS 条目;catalog、hash、媒体等非 UnityFS 文件记录为不支持,不视为同步失败。新 release 会刷新解析缓存;远端和本地都 up-to-date 且已有有效解析缓存时只读取摘要,不重复解析。
|
||||||
|
- 需要将已校验官方 release 导入 CAS + SQLite ResourceRepository 时,使用 `--import-repository` 或 `.env` 中 `BAT_IMPORT_REPOSITORY=1`;默认 CAS 为 `<output>/.cas`,默认索引为 `<output>/resources.sqlite`,可用 `--import-cas-root` / `BAT_IMPORT_CAS_ROOT` 和 `--import-resource-db` / `BAT_IMPORT_RESOURCE_DB` 覆盖。`resource.index` RPC 可查询现有索引,索引不存在时返回 `available=false`,不会创建空库。
|
||||||
|
- 官方同步报告中的 `localized_release_status=not_localized` 表示原版资源已发布、汉化资源未发布,这是当前官方同步阶段的正常完成状态;后续 Patch 发布完成后才应切换为 `localized`,表示原版和汉化两套资源都已发布。
|
||||||
|
|
||||||
资源同步状态文件默认分布如下:
|
资源同步状态文件默认分布如下:
|
||||||
|
|
||||||
- `<output>/current/official-sync-snapshot.json`:上一次成功同步的 v2 snapshot,包含 app version、connection group、bundle version、addressables root、endpoint URL、官方 seed `.hash` 内容、Addressables `catalog_*.hash` marker、launcher metadata 摘要和 `GameMainConfig` 摘要。
|
- `<output>/current/official-sync-snapshot.json`:上一次成功同步的 v2 snapshot,包含 app version、connection group、bundle version、addressables root、endpoint URL、官方 seed `.hash` 内容、Addressables `catalog_*.hash` marker、launcher metadata 摘要和 `GameMainConfig` 摘要;launcher metadata 额外包含 remote manifest 文件列表 digest,用于发现同文件数但内容变化的 launcher manifest。
|
||||||
- `<output>/official-bootstrap-cache.json`:`--auto-discover` 的 `GameMainConfig` 解析缓存。launcher metadata 未变时复用缓存;metadata 变化时才通过官方 HTTP 按 manifest 下载必要 `resources.assets` 或旧版 game zip 到临时目录解析。
|
- `<output>/current/official-launcher-bootstrap.json`:随已发布 release versioned 保存的官方 launcher bootstrap 产物,包含 launcher metadata、launcher CDN config、remote manifest 文件列表、选中的 `resources.assets` 来源、`GameMainConfig` 摘要和当前资源上下文。
|
||||||
|
- `<output>/official-launcher-bootstrap.pending.json`:官方 launcher/server-info 已前进但 client-patch seed marker 或必需 seed catalog 尚未开放时写入的待处理 bootstrap 证据;它不代表资源已发布,也不会改变 `current`。
|
||||||
|
- `<output>/official-bootstrap-cache.json`:`--auto-discover` 的 `GameMainConfig` 解析缓存。launcher metadata 与 remote manifest 文件列表 digest 都未变时复用缓存;任一变化时才通过官方 HTTP 按 manifest 下载必要 `resources.assets` 或旧版 game zip 到临时目录解析。
|
||||||
- `<output>/official-version-state.json`:资源发布根目录的持久版本状态,包含当前已完成版本、正在拉取版本、上一个可用版本和失败版本。
|
- `<output>/official-version-state.json`:资源发布根目录的持久版本状态,包含当前已完成版本、正在拉取版本、上一个可用版本和失败版本。
|
||||||
- `<output>/current/official-download-manifest.json`:本地下载强校验清单,记录 URL、相对路径、size 和 BLAKE3。
|
- `<output>/current/official-download-manifest.json`:本地下载强校验清单,记录 URL、相对路径、size 和 BLAKE3。
|
||||||
|
- `<output>/current/official-resource-changes.json`:当前 release 相对上一完整 release 的资源差异,记录新增、变更、删除以及解析/翻译候选计数。
|
||||||
|
- `<output>/current/crowdin-translation-handoff.json`:为后续 Crowdin worker 预留的本地队列,只包含新增+变更资源;它不是 Crowdin API 调用结果。
|
||||||
|
- `<output>/current/official-parse-cache.json`:官方资源发布后的派生解析缓存,记录 bundle/zip 条目解析摘要和缓存复用情况;它不是汉化产物。
|
||||||
- `<output>/current/official-download-quarantine.json` 或当前 staging 下同名文件:下载最终失败的 URL 诊断记录,包含失败类型、HTTP 状态、是否可重试、尝试次数和最后错误。
|
- `<output>/current/official-download-quarantine.json` 或当前 staging 下同名文件:下载最终失败的 URL 诊断记录,包含失败类型、HTTP 状态、是否可重试、尝试次数和最后错误。
|
||||||
|
|
||||||
先 dry-run:
|
先 dry-run:
|
||||||
@@ -251,7 +263,7 @@ cargo run -p bat-infrastructure --bin bat -- \
|
|||||||
--watch
|
--watch
|
||||||
```
|
```
|
||||||
|
|
||||||
后台自动运行使用 `--daemon`。它会启动一个脱离终端的 watch 子进程,资源默认写入 `./bat-resources`,后台控制和状态默认写入 `/tmp/bat-pid`:
|
后台自动运行使用 `--daemon`。它会启动一个脱离终端的 watch 子进程,官方原版资源默认写入 `./bat-resources`,汉化产物默认写入 `./bat-localized`,后台控制和状态默认写入 `/tmp/bat-pid`:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cargo run -p bat-infrastructure --bin bat -- \
|
cargo run -p bat-infrastructure --bin bat -- \
|
||||||
@@ -310,7 +322,7 @@ cargo run -p bat-infrastructure --bin bat -- \
|
|||||||
|
|
||||||
默认平台是 `Windows,Android`,无需显式传 `--platforms`;只有要覆盖默认平台时才传。`--interval` 是正常检查周期,默认 `1h`;watch/daemon 模式还会在每天北京时间(UTC+8)`03:00`、`16:00`、`18:00` 强制执行一次自动刷新,该轮会注入 `force=true`,并且会中断普通 interval 的 sleep。`--error-retry` 是下载、发现或校验失败后的重试周期,默认 `60s`,也可以用 `--error-retry-seconds 60`。CLI 默认启动时向 stderr 打印 `BlueArchiveToolkit` ASCII banner,并把阶段进度日志写到 stderr,包括自动发现、proxy、server-info、marker、catalog、audit、download、snapshot 和 publish 阶段;download 阶段会输出已完成计数和单文件开始/完成状态,下载执行保持顺序处理,已完成计数保持单调不倒退,audit 阶段会输出官方 `.hash`、本地 BLAKE3、需修复项和 ZIP 结构校验结果摘要。daemon 还会写 `bat-events.jsonl` 结构化日志并按大小轮转。命令结果默认以人类可读摘要写到 stdout。需要纯机器输出时加 `--json --no-progress`,需要显式开启进度日志则用 `--progress`;只想关闭横幅但保留日志时可加 `--no-banner`。错误时 stderr 输出 JSON error,watch 模式下错误 JSON 的 `next_retry_seconds` 使用失败重试周期;如果未关闭 progress,错误 JSON 前可能已有 banner 和进度日志。普通错误 exit `1`,资源目录锁冲突 exit `75`,`verify` 或 `doctor` 发现问题也返回非 0。
|
默认平台是 `Windows,Android`,无需显式传 `--platforms`;只有要覆盖默认平台时才传。`--interval` 是正常检查周期,默认 `1h`;watch/daemon 模式还会在每天北京时间(UTC+8)`03:00`、`16:00`、`18:00` 强制执行一次自动刷新,该轮会注入 `force=true`,并且会中断普通 interval 的 sleep。`--error-retry` 是下载、发现或校验失败后的重试周期,默认 `60s`,也可以用 `--error-retry-seconds 60`。CLI 默认启动时向 stderr 打印 `BlueArchiveToolkit` ASCII banner,并把阶段进度日志写到 stderr,包括自动发现、proxy、server-info、marker、catalog、audit、download、snapshot 和 publish 阶段;download 阶段会输出已完成计数和单文件开始/完成状态,下载执行保持顺序处理,已完成计数保持单调不倒退,audit 阶段会输出官方 `.hash`、本地 BLAKE3、需修复项和 ZIP 结构校验结果摘要。daemon 还会写 `bat-events.jsonl` 结构化日志并按大小轮转。命令结果默认以人类可读摘要写到 stdout。需要纯机器输出时加 `--json --no-progress`,需要显式开启进度日志则用 `--progress`;只想关闭横幅但保留日志时可加 `--no-banner`。错误时 stderr 输出 JSON error,watch 模式下错误 JSON 的 `next_retry_seconds` 使用失败重试周期;如果未关闭 progress,错误 JSON 前可能已有 banner 和进度日志。普通错误 exit `1`,资源目录锁冲突 exit `75`,`verify` 或 `doctor` 发现问题也返回非 0。
|
||||||
|
|
||||||
生产可以直接运行 `--watch`,也可以用 `--daemon` 后台运行,或者用 systemd service、容器或 Go 进程守护它。cron/systemd timer 仍可调用单次模式,但不再是 Rust 自动更新的唯一方式。项目是否热更新、热重载或重启进程,由上层业务集成决定。生产资源目录应使用独立输出目录,不要指向现有客户端或人工维护的资源目录;上层读取资源时应读取 `--output/current`,不要读取 `.staging` 或 `versions` 中未切换的目录。非 dry-run 每轮会创建 `--output/.official-sync.lock`,防止并发写同一资源目录;live daemon 还会阻止前台写命令直接修改它正在管理的同一目录。
|
生产可以直接运行 `--watch`,也可以用 `--daemon` 后台运行,或者用 systemd service、容器或 Go 进程守护它。cron/systemd timer 仍可调用单次模式,但不再是 Rust 自动更新的唯一方式。项目是否热更新、热重载或重启进程,由上层业务集成决定。生产官方资源目录应使用独立输出目录,不要指向现有客户端或人工维护的资源目录;上层读取原版资源时应读取 `--output/current`,不要读取 `.staging` 或 `versions` 中未切换的目录。汉化 Patch/导出应写入 `--localized-output`,并保留官方相对目录结构,不能写回 `--output/current`。发布状态分两档:`not_localized` 只发布原版资源、不发布汉化资源;`localized` 发布原版和汉化两套资源。非 dry-run 每轮会创建 `--output/.official-sync.lock`,防止并发写同一官方资源目录;live daemon 还会阻止前台写命令直接修改它正在管理的同一目录。
|
||||||
|
|
||||||
需要只做探测时可以加 `--dry-run`。需要关闭本地 audit 或 repair 时可以显式使用 `--no-audit-local` 或 `--no-repair`,但生产同步默认应保持开启。
|
需要只做探测时可以加 `--dry-run`。需要关闭本地 audit 或 repair 时可以显式使用 `--no-audit-local` 或 `--no-repair`,但生产同步默认应保持开启。
|
||||||
|
|
||||||
|
|||||||
@@ -102,9 +102,95 @@ contract 为准,不应绕过 daemon 状态文件或扩展 `bat-ffi` 作为主
|
|||||||
| `resource.repair` | 已实现 | `null` | `{ "task_id": "...", "kind": "resource.repair" }`。 |
|
| `resource.repair` | 已实现 | `null` | `{ "task_id": "...", "kind": "resource.repair" }`。 |
|
||||||
| `resource.manifest` | 已实现 | `{ "offset": 0, "limit": 100 }` | 当前 download manifest 分页。 |
|
| `resource.manifest` | 已实现 | `{ "offset": 0, "limit": 100 }` | 当前 download manifest 分页。 |
|
||||||
| `resource.list` | 已实现 | `{ "offset": 0, "limit": 100 }` | `resource.manifest` 的兼容别名。 |
|
| `resource.list` | 已实现 | `{ "offset": 0, "limit": 100 }` | `resource.manifest` 的兼容别名。 |
|
||||||
|
| `resource.index` | 已实现 | `{ "offset": 0, "limit": 100, "type": "asset_bundle", "hash": "...", "path_pattern": "*" }` | 当前 `ResourceRepository` 分页/过滤查询。 |
|
||||||
|
|
||||||
`resource.repair` 会开启本地 manifest audit + repair,不继承 `force`。
|
`resource.repair` 会开启本地 manifest audit + repair,不继承 `force`。
|
||||||
`limit` 范围是 `1..=1000`,非法参数返回 `BAT-ERR-700002`。
|
`resource.manifest` / `resource.list` 查询当前已发布 release 的
|
||||||
|
`official-download-manifest.json`;`resource.index` 查询可选导入产生的
|
||||||
|
SQLite `ResourceRepository`,索引不存在时返回 `ok=true` 且
|
||||||
|
`data.available=false`,不会隐式创建数据库。`limit` 范围是 `1..=1000`,
|
||||||
|
非法参数返回 `BAT-ERR-700002`。
|
||||||
|
|
||||||
|
`resource.index` 的 `entries[]` 是 `Resource` JSON,除 `id`、`local_path`、
|
||||||
|
`entry` 外会包含 `metadata`:`official_release_id`、`platform`、
|
||||||
|
`bundle_path`、`archive_entries`、`parse_statuses`、`unity_versions`、
|
||||||
|
`text_assets`、`text_unit_count`、`text_unit_formats` 和
|
||||||
|
`text_unit_error_count` 等字段。旧索引库会通过 `metadata_json` 迁移列得到
|
||||||
|
默认空 metadata。
|
||||||
|
|
||||||
|
官方资源完整新版本发布后,Rust 侧会先比较上一完整 release 与当前 release
|
||||||
|
的 download manifest,并在当前 release 根目录写出:
|
||||||
|
|
||||||
|
- `official-resource-changes.json`:记录 added / modified / removed 资源。
|
||||||
|
同一 destination 只有 size 或 BLAKE3 变化才算 modified;URL 或 CDN root
|
||||||
|
变化但内容一致时不进入解析/翻译候选。
|
||||||
|
- `crowdin-translation-handoff.json`:只包含 added + modified 资源,作为后续
|
||||||
|
Crowdin worker 的稳定本地队列输入;当前 RPC 不直接调用 Crowdin API。
|
||||||
|
- `official-parse-cache.json`:解析缓存。up-to-date 轮询发现本地文件未变且缓存
|
||||||
|
有效时只读取摘要,不重复解析。
|
||||||
|
- `official-textunit-index.json`:TextUnit 明细和解析错误索引。up-to-date 轮询发现
|
||||||
|
本地文件未变且索引有效时复用,不重复解析。
|
||||||
|
- `official-textunit-tasks.json`:只由 added + modified 资源、parse cache 和
|
||||||
|
TextUnit 明细索引派生,记录 TextUnit 任务、跳过原因和解析诊断。
|
||||||
|
- `crowdin-textunit-queue.json`:只包含已产生 TextUnit 的离线任务,预留给后续
|
||||||
|
Crowdin worker;当前不会发出网络请求。
|
||||||
|
|
||||||
|
删除资源只进入 `official-resource-changes.json`,不进入 Crowdin handoff。
|
||||||
|
|
||||||
|
### parse
|
||||||
|
|
||||||
|
| 方法 | 状态 | params | data |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `parse.status` | 已实现 | `null` | 当前官方 release 的解析缓存状态。 |
|
||||||
|
| `parse.text_units` | 已实现 | `{ "offset": 0, "limit": 100, "destination": "*Table*", "archive_entry": "*.bytes", "path_id": 1, "class_id": 114, "field_path": "*Text*", "format": "json" }` | 当前官方 release 的 TextUnit 明细分页。 |
|
||||||
|
| `parse.errors` | 已实现 | `{ "offset": 0, "limit": 100, "destination": "*Table*", "archive_entry": "*.bytes", "path_id": 1, "class_id": 114, "field_path": "*Text*", "format": "json" }` | 当前官方 release 的解析错误分页。 |
|
||||||
|
|
||||||
|
`parse.status` 是只读查询;没有当前 release 或没有解析缓存时返回
|
||||||
|
`ok=true` 且 `data.available=false`。解析缓存来自官方原版资源目录,不读取
|
||||||
|
汉化输出目录。存在 `official-textunit-index.json` 时,响应会包含
|
||||||
|
`textunit_index_available=true`、`textunit_index_path` 和
|
||||||
|
`textunit_index_summary`;存在 `official-textunit-tasks.json` 时,响应会包含
|
||||||
|
`textunit_queue_available=true`、`textunit_task_queue_path` 和
|
||||||
|
`textunit_task_summary`。
|
||||||
|
|
||||||
|
`parse.text_units` / `parse.errors` 是只读查询;没有当前 release 或没有
|
||||||
|
`official-textunit-index.json` 时返回 `ok=true` 且 `data.available=false`。
|
||||||
|
分页参数 `offset` 默认 0,`limit` 默认 100,范围是 `1..=1000`。过滤参数:
|
||||||
|
|
||||||
|
| 字段 | 类型 | 说明 |
|
||||||
|
|---|---|---|
|
||||||
|
| `destination` | string | official download manifest destination,支持 `*` 通配。 |
|
||||||
|
| `archive_entry` | string | zip 内条目,支持 `*` 通配;直接 bundle 通常为 `null`。 |
|
||||||
|
| `path_id` | integer | Unity object path id。 |
|
||||||
|
| `class_id` | integer | Unity class id。 |
|
||||||
|
| `field_path` | string | TypeTree/TextAsset 字段路径,支持 `*` 通配。 |
|
||||||
|
| `format` | string | TextUnit 格式,例如 `json`、`csv`、`tsv`、`plain` 或 `typetree_string`。 |
|
||||||
|
|
||||||
|
`parse.text_units` 的 `entries[]` 会包含 source text、source URL、
|
||||||
|
destination、archive entry、source kind、Unity version、serialized file、
|
||||||
|
path id、class id、field path、字段 offset/byte size、format、asset name 和
|
||||||
|
context。`parse.errors` 的 `entries[]` 会包含 source URL、destination、
|
||||||
|
archive entry、status、serialized file、path id、class id、field path、
|
||||||
|
offset 和 error。TypeTree-covered managed reference 字段会进入结构化字段遍历;
|
||||||
|
完整 managed reference registry 等暂不支持结构会进入解析错误,而不是静默降级为
|
||||||
|
低保真文本。
|
||||||
|
|
||||||
|
### localized
|
||||||
|
|
||||||
|
| 方法 | 状态 | params | data |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `localized.status` | 已实现 | `null` | 汉化发布状态、当前官方 release 匹配关系和汉化输出目录。 |
|
||||||
|
|
||||||
|
`localized.status` 严格按 daemon / `.env` 中的 `BAT_LOCALIZED_OUTPUT` 或
|
||||||
|
`--localized-output` 查询汉化产物目录,不把 `./bat-resources` 与
|
||||||
|
`./bat-localized` 混用。当前支持未汉化发布状态和已汉化发布状态的只读报告。
|
||||||
|
返回 `localized` 的条件是:`localized-version-state.json` 的官方 release ID
|
||||||
|
匹配当前官方 release,`current` symlink 指向汉化发布根下对应的
|
||||||
|
`versions/<id>`,并且该版本目录中的
|
||||||
|
`localized-patch-manifest.json` 存在且 release ID 匹配。响应会返回
|
||||||
|
`patch_manifest_path`、`patch_manifest_available`、
|
||||||
|
`patch_manifest_matches_release`、`patch_file_count`、
|
||||||
|
`patch_text_asset_operation_count` 和 `rollback_previous_current_target`。
|
||||||
|
|
||||||
### catalog
|
### catalog
|
||||||
|
|
||||||
@@ -151,9 +237,53 @@ daemon 重启后仍处于 `queued` 或 `running` 的历史任务会被标记为
|
|||||||
|
|
||||||
### patch / unityfs
|
### patch / unityfs
|
||||||
|
|
||||||
`patch.*` 和 `unityfs.*` 是已规划命名空间,目前返回
|
已开放的文件级写入方法:
|
||||||
`BAT-ERR-700003`。它们依赖后续 `bat-patch`、`bat-assetbundle`
|
|
||||||
引擎,不作为 issue #1 的关闭阻塞项。
|
- `patch.apply`:对显式 `source_path`、`patch_path`、`target_path` 执行
|
||||||
|
Binary/JSON/Text patch apply,`kind` 取值为 `binary`、`json` 或 `text`。
|
||||||
|
- `unityfs.patch_text_asset`:对显式 UnityFS `bundle_path` 中的
|
||||||
|
`serialized_file_path` / `path_id` TextAsset 应用 `replacement_path`,写入
|
||||||
|
`target_path`,可选 `expected_name`。
|
||||||
|
- `unityfs.patch_string_field`:对显式 UnityFS `bundle_path` 中的
|
||||||
|
`serialized_file_path` / `path_id` / `field_path` TypeTree string 字段应用
|
||||||
|
`replacement_text` 或 UTF-8 `replacement_path`,写入 `target_path`,可选
|
||||||
|
`expected_value`。
|
||||||
|
- `unityfs.patch_field`:对显式 UnityFS `bundle_path` 中的
|
||||||
|
`serialized_file_path` / `path_id` / `field_path` TypeTree 字段应用语义
|
||||||
|
`replacement` JSON,写入 `target_path`,可选 `expected_value`。`replacement`
|
||||||
|
使用 `{"kind":"signed","value":42}` 这类 tagged JSON;支持
|
||||||
|
`bool`、`signed`、`unsigned`、`float32`、`float64`、`string`、`bytes`、
|
||||||
|
`enum`、`bit_field`、`p_ptr`、固定 Unity 叶子结构、object 字段组合和 TypeTree schema 支撑的 array/map 整体替换。enum 形如
|
||||||
|
`{"kind":"enum","value":{"type_name":"ScenarioDifficulty","storage_type":"int","value":3}}`;
|
||||||
|
`type_name` 是 TypeTree enum 类型名,`storage_type` 是 backing integer 类型。`LayerMask` / `BitField` 形如
|
||||||
|
`{"kind":"bit_field","value":{"type_name":"LayerMask","storage_type":"UInt32","bits":9}}`。
|
||||||
|
array 形如
|
||||||
|
`{"kind":"array","value":[{"kind":"string","value":"你好"}]}`;map entry 用
|
||||||
|
object 表达,例如
|
||||||
|
`{"kind":"object","value":[{"name":"first","value":{"kind":"string","value":"jp"}}]}`。
|
||||||
|
扩容时复用当前首个元素或 TypeTree data node 的编码 schema;map entry schema
|
||||||
|
变化、unknown 字段和未覆盖的 managed reference registry 变体仍会返回明确错误。
|
||||||
|
固定 Unity 叶子结构使用 raw bits/bytes 表达,例如
|
||||||
|
`{"kind":"float32_struct","value":{"type_name":"Vector3f","values":[1065353216,1073741824,1077936128]}}`
|
||||||
|
或
|
||||||
|
`{"kind":"fixed_bytes","value":{"type_name":"GUID","bytes":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]}}`。
|
||||||
|
|
||||||
|
这些方法同步执行,不进入 `task.*` 队列;输出文件使用临时文件原子写入,响应
|
||||||
|
`data` 会返回 source / patch 或 replacement / target 的 size 与 BLAKE3。`target_path`
|
||||||
|
不能与输入文件相同。
|
||||||
|
|
||||||
|
仍关闭的范围:发布级 `patch build` / `patch rollback`、复杂 UnityFS 语义编辑、
|
||||||
|
`unityfs.inspect`、通用 manifest 驱动 release 切换。调用这些规划方法仍返回
|
||||||
|
`BAT-ERR-700003`。
|
||||||
|
|
||||||
|
CLI 对应关系:
|
||||||
|
|
||||||
|
| CLI | RPC |
|
||||||
|
|---|---|
|
||||||
|
| `bat patch-apply` | `patch.apply` |
|
||||||
|
| `bat unityfs-patch-text-asset` | `unityfs.patch_text_asset` |
|
||||||
|
| `bat unityfs-patch-string-field` | `unityfs.patch_string_field` |
|
||||||
|
| `bat unityfs-patch-field` | `unityfs.patch_field` |
|
||||||
|
|
||||||
## Go 调用边界
|
## Go 调用边界
|
||||||
|
|
||||||
@@ -161,9 +291,24 @@ daemon 重启后仍处于 `queued` 或 `running` 的历史任务会被标记为
|
|||||||
`bat` binary 是人类 CLI 和进程生命周期工具;默认 `refresh` / `repair`
|
`bat` binary 是人类 CLI 和进程生命周期工具;默认 `refresh` / `repair`
|
||||||
在 daemon 可用时也会作为 RPC client 调用同一个 socket。
|
在 daemon 可用时也会作为 RPC client 调用同一个 socket。
|
||||||
|
|
||||||
|
人类 CLI 的只读查询命令与 RPC 对应关系如下:
|
||||||
|
|
||||||
|
| CLI | RPC |
|
||||||
|
|---|---|
|
||||||
|
| `bat parse-status` | `parse.status` |
|
||||||
|
| `bat parse-text-units` | `parse.text_units` |
|
||||||
|
| `bat parse-errors` | `parse.errors` |
|
||||||
|
| `bat localized-status` | `localized.status` |
|
||||||
|
| `bat resource-index` | `resource.index` |
|
||||||
|
|
||||||
|
`bat resource-index` 支持 `--offset`、`--limit`、`--resource-type`、`--hash`
|
||||||
|
和 `--path-pattern`;`bat parse-text-units` / `bat parse-errors` 支持
|
||||||
|
`--offset`、`--limit`、`--destination`、`--archive-entry`、`--path-id`、
|
||||||
|
`--class-id`、`--field-path` 和 `--format`;这些过滤参数不适用于
|
||||||
|
`parse-status` 或 `localized-status`。
|
||||||
|
|
||||||
禁止事项:
|
禁止事项:
|
||||||
|
|
||||||
- Go 服务层不直接读写 `bat-status.json`、`bat-tasks.json` 等 daemon 内部状态文件。
|
- Go 服务层不直接读写 `bat-status.json`、`bat-tasks.json` 等 daemon 内部状态文件。
|
||||||
- Go 服务层不扩展 `bat-ffi` 为主控制面。
|
- Go 服务层不扩展 `bat-ffi` 为主控制面。
|
||||||
- Go 服务层不通过 stdout 解析 `bat status --json` 作为常规调用路径。
|
- Go 服务层不通过 stdout 解析 `bat status --json` 作为常规调用路径。
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,196 @@
|
|||||||
|
# bat-api / Rust bat Contract Fixture Handoff
|
||||||
|
|
||||||
|
更新时间:2026-07-28
|
||||||
|
|
||||||
|
本文用于两个 Codex 窗口之间间接联调 `bat-api` 与 Rust `bat` 的跨语言 contract fixture。它只定义协作协议和验收标准,不包含已审核 fixture。
|
||||||
|
|
||||||
|
## 最小上下文包
|
||||||
|
|
||||||
|
另一个窗口不需要知道本窗口的完整对话,只需要遵守以下上下文:
|
||||||
|
|
||||||
|
- 本次联调对象是 Rust `bat` RPC / snapshot JSON 与 Go `bat-api` mirror struct 的 contract fixture。
|
||||||
|
- 联调不要求本地运行全量长期服务端 `bat`;允许 Rust 侧使用 fixture root 或临时目录走真实代码路径导出 JSON。
|
||||||
|
- fixture 审核前只能放在 `/tmp/bat-contract-fixture/`,不能直接提交到仓库。
|
||||||
|
- Go 侧已经实现 player-facing HTTP 鉴权、限流、访问日志、反代适配、OpenAPI 和 `/admin/` 预留;contract fixture 是剩余跨语言强契约工作。
|
||||||
|
- Go 侧当前相关代码入口:
|
||||||
|
- `internal/api/rpc_release.go`
|
||||||
|
- `internal/api/release_index.go`
|
||||||
|
- `internal/api/responses.go`
|
||||||
|
- `internal/backendrpc/`
|
||||||
|
|
||||||
|
## 背景
|
||||||
|
|
||||||
|
- Rust `bat` 是资源同步、状态发布和 `bat.sock` RPC 的权威实现。
|
||||||
|
- Go `bat-api` 是只读 HTTP bootstrap / 分发服务,消费 Rust RPC 输出和已发布资源目录。
|
||||||
|
- contract fixture 不能由任一侧手写猜测;必须由 Rust 侧真实输出,经归一化和用户审核后,再由 Go 侧固化测试。
|
||||||
|
|
||||||
|
## 非目标
|
||||||
|
|
||||||
|
- 不引入真实玩家账号、登录、网关、鉴权绕过或游戏业务 API fixture。
|
||||||
|
- 不写入开发机绝对资源路径,例如 `/home/wanye/D/BlueArchive`。
|
||||||
|
- 不把当前某个真实版本号、日期、远程目录或本地目录写成长期契约。
|
||||||
|
- 不让 Go fixture 反向约束 Rust 内部实现;只约束对外 JSON contract。
|
||||||
|
|
||||||
|
## 建议共享目录
|
||||||
|
|
||||||
|
联调前使用临时目录交换未审核产物:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/tmp/bat-contract-fixture/
|
||||||
|
rust/
|
||||||
|
catalog-status.available.raw.json
|
||||||
|
catalog-status.unavailable.raw.json
|
||||||
|
resource-manifest.page0.raw.json
|
||||||
|
official-sync-snapshot.raw.json
|
||||||
|
normalized/
|
||||||
|
catalog-status.available.json
|
||||||
|
catalog-status.unavailable.json
|
||||||
|
resource-manifest.page0.json
|
||||||
|
official-sync-snapshot.json
|
||||||
|
notes.md
|
||||||
|
```
|
||||||
|
|
||||||
|
只有用户审核通过后,才允许把归一化 fixture 落入仓库,例如:
|
||||||
|
|
||||||
|
```text
|
||||||
|
internal/api/testdata/contract/
|
||||||
|
```
|
||||||
|
|
||||||
|
## Rust 侧需要产出
|
||||||
|
|
||||||
|
Rust 窗口请基于当前真实代码生成或导出以下 JSON:
|
||||||
|
|
||||||
|
1. `catalog.status` available=true 响应。
|
||||||
|
2. `catalog.status` available=false 响应。
|
||||||
|
3. `resource.manifest` 第一页响应,至少包含 1 到 2 个 entries。
|
||||||
|
4. 对应 release 的 `official-sync-snapshot.json`。
|
||||||
|
|
||||||
|
输出应来自 Rust 代码路径,而不是手写 JSON。允许使用 fixture resource root 或临时目录,但不能依赖开发机真实资源目录。
|
||||||
|
|
||||||
|
## 归一化规则
|
||||||
|
|
||||||
|
归一化只允许处理环境相关值,不改变 schema:
|
||||||
|
|
||||||
|
- 绝对路径归一化为 `${RESOURCE_ROOT}` 或 `${STATE_DIR}`。
|
||||||
|
- 版本 id 归一化为 `${VERSION_ID}`。
|
||||||
|
- 时间戳可归一化为固定小整数或 `${COMPLETED_UNIX_SECONDS}`。
|
||||||
|
- 真实 URL host 保留;路径中若含具体 release token,可归一化为 `{addressables-root}` / `{manifest-path}`。
|
||||||
|
- 字段名、字段类型、字段层级、null / missing / array / number 语义不得修改。
|
||||||
|
|
||||||
|
## Go 侧验证范围
|
||||||
|
|
||||||
|
Go 窗口读取归一化后的 JSON,验证:
|
||||||
|
|
||||||
|
1. `parseCatalogStatus` 能解析 `available=true`,并正确映射:
|
||||||
|
- `app_version`
|
||||||
|
- `bundle_version`
|
||||||
|
- `connection_group_name`
|
||||||
|
- `addressables_root`
|
||||||
|
- `version.id`
|
||||||
|
- `version.completed_unix_seconds`
|
||||||
|
- `version.resource_root`
|
||||||
|
- `launcher_metadata`
|
||||||
|
- `game_main_config`
|
||||||
|
2. `parseCatalogStatus` 对 `available=false` 返回不可用而不是错误。
|
||||||
|
3. `resource.manifest` entry 字段能映射为 Go `ResourceManifestEntry`:
|
||||||
|
- `url`
|
||||||
|
- `destination`
|
||||||
|
- `bytes`
|
||||||
|
- `blake3`
|
||||||
|
4. 本地 snapshot fixture 使用 `game_main_config_bootstrap`,RPC `catalog.status` 使用 `game_main_config`。
|
||||||
|
5. `bat-api` bootstrap 和 launcher bootstrap 不泄露归一化前的开发机路径。
|
||||||
|
|
||||||
|
Go 侧审核通过后的落地建议:
|
||||||
|
|
||||||
|
- `internal/api/testdata/contract/catalog-status.available.json`
|
||||||
|
- `internal/api/testdata/contract/catalog-status.unavailable.json`
|
||||||
|
- `internal/api/testdata/contract/resource-manifest.page0.json`
|
||||||
|
- `internal/api/testdata/contract/official-sync-snapshot.json`
|
||||||
|
- `internal/api/contract_fixture_test.go`
|
||||||
|
|
||||||
|
测试不应依赖 `/tmp/bat-contract-fixture/`;该目录只用于两窗口交接未审核产物。
|
||||||
|
|
||||||
|
## 必须覆盖的 optional 语义
|
||||||
|
|
||||||
|
至少需要两组 Rust 输出或派生 fixture 覆盖:
|
||||||
|
|
||||||
|
1. optional 字段非空:
|
||||||
|
- `launcher_metadata.game_lowest_version`
|
||||||
|
- `launcher_metadata.game_start_exe_name`
|
||||||
|
- `launcher_metadata.manifest_source`
|
||||||
|
- `game_main_config.server_info_data_url`
|
||||||
|
- `game_main_config.default_connection_group`
|
||||||
|
2. optional 字段为 null 或缺省:
|
||||||
|
- Go mirror 不应崩溃。
|
||||||
|
- HTTP response 中按当前 Go struct `omitempty` 策略输出。
|
||||||
|
|
||||||
|
## 用户审核点
|
||||||
|
|
||||||
|
落仓库前请用户审核:
|
||||||
|
|
||||||
|
- 归一化是否过度改变 Rust 真实输出。
|
||||||
|
- fixture 是否意外绑定真实版本、日期、本机路径或私有部署路径。
|
||||||
|
- `game_main_config` 与 `game_main_config_bootstrap` 的 RPC / snapshot 差异是否符合预期。
|
||||||
|
- optional 字段覆盖是否足够。
|
||||||
|
|
||||||
|
## notes.md 模板
|
||||||
|
|
||||||
|
Rust 侧生成 `/tmp/bat-contract-fixture/notes.md` 时建议使用以下结构:
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
# bat contract fixture notes
|
||||||
|
|
||||||
|
## 生成命令
|
||||||
|
|
||||||
|
- catalog.status available=true: ...
|
||||||
|
- catalog.status available=false: ...
|
||||||
|
- resource.manifest page0: ...
|
||||||
|
- official-sync-snapshot: ...
|
||||||
|
|
||||||
|
## 原始输出来源
|
||||||
|
|
||||||
|
- Rust commit / working tree: ...
|
||||||
|
- 使用的 fixture root 或临时目录: ...
|
||||||
|
- 是否依赖真实开发机资源目录: 否
|
||||||
|
|
||||||
|
## 归一化
|
||||||
|
|
||||||
|
- `${RESOURCE_ROOT}`: ...
|
||||||
|
- `${STATE_DIR}`: ...
|
||||||
|
- `${VERSION_ID}`: ...
|
||||||
|
- `${COMPLETED_UNIX_SECONDS}`: ...
|
||||||
|
- URL 路径占位符: ...
|
||||||
|
|
||||||
|
## 需要用户审核
|
||||||
|
|
||||||
|
- ...
|
||||||
|
```
|
||||||
|
|
||||||
|
## 完成判定
|
||||||
|
|
||||||
|
contract fixture 工作只有在以下条件同时满足时才算完成:
|
||||||
|
|
||||||
|
1. Rust 侧原始 JSON 来自真实 Rust 代码路径。
|
||||||
|
2. 归一化 JSON 经过用户审核。
|
||||||
|
3. Go 侧测试读取归一化 fixture 并验证 mirror struct / launcher bootstrap 行为。
|
||||||
|
4. Go 测试不依赖开发机资源目录、远程长期运行 `bat` 或 `/tmp` 中的交接目录。
|
||||||
|
5. 文档记录 fixture 覆盖的风险和仍未覆盖的字段。
|
||||||
|
|
||||||
|
## 建议给另一个窗口的短指令
|
||||||
|
|
||||||
|
```text
|
||||||
|
请读取 docs/reports/BAT_API_CONTRACT_FIXTURE_HANDOFF.md。
|
||||||
|
你负责 Rust bat 侧 contract fixture 原始输出:
|
||||||
|
1. catalog.status available=true
|
||||||
|
2. catalog.status available=false
|
||||||
|
3. resource.manifest page0
|
||||||
|
4. 对应 official-sync-snapshot.json
|
||||||
|
请输出到 /tmp/bat-contract-fixture/rust/,不要手写 JSON,不要引用开发机真实资源目录。
|
||||||
|
输出后在 /tmp/bat-contract-fixture/notes.md 说明生成命令、是否做过归一化、哪些字段需要用户审核。
|
||||||
|
```
|
||||||
|
|
||||||
|
## 当前状态
|
||||||
|
|
||||||
|
- Go `bat-api` 已具备消费 `launcher_metadata` / `game_main_config` 的 mirror struct。
|
||||||
|
- Go `bat-api` 已具备 player-facing HTTP 控制面、OpenAPI 和管理面板预留。
|
||||||
|
- contract fixture 尚未落仓库,等待 Rust 侧真实输出与用户审核。
|
||||||
+140
-60
@@ -1,6 +1,8 @@
|
|||||||
# 当前实现缺口清单
|
# 当前实现缺口清单
|
||||||
|
|
||||||
- **更新时间**:2026-07-20
|
- **更新时间**:2026-07-24
|
||||||
|
- **Go 进度权威**:`GO_STATUS.md`
|
||||||
|
- **资源布局 / 逆向契约**:`../architecture/resource-release-layout.md`
|
||||||
- **用途**:集中跟踪当前代码中的占位实现、设计缺口和下一步验收项。
|
- **用途**:集中跟踪当前代码中的占位实现、设计缺口和下一步验收项。
|
||||||
- **权威计划**:`../../PROJECT_PLAN.md`
|
- **权威计划**:`../../PROJECT_PLAN.md`
|
||||||
|
|
||||||
@@ -109,41 +111,70 @@
|
|||||||
|
|
||||||
### G-005:AssetBundle 引擎解析器仍未完成
|
### G-005:AssetBundle 引擎解析器仍未完成
|
||||||
|
|
||||||
|
状态:**部分完成**
|
||||||
|
|
||||||
|
冻结状态:自 2026-07-30 起,G-005 不再作为默认推进项。解析层只接受维护冻结规则允许的稳定性修复、诊断修复、真实回归修复和文档校正;新增 TypeTree 语义类型、扩大解析覆盖和新增写入型解析入口全部暂停。冻结细则见 `docs/reports/PARSER_FREEZE.md`。
|
||||||
|
|
||||||
现象:
|
现象:
|
||||||
|
|
||||||
- `crates/bat-assetbundle/src/parser.rs` 只有 `Parser::name`。
|
- `crates/bat-assetbundle` 已接管 UnityFS 解析,提供 `UnityFsParser`、`UnityFsBundle`、header、block info、directory、压缩模式、block info at end、LZ4/LZMA block info 解压、数据 block 解压、directory 文件提取和边界诊断。
|
||||||
- `types.rs` 只有 `AssetType::TextAsset`。
|
- `crates/bat-assetbundle::serialized` 已提供 Unity serialized file header、type table、TypeTree node 元数据、object table、TextAsset bytes 和基础 TypeTree field reader。
|
||||||
- `adapters/src/unity/unity_2021_3.rs` 已能解析 UnityFS header、block info、directory,并校验 directory `offset+size` 不越界;这属于 adapter 层基础摘要能力,不等于 `bat-assetbundle` 引擎已完成。
|
- `adapters/src/unity/unity_2021_3.rs` 已降为 Unity 版本选择薄层,复用 `bat-assetbundle`,不再维护第二套 UnityFS parser。
|
||||||
- 仍没有对象表、TypeTree、TextAsset、MonoBehaviour、ScriptableObject 或可扩展提取入口。
|
- `ResourceImportService` 的 UnityFS 摘要已经能暴露解包文件数、serialized file 数、TextAsset 名称、TextUnit 数量/格式和字段诊断。
|
||||||
|
- `MonoBehaviour`、`ScriptableObject` 已有基础 TypeTree 字段级反序列化和字符串提取入口;array/vector/staticvector/`List<T>`/`HashSet<T>`/map 元素与 TypeTree-covered managed reference registry payload 已保留独立 field path、offset 和 byte size,enum `value__` backing field 会暴露为语义化 `{type_name, storage_type, value}`,`LayerMask` / `BitField` 的 `m_Bits` backing field 会暴露为语义化 `{type_name, storage_type, bits}`,managed-reference full typename 可拆为 assembly/namespace/class,常见 `m_ManagedReferences` / `RefIds` / `m_RefIds` / verbose type 字段命名、`managedReference*` / `serializedReference*` prefixed metadata、`SerializedReference` 节点 alias 和 `data` / `value` / `payload` / `object` / `managedReferencePayload` / `referencePayload` / `serializedReferencePayload` / `managedReferenceValue` / `referenceValue` / `serializedReferenceValue` / `managedReferenceObject` / `referenceObject` / `serializedReferenceObject` / `managedReferenceData` / `referenceData` / `serializedData` / `serializedReferenceData` payload 命名已有回归覆盖,多记录 registry 聚合已有单元回归,TextUnit 提取会跳过 registry 元数据字符串并把它们作为 payload 文本上下文,fallback 字段遍历也会跳过常见 managed-reference 元数据别名,并按 `RefIds[n]` 等记录前缀或子字段推导 metadata 写入 payload TextUnit context;当前可对 string、bool、integer、float raw bits、bytes、enum、bit_field、常见固定 Unity float/int/hash 值类型的 leaf/direct-child 形态、unknown fixed-size raw bytes 同长度替换、PPtr、object 字段组合和 TypeTree schema 支撑的 array/vector/List/HashSet/map 整体替换执行文件级 patch,map entry 的 `first/second` 与 `key/value` 字段命名已有 serialized 和 UnityFS 重建回归,ScriptableObject `key/value` map 解析、变长替换和 UnityFS 重建已有专门回归;真实版本差异、未见样本驱动的复杂 managed reference registry / map entry 变体、unknown 字段结构语义和发布级重打包入口仍未完成。
|
||||||
|
|
||||||
影响:
|
影响:
|
||||||
|
|
||||||
- 可以对部分 UnityFS 样本做基础结构校验,但无法完成真实资源对象解析和文本提取。
|
- 可以对 UnityFS 容器做结构校验、解包 directory 文件,并提取 serialized file 中的 TextAsset 原始 bytes。
|
||||||
- 无法提取 TextAsset 或配置文本。
|
- 对日语汉化最关键的 TextAsset 索引、bytes、JSONL TextUnit、MonoBehaviour/ScriptableObject 基础字符串字段、array/vector/List/HashSet/map 字符串元素和 TypeTree-covered managed reference payload 提取已有入口;managed-reference 类型元数据作为上下文保留,不进入翻译文本队列,fallback registry 字段遍历也会过滤常见元数据别名,并按记录前缀或子字段保留可推导 metadata。文件级 UnityFS 重建可覆盖 TextAsset、TypeTree string 字段、managed-reference registry payload 字符串、基础语义字段、enum、bit_field、unknown fixed-size raw bytes、object 字段组合和 TypeTree schema 支撑的 array/vector/List/HashSet/map 整体替换,但还不能完成发布级复杂对象重打包。
|
||||||
|
|
||||||
验收:
|
当前验收证据:
|
||||||
|
|
||||||
- `crates/bat-assetbundle` 能解析结构化测试样本和隔离真实样本。
|
- `crates/bat-assetbundle` 能解析结构化测试样本和隔离真实样本。
|
||||||
- 支持 UnityFS header、blocks、directory、metadata、object table。
|
- 支持 UnityFS header、blocks、directory、metadata 摘要、directory 文件提取。
|
||||||
|
- 支持 Unity serialized file object table、TypeTree node 元数据和 TextAsset 提取的合成 fixture。
|
||||||
- 错误包含偏移和字段上下文。
|
- 错误包含偏移和字段上下文。
|
||||||
|
|
||||||
### G-006:Patch 引擎仍是占位
|
关闭前仍需:
|
||||||
|
|
||||||
现象:
|
- 冻结解除前不继续扩大 TypeTree 字段 reader 覆盖。当前 TypeTree-covered managed reference 字段与 registry 记录已可结构化解码,`m_ManagedReferences`、`RefIds` / `m_RefIds`、verbose type 字段、`managedReference*` / `serializedReference*` metadata、payload/value/object 家族、`managedReferenceData` / `referenceData` / `serializedData` / `serializedReferenceData` payload、多记录 registry 聚合、enum `value__` backing field、`LayerMask` / `BitField` 的 `m_Bits` backing field、固定 Unity float/int/hash 值类型 leaf/direct-child 形态、unknown fixed-size raw bytes 同长度替换、嵌套 vector `Array` 形态、`List<T>` / `HashSet<T>` 集合 alias、`first/second` 与 `key/value` map entry schema、空 array/vector/List/HashSet/map 扩容已有合成 fixture 覆盖;剩余真实版本差异、unknown 字段结构语义、更多 nested collection、managed reference registry / map entry 变体只记录为冻结后的工作。
|
||||||
|
- 用真实 fixture 继续覆盖 MonoBehaviour、ScriptableObject 字段级遍历和字符串策略。
|
||||||
|
- 输出可追溯文本定位:bundle path、archive entry、serialized file、path id、class id、field path、字段 offset/byte size。
|
||||||
|
- 用真实资源 fixture 覆盖对象级解析、TextAsset 提取和字段级文本提取。
|
||||||
|
- 将解析结果作为 Patch 输入;真正的重打包、Patch 生成和 `localized` 发布切换归 G-006/G-011D。
|
||||||
|
|
||||||
- `binary::apply_patch` 明确返回 `PatchError::ApplyFailed`,提示 Binary patch 尚未实现。
|
解析补全路线图:
|
||||||
- `json::apply_json_patch` 明确返回 `PatchError::ApplyFailed`,提示 JSON patch 尚未实现。
|
|
||||||
|
- 见 `docs/architecture/assetbundle.md` 的 P2/P3/P5。
|
||||||
|
|
||||||
|
### G-006:Patch 引擎基础已落地,发布入口仍未完成
|
||||||
|
|
||||||
|
状态:**部分关闭**
|
||||||
|
|
||||||
|
已完成:
|
||||||
|
|
||||||
|
- `bat-patch::binary` 已提供确定性 Binary hunk diff/apply,应用前校验 source size/BLAKE3,应用后校验 target size/BLAKE3。
|
||||||
|
- `bat-patch::json` 已提供 RFC 6902 JSON Patch apply,覆盖 add/remove/replace/move/copy/test 和 JSON Pointer escape。
|
||||||
|
- `bat-patch::text` 已提供 UTF-8 Text Patch,按 source-relative byte range 替换,支持 expected 文本校验、UTF-8 边界校验、source/target BLAKE3 和 size 校验。
|
||||||
|
- `bat-patch::manifest` 已定义通用 Patch manifest、文件级 patch kind、source/target BLAKE3、size、rollback 元数据和 manifest 文件完整性校验。
|
||||||
|
- `LocalizedPatchManifest` 可转换为通用 `bat_patch::PatchManifest`,UnityFS TextAsset 发布链路和通用 Patch manifest 已有类型对齐点。
|
||||||
|
- `infrastructure::patch_ops`、`patch.apply` RPC 和 `patch-apply` CLI 已开放文件级 Binary/JSON/Text patch apply;输入/输出为显式文件路径,输出原子写入并返回 size/BLAKE3。
|
||||||
|
- `unityfs.patch_text_asset` / `unityfs.patch_string_field` / `unityfs.patch_field` RPC 和 `unityfs-patch-text-asset` / `unityfs-patch-string-field` / `unityfs-patch-field` CLI 已开放显式 UnityFS bundle 文件写入;TextAsset、TypeTree string 字段、managed-reference registry `data` / `managedReferenceData` payload 字符串、基础语义字段、enum、bit_field、固定 Unity 值类型 leaf/direct-child 形态、unknown fixed-size raw bytes、object 字段组合和 TypeTree schema 支撑的 array/vector/List/HashSet/map 整体替换会在重建后重新解析校验。
|
||||||
|
|
||||||
影响:
|
影响:
|
||||||
|
|
||||||
- 无法生成或应用补丁。
|
- 通用 Binary/JSON/Text Patch crate 能力可作为后续发布流程输入。
|
||||||
- 回滚和完整性校验无法落地。
|
- 文件级写入入口可用于隔离测试和上层工具显式产物生成。
|
||||||
|
- 发布级 `patch build` / `patch rollback`、复杂 AssetBundle 重打包和通用 manifest 在发布命令中的正式使用仍未完成。
|
||||||
|
|
||||||
验收:
|
验收:
|
||||||
|
|
||||||
- Binary patch 能完成 diff/apply 往返。
|
- Binary patch 能完成 diff/apply 往返。
|
||||||
- JSON patch 能应用 RFC 6902 patch。
|
- JSON patch 能应用 RFC 6902 patch。
|
||||||
- Patch manifest 包含 hash、版本和回滚信息。
|
- Patch manifest 包含 hash、版本和回滚信息。
|
||||||
|
- Patch 构建/应用必须写 staging,完整性校验通过后才能发布。
|
||||||
|
- `patch.apply` / `patch-apply` 对显式文件执行 apply 时必须原子写目标文件,并返回 source/patch/target hash 与 size。
|
||||||
|
- 失败时不得影响 `bat-resources/current` 或已发布 `bat-localized/current`。
|
||||||
|
|
||||||
### G-007:Addressables Catalog 解析不完整
|
### G-007:Addressables Catalog 解析不完整
|
||||||
|
|
||||||
@@ -153,8 +184,9 @@
|
|||||||
|
|
||||||
- `AddressablesCatalogDriver` 已能解析当前真实形态 JSON catalog fixture/golden。
|
- `AddressablesCatalogDriver` 已能解析当前真实形态 JSON catalog fixture/golden。
|
||||||
- 已输出 path、hash、size、resource_type、address、dependencies、metadata,并已提取 `m_Crc` 到 `crc` 字段。
|
- 已输出 path、hash、size、resource_type、address、dependencies、metadata,并已提取 `m_Crc` 到 `crc` 字段。
|
||||||
|
- compact catalog 解析已补充 hash/size/CRC 的非 0 回归、资源计数 metadata,以及 blob 解码失败时的明确错误;不再在 compact 字段损坏时静默退回低保真 `m_InternalIds`。
|
||||||
- `bat-core` 已提供 `crc32_ieee` 和 `ResourceEntry::verify_downloaded_bytes`,SQLite `ResourceRepository` 已有 `crc` 列迁移。
|
- `bat-core` 已提供 `crc32_ieee` 和 `ResourceEntry::verify_downloaded_bytes`,SQLite `ResourceRepository` 已有 `crc` 列迁移。
|
||||||
- 仍需覆盖更多官方 catalog 结构变体、二进制/压缩字段组合和更明确的失败诊断。
|
- 仍需覆盖更多官方 catalog 结构变体、provider/bundle name 持久化字段,以及真实 Windows/Android catalog 样本集合。
|
||||||
|
|
||||||
影响:
|
影响:
|
||||||
|
|
||||||
@@ -165,54 +197,61 @@
|
|||||||
- 能解析项目目标版本的真实 Catalog 样本集合。
|
- 能解析项目目标版本的真实 Catalog 样本集合。
|
||||||
- 解析结果包含资源 key、provider、dependency、hash、size、path、CRC。
|
- 解析结果包含资源 key、provider、dependency、hash、size、path、CRC。
|
||||||
- 对不支持的 catalog 结构返回明确错误,而不是静默丢字段。
|
- 对不支持的 catalog 结构返回明确错误,而不是静默丢字段。
|
||||||
|
- 解析结果能反查 bundle 文件、依赖链和本地下载 manifest 条目。
|
||||||
|
- Windows/Android 样本集合需要覆盖 JSON、compact JSON 和后续二进制 catalog 入口。
|
||||||
|
|
||||||
|
解析补全路线图:
|
||||||
|
|
||||||
|
- 见 `docs/architecture/assetbundle.md` 的 P1。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 3. 应用层缺口
|
## 3. 应用层缺口
|
||||||
|
|
||||||
### G-008:Go CLI 产品入口尚未完成
|
### G-008:Go 同步/运维 CLI 产品入口
|
||||||
|
|
||||||
状态:**未完成(此前“并入 G-009”只是短期跟踪调整,不代表能力完成)**
|
状态:**已决策关闭(wontfix)**
|
||||||
|
|
||||||
现象:
|
决策(2026-07-24,见 `GO_STATUS.md`):
|
||||||
|
|
||||||
- 当前可用的用户同步/运维入口是 Rust `bat` binary。
|
- **正式同步/运维命令行 = Rust `bat`**(近乎全自动:auto-discover + watch/daemon,无需持久手操维护)。
|
||||||
- `internal/backendrpc` 已提供 Go 到 Rust daemon 的 typed JSON-RPC client;`cmd/bat` 已存在,但仅有 `doctor`、`manifest inspect`、`sync plan` 试验能力;`doctor` 只输出固定 `ok`,`manifest`/`sync` 依赖可选 CGO/FFI helper。
|
- **不另做**产品级 Go 同步 CLI,避免与 Rust `bat` 双轨。
|
||||||
- Go 侧尚未实现通过 Rust `bat --json` 或 daemon RPC 包装官方同步命令、稳定 human/json 输出、真实 doctor 检查和端到端测试。
|
- Go 试验入口 `cmd/bat` 可保留为 experimental,产物必须为 `bin/bat-go`,**禁止**再构建为 `bin/bat`。
|
||||||
- 如果项目决策改为“用户 CLI 永久由 Rust `bat` 承担,Go 只做 `bat-api`/服务层”,必须同步更新 `AGENTS.md`、`PROJECT_PLAN.md` 和 issue 跟踪;在完成该决策前,不能把 Go CLI 写成已完成。
|
- Go 正式产品入口集中在 **`bat-api` 资源 bootstrap/分发服务** + `internal/backendrpc`(G-009)。
|
||||||
|
|
||||||
验收:
|
原验收(真实 doctor / Go sync 包装)**不再作为当前里程碑**。
|
||||||
|
|
||||||
- `cmd/bat doctor` 做真实环境诊断,而不是固定字符串。
|
### G-009:API Server(`bat-api`,资源 bootstrap/分发)部分完成
|
||||||
- `cmd/bat sync` 能通过 `internal/backendrpc` daemon RPC 或 Rust `bat --json` one-shot/fallback 触发/查询官方同步,不走 FFI 控制下载器或 daemon。
|
|
||||||
- human/json 输出、退出码和错误码与 Rust `bat` 契约一致。
|
|
||||||
- `go test ./...`、`go vet ./...` 覆盖命令解析、错误输出和至少一个 mocked Rust 边界。
|
|
||||||
|
|
||||||
### G-009:API Server(`bat-api`,仿官方 API)尚未实现
|
状态:**资源 bootstrap + CDN MVP 已落地;非完整官方游戏 API**
|
||||||
|
|
||||||
现象:
|
目标(对应 issue #19,**按资源面收窄**):
|
||||||
|
|
||||||
- `api/` 只有目录结构,无 handler、service、路由。
|
- `cmd/bat-api`:组织 Rust `bat` 已发布 release 的启动前资源入口,并只读分发官方 CDN host/path 形态资源。
|
||||||
- Go 侧尚无对接 daemon RPC / `current/` 发布布局的服务端入口。
|
- **拉取归属 Rust `bat`**;`bat-api` 不做下载器。
|
||||||
|
- 发现经 `bat.sock`:先 `daemon.status`,再 `daemon.doctor`,再 `catalog.status` / `resource.manifest`。
|
||||||
|
- 生产与 Rust `bat` 同环境运行,资源根来自 RPC 返回的 `resource_root`;`--resource-root` 仅用于 fixture 或应急只读诊断。
|
||||||
|
- `.env` 配置端口 / public base / RPC socket / RPC 刷新周期;预留 database/redis。
|
||||||
|
- `/v1/bootstrap` 返回 RPC 健康、release 摘要、server-info URL、client-patch base 和改写后的 Addressables root。
|
||||||
|
- `/v1/launcher/bootstrap` 和 `/api/launcher/...` 形状端点返回资源引导兼容信息,当前来源是 Rust `bat` 已发布 snapshot/RPC 中的 launcher metadata 与 GameMainConfig 摘要;Rust 侧已新增 release 内 `official-launcher-bootstrap.json` 版本化产物,后续 bat-api 字段统一和联调应以该产物加 RPC contract fixture 为准。
|
||||||
|
- `/healthz` 暴露最近一次 RPC refresh 诊断;`/readyz` 在无可分发 release 时返回 `503`。
|
||||||
|
- 玩家-facing HTTP 控制面必须支持 token 鉴权、限流、访问日志、反代 IP 适配、动态 JSON `no-store` 和 `/v1/resources` 分页上限。
|
||||||
|
- CDN path 支持 `GET` / `HEAD` / `Range`、ETag、Last-Modified、Accept-Ranges 和长期缓存头。
|
||||||
|
- launcher 完整安装包更新链、账号、登录、网关、游戏业务 API 和鉴权全链 **非关闭条件**;USERGUIDE 已补基础章节,联调后补生产排障样例。
|
||||||
|
|
||||||
目标(对应 issue #19):
|
已完成:
|
||||||
|
|
||||||
- 新建 `cmd/bat-api`:**完全仿照 BlueArchive 官方 API** 的 Go HTTP 服务,把 Rust `bat` 后端发布的 `current/` release 按官方接口形态对外提供,使真实客户端/工具可将其当作官方服务端。
|
- `cmd/bat-api`、`internal/api`、`/v1/bootstrap`、`/v1/launcher/bootstrap`、launcher 资源 metadata 兼容端点、HTTP token 鉴权、限流、访问日志、反代 IP 适配、动态 JSON `no-store`、`/v1/resources` 分页上限、OpenAPI、`/admin/` 预留、RPC 周期刷新/诊断、`/readyz`、CDN Range/缓存头、fixture 单测、USERGUIDE 基础章节、systemd bat-api 模板、`make build-go-api` / `test-go-api`
|
||||||
- 仿真面:资源 CDN 面(TableCatalog/MediaCatalog/BundlePackingInfo、bundle、seed `.hash`)、server-info 面、launcher API 面。
|
- 进度权威:`docs/reports/GO_STATUS.md`
|
||||||
- 鉴权/签名**完全仿照**官方实现,服务端做验签(对齐 `adapters/src/official/launcher.rs` 的签名逻辑)。
|
|
||||||
- 版本/状态经 daemon RPC(`catalog.*` / `resource.manifest`)发现,资源字节从 `current/` 读取;不读写 daemon 状态文件内部。
|
|
||||||
|
|
||||||
影响:
|
验收(剩余):
|
||||||
|
|
||||||
- 客户端、工具和第三方集成无服务端入口。
|
- 与远程长期运行的 `bat` / 全量 release 联调(覆盖 bootstrap、server-info、CDN path;SSH 实勘可后置)
|
||||||
|
- Rust snapshot schema 与 Go mirror struct 的跨语言 contract fixture(需要用户审核后落入 fixture)
|
||||||
|
- refresh 中 manifest 磁盘校验的 mtime/size 增量缓存优化(真实全量 release 观测后决定)
|
||||||
|
- 文档与 GO_STATUS 持续一致
|
||||||
|
|
||||||
验收:
|
排期:P2 主体可联调;持久化 API 层与完整 launcher/业务链另议。
|
||||||
|
|
||||||
- Go 单测覆盖路由、签名验签、错误响应形态。
|
|
||||||
- 真机 e2e:daemon 发布 fixture release → 启动 `bat-api` → 按官方 URL 与鉴权头请求 server-info / catalog / bundle / launcher 链,断言字节与形态正确、验签生效(缺签/错签被拒)。
|
|
||||||
- 统一错误结构与官方响应 envelope 对齐;Makefile 增加 Go 构建/测试目标。
|
|
||||||
|
|
||||||
排期:P2,排在 issue #17 验收收口以及 issue #2 / #3 的解析能力继续推进之后启动;可与 G-008 的 Go 产品入口边界收敛并行。
|
|
||||||
|
|
||||||
### G-010:Web 管理后台尚未实现
|
### G-010:Web 管理后台尚未实现
|
||||||
|
|
||||||
@@ -232,7 +271,7 @@
|
|||||||
|
|
||||||
## 4. 数据与翻译缺口
|
## 4. 数据与翻译缺口
|
||||||
|
|
||||||
### G-011:Resource Repository 未持久化
|
### G-011:Resource Repository 查询面仍不完整
|
||||||
|
|
||||||
状态:**部分关闭**
|
状态:**部分关闭**
|
||||||
|
|
||||||
@@ -240,14 +279,27 @@
|
|||||||
|
|
||||||
- `SqliteResourceRepository` 已存在,可按领域 repository 接口保存资源元数据。
|
- `SqliteResourceRepository` 已存在,可按领域 repository 接口保存资源元数据。
|
||||||
- `ResourceImportService` 已能把 manifest 中有数据的资源写入 CAS + `ResourceRepository`,AssetBundle 会记录 UnityFS 摘要,TextAsset/Table/Media 会分类索引。
|
- `ResourceImportService` 已能把 manifest 中有数据的资源写入 CAS + `ResourceRepository`,AssetBundle 会记录 UnityFS 摘要,TextAsset/Table/Media 会分类索引。
|
||||||
- 官方同步下载结果尚未作为用户级流程自动触发导入 CAS + ResourceRepository。
|
- 官方同步下载结果可用 `--import-repository` / `BAT_IMPORT_REPOSITORY=1` 在已校验 release 发布后自动导入 CAS + ResourceRepository;默认 CAS 为 `<output>/.cas`,默认 SQLite 索引为 `<output>/resources.sqlite`,也可通过 `--import-cas-root`、`--import-resource-db`、`BAT_IMPORT_CAS_ROOT`、`BAT_IMPORT_RESOURCE_DB` 覆盖。
|
||||||
- 迁移、版本化 schema 和 CLI 查询入口仍需补齐。
|
- `resource.index` RPC 已能按资源类型、hash、路径模式分页查询现有 SQLite 索引;数据库不存在时返回 `available=false`,不会因查询创建空库。
|
||||||
|
- 官方 release 发布后会写出 `official-resource-changes.json` 和 `crowdin-translation-handoff.json`,新增+变更资源进入解析/翻译 handoff,删除资源只进入差异记录。
|
||||||
|
- `Resource` metadata 已通过 SQLite `metadata_json` 兼容迁移保存 release、平台、bundle path、parse status、TextAsset 名称和 TextUnit 数量/格式;`resource.index` 会返回这些 metadata。
|
||||||
|
- 官方 release 发布后会持久化 `official-textunit-index.json`,记录单条 TextUnit 和解析错误;`parse.text_units` / `parse.errors` RPC 和 `parse-text-units` / `parse-errors` CLI 可按 destination、archive entry、path id、class id、field path 和 format 分页过滤。
|
||||||
|
- 官方 release 发布后会从 Added/Modified 资源、parse cache 和 TextUnit 明细索引派生 `official-textunit-tasks.json` 和 `crowdin-textunit-queue.json`;删除资源不会进入队列。
|
||||||
|
- 翻译任务状态和 Crowdin worker 失败原因查询仍需补齐。
|
||||||
|
|
||||||
验收:
|
验收:
|
||||||
|
|
||||||
- schema 和迁移可重复执行。
|
- schema 和迁移可重复执行。
|
||||||
- 可按版本、类型、hash、路径查询资源。
|
- 可按版本、类型、hash、路径查询资源,且能明确区分索引缺失、版本缺失和空结果。
|
||||||
- 官方同步后的资源可通过 CLI 查询并能追溯到 CAS 对象。
|
- 官方同步后的资源可通过 CLI/RPC 查询并能追溯到 CAS 对象。
|
||||||
|
- `official-parse-cache.json` 的 bundle、zip entry、TextAsset 和 TextUnit 摘要能进入 ResourceRepository 查询面。
|
||||||
|
- `parse.status` 能报告 TextUnit 索引、TextUnit 队列路径与摘要。
|
||||||
|
- `parse.text_units` / `parse.errors` 能分页查询当前 release 的 TextUnit 明细和解析错误。
|
||||||
|
- Crowdin handoff 被后续翻译 worker 消费后,任务状态和失败原因能反查到对应官方 release 与资源 destination。
|
||||||
|
|
||||||
|
解析补全路线图:
|
||||||
|
|
||||||
|
- 见 `docs/architecture/assetbundle.md` 的 P4。
|
||||||
|
|
||||||
### G-011A:资源导入链路基础能力不足
|
### G-011A:资源导入链路基础能力不足
|
||||||
|
|
||||||
@@ -294,6 +346,32 @@
|
|||||||
- `cargo test -p bat-infrastructure version_state`
|
- `cargo test -p bat-infrastructure version_state`
|
||||||
- `cargo test -p bat-infrastructure --test official_game_main_config_bootstrap`
|
- `cargo test -p bat-infrastructure --test official_game_main_config_bootstrap`
|
||||||
|
|
||||||
|
### G-011D:汉化发布状态与 Patch 发布流程未完成
|
||||||
|
|
||||||
|
状态:**部分关闭**
|
||||||
|
|
||||||
|
当前已完成:
|
||||||
|
|
||||||
|
- 官方原版资源发布根为 `./bat-resources`,汉化产物发布根为 `./bat-localized`。
|
||||||
|
- CLI 支持 `--localized-output` / `BAT_LOCALIZED_OUTPUT`,并拒绝官方目录和汉化目录相同或互相嵌套。
|
||||||
|
- 官方同步报告新增 `localized_release_status=not_localized`,明确表示原版资源已发布、汉化资源未发布。
|
||||||
|
- `LocalizedPatchService` 已具备将给定汉化文件按官方相对路径发布到 `--localized-output` / `BAT_LOCALIZED_OUTPUT` 指定目录下的 `.staging/<id>`、校验后移动到 `versions/<id>`、原子切换 `current` 并写入 `localized-version-state.json` 的基础能力。
|
||||||
|
- `LocalizedPatchService` 已写入结构化 `localized-patch-manifest.json`,记录 TextAsset 操作、原始/汉化 hash、size、byte delta 和 rollback 信息;发布前后会校验 manifest hash/size 与 current symlink,失败时清理 staging / 未完成 version。
|
||||||
|
- `localized.status` RPC 会读取 `.env` / daemon 配置中的汉化输出目录,校验汉化状态是否匹配当前官方 release,且要求 patch manifest 存在并匹配 release,避免写死 `./bat-localized` 或误报手工状态。
|
||||||
|
|
||||||
|
仍未完成:
|
||||||
|
|
||||||
|
- 真实 Patch/翻译构建阶段尚未从 `crowdin-textunit-queue.json`、翻译记忆和 Crowdin 结果生成完整汉化文件集合。
|
||||||
|
- 通用 Binary/JSON/Text Patch crate 基础和文件级 `patch.apply` / UnityFS 写入入口已经实现;复杂 AssetBundle 重打包、发布级 patch build/rollback 和与汉化发布流程的统一仍未完成。
|
||||||
|
- 尚未实现原版资源与汉化资源双发布后的查询、分发和清理策略。
|
||||||
|
|
||||||
|
验收:
|
||||||
|
|
||||||
|
- 原版资源同步成功后保持 `not_localized`,不发布半成品汉化资源。
|
||||||
|
- Patch 构建和校验成功后,汉化产物按官方相对路径写入配置化汉化发布根下的 `versions/<id>`。
|
||||||
|
- 汉化发布必须原子切换配置化汉化发布根下的 `current`,失败时不影响已发布原版资源。
|
||||||
|
- `localized` 状态能证明原版和汉化两套资源都可发布,并能被 CLI/RPC/API 查询;缺 patch manifest 或 release 不匹配时不得返回 `localized`。
|
||||||
|
|
||||||
### G-011C:真实 fixture 与回归样本不足
|
### G-011C:真实 fixture 与回归样本不足
|
||||||
|
|
||||||
状态:**已关闭当前阶段**
|
状态:**已关闭当前阶段**
|
||||||
@@ -470,13 +548,15 @@
|
|||||||
|
|
||||||
## 6. 当前关闭顺序建议
|
## 6. 当前关闭顺序建议
|
||||||
|
|
||||||
1. issue #24:失败 staging 复用已补离线回归;继续核对 issue 口径、状态与后续是否仍有真实场景无法复现的残余。
|
1. issue #24:失败 staging 复用回归已补;核对残余场景。
|
||||||
2. issue #1:Rust daemon/backend API 边界收口;`resource.repair`、`resource.list`、`daemon.doctor`、`internal/backendrpc` Go client 和稳定 RPC reference 已补齐,剩余确认 `patch.*` / `unityfs.*`(待引擎)、`task.create`(暂不开放通用入口)以及 `daemon.restart` / `daemon.clean-stable`(CLI 生命周期入口)的 issue 验收口径。
|
2. issue #1:RPC 主体、文件级 `patch.apply` / `unityfs.patch_text_asset` / `unityfs.patch_string_field` / `unityfs.patch_field` 已落地;剩余发布级 patch build/rollback、复杂 UnityFS 语义编辑与设计边界确认。
|
||||||
3. issue #17/#20/#21/#22:多线程下载入口已按最新决定移除,下载回归顺序执行并保留指数退避与单调进度上报;daemon 子进程不再透传并发参数,但 GitHub issue 仍 open。
|
3. issue #17 及子 issue:已按 wontfix 关闭多线程下载(顺序下载 + 指数退避)。
|
||||||
4. issue #2 / G-007:继续扩大 Addressables 可校验字段和结构变体覆盖。
|
4. **G-008:已决策关闭**(同步 CLI = Rust `bat`;见 `GO_STATUS.md`)。
|
||||||
5. issue #3 / G-005:把 UnityFS 基础摘要推进到 `bat-assetbundle` 引擎级解析。
|
5. **G-009 / issue #19**:资源 bootstrap/分发 MVP 已编码;优先服务器联调与索引实勘,非「从零实现」。
|
||||||
6. G-011:官方同步结果接入 CAS + ResourceRepository 用户级工作流。
|
6. issue #2 / G-007(P1):Addressables 可校验字段。
|
||||||
7. G-008 / G-009:收敛 Go 产品入口边界,并实现 `bat-api`(issue #19)。
|
7. issue #3 / G-005(P1):UnityFS 容器基础解析已落地;对象级引擎解析继续跟踪 G-005。
|
||||||
8. G-012 / G-006:翻译系统、Patch 引擎。
|
8. G-011:翻译任务状态、CAS 诊断和 ResourceRepository 查询面扩展。
|
||||||
|
9. G-012 / G-006:Crowdin/翻译系统、复杂 AssetBundle 重打包和 Patch 发布流程统一。
|
||||||
|
10. G-011D:原版/汉化双发布后的查询、分发和清理策略。
|
||||||
|
|
||||||
这个顺序优先把 Rust `bat` 后端做扎实(解析能力 + 下载性能),再把官方同步结果进入可查询资源库,之后收敛 Go 入口和仿官方 API 服务端,最后推进翻译和补丁。G-018 已固化为可重复 smoke 命令并关闭;G-017 已按“不引入托管 CI”决策关闭。
|
Go 进度以 `docs/reports/GO_STATUS.md` 为准。G-018 / G-017 已关闭。
|
||||||
|
|||||||
@@ -0,0 +1,151 @@
|
|||||||
|
# Go 侧进度与边界(权威)
|
||||||
|
|
||||||
|
- **更新时间**:2026-07-27
|
||||||
|
- **用途**:统一 Go module `bat-api` 的产品边界、既有约定和组件进度;其他文档与此冲突时以本文为准。
|
||||||
|
- **关联**:issue #19 / G-009(资源 bootstrap/分发)、G-008(已决策关闭)、`docs/architecture/official-resource-backend.md` §7
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. 三个入口分别是什么
|
||||||
|
|
||||||
|
| 名称 | 路径 / 产物 | 角色 | 是否产品入口 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| **Rust `bat`** | `infrastructure` bin → 正式同步二进制 | 官方资源**自动**发现 / 拉取 / 校验 / 发布 / watch·daemon / 运维子命令 | **是(同步与运维命令行)** |
|
||||||
|
| **Go `bat-api`** | `cmd/bat-api` → `bin/bat-api` | **资源 bootstrap + 分发 HTTP 服务**(官方 CDN path 形态)+ release 观察 API | **是(bootstrap/分发服务)** |
|
||||||
|
| **Go 试验 CLI** | `cmd/bat` → `bin/bat-go`(不得再叫 `bin/bat`) | FFI 演示骨架 | **否** |
|
||||||
|
|
||||||
|
### 1.1 「同步命令行 = Rust `bat`」的含义
|
||||||
|
|
||||||
|
人类做资源同步与运维时,正式命令行是 **Rust 编译的 `bat`**(近乎全自动:`--auto-discover`、`--watch` / `--daemon` 后只需偶发 `status` / `refresh` / `repair`,不需要持久手操维护)。
|
||||||
|
|
||||||
|
这**不是**说整个项目只有 Rust,也**不是**取消 Go 入口:
|
||||||
|
|
||||||
|
- Go 的正式产品入口是 **`bat-api` 服务进程**(给客户端/工具提供启动前资源 bootstrap、server-info 改写和已发布资源字节),不是再做一套同步 CLI。
|
||||||
|
- Go `cmd/bat` 仅试验,禁止与 Rust `bat` 二进制重名。
|
||||||
|
|
||||||
|
### 1.2 `bat` 与 `bat-api` 的关系
|
||||||
|
|
||||||
|
`bat` 是资源生产者和状态拥有者;`bat-api` 是资源读侧和 HTTP 入口。
|
||||||
|
|
||||||
|
| 关系面 | Rust `bat` / daemon | Go `bat-api` |
|
||||||
|
|---|---|---|
|
||||||
|
| 资源发现 | 读取官方 launcher/resource metadata,解析 `GameMainConfig`、server-info 和 Addressables root | 通过 `bat.sock` 读取已发布版本摘要,不重新探测官方 metadata |
|
||||||
|
| 下载与发布 | 下载、校验、staging、原子发布 `current -> versions/<id>`,维护 manifest/snapshot/version-state | 不下载、不写 staging、不改 version-state;生产资源根来自 RPC 返回的 `resource_root` |
|
||||||
|
| 启动前资源入口 | 暴露 `catalog.status` / `resource.manifest` 等 RPC 数据 | 提供 `/v1/bootstrap`、`/v1/launcher/bootstrap`、launcher 资源 metadata 兼容端点、`/v1/server-info` 和 CDN path,组织给客户端/补丁器使用 |
|
||||||
|
| 长期状态 | watch/daemon、任务队列、日志、错误码、repair/sync/verify | 周期性经 RPC 刷新内存索引,只展示 ready、RPC 健康和 release;需要拉取/修复时由外部运维调用 `bat` 或 RPC 任务 |
|
||||||
|
|
||||||
|
这条边界允许 `bat-api` 做资源 bootstrap 兼容,但不允许它复制 Rust 下载器或伪装完整游戏业务服务。
|
||||||
|
|
||||||
|
### 1.3 决策(已核验)
|
||||||
|
|
||||||
|
1. **G-008 决策关闭(wontfix)**:不另做产品级 Go 同步/运维 CLI。
|
||||||
|
2. **G-009**:资源 bootstrap/分发 MVP 部分完成;非完整游戏业务 API。
|
||||||
|
3. **USERGUIDE 的 bat-api 基础章节已补**;全量 release 联调后继续补充生产参数和排障样例。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. 既有约定核对表(不可丢)
|
||||||
|
|
||||||
|
### 职责
|
||||||
|
|
||||||
|
| ID | 约定 |
|
||||||
|
|---|---|
|
||||||
|
| A | **自动发现 / 下载 / 校验 / 发布 / watch·daemon** 只在 **Rust `bat`** |
|
||||||
|
| B | **`bat-api` 只读分发**已发布 release,不实现下载器,不写 staging/version-state |
|
||||||
|
| C | 仿真范围 = **资源拉取相关**(resource bootstrap + CDN path + 可选 server-info);**不是**完整游戏业务 API |
|
||||||
|
| D | launcher 资源 metadata 可作为 bootstrap 输入/输出兼容;账号、登录、网关和鉴权全链 **非 G-009 关闭条件** |
|
||||||
|
| E | USERGUIDE bat-api 基础章节已补;联调后补充实战样例 |
|
||||||
|
|
||||||
|
### 发现与数据
|
||||||
|
|
||||||
|
| ID | 约定 |
|
||||||
|
|---|---|
|
||||||
|
| F | 版本/清单经 **`bat.sock` JSON-RPC**(`--socket`);不读 daemon 内部状态文件 |
|
||||||
|
| G | RPC 顺序:先 **`daemon.status`**,再 **`daemon.doctor`**,再 catalog/manifest |
|
||||||
|
| H | 生产文件字节从 RPC 返回的 `resource_root` 读盘;`bat-api` 与 daemon 同服务器/同容器/共享文件系统部署;`--resource-root` 仅 fixture 或应急只读诊断 |
|
||||||
|
| I | 真数据在**已全量拉取且长期运行 Rust `bat` 的远程服务器**;开发机不跑全量 `bat`,用 fixture、mock RPC 和 Go 门禁验证;远程联调等连接信息 |
|
||||||
|
| J | 索引以 **manifest + 磁盘 Present/size** 为准 |
|
||||||
|
|
||||||
|
### 进程配置
|
||||||
|
|
||||||
|
| ID | 约定 |
|
||||||
|
|---|---|
|
||||||
|
| K | `.env` / 环境变量 / CLI:端口、public base、RPC socket、RPC 刷新周期;**预留** database/redis |
|
||||||
|
| L | 管理面 / bootstrap:`/healthz`、`/readyz`、`/v1/bootstrap`、`/v1/release`、`/v1/resources`、`/openapi.yaml`、`/admin/` 预留 |
|
||||||
|
| M | CDN:`GET/HEAD /prod-clientpatch.bluearchiveyostar.com/...`,支持 Range、ETag、Last-Modified、长期缓存头 |
|
||||||
|
| N | server-info 可选;**只改 AddressablesCatalogUrlRoot** |
|
||||||
|
| N2 | launcher 兼容仅限资源引导:`/v1/launcher/bootstrap` 与 `/api/launcher/...` 形状端点输出已发布 release、launcher metadata 和 GameMainConfig 摘要;不下载 launcher 包、不生成完整 PC package update manifest、不仿造登录/网关 |
|
||||||
|
| N3 | 玩家-facing HTTP 控制面:可配置 token 鉴权、进程内限流、访问日志、反代 IP 适配、动态 JSON `no-store`、`/v1/resources` 分页上限 |
|
||||||
|
|
||||||
|
### 工程
|
||||||
|
|
||||||
|
| ID | 约定 |
|
||||||
|
|---|---|
|
||||||
|
| O | 权威文档与 `go list` 一致,禁止「API 完全没有」等过时句 |
|
||||||
|
| P | 试验 CLI 产物 **`bin/bat-go`**,禁止 `bin/bat` |
|
||||||
|
| Q | 空目录标明 reserved empty |
|
||||||
|
| R | 默认门禁:`make test-go-api` + `make build-go-api`(无 FFI) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. 组件进度
|
||||||
|
|
||||||
|
| 组件 | 路径 | 状态 | 说明 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| Module | `go.mod` → `bat-api` | 已用 | 服务层模块名 |
|
||||||
|
| RPC client | `internal/backendrpc` | **完成** | typed JSON-RPC;fake transport 单测 |
|
||||||
|
| 资源 bootstrap/分发 | `cmd/bat-api` + `internal/api` | **MVP+生产控制面** | RPC 发现 + 周期刷新/诊断 + `/v1/bootstrap` + `/v1/launcher/bootstrap` + launcher 资源 metadata 兼容 + `/readyz` + CDN Range/缓存头 + 鉴权/限流/访问日志/反代适配 + OpenAPI + 管理面预留 + `.env` |
|
||||||
|
| 试验 CLI | `cmd/bat` | **试验** | doctor 固定 ok;manifest/sync 走 FFI |
|
||||||
|
| FFI | `internal/ffi` | **可选** | 需 `build-ffi` |
|
||||||
|
| 空骨架 | `api/`、`pkg/*`、部分 `internal/*` | **空** | 见各目录 README |
|
||||||
|
| Web | `web/` | **空** | G-010 |
|
||||||
|
|
||||||
|
`go list ./...` 当前包:
|
||||||
|
|
||||||
|
- `bat-api/cmd/bat-api`
|
||||||
|
- `bat-api/cmd/bat`
|
||||||
|
- `bat-api/internal/api`
|
||||||
|
- `bat-api/internal/backendrpc`
|
||||||
|
- `bat-api/internal/ffi`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. 验证门禁
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 默认(提交前 / CI 建议)
|
||||||
|
make test-go-api
|
||||||
|
make build-go-api
|
||||||
|
go vet ./internal/api/... ./internal/backendrpc/... ./cmd/bat-api/...
|
||||||
|
|
||||||
|
# 可选:改 FFI 或试验 CLI 时
|
||||||
|
make build-ffi
|
||||||
|
make test-go-ffi
|
||||||
|
make build-go-cli # 产出 bin/bat-go
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. 与缺口 / issue 对应
|
||||||
|
|
||||||
|
| 项 | 状态 |
|
||||||
|
|---|---|
|
||||||
|
| G-008 Go 同步 CLI | **已决策关闭**(正式同步 CLI = Rust `bat`) |
|
||||||
|
| G-009 bat-api 资源 bootstrap/分发 | **部分完成**(MVP+生产控制面);已含资源 bootstrap 关系面、launcher 资源 metadata 兼容、HTTP 鉴权/限流/日志/反代适配、RPC 周期刷新/诊断、readiness、OpenAPI、管理面预留和部署模板,后续远程服务器联调/可选持久化 |
|
||||||
|
| issue #19 | 资源面 MVP 与 USERGUIDE 基础章节已编码;真机联调后继续补充实战样例;**未自动关 issue** |
|
||||||
|
| G-010 Web | 未开始 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. 资源布局与逆向
|
||||||
|
|
||||||
|
- **Release / URL / 分发契约(权威)**:`docs/architecture/resource-release-layout.md`
|
||||||
|
- 真机全量实勘、seed inventory diff、issue #2/#3 样本采集按该文档 §9–§10 执行
|
||||||
|
|
||||||
|
## 7. 后续(不在进度统一范围内)
|
||||||
|
|
||||||
|
1. 服务器 SSH 只读实勘(连接信息到位后)
|
||||||
|
2. bat-api 与远程长期运行的 `bat` / 全量 release 联调(含 `/v1/bootstrap`、server-info 和 CDN path)
|
||||||
|
3. 预留 database/redis 的接入时机另议
|
||||||
|
4. USERGUIDE bat-api 联调排障样例(全量 release 验证后)
|
||||||
|
5. launcher 完整安装包更新链 / 登录网关链(若需要,新 issue)
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
# 解析模块维护冻结
|
||||||
|
|
||||||
|
状态:**生效中**
|
||||||
|
|
||||||
|
生效时间:2026-07-30
|
||||||
|
|
||||||
|
冻结目标:停止继续扩大 UnityFS / AssetBundle / Addressables / TypeTree 解析能力,把当前工作重心切换到运行稳定性、代码审核问题、文档一致性和发布链路可靠性。
|
||||||
|
|
||||||
|
## 冻结范围
|
||||||
|
|
||||||
|
冻结覆盖以下 Rust 解析相关模块和对外入口:
|
||||||
|
|
||||||
|
- `crates/bat-assetbundle`
|
||||||
|
- `adapters/src/unity*`
|
||||||
|
- `infrastructure/src/official_parse.rs`
|
||||||
|
- `infrastructure/src/resources.rs` 中解析缓存、TextUnit 索引和解析状态相关逻辑
|
||||||
|
- `unityfs.*`、`parse.*`、`text.*` 相关 RPC / CLI 契约
|
||||||
|
- Addressables catalog、UnityFS、serialized file、TypeTree、TextUnit、AssetBundle patch 相关文档声明
|
||||||
|
|
||||||
|
## 允许变更
|
||||||
|
|
||||||
|
冻结期只允许以下解析相关变更:
|
||||||
|
|
||||||
|
- 修复编译失败、格式化失败、clippy 报错和测试失败。
|
||||||
|
- 修复真实运行中已经复现的 panic、错误状态污染、重复解析、缓存失效、状态不一致或诊断误导。
|
||||||
|
- 补充回归测试,前提是测试覆盖的是已存在能力的稳定性问题,不宣称新增解析能力。
|
||||||
|
- 修正文档、CLI 帮助、RPC 参考和状态文件中与当前实现不一致的解析能力声明。
|
||||||
|
- 改善错误信息、日志字段、状态记录和失败恢复,但不得改变解析输出契约,除非是修复错误契约且同步迁移说明。
|
||||||
|
|
||||||
|
## 禁止变更
|
||||||
|
|
||||||
|
冻结期禁止以下解析相关变更:
|
||||||
|
|
||||||
|
- 新增 TypeTree 语义类型、字段族、managed reference 变体、Unity 内建结构体覆盖或 Addressables catalog 结构覆盖。
|
||||||
|
- 用纯合成 fixture 推进“完整解析”并把它记录为已支持能力。
|
||||||
|
- 开放新的写入型 `unityfs.*` / `patch.*` RPC 或 CLI。
|
||||||
|
- 修改解析结果 schema、TextUnit schema、patch field JSON 语义或缓存状态格式,除非它是阻断级 bug 修复并附带兼容策略。
|
||||||
|
- 将解析器和官方同步、汉化发布、Go API、Crowdin 或客户端流程进一步耦合。
|
||||||
|
|
||||||
|
## 解冻条件
|
||||||
|
|
||||||
|
解析扩展重新启动前必须同时满足:
|
||||||
|
|
||||||
|
- Rust `bat` 官方同步、daemon、status、校验、断点续传、增量更新和解析缓存链路稳定。
|
||||||
|
- 当前 P0/P1 维护 issue 已关闭或被明确降级。
|
||||||
|
- `bat-api` 与 Rust RPC / CLI 契约完成字段统一和联调验证。
|
||||||
|
- 真实资源 fixture、验证命令和验收标准已写入文档,不能只依赖合成样本。
|
||||||
|
|
||||||
|
## 冻结期验证
|
||||||
|
|
||||||
|
解析相关维护变更至少运行:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo fmt --check
|
||||||
|
cargo test -p bat-assetbundle --locked
|
||||||
|
cargo clippy -p bat-assetbundle --all-targets --locked -- -D warnings
|
||||||
|
```
|
||||||
|
|
||||||
|
如果变更影响 `bat` CLI、RPC、官方解析缓存或 TextUnit 索引,还必须补充对应 `bat-infrastructure` 测试或说明未运行原因。
|
||||||
@@ -12,7 +12,9 @@ path = "src/bin/bat_official_sync.rs"
|
|||||||
[dependencies]
|
[dependencies]
|
||||||
bat-core = { path = "../core" }
|
bat-core = { path = "../core" }
|
||||||
bat-adapters = { path = "../adapters" }
|
bat-adapters = { path = "../adapters" }
|
||||||
|
bat-assetbundle = { path = "../crates/bat-assetbundle" }
|
||||||
bat-cas-engine = { path = "../crates/bat-cas-engine" }
|
bat-cas-engine = { path = "../crates/bat-cas-engine" }
|
||||||
|
bat-patch = { path = "../crates/bat-patch" }
|
||||||
anyhow.workspace = true
|
anyhow.workspace = true
|
||||||
thiserror.workspace = true
|
thiserror.workspace = true
|
||||||
serde.workspace = true
|
serde.workspace = true
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
@@ -2,9 +2,10 @@
|
|||||||
|
|
||||||
use bat_adapters::manifest::GenericManifest;
|
use bat_adapters::manifest::GenericManifest;
|
||||||
use bat_adapters::unity::{RawAssetBundle, UnityAdapterRegistry};
|
use bat_adapters::unity::{RawAssetBundle, UnityAdapterRegistry};
|
||||||
use bat_core::domain::{Resource, ResourceEntry, ResourceType};
|
use bat_assetbundle::TextUnitExtractor;
|
||||||
|
use bat_core::domain::{Resource, ResourceEntry, ResourceMetadata, ResourceType};
|
||||||
use bat_core::repositories::{CasRepository, ResourceRepository};
|
use bat_core::repositories::{CasRepository, ResourceRepository};
|
||||||
use std::collections::BTreeMap;
|
use std::collections::{BTreeMap, BTreeSet};
|
||||||
use std::path::{Path, PathBuf};
|
use std::path::{Path, PathBuf};
|
||||||
|
|
||||||
/// 待导入的 AssetBundle 数据。
|
/// 待导入的 AssetBundle 数据。
|
||||||
@@ -87,6 +88,22 @@ pub struct UnityFsImportSummary {
|
|||||||
pub directory_count: usize,
|
pub directory_count: usize,
|
||||||
/// UnityFS directory 路径。
|
/// UnityFS directory 路径。
|
||||||
pub directories: Vec<String>,
|
pub directories: Vec<String>,
|
||||||
|
/// UnityFS directory 解出的文件数量。
|
||||||
|
pub file_count: usize,
|
||||||
|
/// 成功解析出的 Unity serialized file 数量。
|
||||||
|
pub serialized_file_count: usize,
|
||||||
|
/// 成功解析出的 TextAsset 数量。
|
||||||
|
pub text_asset_count: usize,
|
||||||
|
/// TextAsset 名称列表。
|
||||||
|
pub text_assets: Vec<String>,
|
||||||
|
/// 非致命 serialized-file 解析诊断数量。
|
||||||
|
pub serialized_parse_error_count: usize,
|
||||||
|
/// 从 TextAsset 和 TypeTree 字段提取出的 TextUnit 数量。
|
||||||
|
pub text_unit_count: usize,
|
||||||
|
/// TextUnit 格式标签。
|
||||||
|
pub text_unit_formats: Vec<String>,
|
||||||
|
/// TextUnit 提取阶段的非致命诊断数量。
|
||||||
|
pub text_unit_error_count: usize,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Manifest 导入报告。
|
/// Manifest 导入报告。
|
||||||
@@ -198,6 +215,7 @@ impl<'a> ResourceImportService<'a> {
|
|||||||
id: resource_id_for_path(&entry.path),
|
id: resource_id_for_path(&entry.path),
|
||||||
local_path: PathBuf::from(&entry.path),
|
local_path: PathBuf::from(&entry.path),
|
||||||
entry: stored_entry,
|
entry: stored_entry,
|
||||||
|
metadata: ResourceMetadata::default(),
|
||||||
};
|
};
|
||||||
let id = self.resources.add(resource).await?;
|
let id = self.resources.add(resource).await?;
|
||||||
added_resources.push(id.clone());
|
added_resources.push(id.clone());
|
||||||
@@ -256,6 +274,14 @@ impl<'a> ResourceImportService<'a> {
|
|||||||
manifest_path, error
|
manifest_path, error
|
||||||
))
|
))
|
||||||
})?;
|
})?;
|
||||||
|
let text_units = TextUnitExtractor::new().extract_bundle(&parsed, Some(manifest_path));
|
||||||
|
let text_unit_formats = text_units
|
||||||
|
.units
|
||||||
|
.iter()
|
||||||
|
.filter_map(|unit| unit.context.get("format").cloned())
|
||||||
|
.collect::<BTreeSet<_>>()
|
||||||
|
.into_iter()
|
||||||
|
.collect();
|
||||||
|
|
||||||
Ok(UnityFsImportSummary {
|
Ok(UnityFsImportSummary {
|
||||||
unity_version: parsed.unity_version,
|
unity_version: parsed.unity_version,
|
||||||
@@ -266,6 +292,18 @@ impl<'a> ResourceImportService<'a> {
|
|||||||
.into_iter()
|
.into_iter()
|
||||||
.map(|directory| directory.path)
|
.map(|directory| directory.path)
|
||||||
.collect(),
|
.collect(),
|
||||||
|
file_count: parsed.files.len(),
|
||||||
|
serialized_file_count: parsed.serialized_files.len(),
|
||||||
|
text_asset_count: parsed.text_assets.len(),
|
||||||
|
text_assets: parsed
|
||||||
|
.text_assets
|
||||||
|
.into_iter()
|
||||||
|
.map(|asset| asset.name)
|
||||||
|
.collect(),
|
||||||
|
serialized_parse_error_count: parsed.serialized_parse_errors.len(),
|
||||||
|
text_unit_count: text_units.units.len(),
|
||||||
|
text_unit_formats,
|
||||||
|
text_unit_error_count: text_units.errors.len(),
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -375,6 +413,26 @@ mod tests {
|
|||||||
data.extend_from_slice(&value.to_be_bytes());
|
data.extend_from_slice(&value.to_be_bytes());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn push_i16_le(data: &mut Vec<u8>, value: i16) {
|
||||||
|
data.extend_from_slice(&value.to_le_bytes());
|
||||||
|
}
|
||||||
|
|
||||||
|
fn push_u32_le(data: &mut Vec<u8>, value: u32) {
|
||||||
|
data.extend_from_slice(&value.to_le_bytes());
|
||||||
|
}
|
||||||
|
|
||||||
|
fn push_i32_le(data: &mut Vec<u8>, value: i32) {
|
||||||
|
data.extend_from_slice(&value.to_le_bytes());
|
||||||
|
}
|
||||||
|
|
||||||
|
fn push_i64_le(data: &mut Vec<u8>, value: i64) {
|
||||||
|
data.extend_from_slice(&value.to_le_bytes());
|
||||||
|
}
|
||||||
|
|
||||||
|
fn push_u64_le(data: &mut Vec<u8>, value: u64) {
|
||||||
|
data.extend_from_slice(&value.to_le_bytes());
|
||||||
|
}
|
||||||
|
|
||||||
fn align(data: &mut Vec<u8>, alignment: usize) {
|
fn align(data: &mut Vec<u8>, alignment: usize) {
|
||||||
let remainder = data.len() % alignment;
|
let remainder = data.len() % alignment;
|
||||||
if remainder != 0 {
|
if remainder != 0 {
|
||||||
@@ -414,6 +472,83 @@ mod tests {
|
|||||||
data
|
data
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn synthetic_text_asset_unityfs_bundle() -> Vec<u8> {
|
||||||
|
let serialized_file = synthetic_serialized_text_asset();
|
||||||
|
let mut blocks_info = Vec::new();
|
||||||
|
blocks_info.extend_from_slice(&[1; 16]);
|
||||||
|
push_i32(&mut blocks_info, 1);
|
||||||
|
push_u32(&mut blocks_info, serialized_file.len() as u32);
|
||||||
|
push_u32(&mut blocks_info, serialized_file.len() as u32);
|
||||||
|
push_u16(&mut blocks_info, 0);
|
||||||
|
push_i32(&mut blocks_info, 1);
|
||||||
|
push_u64(&mut blocks_info, 0);
|
||||||
|
push_u64(&mut blocks_info, serialized_file.len() as u64);
|
||||||
|
push_u32(&mut blocks_info, 0);
|
||||||
|
push_c_string(&mut blocks_info, "CAB-scenario");
|
||||||
|
|
||||||
|
let mut data = Vec::new();
|
||||||
|
push_c_string(&mut data, "UnityFS");
|
||||||
|
push_u32(&mut data, 8);
|
||||||
|
push_c_string(&mut data, "5.x.x");
|
||||||
|
push_c_string(&mut data, "2021.3.56f2");
|
||||||
|
push_u64(&mut data, 0);
|
||||||
|
push_u32(&mut data, blocks_info.len() as u32);
|
||||||
|
push_u32(&mut data, blocks_info.len() as u32);
|
||||||
|
push_u32(&mut data, 0);
|
||||||
|
align(&mut data, 16);
|
||||||
|
data.extend_from_slice(&blocks_info);
|
||||||
|
data.extend_from_slice(&serialized_file);
|
||||||
|
|
||||||
|
let total_size = data.len() as u64;
|
||||||
|
let total_size_offset = b"UnityFS\0".len() + 4 + b"5.x.x\0".len() + b"2021.3.56f2\0".len();
|
||||||
|
data[total_size_offset..total_size_offset + 8].copy_from_slice(&total_size.to_be_bytes());
|
||||||
|
data
|
||||||
|
}
|
||||||
|
|
||||||
|
fn synthetic_serialized_text_asset() -> Vec<u8> {
|
||||||
|
let mut object_data = Vec::new();
|
||||||
|
push_u32_le(&mut object_data, 8);
|
||||||
|
object_data.extend_from_slice(b"Scenario");
|
||||||
|
align(&mut object_data, 4);
|
||||||
|
push_u32_le(&mut object_data, 15);
|
||||||
|
object_data.extend_from_slice("こんにちは".as_bytes());
|
||||||
|
|
||||||
|
let mut metadata = Vec::new();
|
||||||
|
metadata.extend_from_slice(b"2021.3.56f2\0");
|
||||||
|
push_i32_le(&mut metadata, 19);
|
||||||
|
metadata.push(0);
|
||||||
|
push_i32_le(&mut metadata, 1);
|
||||||
|
push_i32_le(&mut metadata, 49);
|
||||||
|
metadata.push(0);
|
||||||
|
push_i16_le(&mut metadata, 0);
|
||||||
|
metadata.extend_from_slice(&[0; 16]);
|
||||||
|
push_i32_le(&mut metadata, 1);
|
||||||
|
align(&mut metadata, 4);
|
||||||
|
push_i64_le(&mut metadata, 1);
|
||||||
|
push_u64_le(&mut metadata, 0);
|
||||||
|
push_u32_le(&mut metadata, object_data.len() as u32);
|
||||||
|
push_i32_le(&mut metadata, 0);
|
||||||
|
|
||||||
|
let header_len = 48usize;
|
||||||
|
let data_offset = header_len + metadata.len();
|
||||||
|
let file_size = data_offset + object_data.len();
|
||||||
|
|
||||||
|
let mut file = Vec::new();
|
||||||
|
push_u32(&mut file, metadata.len() as u32);
|
||||||
|
push_u32(&mut file, file_size as u32);
|
||||||
|
push_u32(&mut file, 22);
|
||||||
|
push_u32(&mut file, 0);
|
||||||
|
file.push(0);
|
||||||
|
file.extend_from_slice(&[0, 0, 0]);
|
||||||
|
push_u32(&mut file, metadata.len() as u32);
|
||||||
|
push_u64(&mut file, file_size as u64);
|
||||||
|
push_u64(&mut file, data_offset as u64);
|
||||||
|
push_u64(&mut file, 0);
|
||||||
|
file.extend_from_slice(&metadata);
|
||||||
|
file.extend_from_slice(&object_data);
|
||||||
|
file
|
||||||
|
}
|
||||||
|
|
||||||
fn synthetic_manifest() -> GenericManifest {
|
fn synthetic_manifest() -> GenericManifest {
|
||||||
GenericManifest {
|
GenericManifest {
|
||||||
format: ManifestFormat::AddressablesCatalog,
|
format: ManifestFormat::AddressablesCatalog,
|
||||||
@@ -594,6 +729,14 @@ mod tests {
|
|||||||
assert_eq!(unityfs.block_count, 1);
|
assert_eq!(unityfs.block_count, 1);
|
||||||
assert_eq!(unityfs.directory_count, 1);
|
assert_eq!(unityfs.directory_count, 1);
|
||||||
assert_eq!(unityfs.directories, vec!["SYNTHETIC-CAB".to_string()]);
|
assert_eq!(unityfs.directories, vec!["SYNTHETIC-CAB".to_string()]);
|
||||||
|
assert_eq!(unityfs.file_count, 1);
|
||||||
|
assert_eq!(unityfs.serialized_file_count, 0);
|
||||||
|
assert_eq!(unityfs.text_asset_count, 0);
|
||||||
|
assert!(unityfs.text_assets.is_empty());
|
||||||
|
assert_eq!(unityfs.serialized_parse_error_count, 0);
|
||||||
|
assert_eq!(unityfs.text_unit_count, 0);
|
||||||
|
assert!(unityfs.text_unit_formats.is_empty());
|
||||||
|
assert_eq!(unityfs.text_unit_error_count, 0);
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
report.imported[1].category,
|
report.imported[1].category,
|
||||||
ResourceImportCategory::TextAsset
|
ResourceImportCategory::TextAsset
|
||||||
@@ -645,6 +788,44 @@ mod tests {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn import_summary_reports_text_assets_inside_assetbundle() {
|
||||||
|
let temp_dir = TempDir::new().unwrap();
|
||||||
|
let cas = FileSystemCasRepository::new(temp_dir.path().join("cas"));
|
||||||
|
let resources = InMemoryResourceRepository::new();
|
||||||
|
let service = ResourceImportService::new(&cas, &resources);
|
||||||
|
let manifest = manifest_with(vec![ResourceEntry {
|
||||||
|
path: "synthetic/scenario.bundle".to_string(),
|
||||||
|
hash: "synthetic-scenario-hash".to_string(),
|
||||||
|
size: 1,
|
||||||
|
resource_type: ResourceType::AssetBundle,
|
||||||
|
address: None,
|
||||||
|
dependencies: Vec::new(),
|
||||||
|
crc: None,
|
||||||
|
}]);
|
||||||
|
|
||||||
|
let report = service
|
||||||
|
.import_manifest_bundles(
|
||||||
|
&manifest,
|
||||||
|
&[BundleSource::new(
|
||||||
|
"scenario.bundle",
|
||||||
|
synthetic_text_asset_unityfs_bundle(),
|
||||||
|
)],
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
let unityfs = report.imported[0].unityfs.as_ref().unwrap();
|
||||||
|
assert_eq!(unityfs.file_count, 1);
|
||||||
|
assert_eq!(unityfs.serialized_file_count, 1);
|
||||||
|
assert_eq!(unityfs.text_asset_count, 1);
|
||||||
|
assert_eq!(unityfs.text_assets, vec!["Scenario".to_string()]);
|
||||||
|
assert_eq!(unityfs.serialized_parse_error_count, 0);
|
||||||
|
assert_eq!(unityfs.text_unit_count, 1);
|
||||||
|
assert_eq!(unityfs.text_unit_formats, vec!["plain".to_string()]);
|
||||||
|
assert_eq!(unityfs.text_unit_error_count, 0);
|
||||||
|
}
|
||||||
|
|
||||||
#[tokio::test]
|
#[tokio::test]
|
||||||
async fn returns_error_when_bundle_data_is_missing() {
|
async fn returns_error_when_bundle_data_is_missing() {
|
||||||
let temp_dir = TempDir::new().unwrap();
|
let temp_dir = TempDir::new().unwrap();
|
||||||
|
|||||||
@@ -13,12 +13,18 @@
|
|||||||
pub mod cas;
|
pub mod cas;
|
||||||
mod curl_transfer;
|
mod curl_transfer;
|
||||||
pub mod import;
|
pub mod import;
|
||||||
|
pub mod localized_patch;
|
||||||
|
pub mod official_changes;
|
||||||
pub mod official_download;
|
pub mod official_download;
|
||||||
pub mod official_game_main_config;
|
pub mod official_game_main_config;
|
||||||
pub mod official_launcher;
|
pub mod official_launcher;
|
||||||
|
pub mod official_parse;
|
||||||
pub mod official_pull;
|
pub mod official_pull;
|
||||||
|
pub mod official_repository;
|
||||||
pub mod official_sync;
|
pub mod official_sync;
|
||||||
|
pub mod official_textunit_queue;
|
||||||
pub mod official_update;
|
pub mod official_update;
|
||||||
|
pub mod patch_ops;
|
||||||
pub mod path_security;
|
pub mod path_security;
|
||||||
pub mod resources;
|
pub mod resources;
|
||||||
mod zip_validation;
|
mod zip_validation;
|
||||||
@@ -31,6 +37,24 @@ pub use import::{
|
|||||||
BundleSource, ImportedResource, ResourceImportCategory, ResourceImportReport,
|
BundleSource, ImportedResource, ResourceImportCategory, ResourceImportReport,
|
||||||
ResourceImportService,
|
ResourceImportService,
|
||||||
};
|
};
|
||||||
|
pub use localized_patch::{
|
||||||
|
read_localized_patch_manifest_at, read_localized_version_state, LocalizedPatchConfig,
|
||||||
|
LocalizedPatchFile, LocalizedPatchIntegrity, LocalizedPatchManifest, LocalizedPatchOperation,
|
||||||
|
LocalizedPatchReport, LocalizedPatchRollbackInfo, LocalizedPatchService,
|
||||||
|
LocalizedTextAssetPatch, LocalizedVersionState, LOCALIZED_CURRENT_LINK,
|
||||||
|
LOCALIZED_PATCH_MANIFEST_FILE, LOCALIZED_PATCH_MANIFEST_VERSION, LOCALIZED_STAGING_DIR,
|
||||||
|
LOCALIZED_VERSIONS_DIR, LOCALIZED_VERSION_STATE_FILE,
|
||||||
|
};
|
||||||
|
pub use official_changes::{
|
||||||
|
read_resource_change_set_at, write_crowdin_translation_handoff_at,
|
||||||
|
write_official_resource_change_handoff, write_resource_change_set_at,
|
||||||
|
CrowdinTranslationHandoff, OfficialResourceChange, OfficialResourceChangeHandoffReport,
|
||||||
|
OfficialResourceChangeKind, OfficialResourceChangeSet, OfficialResourceChangeSummary,
|
||||||
|
OfficialResourceDescriptor, TranslationHandoffProvider, TranslationHandoffResource,
|
||||||
|
TranslationHandoffStatus, CROWDIN_TRANSLATION_HANDOFF_FILE,
|
||||||
|
CROWDIN_TRANSLATION_HANDOFF_VERSION, OFFICIAL_RESOURCE_CHANGES_FILE,
|
||||||
|
OFFICIAL_RESOURCE_CHANGES_VERSION,
|
||||||
|
};
|
||||||
pub use official_download::{
|
pub use official_download::{
|
||||||
read_download_manifest_at, DownloadError, OfficialDownloadManifest,
|
read_download_manifest_at, DownloadError, OfficialDownloadManifest,
|
||||||
OfficialDownloadManifestEntry, OfficialLocalManifestAuditItem,
|
OfficialDownloadManifestEntry, OfficialLocalManifestAuditItem,
|
||||||
@@ -45,24 +69,51 @@ pub use official_launcher::{
|
|||||||
OfficialLauncherBootstrapService, YostarJpLauncherCdnConfig, YostarJpLauncherGameConfig,
|
OfficialLauncherBootstrapService, YostarJpLauncherCdnConfig, YostarJpLauncherGameConfig,
|
||||||
YostarJpLauncherManifestUrl, YostarJpLauncherRemoteManifest,
|
YostarJpLauncherManifestUrl, YostarJpLauncherRemoteManifest,
|
||||||
};
|
};
|
||||||
|
pub use official_parse::{
|
||||||
|
query_textunit_index_errors, query_textunit_index_units, read_parse_cache_at,
|
||||||
|
read_textunit_index_at, write_parse_cache_at, write_textunit_index_at, OfficialParseCache,
|
||||||
|
OfficialParseCacheEntry, OfficialParseCacheService, OfficialParseConfig, OfficialParseReport,
|
||||||
|
OfficialParseSourceFingerprint, OfficialParseSourceKind, OfficialParseStatus,
|
||||||
|
OfficialParseSummary, OfficialTextUnitIndex, OfficialTextUnitIndexError,
|
||||||
|
OfficialTextUnitIndexSummary, OfficialTextUnitIndexUnit, OfficialTextUnitQuery,
|
||||||
|
OFFICIAL_PARSE_CACHE_FILE, OFFICIAL_PARSE_CACHE_VERSION, OFFICIAL_TEXTUNIT_INDEX_FILE,
|
||||||
|
OFFICIAL_TEXTUNIT_INDEX_VERSION,
|
||||||
|
};
|
||||||
pub use official_pull::{
|
pub use official_pull::{
|
||||||
build_official_pull_plan, build_official_pull_plan_for_platform_inventory,
|
build_official_pull_plan, build_official_pull_plan_for_platform_inventory,
|
||||||
build_official_pull_plan_for_platforms, build_official_pull_plan_from_platform_inventory,
|
build_official_pull_plan_for_platforms, build_official_pull_plan_from_platform_inventory,
|
||||||
OfficialResourcePullPlan,
|
OfficialResourcePullPlan,
|
||||||
};
|
};
|
||||||
|
pub use official_repository::{
|
||||||
|
OfficialReleaseImportConfig, OfficialReleaseImportReport, OfficialReleaseImportService,
|
||||||
|
};
|
||||||
pub use official_sync::{
|
pub use official_sync::{
|
||||||
build_official_sync_plan, changed_endpoint_urls, classify_sync_decision,
|
build_official_sync_plan, changed_endpoint_urls, classify_sync_decision,
|
||||||
default_official_platforms, OfficialSyncDecision, OfficialSyncPlan,
|
default_official_platforms, OfficialSyncDecision, OfficialSyncPlan,
|
||||||
};
|
};
|
||||||
|
pub use official_textunit_queue::{
|
||||||
|
read_textunit_task_queue_at, write_crowdin_textunit_queue_at, write_official_textunit_queues,
|
||||||
|
write_textunit_task_queue_at, CrowdinTextUnitQueue, CrowdinTextUnitQueueItem,
|
||||||
|
OfficialTextUnitQueueReport, OfficialTextUnitTask, OfficialTextUnitTaskQueue,
|
||||||
|
OfficialTextUnitTaskStatus, OfficialTextUnitTaskSummary, CROWDIN_TEXTUNIT_QUEUE_FILE,
|
||||||
|
CROWDIN_TEXTUNIT_QUEUE_VERSION, OFFICIAL_TEXTUNIT_TASK_QUEUE_FILE,
|
||||||
|
OFFICIAL_TEXTUNIT_TASK_QUEUE_VERSION,
|
||||||
|
};
|
||||||
pub use official_update::{
|
pub use official_update::{
|
||||||
cached_game_main_config_for_metadata, diff_extended_snapshot, gc_orphan_staging,
|
cached_game_main_config_for_metadata, diff_extended_snapshot, gc_orphan_staging,
|
||||||
read_bootstrap_cache, read_snapshot, read_version_state, write_bootstrap_cache, write_snapshot,
|
read_bootstrap_cache, read_snapshot, read_version_state, write_bootstrap_cache, write_snapshot,
|
||||||
write_version_state, ExtendedSnapshotDelta, GameMainConfigSnapshot, LauncherMetadataSnapshot,
|
write_version_state, ExtendedSnapshotDelta, GameMainConfigSnapshot, LauncherMetadataSnapshot,
|
||||||
OfficialBootstrapCache, OfficialEndpointMarkerRole, OfficialEndpointMarkerSnapshot,
|
LocalizedReleaseStatus, OfficialBootstrapCache, OfficialEndpointMarkerRole,
|
||||||
OfficialFailedVersionRecord, OfficialServerInfoSource, OfficialUpdateConfig,
|
OfficialEndpointMarkerSnapshot, OfficialFailedVersionRecord, OfficialServerInfoSource,
|
||||||
OfficialUpdateProgress, OfficialUpdateReport, OfficialUpdateService, OfficialUpdateSnapshot,
|
OfficialUpdateConfig, OfficialUpdateProgress, OfficialUpdateReport, OfficialUpdateService,
|
||||||
OfficialUpdateStatus, OfficialVerificationSummary, OfficialVersionRecord, OfficialVersionState,
|
OfficialUpdateSnapshot, OfficialUpdateStatus, OfficialVerificationSummary,
|
||||||
ResolvedBootstrap,
|
OfficialVersionRecord, OfficialVersionState, ResolvedBootstrap,
|
||||||
|
};
|
||||||
|
pub use patch_ops::{
|
||||||
|
apply_patch_file, apply_unityfs_field_patch_file, apply_unityfs_string_field_patch_file,
|
||||||
|
apply_unityfs_text_asset_patch_file, PatchApplyKind, PatchApplyParams, PatchApplyReport,
|
||||||
|
UnityFsFieldPatchParams, UnityFsPatchReport, UnityFsStringFieldPatchParams,
|
||||||
|
UnityFsTextAssetPatchParams,
|
||||||
};
|
};
|
||||||
pub use path_security::{
|
pub use path_security::{
|
||||||
ensure_path_within_root, ensure_safe_directory_path, ensure_safe_file_target, lexical_absolute,
|
ensure_path_within_root, ensure_safe_directory_path, ensure_safe_file_target, lexical_absolute,
|
||||||
|
|||||||
@@ -0,0 +1,857 @@
|
|||||||
|
//! Localized release publishing for verified TextAsset patches.
|
||||||
|
|
||||||
|
use bat_assetbundle::{patch_unityfs_text_asset, TextAssetPatch};
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use std::fs;
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
use std::time::{SystemTime, UNIX_EPOCH};
|
||||||
|
|
||||||
|
use crate::path_security::{
|
||||||
|
ensure_path_within_root, ensure_safe_directory_path, ensure_safe_file_target, lexical_absolute,
|
||||||
|
read_file_no_symlink, write_file_atomic, STATE_FILE_MODE,
|
||||||
|
};
|
||||||
|
|
||||||
|
/// Atomic current pointer under the localized output root.
|
||||||
|
pub const LOCALIZED_CURRENT_LINK: &str = "current";
|
||||||
|
/// Staging directory under the localized output root.
|
||||||
|
pub const LOCALIZED_STAGING_DIR: &str = ".staging";
|
||||||
|
/// Version directory under the localized output root.
|
||||||
|
pub const LOCALIZED_VERSIONS_DIR: &str = "versions";
|
||||||
|
/// Persisted localized release state file name.
|
||||||
|
pub const LOCALIZED_VERSION_STATE_FILE: &str = "localized-version-state.json";
|
||||||
|
/// Per-release patch manifest file name.
|
||||||
|
pub const LOCALIZED_PATCH_MANIFEST_FILE: &str = "localized-patch-manifest.json";
|
||||||
|
/// Current localized patch manifest schema version.
|
||||||
|
pub const LOCALIZED_PATCH_MANIFEST_VERSION: u32 = 1;
|
||||||
|
|
||||||
|
/// One patch operation against a bundle in an official release.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub struct LocalizedTextAssetPatch {
|
||||||
|
/// Relative path of the UnityFS bundle under the official release.
|
||||||
|
pub bundle_path: String,
|
||||||
|
/// TextAsset replacement inside the bundle.
|
||||||
|
pub text_asset: TextAssetPatch,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Configuration for one localized release publication.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub struct LocalizedPatchConfig {
|
||||||
|
/// Immutable, verified official release root.
|
||||||
|
pub official_release_root: PathBuf,
|
||||||
|
/// Separate localized publication root.
|
||||||
|
pub localized_output_root: PathBuf,
|
||||||
|
/// Version identifier shared with the official release.
|
||||||
|
pub release_id: String,
|
||||||
|
/// Patch operations to apply.
|
||||||
|
pub patches: Vec<LocalizedTextAssetPatch>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl LocalizedPatchConfig {
|
||||||
|
/// Creates a localized patch configuration.
|
||||||
|
pub fn new(
|
||||||
|
official_release_root: impl Into<PathBuf>,
|
||||||
|
localized_output_root: impl Into<PathBuf>,
|
||||||
|
release_id: impl Into<String>,
|
||||||
|
patches: Vec<LocalizedTextAssetPatch>,
|
||||||
|
) -> Self {
|
||||||
|
Self {
|
||||||
|
official_release_root: official_release_root.into(),
|
||||||
|
localized_output_root: localized_output_root.into(),
|
||||||
|
release_id: release_id.into(),
|
||||||
|
patches,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Persisted localized release state.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct LocalizedVersionState {
|
||||||
|
/// State schema version.
|
||||||
|
pub state_version: u32,
|
||||||
|
/// Official release ID used as the patch source.
|
||||||
|
pub official_release_id: String,
|
||||||
|
/// Published localized release ID.
|
||||||
|
pub current_release_id: Option<String>,
|
||||||
|
/// Stable status label.
|
||||||
|
pub status: String,
|
||||||
|
/// Last update time.
|
||||||
|
pub updated_unix_seconds: u64,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One changed file in a localized patch manifest.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct LocalizedPatchFile {
|
||||||
|
/// Relative bundle path.
|
||||||
|
pub path: String,
|
||||||
|
/// BLAKE3 before applying the patch.
|
||||||
|
pub original_blake3: String,
|
||||||
|
/// BLAKE3 after applying the patch.
|
||||||
|
pub localized_blake3: String,
|
||||||
|
/// Original file size in bytes.
|
||||||
|
#[serde(default)]
|
||||||
|
pub original_bytes: u64,
|
||||||
|
/// Localized file size in bytes.
|
||||||
|
#[serde(default)]
|
||||||
|
pub localized_bytes: u64,
|
||||||
|
/// Localized minus original byte size.
|
||||||
|
#[serde(default)]
|
||||||
|
pub byte_delta: i64,
|
||||||
|
/// TextAsset operations applied to this file.
|
||||||
|
#[serde(default)]
|
||||||
|
pub text_asset_operations: Vec<LocalizedPatchOperation>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One TextAsset patch operation recorded in the localized patch manifest.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct LocalizedPatchOperation {
|
||||||
|
/// UnityFS directory path of the serialized file.
|
||||||
|
pub serialized_file_path: String,
|
||||||
|
/// Unity object path ID.
|
||||||
|
pub path_id: i64,
|
||||||
|
/// Expected TextAsset name, when provided.
|
||||||
|
pub expected_name: Option<String>,
|
||||||
|
/// Replacement payload size.
|
||||||
|
pub replacement_bytes: u64,
|
||||||
|
/// BLAKE3 of the replacement payload.
|
||||||
|
pub replacement_blake3: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Rollback information recorded for a localized publication.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct LocalizedPatchRollbackInfo {
|
||||||
|
/// Previous `current` symlink target before this publication.
|
||||||
|
pub previous_current_target: Option<PathBuf>,
|
||||||
|
/// Version directory that should be removed when rolling this publication back.
|
||||||
|
pub remove_version_path: PathBuf,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Integrity summary for a published localized release.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct LocalizedPatchIntegrity {
|
||||||
|
/// Number of changed files verified against the manifest.
|
||||||
|
pub verified_changed_file_count: usize,
|
||||||
|
/// Number of TextAsset operations recorded in the manifest.
|
||||||
|
pub verified_text_asset_operation_count: usize,
|
||||||
|
/// Whether `current` points at this localized release.
|
||||||
|
pub current_points_to_release: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Persisted manifest for one localized release.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct LocalizedPatchManifest {
|
||||||
|
/// Manifest schema version.
|
||||||
|
#[serde(default = "default_patch_manifest_version")]
|
||||||
|
pub manifest_version: u32,
|
||||||
|
/// Official release ID used as the patch source.
|
||||||
|
pub official_release_id: String,
|
||||||
|
/// Published localized release ID.
|
||||||
|
pub localized_release_id: String,
|
||||||
|
/// Manifest generation time as Unix seconds.
|
||||||
|
pub generated_unix_seconds: u64,
|
||||||
|
/// Changed file count.
|
||||||
|
pub file_count: usize,
|
||||||
|
/// TextAsset operation count.
|
||||||
|
pub text_asset_operation_count: usize,
|
||||||
|
/// Changed files and their before/after hashes.
|
||||||
|
pub files: Vec<LocalizedPatchFile>,
|
||||||
|
/// Rollback information for this release.
|
||||||
|
pub rollback: LocalizedPatchRollbackInfo,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl LocalizedPatchManifest {
|
||||||
|
/// Converts the localized TextAsset manifest to the generic patch manifest model.
|
||||||
|
pub fn to_patch_manifest(&self) -> bat_patch::PatchManifest {
|
||||||
|
bat_patch::PatchManifest {
|
||||||
|
version: bat_patch::PATCH_MANIFEST_VERSION,
|
||||||
|
patch_id: self.localized_release_id.clone(),
|
||||||
|
source_version: self.official_release_id.clone(),
|
||||||
|
target_version: self.localized_release_id.clone(),
|
||||||
|
files: self
|
||||||
|
.files
|
||||||
|
.iter()
|
||||||
|
.map(|file| bat_patch::PatchManifestFile {
|
||||||
|
path: PathBuf::from(&file.path),
|
||||||
|
patch_kind: bat_patch::PatchKind::UnityFsTextAsset,
|
||||||
|
source_blake3: file.original_blake3.clone(),
|
||||||
|
target_blake3: file.localized_blake3.clone(),
|
||||||
|
source_size: file.original_bytes,
|
||||||
|
target_size: file.localized_bytes,
|
||||||
|
})
|
||||||
|
.collect(),
|
||||||
|
rollback: bat_patch::PatchRollback {
|
||||||
|
previous_current_target: self.rollback.previous_current_target.clone(),
|
||||||
|
remove_target_path: Some(self.rollback.remove_version_path.clone()),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Result of a successful localized release publication.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub struct LocalizedPatchReport {
|
||||||
|
/// Published version directory.
|
||||||
|
pub version_path: PathBuf,
|
||||||
|
/// Atomic current pointer.
|
||||||
|
pub current_path: PathBuf,
|
||||||
|
/// Version state path.
|
||||||
|
pub state_path: PathBuf,
|
||||||
|
/// Patch manifest path.
|
||||||
|
pub patch_manifest_path: PathBuf,
|
||||||
|
/// Changed files.
|
||||||
|
pub files: Vec<LocalizedPatchFile>,
|
||||||
|
/// Persisted patch manifest.
|
||||||
|
pub manifest: LocalizedPatchManifest,
|
||||||
|
/// Integrity check performed after publication.
|
||||||
|
pub integrity: LocalizedPatchIntegrity,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Applies TextAsset patches and atomically publishes a localized release.
|
||||||
|
#[derive(Debug, Default, Clone, Copy)]
|
||||||
|
pub struct LocalizedPatchService;
|
||||||
|
|
||||||
|
impl LocalizedPatchService {
|
||||||
|
/// Creates the publisher.
|
||||||
|
pub fn new() -> Self {
|
||||||
|
Self
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Copies the official release, applies patches in staging and publishes it.
|
||||||
|
pub fn publish(&self, config: &LocalizedPatchConfig) -> anyhow::Result<LocalizedPatchReport> {
|
||||||
|
let staging = config
|
||||||
|
.localized_output_root
|
||||||
|
.join(LOCALIZED_STAGING_DIR)
|
||||||
|
.join(&config.release_id);
|
||||||
|
let version_path = config
|
||||||
|
.localized_output_root
|
||||||
|
.join(LOCALIZED_VERSIONS_DIR)
|
||||||
|
.join(&config.release_id);
|
||||||
|
let current_path = config.localized_output_root.join(LOCALIZED_CURRENT_LINK);
|
||||||
|
let previous_current_target = current_symlink_target(¤t_path).ok().flatten();
|
||||||
|
let version_existed_before = version_path.exists();
|
||||||
|
match self.publish_inner(config, previous_current_target.clone()) {
|
||||||
|
Ok(report) => Ok(report),
|
||||||
|
Err(error) => {
|
||||||
|
if let Err(rollback_error) = rollback_failed_publish(
|
||||||
|
&config.localized_output_root,
|
||||||
|
&staging,
|
||||||
|
&version_path,
|
||||||
|
!version_existed_before,
|
||||||
|
¤t_path,
|
||||||
|
previous_current_target.as_ref(),
|
||||||
|
) {
|
||||||
|
return Err(anyhow::anyhow!(
|
||||||
|
"{error}; rollback failed: {rollback_error}"
|
||||||
|
));
|
||||||
|
}
|
||||||
|
Err(error)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn publish_inner(
|
||||||
|
&self,
|
||||||
|
config: &LocalizedPatchConfig,
|
||||||
|
previous_current_target: Option<PathBuf>,
|
||||||
|
) -> anyhow::Result<LocalizedPatchReport> {
|
||||||
|
validate_config(config).map_err(anyhow::Error::msg)?;
|
||||||
|
let staging = config
|
||||||
|
.localized_output_root
|
||||||
|
.join(LOCALIZED_STAGING_DIR)
|
||||||
|
.join(&config.release_id);
|
||||||
|
let version_path = config
|
||||||
|
.localized_output_root
|
||||||
|
.join(LOCALIZED_VERSIONS_DIR)
|
||||||
|
.join(&config.release_id);
|
||||||
|
let current_path = config.localized_output_root.join(LOCALIZED_CURRENT_LINK);
|
||||||
|
let state_path = config
|
||||||
|
.localized_output_root
|
||||||
|
.join(LOCALIZED_VERSION_STATE_FILE);
|
||||||
|
let patch_manifest_path = version_path.join(LOCALIZED_PATCH_MANIFEST_FILE);
|
||||||
|
|
||||||
|
if version_path.exists() {
|
||||||
|
return Err(anyhow::anyhow!(
|
||||||
|
"localized release already exists: {}",
|
||||||
|
version_path.display()
|
||||||
|
));
|
||||||
|
}
|
||||||
|
remove_owned_staging(&staging)?;
|
||||||
|
fs::create_dir_all(&staging)?;
|
||||||
|
copy_tree(&config.official_release_root, &staging)?;
|
||||||
|
|
||||||
|
let mut changed_files = Vec::with_capacity(config.patches.len());
|
||||||
|
for operation in &config.patches {
|
||||||
|
let target = staging.join(Path::new(&operation.bundle_path));
|
||||||
|
ensure_path_within_root(&staging, &target).map_err(anyhow::Error::msg)?;
|
||||||
|
ensure_safe_file_target(&staging, &target, "汉化 patch 输入")
|
||||||
|
.map_err(anyhow::Error::msg)?;
|
||||||
|
let original = fs::read(&target)?;
|
||||||
|
let patched = patch_unityfs_text_asset(&original, &operation.text_asset)
|
||||||
|
.map_err(|error| anyhow::anyhow!("{}: {error}", operation.bundle_path))?;
|
||||||
|
if original == patched {
|
||||||
|
return Err(anyhow::anyhow!(
|
||||||
|
"patch produced no change: {}",
|
||||||
|
operation.bundle_path
|
||||||
|
));
|
||||||
|
}
|
||||||
|
write_file_atomic(&target, &patched, STATE_FILE_MODE, "汉化 patch 输出")
|
||||||
|
.map_err(anyhow::Error::msg)?;
|
||||||
|
let original_blake3 = blake3::hash(&original).to_hex().to_string();
|
||||||
|
let localized_blake3 = blake3::hash(&patched).to_hex().to_string();
|
||||||
|
changed_files.push(LocalizedPatchFile {
|
||||||
|
path: operation.bundle_path.clone(),
|
||||||
|
original_blake3,
|
||||||
|
localized_blake3,
|
||||||
|
original_bytes: original.len() as u64,
|
||||||
|
localized_bytes: patched.len() as u64,
|
||||||
|
byte_delta: patched.len() as i64 - original.len() as i64,
|
||||||
|
text_asset_operations: vec![LocalizedPatchOperation::from_text_asset_patch(
|
||||||
|
&operation.text_asset,
|
||||||
|
)],
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
let manifest = LocalizedPatchManifest {
|
||||||
|
manifest_version: LOCALIZED_PATCH_MANIFEST_VERSION,
|
||||||
|
official_release_id: config.release_id.clone(),
|
||||||
|
localized_release_id: config.release_id.clone(),
|
||||||
|
generated_unix_seconds: unix_seconds_now(),
|
||||||
|
file_count: changed_files.len(),
|
||||||
|
text_asset_operation_count: changed_files
|
||||||
|
.iter()
|
||||||
|
.map(|file| file.text_asset_operations.len())
|
||||||
|
.sum(),
|
||||||
|
files: changed_files.clone(),
|
||||||
|
rollback: LocalizedPatchRollbackInfo {
|
||||||
|
previous_current_target: previous_current_target.clone(),
|
||||||
|
remove_version_path: version_path.clone(),
|
||||||
|
},
|
||||||
|
};
|
||||||
|
write_file_atomic(
|
||||||
|
&staging.join(LOCALIZED_PATCH_MANIFEST_FILE),
|
||||||
|
&serde_json::to_vec_pretty(&manifest)?,
|
||||||
|
STATE_FILE_MODE,
|
||||||
|
"汉化 patch manifest",
|
||||||
|
)
|
||||||
|
.map_err(anyhow::Error::msg)?;
|
||||||
|
verify_patch_manifest_files(&config.official_release_root, &staging, &manifest)?;
|
||||||
|
fs::create_dir_all(config.localized_output_root.join(LOCALIZED_VERSIONS_DIR))?;
|
||||||
|
fs::rename(&staging, &version_path)?;
|
||||||
|
switch_current_symlink(
|
||||||
|
&config.localized_output_root,
|
||||||
|
¤t_path,
|
||||||
|
&config.release_id,
|
||||||
|
)?;
|
||||||
|
|
||||||
|
let state = LocalizedVersionState {
|
||||||
|
state_version: 1,
|
||||||
|
official_release_id: config.release_id.clone(),
|
||||||
|
current_release_id: Some(config.release_id.clone()),
|
||||||
|
status: "localized".to_string(),
|
||||||
|
updated_unix_seconds: unix_seconds_now(),
|
||||||
|
};
|
||||||
|
write_file_atomic(
|
||||||
|
&state_path,
|
||||||
|
&serde_json::to_vec_pretty(&state)?,
|
||||||
|
STATE_FILE_MODE,
|
||||||
|
"汉化版本状态",
|
||||||
|
)
|
||||||
|
.map_err(anyhow::Error::msg)?;
|
||||||
|
let integrity = verify_published_localized_release(
|
||||||
|
&config.official_release_root,
|
||||||
|
&version_path,
|
||||||
|
¤t_path,
|
||||||
|
)?;
|
||||||
|
|
||||||
|
Ok(LocalizedPatchReport {
|
||||||
|
version_path,
|
||||||
|
current_path,
|
||||||
|
state_path,
|
||||||
|
patch_manifest_path,
|
||||||
|
files: changed_files,
|
||||||
|
manifest,
|
||||||
|
integrity,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Reads the localized release state without following a symlink at the file
|
||||||
|
/// path. A missing state file means no localized release has been published.
|
||||||
|
pub fn read_localized_version_state(
|
||||||
|
localized_output_root: &Path,
|
||||||
|
) -> anyhow::Result<Option<LocalizedVersionState>> {
|
||||||
|
let path = localized_output_root.join(LOCALIZED_VERSION_STATE_FILE);
|
||||||
|
let Some(bytes) = read_file_no_symlink(&path, "汉化版本状态").map_err(anyhow::Error::msg)?
|
||||||
|
else {
|
||||||
|
return Ok(None);
|
||||||
|
};
|
||||||
|
let state: LocalizedVersionState = serde_json::from_slice(&bytes)?;
|
||||||
|
if state.state_version != 1 {
|
||||||
|
return Err(anyhow::anyhow!(
|
||||||
|
"不支持的汉化版本状态 schema:{},当前版本=1",
|
||||||
|
state.state_version
|
||||||
|
));
|
||||||
|
}
|
||||||
|
Ok(Some(state))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Reads a localized patch manifest from a published version directory.
|
||||||
|
pub fn read_localized_patch_manifest_at(
|
||||||
|
version_path: &Path,
|
||||||
|
) -> anyhow::Result<Option<LocalizedPatchManifest>> {
|
||||||
|
let path = version_path.join(LOCALIZED_PATCH_MANIFEST_FILE);
|
||||||
|
let Some(bytes) =
|
||||||
|
read_file_no_symlink(&path, "汉化 patch manifest").map_err(anyhow::Error::msg)?
|
||||||
|
else {
|
||||||
|
return Ok(None);
|
||||||
|
};
|
||||||
|
let manifest: LocalizedPatchManifest = serde_json::from_slice(&bytes)?;
|
||||||
|
if manifest.manifest_version != LOCALIZED_PATCH_MANIFEST_VERSION {
|
||||||
|
return Err(anyhow::anyhow!(
|
||||||
|
"不支持的汉化 patch manifest schema:{},当前版本={}",
|
||||||
|
manifest.manifest_version,
|
||||||
|
LOCALIZED_PATCH_MANIFEST_VERSION
|
||||||
|
));
|
||||||
|
}
|
||||||
|
Ok(Some(manifest))
|
||||||
|
}
|
||||||
|
|
||||||
|
impl LocalizedPatchOperation {
|
||||||
|
fn from_text_asset_patch(patch: &TextAssetPatch) -> Self {
|
||||||
|
Self {
|
||||||
|
serialized_file_path: patch.serialized_file_path.clone(),
|
||||||
|
path_id: patch.path_id,
|
||||||
|
expected_name: patch.expected_name.clone(),
|
||||||
|
replacement_bytes: patch.replacement.len() as u64,
|
||||||
|
replacement_blake3: blake3::hash(&patch.replacement).to_hex().to_string(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn verify_published_localized_release(
|
||||||
|
official_release_root: &Path,
|
||||||
|
version_path: &Path,
|
||||||
|
current_path: &Path,
|
||||||
|
) -> anyhow::Result<LocalizedPatchIntegrity> {
|
||||||
|
let manifest = read_localized_patch_manifest_at(version_path)?.ok_or_else(|| {
|
||||||
|
anyhow::anyhow!(
|
||||||
|
"缺少汉化 patch manifest:{}",
|
||||||
|
version_path.join(LOCALIZED_PATCH_MANIFEST_FILE).display()
|
||||||
|
)
|
||||||
|
})?;
|
||||||
|
let mut integrity =
|
||||||
|
verify_patch_manifest_files(official_release_root, version_path, &manifest)?;
|
||||||
|
integrity.current_points_to_release = current_points_to_version(current_path, version_path)?;
|
||||||
|
if !integrity.current_points_to_release {
|
||||||
|
return Err(anyhow::anyhow!(
|
||||||
|
"汉化 current 未指向发布版本:current={} version={}",
|
||||||
|
current_path.display(),
|
||||||
|
version_path.display()
|
||||||
|
));
|
||||||
|
}
|
||||||
|
Ok(integrity)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn verify_patch_manifest_files(
|
||||||
|
official_release_root: &Path,
|
||||||
|
localized_release_root: &Path,
|
||||||
|
manifest: &LocalizedPatchManifest,
|
||||||
|
) -> anyhow::Result<LocalizedPatchIntegrity> {
|
||||||
|
let mut operation_count = 0usize;
|
||||||
|
for file in &manifest.files {
|
||||||
|
let relative = Path::new(&file.path);
|
||||||
|
let official_path = official_release_root.join(relative);
|
||||||
|
let localized_path = localized_release_root.join(relative);
|
||||||
|
ensure_path_within_root(official_release_root, &official_path)
|
||||||
|
.map_err(anyhow::Error::msg)?;
|
||||||
|
ensure_path_within_root(localized_release_root, &localized_path)
|
||||||
|
.map_err(anyhow::Error::msg)?;
|
||||||
|
ensure_safe_file_target(official_release_root, &official_path, "官方 patch 原文件")
|
||||||
|
.map_err(anyhow::Error::msg)?;
|
||||||
|
ensure_safe_file_target(localized_release_root, &localized_path, "汉化 patch 产物")
|
||||||
|
.map_err(anyhow::Error::msg)?;
|
||||||
|
let original = fs::read(&official_path)?;
|
||||||
|
let localized = fs::read(&localized_path)?;
|
||||||
|
let original_hash = blake3::hash(&original).to_hex().to_string();
|
||||||
|
let localized_hash = blake3::hash(&localized).to_hex().to_string();
|
||||||
|
if original_hash != file.original_blake3 || original.len() as u64 != file.original_bytes {
|
||||||
|
return Err(anyhow::anyhow!(
|
||||||
|
"汉化 manifest 原文件校验失败 {}:期望 hash={} bytes={},实际 hash={} bytes={}",
|
||||||
|
file.path,
|
||||||
|
file.original_blake3,
|
||||||
|
file.original_bytes,
|
||||||
|
original_hash,
|
||||||
|
original.len()
|
||||||
|
));
|
||||||
|
}
|
||||||
|
if localized_hash != file.localized_blake3 || localized.len() as u64 != file.localized_bytes
|
||||||
|
{
|
||||||
|
return Err(anyhow::anyhow!(
|
||||||
|
"汉化 manifest 产物校验失败 {}:期望 hash={} bytes={},实际 hash={} bytes={}",
|
||||||
|
file.path,
|
||||||
|
file.localized_blake3,
|
||||||
|
file.localized_bytes,
|
||||||
|
localized_hash,
|
||||||
|
localized.len()
|
||||||
|
));
|
||||||
|
}
|
||||||
|
if localized_hash == original_hash {
|
||||||
|
return Err(anyhow::anyhow!(
|
||||||
|
"汉化 manifest 文件未发生变化:{}",
|
||||||
|
file.path
|
||||||
|
));
|
||||||
|
}
|
||||||
|
operation_count += file.text_asset_operations.len();
|
||||||
|
}
|
||||||
|
if manifest.file_count != manifest.files.len() {
|
||||||
|
return Err(anyhow::anyhow!(
|
||||||
|
"汉化 manifest file_count 不一致:声明 {},实际 {}",
|
||||||
|
manifest.file_count,
|
||||||
|
manifest.files.len()
|
||||||
|
));
|
||||||
|
}
|
||||||
|
if manifest.text_asset_operation_count != operation_count {
|
||||||
|
return Err(anyhow::anyhow!(
|
||||||
|
"汉化 manifest operation_count 不一致:声明 {},实际 {}",
|
||||||
|
manifest.text_asset_operation_count,
|
||||||
|
operation_count
|
||||||
|
));
|
||||||
|
}
|
||||||
|
Ok(LocalizedPatchIntegrity {
|
||||||
|
verified_changed_file_count: manifest.files.len(),
|
||||||
|
verified_text_asset_operation_count: operation_count,
|
||||||
|
current_points_to_release: false,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
fn current_symlink_target(current_path: &Path) -> anyhow::Result<Option<PathBuf>> {
|
||||||
|
match fs::symlink_metadata(current_path) {
|
||||||
|
Ok(metadata) if metadata.file_type().is_symlink() => Ok(Some(fs::read_link(current_path)?)),
|
||||||
|
Ok(_) => Err(anyhow::anyhow!(
|
||||||
|
"汉化 current 已存在但不是 symlink:{}",
|
||||||
|
current_path.display()
|
||||||
|
)),
|
||||||
|
Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(None),
|
||||||
|
Err(error) => Err(error.into()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn current_points_to_version(current_path: &Path, version_path: &Path) -> anyhow::Result<bool> {
|
||||||
|
let metadata = fs::symlink_metadata(current_path)?;
|
||||||
|
if !metadata.file_type().is_symlink() {
|
||||||
|
return Ok(false);
|
||||||
|
}
|
||||||
|
Ok(fs::canonicalize(current_path)? == fs::canonicalize(version_path)?)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn rollback_failed_publish(
|
||||||
|
localized_output_root: &Path,
|
||||||
|
staging: &Path,
|
||||||
|
version_path: &Path,
|
||||||
|
remove_version_path: bool,
|
||||||
|
current_path: &Path,
|
||||||
|
previous_current_target: Option<&PathBuf>,
|
||||||
|
) -> anyhow::Result<()> {
|
||||||
|
remove_owned_path(staging)?;
|
||||||
|
if remove_version_path {
|
||||||
|
remove_owned_path(version_path)?;
|
||||||
|
}
|
||||||
|
remove_owned_path(&localized_output_root.join(".current.tmp"))?;
|
||||||
|
remove_owned_path(&localized_output_root.join(".current.rollback.tmp"))?;
|
||||||
|
let failed_target = version_path
|
||||||
|
.file_name()
|
||||||
|
.map(|release_id| Path::new(LOCALIZED_VERSIONS_DIR).join(release_id));
|
||||||
|
if failed_target.as_ref().is_some_and(|target| {
|
||||||
|
fs::read_link(current_path)
|
||||||
|
.map(|current_target| current_target == *target)
|
||||||
|
.unwrap_or(false)
|
||||||
|
}) {
|
||||||
|
restore_current_symlink(localized_output_root, current_path, previous_current_target)?;
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn remove_owned_path(path: &Path) -> anyhow::Result<()> {
|
||||||
|
if let Ok(metadata) = fs::symlink_metadata(path) {
|
||||||
|
if metadata.file_type().is_symlink() || metadata.is_file() {
|
||||||
|
fs::remove_file(path)?;
|
||||||
|
} else if metadata.is_dir() {
|
||||||
|
fs::remove_dir_all(path)?;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(unix)]
|
||||||
|
fn restore_current_symlink(
|
||||||
|
root: &Path,
|
||||||
|
current_path: &Path,
|
||||||
|
previous_current_target: Option<&PathBuf>,
|
||||||
|
) -> anyhow::Result<()> {
|
||||||
|
use std::os::unix::fs::symlink;
|
||||||
|
|
||||||
|
if let Some(previous_target) = previous_current_target {
|
||||||
|
let temporary = root.join(".current.rollback.tmp");
|
||||||
|
remove_owned_path(&temporary)?;
|
||||||
|
symlink(previous_target, &temporary)?;
|
||||||
|
fs::rename(temporary, current_path)?;
|
||||||
|
} else if fs::symlink_metadata(current_path)
|
||||||
|
.map(|metadata| metadata.file_type().is_symlink())
|
||||||
|
.unwrap_or(false)
|
||||||
|
{
|
||||||
|
fs::remove_file(current_path)?;
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(not(unix))]
|
||||||
|
fn restore_current_symlink(
|
||||||
|
_root: &Path,
|
||||||
|
_current_path: &Path,
|
||||||
|
_previous_current_target: Option<&PathBuf>,
|
||||||
|
) -> anyhow::Result<()> {
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn validate_config(config: &LocalizedPatchConfig) -> Result<(), String> {
|
||||||
|
let official = lexical_absolute(&config.official_release_root)?;
|
||||||
|
let localized = lexical_absolute(&config.localized_output_root)?;
|
||||||
|
if official == localized || official.starts_with(&localized) || localized.starts_with(&official)
|
||||||
|
{
|
||||||
|
return Err(format!(
|
||||||
|
"官方 release 与汉化输出目录不能相同或互相嵌套:官方={} 汉化={}",
|
||||||
|
official.display(),
|
||||||
|
localized.display()
|
||||||
|
));
|
||||||
|
}
|
||||||
|
if config.release_id.is_empty()
|
||||||
|
|| config.release_id.contains('/')
|
||||||
|
|| config.release_id.contains('\\')
|
||||||
|
|| config.release_id == "."
|
||||||
|
|| config.release_id == ".."
|
||||||
|
{
|
||||||
|
return Err(format!("非法汉化 release id:{}", config.release_id));
|
||||||
|
}
|
||||||
|
ensure_safe_directory_path(&config.official_release_root, "官方 release")?;
|
||||||
|
ensure_safe_directory_path(&config.localized_output_root, "汉化输出目录")?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn copy_tree(source: &Path, destination: &Path) -> anyhow::Result<()> {
|
||||||
|
let metadata = fs::symlink_metadata(source)?;
|
||||||
|
if metadata.file_type().is_symlink() {
|
||||||
|
return Err(anyhow::anyhow!(
|
||||||
|
"官方 release 不能包含 symlink: {}",
|
||||||
|
source.display()
|
||||||
|
));
|
||||||
|
}
|
||||||
|
if metadata.is_dir() {
|
||||||
|
fs::create_dir_all(destination)?;
|
||||||
|
for entry in fs::read_dir(source)? {
|
||||||
|
let entry = entry?;
|
||||||
|
copy_tree(&entry.path(), &destination.join(entry.file_name()))?;
|
||||||
|
}
|
||||||
|
} else if metadata.is_file() {
|
||||||
|
if let Some(parent) = destination.parent() {
|
||||||
|
fs::create_dir_all(parent)?;
|
||||||
|
}
|
||||||
|
fs::copy(source, destination)?;
|
||||||
|
} else {
|
||||||
|
return Err(anyhow::anyhow!(
|
||||||
|
"官方 release 中存在非普通文件: {}",
|
||||||
|
source.display()
|
||||||
|
));
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn remove_owned_staging(path: &Path) -> anyhow::Result<()> {
|
||||||
|
if let Ok(metadata) = fs::symlink_metadata(path) {
|
||||||
|
if metadata.file_type().is_symlink() {
|
||||||
|
return Err(anyhow::anyhow!(
|
||||||
|
"汉化 staging 不能是 symlink: {}",
|
||||||
|
path.display()
|
||||||
|
));
|
||||||
|
}
|
||||||
|
if metadata.is_dir() {
|
||||||
|
fs::remove_dir_all(path)?;
|
||||||
|
} else {
|
||||||
|
fs::remove_file(path)?;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(unix)]
|
||||||
|
fn switch_current_symlink(root: &Path, current: &Path, release_id: &str) -> anyhow::Result<()> {
|
||||||
|
use std::os::unix::fs::symlink;
|
||||||
|
|
||||||
|
let temporary = root.join(".current.tmp");
|
||||||
|
if let Ok(metadata) = fs::symlink_metadata(&temporary) {
|
||||||
|
if metadata.file_type().is_symlink() || metadata.is_file() {
|
||||||
|
fs::remove_file(&temporary)?;
|
||||||
|
} else if metadata.is_dir() {
|
||||||
|
fs::remove_dir_all(&temporary)?;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
symlink(
|
||||||
|
Path::new(LOCALIZED_VERSIONS_DIR).join(release_id),
|
||||||
|
&temporary,
|
||||||
|
)?;
|
||||||
|
fs::rename(temporary, current)?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(not(unix))]
|
||||||
|
fn switch_current_symlink(_root: &Path, _current: &Path, _release_id: &str) -> anyhow::Result<()> {
|
||||||
|
Err(anyhow::anyhow!(
|
||||||
|
"localized release publication requires a Unix symlink-capable platform"
|
||||||
|
))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn unix_seconds_now() -> u64 {
|
||||||
|
SystemTime::now()
|
||||||
|
.duration_since(UNIX_EPOCH)
|
||||||
|
.unwrap_or_default()
|
||||||
|
.as_secs()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn default_patch_manifest_version() -> u32 {
|
||||||
|
LOCALIZED_PATCH_MANIFEST_VERSION
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use tempfile::TempDir;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn localized_manifest_converts_to_generic_patch_manifest() {
|
||||||
|
let source = b"source";
|
||||||
|
let target = b"target";
|
||||||
|
let manifest = LocalizedPatchManifest {
|
||||||
|
manifest_version: LOCALIZED_PATCH_MANIFEST_VERSION,
|
||||||
|
official_release_id: "official-v1".to_string(),
|
||||||
|
localized_release_id: "localized-v1".to_string(),
|
||||||
|
generated_unix_seconds: 123,
|
||||||
|
file_count: 1,
|
||||||
|
text_asset_operation_count: 1,
|
||||||
|
files: vec![LocalizedPatchFile {
|
||||||
|
path: "Bundles/file.bundle".to_string(),
|
||||||
|
original_blake3: blake3::hash(source).to_hex().to_string(),
|
||||||
|
localized_blake3: blake3::hash(target).to_hex().to_string(),
|
||||||
|
original_bytes: source.len() as u64,
|
||||||
|
localized_bytes: target.len() as u64,
|
||||||
|
byte_delta: target.len() as i64 - source.len() as i64,
|
||||||
|
text_asset_operations: vec![LocalizedPatchOperation {
|
||||||
|
serialized_file_path: "CAB-asset".to_string(),
|
||||||
|
path_id: 1,
|
||||||
|
expected_name: Some("Text".to_string()),
|
||||||
|
replacement_bytes: target.len() as u64,
|
||||||
|
replacement_blake3: blake3::hash(target).to_hex().to_string(),
|
||||||
|
}],
|
||||||
|
}],
|
||||||
|
rollback: LocalizedPatchRollbackInfo {
|
||||||
|
previous_current_target: Some(PathBuf::from("versions/previous")),
|
||||||
|
remove_version_path: PathBuf::from("versions/localized-v1"),
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
let patch_manifest = manifest.to_patch_manifest();
|
||||||
|
|
||||||
|
assert_eq!(patch_manifest.version, bat_patch::PATCH_MANIFEST_VERSION);
|
||||||
|
assert_eq!(patch_manifest.patch_id, "localized-v1");
|
||||||
|
assert_eq!(patch_manifest.source_version, "official-v1");
|
||||||
|
assert_eq!(patch_manifest.target_version, "localized-v1");
|
||||||
|
assert_eq!(patch_manifest.files.len(), 1);
|
||||||
|
assert_eq!(
|
||||||
|
patch_manifest.files[0].patch_kind,
|
||||||
|
bat_patch::PatchKind::UnityFsTextAsset
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
patch_manifest.rollback.previous_current_target,
|
||||||
|
Some(PathBuf::from("versions/previous"))
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
patch_manifest.rollback.remove_target_path,
|
||||||
|
Some(PathBuf::from("versions/localized-v1"))
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(unix)]
|
||||||
|
#[test]
|
||||||
|
fn publishes_a_separate_localized_release_atomically() {
|
||||||
|
let temp = TempDir::new().unwrap();
|
||||||
|
let official = temp.path().join("official-release");
|
||||||
|
let localized = temp.path().join("localized");
|
||||||
|
fs::create_dir_all(official.join("TableBundles")).unwrap();
|
||||||
|
fs::write(
|
||||||
|
official.join("TableBundles/TableCatalog.bytes"),
|
||||||
|
b"official",
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
let report = LocalizedPatchService::new()
|
||||||
|
.publish(&LocalizedPatchConfig::new(
|
||||||
|
&official,
|
||||||
|
&localized,
|
||||||
|
"release-1",
|
||||||
|
Vec::new(),
|
||||||
|
))
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
assert_eq!(
|
||||||
|
fs::read(report.version_path.join("TableBundles/TableCatalog.bytes")).unwrap(),
|
||||||
|
b"official"
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
fs::read_link(report.current_path).unwrap(),
|
||||||
|
PathBuf::from("versions/release-1")
|
||||||
|
);
|
||||||
|
let state: LocalizedVersionState =
|
||||||
|
serde_json::from_slice(&fs::read(report.state_path).unwrap()).unwrap();
|
||||||
|
assert_eq!(state.status, "localized");
|
||||||
|
assert_eq!(state.current_release_id.as_deref(), Some("release-1"));
|
||||||
|
assert!(report.patch_manifest_path.is_file());
|
||||||
|
assert_eq!(report.manifest.file_count, 0);
|
||||||
|
assert_eq!(report.integrity.verified_changed_file_count, 0);
|
||||||
|
assert!(report.integrity.current_points_to_release);
|
||||||
|
let manifest = read_localized_patch_manifest_at(&report.version_path)
|
||||||
|
.unwrap()
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(manifest.localized_release_id, "release-1");
|
||||||
|
assert_eq!(manifest.rollback.previous_current_target, None);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(unix)]
|
||||||
|
#[test]
|
||||||
|
fn failed_patch_publish_cleans_staging_and_unpublished_version() {
|
||||||
|
let temp = TempDir::new().unwrap();
|
||||||
|
let official = temp.path().join("official-release");
|
||||||
|
let localized = temp.path().join("localized");
|
||||||
|
fs::create_dir_all(official.join("Bundles")).unwrap();
|
||||||
|
fs::write(official.join("Bundles/bad.bundle"), b"not-unityfs").unwrap();
|
||||||
|
|
||||||
|
let error = LocalizedPatchService::new()
|
||||||
|
.publish(&LocalizedPatchConfig::new(
|
||||||
|
&official,
|
||||||
|
&localized,
|
||||||
|
"release-1",
|
||||||
|
vec![LocalizedTextAssetPatch {
|
||||||
|
bundle_path: "Bundles/bad.bundle".to_string(),
|
||||||
|
text_asset: TextAssetPatch::new("CAB-bad", 1, b"replacement".to_vec()),
|
||||||
|
}],
|
||||||
|
))
|
||||||
|
.unwrap_err();
|
||||||
|
|
||||||
|
assert!(error.to_string().contains("Bundles/bad.bundle"));
|
||||||
|
assert!(!localized
|
||||||
|
.join(LOCALIZED_STAGING_DIR)
|
||||||
|
.join("release-1")
|
||||||
|
.exists());
|
||||||
|
assert!(!localized
|
||||||
|
.join(LOCALIZED_VERSIONS_DIR)
|
||||||
|
.join("release-1")
|
||||||
|
.exists());
|
||||||
|
assert!(!localized.join(LOCALIZED_CURRENT_LINK).exists());
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,653 @@
|
|||||||
|
//! Official resource change sets and translation handoff files.
|
||||||
|
//!
|
||||||
|
//! This module is intentionally file-based. Official update generates the
|
||||||
|
//! durable change set after a new release has been fully downloaded and
|
||||||
|
//! verified; parser and translation modules can then consume the same immutable
|
||||||
|
//! handoff without depending on daemon internals.
|
||||||
|
|
||||||
|
use crate::official_download::{
|
||||||
|
read_download_manifest_at, OfficialDownloadManifest, OfficialDownloadManifestEntry,
|
||||||
|
};
|
||||||
|
use crate::path_security::{
|
||||||
|
ensure_path_within_root, ensure_safe_file_target, read_file_no_symlink, write_file_atomic,
|
||||||
|
STATE_FILE_MODE,
|
||||||
|
};
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use std::collections::{BTreeMap, BTreeSet};
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
use std::time::{SystemTime, UNIX_EPOCH};
|
||||||
|
|
||||||
|
/// Current resource-change-set schema version.
|
||||||
|
pub const OFFICIAL_RESOURCE_CHANGES_VERSION: u32 = 1;
|
||||||
|
/// File name stored under a published official release root.
|
||||||
|
pub const OFFICIAL_RESOURCE_CHANGES_FILE: &str = "official-resource-changes.json";
|
||||||
|
/// Current Crowdin handoff schema version.
|
||||||
|
pub const CROWDIN_TRANSLATION_HANDOFF_VERSION: u32 = 1;
|
||||||
|
/// File name stored under a published official release root.
|
||||||
|
pub const CROWDIN_TRANSLATION_HANDOFF_FILE: &str = "crowdin-translation-handoff.json";
|
||||||
|
|
||||||
|
/// Change kind for one official resource destination.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
#[serde(rename_all = "snake_case")]
|
||||||
|
pub enum OfficialResourceChangeKind {
|
||||||
|
/// Destination did not exist in the previous complete release.
|
||||||
|
Added,
|
||||||
|
/// Destination exists in both releases, but verified size or BLAKE3 changed.
|
||||||
|
Modified,
|
||||||
|
/// Destination existed in the previous release but is absent from the new one.
|
||||||
|
Removed,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl OfficialResourceChangeKind {
|
||||||
|
/// Returns the stable JSON/RPC label for the change kind.
|
||||||
|
pub fn as_str(self) -> &'static str {
|
||||||
|
match self {
|
||||||
|
Self::Added => "added",
|
||||||
|
Self::Modified => "modified",
|
||||||
|
Self::Removed => "removed",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Returns true when the changed resource should be offered to parser and
|
||||||
|
/// translation modules.
|
||||||
|
pub fn is_incremental_candidate(self) -> bool {
|
||||||
|
matches!(self, Self::Added | Self::Modified)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Verified manifest attributes for one official resource.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct OfficialResourceDescriptor {
|
||||||
|
/// Official URL from the download manifest.
|
||||||
|
pub url: String,
|
||||||
|
/// Relative path under the official release root.
|
||||||
|
pub destination: String,
|
||||||
|
/// Verified byte count from the download manifest.
|
||||||
|
pub bytes: u64,
|
||||||
|
/// Verified BLAKE3 digest from the download manifest.
|
||||||
|
pub blake3: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl From<&OfficialDownloadManifestEntry> for OfficialResourceDescriptor {
|
||||||
|
fn from(entry: &OfficialDownloadManifestEntry) -> Self {
|
||||||
|
Self {
|
||||||
|
url: entry.url.clone(),
|
||||||
|
destination: entry.destination.clone(),
|
||||||
|
bytes: entry.bytes,
|
||||||
|
blake3: entry.blake3.clone(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One resource-level change between two complete official releases.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct OfficialResourceChange {
|
||||||
|
/// Relative path used as the stable comparison key.
|
||||||
|
pub destination: String,
|
||||||
|
/// Change kind for this destination.
|
||||||
|
pub kind: OfficialResourceChangeKind,
|
||||||
|
/// Previous release descriptor, when the destination existed before.
|
||||||
|
pub previous: Option<OfficialResourceDescriptor>,
|
||||||
|
/// Current release descriptor, when the destination exists now.
|
||||||
|
pub current: Option<OfficialResourceDescriptor>,
|
||||||
|
/// Whether parser modules should inspect this resource in incremental mode.
|
||||||
|
pub parse_candidate: bool,
|
||||||
|
/// Whether translation modules should enqueue this resource in incremental mode.
|
||||||
|
pub translation_candidate: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl OfficialResourceChange {
|
||||||
|
fn new(
|
||||||
|
destination: String,
|
||||||
|
kind: OfficialResourceChangeKind,
|
||||||
|
previous: Option<OfficialResourceDescriptor>,
|
||||||
|
current: Option<OfficialResourceDescriptor>,
|
||||||
|
) -> Self {
|
||||||
|
let is_candidate = kind.is_incremental_candidate();
|
||||||
|
Self {
|
||||||
|
destination,
|
||||||
|
kind,
|
||||||
|
previous,
|
||||||
|
current,
|
||||||
|
parse_candidate: is_candidate,
|
||||||
|
translation_candidate: is_candidate,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Aggregate counters for one official resource change set.
|
||||||
|
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct OfficialResourceChangeSummary {
|
||||||
|
/// Whether a previous complete release manifest was available.
|
||||||
|
pub previous_manifest_present: bool,
|
||||||
|
/// Number of entries in the previous release manifest.
|
||||||
|
pub previous_manifest_entry_count: usize,
|
||||||
|
/// Number of entries in the current release manifest.
|
||||||
|
pub current_manifest_entry_count: usize,
|
||||||
|
/// Number of newly added destinations.
|
||||||
|
pub added_count: usize,
|
||||||
|
/// Number of destinations whose verified bytes or BLAKE3 changed.
|
||||||
|
pub modified_count: usize,
|
||||||
|
/// Number of destinations removed from the current release.
|
||||||
|
pub removed_count: usize,
|
||||||
|
/// Number of resources to offer to parser modules.
|
||||||
|
pub parse_candidate_count: usize,
|
||||||
|
/// Number of resources to offer to translation modules.
|
||||||
|
pub translation_candidate_count: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Durable comparison result between a previous complete official release and
|
||||||
|
/// the newly published official release.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct OfficialResourceChangeSet {
|
||||||
|
/// Change-set schema version.
|
||||||
|
#[serde(default = "default_resource_changes_version")]
|
||||||
|
pub change_set_version: u32,
|
||||||
|
/// Current official release ID.
|
||||||
|
pub official_release_id: String,
|
||||||
|
/// Previous official release ID, when known.
|
||||||
|
pub previous_release_id: Option<String>,
|
||||||
|
/// Generation time as Unix seconds.
|
||||||
|
pub generated_unix_seconds: u64,
|
||||||
|
/// Previous complete release root, when available.
|
||||||
|
pub previous_resource_root: Option<PathBuf>,
|
||||||
|
/// Current complete release root.
|
||||||
|
pub current_resource_root: PathBuf,
|
||||||
|
/// Aggregate counters.
|
||||||
|
pub summary: OfficialResourceChangeSummary,
|
||||||
|
/// Stable, destination-sorted list of changed resources.
|
||||||
|
pub changes: Vec<OfficialResourceChange>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl OfficialResourceChangeSet {
|
||||||
|
/// Builds a change set from already loaded manifests.
|
||||||
|
pub fn from_manifests(
|
||||||
|
official_release_id: impl Into<String>,
|
||||||
|
previous_release_id: Option<String>,
|
||||||
|
previous_resource_root: Option<PathBuf>,
|
||||||
|
current_resource_root: PathBuf,
|
||||||
|
previous_manifest: Option<&OfficialDownloadManifest>,
|
||||||
|
current_manifest: &OfficialDownloadManifest,
|
||||||
|
) -> Self {
|
||||||
|
let previous_by_destination = previous_manifest
|
||||||
|
.map(entries_by_destination)
|
||||||
|
.unwrap_or_default();
|
||||||
|
let current_by_destination = entries_by_destination(current_manifest);
|
||||||
|
let destinations = previous_by_destination
|
||||||
|
.keys()
|
||||||
|
.chain(current_by_destination.keys())
|
||||||
|
.cloned()
|
||||||
|
.collect::<BTreeSet<_>>();
|
||||||
|
|
||||||
|
let mut changes = Vec::new();
|
||||||
|
let mut summary = OfficialResourceChangeSummary {
|
||||||
|
previous_manifest_present: previous_manifest.is_some(),
|
||||||
|
previous_manifest_entry_count: previous_manifest
|
||||||
|
.map(|manifest| manifest.entries.len())
|
||||||
|
.unwrap_or(0),
|
||||||
|
current_manifest_entry_count: current_manifest.entries.len(),
|
||||||
|
..OfficialResourceChangeSummary::default()
|
||||||
|
};
|
||||||
|
|
||||||
|
for destination in destinations {
|
||||||
|
match (
|
||||||
|
previous_by_destination.get(&destination),
|
||||||
|
current_by_destination.get(&destination),
|
||||||
|
) {
|
||||||
|
(None, Some(current)) => {
|
||||||
|
summary.added_count += 1;
|
||||||
|
changes.push(OfficialResourceChange::new(
|
||||||
|
destination,
|
||||||
|
OfficialResourceChangeKind::Added,
|
||||||
|
None,
|
||||||
|
Some((*current).into()),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
(Some(previous), Some(current)) if content_changed(previous, current) => {
|
||||||
|
summary.modified_count += 1;
|
||||||
|
changes.push(OfficialResourceChange::new(
|
||||||
|
destination,
|
||||||
|
OfficialResourceChangeKind::Modified,
|
||||||
|
Some((*previous).into()),
|
||||||
|
Some((*current).into()),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
(Some(previous), None) => {
|
||||||
|
summary.removed_count += 1;
|
||||||
|
changes.push(OfficialResourceChange::new(
|
||||||
|
destination,
|
||||||
|
OfficialResourceChangeKind::Removed,
|
||||||
|
Some((*previous).into()),
|
||||||
|
None,
|
||||||
|
));
|
||||||
|
}
|
||||||
|
_ => {}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
summary.parse_candidate_count = changes
|
||||||
|
.iter()
|
||||||
|
.filter(|change| change.parse_candidate)
|
||||||
|
.count();
|
||||||
|
summary.translation_candidate_count = changes
|
||||||
|
.iter()
|
||||||
|
.filter(|change| change.translation_candidate)
|
||||||
|
.count();
|
||||||
|
|
||||||
|
Self {
|
||||||
|
change_set_version: OFFICIAL_RESOURCE_CHANGES_VERSION,
|
||||||
|
official_release_id: official_release_id.into(),
|
||||||
|
previous_release_id,
|
||||||
|
generated_unix_seconds: unix_seconds_now(),
|
||||||
|
previous_resource_root,
|
||||||
|
current_resource_root,
|
||||||
|
summary,
|
||||||
|
changes,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Returns the resources that parser modules should inspect for
|
||||||
|
/// incremental work.
|
||||||
|
pub fn parse_candidates(&self) -> Vec<&OfficialResourceChange> {
|
||||||
|
self.changes
|
||||||
|
.iter()
|
||||||
|
.filter(|change| change.parse_candidate)
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Returns the resources that translation modules should enqueue.
|
||||||
|
pub fn translation_candidates(&self) -> Vec<&OfficialResourceChange> {
|
||||||
|
self.changes
|
||||||
|
.iter()
|
||||||
|
.filter(|change| change.translation_candidate)
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Provider reserved for translation handoff consumers.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
#[serde(rename_all = "snake_case")]
|
||||||
|
pub enum TranslationHandoffProvider {
|
||||||
|
/// Crowdin provider. The handoff file does not make a network request.
|
||||||
|
Crowdin,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl TranslationHandoffProvider {
|
||||||
|
/// Returns the stable provider label.
|
||||||
|
pub fn as_str(self) -> &'static str {
|
||||||
|
match self {
|
||||||
|
Self::Crowdin => "crowdin",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Status of a generated translation handoff.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
#[serde(rename_all = "snake_case")]
|
||||||
|
pub enum TranslationHandoffStatus {
|
||||||
|
/// The handoff was written locally and is waiting for a translation worker.
|
||||||
|
QueuedOffline,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl TranslationHandoffStatus {
|
||||||
|
/// Returns the stable status label.
|
||||||
|
pub fn as_str(self) -> &'static str {
|
||||||
|
match self {
|
||||||
|
Self::QueuedOffline => "queued_offline",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One resource entry queued for translation-provider processing.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct TranslationHandoffResource {
|
||||||
|
/// Relative path under the official release root.
|
||||||
|
pub destination: String,
|
||||||
|
/// Change kind that caused this resource to be queued.
|
||||||
|
pub kind: OfficialResourceChangeKind,
|
||||||
|
/// Current official URL.
|
||||||
|
pub url: String,
|
||||||
|
/// Verified byte count.
|
||||||
|
pub bytes: u64,
|
||||||
|
/// Verified BLAKE3 digest.
|
||||||
|
pub blake3: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Crowdin-ready local queue file for added or modified official resources.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct CrowdinTranslationHandoff {
|
||||||
|
/// Handoff schema version.
|
||||||
|
#[serde(default = "default_crowdin_handoff_version")]
|
||||||
|
pub handoff_version: u32,
|
||||||
|
/// Translation provider reserved for this queue.
|
||||||
|
pub provider: TranslationHandoffProvider,
|
||||||
|
/// Current queue status.
|
||||||
|
pub status: TranslationHandoffStatus,
|
||||||
|
/// Current official release ID.
|
||||||
|
pub official_release_id: String,
|
||||||
|
/// Previous official release ID, when known.
|
||||||
|
pub previous_release_id: Option<String>,
|
||||||
|
/// Generation time as Unix seconds.
|
||||||
|
pub generated_unix_seconds: u64,
|
||||||
|
/// Number of queued resources.
|
||||||
|
pub resource_count: usize,
|
||||||
|
/// Added or modified resources to pass into parsing/translation workers.
|
||||||
|
pub resources: Vec<TranslationHandoffResource>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl CrowdinTranslationHandoff {
|
||||||
|
/// Builds a local Crowdin handoff from a resource change set.
|
||||||
|
pub fn from_change_set(change_set: &OfficialResourceChangeSet) -> Self {
|
||||||
|
let resources = change_set
|
||||||
|
.translation_candidates()
|
||||||
|
.into_iter()
|
||||||
|
.filter_map(|change| {
|
||||||
|
let current = change.current.as_ref()?;
|
||||||
|
Some(TranslationHandoffResource {
|
||||||
|
destination: change.destination.clone(),
|
||||||
|
kind: change.kind,
|
||||||
|
url: current.url.clone(),
|
||||||
|
bytes: current.bytes,
|
||||||
|
blake3: current.blake3.clone(),
|
||||||
|
})
|
||||||
|
})
|
||||||
|
.collect::<Vec<_>>();
|
||||||
|
|
||||||
|
Self {
|
||||||
|
handoff_version: CROWDIN_TRANSLATION_HANDOFF_VERSION,
|
||||||
|
provider: TranslationHandoffProvider::Crowdin,
|
||||||
|
status: TranslationHandoffStatus::QueuedOffline,
|
||||||
|
official_release_id: change_set.official_release_id.clone(),
|
||||||
|
previous_release_id: change_set.previous_release_id.clone(),
|
||||||
|
generated_unix_seconds: unix_seconds_now(),
|
||||||
|
resource_count: resources.len(),
|
||||||
|
resources,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Generates a change set for two complete release roots and writes both the
|
||||||
|
/// change set and the Crowdin handoff under the current release root.
|
||||||
|
pub fn write_official_resource_change_handoff(
|
||||||
|
previous_resource_root: Option<&Path>,
|
||||||
|
current_resource_root: &Path,
|
||||||
|
official_release_id: &str,
|
||||||
|
previous_release_id: Option<String>,
|
||||||
|
) -> Result<OfficialResourceChangeHandoffReport, String> {
|
||||||
|
let current_manifest = read_download_manifest_at(current_resource_root)?.ok_or_else(|| {
|
||||||
|
format!(
|
||||||
|
"缺少当前官方下载 manifest,无法生成资源变更集:{}",
|
||||||
|
current_resource_root.display()
|
||||||
|
)
|
||||||
|
})?;
|
||||||
|
let previous_manifest = match previous_resource_root {
|
||||||
|
Some(root) => read_download_manifest_at(root)?,
|
||||||
|
None => None,
|
||||||
|
};
|
||||||
|
let previous_manifest_root = previous_manifest
|
||||||
|
.as_ref()
|
||||||
|
.and(previous_resource_root)
|
||||||
|
.map(Path::to_path_buf);
|
||||||
|
let previous_release_id = previous_manifest.as_ref().and(previous_release_id);
|
||||||
|
let change_set = OfficialResourceChangeSet::from_manifests(
|
||||||
|
official_release_id,
|
||||||
|
previous_release_id,
|
||||||
|
previous_manifest_root,
|
||||||
|
current_resource_root.to_path_buf(),
|
||||||
|
previous_manifest.as_ref(),
|
||||||
|
¤t_manifest,
|
||||||
|
);
|
||||||
|
write_resource_change_set_at(current_resource_root, &change_set)?;
|
||||||
|
|
||||||
|
let handoff = CrowdinTranslationHandoff::from_change_set(&change_set);
|
||||||
|
write_crowdin_translation_handoff_at(current_resource_root, &handoff)?;
|
||||||
|
|
||||||
|
Ok(OfficialResourceChangeHandoffReport {
|
||||||
|
change_set_path: current_resource_root.join(OFFICIAL_RESOURCE_CHANGES_FILE),
|
||||||
|
crowdin_handoff_path: current_resource_root.join(CROWDIN_TRANSLATION_HANDOFF_FILE),
|
||||||
|
summary: change_set.summary,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Paths and summary produced after writing a resource-change handoff.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct OfficialResourceChangeHandoffReport {
|
||||||
|
/// Path to `official-resource-changes.json`.
|
||||||
|
pub change_set_path: PathBuf,
|
||||||
|
/// Path to `crowdin-translation-handoff.json`.
|
||||||
|
pub crowdin_handoff_path: PathBuf,
|
||||||
|
/// Aggregate change counters.
|
||||||
|
pub summary: OfficialResourceChangeSummary,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Reads a generated resource change set from a release root.
|
||||||
|
pub fn read_resource_change_set_at(
|
||||||
|
resource_root: &Path,
|
||||||
|
) -> Result<Option<OfficialResourceChangeSet>, String> {
|
||||||
|
let path = resource_root.join(OFFICIAL_RESOURCE_CHANGES_FILE);
|
||||||
|
let Some(bytes) = read_file_no_symlink(&path, "官方资源变更集")? else {
|
||||||
|
return Ok(None);
|
||||||
|
};
|
||||||
|
let change_set: OfficialResourceChangeSet = serde_json::from_slice(&bytes)
|
||||||
|
.map_err(|error| format!("解析官方资源变更集失败 {}:{error}", path.display()))?;
|
||||||
|
if change_set.change_set_version != OFFICIAL_RESOURCE_CHANGES_VERSION {
|
||||||
|
return Err(format!(
|
||||||
|
"不支持的官方资源变更集版本 {},文件 {}",
|
||||||
|
change_set.change_set_version,
|
||||||
|
path.display()
|
||||||
|
));
|
||||||
|
}
|
||||||
|
Ok(Some(change_set))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Writes a generated resource change set under a release root.
|
||||||
|
pub fn write_resource_change_set_at(
|
||||||
|
resource_root: &Path,
|
||||||
|
change_set: &OfficialResourceChangeSet,
|
||||||
|
) -> Result<(), String> {
|
||||||
|
let path = resource_root.join(OFFICIAL_RESOURCE_CHANGES_FILE);
|
||||||
|
ensure_path_within_root(resource_root, &path)?;
|
||||||
|
ensure_safe_file_target(resource_root, &path, "官方资源变更集")?;
|
||||||
|
let bytes = serde_json::to_vec_pretty(change_set)
|
||||||
|
.map_err(|error| format!("序列化官方资源变更集失败:{error}"))?;
|
||||||
|
write_file_atomic(&path, &bytes, STATE_FILE_MODE, "官方资源变更集")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Writes a generated Crowdin handoff under a release root.
|
||||||
|
pub fn write_crowdin_translation_handoff_at(
|
||||||
|
resource_root: &Path,
|
||||||
|
handoff: &CrowdinTranslationHandoff,
|
||||||
|
) -> Result<(), String> {
|
||||||
|
let path = resource_root.join(CROWDIN_TRANSLATION_HANDOFF_FILE);
|
||||||
|
ensure_path_within_root(resource_root, &path)?;
|
||||||
|
ensure_safe_file_target(resource_root, &path, "Crowdin 翻译 handoff")?;
|
||||||
|
let bytes = serde_json::to_vec_pretty(handoff)
|
||||||
|
.map_err(|error| format!("序列化 Crowdin 翻译 handoff 失败:{error}"))?;
|
||||||
|
write_file_atomic(&path, &bytes, STATE_FILE_MODE, "Crowdin 翻译 handoff")
|
||||||
|
}
|
||||||
|
|
||||||
|
fn entries_by_destination(
|
||||||
|
manifest: &OfficialDownloadManifest,
|
||||||
|
) -> BTreeMap<String, &OfficialDownloadManifestEntry> {
|
||||||
|
manifest
|
||||||
|
.entries
|
||||||
|
.values()
|
||||||
|
.map(|entry| (entry.destination.clone(), entry))
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn content_changed(
|
||||||
|
previous: &OfficialDownloadManifestEntry,
|
||||||
|
current: &OfficialDownloadManifestEntry,
|
||||||
|
) -> bool {
|
||||||
|
previous.bytes != current.bytes || previous.blake3 != current.blake3
|
||||||
|
}
|
||||||
|
|
||||||
|
fn unix_seconds_now() -> u64 {
|
||||||
|
SystemTime::now()
|
||||||
|
.duration_since(UNIX_EPOCH)
|
||||||
|
.unwrap_or_default()
|
||||||
|
.as_secs()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn default_resource_changes_version() -> u32 {
|
||||||
|
OFFICIAL_RESOURCE_CHANGES_VERSION
|
||||||
|
}
|
||||||
|
|
||||||
|
fn default_crowdin_handoff_version() -> u32 {
|
||||||
|
CROWDIN_TRANSLATION_HANDOFF_VERSION
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
fn manifest(entries: &[(&str, &str, &[u8])]) -> OfficialDownloadManifest {
|
||||||
|
let mut manifest = OfficialDownloadManifest::default();
|
||||||
|
for (url, destination, bytes) in entries {
|
||||||
|
manifest.entries.insert(
|
||||||
|
(*url).to_string(),
|
||||||
|
OfficialDownloadManifestEntry {
|
||||||
|
url: (*url).to_string(),
|
||||||
|
destination: (*destination).to_string(),
|
||||||
|
bytes: bytes.len() as u64,
|
||||||
|
blake3: blake3::hash(bytes).to_hex().to_string(),
|
||||||
|
},
|
||||||
|
);
|
||||||
|
}
|
||||||
|
manifest
|
||||||
|
}
|
||||||
|
|
||||||
|
fn write_manifest(root: &Path, manifest: &OfficialDownloadManifest) {
|
||||||
|
std::fs::create_dir_all(root).unwrap();
|
||||||
|
std::fs::write(
|
||||||
|
root.join("official-download-manifest.json"),
|
||||||
|
serde_json::to_vec(manifest).unwrap(),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn change_set_classifies_added_modified_and_removed_resources() {
|
||||||
|
let previous = manifest(&[
|
||||||
|
("https://old/a", "TableBundles/a.bytes", b"old-a"),
|
||||||
|
("https://old/b", "TableBundles/b.bytes", b"same"),
|
||||||
|
("https://old/c", "TableBundles/c.bytes", b"removed"),
|
||||||
|
(
|
||||||
|
"https://old/u",
|
||||||
|
"TableBundles/url-only.bytes",
|
||||||
|
b"same-url-only",
|
||||||
|
),
|
||||||
|
]);
|
||||||
|
let current = manifest(&[
|
||||||
|
("https://new/a", "TableBundles/a.bytes", b"new-a"),
|
||||||
|
("https://new/b", "TableBundles/b.bytes", b"same"),
|
||||||
|
("https://new/d", "TableBundles/d.bytes", b"added"),
|
||||||
|
(
|
||||||
|
"https://changed-host/u",
|
||||||
|
"TableBundles/url-only.bytes",
|
||||||
|
b"same-url-only",
|
||||||
|
),
|
||||||
|
]);
|
||||||
|
|
||||||
|
let change_set = OfficialResourceChangeSet::from_manifests(
|
||||||
|
"release-new",
|
||||||
|
Some("release-old".to_string()),
|
||||||
|
Some(PathBuf::from("/previous")),
|
||||||
|
PathBuf::from("/current"),
|
||||||
|
Some(&previous),
|
||||||
|
¤t,
|
||||||
|
);
|
||||||
|
|
||||||
|
assert_eq!(change_set.summary.added_count, 1);
|
||||||
|
assert_eq!(change_set.summary.modified_count, 1);
|
||||||
|
assert_eq!(change_set.summary.removed_count, 1);
|
||||||
|
assert_eq!(change_set.summary.parse_candidate_count, 2);
|
||||||
|
assert_eq!(change_set.summary.translation_candidate_count, 2);
|
||||||
|
let destinations = change_set
|
||||||
|
.translation_candidates()
|
||||||
|
.into_iter()
|
||||||
|
.map(|change| change.destination.as_str())
|
||||||
|
.collect::<Vec<_>>();
|
||||||
|
assert_eq!(
|
||||||
|
destinations,
|
||||||
|
vec!["TableBundles/a.bytes", "TableBundles/d.bytes"]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn first_release_treats_all_current_resources_as_added() {
|
||||||
|
let current = manifest(&[
|
||||||
|
("https://new/a", "a.bundle", b"a"),
|
||||||
|
("https://new/b", "b.bundle", b"b"),
|
||||||
|
]);
|
||||||
|
|
||||||
|
let change_set = OfficialResourceChangeSet::from_manifests(
|
||||||
|
"release-new",
|
||||||
|
None,
|
||||||
|
None,
|
||||||
|
PathBuf::from("/current"),
|
||||||
|
None,
|
||||||
|
¤t,
|
||||||
|
);
|
||||||
|
|
||||||
|
assert!(!change_set.summary.previous_manifest_present);
|
||||||
|
assert_eq!(change_set.summary.added_count, 2);
|
||||||
|
assert_eq!(change_set.summary.translation_candidate_count, 2);
|
||||||
|
assert_eq!(change_set.changes.len(), 2);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn crowdin_handoff_excludes_removed_resources() {
|
||||||
|
let previous = manifest(&[("https://old/a", "a.bundle", b"a")]);
|
||||||
|
let current = manifest(&[("https://new/b", "b.bundle", b"b")]);
|
||||||
|
let change_set = OfficialResourceChangeSet::from_manifests(
|
||||||
|
"release-new",
|
||||||
|
Some("release-old".to_string()),
|
||||||
|
None,
|
||||||
|
PathBuf::from("/current"),
|
||||||
|
Some(&previous),
|
||||||
|
¤t,
|
||||||
|
);
|
||||||
|
|
||||||
|
let handoff = CrowdinTranslationHandoff::from_change_set(&change_set);
|
||||||
|
|
||||||
|
assert_eq!(handoff.provider, TranslationHandoffProvider::Crowdin);
|
||||||
|
assert_eq!(handoff.status, TranslationHandoffStatus::QueuedOffline);
|
||||||
|
assert_eq!(handoff.resource_count, 1);
|
||||||
|
assert_eq!(handoff.resources[0].destination, "b.bundle");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn write_handoff_persists_change_set_and_crowdin_queue() {
|
||||||
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
let previous_root = temp.path().join("previous");
|
||||||
|
let current_root = temp.path().join("current");
|
||||||
|
write_manifest(
|
||||||
|
&previous_root,
|
||||||
|
&manifest(&[("https://old/a", "a.bundle", b"old")]),
|
||||||
|
);
|
||||||
|
write_manifest(
|
||||||
|
¤t_root,
|
||||||
|
&manifest(&[
|
||||||
|
("https://new/a", "a.bundle", b"new"),
|
||||||
|
("https://new/b", "b.bundle", b"added"),
|
||||||
|
]),
|
||||||
|
);
|
||||||
|
|
||||||
|
let report = write_official_resource_change_handoff(
|
||||||
|
Some(&previous_root),
|
||||||
|
¤t_root,
|
||||||
|
"release-new",
|
||||||
|
Some("release-old".to_string()),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
assert_eq!(report.summary.modified_count, 1);
|
||||||
|
assert_eq!(report.summary.added_count, 1);
|
||||||
|
assert!(report.change_set_path.exists());
|
||||||
|
assert!(report.crowdin_handoff_path.exists());
|
||||||
|
let change_set = read_resource_change_set_at(¤t_root).unwrap().unwrap();
|
||||||
|
assert_eq!(change_set.summary.translation_candidate_count, 2);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -115,8 +115,8 @@ pub struct OfficialResourcePullProgress {
|
|||||||
///
|
///
|
||||||
/// `Started` events report the currently completed count before the URL
|
/// `Started` events report the currently completed count before the URL
|
||||||
/// finishes; `Finished` events report the count after completion. This is
|
/// finishes; `Finished` events report the count after completion. This is
|
||||||
/// intentionally not the URL's plan position, because concurrent downloads
|
/// intentionally not the URL's plan position, so status percentages stay
|
||||||
/// finish out of plan order and status percentages must not move backward.
|
/// monotonic even if execution order or skip/resume mix changes.
|
||||||
pub index: usize,
|
pub index: usize,
|
||||||
/// Total URL count in the pull plan.
|
/// Total URL count in the pull plan.
|
||||||
pub total: usize,
|
pub total: usize,
|
||||||
@@ -2272,14 +2272,16 @@ exit 22
|
|||||||
}
|
}
|
||||||
|
|
||||||
fn write_shell_script(path: &Path, script: &str) {
|
fn write_shell_script(path: &Path, script: &str) {
|
||||||
fs::write(path, script).unwrap();
|
let temp_path = path.with_extension("tmp");
|
||||||
|
fs::write(&temp_path, script).unwrap();
|
||||||
#[cfg(unix)]
|
#[cfg(unix)]
|
||||||
{
|
{
|
||||||
use std::os::unix::fs::PermissionsExt;
|
use std::os::unix::fs::PermissionsExt;
|
||||||
let mut permissions = fs::metadata(path).unwrap().permissions();
|
let mut permissions = fs::metadata(&temp_path).unwrap().permissions();
|
||||||
permissions.set_mode(0o755);
|
permissions.set_mode(0o755);
|
||||||
fs::set_permissions(path, permissions).unwrap();
|
fs::set_permissions(&temp_path, permissions).unwrap();
|
||||||
}
|
}
|
||||||
|
fs::rename(temp_path, path).unwrap();
|
||||||
}
|
}
|
||||||
|
|
||||||
fn write_fake_curl(path: &Path) {
|
fn write_fake_curl(path: &Path) {
|
||||||
|
|||||||
@@ -9,7 +9,8 @@ use crate::official_download::DownloadError;
|
|||||||
use crate::official_launcher::launcher_package_url;
|
use crate::official_launcher::launcher_package_url;
|
||||||
use crate::official_launcher::OfficialLauncherBootstrapService;
|
use crate::official_launcher::OfficialLauncherBootstrapService;
|
||||||
use crate::official_launcher::{
|
use crate::official_launcher::{
|
||||||
YostarJpLauncherCdnConfig, YostarJpLauncherGameConfig, YostarJpLauncherRemoteManifest,
|
YostarJpLauncherCdnConfig, YostarJpLauncherGameConfig, YostarJpLauncherManifestUrl,
|
||||||
|
YostarJpLauncherRemoteManifest,
|
||||||
};
|
};
|
||||||
use crate::zip_validation::validate_zip_structure;
|
use crate::zip_validation::validate_zip_structure;
|
||||||
use bat_adapters::official::game_main_config::YostarJpGameMainConfig;
|
use bat_adapters::official::game_main_config::YostarJpGameMainConfig;
|
||||||
@@ -39,12 +40,44 @@ pub struct OfficialGameMainConfigBootstrap {
|
|||||||
/// may instead point to a directory source plus per-file entries; in that
|
/// may instead point to a directory source plus per-file entries; in that
|
||||||
/// case this is the direct `resources.assets` URL.
|
/// case this is the direct `resources.assets` URL.
|
||||||
pub game_zip_url: String,
|
pub game_zip_url: String,
|
||||||
|
/// Remote launcher manifest used for this bootstrap.
|
||||||
|
pub remote_manifest: YostarJpLauncherRemoteManifest,
|
||||||
|
/// Exact source selected to obtain `GameMainConfig`.
|
||||||
|
pub selected_source: OfficialGameMainConfigSelectedSource,
|
||||||
/// Number of files declared by the remote manifest.
|
/// Number of files declared by the remote manifest.
|
||||||
pub manifest_file_count: usize,
|
pub manifest_file_count: usize,
|
||||||
/// Decrypted `GameMainConfig`.
|
/// Decrypted `GameMainConfig`.
|
||||||
pub game_main_config: YostarJpGameMainConfig,
|
pub game_main_config: YostarJpGameMainConfig,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Kind of official launcher package artifact selected for `GameMainConfig`.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
|
pub enum OfficialGameMainConfigSourceKind {
|
||||||
|
/// Older launcher manifests point to a single game ZIP archive.
|
||||||
|
Archive,
|
||||||
|
/// Current launcher manifests expose a directory plus per-file entries.
|
||||||
|
ManifestFile,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Exact launcher artifact selected to obtain `resources.assets`.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub struct OfficialGameMainConfigSelectedSource {
|
||||||
|
/// Source kind.
|
||||||
|
pub kind: OfficialGameMainConfigSourceKind,
|
||||||
|
/// Official URL fetched for this source.
|
||||||
|
pub url: String,
|
||||||
|
/// Relative path under the official launcher package CDN root.
|
||||||
|
pub relative_path: String,
|
||||||
|
/// Original manifest file path when the source is a manifest file entry.
|
||||||
|
pub manifest_path: Option<String>,
|
||||||
|
/// Declared file size from the manifest, when available.
|
||||||
|
pub declared_size: Option<u64>,
|
||||||
|
/// Official launcher manifest `hash` field, when available.
|
||||||
|
pub official_hash: Option<String>,
|
||||||
|
/// Official launcher manifest per-file `vc`, when available.
|
||||||
|
pub vc: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
/// Loads and decrypts the official `GameMainConfig` by following the official
|
/// Loads and decrypts the official `GameMainConfig` by following the official
|
||||||
/// launcher package chain.
|
/// launcher package chain.
|
||||||
#[derive(Debug, Clone)]
|
#[derive(Debug, Clone)]
|
||||||
@@ -92,6 +125,18 @@ impl OfficialGameMainConfigBootstrapService {
|
|||||||
/// launcher API, extracts `resources.assets`, and decrypts `GameMainConfig`.
|
/// launcher API, extracts `resources.assets`, and decrypts `GameMainConfig`.
|
||||||
pub fn fetch_bootstrap(&self) -> Result<OfficialGameMainConfigBootstrap, DownloadError> {
|
pub fn fetch_bootstrap(&self) -> Result<OfficialGameMainConfigBootstrap, DownloadError> {
|
||||||
let (game_config, manifest_url, manifest) = self.launcher.fetch_latest_remote_manifest()?;
|
let (game_config, manifest_url, manifest) = self.launcher.fetch_latest_remote_manifest()?;
|
||||||
|
let cdn_config = self.launcher.fetch_cdn_config()?;
|
||||||
|
self.fetch_bootstrap_from_parts(game_config, cdn_config, manifest_url, manifest)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Extracts `GameMainConfig` from already fetched launcher metadata.
|
||||||
|
pub fn fetch_bootstrap_from_parts(
|
||||||
|
&self,
|
||||||
|
game_config: YostarJpLauncherGameConfig,
|
||||||
|
cdn_config: YostarJpLauncherCdnConfig,
|
||||||
|
manifest_url: YostarJpLauncherManifestUrl,
|
||||||
|
manifest: YostarJpLauncherRemoteManifest,
|
||||||
|
) -> Result<OfficialGameMainConfigBootstrap, DownloadError> {
|
||||||
let manifest_source = manifest
|
let manifest_source = manifest
|
||||||
.source
|
.source
|
||||||
.clone()
|
.clone()
|
||||||
@@ -102,9 +147,8 @@ impl OfficialGameMainConfigBootstrapService {
|
|||||||
"官方启动器远端 manifest 缺少 source",
|
"官方启动器远端 manifest 缺少 source",
|
||||||
)
|
)
|
||||||
})?;
|
})?;
|
||||||
let cdn_config = self.launcher.fetch_cdn_config()?;
|
|
||||||
let temp_dir = TempDir::new().map_err(|error| format!("创建临时目录失败:{error}"))?;
|
let temp_dir = TempDir::new().map_err(|error| format!("创建临时目录失败:{error}"))?;
|
||||||
let (game_zip_url, resources_assets) = self.fetch_resources_assets(
|
let (game_zip_url, resources_assets, selected_source) = self.fetch_resources_assets(
|
||||||
&game_config,
|
&game_config,
|
||||||
&manifest,
|
&manifest,
|
||||||
&manifest_source,
|
&manifest_source,
|
||||||
@@ -113,6 +157,7 @@ impl OfficialGameMainConfigBootstrapService {
|
|||||||
)?;
|
)?;
|
||||||
let game_main_config = YostarJpGameMainConfig::from_resources_assets(resources_assets)
|
let game_main_config = YostarJpGameMainConfig::from_resources_assets(resources_assets)
|
||||||
.map_err(|error| DownloadError::new(ErrorCode::GAME_MAIN_CONFIG_FAILED, error))?;
|
.map_err(|error| DownloadError::new(ErrorCode::GAME_MAIN_CONFIG_FAILED, error))?;
|
||||||
|
let manifest_file_count = manifest.files.len();
|
||||||
|
|
||||||
Ok(OfficialGameMainConfigBootstrap {
|
Ok(OfficialGameMainConfigBootstrap {
|
||||||
game_config,
|
game_config,
|
||||||
@@ -120,7 +165,9 @@ impl OfficialGameMainConfigBootstrapService {
|
|||||||
manifest_url: manifest_url.url,
|
manifest_url: manifest_url.url,
|
||||||
manifest_source: Some(manifest_source),
|
manifest_source: Some(manifest_source),
|
||||||
game_zip_url,
|
game_zip_url,
|
||||||
manifest_file_count: manifest.files.len(),
|
remote_manifest: manifest,
|
||||||
|
selected_source,
|
||||||
|
manifest_file_count,
|
||||||
game_main_config,
|
game_main_config,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -203,10 +250,12 @@ impl OfficialGameMainConfigBootstrapService {
|
|||||||
manifest_source: &str,
|
manifest_source: &str,
|
||||||
cdn_config: &YostarJpLauncherCdnConfig,
|
cdn_config: &YostarJpLauncherCdnConfig,
|
||||||
temp_root: &Path,
|
temp_root: &Path,
|
||||||
) -> Result<(String, PathBuf), DownloadError> {
|
) -> Result<(String, PathBuf, OfficialGameMainConfigSelectedSource), DownloadError> {
|
||||||
|
let selected_source =
|
||||||
|
resolve_game_main_config_source(game_config, manifest, manifest_source, cdn_config)?;
|
||||||
match select_game_main_config_source(game_config, manifest_source, manifest)? {
|
match select_game_main_config_source(game_config, manifest_source, manifest)? {
|
||||||
GameMainConfigSource::Archive(package_path) => {
|
GameMainConfigSource::Archive(package_path) => {
|
||||||
let game_zip_url = launcher_package_url(&cdn_config.primary_cdn, package_path)?;
|
let game_zip_url = selected_source.url.clone();
|
||||||
let archive_path = temp_root.join("official-game.zip");
|
let archive_path = temp_root.join("official-game.zip");
|
||||||
self.download_file_with_fallback(
|
self.download_file_with_fallback(
|
||||||
&game_zip_url,
|
&game_zip_url,
|
||||||
@@ -227,12 +276,11 @@ impl OfficialGameMainConfigBootstrapService {
|
|||||||
"官方启动器包内没有找到 resources.assets",
|
"官方启动器包内没有找到 resources.assets",
|
||||||
)
|
)
|
||||||
})?;
|
})?;
|
||||||
Ok((game_zip_url, resources_assets))
|
Ok((game_zip_url, resources_assets, selected_source))
|
||||||
}
|
}
|
||||||
GameMainConfigSource::ManifestFile { source_dir, file } => {
|
GameMainConfigSource::ManifestFile { source_dir, file } => {
|
||||||
let relative_path = launcher_manifest_file_relative_path(source_dir, &file.path)?;
|
let relative_path = launcher_manifest_file_relative_path(source_dir, &file.path)?;
|
||||||
let resources_assets_url =
|
let resources_assets_url = selected_source.url.clone();
|
||||||
launcher_package_url(&cdn_config.primary_cdn, &relative_path)?;
|
|
||||||
let resources_assets = temp_root.join("resources.assets");
|
let resources_assets = temp_root.join("resources.assets");
|
||||||
self.download_file_with_fallback(
|
self.download_file_with_fallback(
|
||||||
&resources_assets_url,
|
&resources_assets_url,
|
||||||
@@ -241,7 +289,7 @@ impl OfficialGameMainConfigBootstrapService {
|
|||||||
&resources_assets,
|
&resources_assets,
|
||||||
)?;
|
)?;
|
||||||
verify_manifest_file_size(&resources_assets, file)?;
|
verify_manifest_file_size(&resources_assets, file)?;
|
||||||
Ok((resources_assets_url, resources_assets))
|
Ok((resources_assets_url, resources_assets, selected_source))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -283,6 +331,49 @@ impl OfficialGameMainConfigBootstrapService {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Resolves the exact official launcher artifact used to obtain `GameMainConfig`.
|
||||||
|
pub fn resolve_game_main_config_source(
|
||||||
|
game_config: &YostarJpLauncherGameConfig,
|
||||||
|
manifest: &YostarJpLauncherRemoteManifest,
|
||||||
|
manifest_source: &str,
|
||||||
|
cdn_config: &YostarJpLauncherCdnConfig,
|
||||||
|
) -> Result<OfficialGameMainConfigSelectedSource, DownloadError> {
|
||||||
|
match select_game_main_config_source(game_config, manifest_source, manifest)? {
|
||||||
|
GameMainConfigSource::Archive(package_path) => {
|
||||||
|
let url = launcher_package_url(&cdn_config.primary_cdn, package_path)?;
|
||||||
|
Ok(OfficialGameMainConfigSelectedSource {
|
||||||
|
kind: OfficialGameMainConfigSourceKind::Archive,
|
||||||
|
url,
|
||||||
|
relative_path: package_path.to_string(),
|
||||||
|
manifest_path: None,
|
||||||
|
declared_size: None,
|
||||||
|
official_hash: None,
|
||||||
|
vc: None,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
GameMainConfigSource::ManifestFile { source_dir, file } => {
|
||||||
|
let relative_path = launcher_manifest_file_relative_path(source_dir, &file.path)
|
||||||
|
.map_err(|error| DownloadError::new(ErrorCode::LAUNCHER_RESPONSE_INVALID, error))?;
|
||||||
|
let url = launcher_package_url(&cdn_config.primary_cdn, &relative_path)?;
|
||||||
|
let declared_size = file.size.parse::<u64>().map_err(|error| {
|
||||||
|
DownloadError::new(
|
||||||
|
ErrorCode::LAUNCHER_RESPONSE_INVALID,
|
||||||
|
format!("官方启动器 manifest 中 {} 的 size 无效:{error}", file.path),
|
||||||
|
)
|
||||||
|
})?;
|
||||||
|
Ok(OfficialGameMainConfigSelectedSource {
|
||||||
|
kind: OfficialGameMainConfigSourceKind::ManifestFile,
|
||||||
|
url,
|
||||||
|
relative_path,
|
||||||
|
manifest_path: Some(file.path.clone()),
|
||||||
|
declared_size: Some(declared_size),
|
||||||
|
official_hash: Some(file.hash.clone()),
|
||||||
|
vc: file.vc.clone(),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
#[derive(Debug)]
|
#[derive(Debug)]
|
||||||
enum GameMainConfigSource<'a> {
|
enum GameMainConfigSource<'a> {
|
||||||
Archive(&'a str),
|
Archive(&'a str),
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,543 @@
|
|||||||
|
//! Import of a verified official release into CAS and ResourceRepository.
|
||||||
|
|
||||||
|
use crate::official_download::{read_download_manifest_at, OfficialDownloadManifestEntry};
|
||||||
|
use crate::official_parse::{
|
||||||
|
read_parse_cache_at, OfficialParseCache, OfficialParseCacheEntry, OfficialParseStatus,
|
||||||
|
OfficialParseSummary,
|
||||||
|
};
|
||||||
|
use crate::path_security::{
|
||||||
|
ensure_path_within_root, ensure_safe_file_target, read_file_no_symlink,
|
||||||
|
};
|
||||||
|
use bat_core::domain::{Resource, ResourceEntry, ResourceMetadata, ResourceType};
|
||||||
|
use bat_core::repositories::{CasRepository, ResourceRepository};
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use std::collections::{BTreeMap, BTreeSet};
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
|
||||||
|
/// Configuration for importing one already-published official release.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub struct OfficialReleaseImportConfig {
|
||||||
|
/// Published release root containing `official-download-manifest.json`.
|
||||||
|
pub release_root: PathBuf,
|
||||||
|
/// Official release ID associated with this root, when known.
|
||||||
|
pub official_release_id: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl OfficialReleaseImportConfig {
|
||||||
|
/// Creates an import configuration.
|
||||||
|
pub fn new(release_root: impl Into<PathBuf>) -> Self {
|
||||||
|
Self {
|
||||||
|
release_root: release_root.into(),
|
||||||
|
official_release_id: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Attaches the official release ID that should be stored in resource metadata.
|
||||||
|
pub fn with_official_release_id(mut self, release_id: impl Into<String>) -> Self {
|
||||||
|
self.official_release_id = Some(release_id.into());
|
||||||
|
self
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Summary returned by an official release repository import.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct OfficialReleaseImportReport {
|
||||||
|
/// Number of verified manifest entries.
|
||||||
|
pub manifest_entry_count: usize,
|
||||||
|
/// Number of new or changed repository rows.
|
||||||
|
pub imported_count: usize,
|
||||||
|
/// Number of rows already pointing at the same CAS object.
|
||||||
|
pub unchanged_count: usize,
|
||||||
|
/// Number of unchanged rows whose metadata was refreshed from parse cache.
|
||||||
|
#[serde(default)]
|
||||||
|
pub metadata_updated_count: usize,
|
||||||
|
/// Number of resources classified as AssetBundle.
|
||||||
|
pub asset_bundle_count: usize,
|
||||||
|
/// Number of resources classified as text-like payloads.
|
||||||
|
pub text_asset_count: usize,
|
||||||
|
/// Number of resources classified as tables.
|
||||||
|
pub table_count: usize,
|
||||||
|
/// Number of resources classified as media.
|
||||||
|
pub media_count: usize,
|
||||||
|
/// Parse-cache summary associated with this release, when available.
|
||||||
|
pub parse_summary: Option<OfficialParseSummary>,
|
||||||
|
/// Non-fatal cleanup warnings, such as an old CAS reference that could not
|
||||||
|
/// be decremented after a successful row replacement.
|
||||||
|
pub warnings: Vec<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Imports verified official resources into CAS and the resource index.
|
||||||
|
pub struct OfficialReleaseImportService<'a> {
|
||||||
|
cas: &'a dyn CasRepository,
|
||||||
|
resources: &'a dyn ResourceRepository,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<'a> OfficialReleaseImportService<'a> {
|
||||||
|
/// Creates an import service.
|
||||||
|
pub fn new(cas: &'a dyn CasRepository, resources: &'a dyn ResourceRepository) -> Self {
|
||||||
|
Self { cas, resources }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Imports every entry in one published release manifest.
|
||||||
|
///
|
||||||
|
/// The source files remain untouched. Every file is checked against the
|
||||||
|
/// verified download manifest before it can enter CAS. Repository IDs are
|
||||||
|
/// stable by destination, making repeated imports idempotent.
|
||||||
|
pub async fn import_release(
|
||||||
|
&self,
|
||||||
|
config: &OfficialReleaseImportConfig,
|
||||||
|
) -> bat_core::Result<OfficialReleaseImportReport> {
|
||||||
|
let manifest = read_download_manifest_at(&config.release_root)
|
||||||
|
.map_err(bat_core::Error::InvalidArgument)?
|
||||||
|
.ok_or_else(|| {
|
||||||
|
bat_core::Error::NotFound(format!(
|
||||||
|
"官方下载 manifest 不存在:{}",
|
||||||
|
config.release_root.display()
|
||||||
|
))
|
||||||
|
})?;
|
||||||
|
let parse_cache =
|
||||||
|
read_parse_cache_at(&config.release_root).map_err(bat_core::Error::InvalidArgument)?;
|
||||||
|
let parse_summary = parse_cache.as_ref().map(|cache| cache.summary.clone());
|
||||||
|
let parse_entries_by_destination = parse_cache
|
||||||
|
.as_ref()
|
||||||
|
.map(parse_entries_by_destination)
|
||||||
|
.unwrap_or_default();
|
||||||
|
let release_id = config
|
||||||
|
.official_release_id
|
||||||
|
.clone()
|
||||||
|
.or_else(|| release_id_from_root(&config.release_root));
|
||||||
|
let mut report = OfficialReleaseImportReport {
|
||||||
|
manifest_entry_count: manifest.entries.len(),
|
||||||
|
imported_count: 0,
|
||||||
|
unchanged_count: 0,
|
||||||
|
metadata_updated_count: 0,
|
||||||
|
asset_bundle_count: 0,
|
||||||
|
text_asset_count: 0,
|
||||||
|
table_count: 0,
|
||||||
|
media_count: 0,
|
||||||
|
parse_summary,
|
||||||
|
warnings: Vec::new(),
|
||||||
|
};
|
||||||
|
|
||||||
|
for entry in manifest.entries.values() {
|
||||||
|
let parse_entries = parse_entries_by_destination
|
||||||
|
.get(&entry.destination)
|
||||||
|
.map(Vec::as_slice)
|
||||||
|
.unwrap_or(&[]);
|
||||||
|
self.import_entry(
|
||||||
|
&config.release_root,
|
||||||
|
release_id.as_deref(),
|
||||||
|
entry,
|
||||||
|
parse_entries,
|
||||||
|
&mut report,
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
}
|
||||||
|
|
||||||
|
Ok(report)
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn import_entry(
|
||||||
|
&self,
|
||||||
|
release_root: &Path,
|
||||||
|
official_release_id: Option<&str>,
|
||||||
|
manifest_entry: &OfficialDownloadManifestEntry,
|
||||||
|
parse_entries: &[&OfficialParseCacheEntry],
|
||||||
|
report: &mut OfficialReleaseImportReport,
|
||||||
|
) -> bat_core::Result<()> {
|
||||||
|
let path = release_root.join(Path::new(&manifest_entry.destination));
|
||||||
|
ensure_path_within_root(release_root, &path).map_err(bat_core::Error::InvalidArgument)?;
|
||||||
|
ensure_safe_file_target(release_root, &path, "官方资源导入输入")
|
||||||
|
.map_err(bat_core::Error::InvalidArgument)?;
|
||||||
|
let bytes = read_file_no_symlink(&path, "官方资源导入输入")
|
||||||
|
.map_err(bat_core::Error::InvalidArgument)?
|
||||||
|
.ok_or_else(|| bat_core::Error::NotFound(path.display().to_string()))?;
|
||||||
|
if bytes.len() as u64 != manifest_entry.bytes {
|
||||||
|
return Err(bat_core::Error::InvalidArgument(format!(
|
||||||
|
"官方资源导入 size 校验失败 {}:期望 {},实际 {}",
|
||||||
|
manifest_entry.destination,
|
||||||
|
manifest_entry.bytes,
|
||||||
|
bytes.len()
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
let actual_hash = blake3::hash(&bytes).to_hex().to_string();
|
||||||
|
if actual_hash != manifest_entry.blake3 {
|
||||||
|
return Err(bat_core::Error::InvalidArgument(format!(
|
||||||
|
"官方资源导入 BLAKE3 校验失败 {}:期望 {},实际 {}",
|
||||||
|
manifest_entry.destination, manifest_entry.blake3, actual_hash
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
|
||||||
|
let resource_type = resource_type_for_path(&manifest_entry.destination);
|
||||||
|
let metadata =
|
||||||
|
metadata_for_manifest_entry(official_release_id, manifest_entry, parse_entries);
|
||||||
|
let resource_id = resource_id_for_destination(&manifest_entry.destination);
|
||||||
|
let previous = match self.resources.find_by_id(&resource_id).await {
|
||||||
|
Ok(resource) => Some(resource),
|
||||||
|
Err(bat_core::Error::NotFound(_)) => None,
|
||||||
|
Err(error) => return Err(error),
|
||||||
|
};
|
||||||
|
if previous
|
||||||
|
.as_ref()
|
||||||
|
.is_some_and(|resource| resource.entry.hash == actual_hash)
|
||||||
|
{
|
||||||
|
if let Some(mut resource) = previous {
|
||||||
|
let should_refresh_metadata = resource.metadata != metadata
|
||||||
|
|| resource.entry.resource_type != resource_type
|
||||||
|
|| resource.entry.size != manifest_entry.bytes
|
||||||
|
|| resource.local_path.as_path() != Path::new(&manifest_entry.destination);
|
||||||
|
if should_refresh_metadata {
|
||||||
|
resource.local_path = PathBuf::from(&manifest_entry.destination);
|
||||||
|
resource.entry.size = manifest_entry.bytes;
|
||||||
|
resource.entry.resource_type = resource_type;
|
||||||
|
resource.metadata = metadata;
|
||||||
|
self.resources.update(resource).await?;
|
||||||
|
report.metadata_updated_count += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
report.unchanged_count += 1;
|
||||||
|
count_resource_type(report, resource_type);
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
|
||||||
|
let object_id = self.cas.store(&bytes).await?;
|
||||||
|
let resource = Resource {
|
||||||
|
id: resource_id,
|
||||||
|
local_path: PathBuf::from(&manifest_entry.destination),
|
||||||
|
entry: ResourceEntry {
|
||||||
|
path: manifest_entry.destination.clone(),
|
||||||
|
hash: object_id.clone(),
|
||||||
|
size: manifest_entry.bytes,
|
||||||
|
resource_type,
|
||||||
|
address: None,
|
||||||
|
dependencies: Vec::new(),
|
||||||
|
crc: None,
|
||||||
|
},
|
||||||
|
metadata,
|
||||||
|
};
|
||||||
|
if let Err(error) = self.resources.add(resource).await {
|
||||||
|
let _ = self.cas.remove_reference(&object_id).await;
|
||||||
|
return Err(error);
|
||||||
|
}
|
||||||
|
|
||||||
|
if let Some(previous) = previous {
|
||||||
|
if previous.entry.hash != object_id {
|
||||||
|
match self.cas.remove_reference(&previous.entry.hash).await {
|
||||||
|
Ok(_) => {}
|
||||||
|
Err(error) => report.warnings.push(format!(
|
||||||
|
"旧 CAS 引用清理失败 {}:{}",
|
||||||
|
previous.entry.hash, error
|
||||||
|
)),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
report.imported_count += 1;
|
||||||
|
count_resource_type(report, resource_type);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn parse_entries_by_destination(
|
||||||
|
cache: &OfficialParseCache,
|
||||||
|
) -> BTreeMap<String, Vec<&OfficialParseCacheEntry>> {
|
||||||
|
let mut by_destination: BTreeMap<String, Vec<&OfficialParseCacheEntry>> = BTreeMap::new();
|
||||||
|
for entry in cache.entries.values() {
|
||||||
|
by_destination
|
||||||
|
.entry(entry.destination.clone())
|
||||||
|
.or_default()
|
||||||
|
.push(entry);
|
||||||
|
}
|
||||||
|
by_destination
|
||||||
|
}
|
||||||
|
|
||||||
|
fn metadata_for_manifest_entry(
|
||||||
|
official_release_id: Option<&str>,
|
||||||
|
manifest_entry: &OfficialDownloadManifestEntry,
|
||||||
|
parse_entries: &[&OfficialParseCacheEntry],
|
||||||
|
) -> ResourceMetadata {
|
||||||
|
let mut metadata = ResourceMetadata {
|
||||||
|
official_release_id: official_release_id.map(ToOwned::to_owned),
|
||||||
|
platform: platform_for_destination(&manifest_entry.destination),
|
||||||
|
bundle_path: Some(manifest_entry.destination.clone()),
|
||||||
|
..ResourceMetadata::default()
|
||||||
|
};
|
||||||
|
|
||||||
|
let mut archive_entries = BTreeSet::new();
|
||||||
|
let mut parse_statuses = BTreeSet::new();
|
||||||
|
let mut unity_versions = BTreeSet::new();
|
||||||
|
let mut text_assets = BTreeSet::new();
|
||||||
|
let mut text_unit_formats = BTreeSet::new();
|
||||||
|
|
||||||
|
for entry in parse_entries {
|
||||||
|
if let Some(archive_entry) = entry.archive_entry.as_ref() {
|
||||||
|
archive_entries.insert(archive_entry.clone());
|
||||||
|
}
|
||||||
|
parse_statuses.insert(parse_status_label(entry.status).to_string());
|
||||||
|
if let Some(unity_version) = entry.unity_version.as_ref() {
|
||||||
|
unity_versions.insert(unity_version.clone());
|
||||||
|
}
|
||||||
|
metadata.unityfs_file_count += entry.file_count as u64;
|
||||||
|
metadata.serialized_file_count += entry.serialized_file_count as u64;
|
||||||
|
metadata.text_asset_count += entry.text_asset_count as u64;
|
||||||
|
metadata.text_unit_count += entry.text_unit_count as u64;
|
||||||
|
metadata.text_unit_error_count += entry.text_unit_error_count as u64;
|
||||||
|
text_assets.extend(entry.text_assets.iter().cloned());
|
||||||
|
text_unit_formats.extend(entry.text_unit_formats.iter().cloned());
|
||||||
|
}
|
||||||
|
|
||||||
|
metadata.archive_entries = archive_entries.into_iter().collect();
|
||||||
|
metadata.parse_statuses = parse_statuses.into_iter().collect();
|
||||||
|
metadata.unity_versions = unity_versions.into_iter().collect();
|
||||||
|
metadata.text_assets = text_assets.into_iter().collect();
|
||||||
|
metadata.text_unit_formats = text_unit_formats.into_iter().collect();
|
||||||
|
metadata
|
||||||
|
}
|
||||||
|
|
||||||
|
fn parse_status_label(status: OfficialParseStatus) -> &'static str {
|
||||||
|
match status {
|
||||||
|
OfficialParseStatus::Parsed => "parsed",
|
||||||
|
OfficialParseStatus::SkippedUnsupported => "skipped_unsupported",
|
||||||
|
OfficialParseStatus::Failed => "failed",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn platform_for_destination(destination: &str) -> Option<String> {
|
||||||
|
let normalized = destination.replace('\\', "/").to_ascii_lowercase();
|
||||||
|
if normalized.contains("windows") || normalized.contains("/win/") {
|
||||||
|
Some("windows".to_string())
|
||||||
|
} else if normalized.contains("android") {
|
||||||
|
Some("android".to_string())
|
||||||
|
} else {
|
||||||
|
None
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn release_id_from_root(release_root: &Path) -> Option<String> {
|
||||||
|
release_root
|
||||||
|
.file_name()
|
||||||
|
.and_then(|name| name.to_str())
|
||||||
|
.filter(|name| !name.is_empty() && *name != "current")
|
||||||
|
.map(ToOwned::to_owned)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn resource_id_for_destination(destination: &str) -> String {
|
||||||
|
format!("official/{}", destination.replace('\\', "/"))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn resource_type_for_path(path: &str) -> ResourceType {
|
||||||
|
let normalized = path.replace('\\', "/").to_ascii_lowercase();
|
||||||
|
if normalized.ends_with(".bundle") || normalized.ends_with(".unity3d") {
|
||||||
|
ResourceType::AssetBundle
|
||||||
|
} else if normalized.contains("tablebundles/") {
|
||||||
|
ResourceType::TableBundle
|
||||||
|
} else if normalized.contains("textassets/")
|
||||||
|
|| matches!(
|
||||||
|
normalized.rsplit('.').next(),
|
||||||
|
Some("txt" | "csv" | "json" | "xml" | "yaml" | "yml")
|
||||||
|
)
|
||||||
|
{
|
||||||
|
ResourceType::TextAsset
|
||||||
|
} else if normalized.contains("mediaresources/")
|
||||||
|
|| matches!(
|
||||||
|
normalized.rsplit('.').next(),
|
||||||
|
Some("acb" | "awb" | "jpg" | "jpeg" | "mp3" | "mp4" | "ogg" | "png" | "wav" | "webp")
|
||||||
|
)
|
||||||
|
{
|
||||||
|
ResourceType::Media
|
||||||
|
} else if normalized.contains("catalog") || normalized.ends_with(".hash") {
|
||||||
|
ResourceType::Manifest
|
||||||
|
} else {
|
||||||
|
ResourceType::Other
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn count_resource_type(report: &mut OfficialReleaseImportReport, resource_type: ResourceType) {
|
||||||
|
match resource_type {
|
||||||
|
ResourceType::AssetBundle => report.asset_bundle_count += 1,
|
||||||
|
ResourceType::TextAsset => report.text_asset_count += 1,
|
||||||
|
ResourceType::TableBundle => report.table_count += 1,
|
||||||
|
ResourceType::Media => report.media_count += 1,
|
||||||
|
ResourceType::Manifest | ResourceType::Other => {}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use crate::{FileSystemCasRepository, InMemoryResourceRepository};
|
||||||
|
use crate::{
|
||||||
|
OfficialParseSourceFingerprint, OfficialParseSourceKind, OFFICIAL_PARSE_CACHE_VERSION,
|
||||||
|
};
|
||||||
|
use bat_core::repositories::resource_repository::{ResourceQuery, ResourceRepository};
|
||||||
|
use std::collections::BTreeMap;
|
||||||
|
use tempfile::TempDir;
|
||||||
|
|
||||||
|
fn write_manifest(root: &Path, destination: &str, bytes: &[u8]) {
|
||||||
|
let path = root.join(destination);
|
||||||
|
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
|
||||||
|
std::fs::write(&path, bytes).unwrap();
|
||||||
|
let url = format!("https://prod-clientpatch.bluearchiveyostar.com/r93/{destination}");
|
||||||
|
let entry = OfficialDownloadManifestEntry {
|
||||||
|
url: url.clone(),
|
||||||
|
destination: destination.to_string(),
|
||||||
|
bytes: bytes.len() as u64,
|
||||||
|
blake3: blake3::hash(bytes).to_hex().to_string(),
|
||||||
|
};
|
||||||
|
let manifest = serde_json::json!({
|
||||||
|
"version": 1,
|
||||||
|
"entries": BTreeMap::from([(url, entry)]),
|
||||||
|
});
|
||||||
|
std::fs::write(
|
||||||
|
root.join("official-download-manifest.json"),
|
||||||
|
serde_json::to_vec(&manifest).unwrap(),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
fn write_parse_cache(root: &Path, destination: &str, bytes: &[u8]) {
|
||||||
|
let source_url =
|
||||||
|
format!("https://prod-clientpatch.bluearchiveyostar.com/r93/{destination}");
|
||||||
|
let entry = OfficialParseCacheEntry {
|
||||||
|
key: format!("direct:{source_url}"),
|
||||||
|
source_url: source_url.clone(),
|
||||||
|
destination: destination.to_string(),
|
||||||
|
archive_entry: None,
|
||||||
|
source_kind: OfficialParseSourceKind::DirectBundle,
|
||||||
|
fingerprint: OfficialParseSourceFingerprint {
|
||||||
|
source_url,
|
||||||
|
destination: destination.to_string(),
|
||||||
|
bytes: bytes.len() as u64,
|
||||||
|
blake3: blake3::hash(bytes).to_hex().to_string(),
|
||||||
|
},
|
||||||
|
status: OfficialParseStatus::Parsed,
|
||||||
|
reused_from_previous_cache: false,
|
||||||
|
unity_version: Some("2021.3.56f2".to_string()),
|
||||||
|
file_count: 1,
|
||||||
|
serialized_file_count: 1,
|
||||||
|
text_asset_count: 1,
|
||||||
|
text_assets: vec!["Scenario".to_string()],
|
||||||
|
serialized_parse_error_count: 0,
|
||||||
|
text_unit_count: 2,
|
||||||
|
text_unit_formats: vec!["json".to_string(), "plain".to_string()],
|
||||||
|
skipped_binary_text_asset_count: 0,
|
||||||
|
text_unit_error_count: 1,
|
||||||
|
error: None,
|
||||||
|
};
|
||||||
|
let cache = OfficialParseCache {
|
||||||
|
version: OFFICIAL_PARSE_CACHE_VERSION,
|
||||||
|
generated_unix_seconds: 123,
|
||||||
|
summary: OfficialParseSummary {
|
||||||
|
manifest_entry_count: 1,
|
||||||
|
cache_entry_count: 1,
|
||||||
|
candidate_file_count: 1,
|
||||||
|
zip_entry_count: 0,
|
||||||
|
skipped_unchanged_count: 0,
|
||||||
|
parsed_bundle_count: 1,
|
||||||
|
unsupported_count: 0,
|
||||||
|
failed_count: 0,
|
||||||
|
text_asset_count: 1,
|
||||||
|
text_unit_count: 2,
|
||||||
|
skipped_binary_text_asset_count: 0,
|
||||||
|
text_unit_error_count: 1,
|
||||||
|
},
|
||||||
|
entries: BTreeMap::from([(entry.key.clone(), entry)]),
|
||||||
|
};
|
||||||
|
std::fs::write(
|
||||||
|
root.join("official-parse-cache.json"),
|
||||||
|
serde_json::to_vec(&cache).unwrap(),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn imports_verified_release_idempotently() {
|
||||||
|
let temp = TempDir::new().unwrap();
|
||||||
|
write_manifest(temp.path(), "TableBundles/TableCatalog.bytes", b"catalog");
|
||||||
|
let cas = FileSystemCasRepository::new(temp.path().join("cas"));
|
||||||
|
let resources = InMemoryResourceRepository::new();
|
||||||
|
let service = OfficialReleaseImportService::new(&cas, &resources);
|
||||||
|
let config = OfficialReleaseImportConfig::new(temp.path());
|
||||||
|
|
||||||
|
let first = service.import_release(&config).await.unwrap();
|
||||||
|
let second = service.import_release(&config).await.unwrap();
|
||||||
|
|
||||||
|
assert_eq!(first.imported_count, 1);
|
||||||
|
assert_eq!(second.imported_count, 0);
|
||||||
|
assert_eq!(second.unchanged_count, 1);
|
||||||
|
assert_eq!(
|
||||||
|
resources
|
||||||
|
.count(ResourceQuery::by_type(ResourceType::TableBundle))
|
||||||
|
.await
|
||||||
|
.unwrap(),
|
||||||
|
1
|
||||||
|
);
|
||||||
|
let id = resource_id_for_destination("TableBundles/TableCatalog.bytes");
|
||||||
|
let resource = resources.find_by_id(&id).await.unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
cas.get_reference_count(&resource.entry.hash).await.unwrap(),
|
||||||
|
1
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn imports_parse_cache_metadata_into_resource_index() {
|
||||||
|
let temp = TempDir::new().unwrap();
|
||||||
|
let destination = "Windows/Bundles/scenario.bundle";
|
||||||
|
let bytes = b"bundle-bytes";
|
||||||
|
write_manifest(temp.path(), destination, bytes);
|
||||||
|
write_parse_cache(temp.path(), destination, bytes);
|
||||||
|
let cas = FileSystemCasRepository::new(temp.path().join("cas"));
|
||||||
|
let resources = InMemoryResourceRepository::new();
|
||||||
|
let service = OfficialReleaseImportService::new(&cas, &resources);
|
||||||
|
|
||||||
|
let report = service
|
||||||
|
.import_release(
|
||||||
|
&OfficialReleaseImportConfig::new(temp.path())
|
||||||
|
.with_official_release_id("release-current"),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
assert_eq!(report.imported_count, 1);
|
||||||
|
assert_eq!(report.parse_summary.as_ref().unwrap().text_unit_count, 2);
|
||||||
|
let id = resource_id_for_destination(destination);
|
||||||
|
let resource = resources.find_by_id(&id).await.unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
resource.metadata.official_release_id.as_deref(),
|
||||||
|
Some("release-current")
|
||||||
|
);
|
||||||
|
assert_eq!(resource.metadata.platform.as_deref(), Some("windows"));
|
||||||
|
assert_eq!(resource.metadata.bundle_path.as_deref(), Some(destination));
|
||||||
|
assert_eq!(resource.metadata.parse_statuses, vec!["parsed".to_string()]);
|
||||||
|
assert_eq!(
|
||||||
|
resource.metadata.unity_versions,
|
||||||
|
vec!["2021.3.56f2".to_string()]
|
||||||
|
);
|
||||||
|
assert_eq!(resource.metadata.text_assets, vec!["Scenario".to_string()]);
|
||||||
|
assert_eq!(resource.metadata.text_unit_count, 2);
|
||||||
|
assert_eq!(
|
||||||
|
resource.metadata.text_unit_formats,
|
||||||
|
vec!["json".to_string(), "plain".to_string()]
|
||||||
|
);
|
||||||
|
assert_eq!(resource.metadata.text_unit_error_count, 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn rejects_manifest_hash_mismatch_before_cas_write() {
|
||||||
|
let temp = TempDir::new().unwrap();
|
||||||
|
write_manifest(temp.path(), "TextAssets/dialogue.txt", b"original");
|
||||||
|
let path = temp.path().join("TextAssets/dialogue.txt");
|
||||||
|
std::fs::write(&path, b"tampered").unwrap();
|
||||||
|
let cas = FileSystemCasRepository::new(temp.path().join("cas"));
|
||||||
|
let resources = InMemoryResourceRepository::new();
|
||||||
|
let service = OfficialReleaseImportService::new(&cas, &resources);
|
||||||
|
|
||||||
|
let error = service
|
||||||
|
.import_release(&OfficialReleaseImportConfig::new(temp.path()))
|
||||||
|
.await
|
||||||
|
.unwrap_err();
|
||||||
|
|
||||||
|
assert!(error.to_string().contains("BLAKE3"));
|
||||||
|
assert_eq!(resources.count(ResourceQuery::all()).await.unwrap(), 0);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,834 @@
|
|||||||
|
//! Incremental TextUnit task and Crowdin offline queues for official releases.
|
||||||
|
//!
|
||||||
|
//! The queue is derived only from a verified official release, its
|
||||||
|
//! `official-resource-changes.json`, and its `official-parse-cache.json`.
|
||||||
|
//! It does not call Crowdin or any network service.
|
||||||
|
|
||||||
|
use crate::official_changes::{
|
||||||
|
read_resource_change_set_at, OfficialResourceChange, OfficialResourceChangeKind,
|
||||||
|
OfficialResourceChangeSet, TranslationHandoffProvider, TranslationHandoffStatus,
|
||||||
|
};
|
||||||
|
use crate::official_parse::{
|
||||||
|
read_parse_cache_at, OfficialParseCache, OfficialParseCacheEntry, OfficialParseSourceKind,
|
||||||
|
OfficialParseStatus,
|
||||||
|
};
|
||||||
|
use crate::path_security::{
|
||||||
|
ensure_path_within_root, ensure_safe_file_target, read_file_no_symlink, write_file_atomic,
|
||||||
|
STATE_FILE_MODE,
|
||||||
|
};
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use std::collections::BTreeMap;
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
use std::time::{SystemTime, UNIX_EPOCH};
|
||||||
|
|
||||||
|
/// Current TextUnit task-queue schema version.
|
||||||
|
pub const OFFICIAL_TEXTUNIT_TASK_QUEUE_VERSION: u32 = 1;
|
||||||
|
/// File name stored under a published official release root.
|
||||||
|
pub const OFFICIAL_TEXTUNIT_TASK_QUEUE_FILE: &str = "official-textunit-tasks.json";
|
||||||
|
/// Current Crowdin TextUnit queue schema version.
|
||||||
|
pub const CROWDIN_TEXTUNIT_QUEUE_VERSION: u32 = 1;
|
||||||
|
/// File name stored under a published official release root.
|
||||||
|
pub const CROWDIN_TEXTUNIT_QUEUE_FILE: &str = "crowdin-textunit-queue.json";
|
||||||
|
|
||||||
|
/// Processing status for one incremental TextUnit task candidate.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
#[serde(rename_all = "snake_case")]
|
||||||
|
pub enum OfficialTextUnitTaskStatus {
|
||||||
|
/// The task has parse metadata and at least one TextUnit.
|
||||||
|
QueuedOffline,
|
||||||
|
/// No parse-cache entry exists for this changed resource yet.
|
||||||
|
SkippedNoParseEntry,
|
||||||
|
/// The resource parsed successfully, but did not produce TextUnits.
|
||||||
|
SkippedNoTextUnit,
|
||||||
|
/// Parse cache says the candidate failed to parse.
|
||||||
|
SkippedParseFailed,
|
||||||
|
/// Parse cache says the candidate is unsupported.
|
||||||
|
SkippedUnsupported,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl OfficialTextUnitTaskStatus {
|
||||||
|
/// Returns the stable status label.
|
||||||
|
pub fn as_str(self) -> &'static str {
|
||||||
|
match self {
|
||||||
|
Self::QueuedOffline => "queued_offline",
|
||||||
|
Self::SkippedNoParseEntry => "skipped_no_parse_entry",
|
||||||
|
Self::SkippedNoTextUnit => "skipped_no_text_unit",
|
||||||
|
Self::SkippedParseFailed => "skipped_parse_failed",
|
||||||
|
Self::SkippedUnsupported => "skipped_unsupported",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One parse-cache-backed incremental task.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct OfficialTextUnitTask {
|
||||||
|
/// Stable task ID.
|
||||||
|
pub task_id: String,
|
||||||
|
/// Current official release ID.
|
||||||
|
pub official_release_id: String,
|
||||||
|
/// Relative resource path under the official release root.
|
||||||
|
pub destination: String,
|
||||||
|
/// Change kind that caused this task candidate.
|
||||||
|
pub change_kind: OfficialResourceChangeKind,
|
||||||
|
/// Current official URL.
|
||||||
|
pub url: String,
|
||||||
|
/// Verified byte count.
|
||||||
|
pub bytes: u64,
|
||||||
|
/// Verified BLAKE3 digest.
|
||||||
|
pub blake3: String,
|
||||||
|
/// Parse-cache entry key, when available.
|
||||||
|
pub parse_entry_key: Option<String>,
|
||||||
|
/// ZIP/archive entry containing the parsed bundle, when available.
|
||||||
|
pub archive_entry: Option<String>,
|
||||||
|
/// Parse-cache source kind, when available.
|
||||||
|
pub source_kind: Option<OfficialParseSourceKind>,
|
||||||
|
/// Parse status, when available.
|
||||||
|
pub parse_status: Option<OfficialParseStatus>,
|
||||||
|
/// Number of TextAsset objects in this parse entry.
|
||||||
|
pub text_asset_count: usize,
|
||||||
|
/// TextAsset names in this parse entry.
|
||||||
|
pub text_assets: Vec<String>,
|
||||||
|
/// Number of TextUnits in this parse entry.
|
||||||
|
pub text_unit_count: usize,
|
||||||
|
/// TextUnit format labels.
|
||||||
|
pub text_unit_formats: Vec<String>,
|
||||||
|
/// Non-fatal TextUnit extraction diagnostic count.
|
||||||
|
pub text_unit_error_count: usize,
|
||||||
|
/// Queue status.
|
||||||
|
pub status: OfficialTextUnitTaskStatus,
|
||||||
|
/// Diagnostic reason for skipped tasks.
|
||||||
|
pub reason: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Aggregate counters for an incremental TextUnit task queue.
|
||||||
|
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct OfficialTextUnitTaskSummary {
|
||||||
|
/// Number of added/modified resources considered.
|
||||||
|
pub resource_candidate_count: usize,
|
||||||
|
/// Number of parse-cache entries inspected for those resources.
|
||||||
|
pub parse_entry_count: usize,
|
||||||
|
/// Number of tasks queued for offline translation.
|
||||||
|
pub queued_task_count: usize,
|
||||||
|
/// Number of changed resources with no parse-cache entry.
|
||||||
|
pub skipped_no_parse_entry_count: usize,
|
||||||
|
/// Number of parse entries with zero TextUnits.
|
||||||
|
pub skipped_no_text_unit_count: usize,
|
||||||
|
/// Number of failed parse entries.
|
||||||
|
pub skipped_parse_failed_count: usize,
|
||||||
|
/// Number of unsupported parse entries.
|
||||||
|
pub skipped_unsupported_count: usize,
|
||||||
|
/// Total queued TextUnit count.
|
||||||
|
pub text_unit_count: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl OfficialTextUnitTaskSummary {
|
||||||
|
fn record(&mut self, task: &OfficialTextUnitTask) {
|
||||||
|
match task.status {
|
||||||
|
OfficialTextUnitTaskStatus::QueuedOffline => {
|
||||||
|
self.queued_task_count += 1;
|
||||||
|
self.text_unit_count += task.text_unit_count;
|
||||||
|
}
|
||||||
|
OfficialTextUnitTaskStatus::SkippedNoParseEntry => {
|
||||||
|
self.skipped_no_parse_entry_count += 1;
|
||||||
|
}
|
||||||
|
OfficialTextUnitTaskStatus::SkippedNoTextUnit => {
|
||||||
|
self.skipped_no_text_unit_count += 1;
|
||||||
|
}
|
||||||
|
OfficialTextUnitTaskStatus::SkippedParseFailed => {
|
||||||
|
self.skipped_parse_failed_count += 1;
|
||||||
|
}
|
||||||
|
OfficialTextUnitTaskStatus::SkippedUnsupported => {
|
||||||
|
self.skipped_unsupported_count += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Durable incremental TextUnit task queue for one official release.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct OfficialTextUnitTaskQueue {
|
||||||
|
/// Queue schema version.
|
||||||
|
#[serde(default = "default_textunit_task_queue_version")]
|
||||||
|
pub queue_version: u32,
|
||||||
|
/// Current official release ID.
|
||||||
|
pub official_release_id: String,
|
||||||
|
/// Previous official release ID, when known.
|
||||||
|
pub previous_release_id: Option<String>,
|
||||||
|
/// Generation time as Unix seconds.
|
||||||
|
pub generated_unix_seconds: u64,
|
||||||
|
/// Current official release root.
|
||||||
|
pub current_resource_root: PathBuf,
|
||||||
|
/// Queue summary.
|
||||||
|
pub summary: OfficialTextUnitTaskSummary,
|
||||||
|
/// All task candidates, including skipped diagnostics.
|
||||||
|
pub tasks: Vec<OfficialTextUnitTask>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl OfficialTextUnitTaskQueue {
|
||||||
|
/// Builds a task queue from an official change set and parse cache.
|
||||||
|
pub fn from_change_set_and_parse_cache(
|
||||||
|
change_set: &OfficialResourceChangeSet,
|
||||||
|
parse_cache: &OfficialParseCache,
|
||||||
|
) -> Self {
|
||||||
|
let parse_entries = parse_entries_by_destination(parse_cache);
|
||||||
|
let mut summary = OfficialTextUnitTaskSummary {
|
||||||
|
resource_candidate_count: change_set.parse_candidates().len(),
|
||||||
|
..OfficialTextUnitTaskSummary::default()
|
||||||
|
};
|
||||||
|
let mut tasks = Vec::new();
|
||||||
|
|
||||||
|
for change in change_set.parse_candidates() {
|
||||||
|
let entries = parse_entries
|
||||||
|
.get(&change.destination)
|
||||||
|
.map(Vec::as_slice)
|
||||||
|
.unwrap_or(&[]);
|
||||||
|
if entries.is_empty() {
|
||||||
|
let task = skipped_no_parse_entry_task(change_set, change);
|
||||||
|
summary.record(&task);
|
||||||
|
tasks.push(task);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
summary.parse_entry_count += entries.len();
|
||||||
|
for entry in entries {
|
||||||
|
let task = task_from_parse_entry(change_set, change, entry);
|
||||||
|
summary.record(&task);
|
||||||
|
tasks.push(task);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Self {
|
||||||
|
queue_version: OFFICIAL_TEXTUNIT_TASK_QUEUE_VERSION,
|
||||||
|
official_release_id: change_set.official_release_id.clone(),
|
||||||
|
previous_release_id: change_set.previous_release_id.clone(),
|
||||||
|
generated_unix_seconds: unix_seconds_now(),
|
||||||
|
current_resource_root: change_set.current_resource_root.clone(),
|
||||||
|
summary,
|
||||||
|
tasks,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn matches_derivation(&self, expected: &Self) -> bool {
|
||||||
|
self.official_release_id == expected.official_release_id
|
||||||
|
&& self.previous_release_id == expected.previous_release_id
|
||||||
|
&& self.current_resource_root == expected.current_resource_root
|
||||||
|
&& self.summary == expected.summary
|
||||||
|
&& self.tasks == expected.tasks
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One Crowdin offline queue item referencing a TextUnit task.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct CrowdinTextUnitQueueItem {
|
||||||
|
/// Stable TextUnit task ID.
|
||||||
|
pub task_id: String,
|
||||||
|
/// Relative resource path under the official release root.
|
||||||
|
pub destination: String,
|
||||||
|
/// ZIP/archive entry, when applicable.
|
||||||
|
pub archive_entry: Option<String>,
|
||||||
|
/// Number of queued TextUnits.
|
||||||
|
pub text_unit_count: usize,
|
||||||
|
/// TextUnit format labels.
|
||||||
|
pub text_unit_formats: Vec<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Crowdin offline queue derived from queued TextUnit tasks.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct CrowdinTextUnitQueue {
|
||||||
|
/// Queue schema version.
|
||||||
|
#[serde(default = "default_crowdin_textunit_queue_version")]
|
||||||
|
pub queue_version: u32,
|
||||||
|
/// Provider reserved for this queue.
|
||||||
|
pub provider: TranslationHandoffProvider,
|
||||||
|
/// Queue status.
|
||||||
|
pub status: TranslationHandoffStatus,
|
||||||
|
/// Current official release ID.
|
||||||
|
pub official_release_id: String,
|
||||||
|
/// Generation time as Unix seconds.
|
||||||
|
pub generated_unix_seconds: u64,
|
||||||
|
/// Path to the source TextUnit task queue.
|
||||||
|
pub textunit_task_queue_path: PathBuf,
|
||||||
|
/// Number of queued task items.
|
||||||
|
pub task_count: usize,
|
||||||
|
/// Total queued TextUnit count.
|
||||||
|
pub text_unit_count: usize,
|
||||||
|
/// Queued items for the future Crowdin worker.
|
||||||
|
pub items: Vec<CrowdinTextUnitQueueItem>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl CrowdinTextUnitQueue {
|
||||||
|
/// Builds a Crowdin offline queue from queued TextUnit tasks.
|
||||||
|
pub fn from_textunit_task_queue(
|
||||||
|
task_queue_path: PathBuf,
|
||||||
|
task_queue: &OfficialTextUnitTaskQueue,
|
||||||
|
) -> Self {
|
||||||
|
let items = task_queue
|
||||||
|
.tasks
|
||||||
|
.iter()
|
||||||
|
.filter(|task| task.status == OfficialTextUnitTaskStatus::QueuedOffline)
|
||||||
|
.map(|task| CrowdinTextUnitQueueItem {
|
||||||
|
task_id: task.task_id.clone(),
|
||||||
|
destination: task.destination.clone(),
|
||||||
|
archive_entry: task.archive_entry.clone(),
|
||||||
|
text_unit_count: task.text_unit_count,
|
||||||
|
text_unit_formats: task.text_unit_formats.clone(),
|
||||||
|
})
|
||||||
|
.collect::<Vec<_>>();
|
||||||
|
let text_unit_count = items.iter().map(|item| item.text_unit_count).sum();
|
||||||
|
|
||||||
|
Self {
|
||||||
|
queue_version: CROWDIN_TEXTUNIT_QUEUE_VERSION,
|
||||||
|
provider: TranslationHandoffProvider::Crowdin,
|
||||||
|
status: TranslationHandoffStatus::QueuedOffline,
|
||||||
|
official_release_id: task_queue.official_release_id.clone(),
|
||||||
|
generated_unix_seconds: unix_seconds_now(),
|
||||||
|
textunit_task_queue_path: task_queue_path,
|
||||||
|
task_count: items.len(),
|
||||||
|
text_unit_count,
|
||||||
|
items,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn matches_derivation(&self, expected: &Self) -> bool {
|
||||||
|
self.provider == expected.provider
|
||||||
|
&& self.status == expected.status
|
||||||
|
&& self.official_release_id == expected.official_release_id
|
||||||
|
&& self.textunit_task_queue_path == expected.textunit_task_queue_path
|
||||||
|
&& self.task_count == expected.task_count
|
||||||
|
&& self.text_unit_count == expected.text_unit_count
|
||||||
|
&& self.items == expected.items
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Paths and summary produced after writing incremental TextUnit queues.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct OfficialTextUnitQueueReport {
|
||||||
|
/// Path to `official-textunit-tasks.json`.
|
||||||
|
pub textunit_task_queue_path: PathBuf,
|
||||||
|
/// Path to `crowdin-textunit-queue.json`.
|
||||||
|
pub crowdin_textunit_queue_path: PathBuf,
|
||||||
|
/// Aggregate queue counters.
|
||||||
|
pub summary: OfficialTextUnitTaskSummary,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Reads change/parse derived queues from a release root and writes them back.
|
||||||
|
pub fn write_official_textunit_queues(
|
||||||
|
resource_root: &Path,
|
||||||
|
) -> Result<OfficialTextUnitQueueReport, String> {
|
||||||
|
let change_set = read_resource_change_set_at(resource_root)?.ok_or_else(|| {
|
||||||
|
format!(
|
||||||
|
"缺少官方资源变更集,无法生成 TextUnit 队列:{}",
|
||||||
|
resource_root.display()
|
||||||
|
)
|
||||||
|
})?;
|
||||||
|
let parse_cache = read_parse_cache_at(resource_root)?.ok_or_else(|| {
|
||||||
|
format!(
|
||||||
|
"缺少官方解析缓存,无法生成 TextUnit 队列:{}",
|
||||||
|
resource_root.display()
|
||||||
|
)
|
||||||
|
})?;
|
||||||
|
let task_queue =
|
||||||
|
OfficialTextUnitTaskQueue::from_change_set_and_parse_cache(&change_set, &parse_cache);
|
||||||
|
write_textunit_task_queue_at(resource_root, &task_queue)?;
|
||||||
|
|
||||||
|
let task_queue_path = resource_root.join(OFFICIAL_TEXTUNIT_TASK_QUEUE_FILE);
|
||||||
|
let crowdin_queue =
|
||||||
|
CrowdinTextUnitQueue::from_textunit_task_queue(task_queue_path.clone(), &task_queue);
|
||||||
|
write_crowdin_textunit_queue_at(resource_root, &crowdin_queue)?;
|
||||||
|
|
||||||
|
Ok(OfficialTextUnitQueueReport {
|
||||||
|
textunit_task_queue_path: task_queue_path,
|
||||||
|
crowdin_textunit_queue_path: resource_root.join(CROWDIN_TEXTUNIT_QUEUE_FILE),
|
||||||
|
summary: task_queue.summary,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Reads a generated TextUnit task queue from a release root.
|
||||||
|
pub fn read_textunit_task_queue_at(
|
||||||
|
resource_root: &Path,
|
||||||
|
) -> Result<Option<OfficialTextUnitTaskQueue>, String> {
|
||||||
|
let path = resource_root.join(OFFICIAL_TEXTUNIT_TASK_QUEUE_FILE);
|
||||||
|
let Some(bytes) = read_file_no_symlink(&path, "官方 TextUnit 任务队列")? else {
|
||||||
|
return Ok(None);
|
||||||
|
};
|
||||||
|
let queue: OfficialTextUnitTaskQueue = serde_json::from_slice(&bytes)
|
||||||
|
.map_err(|error| format!("解析官方 TextUnit 任务队列失败 {}:{error}", path.display()))?;
|
||||||
|
if queue.queue_version != OFFICIAL_TEXTUNIT_TASK_QUEUE_VERSION {
|
||||||
|
return Err(format!(
|
||||||
|
"不支持的官方 TextUnit 任务队列版本 {},文件 {}",
|
||||||
|
queue.queue_version,
|
||||||
|
path.display()
|
||||||
|
));
|
||||||
|
}
|
||||||
|
Ok(Some(queue))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Returns whether the persisted TextUnit queue still matches current inputs.
|
||||||
|
pub fn is_textunit_task_queue_current(
|
||||||
|
resource_root: &Path,
|
||||||
|
queue: &OfficialTextUnitTaskQueue,
|
||||||
|
) -> Result<bool, String> {
|
||||||
|
let Some(change_set) = read_resource_change_set_at(resource_root)? else {
|
||||||
|
return Ok(false);
|
||||||
|
};
|
||||||
|
let Some(parse_cache) = read_parse_cache_at(resource_root)? else {
|
||||||
|
return Ok(false);
|
||||||
|
};
|
||||||
|
let expected =
|
||||||
|
OfficialTextUnitTaskQueue::from_change_set_and_parse_cache(&change_set, &parse_cache);
|
||||||
|
Ok(queue.matches_derivation(&expected))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Writes a TextUnit task queue under a release root.
|
||||||
|
pub fn write_textunit_task_queue_at(
|
||||||
|
resource_root: &Path,
|
||||||
|
queue: &OfficialTextUnitTaskQueue,
|
||||||
|
) -> Result<(), String> {
|
||||||
|
let path = resource_root.join(OFFICIAL_TEXTUNIT_TASK_QUEUE_FILE);
|
||||||
|
ensure_path_within_root(resource_root, &path)?;
|
||||||
|
ensure_safe_file_target(resource_root, &path, "官方 TextUnit 任务队列")?;
|
||||||
|
let bytes = serde_json::to_vec_pretty(queue)
|
||||||
|
.map_err(|error| format!("序列化官方 TextUnit 任务队列失败:{error}"))?;
|
||||||
|
write_file_atomic(&path, &bytes, STATE_FILE_MODE, "官方 TextUnit 任务队列")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Reads a generated Crowdin offline TextUnit queue from a release root.
|
||||||
|
pub fn read_crowdin_textunit_queue_at(
|
||||||
|
resource_root: &Path,
|
||||||
|
) -> Result<Option<CrowdinTextUnitQueue>, String> {
|
||||||
|
let path = resource_root.join(CROWDIN_TEXTUNIT_QUEUE_FILE);
|
||||||
|
let Some(bytes) = read_file_no_symlink(&path, "Crowdin TextUnit 离线队列")? else {
|
||||||
|
return Ok(None);
|
||||||
|
};
|
||||||
|
let queue: CrowdinTextUnitQueue = serde_json::from_slice(&bytes).map_err(|error| {
|
||||||
|
format!(
|
||||||
|
"解析 Crowdin TextUnit 离线队列失败 {}:{error}",
|
||||||
|
path.display()
|
||||||
|
)
|
||||||
|
})?;
|
||||||
|
if queue.queue_version != CROWDIN_TEXTUNIT_QUEUE_VERSION {
|
||||||
|
return Err(format!(
|
||||||
|
"不支持的 Crowdin TextUnit 离线队列版本 {},文件 {}",
|
||||||
|
queue.queue_version,
|
||||||
|
path.display()
|
||||||
|
));
|
||||||
|
}
|
||||||
|
Ok(Some(queue))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Returns whether the persisted Crowdin queue still matches a TextUnit queue.
|
||||||
|
pub fn is_crowdin_textunit_queue_current(
|
||||||
|
resource_root: &Path,
|
||||||
|
task_queue: &OfficialTextUnitTaskQueue,
|
||||||
|
) -> Result<bool, String> {
|
||||||
|
let Some(queue) = read_crowdin_textunit_queue_at(resource_root)? else {
|
||||||
|
return Ok(false);
|
||||||
|
};
|
||||||
|
let expected = CrowdinTextUnitQueue::from_textunit_task_queue(
|
||||||
|
resource_root.join(OFFICIAL_TEXTUNIT_TASK_QUEUE_FILE),
|
||||||
|
task_queue,
|
||||||
|
);
|
||||||
|
Ok(queue.matches_derivation(&expected))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Writes a Crowdin offline TextUnit queue under a release root.
|
||||||
|
pub fn write_crowdin_textunit_queue_at(
|
||||||
|
resource_root: &Path,
|
||||||
|
queue: &CrowdinTextUnitQueue,
|
||||||
|
) -> Result<(), String> {
|
||||||
|
let path = resource_root.join(CROWDIN_TEXTUNIT_QUEUE_FILE);
|
||||||
|
ensure_path_within_root(resource_root, &path)?;
|
||||||
|
ensure_safe_file_target(resource_root, &path, "Crowdin TextUnit 离线队列")?;
|
||||||
|
let bytes = serde_json::to_vec_pretty(queue)
|
||||||
|
.map_err(|error| format!("序列化 Crowdin TextUnit 离线队列失败:{error}"))?;
|
||||||
|
write_file_atomic(&path, &bytes, STATE_FILE_MODE, "Crowdin TextUnit 离线队列")
|
||||||
|
}
|
||||||
|
|
||||||
|
fn parse_entries_by_destination(
|
||||||
|
cache: &OfficialParseCache,
|
||||||
|
) -> BTreeMap<String, Vec<&OfficialParseCacheEntry>> {
|
||||||
|
let mut by_destination: BTreeMap<String, Vec<&OfficialParseCacheEntry>> = BTreeMap::new();
|
||||||
|
for entry in cache.entries.values() {
|
||||||
|
by_destination
|
||||||
|
.entry(entry.destination.clone())
|
||||||
|
.or_default()
|
||||||
|
.push(entry);
|
||||||
|
}
|
||||||
|
by_destination
|
||||||
|
}
|
||||||
|
|
||||||
|
fn skipped_no_parse_entry_task(
|
||||||
|
change_set: &OfficialResourceChangeSet,
|
||||||
|
change: &OfficialResourceChange,
|
||||||
|
) -> OfficialTextUnitTask {
|
||||||
|
let current = change
|
||||||
|
.current
|
||||||
|
.as_ref()
|
||||||
|
.expect("parse candidate has current");
|
||||||
|
OfficialTextUnitTask {
|
||||||
|
task_id: task_id_for(&change_set.official_release_id, &change.destination, None),
|
||||||
|
official_release_id: change_set.official_release_id.clone(),
|
||||||
|
destination: change.destination.clone(),
|
||||||
|
change_kind: change.kind,
|
||||||
|
url: current.url.clone(),
|
||||||
|
bytes: current.bytes,
|
||||||
|
blake3: current.blake3.clone(),
|
||||||
|
parse_entry_key: None,
|
||||||
|
archive_entry: None,
|
||||||
|
source_kind: None,
|
||||||
|
parse_status: None,
|
||||||
|
text_asset_count: 0,
|
||||||
|
text_assets: Vec::new(),
|
||||||
|
text_unit_count: 0,
|
||||||
|
text_unit_formats: Vec::new(),
|
||||||
|
text_unit_error_count: 0,
|
||||||
|
status: OfficialTextUnitTaskStatus::SkippedNoParseEntry,
|
||||||
|
reason: Some("parse cache entry not found for changed resource".to_string()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn task_from_parse_entry(
|
||||||
|
change_set: &OfficialResourceChangeSet,
|
||||||
|
change: &OfficialResourceChange,
|
||||||
|
entry: &OfficialParseCacheEntry,
|
||||||
|
) -> OfficialTextUnitTask {
|
||||||
|
let current = change
|
||||||
|
.current
|
||||||
|
.as_ref()
|
||||||
|
.expect("parse candidate has current");
|
||||||
|
let (status, reason) = status_for_parse_entry(entry);
|
||||||
|
OfficialTextUnitTask {
|
||||||
|
task_id: task_id_for(
|
||||||
|
&change_set.official_release_id,
|
||||||
|
&change.destination,
|
||||||
|
entry.archive_entry.as_deref(),
|
||||||
|
),
|
||||||
|
official_release_id: change_set.official_release_id.clone(),
|
||||||
|
destination: change.destination.clone(),
|
||||||
|
change_kind: change.kind,
|
||||||
|
url: current.url.clone(),
|
||||||
|
bytes: current.bytes,
|
||||||
|
blake3: current.blake3.clone(),
|
||||||
|
parse_entry_key: Some(entry.key.clone()),
|
||||||
|
archive_entry: entry.archive_entry.clone(),
|
||||||
|
source_kind: Some(entry.source_kind),
|
||||||
|
parse_status: Some(entry.status),
|
||||||
|
text_asset_count: entry.text_asset_count,
|
||||||
|
text_assets: entry.text_assets.clone(),
|
||||||
|
text_unit_count: entry.text_unit_count,
|
||||||
|
text_unit_formats: entry.text_unit_formats.clone(),
|
||||||
|
text_unit_error_count: entry.text_unit_error_count,
|
||||||
|
status,
|
||||||
|
reason,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn status_for_parse_entry(
|
||||||
|
entry: &OfficialParseCacheEntry,
|
||||||
|
) -> (OfficialTextUnitTaskStatus, Option<String>) {
|
||||||
|
match entry.status {
|
||||||
|
OfficialParseStatus::Parsed if entry.text_unit_count > 0 => {
|
||||||
|
(OfficialTextUnitTaskStatus::QueuedOffline, None)
|
||||||
|
}
|
||||||
|
OfficialParseStatus::Parsed => (
|
||||||
|
OfficialTextUnitTaskStatus::SkippedNoTextUnit,
|
||||||
|
Some("parsed resource has no TextUnit".to_string()),
|
||||||
|
),
|
||||||
|
OfficialParseStatus::Failed => (
|
||||||
|
OfficialTextUnitTaskStatus::SkippedParseFailed,
|
||||||
|
entry.error.clone(),
|
||||||
|
),
|
||||||
|
OfficialParseStatus::SkippedUnsupported => (
|
||||||
|
OfficialTextUnitTaskStatus::SkippedUnsupported,
|
||||||
|
entry.error.clone(),
|
||||||
|
),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn task_id_for(
|
||||||
|
official_release_id: &str,
|
||||||
|
destination: &str,
|
||||||
|
archive_entry: Option<&str>,
|
||||||
|
) -> String {
|
||||||
|
match archive_entry {
|
||||||
|
Some(entry) => format!("textunit/{official_release_id}/{destination}#{entry}"),
|
||||||
|
None => format!("textunit/{official_release_id}/{destination}"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn unix_seconds_now() -> u64 {
|
||||||
|
SystemTime::now()
|
||||||
|
.duration_since(UNIX_EPOCH)
|
||||||
|
.unwrap_or_default()
|
||||||
|
.as_secs()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn default_textunit_task_queue_version() -> u32 {
|
||||||
|
OFFICIAL_TEXTUNIT_TASK_QUEUE_VERSION
|
||||||
|
}
|
||||||
|
|
||||||
|
fn default_crowdin_textunit_queue_version() -> u32 {
|
||||||
|
CROWDIN_TEXTUNIT_QUEUE_VERSION
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use crate::official_changes::{
|
||||||
|
OfficialResourceChangeSet, OfficialResourceDescriptor, OFFICIAL_RESOURCE_CHANGES_VERSION,
|
||||||
|
};
|
||||||
|
use crate::official_parse::{
|
||||||
|
OfficialParseCache, OfficialParseCacheEntry, OfficialParseSourceFingerprint,
|
||||||
|
OfficialParseSourceKind, OfficialParseSummary, OFFICIAL_PARSE_CACHE_VERSION,
|
||||||
|
};
|
||||||
|
|
||||||
|
fn change_set(root: &Path) -> OfficialResourceChangeSet {
|
||||||
|
OfficialResourceChangeSet {
|
||||||
|
change_set_version: OFFICIAL_RESOURCE_CHANGES_VERSION,
|
||||||
|
official_release_id: "release-new".to_string(),
|
||||||
|
previous_release_id: Some("release-old".to_string()),
|
||||||
|
generated_unix_seconds: 123,
|
||||||
|
previous_resource_root: None,
|
||||||
|
current_resource_root: root.to_path_buf(),
|
||||||
|
summary: crate::OfficialResourceChangeSummary {
|
||||||
|
previous_manifest_present: true,
|
||||||
|
previous_manifest_entry_count: 2,
|
||||||
|
current_manifest_entry_count: 3,
|
||||||
|
added_count: 2,
|
||||||
|
modified_count: 1,
|
||||||
|
removed_count: 1,
|
||||||
|
parse_candidate_count: 3,
|
||||||
|
translation_candidate_count: 3,
|
||||||
|
},
|
||||||
|
changes: vec![
|
||||||
|
change("Bundles/a.bundle", OfficialResourceChangeKind::Added, b"a"),
|
||||||
|
change(
|
||||||
|
"Bundles/b.bundle",
|
||||||
|
OfficialResourceChangeKind::Modified,
|
||||||
|
b"b",
|
||||||
|
),
|
||||||
|
change("Bundles/c.bundle", OfficialResourceChangeKind::Added, b"c"),
|
||||||
|
crate::OfficialResourceChange {
|
||||||
|
destination: "Bundles/removed.bundle".to_string(),
|
||||||
|
kind: OfficialResourceChangeKind::Removed,
|
||||||
|
previous: Some(OfficialResourceDescriptor {
|
||||||
|
url: "https://old/removed".to_string(),
|
||||||
|
destination: "Bundles/removed.bundle".to_string(),
|
||||||
|
bytes: 1,
|
||||||
|
blake3: "old".to_string(),
|
||||||
|
}),
|
||||||
|
current: None,
|
||||||
|
parse_candidate: false,
|
||||||
|
translation_candidate: false,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn change(
|
||||||
|
destination: &str,
|
||||||
|
kind: OfficialResourceChangeKind,
|
||||||
|
bytes: &[u8],
|
||||||
|
) -> crate::OfficialResourceChange {
|
||||||
|
crate::OfficialResourceChange {
|
||||||
|
destination: destination.to_string(),
|
||||||
|
kind,
|
||||||
|
previous: None,
|
||||||
|
current: Some(OfficialResourceDescriptor {
|
||||||
|
url: format!("https://new/{destination}"),
|
||||||
|
destination: destination.to_string(),
|
||||||
|
bytes: bytes.len() as u64,
|
||||||
|
blake3: blake3::hash(bytes).to_hex().to_string(),
|
||||||
|
}),
|
||||||
|
parse_candidate: true,
|
||||||
|
translation_candidate: true,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn parse_cache() -> OfficialParseCache {
|
||||||
|
OfficialParseCache {
|
||||||
|
version: OFFICIAL_PARSE_CACHE_VERSION,
|
||||||
|
generated_unix_seconds: 124,
|
||||||
|
summary: OfficialParseSummary {
|
||||||
|
manifest_entry_count: 3,
|
||||||
|
cache_entry_count: 2,
|
||||||
|
candidate_file_count: 2,
|
||||||
|
zip_entry_count: 0,
|
||||||
|
skipped_unchanged_count: 0,
|
||||||
|
parsed_bundle_count: 1,
|
||||||
|
unsupported_count: 1,
|
||||||
|
failed_count: 0,
|
||||||
|
text_asset_count: 1,
|
||||||
|
text_unit_count: 2,
|
||||||
|
skipped_binary_text_asset_count: 0,
|
||||||
|
text_unit_error_count: 0,
|
||||||
|
},
|
||||||
|
entries: BTreeMap::from([
|
||||||
|
(
|
||||||
|
"direct:a".to_string(),
|
||||||
|
parse_entry(
|
||||||
|
"direct:a",
|
||||||
|
"Bundles/a.bundle",
|
||||||
|
OfficialParseStatus::Parsed,
|
||||||
|
2,
|
||||||
|
),
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"direct:b".to_string(),
|
||||||
|
parse_entry(
|
||||||
|
"direct:b",
|
||||||
|
"Bundles/b.bundle",
|
||||||
|
OfficialParseStatus::SkippedUnsupported,
|
||||||
|
0,
|
||||||
|
),
|
||||||
|
),
|
||||||
|
]),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn parse_entry(
|
||||||
|
key: &str,
|
||||||
|
destination: &str,
|
||||||
|
status: OfficialParseStatus,
|
||||||
|
text_unit_count: usize,
|
||||||
|
) -> OfficialParseCacheEntry {
|
||||||
|
OfficialParseCacheEntry {
|
||||||
|
key: key.to_string(),
|
||||||
|
source_url: format!("https://new/{destination}"),
|
||||||
|
destination: destination.to_string(),
|
||||||
|
archive_entry: None,
|
||||||
|
source_kind: OfficialParseSourceKind::DirectBundle,
|
||||||
|
fingerprint: OfficialParseSourceFingerprint {
|
||||||
|
source_url: format!("https://new/{destination}"),
|
||||||
|
destination: destination.to_string(),
|
||||||
|
bytes: 1,
|
||||||
|
blake3: "hash".to_string(),
|
||||||
|
},
|
||||||
|
status,
|
||||||
|
reused_from_previous_cache: false,
|
||||||
|
unity_version: Some("2021.3.56f2".to_string()),
|
||||||
|
file_count: 1,
|
||||||
|
serialized_file_count: 1,
|
||||||
|
text_asset_count: usize::from(text_unit_count > 0),
|
||||||
|
text_assets: if text_unit_count > 0 {
|
||||||
|
vec!["Scenario".to_string()]
|
||||||
|
} else {
|
||||||
|
Vec::new()
|
||||||
|
},
|
||||||
|
serialized_parse_error_count: 0,
|
||||||
|
text_unit_count,
|
||||||
|
text_unit_formats: if text_unit_count > 0 {
|
||||||
|
vec!["plain".to_string()]
|
||||||
|
} else {
|
||||||
|
Vec::new()
|
||||||
|
},
|
||||||
|
skipped_binary_text_asset_count: 0,
|
||||||
|
text_unit_error_count: 0,
|
||||||
|
error: (status != OfficialParseStatus::Parsed).then(|| "unsupported".to_string()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn textunit_queue_uses_only_added_and_modified_parse_candidates() {
|
||||||
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
let queue = OfficialTextUnitTaskQueue::from_change_set_and_parse_cache(
|
||||||
|
&change_set(temp.path()),
|
||||||
|
&parse_cache(),
|
||||||
|
);
|
||||||
|
|
||||||
|
assert_eq!(queue.summary.resource_candidate_count, 3);
|
||||||
|
assert_eq!(queue.summary.parse_entry_count, 2);
|
||||||
|
assert_eq!(queue.summary.queued_task_count, 1);
|
||||||
|
assert_eq!(queue.summary.skipped_unsupported_count, 1);
|
||||||
|
assert_eq!(queue.summary.skipped_no_parse_entry_count, 1);
|
||||||
|
assert_eq!(queue.summary.text_unit_count, 2);
|
||||||
|
assert_eq!(
|
||||||
|
queue
|
||||||
|
.tasks
|
||||||
|
.iter()
|
||||||
|
.filter(|task| task.status == OfficialTextUnitTaskStatus::QueuedOffline)
|
||||||
|
.map(|task| task.destination.as_str())
|
||||||
|
.collect::<Vec<_>>(),
|
||||||
|
vec!["Bundles/a.bundle"]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn crowdin_queue_contains_only_queued_textunit_tasks() {
|
||||||
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
let queue = OfficialTextUnitTaskQueue::from_change_set_and_parse_cache(
|
||||||
|
&change_set(temp.path()),
|
||||||
|
&parse_cache(),
|
||||||
|
);
|
||||||
|
let crowdin = CrowdinTextUnitQueue::from_textunit_task_queue(
|
||||||
|
temp.path().join(OFFICIAL_TEXTUNIT_TASK_QUEUE_FILE),
|
||||||
|
&queue,
|
||||||
|
);
|
||||||
|
|
||||||
|
assert_eq!(crowdin.provider, TranslationHandoffProvider::Crowdin);
|
||||||
|
assert_eq!(crowdin.status, TranslationHandoffStatus::QueuedOffline);
|
||||||
|
assert_eq!(crowdin.task_count, 1);
|
||||||
|
assert_eq!(crowdin.text_unit_count, 2);
|
||||||
|
assert_eq!(crowdin.items[0].destination, "Bundles/a.bundle");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn write_textunit_queues_persists_files() {
|
||||||
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
crate::write_resource_change_set_at(temp.path(), &change_set(temp.path())).unwrap();
|
||||||
|
crate::write_parse_cache_at(temp.path(), &parse_cache()).unwrap();
|
||||||
|
|
||||||
|
let report = write_official_textunit_queues(temp.path()).unwrap();
|
||||||
|
|
||||||
|
assert_eq!(report.summary.queued_task_count, 1);
|
||||||
|
assert!(report.textunit_task_queue_path.is_file());
|
||||||
|
assert!(report.crowdin_textunit_queue_path.is_file());
|
||||||
|
let persisted = read_textunit_task_queue_at(temp.path()).unwrap().unwrap();
|
||||||
|
assert_eq!(persisted.summary.text_unit_count, 2);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn textunit_queue_current_detects_parse_cache_changes() {
|
||||||
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
crate::write_resource_change_set_at(temp.path(), &change_set(temp.path())).unwrap();
|
||||||
|
crate::write_parse_cache_at(temp.path(), &parse_cache()).unwrap();
|
||||||
|
write_official_textunit_queues(temp.path()).unwrap();
|
||||||
|
|
||||||
|
let queue = read_textunit_task_queue_at(temp.path()).unwrap().unwrap();
|
||||||
|
assert!(is_textunit_task_queue_current(temp.path(), &queue).unwrap());
|
||||||
|
|
||||||
|
let mut changed_cache = parse_cache();
|
||||||
|
changed_cache.entries.insert(
|
||||||
|
"direct:c".to_string(),
|
||||||
|
parse_entry(
|
||||||
|
"direct:c",
|
||||||
|
"Bundles/c.bundle",
|
||||||
|
OfficialParseStatus::Parsed,
|
||||||
|
1,
|
||||||
|
),
|
||||||
|
);
|
||||||
|
crate::write_parse_cache_at(temp.path(), &changed_cache).unwrap();
|
||||||
|
|
||||||
|
assert!(!is_textunit_task_queue_current(temp.path(), &queue).unwrap());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn crowdin_queue_current_requires_matching_file() {
|
||||||
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
let queue = OfficialTextUnitTaskQueue::from_change_set_and_parse_cache(
|
||||||
|
&change_set(temp.path()),
|
||||||
|
&parse_cache(),
|
||||||
|
);
|
||||||
|
|
||||||
|
write_textunit_task_queue_at(temp.path(), &queue).unwrap();
|
||||||
|
assert!(!is_crowdin_textunit_queue_current(temp.path(), &queue).unwrap());
|
||||||
|
|
||||||
|
let crowdin = CrowdinTextUnitQueue::from_textunit_task_queue(
|
||||||
|
temp.path().join(OFFICIAL_TEXTUNIT_TASK_QUEUE_FILE),
|
||||||
|
&queue,
|
||||||
|
);
|
||||||
|
write_crowdin_textunit_queue_at(temp.path(), &crowdin).unwrap();
|
||||||
|
|
||||||
|
assert!(is_crowdin_textunit_queue_current(temp.path(), &queue).unwrap());
|
||||||
|
}
|
||||||
|
}
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,424 @@
|
|||||||
|
//! File-level Patch and UnityFS write operations.
|
||||||
|
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
|
||||||
|
use bat_assetbundle::{
|
||||||
|
patch_unityfs_field, patch_unityfs_string_field, patch_unityfs_text_asset, FieldPatch,
|
||||||
|
StringFieldPatch, TextAssetPatch, UnitySerializedReplacementValue,
|
||||||
|
};
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
|
||||||
|
use crate::path_security::{
|
||||||
|
ensure_safe_file_target, lexical_absolute, read_file_no_symlink, write_file_atomic,
|
||||||
|
STATE_FILE_MODE,
|
||||||
|
};
|
||||||
|
|
||||||
|
/// File patch algorithm selected by `patch.apply`.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
#[serde(rename_all = "snake_case")]
|
||||||
|
pub enum PatchApplyKind {
|
||||||
|
/// Deterministic binary hunk patch.
|
||||||
|
Binary,
|
||||||
|
/// RFC 6902 JSON Patch.
|
||||||
|
Json,
|
||||||
|
/// UTF-8 Text Patch.
|
||||||
|
Text,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl PatchApplyKind {
|
||||||
|
/// Stable RPC/CLI label.
|
||||||
|
pub fn as_str(self) -> &'static str {
|
||||||
|
match self {
|
||||||
|
Self::Binary => "binary",
|
||||||
|
Self::Json => "json",
|
||||||
|
Self::Text => "text",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Parameters for applying one file-level patch.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct PatchApplyParams {
|
||||||
|
/// Patch algorithm.
|
||||||
|
pub kind: PatchApplyKind,
|
||||||
|
/// Source file path.
|
||||||
|
pub source_path: PathBuf,
|
||||||
|
/// Patch document path.
|
||||||
|
pub patch_path: PathBuf,
|
||||||
|
/// Target file path written atomically.
|
||||||
|
pub target_path: PathBuf,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Parameters for patching one UnityFS TextAsset object.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct UnityFsTextAssetPatchParams {
|
||||||
|
/// Source UnityFS bundle file.
|
||||||
|
pub bundle_path: PathBuf,
|
||||||
|
/// Serialized file path inside the UnityFS directory table.
|
||||||
|
pub serialized_file_path: String,
|
||||||
|
/// Unity object path ID.
|
||||||
|
pub path_id: i64,
|
||||||
|
/// Replacement raw bytes path.
|
||||||
|
pub replacement_path: PathBuf,
|
||||||
|
/// Target bundle file path written atomically.
|
||||||
|
pub target_path: PathBuf,
|
||||||
|
/// Optional expected TextAsset name.
|
||||||
|
#[serde(default)]
|
||||||
|
pub expected_name: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Parameters for patching one TypeTree string field inside a UnityFS bundle.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct UnityFsStringFieldPatchParams {
|
||||||
|
/// Source UnityFS bundle file.
|
||||||
|
pub bundle_path: PathBuf,
|
||||||
|
/// Serialized file path inside the UnityFS directory table.
|
||||||
|
pub serialized_file_path: String,
|
||||||
|
/// Unity object path ID.
|
||||||
|
pub path_id: i64,
|
||||||
|
/// Stable TypeTree field path.
|
||||||
|
pub field_path: String,
|
||||||
|
/// Replacement string supplied inline.
|
||||||
|
#[serde(default)]
|
||||||
|
pub replacement_text: Option<String>,
|
||||||
|
/// Replacement UTF-8 file path.
|
||||||
|
#[serde(default)]
|
||||||
|
pub replacement_path: Option<PathBuf>,
|
||||||
|
/// Target bundle file path written atomically.
|
||||||
|
pub target_path: PathBuf,
|
||||||
|
/// Optional expected source string.
|
||||||
|
#[serde(default)]
|
||||||
|
pub expected_value: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Parameters for patching one semantic TypeTree field inside a UnityFS bundle.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct UnityFsFieldPatchParams {
|
||||||
|
/// Source UnityFS bundle file.
|
||||||
|
pub bundle_path: PathBuf,
|
||||||
|
/// Serialized file path inside the UnityFS directory table.
|
||||||
|
pub serialized_file_path: String,
|
||||||
|
/// Unity object path ID.
|
||||||
|
pub path_id: i64,
|
||||||
|
/// Stable TypeTree field path.
|
||||||
|
pub field_path: String,
|
||||||
|
/// Replacement value encoded according to the current TypeTree field type.
|
||||||
|
pub replacement: UnitySerializedReplacementValue,
|
||||||
|
/// Target bundle file path written atomically.
|
||||||
|
pub target_path: PathBuf,
|
||||||
|
/// Optional expected source value.
|
||||||
|
#[serde(default)]
|
||||||
|
pub expected_value: Option<UnitySerializedReplacementValue>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Result of a file-level patch operation.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
|
||||||
|
pub struct PatchApplyReport {
|
||||||
|
/// RPC/CLI command name.
|
||||||
|
pub command: &'static str,
|
||||||
|
/// Operation status.
|
||||||
|
pub status: &'static str,
|
||||||
|
/// Human-readable summary.
|
||||||
|
pub message: &'static str,
|
||||||
|
/// Patch algorithm.
|
||||||
|
pub kind: PatchApplyKind,
|
||||||
|
/// Absolute source path.
|
||||||
|
pub source_path: PathBuf,
|
||||||
|
/// Absolute patch path.
|
||||||
|
pub patch_path: PathBuf,
|
||||||
|
/// Absolute target path.
|
||||||
|
pub target_path: PathBuf,
|
||||||
|
/// Source byte length.
|
||||||
|
pub source_size: u64,
|
||||||
|
/// Patch document byte length.
|
||||||
|
pub patch_size: u64,
|
||||||
|
/// Target byte length.
|
||||||
|
pub target_size: u64,
|
||||||
|
/// Source BLAKE3 hash.
|
||||||
|
pub source_blake3: String,
|
||||||
|
/// Patch document BLAKE3 hash.
|
||||||
|
pub patch_blake3: String,
|
||||||
|
/// Target BLAKE3 hash.
|
||||||
|
pub target_blake3: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Result of a UnityFS write operation.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
|
||||||
|
pub struct UnityFsPatchReport {
|
||||||
|
/// RPC/CLI command name.
|
||||||
|
pub command: &'static str,
|
||||||
|
/// Operation status.
|
||||||
|
pub status: &'static str,
|
||||||
|
/// Human-readable summary.
|
||||||
|
pub message: &'static str,
|
||||||
|
/// Absolute source UnityFS bundle path.
|
||||||
|
pub bundle_path: PathBuf,
|
||||||
|
/// Serialized file path inside the UnityFS directory table.
|
||||||
|
pub serialized_file_path: String,
|
||||||
|
/// Unity object path ID.
|
||||||
|
pub path_id: i64,
|
||||||
|
/// Optional TypeTree field path for string-field patches.
|
||||||
|
#[serde(skip_serializing_if = "Option::is_none")]
|
||||||
|
pub field_path: Option<String>,
|
||||||
|
/// Absolute target bundle path.
|
||||||
|
pub target_path: PathBuf,
|
||||||
|
/// Source bundle byte length.
|
||||||
|
pub source_size: u64,
|
||||||
|
/// Replacement byte length.
|
||||||
|
pub replacement_size: u64,
|
||||||
|
/// Target bundle byte length.
|
||||||
|
pub target_size: u64,
|
||||||
|
/// Source bundle BLAKE3 hash.
|
||||||
|
pub source_blake3: String,
|
||||||
|
/// Replacement BLAKE3 hash.
|
||||||
|
pub replacement_blake3: String,
|
||||||
|
/// Target bundle BLAKE3 hash.
|
||||||
|
pub target_blake3: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Applies a Binary/JSON/Text patch document to one source file.
|
||||||
|
pub fn apply_patch_file(params: &PatchApplyParams) -> anyhow::Result<PatchApplyReport> {
|
||||||
|
let source_path = lexical_absolute(¶ms.source_path).map_err(anyhow::Error::msg)?;
|
||||||
|
let patch_path = lexical_absolute(¶ms.patch_path).map_err(anyhow::Error::msg)?;
|
||||||
|
let target_path = lexical_absolute(¶ms.target_path).map_err(anyhow::Error::msg)?;
|
||||||
|
ensure_target_is_not_input(&target_path, &[&source_path, &patch_path])?;
|
||||||
|
|
||||||
|
let source = read_required_file(&source_path, "patch source")?;
|
||||||
|
let patch = read_required_file(&patch_path, "patch document")?;
|
||||||
|
let target = match params.kind {
|
||||||
|
PatchApplyKind::Binary => bat_patch::binary::apply_patch(&source, &patch)?,
|
||||||
|
PatchApplyKind::Json => {
|
||||||
|
let source = std::str::from_utf8(&source)
|
||||||
|
.map_err(|error| anyhow::anyhow!("JSON patch source 不是 UTF-8:{error}"))?;
|
||||||
|
let patch = std::str::from_utf8(&patch)
|
||||||
|
.map_err(|error| anyhow::anyhow!("JSON patch document 不是 UTF-8:{error}"))?;
|
||||||
|
bat_patch::json::apply_json_patch(source, patch)?.into_bytes()
|
||||||
|
}
|
||||||
|
PatchApplyKind::Text => bat_patch::text::apply_patch_bytes(&source, &patch)?,
|
||||||
|
};
|
||||||
|
write_output_file(&target_path, &target, "patch target")?;
|
||||||
|
|
||||||
|
Ok(PatchApplyReport {
|
||||||
|
command: "patch.apply",
|
||||||
|
status: "patched",
|
||||||
|
message: "patch 已应用并原子写入目标文件",
|
||||||
|
kind: params.kind,
|
||||||
|
source_path,
|
||||||
|
patch_path,
|
||||||
|
target_path,
|
||||||
|
source_size: source.len() as u64,
|
||||||
|
patch_size: patch.len() as u64,
|
||||||
|
target_size: target.len() as u64,
|
||||||
|
source_blake3: blake3_hex(&source),
|
||||||
|
patch_blake3: blake3_hex(&patch),
|
||||||
|
target_blake3: blake3_hex(&target),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Patches one UnityFS TextAsset and writes the rebuilt bundle atomically.
|
||||||
|
pub fn apply_unityfs_text_asset_patch_file(
|
||||||
|
params: &UnityFsTextAssetPatchParams,
|
||||||
|
) -> anyhow::Result<UnityFsPatchReport> {
|
||||||
|
let bundle_path = lexical_absolute(¶ms.bundle_path).map_err(anyhow::Error::msg)?;
|
||||||
|
let replacement_path =
|
||||||
|
lexical_absolute(¶ms.replacement_path).map_err(anyhow::Error::msg)?;
|
||||||
|
let target_path = lexical_absolute(¶ms.target_path).map_err(anyhow::Error::msg)?;
|
||||||
|
ensure_target_is_not_input(&target_path, &[&bundle_path, &replacement_path])?;
|
||||||
|
|
||||||
|
let bundle = read_required_file(&bundle_path, "UnityFS bundle")?;
|
||||||
|
let replacement = read_required_file(&replacement_path, "TextAsset replacement")?;
|
||||||
|
let mut patch = TextAssetPatch::new(
|
||||||
|
params.serialized_file_path.clone(),
|
||||||
|
params.path_id,
|
||||||
|
replacement.clone(),
|
||||||
|
);
|
||||||
|
patch.expected_name = params.expected_name.clone();
|
||||||
|
let target = patch_unityfs_text_asset(&bundle, &patch)?;
|
||||||
|
write_output_file(&target_path, &target, "UnityFS target")?;
|
||||||
|
|
||||||
|
Ok(UnityFsPatchReport {
|
||||||
|
command: "unityfs.patch_text_asset",
|
||||||
|
status: "patched",
|
||||||
|
message: "UnityFS TextAsset patch 已应用并原子写入目标 bundle",
|
||||||
|
bundle_path,
|
||||||
|
serialized_file_path: params.serialized_file_path.clone(),
|
||||||
|
path_id: params.path_id,
|
||||||
|
field_path: None,
|
||||||
|
target_path,
|
||||||
|
source_size: bundle.len() as u64,
|
||||||
|
replacement_size: replacement.len() as u64,
|
||||||
|
target_size: target.len() as u64,
|
||||||
|
source_blake3: blake3_hex(&bundle),
|
||||||
|
replacement_blake3: blake3_hex(&replacement),
|
||||||
|
target_blake3: blake3_hex(&target),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Patches one TypeTree string field and writes the rebuilt UnityFS bundle atomically.
|
||||||
|
pub fn apply_unityfs_string_field_patch_file(
|
||||||
|
params: &UnityFsStringFieldPatchParams,
|
||||||
|
) -> anyhow::Result<UnityFsPatchReport> {
|
||||||
|
let bundle_path = lexical_absolute(¶ms.bundle_path).map_err(anyhow::Error::msg)?;
|
||||||
|
let target_path = lexical_absolute(¶ms.target_path).map_err(anyhow::Error::msg)?;
|
||||||
|
let replacement = replacement_text(params)?;
|
||||||
|
let extra_inputs = params
|
||||||
|
.replacement_path
|
||||||
|
.as_ref()
|
||||||
|
.map(|path| lexical_absolute(path).map_err(anyhow::Error::msg))
|
||||||
|
.transpose()?;
|
||||||
|
let mut inputs = vec![bundle_path.as_path()];
|
||||||
|
if let Some(path) = extra_inputs.as_ref() {
|
||||||
|
inputs.push(path.as_path());
|
||||||
|
}
|
||||||
|
ensure_target_is_not_input(&target_path, &inputs)?;
|
||||||
|
|
||||||
|
let bundle = read_required_file(&bundle_path, "UnityFS bundle")?;
|
||||||
|
let patch = StringFieldPatch {
|
||||||
|
serialized_file_path: params.serialized_file_path.clone(),
|
||||||
|
path_id: params.path_id,
|
||||||
|
field_path: params.field_path.clone(),
|
||||||
|
expected_value: params.expected_value.clone(),
|
||||||
|
replacement: replacement.clone(),
|
||||||
|
};
|
||||||
|
let target = patch_unityfs_string_field(&bundle, &patch)?;
|
||||||
|
write_output_file(&target_path, &target, "UnityFS target")?;
|
||||||
|
|
||||||
|
Ok(UnityFsPatchReport {
|
||||||
|
command: "unityfs.patch_string_field",
|
||||||
|
status: "patched",
|
||||||
|
message: "UnityFS TypeTree string field patch 已应用并原子写入目标 bundle",
|
||||||
|
bundle_path,
|
||||||
|
serialized_file_path: params.serialized_file_path.clone(),
|
||||||
|
path_id: params.path_id,
|
||||||
|
field_path: Some(params.field_path.clone()),
|
||||||
|
target_path,
|
||||||
|
source_size: bundle.len() as u64,
|
||||||
|
replacement_size: replacement.len() as u64,
|
||||||
|
target_size: target.len() as u64,
|
||||||
|
source_blake3: blake3_hex(&bundle),
|
||||||
|
replacement_blake3: blake3_hex(replacement.as_bytes()),
|
||||||
|
target_blake3: blake3_hex(&target),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Patches one semantic TypeTree field and writes the rebuilt UnityFS bundle atomically.
|
||||||
|
pub fn apply_unityfs_field_patch_file(
|
||||||
|
params: &UnityFsFieldPatchParams,
|
||||||
|
) -> anyhow::Result<UnityFsPatchReport> {
|
||||||
|
let bundle_path = lexical_absolute(¶ms.bundle_path).map_err(anyhow::Error::msg)?;
|
||||||
|
let target_path = lexical_absolute(¶ms.target_path).map_err(anyhow::Error::msg)?;
|
||||||
|
ensure_target_is_not_input(&target_path, &[&bundle_path])?;
|
||||||
|
|
||||||
|
let bundle = read_required_file(&bundle_path, "UnityFS bundle")?;
|
||||||
|
let replacement_size = serde_json::to_vec(¶ms.replacement)
|
||||||
|
.map_err(anyhow::Error::from)?
|
||||||
|
.len() as u64;
|
||||||
|
let patch = FieldPatch {
|
||||||
|
serialized_file_path: params.serialized_file_path.clone(),
|
||||||
|
path_id: params.path_id,
|
||||||
|
field_path: params.field_path.clone(),
|
||||||
|
expected_value: params.expected_value.clone(),
|
||||||
|
replacement: params.replacement.clone(),
|
||||||
|
};
|
||||||
|
let target = patch_unityfs_field(&bundle, &patch)?;
|
||||||
|
write_output_file(&target_path, &target, "UnityFS target")?;
|
||||||
|
|
||||||
|
Ok(UnityFsPatchReport {
|
||||||
|
command: "unityfs.patch_field",
|
||||||
|
status: "patched",
|
||||||
|
message: "UnityFS TypeTree field patch 已应用并原子写入目标 bundle",
|
||||||
|
bundle_path,
|
||||||
|
serialized_file_path: params.serialized_file_path.clone(),
|
||||||
|
path_id: params.path_id,
|
||||||
|
field_path: Some(params.field_path.clone()),
|
||||||
|
target_path,
|
||||||
|
source_size: bundle.len() as u64,
|
||||||
|
replacement_size,
|
||||||
|
target_size: target.len() as u64,
|
||||||
|
source_blake3: blake3_hex(&bundle),
|
||||||
|
replacement_blake3: blake3_hex(&serde_json::to_vec(¶ms.replacement)?),
|
||||||
|
target_blake3: blake3_hex(&target),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
fn replacement_text(params: &UnityFsStringFieldPatchParams) -> anyhow::Result<String> {
|
||||||
|
match (¶ms.replacement_text, ¶ms.replacement_path) {
|
||||||
|
(Some(_), Some(_)) => Err(anyhow::anyhow!(
|
||||||
|
"replacement_text 和 replacement_path 只能指定一个"
|
||||||
|
)),
|
||||||
|
(Some(text), None) => Ok(text.clone()),
|
||||||
|
(None, Some(path)) => {
|
||||||
|
let path = lexical_absolute(path).map_err(anyhow::Error::msg)?;
|
||||||
|
let bytes = read_required_file(&path, "string replacement")?;
|
||||||
|
String::from_utf8(bytes)
|
||||||
|
.map_err(|error| anyhow::anyhow!("string replacement 不是 UTF-8:{error}"))
|
||||||
|
}
|
||||||
|
(None, None) => Err(anyhow::anyhow!(
|
||||||
|
"必须指定 replacement_text 或 replacement_path"
|
||||||
|
)),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn read_required_file(path: &Path, label: &str) -> anyhow::Result<Vec<u8>> {
|
||||||
|
read_file_no_symlink(path, label)
|
||||||
|
.map_err(anyhow::Error::msg)?
|
||||||
|
.ok_or_else(|| anyhow::anyhow!("{label} 不存在:{}", path.display()))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn write_output_file(path: &Path, bytes: &[u8], label: &str) -> anyhow::Result<()> {
|
||||||
|
let parent = path
|
||||||
|
.parent()
|
||||||
|
.ok_or_else(|| anyhow::anyhow!("{label} 缺少父目录:{}", path.display()))?;
|
||||||
|
ensure_safe_file_target(parent, path, label).map_err(anyhow::Error::msg)?;
|
||||||
|
write_file_atomic(path, bytes, STATE_FILE_MODE, label).map_err(anyhow::Error::msg)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn ensure_target_is_not_input(target: &Path, inputs: &[&Path]) -> anyhow::Result<()> {
|
||||||
|
for input in inputs {
|
||||||
|
if target == *input {
|
||||||
|
return Err(anyhow::anyhow!(
|
||||||
|
"target_path 不能与输入文件相同:{}",
|
||||||
|
target.display()
|
||||||
|
));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn blake3_hex(bytes: &[u8]) -> String {
|
||||||
|
blake3::hash(bytes).to_hex().to_string()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn patch_apply_rejects_in_place_target() {
|
||||||
|
let path = PathBuf::from("/tmp/source.bin");
|
||||||
|
let params = PatchApplyParams {
|
||||||
|
kind: PatchApplyKind::Binary,
|
||||||
|
source_path: path.clone(),
|
||||||
|
patch_path: PathBuf::from("/tmp/patch.json"),
|
||||||
|
target_path: path,
|
||||||
|
};
|
||||||
|
let error = apply_patch_file(¶ms).unwrap_err().to_string();
|
||||||
|
assert!(error.contains("target_path 不能与输入文件相同"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn string_field_params_require_one_replacement_source() {
|
||||||
|
let params = UnityFsStringFieldPatchParams {
|
||||||
|
bundle_path: PathBuf::from("/tmp/source.bundle"),
|
||||||
|
serialized_file_path: "CAB".to_string(),
|
||||||
|
path_id: 1,
|
||||||
|
field_path: "message".to_string(),
|
||||||
|
replacement_text: Some("a".to_string()),
|
||||||
|
replacement_path: Some(PathBuf::from("/tmp/replacement.txt")),
|
||||||
|
target_path: PathBuf::from("/tmp/target.bundle"),
|
||||||
|
expected_value: None,
|
||||||
|
};
|
||||||
|
let error = replacement_text(¶ms).unwrap_err().to_string();
|
||||||
|
assert!(error.contains("只能指定一个"));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
//! 内存资源仓储实现。
|
//! 内存资源仓储实现。
|
||||||
|
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
use bat_core::domain::{Resource, ResourceEntry, ResourceType};
|
use bat_core::domain::{Resource, ResourceEntry, ResourceMetadata, ResourceType};
|
||||||
use bat_core::repositories::resource_repository::{ResourceQuery, ResourceRepository};
|
use bat_core::repositories::resource_repository::{ResourceQuery, ResourceRepository};
|
||||||
use sqlx::sqlite::{SqliteConnectOptions, SqlitePoolOptions, SqliteQueryResult};
|
use sqlx::sqlite::{SqliteConnectOptions, SqlitePoolOptions, SqliteQueryResult};
|
||||||
use sqlx::{QueryBuilder, Sqlite, SqlitePool};
|
use sqlx::{QueryBuilder, Sqlite, SqlitePool};
|
||||||
@@ -130,7 +130,8 @@ impl SqliteResourceRepository {
|
|||||||
local_path TEXT NOT NULL,
|
local_path TEXT NOT NULL,
|
||||||
address TEXT,
|
address TEXT,
|
||||||
dependencies_json TEXT NOT NULL DEFAULT '[]',
|
dependencies_json TEXT NOT NULL DEFAULT '[]',
|
||||||
crc INTEGER
|
crc INTEGER,
|
||||||
|
metadata_json TEXT NOT NULL DEFAULT '{}'
|
||||||
)
|
)
|
||||||
"#,
|
"#,
|
||||||
),
|
),
|
||||||
@@ -140,6 +141,13 @@ impl SqliteResourceRepository {
|
|||||||
// 向后兼容:早于 crc 列的旧库缺少该列,按需补加(新建库已含该列,
|
// 向后兼容:早于 crc 列的旧库缺少该列,按需补加(新建库已含该列,
|
||||||
// pragma 检查后不会重复 ALTER)。
|
// pragma 检查后不会重复 ALTER)。
|
||||||
Self::ensure_column(&self.pool, "resources", "crc", "INTEGER").await?;
|
Self::ensure_column(&self.pool, "resources", "crc", "INTEGER").await?;
|
||||||
|
Self::ensure_column(
|
||||||
|
&self.pool,
|
||||||
|
"resources",
|
||||||
|
"metadata_json",
|
||||||
|
"TEXT NOT NULL DEFAULT '{}'",
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
|
||||||
Self::execute_query(
|
Self::execute_query(
|
||||||
&self.pool,
|
&self.pool,
|
||||||
@@ -250,9 +258,32 @@ impl SqliteResourceRepository {
|
|||||||
.map_err(|error| bat_core::Error::Serialization(error.to_string()))
|
.map_err(|error| bat_core::Error::Serialization(error.to_string()))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn metadata_to_json(metadata: &ResourceMetadata) -> bat_core::Result<String> {
|
||||||
|
serde_json::to_string(metadata)
|
||||||
|
.map_err(|error| bat_core::Error::Serialization(error.to_string()))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn metadata_from_json(value: &str) -> bat_core::Result<ResourceMetadata> {
|
||||||
|
if value.trim().is_empty() {
|
||||||
|
return Ok(ResourceMetadata::default());
|
||||||
|
}
|
||||||
|
serde_json::from_str(value)
|
||||||
|
.map_err(|error| bat_core::Error::Serialization(error.to_string()))
|
||||||
|
}
|
||||||
|
|
||||||
fn resource_from_row(row: ResourceRow) -> bat_core::Result<Resource> {
|
fn resource_from_row(row: ResourceRow) -> bat_core::Result<Resource> {
|
||||||
let (id, path, hash, size, resource_type, local_path, address, dependencies_json, crc) =
|
let (
|
||||||
row;
|
id,
|
||||||
|
path,
|
||||||
|
hash,
|
||||||
|
size,
|
||||||
|
resource_type,
|
||||||
|
local_path,
|
||||||
|
address,
|
||||||
|
dependencies_json,
|
||||||
|
crc,
|
||||||
|
metadata_json,
|
||||||
|
) = row;
|
||||||
Ok(Resource {
|
Ok(Resource {
|
||||||
id,
|
id,
|
||||||
local_path: PathBuf::from(local_path),
|
local_path: PathBuf::from(local_path),
|
||||||
@@ -265,6 +296,7 @@ impl SqliteResourceRepository {
|
|||||||
dependencies: Self::dependencies_from_json(&dependencies_json)?,
|
dependencies: Self::dependencies_from_json(&dependencies_json)?,
|
||||||
crc: crc.and_then(|value| u32::try_from(value).ok()),
|
crc: crc.and_then(|value| u32::try_from(value).ok()),
|
||||||
},
|
},
|
||||||
|
metadata: Self::metadata_from_json(&metadata_json)?,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -303,7 +335,7 @@ impl SqliteResourceRepository {
|
|||||||
limit: Option<usize>,
|
limit: Option<usize>,
|
||||||
) -> bat_core::Result<Vec<Resource>> {
|
) -> bat_core::Result<Vec<Resource>> {
|
||||||
let mut builder = QueryBuilder::<Sqlite>::new(
|
let mut builder = QueryBuilder::<Sqlite>::new(
|
||||||
"SELECT id, path, hash, size, resource_type, local_path, address, dependencies_json, crc FROM resources",
|
"SELECT id, path, hash, size, resource_type, local_path, address, dependencies_json, crc, metadata_json FROM resources",
|
||||||
);
|
);
|
||||||
Self::apply_filters(&mut builder, query)?;
|
Self::apply_filters(&mut builder, query)?;
|
||||||
builder.push(" ORDER BY id");
|
builder.push(" ORDER BY id");
|
||||||
@@ -337,14 +369,15 @@ impl SqliteResourceRepository {
|
|||||||
impl ResourceRepository for SqliteResourceRepository {
|
impl ResourceRepository for SqliteResourceRepository {
|
||||||
async fn add(&self, resource: Resource) -> bat_core::Result<String> {
|
async fn add(&self, resource: Resource) -> bat_core::Result<String> {
|
||||||
let dependencies = Self::dependencies_to_json(&resource.entry.dependencies)?;
|
let dependencies = Self::dependencies_to_json(&resource.entry.dependencies)?;
|
||||||
|
let metadata = Self::metadata_to_json(&resource.metadata)?;
|
||||||
Self::execute_query(
|
Self::execute_query(
|
||||||
&self.pool,
|
&self.pool,
|
||||||
sqlx::query(
|
sqlx::query(
|
||||||
r#"
|
r#"
|
||||||
INSERT INTO resources (
|
INSERT INTO resources (
|
||||||
id, path, hash, size, resource_type, local_path, address, dependencies_json, crc
|
id, path, hash, size, resource_type, local_path, address, dependencies_json, crc, metadata_json
|
||||||
)
|
)
|
||||||
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9)
|
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10)
|
||||||
ON CONFLICT(id) DO UPDATE SET
|
ON CONFLICT(id) DO UPDATE SET
|
||||||
path = excluded.path,
|
path = excluded.path,
|
||||||
hash = excluded.hash,
|
hash = excluded.hash,
|
||||||
@@ -353,7 +386,8 @@ impl ResourceRepository for SqliteResourceRepository {
|
|||||||
local_path = excluded.local_path,
|
local_path = excluded.local_path,
|
||||||
address = excluded.address,
|
address = excluded.address,
|
||||||
dependencies_json = excluded.dependencies_json,
|
dependencies_json = excluded.dependencies_json,
|
||||||
crc = excluded.crc
|
crc = excluded.crc,
|
||||||
|
metadata_json = excluded.metadata_json
|
||||||
"#,
|
"#,
|
||||||
)
|
)
|
||||||
.bind(resource.id.clone())
|
.bind(resource.id.clone())
|
||||||
@@ -364,7 +398,8 @@ impl ResourceRepository for SqliteResourceRepository {
|
|||||||
.bind(resource.local_path.to_string_lossy().to_string())
|
.bind(resource.local_path.to_string_lossy().to_string())
|
||||||
.bind(resource.entry.address.clone())
|
.bind(resource.entry.address.clone())
|
||||||
.bind(dependencies)
|
.bind(dependencies)
|
||||||
.bind(resource.entry.crc.map(i64::from)),
|
.bind(resource.entry.crc.map(i64::from))
|
||||||
|
.bind(metadata),
|
||||||
)
|
)
|
||||||
.await?;
|
.await?;
|
||||||
|
|
||||||
@@ -374,7 +409,7 @@ impl ResourceRepository for SqliteResourceRepository {
|
|||||||
async fn find_by_id(&self, id: &str) -> bat_core::Result<Resource> {
|
async fn find_by_id(&self, id: &str) -> bat_core::Result<Resource> {
|
||||||
let row: Option<ResourceRow> = sqlx::query_as(
|
let row: Option<ResourceRow> = sqlx::query_as(
|
||||||
r#"
|
r#"
|
||||||
SELECT id, path, hash, size, resource_type, local_path, address, dependencies_json, crc
|
SELECT id, path, hash, size, resource_type, local_path, address, dependencies_json, crc, metadata_json
|
||||||
FROM resources
|
FROM resources
|
||||||
WHERE id = ?1
|
WHERE id = ?1
|
||||||
"#,
|
"#,
|
||||||
@@ -392,7 +427,7 @@ impl ResourceRepository for SqliteResourceRepository {
|
|||||||
async fn find_by_hash(&self, hash: &str) -> bat_core::Result<Resource> {
|
async fn find_by_hash(&self, hash: &str) -> bat_core::Result<Resource> {
|
||||||
let row: Option<ResourceRow> = sqlx::query_as(
|
let row: Option<ResourceRow> = sqlx::query_as(
|
||||||
r#"
|
r#"
|
||||||
SELECT id, path, hash, size, resource_type, local_path, address, dependencies_json, crc
|
SELECT id, path, hash, size, resource_type, local_path, address, dependencies_json, crc, metadata_json
|
||||||
FROM resources
|
FROM resources
|
||||||
WHERE hash = ?1
|
WHERE hash = ?1
|
||||||
ORDER BY id
|
ORDER BY id
|
||||||
@@ -455,6 +490,7 @@ type ResourceRow = (
|
|||||||
Option<String>,
|
Option<String>,
|
||||||
String,
|
String,
|
||||||
Option<i64>,
|
Option<i64>,
|
||||||
|
String,
|
||||||
);
|
);
|
||||||
|
|
||||||
fn glob_to_like(pattern: &str) -> String {
|
fn glob_to_like(pattern: &str) -> String {
|
||||||
@@ -544,6 +580,7 @@ mod tests {
|
|||||||
dependencies: Vec::new(),
|
dependencies: Vec::new(),
|
||||||
crc: None,
|
crc: None,
|
||||||
},
|
},
|
||||||
|
metadata: ResourceMetadata::default(),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -638,6 +675,10 @@ mod tests {
|
|||||||
.entry
|
.entry
|
||||||
.dependencies
|
.dependencies
|
||||||
.push("assets/shared.bundle".to_string());
|
.push("assets/shared.bundle".to_string());
|
||||||
|
resource.metadata.official_release_id = Some("release-1".to_string());
|
||||||
|
resource.metadata.platform = Some("windows".to_string());
|
||||||
|
resource.metadata.text_assets = vec!["Scenario".to_string()];
|
||||||
|
resource.metadata.text_unit_count = 3;
|
||||||
|
|
||||||
repository.add(resource.clone()).await.unwrap();
|
repository.add(resource.clone()).await.unwrap();
|
||||||
|
|
||||||
@@ -647,6 +688,13 @@ mod tests {
|
|||||||
by_id.entry.dependencies,
|
by_id.entry.dependencies,
|
||||||
vec!["assets/shared.bundle".to_string()]
|
vec!["assets/shared.bundle".to_string()]
|
||||||
);
|
);
|
||||||
|
assert_eq!(
|
||||||
|
by_id.metadata.official_release_id.as_deref(),
|
||||||
|
Some("release-1")
|
||||||
|
);
|
||||||
|
assert_eq!(by_id.metadata.platform.as_deref(), Some("windows"));
|
||||||
|
assert_eq!(by_id.metadata.text_assets, vec!["Scenario".to_string()]);
|
||||||
|
assert_eq!(by_id.metadata.text_unit_count, 3);
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
repository.find_by_hash("hash-sqlite-a").await.unwrap().id,
|
repository.find_by_hash("hash-sqlite-a").await.unwrap().id,
|
||||||
resource.id
|
resource.id
|
||||||
|
|||||||
@@ -367,14 +367,16 @@ fn official_update_reuses_failed_staging_after_interrupted_download() {
|
|||||||
let curl_failure_state = harness.temp.path().join("curl-failure.state");
|
let curl_failure_state = harness.temp.path().join("curl-failure.state");
|
||||||
write_executable(
|
write_executable(
|
||||||
&harness.curl_script,
|
&harness.curl_script,
|
||||||
&official_curl_script(
|
&official_curl_script(OfficialCurlScriptFixture {
|
||||||
&harness.curl_log,
|
curl_log: &harness.curl_log,
|
||||||
&zip_fixture,
|
zip_fixture: &zip_fixture,
|
||||||
&resources_assets_path,
|
resources_assets_fixture: &resources_assets_path,
|
||||||
fs::metadata(&resources_assets_path).unwrap().len() as usize,
|
resources_assets_size: fs::metadata(&resources_assets_path).unwrap().len() as usize,
|
||||||
TEST_LAUNCHER_MANIFEST_SOURCE,
|
manifest_source: TEST_LAUNCHER_MANIFEST_SOURCE,
|
||||||
Some(&curl_failure_state),
|
failure_state: Some(&curl_failure_state),
|
||||||
),
|
clientpatch_unavailable: false,
|
||||||
|
seed_unavailable_only: false,
|
||||||
|
}),
|
||||||
);
|
);
|
||||||
|
|
||||||
let config = harness.sync_config("failed-staging-output");
|
let config = harness.sync_config("failed-staging-output");
|
||||||
@@ -399,6 +401,77 @@ fn official_update_reuses_failed_staging_after_interrupted_download() {
|
|||||||
assert!(version_state.current_completed_version.is_some());
|
assert!(version_state.current_completed_version.is_some());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn official_update_waits_when_client_patch_markers_are_not_ready() {
|
||||||
|
let harness = TestHarness::new_with_clientpatch_unavailable(false);
|
||||||
|
let config = harness.sync_config("clientpatch-marker-wait-output");
|
||||||
|
let mut events = Vec::<OfficialUpdateProgress>::new();
|
||||||
|
|
||||||
|
let report = OfficialUpdateService::new()
|
||||||
|
.run_with_progress(&config, |event| events.push(event))
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
assert_eq!(
|
||||||
|
report.update_status,
|
||||||
|
OfficialUpdateStatus::WaitingForOfficialResources
|
||||||
|
);
|
||||||
|
assert!(report.waiting_for_official_resources);
|
||||||
|
assert!(!report.should_download);
|
||||||
|
assert!(report
|
||||||
|
.unavailable_endpoints
|
||||||
|
.iter()
|
||||||
|
.any(|endpoint| endpoint.url.ends_with("/TableBundles/TableCatalog.hash")));
|
||||||
|
assert!(report
|
||||||
|
.unavailable_endpoints
|
||||||
|
.iter()
|
||||||
|
.all(|endpoint| endpoint.error_kind == "http_forbidden"));
|
||||||
|
assert!(report.staging_path.is_none());
|
||||||
|
assert!(report.published_version_path.is_none());
|
||||||
|
assert!(!config.output_root.join("current").exists());
|
||||||
|
assert!(!config.output_root.join(".staging").exists());
|
||||||
|
assert!(read_version_state(&report.version_state_path)
|
||||||
|
.unwrap()
|
||||||
|
.is_none());
|
||||||
|
assert!(events
|
||||||
|
.iter()
|
||||||
|
.any(|event| event.stage == "upstream" && event.message.contains("尚未开放")));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn official_update_waits_when_required_seed_catalog_is_not_ready() {
|
||||||
|
let harness = TestHarness::new_with_clientpatch_unavailable(true);
|
||||||
|
let config = harness.sync_config("clientpatch-seed-wait-output");
|
||||||
|
let mut events = Vec::<OfficialUpdateProgress>::new();
|
||||||
|
|
||||||
|
let report = OfficialUpdateService::new()
|
||||||
|
.run_with_progress(&config, |event| events.push(event))
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
assert_eq!(
|
||||||
|
report.update_status,
|
||||||
|
OfficialUpdateStatus::WaitingForOfficialResources
|
||||||
|
);
|
||||||
|
assert!(report.waiting_for_official_resources);
|
||||||
|
assert_eq!(report.unavailable_endpoints.len(), 1);
|
||||||
|
assert!(report.unavailable_endpoints[0]
|
||||||
|
.url
|
||||||
|
.ends_with("/TableBundles/TableCatalog.bytes"));
|
||||||
|
assert_eq!(report.unavailable_endpoints[0].http_status, Some(403));
|
||||||
|
assert!(report.staging_path.is_none());
|
||||||
|
assert!(report.published_version_path.is_none());
|
||||||
|
assert!(!config.output_root.join("current").exists());
|
||||||
|
assert!(!config.output_root.join(".staging").exists());
|
||||||
|
assert!(read_version_state(&report.version_state_path)
|
||||||
|
.unwrap()
|
||||||
|
.is_none());
|
||||||
|
assert!(events
|
||||||
|
.iter()
|
||||||
|
.any(|event| event.stage == "catalog" && event.message.contains("TableCatalog.bytes")));
|
||||||
|
assert!(events
|
||||||
|
.iter()
|
||||||
|
.any(|event| event.stage == "upstream" && event.message.contains("必需资源")));
|
||||||
|
}
|
||||||
|
|
||||||
struct TestHarness {
|
struct TestHarness {
|
||||||
temp: TempDir,
|
temp: TempDir,
|
||||||
curl_script: std::path::PathBuf,
|
curl_script: std::path::PathBuf,
|
||||||
@@ -412,6 +485,18 @@ impl TestHarness {
|
|||||||
}
|
}
|
||||||
|
|
||||||
fn new_with_manifest_source(manifest_source: &str) -> Self {
|
fn new_with_manifest_source(manifest_source: &str) -> Self {
|
||||||
|
Self::new_with_options(manifest_source, false, false)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn new_with_clientpatch_unavailable(seed_only: bool) -> Self {
|
||||||
|
Self::new_with_options(TEST_LAUNCHER_MANIFEST_SOURCE, true, seed_only)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn new_with_options(
|
||||||
|
manifest_source: &str,
|
||||||
|
clientpatch_unavailable: bool,
|
||||||
|
seed_unavailable_only: bool,
|
||||||
|
) -> Self {
|
||||||
let temp = TempDir::new().unwrap();
|
let temp = TempDir::new().unwrap();
|
||||||
let resources_assets = synthetic_resources_assets();
|
let resources_assets = synthetic_resources_assets();
|
||||||
|
|
||||||
@@ -425,14 +510,16 @@ impl TestHarness {
|
|||||||
let curl_script = temp.path().join("fake-curl");
|
let curl_script = temp.path().join("fake-curl");
|
||||||
write_executable(
|
write_executable(
|
||||||
&curl_script,
|
&curl_script,
|
||||||
&official_curl_script(
|
&official_curl_script(OfficialCurlScriptFixture {
|
||||||
&curl_log,
|
curl_log: &curl_log,
|
||||||
&zip_fixture,
|
zip_fixture: &zip_fixture,
|
||||||
&resources_assets_path,
|
resources_assets_fixture: &resources_assets_path,
|
||||||
resources_assets.len(),
|
resources_assets_size: resources_assets.len(),
|
||||||
manifest_source,
|
manifest_source,
|
||||||
None,
|
failure_state: None,
|
||||||
),
|
clientpatch_unavailable,
|
||||||
|
seed_unavailable_only,
|
||||||
|
}),
|
||||||
);
|
);
|
||||||
|
|
||||||
let unzip_script = temp.path().join("fake-unzip");
|
let unzip_script = temp.path().join("fake-unzip");
|
||||||
@@ -542,15 +629,20 @@ fn one_file_zip(name: &[u8], data: &[u8]) -> Vec<u8> {
|
|||||||
bytes
|
bytes
|
||||||
}
|
}
|
||||||
|
|
||||||
fn official_curl_script(
|
struct OfficialCurlScriptFixture<'a> {
|
||||||
curl_log: &Path,
|
curl_log: &'a Path,
|
||||||
zip_fixture: &Path,
|
zip_fixture: &'a Path,
|
||||||
resources_assets_fixture: &Path,
|
resources_assets_fixture: &'a Path,
|
||||||
resources_assets_size: usize,
|
resources_assets_size: usize,
|
||||||
manifest_source: &str,
|
manifest_source: &'a str,
|
||||||
failure_state: Option<&Path>,
|
failure_state: Option<&'a Path>,
|
||||||
) -> String {
|
clientpatch_unavailable: bool,
|
||||||
let failure_state = failure_state
|
seed_unavailable_only: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
fn official_curl_script(fixture: OfficialCurlScriptFixture<'_>) -> String {
|
||||||
|
let failure_state = fixture
|
||||||
|
.failure_state
|
||||||
.map(shell_quote)
|
.map(shell_quote)
|
||||||
.unwrap_or_else(|| "''".to_string());
|
.unwrap_or_else(|| "''".to_string());
|
||||||
format!(
|
format!(
|
||||||
@@ -614,6 +706,31 @@ maybe_fail_once() {{
|
|||||||
fi
|
fi
|
||||||
}}
|
}}
|
||||||
|
|
||||||
|
clientpatch_is_unavailable() {{
|
||||||
|
if [[ "{clientpatch_unavailable}" != "true" ]]; then
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
if [[ "$url" != "{addressables_root}/"* ]]; then
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
if [[ "{seed_unavailable_only}" == "true" ]]; then
|
||||||
|
case "$url" in
|
||||||
|
*/TableBundles/TableCatalog.bytes|*/BundlePackingInfo.bytes|*/Catalog/MediaCatalog.bytes)
|
||||||
|
return 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
return 1
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
fi
|
||||||
|
return 0
|
||||||
|
}}
|
||||||
|
|
||||||
|
if clientpatch_is_unavailable; then
|
||||||
|
echo "curl: (22) The requested URL returned error: 403" >&2
|
||||||
|
exit 22
|
||||||
|
fi
|
||||||
|
|
||||||
if [[ "$url" == "https://api-launcher-jp.yo-star.com/api/launcher/game/config" ]]; then
|
if [[ "$url" == "https://api-launcher-jp.yo-star.com/api/launcher/game/config" ]]; then
|
||||||
cat <<'JSON'
|
cat <<'JSON'
|
||||||
{{"code":200,"message":"ok","data":{{"game_latest_version":"{launcher_latest_version}","game_latest_file_path":"{launcher_latest_file_path}"}}}}
|
{{"code":200,"message":"ok","data":{{"game_latest_version":"{launcher_latest_version}","game_latest_file_path":"{launcher_latest_file_path}"}}}}
|
||||||
@@ -679,16 +796,16 @@ else
|
|||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
"#,
|
"#,
|
||||||
shell_quote(curl_log),
|
shell_quote(fixture.curl_log),
|
||||||
shell_quote(zip_fixture),
|
shell_quote(fixture.zip_fixture),
|
||||||
shell_quote(resources_assets_fixture),
|
shell_quote(fixture.resources_assets_fixture),
|
||||||
launcher_latest_version = TEST_LAUNCHER_LATEST_VERSION,
|
launcher_latest_version = TEST_LAUNCHER_LATEST_VERSION,
|
||||||
launcher_latest_file_path = TEST_LAUNCHER_LATEST_FILE_PATH,
|
launcher_latest_file_path = TEST_LAUNCHER_LATEST_FILE_PATH,
|
||||||
launcher_game_config_json_url = TEST_LAUNCHER_GAME_CONFIG_JSON_URL,
|
launcher_game_config_json_url = TEST_LAUNCHER_GAME_CONFIG_JSON_URL,
|
||||||
launcher_manifest_url = TEST_LAUNCHER_MANIFEST_URL,
|
launcher_manifest_url = TEST_LAUNCHER_MANIFEST_URL,
|
||||||
launcher_manifest_source = manifest_source,
|
launcher_manifest_source = fixture.manifest_source,
|
||||||
launcher_resources_assets_url = TEST_LAUNCHER_RESOURCES_ASSETS_URL,
|
launcher_resources_assets_url = TEST_LAUNCHER_RESOURCES_ASSETS_URL,
|
||||||
resources_assets_size = resources_assets_size,
|
resources_assets_size = fixture.resources_assets_size,
|
||||||
server_info_url = TEST_SERVER_INFO_URL,
|
server_info_url = TEST_SERVER_INFO_URL,
|
||||||
connection_group = TEST_CONNECTION_GROUP,
|
connection_group = TEST_CONNECTION_GROUP,
|
||||||
addressables_root = TEST_ADDRESSABLES_ROOT,
|
addressables_root = TEST_ADDRESSABLES_ROOT,
|
||||||
@@ -698,6 +815,8 @@ fi
|
|||||||
android_bundle_catalog_hash = xxhash32(b"FullPatch_001.zip"),
|
android_bundle_catalog_hash = xxhash32(b"FullPatch_001.zip"),
|
||||||
android_media_catalog_hash = xxhash32(b"GameData\\Audio\\VOC_JP\\JP_Airi_Android.zip"),
|
android_media_catalog_hash = xxhash32(b"GameData\\Audio\\VOC_JP\\JP_Airi_Android.zip"),
|
||||||
failure_state = failure_state,
|
failure_state = failure_state,
|
||||||
|
clientpatch_unavailable = fixture.clientpatch_unavailable,
|
||||||
|
seed_unavailable_only = fixture.seed_unavailable_only,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,29 @@
|
|||||||
|
package api
|
||||||
|
|
||||||
|
import "net/http"
|
||||||
|
|
||||||
|
func (s *Server) handleAdminIndex(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != http.MethodGet && r.Method != http.MethodHead {
|
||||||
|
writeErrorJSON(w, http.StatusMethodNotAllowed, "method_not_allowed", "method not allowed")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
body := AdminIndexResponse{
|
||||||
|
Service: "bat-api",
|
||||||
|
Panel: "admin",
|
||||||
|
Status: "reserved",
|
||||||
|
Links: []string{
|
||||||
|
"/healthz",
|
||||||
|
"/readyz",
|
||||||
|
"/v1/bootstrap",
|
||||||
|
"/v1/release",
|
||||||
|
"/v1/resources",
|
||||||
|
"/openapi.yaml",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
if r.Method == http.MethodHead {
|
||||||
|
w.Header().Set("Cache-Control", "no-store")
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
writeNoStoreJSON(w, http.StatusOK, body)
|
||||||
|
}
|
||||||
@@ -0,0 +1,962 @@
|
|||||||
|
package api
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"log"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"sync/atomic"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"bat-api/internal/backendrpc"
|
||||||
|
)
|
||||||
|
|
||||||
|
func fixtureRoot(t *testing.T) string {
|
||||||
|
t.Helper()
|
||||||
|
root := filepath.Join("testdata", "release")
|
||||||
|
abs, err := filepath.Abs(root)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(abs, "official-download-manifest.json")); err != nil {
|
||||||
|
t.Fatalf("fixture missing: %v", err)
|
||||||
|
}
|
||||||
|
return abs
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoadIndexFromResourceRoot(t *testing.T) {
|
||||||
|
idx, err := LoadIndexFromResourceRoot(fixtureRoot(t))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(idx.Entries) != 2 {
|
||||||
|
t.Fatalf("entries = %d", len(idx.Entries))
|
||||||
|
}
|
||||||
|
sum := idx.Summary()
|
||||||
|
if !sum.Ready || sum.PresentCount != 2 {
|
||||||
|
t.Fatalf("summary = %+v", sum)
|
||||||
|
}
|
||||||
|
if sum.Snapshot == nil {
|
||||||
|
t.Fatal("snapshot missing")
|
||||||
|
}
|
||||||
|
if sum.Snapshot.LauncherMetadata == nil {
|
||||||
|
t.Fatal("launcher metadata missing")
|
||||||
|
}
|
||||||
|
if sum.Snapshot.LauncherMetadata.GameLatestVersion != "1.70.0" {
|
||||||
|
t.Fatalf("launcher game version=%q", sum.Snapshot.LauncherMetadata.GameLatestVersion)
|
||||||
|
}
|
||||||
|
if got := sum.Snapshot.LauncherMetadata.GameStartParams; len(got) != 1 || got[0] != "BlueArchive.exe" {
|
||||||
|
t.Fatalf("launcher params=%v", got)
|
||||||
|
}
|
||||||
|
if sum.Snapshot.GameMainConfig == nil || sum.Snapshot.GameMainConfig.DefaultConnectionGroup != "Prod" {
|
||||||
|
t.Fatalf("game main config=%+v", sum.Snapshot.GameMainConfig)
|
||||||
|
}
|
||||||
|
rel := "prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.bytes"
|
||||||
|
entry, ok := idx.Lookup(rel)
|
||||||
|
if !ok || !entry.Present {
|
||||||
|
t.Fatalf("lookup failed: %+v", entry)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSplitCDNPathRejectsEscape(t *testing.T) {
|
||||||
|
if _, _, err := SplitCDNPath("/prod-clientpatch.bluearchiveyostar.com/../etc/passwd"); err == nil {
|
||||||
|
t.Fatal("expected error")
|
||||||
|
}
|
||||||
|
if _, _, err := SplitCDNPath("/evil.example/a"); err == nil {
|
||||||
|
t.Fatal("expected unsupported host")
|
||||||
|
}
|
||||||
|
host, rel, err := SplitCDNPath("/prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.bytes")
|
||||||
|
if err != nil || host != ClientPatchHost {
|
||||||
|
t.Fatalf("host=%s rel=%s err=%v", host, rel, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCDNServesIndexedFile(t *testing.T) {
|
||||||
|
root := fixtureRoot(t)
|
||||||
|
idx, err := LoadIndexFromResourceRoot(root)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
cfg.ResourceRoot = root
|
||||||
|
cfg.PublicBaseURL = "http://127.0.0.1:18080"
|
||||||
|
_ = cfg.Normalize()
|
||||||
|
s := NewServer(cfg, nil, nil)
|
||||||
|
s.mu.Lock()
|
||||||
|
s.idx = idx
|
||||||
|
s.meta = DiscoverResult{ResourceRoot: root, Index: idx}
|
||||||
|
s.mu.Unlock()
|
||||||
|
|
||||||
|
req := httptest.NewRequest(http.MethodGet, "/prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.bytes", nil)
|
||||||
|
rr := httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, req)
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
if rr.Body.String() != "TABLE_CATALOG_FIXTURE" {
|
||||||
|
t.Fatalf("body=%q", rr.Body.String())
|
||||||
|
}
|
||||||
|
if rr.Header().Get("Accept-Ranges") != "bytes" {
|
||||||
|
t.Fatalf("Accept-Ranges=%q", rr.Header().Get("Accept-Ranges"))
|
||||||
|
}
|
||||||
|
if rr.Header().Get("ETag") == "" {
|
||||||
|
t.Fatal("ETag missing")
|
||||||
|
}
|
||||||
|
if rr.Header().Get("Last-Modified") == "" {
|
||||||
|
t.Fatal("Last-Modified missing")
|
||||||
|
}
|
||||||
|
if rr.Header().Get("Cache-Control") != "public, max-age=31536000, immutable" {
|
||||||
|
t.Fatalf("Cache-Control=%q", rr.Header().Get("Cache-Control"))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Unknown path
|
||||||
|
req = httptest.NewRequest(http.MethodGet, "/prod-clientpatch.bluearchiveyostar.com/r93_fixture/missing.bin", nil)
|
||||||
|
rr = httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, req)
|
||||||
|
if rr.Code != http.StatusNotFound {
|
||||||
|
t.Fatalf("missing status=%d", rr.Code)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCDNSupportsRangeHeadAndConditionalRequests(t *testing.T) {
|
||||||
|
root := fixtureRoot(t)
|
||||||
|
idx, err := LoadIndexFromResourceRoot(root)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
cfg.ResourceRoot = root
|
||||||
|
_ = cfg.Normalize()
|
||||||
|
s := NewServer(cfg, nil, nil)
|
||||||
|
s.mu.Lock()
|
||||||
|
s.idx = idx
|
||||||
|
s.meta = DiscoverResult{ResourceRoot: root, Index: idx}
|
||||||
|
s.mu.Unlock()
|
||||||
|
|
||||||
|
path := "/prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.bytes"
|
||||||
|
req := httptest.NewRequest(http.MethodGet, path, nil)
|
||||||
|
req.Header.Set("Range", "bytes=0-4")
|
||||||
|
rr := httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, req)
|
||||||
|
if rr.Code != http.StatusPartialContent {
|
||||||
|
t.Fatalf("range status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
if rr.Body.String() != "TABLE" {
|
||||||
|
t.Fatalf("range body=%q", rr.Body.String())
|
||||||
|
}
|
||||||
|
if rr.Header().Get("Content-Range") != "bytes 0-4/21" {
|
||||||
|
t.Fatalf("Content-Range=%q", rr.Header().Get("Content-Range"))
|
||||||
|
}
|
||||||
|
etag := rr.Header().Get("ETag")
|
||||||
|
if etag != `"blake3-0000000000000000000000000000000000000000000000000000000000000000"` {
|
||||||
|
t.Fatalf("ETag=%q", etag)
|
||||||
|
}
|
||||||
|
|
||||||
|
req = httptest.NewRequest(http.MethodHead, path, nil)
|
||||||
|
rr = httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, req)
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("head status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
if rr.Body.Len() != 0 {
|
||||||
|
t.Fatalf("head body=%q", rr.Body.String())
|
||||||
|
}
|
||||||
|
if rr.Header().Get("Content-Length") != "21" {
|
||||||
|
t.Fatalf("head Content-Length=%q", rr.Header().Get("Content-Length"))
|
||||||
|
}
|
||||||
|
|
||||||
|
req = httptest.NewRequest(http.MethodGet, path, nil)
|
||||||
|
req.Header.Set("If-None-Match", etag)
|
||||||
|
rr = httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, req)
|
||||||
|
if rr.Code != http.StatusNotModified {
|
||||||
|
t.Fatalf("conditional status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
req = httptest.NewRequest(http.MethodGet, path, nil)
|
||||||
|
req.Header.Set("Range", "bytes=99-120")
|
||||||
|
rr = httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, req)
|
||||||
|
if rr.Code != http.StatusRequestedRangeNotSatisfiable {
|
||||||
|
t.Fatalf("invalid range status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCDNHashContentType(t *testing.T) {
|
||||||
|
root := fixtureRoot(t)
|
||||||
|
idx, err := LoadIndexFromResourceRoot(root)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
cfg.ResourceRoot = root
|
||||||
|
_ = cfg.Normalize()
|
||||||
|
s := NewServer(cfg, nil, nil)
|
||||||
|
s.mu.Lock()
|
||||||
|
s.idx = idx
|
||||||
|
s.meta = DiscoverResult{ResourceRoot: root, Index: idx}
|
||||||
|
s.mu.Unlock()
|
||||||
|
|
||||||
|
req := httptest.NewRequest(http.MethodGet, "/prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.hash", nil)
|
||||||
|
rr := httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, req)
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
if rr.Header().Get("Content-Type") != "text/plain; charset=utf-8" {
|
||||||
|
t.Fatalf("Content-Type=%q", rr.Header().Get("Content-Type"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestHealthzAndRelease(t *testing.T) {
|
||||||
|
root := fixtureRoot(t)
|
||||||
|
idx, err := LoadIndexFromResourceRoot(root)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
cfg.ResourceRoot = root
|
||||||
|
_ = cfg.Normalize()
|
||||||
|
s := NewServer(cfg, nil, nil)
|
||||||
|
s.mu.Lock()
|
||||||
|
s.idx = idx
|
||||||
|
s.meta = DiscoverResult{Index: idx, ResourceRoot: root}
|
||||||
|
s.mu.Unlock()
|
||||||
|
|
||||||
|
rr := httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/healthz", nil))
|
||||||
|
if rr.Code != 200 {
|
||||||
|
t.Fatal(rr.Body.String())
|
||||||
|
}
|
||||||
|
var health map[string]any
|
||||||
|
if err := json.Unmarshal(rr.Body.Bytes(), &health); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if health["ready"] != true {
|
||||||
|
t.Fatalf("health=%v", health)
|
||||||
|
}
|
||||||
|
|
||||||
|
rr = httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/readyz", nil))
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("readyz status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
rr = httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/v1/resources?limit=1", nil))
|
||||||
|
if rr.Code != 200 {
|
||||||
|
t.Fatal(rr.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBootstrapDescribesBatRelationship(t *testing.T) {
|
||||||
|
root := fixtureRoot(t)
|
||||||
|
idx, err := LoadIndexFromResourceRoot(root)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
cfg.ResourceRoot = root
|
||||||
|
cfg.PublicBaseURL = "http://127.0.0.1:18080"
|
||||||
|
cfg.SocketPath = "/tmp/bat-pid/bat.sock"
|
||||||
|
_ = cfg.Normalize()
|
||||||
|
healthy := true
|
||||||
|
s := NewServer(cfg, nil, nil)
|
||||||
|
s.mu.Lock()
|
||||||
|
s.idx = idx
|
||||||
|
s.meta = DiscoverResult{
|
||||||
|
Index: idx,
|
||||||
|
ResourceRoot: root,
|
||||||
|
RPCAvailable: true,
|
||||||
|
DoctorHealthy: &healthy,
|
||||||
|
}
|
||||||
|
s.mu.Unlock()
|
||||||
|
|
||||||
|
rr := httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/v1/bootstrap", nil))
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
var body map[string]any
|
||||||
|
if err := json.Unmarshal(rr.Body.Bytes(), &body); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if body["ready"] != true {
|
||||||
|
t.Fatalf("ready=%v", body["ready"])
|
||||||
|
}
|
||||||
|
bat := body["bat"].(map[string]any)
|
||||||
|
if bat["role"] != "sync_daemon_and_release_producer" {
|
||||||
|
t.Fatalf("bat role=%v", bat["role"])
|
||||||
|
}
|
||||||
|
resource := body["resource"].(map[string]any)
|
||||||
|
wantRoot := "http://127.0.0.1:18080/prod-clientpatch.bluearchiveyostar.com/r93_fixture"
|
||||||
|
if resource["addressables_catalog_url_root"] != wantRoot {
|
||||||
|
t.Fatalf("addressables root=%v", resource["addressables_catalog_url_root"])
|
||||||
|
}
|
||||||
|
if resource["server_info_url"] != "http://127.0.0.1:18080/yostar-serverinfo.bluearchiveyostar.com/server-info.json" {
|
||||||
|
t.Fatalf("server info url=%v", resource["server_info_url"])
|
||||||
|
}
|
||||||
|
policy := body["policy"].(map[string]any)
|
||||||
|
if policy["pull_owner"] != "rust_bat" || policy["writes_release_state"] != false {
|
||||||
|
t.Fatalf("policy=%v", policy)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLauncherBootstrapDescribesResourceOnlyScope(t *testing.T) {
|
||||||
|
root := fixtureRoot(t)
|
||||||
|
idx, err := LoadIndexFromResourceRoot(root)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
cfg.ResourceRoot = root
|
||||||
|
cfg.PublicBaseURL = "http://127.0.0.1:18080"
|
||||||
|
_ = cfg.Normalize()
|
||||||
|
s := NewServer(cfg, nil, nil)
|
||||||
|
s.mu.Lock()
|
||||||
|
s.idx = idx
|
||||||
|
s.meta = DiscoverResult{Index: idx, ResourceRoot: root}
|
||||||
|
s.mu.Unlock()
|
||||||
|
|
||||||
|
rr := httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/v1/launcher/bootstrap", nil))
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
var body map[string]any
|
||||||
|
if err := json.Unmarshal(rr.Body.Bytes(), &body); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if body["ready"] != true {
|
||||||
|
t.Fatalf("ready=%v", body["ready"])
|
||||||
|
}
|
||||||
|
policy := body["policy"].(map[string]any)
|
||||||
|
if policy["source"] != "rust_bat_snapshot" || policy["emulates_login_or_gateway"] != false {
|
||||||
|
t.Fatalf("policy=%v", policy)
|
||||||
|
}
|
||||||
|
if policy["downloads_launcher_package"] != false || policy["package_update_manifest"] != false {
|
||||||
|
t.Fatalf("policy=%v", policy)
|
||||||
|
}
|
||||||
|
metadata := body["launcher_metadata"].(map[string]any)
|
||||||
|
if metadata["game_latest_file_path"] != "prod/ZIP_TEMP/BlueArchive_JP_TEMP/BlueArchive_JP-1.70.436321-game.zip" {
|
||||||
|
t.Fatalf("launcher metadata=%v", metadata)
|
||||||
|
}
|
||||||
|
gameMainConfig := body["game_main_config"].(map[string]any)
|
||||||
|
if gameMainConfig["server_info_data_url"] != "https://yostar-serverinfo.bluearchiveyostar.com/r93_fixture.json" {
|
||||||
|
t.Fatalf("game main config=%v", gameMainConfig)
|
||||||
|
}
|
||||||
|
if body["addressables_catalog_url_root"] != "http://127.0.0.1:18080/prod-clientpatch.bluearchiveyostar.com/r93_fixture" {
|
||||||
|
t.Fatalf("addressables root=%v", body["addressables_catalog_url_root"])
|
||||||
|
}
|
||||||
|
|
||||||
|
rr = httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, launcherHostPath("/api/launcher/resource/bootstrap.json"), nil))
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("host-shaped bootstrap status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLauncherCompatibilityEndpoints(t *testing.T) {
|
||||||
|
root := fixtureRoot(t)
|
||||||
|
idx, err := LoadIndexFromResourceRoot(root)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
cfg.ResourceRoot = root
|
||||||
|
cfg.PublicBaseURL = "http://127.0.0.1:18080"
|
||||||
|
_ = cfg.Normalize()
|
||||||
|
s := NewServer(cfg, nil, nil)
|
||||||
|
s.mu.Lock()
|
||||||
|
s.idx = idx
|
||||||
|
s.meta = DiscoverResult{Index: idx, ResourceRoot: root}
|
||||||
|
s.mu.Unlock()
|
||||||
|
|
||||||
|
rr := httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, launcherHostPath("/api/launcher/game/config"), nil))
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("game config status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
var envelope map[string]any
|
||||||
|
if err := json.Unmarshal(rr.Body.Bytes(), &envelope); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if envelope["code"] != float64(200) {
|
||||||
|
t.Fatalf("envelope=%v", envelope)
|
||||||
|
}
|
||||||
|
data := envelope["data"].(map[string]any)
|
||||||
|
if data["game_latest_version"] != "1.70.0" {
|
||||||
|
t.Fatalf("game config=%v", data)
|
||||||
|
}
|
||||||
|
if data["game_latest_file_path"] != "prod/ZIP_TEMP/BlueArchive_JP_TEMP/BlueArchive_JP-1.70.436321-game.zip" {
|
||||||
|
t.Fatalf("game config=%v", data)
|
||||||
|
}
|
||||||
|
if data["resource_bootstrap_url"] != "http://127.0.0.1:18080/v1/launcher/bootstrap" {
|
||||||
|
t.Fatalf("game config=%v", data)
|
||||||
|
}
|
||||||
|
if data["server_info_url"] != "http://127.0.0.1:18080/yostar-serverinfo.bluearchiveyostar.com/server-info.json" {
|
||||||
|
t.Fatalf("game config=%v", data)
|
||||||
|
}
|
||||||
|
if params := data["game_start_params"].([]any); len(params) != 1 || params[0] != "BlueArchive.exe" {
|
||||||
|
t.Fatalf("params=%v", data["game_start_params"])
|
||||||
|
}
|
||||||
|
|
||||||
|
rr = httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/api/launcher/game/config", nil))
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("bare game config status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
rr = httptest.NewRecorder()
|
||||||
|
req := httptest.NewRequest(
|
||||||
|
http.MethodGet,
|
||||||
|
launcherHostPath("/api/launcher/game/config/json")+"?version=1.70.0&file_path=prod%2FZIP_TEMP%2FBlueArchive_JP_TEMP%2FBlueArchive_JP-1.70.436321-game.zip",
|
||||||
|
nil,
|
||||||
|
)
|
||||||
|
s.Handler().ServeHTTP(rr, req)
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("game config json status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
envelope = map[string]any{}
|
||||||
|
if err := json.Unmarshal(rr.Body.Bytes(), &envelope); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
data = envelope["data"].(map[string]any)
|
||||||
|
if data["package_update_manifest"] != false || data["scope"] != "resource_bootstrap_only" {
|
||||||
|
t.Fatalf("game config json=%v", data)
|
||||||
|
}
|
||||||
|
url, _ := data["url"].(string)
|
||||||
|
if !strings.HasPrefix(url, "http://127.0.0.1:18080/api-launcher-jp.yo-star.com/api/launcher/resource/bootstrap.json?") {
|
||||||
|
t.Fatalf("bootstrap url=%q", url)
|
||||||
|
}
|
||||||
|
if !strings.Contains(url, "version=1.70.0") || !strings.Contains(url, "file_path=prod%2FZIP_TEMP%2FBlueArchive_JP_TEMP%2FBlueArchive_JP-1.70.436321-game.zip") {
|
||||||
|
t.Fatalf("bootstrap url=%q", url)
|
||||||
|
}
|
||||||
|
|
||||||
|
rr = httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, launcherHostPath("/api/launcher/advanced/game/download/cdn"), nil))
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("cdn config status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
envelope = map[string]any{}
|
||||||
|
if err := json.Unmarshal(rr.Body.Bytes(), &envelope); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
data = envelope["data"].(map[string]any)
|
||||||
|
if data["primary_cdn"] != "http://127.0.0.1:18080" || data["back_up_cdn"] != "http://127.0.0.1:18080" {
|
||||||
|
t.Fatalf("cdn config=%v", data)
|
||||||
|
}
|
||||||
|
if data["package_update_manifest"] != false {
|
||||||
|
t.Fatalf("cdn config=%v", data)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBootstrapNotReadyDoesNotClaimReleaseOwnership(t *testing.T) {
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
cfg.PublicBaseURL = "http://127.0.0.1:18080"
|
||||||
|
cfg.SocketPath = "/tmp/bat-pid/bat.sock"
|
||||||
|
_ = cfg.Normalize()
|
||||||
|
s := NewServer(cfg, nil, nil)
|
||||||
|
|
||||||
|
rr := httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/v1/bootstrap", nil))
|
||||||
|
if rr.Code != http.StatusServiceUnavailable {
|
||||||
|
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
var body map[string]any
|
||||||
|
if err := json.Unmarshal(rr.Body.Bytes(), &body); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if body["ready"] != false {
|
||||||
|
t.Fatalf("ready=%v", body["ready"])
|
||||||
|
}
|
||||||
|
policy := body["policy"].(map[string]any)
|
||||||
|
if policy["pull_owner"] != "rust_bat" {
|
||||||
|
t.Fatalf("pull owner=%v", policy["pull_owner"])
|
||||||
|
}
|
||||||
|
if policy["serves_only_published_release"] != true || policy["writes_release_state"] != false {
|
||||||
|
t.Fatalf("policy=%v", policy)
|
||||||
|
}
|
||||||
|
resource := body["resource"].(map[string]any)
|
||||||
|
if resource["entry_count"] != float64(0) || resource["present_count"] != float64(0) {
|
||||||
|
t.Fatalf("resource=%v", resource)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestServerInfoRewritesAddressablesOnly(t *testing.T) {
|
||||||
|
raw := []byte(`{
|
||||||
|
"ConnectionGroups":[{
|
||||||
|
"Name":"Prod",
|
||||||
|
"AddressablesCatalogUrlRoot":"https://prod-clientpatch.bluearchiveyostar.com/r93_fixture",
|
||||||
|
"ApiUrl":"https://prod-game.example.invalid/"
|
||||||
|
}]
|
||||||
|
}`)
|
||||||
|
out, err := RewriteServerInfoAddressables(raw, "http://127.0.0.1:18080")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var doc map[string]any
|
||||||
|
if err := json.Unmarshal(out, &doc); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
group := doc["ConnectionGroups"].([]any)[0].(map[string]any)
|
||||||
|
got := group["AddressablesCatalogUrlRoot"].(string)
|
||||||
|
want := "http://127.0.0.1:18080/prod-clientpatch.bluearchiveyostar.com/r93_fixture"
|
||||||
|
if got != want {
|
||||||
|
t.Fatalf("root=%s want=%s", got, want)
|
||||||
|
}
|
||||||
|
if group["ApiUrl"] != "https://prod-game.example.invalid/" {
|
||||||
|
t.Fatalf("ApiUrl should be unchanged: %v", group["ApiUrl"])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type fakeBackend struct {
|
||||||
|
statusCalls int
|
||||||
|
doctorCalls int
|
||||||
|
status *backendrpc.DaemonStatusReport
|
||||||
|
doctor *backendrpc.DoctorReport
|
||||||
|
catalog json.RawMessage
|
||||||
|
manifest *backendrpc.ResourceManifestPage
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeBackend) DaemonStatus(ctx context.Context) (*backendrpc.DaemonStatusReport, error) {
|
||||||
|
f.statusCalls++
|
||||||
|
return f.status, nil
|
||||||
|
}
|
||||||
|
func (f *fakeBackend) DaemonDoctor(ctx context.Context) (*backendrpc.DoctorReport, error) {
|
||||||
|
f.doctorCalls++
|
||||||
|
if f.statusCalls == 0 {
|
||||||
|
// status must be called first in real DiscoverAndIndex; this is asserted by call order.
|
||||||
|
}
|
||||||
|
return f.doctor, nil
|
||||||
|
}
|
||||||
|
func (f *fakeBackend) ResourceState(ctx context.Context) (*backendrpc.ResourceState, error) {
|
||||||
|
return &backendrpc.ResourceState{}, nil
|
||||||
|
}
|
||||||
|
func (f *fakeBackend) CatalogStatus(ctx context.Context) (json.RawMessage, error) {
|
||||||
|
return f.catalog, nil
|
||||||
|
}
|
||||||
|
func (f *fakeBackend) ResourceManifest(ctx context.Context, offset int, limit int) (*backendrpc.ResourceManifestPage, error) {
|
||||||
|
return f.manifest, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDiscoverCallsStatusBeforeDoctor(t *testing.T) {
|
||||||
|
root := fixtureRoot(t)
|
||||||
|
bytes := uint64(20)
|
||||||
|
info, err := os.Stat(filepath.Join(root, "prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.bytes"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
b := uint64(info.Size())
|
||||||
|
_ = bytes
|
||||||
|
catalogObj := map[string]any{
|
||||||
|
"available": true,
|
||||||
|
"app_version": "1.70.0",
|
||||||
|
"bundle_version": "s8tloc7lo3",
|
||||||
|
"connection_group_name": "Prod",
|
||||||
|
"addressables_root": "https://prod-clientpatch.bluearchiveyostar.com/r93_fixture",
|
||||||
|
"version": map[string]any{
|
||||||
|
"id": "v1",
|
||||||
|
"resource_root": root,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
catalogRaw, err := json.Marshal(catalogObj)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
fb := &fakeBackend{
|
||||||
|
status: &backendrpc.DaemonStatusReport{Status: "ok", Running: true, RPCAvailable: true},
|
||||||
|
doctor: &backendrpc.DoctorReport{Healthy: true, Status: "ok"},
|
||||||
|
catalog: catalogRaw,
|
||||||
|
manifest: &backendrpc.ResourceManifestPage{
|
||||||
|
Available: true,
|
||||||
|
ResourceRoot: root,
|
||||||
|
ManifestVersion: 1,
|
||||||
|
TotalEntries: 1,
|
||||||
|
Entries: []backendrpc.ResourceManifestEntry{{
|
||||||
|
URL: "https://prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.bytes",
|
||||||
|
Destination: "prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.bytes",
|
||||||
|
Bytes: &b,
|
||||||
|
BLAKE3: strings.Repeat("0", 64),
|
||||||
|
}},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
result, err := DiscoverAndIndex(context.Background(), fb, "")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if fb.statusCalls != 1 || fb.doctorCalls != 1 {
|
||||||
|
t.Fatalf("status=%d doctor=%d", fb.statusCalls, fb.doctorCalls)
|
||||||
|
}
|
||||||
|
if !result.RPCAvailable || result.Index == nil || !result.Index.Summary().Ready {
|
||||||
|
t.Fatalf("result=%+v summary=%+v", result, result.Index.Summary())
|
||||||
|
}
|
||||||
|
if result.DoctorHealthy == nil || !*result.DoctorHealthy {
|
||||||
|
t.Fatal("doctor healthy expected")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoadEnvFileDoesNotOverride(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
path := filepath.Join(dir, ".env")
|
||||||
|
if err := os.WriteFile(path, []byte("BAT_API_LISTEN=:9999\n"), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Setenv("BAT_API_LISTEN", ":1111")
|
||||||
|
if err := LoadEnvFile(path); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if os.Getenv("BAT_API_LISTEN") != ":1111" {
|
||||||
|
t.Fatal(os.Getenv("BAT_API_LISTEN"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRefreshDiagnosticsAndReadyz(t *testing.T) {
|
||||||
|
root := fixtureRoot(t)
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
cfg.ResourceRoot = root
|
||||||
|
cfg.RefreshInterval = 0
|
||||||
|
if err := cfg.Normalize(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s := NewServer(cfg, nil, log.New(io.Discard, "", 0))
|
||||||
|
if err := s.Refresh(context.Background()); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rr := httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/healthz", nil))
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("healthz status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
var health map[string]any
|
||||||
|
if err := json.Unmarshal(rr.Body.Bytes(), &health); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
refresh := health["refresh"].(map[string]any)
|
||||||
|
if refresh["last_success_unix_seconds"] == nil || refresh["last_error"] != "" {
|
||||||
|
t.Fatalf("refresh=%v", refresh)
|
||||||
|
}
|
||||||
|
|
||||||
|
rr = httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/readyz", nil))
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("readyz status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRefreshFailureDiagnosticsAndReadyz(t *testing.T) {
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
cfg.ResourceRoot = filepath.Join(t.TempDir(), "missing-release")
|
||||||
|
cfg.RefreshInterval = 0
|
||||||
|
if err := cfg.Normalize(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s := NewServer(cfg, nil, log.New(io.Discard, "", 0))
|
||||||
|
if err := s.Refresh(context.Background()); err == nil {
|
||||||
|
t.Fatal("expected refresh failure")
|
||||||
|
}
|
||||||
|
|
||||||
|
rr := httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/healthz", nil))
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("healthz status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
var health map[string]any
|
||||||
|
if err := json.Unmarshal(rr.Body.Bytes(), &health); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
refresh := health["refresh"].(map[string]any)
|
||||||
|
if refresh["last_error"] == "" || refresh["last_finished_unix_seconds"] == nil {
|
||||||
|
t.Fatalf("refresh=%v", refresh)
|
||||||
|
}
|
||||||
|
|
||||||
|
rr = httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/readyz", nil))
|
||||||
|
if rr.Code != http.StatusServiceUnavailable {
|
||||||
|
t.Fatalf("readyz status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestApplyEnvParsesRefreshInterval(t *testing.T) {
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
t.Setenv("BAT_API_REFRESH_INTERVAL", "2m")
|
||||||
|
ApplyEnv(&cfg)
|
||||||
|
if cfg.RefreshInterval != 2*time.Minute {
|
||||||
|
t.Fatalf("refresh interval=%s", cfg.RefreshInterval)
|
||||||
|
}
|
||||||
|
cfg.RefreshInterval = -time.Second
|
||||||
|
if err := cfg.Normalize(); err == nil {
|
||||||
|
t.Fatal("expected negative refresh interval error")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type pollingBackend struct {
|
||||||
|
statusCalls atomic.Int64
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *pollingBackend) DaemonStatus(ctx context.Context) (*backendrpc.DaemonStatusReport, error) {
|
||||||
|
p.statusCalls.Add(1)
|
||||||
|
return nil, errors.New("offline")
|
||||||
|
}
|
||||||
|
func (p *pollingBackend) DaemonDoctor(ctx context.Context) (*backendrpc.DoctorReport, error) {
|
||||||
|
return nil, errors.New("unexpected doctor call")
|
||||||
|
}
|
||||||
|
func (p *pollingBackend) ResourceState(ctx context.Context) (*backendrpc.ResourceState, error) {
|
||||||
|
return nil, errors.New("unexpected resource state call")
|
||||||
|
}
|
||||||
|
func (p *pollingBackend) CatalogStatus(ctx context.Context) (json.RawMessage, error) {
|
||||||
|
return nil, errors.New("unexpected catalog call")
|
||||||
|
}
|
||||||
|
func (p *pollingBackend) ResourceManifest(ctx context.Context, offset int, limit int) (*backendrpc.ResourceManifestPage, error) {
|
||||||
|
return nil, errors.New("unexpected manifest call")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestStartRefreshLoopPollsBackend(t *testing.T) {
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
cfg.RefreshInterval = 10 * time.Millisecond
|
||||||
|
cfg.RPCTimeout = 20 * time.Millisecond
|
||||||
|
backend := &pollingBackend{}
|
||||||
|
s := NewServer(cfg, backend, log.New(io.Discard, "", 0))
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
s.StartRefreshLoop(ctx)
|
||||||
|
deadline := time.Now().Add(200 * time.Millisecond)
|
||||||
|
for time.Now().Before(deadline) {
|
||||||
|
if backend.statusCalls.Load() > 0 {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
time.Sleep(5 * time.Millisecond)
|
||||||
|
}
|
||||||
|
t.Fatal("refresh loop did not poll backend")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRefreshWarningsRecordedWhenRPCUnavailable(t *testing.T) {
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
cfg.RefreshInterval = 0
|
||||||
|
backend := &pollingBackend{}
|
||||||
|
s := NewServer(cfg, backend, log.New(io.Discard, "", 0))
|
||||||
|
if err := s.Refresh(context.Background()); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rr := httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/healthz", nil))
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
var health map[string]any
|
||||||
|
if err := json.Unmarshal(rr.Body.Bytes(), &health); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
refresh := health["refresh"].(map[string]any)
|
||||||
|
if refresh["last_error"] != "" || refresh["last_warning_count"] == float64(0) {
|
||||||
|
t.Fatalf("refresh=%v", refresh)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestHTTPAuthRequiresTokenAndAllowsExemptPaths(t *testing.T) {
|
||||||
|
root := fixtureRoot(t)
|
||||||
|
idx, err := LoadIndexFromResourceRoot(root)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
cfg.ResourceRoot = root
|
||||||
|
cfg.AuthToken = "secret-token"
|
||||||
|
cfg.AuthExemptPaths = []string{"/healthz"}
|
||||||
|
if err := cfg.Normalize(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s := NewServer(cfg, nil, nil)
|
||||||
|
s.mu.Lock()
|
||||||
|
s.idx = idx
|
||||||
|
s.meta = DiscoverResult{Index: idx, ResourceRoot: root}
|
||||||
|
s.mu.Unlock()
|
||||||
|
|
||||||
|
rr := httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/v1/bootstrap", nil))
|
||||||
|
if rr.Code != http.StatusUnauthorized {
|
||||||
|
t.Fatalf("missing token status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
if rr.Header().Get("Cache-Control") != "no-store" {
|
||||||
|
t.Fatalf("unauthorized Cache-Control=%q", rr.Header().Get("Cache-Control"))
|
||||||
|
}
|
||||||
|
var errBody ErrorResponse
|
||||||
|
if err := json.Unmarshal(rr.Body.Bytes(), &errBody); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if errBody.Error.Code != "unauthorized" {
|
||||||
|
t.Fatalf("error=%+v", errBody)
|
||||||
|
}
|
||||||
|
|
||||||
|
req := httptest.NewRequest(http.MethodGet, "/v1/bootstrap", nil)
|
||||||
|
req.Header.Set("Authorization", "Bearer secret-token")
|
||||||
|
rr = httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, req)
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("authorized status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
rr = httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/healthz", nil))
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("exempt healthz status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRateLimitUsesForwardedClientWhenTrusted(t *testing.T) {
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
cfg.RateLimitRPS = 1
|
||||||
|
cfg.RateLimitBurst = 1
|
||||||
|
cfg.TrustProxyHeaders = true
|
||||||
|
if err := cfg.Normalize(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s := NewServer(cfg, nil, log.New(io.Discard, "", 0))
|
||||||
|
handler := s.Handler()
|
||||||
|
|
||||||
|
req := httptest.NewRequest(http.MethodGet, "/healthz", nil)
|
||||||
|
req.RemoteAddr = "10.0.0.10:1234"
|
||||||
|
req.Header.Set("X-Forwarded-For", "203.0.113.7, 10.0.0.10")
|
||||||
|
rr := httptest.NewRecorder()
|
||||||
|
handler.ServeHTTP(rr, req)
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("first status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
req = httptest.NewRequest(http.MethodGet, "/healthz", nil)
|
||||||
|
req.RemoteAddr = "10.0.0.10:1234"
|
||||||
|
req.Header.Set("X-Forwarded-For", "203.0.113.7, 10.0.0.10")
|
||||||
|
rr = httptest.NewRecorder()
|
||||||
|
handler.ServeHTTP(rr, req)
|
||||||
|
if rr.Code != http.StatusTooManyRequests {
|
||||||
|
t.Fatalf("second status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
var errBody ErrorResponse
|
||||||
|
if err := json.Unmarshal(rr.Body.Bytes(), &errBody); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if errBody.Error.Code != "rate_limited" {
|
||||||
|
t.Fatalf("error=%+v", errBody)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourcesLimitIsCappedAndDynamicResponsesNoStore(t *testing.T) {
|
||||||
|
root := fixtureRoot(t)
|
||||||
|
idx, err := LoadIndexFromResourceRoot(root)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
cfg.ResourceRoot = root
|
||||||
|
cfg.MaxResourcePageLimit = 1
|
||||||
|
cfg.PublicBaseURL = "http://127.0.0.1:18080"
|
||||||
|
if err := cfg.Normalize(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s := NewServer(cfg, nil, nil)
|
||||||
|
s.mu.Lock()
|
||||||
|
s.idx = idx
|
||||||
|
s.meta = DiscoverResult{Index: idx, ResourceRoot: root}
|
||||||
|
s.mu.Unlock()
|
||||||
|
|
||||||
|
rr := httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/v1/resources?limit=999", nil))
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("resources status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
if rr.Header().Get("Cache-Control") != "no-store" {
|
||||||
|
t.Fatalf("resources Cache-Control=%q", rr.Header().Get("Cache-Control"))
|
||||||
|
}
|
||||||
|
var page ResourceListResponse
|
||||||
|
if err := json.Unmarshal(rr.Body.Bytes(), &page); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if page.Limit != 1 || len(page.Items) != 1 {
|
||||||
|
t.Fatalf("page=%+v", page)
|
||||||
|
}
|
||||||
|
|
||||||
|
rr = httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/v1/bootstrap", nil))
|
||||||
|
if rr.Header().Get("Cache-Control") != "no-store" {
|
||||||
|
t.Fatalf("bootstrap Cache-Control=%q", rr.Header().Get("Cache-Control"))
|
||||||
|
}
|
||||||
|
|
||||||
|
rr = httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/v1/launcher/bootstrap", nil))
|
||||||
|
if rr.Header().Get("Cache-Control") != "no-store" {
|
||||||
|
t.Fatalf("launcher bootstrap Cache-Control=%q", rr.Header().Get("Cache-Control"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAccessLogOmitsQueryString(t *testing.T) {
|
||||||
|
var logs bytes.Buffer
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
cfg.AccessLog = true
|
||||||
|
if err := cfg.Normalize(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s := NewServer(cfg, nil, log.New(&logs, "", 0))
|
||||||
|
|
||||||
|
req := httptest.NewRequest(http.MethodGet, "/healthz?bat_token=secret", nil)
|
||||||
|
req.Header.Set("User-Agent", "bat-test")
|
||||||
|
rr := httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, req)
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
got := logs.String()
|
||||||
|
if !strings.Contains(got, "path=/healthz") {
|
||||||
|
t.Fatalf("log=%q", got)
|
||||||
|
}
|
||||||
|
if strings.Contains(got, "secret") || strings.Contains(got, "bat_token") {
|
||||||
|
t.Fatalf("log leaked query token: %q", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestOpenAPIAndAdminReservedEndpoints(t *testing.T) {
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
if err := cfg.Normalize(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s := NewServer(cfg, nil, nil)
|
||||||
|
|
||||||
|
rr := httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/openapi.yaml", nil))
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("openapi status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
if rr.Header().Get("Cache-Control") != "no-store" {
|
||||||
|
t.Fatalf("openapi Cache-Control=%q", rr.Header().Get("Cache-Control"))
|
||||||
|
}
|
||||||
|
if rr.Header().Get("X-Content-Type-Options") != "nosniff" {
|
||||||
|
t.Fatalf("X-Content-Type-Options=%q", rr.Header().Get("X-Content-Type-Options"))
|
||||||
|
}
|
||||||
|
if !strings.Contains(rr.Body.String(), "/v1/launcher/bootstrap") {
|
||||||
|
t.Fatalf("openapi missing launcher bootstrap")
|
||||||
|
}
|
||||||
|
|
||||||
|
rr = httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/admin/", nil))
|
||||||
|
if rr.Code != http.StatusOK {
|
||||||
|
t.Fatalf("admin status=%d body=%s", rr.Code, rr.Body.String())
|
||||||
|
}
|
||||||
|
var admin AdminIndexResponse
|
||||||
|
if err := json.Unmarshal(rr.Body.Bytes(), &admin); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if admin.Status != "reserved" {
|
||||||
|
t.Fatalf("admin=%+v", admin)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,87 @@
|
|||||||
|
package api
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"mime"
|
||||||
|
"net/http"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
func (s *Server) serveCDN(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != http.MethodGet && r.Method != http.MethodHead {
|
||||||
|
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
idx := s.index()
|
||||||
|
if idx == nil || idx.ResourceRoot == "" {
|
||||||
|
http.Error(w, "resource root not ready", http.StatusServiceUnavailable)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
_, rel, err := SplitCDNPath(r.URL.Path)
|
||||||
|
if err != nil {
|
||||||
|
http.NotFound(w, r)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
var entry ResourceEntry
|
||||||
|
var hasEntry bool
|
||||||
|
if s.cfg.RequireIndexed {
|
||||||
|
entry, hasEntry = idx.Lookup(rel)
|
||||||
|
if !hasEntry || !entry.Present || !entry.SizeMatch {
|
||||||
|
http.NotFound(w, r)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
abs, err := ResolveUnderRoot(idx.ResourceRoot, rel, true)
|
||||||
|
if err != nil {
|
||||||
|
http.NotFound(w, r)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
info, err := os.Stat(abs)
|
||||||
|
if err != nil || !info.Mode().IsRegular() {
|
||||||
|
http.NotFound(w, r)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if s.cfg.RequireIndexed && s.cfg.VerifySize {
|
||||||
|
if hasEntry && entry.Bytes > 0 && uint64(info.Size()) != entry.Bytes {
|
||||||
|
http.Error(w, "size mismatch with release index", http.StatusConflict)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
file, err := os.Open(abs)
|
||||||
|
if err != nil {
|
||||||
|
http.NotFound(w, r)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer file.Close()
|
||||||
|
|
||||||
|
w.Header().Set("Cache-Control", "public, max-age=31536000, immutable")
|
||||||
|
w.Header().Set("ETag", cdnETag(entry, hasEntry, info))
|
||||||
|
w.Header().Set("Content-Type", cdnContentType(abs))
|
||||||
|
http.ServeContent(w, r, info.Name(), info.ModTime(), file)
|
||||||
|
}
|
||||||
|
|
||||||
|
func cdnETag(entry ResourceEntry, hasEntry bool, info os.FileInfo) string {
|
||||||
|
if hasEntry && entry.BLAKE3 != "" {
|
||||||
|
return fmt.Sprintf(`"blake3-%s"`, entry.BLAKE3)
|
||||||
|
}
|
||||||
|
return fmt.Sprintf(`W/"%d-%d"`, info.Size(), info.ModTime().UnixNano())
|
||||||
|
}
|
||||||
|
|
||||||
|
func cdnContentType(abs string) string {
|
||||||
|
ext := strings.ToLower(filepath.Ext(abs))
|
||||||
|
if ext == ".json" {
|
||||||
|
return "application/json; charset=utf-8"
|
||||||
|
}
|
||||||
|
if ext == ".hash" {
|
||||||
|
return "text/plain; charset=utf-8"
|
||||||
|
}
|
||||||
|
if detected := mime.TypeByExtension(ext); detected != "" {
|
||||||
|
return detected
|
||||||
|
}
|
||||||
|
return "application/octet-stream"
|
||||||
|
}
|
||||||
@@ -0,0 +1,372 @@
|
|||||||
|
// Package api implements the resource bootstrap and distribution HTTP surface of bat-api.
|
||||||
|
//
|
||||||
|
// bat-api serves already-published official resources (CDN-shaped paths) and
|
||||||
|
// discovers version/manifest state and the current resource_root through the
|
||||||
|
// Rust bat daemon RPC socket.
|
||||||
|
// Automatic resource pull/sync remains the responsibility of the Rust bat
|
||||||
|
// binary; this package does not download or mutate release contents.
|
||||||
|
package api
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"net/url"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Default listen address and related defaults for the resource API.
|
||||||
|
const (
|
||||||
|
DefaultListen = ":18080"
|
||||||
|
DefaultPublicBaseURL = "http://127.0.0.1:18080"
|
||||||
|
DefaultStateDir = "/tmp/bat-pid"
|
||||||
|
DefaultAuthSkew = 5 * time.Minute
|
||||||
|
ClientPatchHost = "prod-clientpatch.bluearchiveyostar.com"
|
||||||
|
ServerInfoHost = "yostar-serverinfo.bluearchiveyostar.com"
|
||||||
|
EnvFileName = ".env"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Config holds bat-api process configuration.
|
||||||
|
//
|
||||||
|
// Values come from (highest wins): CLI flags > process environment > .env file
|
||||||
|
// > built-in defaults. Database-related keys are reserved for future API
|
||||||
|
// persistence and are loaded but not required for the resource CDN surface.
|
||||||
|
type Config struct {
|
||||||
|
// Listen is the HTTP listen address, e.g. ":18080" or "127.0.0.1:18080".
|
||||||
|
Listen string
|
||||||
|
// PublicBaseURL is used when rewriting AddressablesCatalogUrlRoot.
|
||||||
|
PublicBaseURL string
|
||||||
|
// StateDir is used only to derive the default RPC socket path.
|
||||||
|
StateDir string
|
||||||
|
// SocketPath is the bat.sock Unix socket used for JSON-RPC.
|
||||||
|
SocketPath string
|
||||||
|
// ResourceRoot overrides the release root discovered via RPC (tests / emergency).
|
||||||
|
ResourceRoot string
|
||||||
|
// ServerInfoFile is an optional path to a server-info JSON document.
|
||||||
|
ServerInfoFile string
|
||||||
|
// RequireIndexed rejects CDN paths that are not in the release index.
|
||||||
|
RequireIndexed bool
|
||||||
|
// VerifySize rejects CDN files whose size differs from the index.
|
||||||
|
VerifySize bool
|
||||||
|
// RPCTimeout is the per-call timeout for daemon RPC.
|
||||||
|
RPCTimeout time.Duration
|
||||||
|
// RefreshInterval periodically re-discovers the release through RPC. Zero disables it.
|
||||||
|
RefreshInterval time.Duration
|
||||||
|
// SkipEnvFile disables loading .env when true (BAT_API_SKIP_ENV_FILE=1).
|
||||||
|
SkipEnvFile bool
|
||||||
|
// AuthToken enables HTTP token authentication when non-empty.
|
||||||
|
AuthToken string
|
||||||
|
// AuthQueryParam is the optional query parameter accepted for token auth.
|
||||||
|
AuthQueryParam string
|
||||||
|
// AuthExemptPaths lists exact paths or slash-prefixes that bypass token auth.
|
||||||
|
AuthExemptPaths []string
|
||||||
|
// TrustProxyHeaders allows X-Forwarded-For / X-Real-IP for client identity.
|
||||||
|
TrustProxyHeaders bool
|
||||||
|
// AccessLog enables structured per-request access logs.
|
||||||
|
AccessLog bool
|
||||||
|
// RateLimitRPS limits requests per client identity. Zero disables in-process limiting.
|
||||||
|
RateLimitRPS float64
|
||||||
|
// RateLimitBurst is the token bucket burst size when rate limiting is enabled.
|
||||||
|
RateLimitBurst int
|
||||||
|
// MaxResourcePageLimit caps /v1/resources limit.
|
||||||
|
MaxResourcePageLimit int
|
||||||
|
|
||||||
|
// Reserved for a future persistent API layer (not used by CDN MVP).
|
||||||
|
DatabaseURL string
|
||||||
|
DatabasePassword string
|
||||||
|
RedisURL string
|
||||||
|
RedisPassword string
|
||||||
|
}
|
||||||
|
|
||||||
|
// DefaultConfig returns built-in defaults.
|
||||||
|
func DefaultConfig() Config {
|
||||||
|
return Config{
|
||||||
|
Listen: DefaultListen,
|
||||||
|
PublicBaseURL: DefaultPublicBaseURL,
|
||||||
|
StateDir: DefaultStateDir,
|
||||||
|
RequireIndexed: true,
|
||||||
|
VerifySize: true,
|
||||||
|
RPCTimeout: 30 * time.Second,
|
||||||
|
RefreshInterval: time.Minute,
|
||||||
|
AuthQueryParam: "bat_token",
|
||||||
|
MaxResourcePageLimit: 1000,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Normalize fills derived fields and validates required values.
|
||||||
|
func (c *Config) Normalize() error {
|
||||||
|
if c.Listen == "" {
|
||||||
|
c.Listen = DefaultListen
|
||||||
|
}
|
||||||
|
if c.PublicBaseURL == "" {
|
||||||
|
c.PublicBaseURL = DefaultPublicBaseURL
|
||||||
|
}
|
||||||
|
c.PublicBaseURL = strings.TrimRight(c.PublicBaseURL, "/")
|
||||||
|
if _, err := url.ParseRequestURI(c.PublicBaseURL); err != nil {
|
||||||
|
return fmt.Errorf("invalid public base URL %q: %w", c.PublicBaseURL, err)
|
||||||
|
}
|
||||||
|
if c.StateDir == "" {
|
||||||
|
c.StateDir = DefaultStateDir
|
||||||
|
}
|
||||||
|
if c.SocketPath == "" {
|
||||||
|
c.SocketPath = filepath.Join(c.StateDir, "bat.sock")
|
||||||
|
}
|
||||||
|
if c.RPCTimeout <= 0 {
|
||||||
|
c.RPCTimeout = 30 * time.Second
|
||||||
|
}
|
||||||
|
if c.RefreshInterval < 0 {
|
||||||
|
return fmt.Errorf("refresh interval must be >= 0")
|
||||||
|
}
|
||||||
|
if c.AuthQueryParam == "" {
|
||||||
|
c.AuthQueryParam = "bat_token"
|
||||||
|
}
|
||||||
|
c.AuthExemptPaths = normalizePathList(c.AuthExemptPaths)
|
||||||
|
if c.RateLimitRPS < 0 {
|
||||||
|
return fmt.Errorf("rate limit rps must be >= 0")
|
||||||
|
}
|
||||||
|
if c.RateLimitRPS > 0 && c.RateLimitBurst <= 0 {
|
||||||
|
c.RateLimitBurst = int(c.RateLimitRPS)
|
||||||
|
if c.RateLimitBurst <= 0 {
|
||||||
|
c.RateLimitBurst = 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if c.RateLimitBurst < 0 {
|
||||||
|
return fmt.Errorf("rate limit burst must be >= 0")
|
||||||
|
}
|
||||||
|
if c.MaxResourcePageLimit <= 0 {
|
||||||
|
c.MaxResourcePageLimit = 1000
|
||||||
|
}
|
||||||
|
if c.ResourceRoot != "" {
|
||||||
|
abs, err := filepath.Abs(c.ResourceRoot)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("resolve resource root: %w", err)
|
||||||
|
}
|
||||||
|
c.ResourceRoot = abs
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// LoadEnvFile reads KEY=VALUE lines from path into the process environment
|
||||||
|
// without overriding variables that are already set.
|
||||||
|
func LoadEnvFile(path string) error {
|
||||||
|
data, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
if os.IsNotExist(err) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for _, line := range strings.Split(string(data), "\n") {
|
||||||
|
line = strings.TrimSpace(line)
|
||||||
|
if line == "" || strings.HasPrefix(line, "#") {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
key, value, ok := strings.Cut(line, "=")
|
||||||
|
if !ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
key = strings.TrimSpace(key)
|
||||||
|
value = strings.TrimSpace(value)
|
||||||
|
value = strings.Trim(value, `"'`)
|
||||||
|
if key == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if _, exists := os.LookupEnv(key); exists {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if err := os.Setenv(key, value); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ApplyEnv overlays environment variables onto cfg.
|
||||||
|
func ApplyEnv(cfg *Config) {
|
||||||
|
if v := os.Getenv("BAT_API_LISTEN"); v != "" {
|
||||||
|
cfg.Listen = v
|
||||||
|
}
|
||||||
|
// Alias common port-only style.
|
||||||
|
if v := os.Getenv("BAT_API_PORT"); v != "" && os.Getenv("BAT_API_LISTEN") == "" {
|
||||||
|
if strings.HasPrefix(v, ":") {
|
||||||
|
cfg.Listen = v
|
||||||
|
} else {
|
||||||
|
cfg.Listen = ":" + v
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_PUBLIC_BASE_URL"); v != "" {
|
||||||
|
cfg.PublicBaseURL = v
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_STATE_DIR"); v != "" {
|
||||||
|
cfg.StateDir = v
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_SOCKET"); v != "" {
|
||||||
|
cfg.SocketPath = v
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_RESOURCE_ROOT"); v != "" {
|
||||||
|
cfg.ResourceRoot = v
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_SERVER_INFO_FILE"); v != "" {
|
||||||
|
cfg.ServerInfoFile = v
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_REQUIRE_INDEXED"); v != "" {
|
||||||
|
cfg.RequireIndexed = parseBool(v, cfg.RequireIndexed)
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_VERIFY_SIZE"); v != "" {
|
||||||
|
cfg.VerifySize = parseBool(v, cfg.VerifySize)
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_RPC_TIMEOUT"); v != "" {
|
||||||
|
if d, err := time.ParseDuration(v); err == nil {
|
||||||
|
cfg.RPCTimeout = d
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_REFRESH_INTERVAL"); v != "" {
|
||||||
|
if d, err := time.ParseDuration(v); err == nil {
|
||||||
|
cfg.RefreshInterval = d
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_DATABASE_URL"); v != "" {
|
||||||
|
cfg.DatabaseURL = v
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_DATABASE_PASSWORD"); v != "" {
|
||||||
|
cfg.DatabasePassword = v
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_REDIS_URL"); v != "" {
|
||||||
|
cfg.RedisURL = v
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_REDIS_PASSWORD"); v != "" {
|
||||||
|
cfg.RedisPassword = v
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_SKIP_ENV_FILE"); v != "" {
|
||||||
|
cfg.SkipEnvFile = parseBool(v, false)
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_AUTH_TOKEN"); v != "" {
|
||||||
|
cfg.AuthToken = v
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_AUTH_QUERY_PARAM"); v != "" {
|
||||||
|
cfg.AuthQueryParam = v
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_AUTH_EXEMPT_PATHS"); v != "" {
|
||||||
|
cfg.AuthExemptPaths = splitCSV(v)
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_TRUST_PROXY_HEADERS"); v != "" {
|
||||||
|
cfg.TrustProxyHeaders = parseBool(v, cfg.TrustProxyHeaders)
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_ACCESS_LOG"); v != "" {
|
||||||
|
cfg.AccessLog = parseBool(v, cfg.AccessLog)
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_RATE_LIMIT_RPS"); v != "" {
|
||||||
|
if f, err := strconv.ParseFloat(strings.TrimSpace(v), 64); err == nil {
|
||||||
|
cfg.RateLimitRPS = f
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_RATE_LIMIT_BURST"); v != "" {
|
||||||
|
if i, err := strconv.Atoi(strings.TrimSpace(v)); err == nil {
|
||||||
|
cfg.RateLimitBurst = i
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if v := os.Getenv("BAT_API_MAX_RESOURCE_LIMIT"); v != "" {
|
||||||
|
if i, err := strconv.Atoi(strings.TrimSpace(v)); err == nil {
|
||||||
|
cfg.MaxResourcePageLimit = i
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseBool(raw string, fallback bool) bool {
|
||||||
|
b, err := strconv.ParseBool(strings.TrimSpace(raw))
|
||||||
|
if err != nil {
|
||||||
|
return fallback
|
||||||
|
}
|
||||||
|
return b
|
||||||
|
}
|
||||||
|
|
||||||
|
func splitCSV(raw string) []string {
|
||||||
|
parts := strings.Split(raw, ",")
|
||||||
|
out := make([]string, 0, len(parts))
|
||||||
|
for _, part := range parts {
|
||||||
|
part = strings.TrimSpace(part)
|
||||||
|
if part != "" {
|
||||||
|
out = append(out, part)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func normalizePathList(paths []string) []string {
|
||||||
|
out := make([]string, 0, len(paths))
|
||||||
|
seen := map[string]struct{}{}
|
||||||
|
for _, path := range paths {
|
||||||
|
path = strings.TrimSpace(path)
|
||||||
|
if path == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !strings.HasPrefix(path, "/") {
|
||||||
|
path = "/" + path
|
||||||
|
}
|
||||||
|
if _, ok := seen[path]; ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
seen[path] = struct{}{}
|
||||||
|
out = append(out, path)
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// EnvTemplate is written next to the bat-api binary on first run when missing.
|
||||||
|
const EnvTemplate = `# bat-api configuration
|
||||||
|
# Priority: CLI flags > process environment > this file > built-in defaults.
|
||||||
|
# Resource auto-pull/sync is owned by the Rust bat daemon, not bat-api.
|
||||||
|
# bat-api only bootstraps/distributes already-published resources and exposes
|
||||||
|
# a small management surface for release inspection.
|
||||||
|
|
||||||
|
# HTTP listen address (host:port or :port)
|
||||||
|
BAT_API_LISTEN=:18080
|
||||||
|
# Public base URL used when rewriting AddressablesCatalogUrlRoot
|
||||||
|
BAT_API_PUBLIC_BASE_URL=http://127.0.0.1:18080
|
||||||
|
|
||||||
|
# Rust bat daemon RPC socket (primary discovery path)
|
||||||
|
# Prefer BAT_API_SOCKET; BAT_API_STATE_DIR only derives the default socket path.
|
||||||
|
BAT_API_STATE_DIR=/tmp/bat-pid
|
||||||
|
# BAT_API_SOCKET=/tmp/bat-pid/bat.sock
|
||||||
|
|
||||||
|
# Optional override of the published release root (fixtures / emergency only).
|
||||||
|
# Production obtains resource_root from daemon RPC (catalog.status / resource.manifest).
|
||||||
|
# BAT_API_RESOURCE_ROOT=
|
||||||
|
|
||||||
|
# Optional server-info JSON for Addressables root rewrite
|
||||||
|
# BAT_API_SERVER_INFO_FILE=
|
||||||
|
|
||||||
|
# CDN safety
|
||||||
|
BAT_API_REQUIRE_INDEXED=true
|
||||||
|
BAT_API_VERIFY_SIZE=true
|
||||||
|
BAT_API_RPC_TIMEOUT=30s
|
||||||
|
# Periodically re-read bat.sock so bat-api follows Rust bat release switches.
|
||||||
|
# Set to 0 to disable in fixture-only local development.
|
||||||
|
BAT_API_REFRESH_INTERVAL=1m
|
||||||
|
|
||||||
|
# Player-facing HTTP controls.
|
||||||
|
# Prefer setting BAT_API_AUTH_TOKEN through a secret manager or process
|
||||||
|
# environment. Reverse proxies may inject Authorization: Bearer <token> or
|
||||||
|
# X-BAT-Token to authenticated upstream requests. Query token fallback uses
|
||||||
|
# BAT_API_AUTH_QUERY_PARAM and is supported for clients that cannot set headers.
|
||||||
|
# BAT_API_AUTH_TOKEN=
|
||||||
|
BAT_API_AUTH_QUERY_PARAM=bat_token
|
||||||
|
# BAT_API_AUTH_EXEMPT_PATHS=
|
||||||
|
BAT_API_TRUST_PROXY_HEADERS=false
|
||||||
|
BAT_API_ACCESS_LOG=false
|
||||||
|
# Zero disables in-process limiting; production may still enforce edge limits.
|
||||||
|
BAT_API_RATE_LIMIT_RPS=0
|
||||||
|
BAT_API_RATE_LIMIT_BURST=0
|
||||||
|
BAT_API_MAX_RESOURCE_LIMIT=1000
|
||||||
|
|
||||||
|
# Reserved for future API persistence (not required for resource CDN)
|
||||||
|
# BAT_API_DATABASE_URL=postgres://bat:@127.0.0.1:5432/bat?sslmode=disable
|
||||||
|
# BAT_API_DATABASE_PASSWORD=
|
||||||
|
# BAT_API_REDIS_URL=redis://127.0.0.1:6379/0
|
||||||
|
# BAT_API_REDIS_PASSWORD=
|
||||||
|
|
||||||
|
# Set to 1 to ignore this file entirely
|
||||||
|
# BAT_API_SKIP_ENV_FILE=0
|
||||||
|
`
|
||||||
@@ -0,0 +1,208 @@
|
|||||||
|
package api
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"net/url"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// LauncherAPIHost is the official JP launcher API host observed from the PC launcher.
|
||||||
|
const LauncherAPIHost = "api-launcher-jp.yo-star.com"
|
||||||
|
|
||||||
|
func (s *Server) handleLauncherBootstrap(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != http.MethodGet {
|
||||||
|
writeErrorJSON(w, http.StatusMethodNotAllowed, "method_not_allowed", "method not allowed")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
sum := s.releaseSummary()
|
||||||
|
status := http.StatusOK
|
||||||
|
if !sum.Ready || sum.Snapshot == nil {
|
||||||
|
status = http.StatusServiceUnavailable
|
||||||
|
}
|
||||||
|
writeNoStoreJSON(w, status, s.launcherBootstrapBody(sum))
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleLauncherGameConfig(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != http.MethodGet {
|
||||||
|
writeErrorJSON(w, http.StatusMethodNotAllowed, "method_not_allowed", "method not allowed")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
sum := s.releaseSummary()
|
||||||
|
if !sum.Ready || sum.Snapshot == nil {
|
||||||
|
writeLauncherEnvelope(w, http.StatusServiceUnavailable, 503, "release not ready", LauncherGameConfigJSONData{
|
||||||
|
ResourceBootstrapURL: s.launcherBootstrapURL(),
|
||||||
|
Scope: "resource_bootstrap_only",
|
||||||
|
})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
snap := sum.Snapshot
|
||||||
|
data := LauncherGameConfigData{
|
||||||
|
GameLatestVersion: defaultString(s.launcherLatestVersion(snap), snap.AppVersion),
|
||||||
|
GameLatestFilePath: s.launcherLatestFilePath(snap),
|
||||||
|
GameStartExeName: s.launcherStartExeName(snap),
|
||||||
|
GameStartParams: s.launcherStartParams(snap),
|
||||||
|
ResourceBootstrapURL: s.launcherBootstrapURL(),
|
||||||
|
ServerInfoURL: s.serverInfoURL(),
|
||||||
|
ClientPatchBaseURL: s.clientPatchBaseURL(),
|
||||||
|
Scope: "resource_bootstrap_only",
|
||||||
|
}
|
||||||
|
if snap.LauncherMetadata != nil && snap.LauncherMetadata.GameLowestVersion != "" {
|
||||||
|
data.GameLowestVersion = snap.LauncherMetadata.GameLowestVersion
|
||||||
|
}
|
||||||
|
writeLauncherEnvelope(w, http.StatusOK, 200, "ok", data)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleLauncherGameConfigJSON(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != http.MethodGet {
|
||||||
|
writeErrorJSON(w, http.StatusMethodNotAllowed, "method_not_allowed", "method not allowed")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
sum := s.releaseSummary()
|
||||||
|
if !sum.Ready || sum.Snapshot == nil {
|
||||||
|
writeLauncherEnvelope(w, http.StatusServiceUnavailable, 503, "release not ready", LauncherGameConfigJSONData{
|
||||||
|
URL: s.launcherBootstrapURL(),
|
||||||
|
ResourceBootstrapURL: s.launcherBootstrapURL(),
|
||||||
|
Scope: "resource_bootstrap_only",
|
||||||
|
})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
writeLauncherEnvelope(w, http.StatusOK, 200, "ok", LauncherGameConfigJSONData{
|
||||||
|
URL: s.launcherBootstrapJSONURL(r.URL.Query()),
|
||||||
|
ResourceBootstrapURL: s.launcherBootstrapURL(),
|
||||||
|
PackageUpdateManifest: false,
|
||||||
|
Scope: "resource_bootstrap_only",
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleLauncherCdnConfig(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != http.MethodGet {
|
||||||
|
writeErrorJSON(w, http.StatusMethodNotAllowed, "method_not_allowed", "method not allowed")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
writeLauncherEnvelope(w, http.StatusOK, 200, "ok", LauncherCdnConfigData{
|
||||||
|
PrimaryCDN: strings.TrimRight(s.cfg.PublicBaseURL, "/"),
|
||||||
|
BackupCDN: strings.TrimRight(s.cfg.PublicBaseURL, "/"),
|
||||||
|
ResourceBootstrapURL: s.launcherBootstrapURL(),
|
||||||
|
PackageUpdateManifest: false,
|
||||||
|
Scope: "resource_bootstrap_only",
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) launcherBootstrapBody(sum ReleaseSummary) LauncherBootstrapResponse {
|
||||||
|
publicBase := strings.TrimRight(s.cfg.PublicBaseURL, "/")
|
||||||
|
body := LauncherBootstrapResponse{
|
||||||
|
Service: "bat-api",
|
||||||
|
Ready: sum.Ready,
|
||||||
|
LauncherAPI: LauncherAPIInfo{
|
||||||
|
Host: LauncherAPIHost,
|
||||||
|
ObservedResourceRelevantEndpoints: []string{
|
||||||
|
"/api/launcher/game/config",
|
||||||
|
"/api/launcher/game/config/json",
|
||||||
|
"/api/launcher/advanced/game/download/cdn",
|
||||||
|
},
|
||||||
|
CompatibilityScope: "resource_bootstrap_only",
|
||||||
|
},
|
||||||
|
Policy: LauncherPolicy{
|
||||||
|
Source: "rust_bat_snapshot",
|
||||||
|
DownloadsLauncherPackage: false,
|
||||||
|
EmulatesLoginOrGateway: false,
|
||||||
|
PackageUpdateManifest: false,
|
||||||
|
},
|
||||||
|
Resource: LauncherResource{
|
||||||
|
Release: sum.Snapshot,
|
||||||
|
ServerInfoURL: s.serverInfoURL(),
|
||||||
|
ClientPatchBaseURL: s.clientPatchBaseURL(),
|
||||||
|
},
|
||||||
|
Endpoints: LauncherEndpointSet{
|
||||||
|
Bootstrap: publicBase + "/v1/bootstrap",
|
||||||
|
LauncherBootstrap: publicBase + "/v1/launcher/bootstrap",
|
||||||
|
ServerInfo: s.serverInfoURL(),
|
||||||
|
ClientPatchBase: s.clientPatchBaseURL(),
|
||||||
|
LauncherGameConfig: publicBase + "/" + LauncherAPIHost + "/api/launcher/game/config",
|
||||||
|
LauncherGameConfigJSON: publicBase + "/" + LauncherAPIHost + "/api/launcher/game/config/json",
|
||||||
|
LauncherCdnConfig: publicBase + "/" + LauncherAPIHost + "/api/launcher/advanced/game/download/cdn",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
if sum.Snapshot != nil {
|
||||||
|
body.GameMainConfig = sum.Snapshot.GameMainConfig
|
||||||
|
body.LauncherMetadata = sum.Snapshot.LauncherMetadata
|
||||||
|
if rewritten, ok := rewriteAddressablesRoot(sum.Snapshot.AddressablesRoot, publicBase); ok {
|
||||||
|
body.AddressablesCatalogURLRoot = rewritten
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return body
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) releaseSummary() ReleaseSummary {
|
||||||
|
if idx := s.index(); idx != nil {
|
||||||
|
return idx.Summary()
|
||||||
|
}
|
||||||
|
return ReleaseSummary{}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) serverInfoURL() string {
|
||||||
|
return strings.TrimRight(s.cfg.PublicBaseURL, "/") + "/" + ServerInfoHost + "/server-info.json"
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) clientPatchBaseURL() string {
|
||||||
|
return strings.TrimRight(s.cfg.PublicBaseURL, "/") + "/" + ClientPatchHost
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) launcherBootstrapURL() string {
|
||||||
|
return strings.TrimRight(s.cfg.PublicBaseURL, "/") + "/v1/launcher/bootstrap"
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) launcherBootstrapJSONURL(query url.Values) string {
|
||||||
|
base := strings.TrimRight(s.cfg.PublicBaseURL, "/") + "/" + LauncherAPIHost + "/api/launcher/resource/bootstrap.json"
|
||||||
|
values := url.Values{}
|
||||||
|
for _, key := range []string{"version", "file_path"} {
|
||||||
|
if value := query.Get(key); value != "" {
|
||||||
|
values.Set(key, value)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if encoded := values.Encode(); encoded != "" {
|
||||||
|
return base + "?" + encoded
|
||||||
|
}
|
||||||
|
return base
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) launcherLatestVersion(snap *SnapshotSummary) string {
|
||||||
|
if snap.LauncherMetadata != nil && snap.LauncherMetadata.GameLatestVersion != "" {
|
||||||
|
return snap.LauncherMetadata.GameLatestVersion
|
||||||
|
}
|
||||||
|
return snap.AppVersion
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) launcherLatestFilePath(snap *SnapshotSummary) string {
|
||||||
|
if snap.LauncherMetadata != nil {
|
||||||
|
return snap.LauncherMetadata.GameLatestFilePath
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) launcherStartExeName(snap *SnapshotSummary) string {
|
||||||
|
if snap.LauncherMetadata != nil {
|
||||||
|
return snap.LauncherMetadata.GameStartExeName
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) launcherStartParams(snap *SnapshotSummary) []string {
|
||||||
|
if snap.LauncherMetadata != nil && len(snap.LauncherMetadata.GameStartParams) > 0 {
|
||||||
|
return append([]string(nil), snap.LauncherMetadata.GameStartParams...)
|
||||||
|
}
|
||||||
|
return []string{}
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeLauncherEnvelope[T any](w http.ResponseWriter, status int, code int, message string, data T) {
|
||||||
|
writeNoStoreJSON(w, status, LauncherEnvelope[T]{
|
||||||
|
Code: code,
|
||||||
|
Message: message,
|
||||||
|
Data: data,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func launcherHostPath(path string) string {
|
||||||
|
return fmt.Sprintf("/%s%s", LauncherAPIHost, path)
|
||||||
|
}
|
||||||
@@ -0,0 +1,259 @@
|
|||||||
|
package api
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/subtle"
|
||||||
|
"net"
|
||||||
|
"net/http"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func (s *Server) wrapHandler(next http.Handler) http.Handler {
|
||||||
|
handler := next
|
||||||
|
handler = s.authMiddleware(handler)
|
||||||
|
handler = s.rateLimitMiddleware(handler)
|
||||||
|
handler = s.securityHeadersMiddleware(handler)
|
||||||
|
handler = s.accessLogMiddleware(handler)
|
||||||
|
return handler
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) securityHeadersMiddleware(next http.Handler) http.Handler {
|
||||||
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
w.Header().Set("X-Content-Type-Options", "nosniff")
|
||||||
|
w.Header().Set("X-Frame-Options", "DENY")
|
||||||
|
w.Header().Set("Referrer-Policy", "no-referrer")
|
||||||
|
w.Header().Set("Permissions-Policy", "geolocation=(), microphone=(), camera=()")
|
||||||
|
w.Header().Set("Content-Security-Policy", "default-src 'none'; frame-ancestors 'none'")
|
||||||
|
next.ServeHTTP(w, r)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) authMiddleware(next http.Handler) http.Handler {
|
||||||
|
if s.cfg.AuthToken == "" {
|
||||||
|
return next
|
||||||
|
}
|
||||||
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if s.authExempt(r.URL.Path) {
|
||||||
|
next.ServeHTTP(w, r)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !constantTimeTokenEqual(s.requestToken(r), s.cfg.AuthToken) {
|
||||||
|
w.Header().Set("WWW-Authenticate", `Bearer realm="bat-api"`)
|
||||||
|
writeErrorJSON(w, http.StatusUnauthorized, "unauthorized", "missing or invalid access token")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
next.ServeHTTP(w, r)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) authExempt(path string) bool {
|
||||||
|
for _, exempt := range s.cfg.AuthExemptPaths {
|
||||||
|
if exempt == path {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if strings.HasSuffix(exempt, "/") && strings.HasPrefix(path, exempt) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) requestToken(r *http.Request) string {
|
||||||
|
if auth := r.Header.Get("Authorization"); auth != "" {
|
||||||
|
const prefix = "Bearer "
|
||||||
|
if strings.HasPrefix(auth, prefix) {
|
||||||
|
return strings.TrimSpace(strings.TrimPrefix(auth, prefix))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, header := range []string{"X-BAT-Token", "X-BAT-API-Key"} {
|
||||||
|
if token := strings.TrimSpace(r.Header.Get(header)); token != "" {
|
||||||
|
return token
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if s.cfg.AuthQueryParam != "" {
|
||||||
|
return r.URL.Query().Get(s.cfg.AuthQueryParam)
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
func constantTimeTokenEqual(got, want string) bool {
|
||||||
|
if got == "" || want == "" {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
gotBytes := []byte(got)
|
||||||
|
wantBytes := []byte(want)
|
||||||
|
if len(gotBytes) != len(wantBytes) {
|
||||||
|
subtle.ConstantTimeCompare(wantBytes, wantBytes)
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return subtle.ConstantTimeCompare(gotBytes, wantBytes) == 1
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) rateLimitMiddleware(next http.Handler) http.Handler {
|
||||||
|
if s.limiter == nil {
|
||||||
|
return next
|
||||||
|
}
|
||||||
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
client := s.clientIdentity(r)
|
||||||
|
if !s.limiter.allow(client, time.Now()) {
|
||||||
|
w.Header().Set("Retry-After", "1")
|
||||||
|
writeErrorJSON(w, http.StatusTooManyRequests, "rate_limited", "too many requests")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
next.ServeHTTP(w, r)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) accessLogMiddleware(next http.Handler) http.Handler {
|
||||||
|
if !s.cfg.AccessLog {
|
||||||
|
return next
|
||||||
|
}
|
||||||
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
start := time.Now()
|
||||||
|
rec := &statusRecorder{ResponseWriter: w, status: http.StatusOK}
|
||||||
|
next.ServeHTTP(rec, r)
|
||||||
|
requestID := safeLogValue(r.Header.Get("X-Request-ID"))
|
||||||
|
if requestID == "" {
|
||||||
|
requestID = "-"
|
||||||
|
}
|
||||||
|
s.logger.Printf(
|
||||||
|
"access method=%s path=%s status=%d bytes=%d duration_ms=%d client_ip=%s request_id=%s user_agent=%q",
|
||||||
|
r.Method,
|
||||||
|
r.URL.Path,
|
||||||
|
rec.status,
|
||||||
|
rec.bytes,
|
||||||
|
time.Since(start).Milliseconds(),
|
||||||
|
s.clientIdentity(r),
|
||||||
|
requestID,
|
||||||
|
r.UserAgent(),
|
||||||
|
)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) clientIdentity(r *http.Request) string {
|
||||||
|
if s.cfg.TrustProxyHeaders {
|
||||||
|
if forwarded := r.Header.Get("X-Forwarded-For"); forwarded != "" {
|
||||||
|
for _, part := range strings.Split(forwarded, ",") {
|
||||||
|
if ip := strings.TrimSpace(part); ip != "" {
|
||||||
|
return ip
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if realIP := strings.TrimSpace(r.Header.Get("X-Real-IP")); realIP != "" {
|
||||||
|
return realIP
|
||||||
|
}
|
||||||
|
}
|
||||||
|
host, _, err := net.SplitHostPort(r.RemoteAddr)
|
||||||
|
if err == nil && host != "" {
|
||||||
|
return host
|
||||||
|
}
|
||||||
|
return r.RemoteAddr
|
||||||
|
}
|
||||||
|
|
||||||
|
func safeLogValue(raw string) string {
|
||||||
|
raw = strings.TrimSpace(raw)
|
||||||
|
if raw == "" {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
raw = strings.Map(func(r rune) rune {
|
||||||
|
if r < 32 || r == 127 {
|
||||||
|
return -1
|
||||||
|
}
|
||||||
|
return r
|
||||||
|
}, raw)
|
||||||
|
if len(raw) > 128 {
|
||||||
|
return raw[:128]
|
||||||
|
}
|
||||||
|
return raw
|
||||||
|
}
|
||||||
|
|
||||||
|
type statusRecorder struct {
|
||||||
|
http.ResponseWriter
|
||||||
|
status int
|
||||||
|
bytes int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *statusRecorder) WriteHeader(status int) {
|
||||||
|
r.status = status
|
||||||
|
r.ResponseWriter.WriteHeader(status)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *statusRecorder) Write(data []byte) (int, error) {
|
||||||
|
n, err := r.ResponseWriter.Write(data)
|
||||||
|
r.bytes += n
|
||||||
|
return n, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *statusRecorder) Unwrap() http.ResponseWriter {
|
||||||
|
return r.ResponseWriter
|
||||||
|
}
|
||||||
|
|
||||||
|
type tokenBucketLimiter struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
rate float64
|
||||||
|
burst float64
|
||||||
|
buckets map[string]*tokenBucket
|
||||||
|
lastCleanup time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
type tokenBucket struct {
|
||||||
|
tokens float64
|
||||||
|
last time.Time
|
||||||
|
lastSeen time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
func newTokenBucketLimiter(rps float64, burst int) *tokenBucketLimiter {
|
||||||
|
return &tokenBucketLimiter{
|
||||||
|
rate: rps,
|
||||||
|
burst: float64(burst),
|
||||||
|
buckets: map[string]*tokenBucket{},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (l *tokenBucketLimiter) allow(key string, now time.Time) bool {
|
||||||
|
if key == "" {
|
||||||
|
key = "unknown"
|
||||||
|
}
|
||||||
|
l.mu.Lock()
|
||||||
|
defer l.mu.Unlock()
|
||||||
|
if l.lastCleanup.IsZero() || now.Sub(l.lastCleanup) > time.Minute {
|
||||||
|
l.cleanup(now)
|
||||||
|
}
|
||||||
|
bucket := l.buckets[key]
|
||||||
|
if bucket == nil {
|
||||||
|
bucket = &tokenBucket{tokens: l.burst, last: now, lastSeen: now}
|
||||||
|
l.buckets[key] = bucket
|
||||||
|
}
|
||||||
|
elapsed := now.Sub(bucket.last).Seconds()
|
||||||
|
bucket.tokens += elapsed * l.rate
|
||||||
|
if bucket.tokens > l.burst {
|
||||||
|
bucket.tokens = l.burst
|
||||||
|
}
|
||||||
|
bucket.last = now
|
||||||
|
bucket.lastSeen = now
|
||||||
|
if bucket.tokens < 1 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
bucket.tokens--
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
func (l *tokenBucketLimiter) cleanup(now time.Time) {
|
||||||
|
for key, bucket := range l.buckets {
|
||||||
|
if now.Sub(bucket.lastSeen) > 10*time.Minute {
|
||||||
|
delete(l.buckets, key)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
l.lastCleanup = now
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeErrorJSON(w http.ResponseWriter, status int, code string, message string) {
|
||||||
|
writeNoStoreJSON(w, status, ErrorResponse{
|
||||||
|
Error: ErrorBody{
|
||||||
|
Code: code,
|
||||||
|
Message: message,
|
||||||
|
Status: status,
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}
|
||||||
@@ -0,0 +1,161 @@
|
|||||||
|
package api
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
"strconv"
|
||||||
|
)
|
||||||
|
|
||||||
|
const openAPISpecYAML = `openapi: 3.0.3
|
||||||
|
info:
|
||||||
|
title: BlueArchive Toolkit bat-api
|
||||||
|
version: 0.1.0
|
||||||
|
description: Resource bootstrap and read-only distribution API.
|
||||||
|
servers:
|
||||||
|
- url: http://127.0.0.1:18080
|
||||||
|
security:
|
||||||
|
- bearerAuth: []
|
||||||
|
- queryToken: []
|
||||||
|
paths:
|
||||||
|
/healthz:
|
||||||
|
get:
|
||||||
|
summary: Liveness and refresh diagnostics
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Service is alive.
|
||||||
|
/readyz:
|
||||||
|
get:
|
||||||
|
summary: Release readiness
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: A distributable release is available.
|
||||||
|
"503":
|
||||||
|
description: No distributable release is available.
|
||||||
|
/v1/bootstrap:
|
||||||
|
get:
|
||||||
|
summary: Startup resource bootstrap
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Resource bootstrap response.
|
||||||
|
"503":
|
||||||
|
description: Release is not ready.
|
||||||
|
/v1/launcher/bootstrap:
|
||||||
|
get:
|
||||||
|
summary: Launcher-shaped resource bootstrap
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Launcher bootstrap response.
|
||||||
|
"503":
|
||||||
|
description: Release is not ready.
|
||||||
|
/api/launcher/game/config:
|
||||||
|
get:
|
||||||
|
summary: Resource-only launcher game config compatibility
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Launcher envelope with resource metadata.
|
||||||
|
/api/launcher/game/config/json:
|
||||||
|
get:
|
||||||
|
summary: Resource-only launcher manifest URL compatibility
|
||||||
|
parameters:
|
||||||
|
- name: version
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
- name: file_path
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Launcher envelope pointing to resource bootstrap JSON.
|
||||||
|
/api/launcher/advanced/game/download/cdn:
|
||||||
|
get:
|
||||||
|
summary: Resource-only launcher CDN compatibility
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Launcher envelope with public base URL as CDN root.
|
||||||
|
/v1/release:
|
||||||
|
get:
|
||||||
|
summary: Current release summary
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Release summary.
|
||||||
|
/v1/resources:
|
||||||
|
get:
|
||||||
|
summary: Paginated resource manifest entries
|
||||||
|
parameters:
|
||||||
|
- name: offset
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: integer
|
||||||
|
minimum: 0
|
||||||
|
- name: limit
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
type: integer
|
||||||
|
minimum: 1
|
||||||
|
maximum: 1000
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Resource list page.
|
||||||
|
/v1/server-info:
|
||||||
|
get:
|
||||||
|
summary: Rewritten server-info document
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Server-info JSON with AddressablesCatalogUrlRoot rewritten.
|
||||||
|
/openapi.yaml:
|
||||||
|
get:
|
||||||
|
summary: OpenAPI document
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: OpenAPI YAML.
|
||||||
|
/admin/:
|
||||||
|
get:
|
||||||
|
summary: Reserved admin panel entry
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Admin panel placeholder and links.
|
||||||
|
/prod-clientpatch.bluearchiveyostar.com/{path}:
|
||||||
|
get:
|
||||||
|
summary: CDN-shaped resource bytes
|
||||||
|
parameters:
|
||||||
|
- name: path
|
||||||
|
in: path
|
||||||
|
required: true
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Resource bytes.
|
||||||
|
"206":
|
||||||
|
description: Partial resource bytes.
|
||||||
|
head:
|
||||||
|
summary: CDN-shaped resource metadata
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Resource headers.
|
||||||
|
components:
|
||||||
|
securitySchemes:
|
||||||
|
bearerAuth:
|
||||||
|
type: http
|
||||||
|
scheme: bearer
|
||||||
|
queryToken:
|
||||||
|
type: apiKey
|
||||||
|
in: query
|
||||||
|
name: bat_token
|
||||||
|
`
|
||||||
|
|
||||||
|
func (s *Server) handleOpenAPI(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != http.MethodGet && r.Method != http.MethodHead {
|
||||||
|
writeErrorJSON(w, http.StatusMethodNotAllowed, "method_not_allowed", "method not allowed")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
w.Header().Set("Content-Type", "application/yaml; charset=utf-8")
|
||||||
|
w.Header().Set("Cache-Control", "no-store")
|
||||||
|
w.Header().Set("Content-Length", strconv.Itoa(len(openAPISpecYAML)))
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
if r.Method == http.MethodHead {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
_, _ = w.Write([]byte(openAPISpecYAML))
|
||||||
|
}
|
||||||
@@ -0,0 +1,93 @@
|
|||||||
|
package api
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// OfficialHosts are host path prefixes allowed on the CDN surface.
|
||||||
|
var OfficialHosts = map[string]struct{}{
|
||||||
|
ClientPatchHost: {},
|
||||||
|
ServerInfoHost: {},
|
||||||
|
}
|
||||||
|
|
||||||
|
// SplitCDNPath splits a request URL path into host + relative path under that host.
|
||||||
|
// Expected form: /prod-clientpatch.bluearchiveyostar.com/r93_xxx/TableBundles/...
|
||||||
|
func SplitCDNPath(requestPath string) (host string, rel string, err error) {
|
||||||
|
cleaned := filepath.ToSlash(requestPath)
|
||||||
|
cleaned = strings.TrimPrefix(cleaned, "/")
|
||||||
|
if cleaned == "" {
|
||||||
|
return "", "", fmt.Errorf("empty path")
|
||||||
|
}
|
||||||
|
parts := strings.Split(cleaned, "/")
|
||||||
|
if len(parts) < 2 {
|
||||||
|
return "", "", fmt.Errorf("path must include host and object path")
|
||||||
|
}
|
||||||
|
host = parts[0]
|
||||||
|
if _, ok := OfficialHosts[host]; !ok {
|
||||||
|
return "", "", fmt.Errorf("unsupported host %q", host)
|
||||||
|
}
|
||||||
|
for _, segment := range parts {
|
||||||
|
if segment == "" || segment == "." || segment == ".." {
|
||||||
|
return "", "", fmt.Errorf("unsafe path segment")
|
||||||
|
}
|
||||||
|
if strings.ContainsAny(segment, `?\`) {
|
||||||
|
return "", "", fmt.Errorf("unsafe path character")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
rel = strings.Join(parts, "/")
|
||||||
|
return host, rel, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ResolveUnderRoot joins root with a slash-separated relative path and ensures
|
||||||
|
// the result stays within root. Symlink targets that escape root are rejected
|
||||||
|
// when evalSymlinks is true and the path exists.
|
||||||
|
func ResolveUnderRoot(root, rel string, evalSymlinks bool) (string, error) {
|
||||||
|
if root == "" {
|
||||||
|
return "", fmt.Errorf("resource root is empty")
|
||||||
|
}
|
||||||
|
rootAbs, err := filepath.Abs(root)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
if evalSymlinks {
|
||||||
|
if resolved, err := filepath.EvalSymlinks(rootAbs); err == nil {
|
||||||
|
rootAbs = resolved
|
||||||
|
}
|
||||||
|
}
|
||||||
|
rel = filepath.ToSlash(rel)
|
||||||
|
rel = strings.TrimPrefix(rel, "/")
|
||||||
|
var segments []string
|
||||||
|
for _, segment := range strings.Split(rel, "/") {
|
||||||
|
if segment == "" || segment == "." {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if segment == ".." {
|
||||||
|
return "", fmt.Errorf("path escapes resource root")
|
||||||
|
}
|
||||||
|
if strings.ContainsAny(segment, `?\`) {
|
||||||
|
return "", fmt.Errorf("unsafe path segment %q", segment)
|
||||||
|
}
|
||||||
|
segments = append(segments, segment)
|
||||||
|
}
|
||||||
|
if len(segments) == 0 {
|
||||||
|
return "", fmt.Errorf("empty relative path")
|
||||||
|
}
|
||||||
|
candidate := filepath.Join(append([]string{rootAbs}, segments...)...)
|
||||||
|
// Ensure candidate is under rootAbs even before existence checks.
|
||||||
|
relCheck, err := filepath.Rel(rootAbs, candidate)
|
||||||
|
if err != nil || strings.HasPrefix(relCheck, "..") || filepath.IsAbs(relCheck) {
|
||||||
|
return "", fmt.Errorf("path escapes resource root")
|
||||||
|
}
|
||||||
|
if evalSymlinks {
|
||||||
|
if resolved, err := filepath.EvalSymlinks(candidate); err == nil {
|
||||||
|
relCheck, err = filepath.Rel(rootAbs, resolved)
|
||||||
|
if err != nil || strings.HasPrefix(relCheck, "..") || filepath.IsAbs(relCheck) {
|
||||||
|
return "", fmt.Errorf("symlink escapes resource root")
|
||||||
|
}
|
||||||
|
candidate = resolved
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return candidate, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,292 @@
|
|||||||
|
package api
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ResourceEntry is one distributable object from the published release.
|
||||||
|
type ResourceEntry struct {
|
||||||
|
URL string `json:"url"`
|
||||||
|
RelativePath string `json:"relative_path"`
|
||||||
|
Bytes uint64 `json:"bytes"`
|
||||||
|
BLAKE3 string `json:"blake3,omitempty"`
|
||||||
|
Present bool `json:"present"`
|
||||||
|
SizeMatch bool `json:"size_match"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// SnapshotSummary is a subset of official-sync-snapshot.json / catalog.status.
|
||||||
|
type SnapshotSummary struct {
|
||||||
|
AppVersion string `json:"app_version,omitempty"`
|
||||||
|
BundleVersion string `json:"bundle_version,omitempty"`
|
||||||
|
ConnectionGroupName string `json:"connection_group_name,omitempty"`
|
||||||
|
AddressablesRoot string `json:"addressables_root,omitempty"`
|
||||||
|
VersionID string `json:"version_id,omitempty"`
|
||||||
|
CompletedUnixSeconds *uint64 `json:"completed_unix_seconds,omitempty"`
|
||||||
|
LauncherMetadata *LauncherMetadataSummary `json:"launcher_metadata,omitempty"`
|
||||||
|
GameMainConfig *GameMainConfigSummary `json:"game_main_config,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// LauncherMetadataSummary mirrors the resource-relevant part of Rust's launcher metadata snapshot.
|
||||||
|
type LauncherMetadataSummary struct {
|
||||||
|
LauncherVersion string `json:"launcher_version,omitempty"`
|
||||||
|
GameLatestVersion string `json:"game_latest_version,omitempty"`
|
||||||
|
GameLatestFilePath string `json:"game_latest_file_path,omitempty"`
|
||||||
|
GameLowestVersion string `json:"game_lowest_version,omitempty"`
|
||||||
|
GameStartExeName string `json:"game_start_exe_name,omitempty"`
|
||||||
|
GameStartParams []string `json:"game_start_params,omitempty"`
|
||||||
|
ManifestURL string `json:"manifest_url,omitempty"`
|
||||||
|
ManifestSource string `json:"manifest_source,omitempty"`
|
||||||
|
ManifestFileCount int `json:"manifest_file_count,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// GameMainConfigSummary mirrors the resource-relevant decrypted GameMainConfig fields.
|
||||||
|
type GameMainConfigSummary struct {
|
||||||
|
ServerInfoDataURL string `json:"server_info_data_url,omitempty"`
|
||||||
|
DefaultConnectionGroup string `json:"default_connection_group,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ReleaseIndex is the in-memory view of a published resource root.
|
||||||
|
type ReleaseIndex struct {
|
||||||
|
mu sync.RWMutex
|
||||||
|
|
||||||
|
ResourceRoot string `json:"resource_root"`
|
||||||
|
Source string `json:"source"` // "rpc" | "resource_root" | "empty"
|
||||||
|
RPCAvailable bool `json:"rpc_available"`
|
||||||
|
DoctorHealthy *bool `json:"doctor_healthy,omitempty"`
|
||||||
|
Snapshot *SnapshotSummary `json:"snapshot,omitempty"`
|
||||||
|
ManifestVersion int `json:"manifest_version,omitempty"`
|
||||||
|
Entries []ResourceEntry `json:"entries"`
|
||||||
|
// byRel maps relative path (host/path...) to entry index.
|
||||||
|
byRel map[string]int
|
||||||
|
// MissingOnDisk lists relative paths present in the index but absent on disk.
|
||||||
|
MissingOnDisk []string `json:"missing_on_disk,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Summary returns a JSON-serializable overview without the full entry list.
|
||||||
|
type ReleaseSummary struct {
|
||||||
|
ResourceRoot string `json:"resource_root"`
|
||||||
|
Source string `json:"source"`
|
||||||
|
RPCAvailable bool `json:"rpc_available"`
|
||||||
|
DoctorHealthy *bool `json:"doctor_healthy,omitempty"`
|
||||||
|
Snapshot *SnapshotSummary `json:"snapshot,omitempty"`
|
||||||
|
ManifestVersion int `json:"manifest_version,omitempty"`
|
||||||
|
EntryCount int `json:"entry_count"`
|
||||||
|
PresentCount int `json:"present_count"`
|
||||||
|
MissingCount int `json:"missing_count"`
|
||||||
|
Ready bool `json:"ready"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Summary builds a compact release overview.
|
||||||
|
func (idx *ReleaseIndex) Summary() ReleaseSummary {
|
||||||
|
idx.mu.RLock()
|
||||||
|
defer idx.mu.RUnlock()
|
||||||
|
present := 0
|
||||||
|
for _, e := range idx.Entries {
|
||||||
|
if e.Present && e.SizeMatch {
|
||||||
|
present++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ReleaseSummary{
|
||||||
|
ResourceRoot: idx.ResourceRoot,
|
||||||
|
Source: idx.Source,
|
||||||
|
RPCAvailable: idx.RPCAvailable,
|
||||||
|
DoctorHealthy: idx.DoctorHealthy,
|
||||||
|
Snapshot: idx.Snapshot,
|
||||||
|
ManifestVersion: idx.ManifestVersion,
|
||||||
|
EntryCount: len(idx.Entries),
|
||||||
|
PresentCount: present,
|
||||||
|
MissingCount: len(idx.MissingOnDisk),
|
||||||
|
Ready: idx.ResourceRoot != "" && present > 0,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Lookup returns the entry for a host/path relative path.
|
||||||
|
func (idx *ReleaseIndex) Lookup(rel string) (ResourceEntry, bool) {
|
||||||
|
idx.mu.RLock()
|
||||||
|
defer idx.mu.RUnlock()
|
||||||
|
rel = filepath.ToSlash(rel)
|
||||||
|
rel = strings.TrimPrefix(rel, "/")
|
||||||
|
i, ok := idx.byRel[rel]
|
||||||
|
if !ok {
|
||||||
|
return ResourceEntry{}, false
|
||||||
|
}
|
||||||
|
return idx.Entries[i], true
|
||||||
|
}
|
||||||
|
|
||||||
|
// List returns a stable page of entries.
|
||||||
|
func (idx *ReleaseIndex) List(offset, limit int) (items []ResourceEntry, total int) {
|
||||||
|
idx.mu.RLock()
|
||||||
|
defer idx.mu.RUnlock()
|
||||||
|
total = len(idx.Entries)
|
||||||
|
if offset < 0 {
|
||||||
|
offset = 0
|
||||||
|
}
|
||||||
|
if limit <= 0 {
|
||||||
|
limit = 100
|
||||||
|
}
|
||||||
|
if offset >= total {
|
||||||
|
return []ResourceEntry{}, total
|
||||||
|
}
|
||||||
|
end := offset + limit
|
||||||
|
if end > total {
|
||||||
|
end = total
|
||||||
|
}
|
||||||
|
out := make([]ResourceEntry, end-offset)
|
||||||
|
copy(out, idx.Entries[offset:end])
|
||||||
|
return out, total
|
||||||
|
}
|
||||||
|
|
||||||
|
// BuildIndexFromManifestEntries builds an index against resourceRoot.
|
||||||
|
func BuildIndexFromManifestEntries(
|
||||||
|
resourceRoot string,
|
||||||
|
source string,
|
||||||
|
rpcAvailable bool,
|
||||||
|
doctorHealthy *bool,
|
||||||
|
snapshot *SnapshotSummary,
|
||||||
|
manifestVersion int,
|
||||||
|
entries []manifestEntry,
|
||||||
|
) (*ReleaseIndex, error) {
|
||||||
|
rootAbs, err := filepath.Abs(resourceRoot)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
idx := &ReleaseIndex{
|
||||||
|
ResourceRoot: rootAbs,
|
||||||
|
Source: source,
|
||||||
|
RPCAvailable: rpcAvailable,
|
||||||
|
DoctorHealthy: doctorHealthy,
|
||||||
|
Snapshot: snapshot,
|
||||||
|
ManifestVersion: manifestVersion,
|
||||||
|
byRel: make(map[string]int),
|
||||||
|
}
|
||||||
|
sort.Slice(entries, func(i, j int) bool {
|
||||||
|
return entries[i].URL < entries[j].URL
|
||||||
|
})
|
||||||
|
for _, raw := range entries {
|
||||||
|
rel := filepath.ToSlash(raw.Destination)
|
||||||
|
rel = strings.TrimPrefix(rel, "/")
|
||||||
|
if rel == "" {
|
||||||
|
// Derive from URL when destination is empty.
|
||||||
|
rel = relativePathFromURL(raw.URL)
|
||||||
|
}
|
||||||
|
if rel == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
entry := ResourceEntry{
|
||||||
|
URL: raw.URL,
|
||||||
|
RelativePath: rel,
|
||||||
|
Bytes: raw.Bytes,
|
||||||
|
BLAKE3: raw.BLAKE3,
|
||||||
|
}
|
||||||
|
abs, err := ResolveUnderRoot(rootAbs, rel, false)
|
||||||
|
if err != nil {
|
||||||
|
entry.Present = false
|
||||||
|
entry.SizeMatch = false
|
||||||
|
idx.MissingOnDisk = append(idx.MissingOnDisk, rel)
|
||||||
|
} else {
|
||||||
|
info, err := os.Lstat(abs)
|
||||||
|
if err != nil || !info.Mode().IsRegular() {
|
||||||
|
entry.Present = false
|
||||||
|
entry.SizeMatch = false
|
||||||
|
idx.MissingOnDisk = append(idx.MissingOnDisk, rel)
|
||||||
|
} else {
|
||||||
|
entry.Present = true
|
||||||
|
entry.SizeMatch = uint64(info.Size()) == raw.Bytes || raw.Bytes == 0
|
||||||
|
if !entry.SizeMatch {
|
||||||
|
idx.MissingOnDisk = append(idx.MissingOnDisk, rel+"#size_mismatch")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
idx.byRel[rel] = len(idx.Entries)
|
||||||
|
idx.Entries = append(idx.Entries, entry)
|
||||||
|
}
|
||||||
|
return idx, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// LoadIndexFromResourceRoot reads official-download-manifest.json under root.
|
||||||
|
// Used for tests and --resource-root fallback without RPC.
|
||||||
|
func LoadIndexFromResourceRoot(resourceRoot string) (*ReleaseIndex, error) {
|
||||||
|
rootAbs, err := filepath.Abs(resourceRoot)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
manifestPath := filepath.Join(rootAbs, "official-download-manifest.json")
|
||||||
|
data, err := os.ReadFile(manifestPath)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("read download manifest: %w", err)
|
||||||
|
}
|
||||||
|
var manifest fileManifest
|
||||||
|
if err := json.Unmarshal(data, &manifest); err != nil {
|
||||||
|
return nil, fmt.Errorf("parse download manifest: %w", err)
|
||||||
|
}
|
||||||
|
if manifest.Version != 0 && manifest.Version != 1 {
|
||||||
|
return nil, fmt.Errorf("unsupported download manifest version %d", manifest.Version)
|
||||||
|
}
|
||||||
|
entries := make([]manifestEntry, 0, len(manifest.Entries))
|
||||||
|
for _, e := range manifest.Entries {
|
||||||
|
entries = append(entries, manifestEntry{
|
||||||
|
URL: e.URL,
|
||||||
|
Destination: e.Destination,
|
||||||
|
Bytes: e.Bytes,
|
||||||
|
BLAKE3: e.BLAKE3,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
var snapshot *SnapshotSummary
|
||||||
|
if snapData, err := os.ReadFile(filepath.Join(rootAbs, "official-sync-snapshot.json")); err == nil {
|
||||||
|
var s fileSnapshot
|
||||||
|
if json.Unmarshal(snapData, &s) == nil {
|
||||||
|
snapshot = &SnapshotSummary{
|
||||||
|
AppVersion: s.AppVersion,
|
||||||
|
BundleVersion: s.BundleVersion,
|
||||||
|
ConnectionGroupName: s.ConnectionGroupName,
|
||||||
|
AddressablesRoot: s.AddressablesRoot,
|
||||||
|
LauncherMetadata: s.LauncherMetadata,
|
||||||
|
GameMainConfig: s.GameMainConfigBootstrap,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return BuildIndexFromManifestEntries(rootAbs, "resource_root", false, nil, snapshot, manifest.Version, entries)
|
||||||
|
}
|
||||||
|
|
||||||
|
type manifestEntry struct {
|
||||||
|
URL string
|
||||||
|
Destination string
|
||||||
|
Bytes uint64
|
||||||
|
BLAKE3 string
|
||||||
|
}
|
||||||
|
|
||||||
|
type fileManifest struct {
|
||||||
|
Version int `json:"version"`
|
||||||
|
Entries map[string]struct {
|
||||||
|
URL string `json:"url"`
|
||||||
|
Destination string `json:"destination"`
|
||||||
|
Bytes uint64 `json:"bytes"`
|
||||||
|
BLAKE3 string `json:"blake3"`
|
||||||
|
} `json:"entries"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type fileSnapshot struct {
|
||||||
|
AppVersion string `json:"app_version"`
|
||||||
|
BundleVersion string `json:"bundle_version"`
|
||||||
|
ConnectionGroupName string `json:"connection_group_name"`
|
||||||
|
AddressablesRoot string `json:"addressables_root"`
|
||||||
|
LauncherMetadata *LauncherMetadataSummary `json:"launcher_metadata,omitempty"`
|
||||||
|
GameMainConfigBootstrap *GameMainConfigSummary `json:"game_main_config_bootstrap,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func relativePathFromURL(rawURL string) string {
|
||||||
|
const prefix = "https://"
|
||||||
|
if !strings.HasPrefix(rawURL, prefix) {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
rest := strings.TrimPrefix(rawURL, prefix)
|
||||||
|
rest = strings.SplitN(rest, "?", 2)[0]
|
||||||
|
rest = strings.SplitN(rest, "#", 2)[0]
|
||||||
|
return filepath.ToSlash(rest)
|
||||||
|
}
|
||||||
@@ -0,0 +1,174 @@
|
|||||||
|
package api
|
||||||
|
|
||||||
|
import "net/http"
|
||||||
|
|
||||||
|
type ErrorResponse struct {
|
||||||
|
Error ErrorBody `json:"error"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type ErrorBody struct {
|
||||||
|
Code string `json:"code"`
|
||||||
|
Message string `json:"message"`
|
||||||
|
Status int `json:"status"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type RefreshDiagnostics struct {
|
||||||
|
InProgress bool `json:"in_progress"`
|
||||||
|
LastAttemptUnixSeconds *int64 `json:"last_attempt_unix_seconds"`
|
||||||
|
LastFinishedUnixSeconds *int64 `json:"last_finished_unix_seconds"`
|
||||||
|
LastSuccessUnixSeconds *int64 `json:"last_success_unix_seconds"`
|
||||||
|
LastDurationMilliseconds int64 `json:"last_duration_milliseconds"`
|
||||||
|
LastError string `json:"last_error"`
|
||||||
|
LastWarningCount int `json:"last_warning_count"`
|
||||||
|
LastWarnings []string `json:"last_warnings"`
|
||||||
|
RefreshIntervalSeconds int64 `json:"refresh_interval_seconds"`
|
||||||
|
ResourceRootOverrideActive bool `json:"resource_root_override_active"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type BootstrapResponse struct {
|
||||||
|
Service string `json:"service"`
|
||||||
|
Ready bool `json:"ready"`
|
||||||
|
Bat BootstrapBat `json:"bat"`
|
||||||
|
BatAPI BootstrapAPI `json:"bat_api"`
|
||||||
|
Resource BootstrapResource `json:"resource"`
|
||||||
|
Policy BootstrapPolicy `json:"policy"`
|
||||||
|
Endpoints []string `json:"endpoints"`
|
||||||
|
Warnings []string `json:"warnings"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type BootstrapBat struct {
|
||||||
|
Role string `json:"role"`
|
||||||
|
Socket string `json:"socket"`
|
||||||
|
RPCAvailable bool `json:"rpc_available"`
|
||||||
|
DoctorHealthy *bool `json:"doctor_healthy,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type BootstrapAPI struct {
|
||||||
|
Role string `json:"role"`
|
||||||
|
PublicBase string `json:"public_base"`
|
||||||
|
RefreshIntervalSeconds int64 `json:"refresh_interval_seconds"`
|
||||||
|
Refresh RefreshDiagnostics `json:"refresh"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type BootstrapResource struct {
|
||||||
|
Release *SnapshotSummary `json:"release,omitempty"`
|
||||||
|
ResourceRoot string `json:"resource_root"`
|
||||||
|
Source string `json:"source"`
|
||||||
|
ManifestVersion int `json:"manifest_version,omitempty"`
|
||||||
|
EntryCount int `json:"entry_count"`
|
||||||
|
PresentCount int `json:"present_count"`
|
||||||
|
MissingCount int `json:"missing_count"`
|
||||||
|
ServerInfoURL string `json:"server_info_url"`
|
||||||
|
ClientPatchBaseURL string `json:"client_patch_base_url"`
|
||||||
|
AddressablesCatalogURLRoot string `json:"addressables_catalog_url_root,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type BootstrapPolicy struct {
|
||||||
|
PullOwner string `json:"pull_owner"`
|
||||||
|
ResourceRootDiscovery string `json:"resource_root_discovery"`
|
||||||
|
Deployment string `json:"deployment"`
|
||||||
|
ResourceRootOverride bool `json:"resource_root_override"`
|
||||||
|
ServesOnlyPublishedRelease bool `json:"serves_only_published_release"`
|
||||||
|
WritesReleaseState bool `json:"writes_release_state"`
|
||||||
|
FullGameBusinessAPI bool `json:"full_game_business_api"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type RootResponse struct {
|
||||||
|
Service string `json:"service"`
|
||||||
|
Role string `json:"role"`
|
||||||
|
Note string `json:"note"`
|
||||||
|
Endpoints []string `json:"endpoints"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type ResourceListResponse struct {
|
||||||
|
Total int `json:"total"`
|
||||||
|
Offset int `json:"offset"`
|
||||||
|
Limit int `json:"limit"`
|
||||||
|
Items []ResourceEntry `json:"items"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type LauncherBootstrapResponse struct {
|
||||||
|
Service string `json:"service"`
|
||||||
|
Ready bool `json:"ready"`
|
||||||
|
LauncherAPI LauncherAPIInfo `json:"launcher_api"`
|
||||||
|
Policy LauncherPolicy `json:"policy"`
|
||||||
|
Resource LauncherResource `json:"resource"`
|
||||||
|
Endpoints LauncherEndpointSet `json:"endpoints"`
|
||||||
|
GameMainConfig *GameMainConfigSummary `json:"game_main_config,omitempty"`
|
||||||
|
LauncherMetadata *LauncherMetadataSummary `json:"launcher_metadata,omitempty"`
|
||||||
|
AddressablesCatalogURLRoot string `json:"addressables_catalog_url_root,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type LauncherAPIInfo struct {
|
||||||
|
Host string `json:"host"`
|
||||||
|
ObservedResourceRelevantEndpoints []string `json:"observed_resource_relevant_endpoints"`
|
||||||
|
CompatibilityScope string `json:"compatibility_scope"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type LauncherPolicy struct {
|
||||||
|
Source string `json:"source"`
|
||||||
|
DownloadsLauncherPackage bool `json:"downloads_launcher_package"`
|
||||||
|
EmulatesLoginOrGateway bool `json:"emulates_login_or_gateway"`
|
||||||
|
PackageUpdateManifest bool `json:"package_update_manifest"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type LauncherResource struct {
|
||||||
|
Release *SnapshotSummary `json:"release,omitempty"`
|
||||||
|
ServerInfoURL string `json:"server_info_url"`
|
||||||
|
ClientPatchBaseURL string `json:"client_patch_base_url"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type LauncherEndpointSet struct {
|
||||||
|
Bootstrap string `json:"bootstrap"`
|
||||||
|
LauncherBootstrap string `json:"launcher_bootstrap"`
|
||||||
|
ServerInfo string `json:"server_info"`
|
||||||
|
ClientPatchBase string `json:"client_patch_base"`
|
||||||
|
LauncherGameConfig string `json:"launcher_game_config"`
|
||||||
|
LauncherGameConfigJSON string `json:"launcher_game_config_json"`
|
||||||
|
LauncherCdnConfig string `json:"launcher_cdn_config"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type LauncherGameConfigData struct {
|
||||||
|
GameLatestVersion string `json:"game_latest_version"`
|
||||||
|
GameLatestFilePath string `json:"game_latest_file_path"`
|
||||||
|
GameLowestVersion string `json:"game_lowest_version,omitempty"`
|
||||||
|
GameStartExeName string `json:"game_start_exe_name"`
|
||||||
|
GameStartParams []string `json:"game_start_params"`
|
||||||
|
ResourceBootstrapURL string `json:"resource_bootstrap_url"`
|
||||||
|
ServerInfoURL string `json:"server_info_url"`
|
||||||
|
ClientPatchBaseURL string `json:"client_patch_base_url"`
|
||||||
|
Scope string `json:"scope"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type LauncherGameConfigJSONData struct {
|
||||||
|
URL string `json:"url"`
|
||||||
|
ResourceBootstrapURL string `json:"resource_bootstrap_url"`
|
||||||
|
PackageUpdateManifest bool `json:"package_update_manifest"`
|
||||||
|
Scope string `json:"scope"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type LauncherCdnConfigData struct {
|
||||||
|
PrimaryCDN string `json:"primary_cdn"`
|
||||||
|
BackupCDN string `json:"back_up_cdn"`
|
||||||
|
ResourceBootstrapURL string `json:"resource_bootstrap_url"`
|
||||||
|
PackageUpdateManifest bool `json:"package_update_manifest"`
|
||||||
|
Scope string `json:"scope"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type LauncherEnvelope[T any] struct {
|
||||||
|
Code int `json:"code"`
|
||||||
|
Message string `json:"message"`
|
||||||
|
Data T `json:"data"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type AdminIndexResponse struct {
|
||||||
|
Service string `json:"service"`
|
||||||
|
Panel string `json:"panel"`
|
||||||
|
Status string `json:"status"`
|
||||||
|
Links []string `json:"links"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeNoStoreJSON(w http.ResponseWriter, status int, body any) {
|
||||||
|
w.Header().Set("Cache-Control", "no-store")
|
||||||
|
writeJSON(w, status, body)
|
||||||
|
}
|
||||||
@@ -0,0 +1,269 @@
|
|||||||
|
package api
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"path/filepath"
|
||||||
|
|
||||||
|
"bat-api/internal/backendrpc"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Backend is the subset of daemon RPC used by bat-api.
|
||||||
|
//
|
||||||
|
// Call order for discovery (per plan review):
|
||||||
|
// 1. daemon.status
|
||||||
|
// 2. daemon.doctor
|
||||||
|
// 3. catalog.status / resource.manifest (and resource.state as needed)
|
||||||
|
type Backend interface {
|
||||||
|
DaemonStatus(ctx context.Context) (*backendrpc.DaemonStatusReport, error)
|
||||||
|
DaemonDoctor(ctx context.Context) (*backendrpc.DoctorReport, error)
|
||||||
|
ResourceState(ctx context.Context) (*backendrpc.ResourceState, error)
|
||||||
|
CatalogStatus(ctx context.Context) (json.RawMessage, error)
|
||||||
|
ResourceManifest(ctx context.Context, offset int, limit int) (*backendrpc.ResourceManifestPage, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// RPCClient adapts *backendrpc.Client to Backend.
|
||||||
|
type RPCClient struct {
|
||||||
|
Client *backendrpc.Client
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r RPCClient) DaemonStatus(ctx context.Context) (*backendrpc.DaemonStatusReport, error) {
|
||||||
|
return r.Client.DaemonStatus(ctx)
|
||||||
|
}
|
||||||
|
func (r RPCClient) DaemonDoctor(ctx context.Context) (*backendrpc.DoctorReport, error) {
|
||||||
|
return r.Client.DaemonDoctor(ctx)
|
||||||
|
}
|
||||||
|
func (r RPCClient) ResourceState(ctx context.Context) (*backendrpc.ResourceState, error) {
|
||||||
|
return r.Client.ResourceState(ctx)
|
||||||
|
}
|
||||||
|
func (r RPCClient) CatalogStatus(ctx context.Context) (json.RawMessage, error) {
|
||||||
|
return r.Client.CatalogStatus(ctx)
|
||||||
|
}
|
||||||
|
func (r RPCClient) ResourceManifest(ctx context.Context, offset int, limit int) (*backendrpc.ResourceManifestPage, error) {
|
||||||
|
return r.Client.ResourceManifest(ctx, offset, limit)
|
||||||
|
}
|
||||||
|
|
||||||
|
// DiscoverResult is the outcome of talking to the bat daemon.
|
||||||
|
type DiscoverResult struct {
|
||||||
|
RPCAvailable bool
|
||||||
|
DoctorHealthy *bool
|
||||||
|
Status *backendrpc.DaemonStatusReport
|
||||||
|
Doctor *backendrpc.DoctorReport
|
||||||
|
Snapshot *SnapshotSummary
|
||||||
|
ResourceRoot string
|
||||||
|
Index *ReleaseIndex
|
||||||
|
Warnings []string
|
||||||
|
}
|
||||||
|
|
||||||
|
// DiscoverAndIndex contacts the daemon (status first, then doctor) and builds
|
||||||
|
// a release index from paginated resource.manifest plus on-disk checks.
|
||||||
|
//
|
||||||
|
// If resourceRootOverride is non-empty, it wins over RPC-reported roots after
|
||||||
|
// RPC health probes (still preferred for production to call status/doctor).
|
||||||
|
func DiscoverAndIndex(ctx context.Context, backend Backend, resourceRootOverride string) (*DiscoverResult, error) {
|
||||||
|
out := &DiscoverResult{}
|
||||||
|
if backend == nil {
|
||||||
|
if resourceRootOverride == "" {
|
||||||
|
out.Index = &ReleaseIndex{Source: "empty", byRel: map[string]int{}}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
idx, err := LoadIndexFromResourceRoot(resourceRootOverride)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
out.ResourceRoot = idx.ResourceRoot
|
||||||
|
out.Index = idx
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// 1) daemon.status first
|
||||||
|
status, err := backend.DaemonStatus(ctx)
|
||||||
|
if err != nil {
|
||||||
|
out.Warnings = append(out.Warnings, fmt.Sprintf("daemon.status: %v", err))
|
||||||
|
if resourceRootOverride != "" {
|
||||||
|
idx, loadErr := LoadIndexFromResourceRoot(resourceRootOverride)
|
||||||
|
if loadErr != nil {
|
||||||
|
return out, fmt.Errorf("daemon.status failed (%v) and resource-root load failed: %w", err, loadErr)
|
||||||
|
}
|
||||||
|
out.ResourceRoot = idx.ResourceRoot
|
||||||
|
out.Index = idx
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
out.Index = &ReleaseIndex{Source: "empty", byRel: map[string]int{}}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
out.RPCAvailable = true
|
||||||
|
out.Status = status
|
||||||
|
|
||||||
|
// 2) daemon.doctor second
|
||||||
|
doctor, err := backend.DaemonDoctor(ctx)
|
||||||
|
if err != nil {
|
||||||
|
out.Warnings = append(out.Warnings, fmt.Sprintf("daemon.doctor: %v", err))
|
||||||
|
} else {
|
||||||
|
out.Doctor = doctor
|
||||||
|
h := doctor.Healthy
|
||||||
|
out.DoctorHealthy = &h
|
||||||
|
}
|
||||||
|
|
||||||
|
// Catalog / resource discovery
|
||||||
|
var snapshot *SnapshotSummary
|
||||||
|
var resourceRoot string
|
||||||
|
|
||||||
|
if raw, err := backend.CatalogStatus(ctx); err != nil {
|
||||||
|
out.Warnings = append(out.Warnings, fmt.Sprintf("catalog.status: %v", err))
|
||||||
|
} else {
|
||||||
|
snap, root, ok := parseCatalogStatus(raw)
|
||||||
|
if ok {
|
||||||
|
snapshot = snap
|
||||||
|
resourceRoot = root
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if resourceRoot == "" {
|
||||||
|
if state, err := backend.ResourceState(ctx); err != nil {
|
||||||
|
out.Warnings = append(out.Warnings, fmt.Sprintf("resource.state: %v", err))
|
||||||
|
} else if state.ResourceOutputRoot != nil && *state.ResourceOutputRoot != "" {
|
||||||
|
// Prefer published current under output root when catalog root missing.
|
||||||
|
candidate := filepath.Join(*state.ResourceOutputRoot, "current")
|
||||||
|
resourceRoot = candidate
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if resourceRootOverride != "" {
|
||||||
|
resourceRoot = resourceRootOverride
|
||||||
|
}
|
||||||
|
if resourceRoot == "" {
|
||||||
|
out.Snapshot = snapshot
|
||||||
|
out.Index = &ReleaseIndex{
|
||||||
|
Source: "rpc",
|
||||||
|
RPCAvailable: true,
|
||||||
|
DoctorHealthy: out.DoctorHealthy,
|
||||||
|
Snapshot: snapshot,
|
||||||
|
byRel: map[string]int{},
|
||||||
|
}
|
||||||
|
out.Warnings = append(out.Warnings, "no resource root from RPC; set --resource-root or publish a version")
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
entries, manifestVersion, rootFromManifest, err := fetchAllManifestEntries(ctx, backend)
|
||||||
|
if err != nil {
|
||||||
|
out.Warnings = append(out.Warnings, fmt.Sprintf("resource.manifest: %v", err))
|
||||||
|
// Fallback: load local manifest file under root.
|
||||||
|
idx, loadErr := LoadIndexFromResourceRoot(resourceRoot)
|
||||||
|
if loadErr != nil {
|
||||||
|
return out, fmt.Errorf("manifest RPC and local load failed: rpc=%v local=%w", err, loadErr)
|
||||||
|
}
|
||||||
|
idx.Source = "rpc+local_manifest"
|
||||||
|
idx.RPCAvailable = true
|
||||||
|
idx.DoctorHealthy = out.DoctorHealthy
|
||||||
|
if snapshot != nil {
|
||||||
|
idx.Snapshot = snapshot
|
||||||
|
}
|
||||||
|
out.ResourceRoot = idx.ResourceRoot
|
||||||
|
out.Snapshot = idx.Snapshot
|
||||||
|
out.Index = idx
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
if rootFromManifest != "" {
|
||||||
|
resourceRoot = rootFromManifest
|
||||||
|
}
|
||||||
|
if resourceRootOverride != "" {
|
||||||
|
resourceRoot = resourceRootOverride
|
||||||
|
}
|
||||||
|
|
||||||
|
idx, err := BuildIndexFromManifestEntries(
|
||||||
|
resourceRoot,
|
||||||
|
"rpc",
|
||||||
|
true,
|
||||||
|
out.DoctorHealthy,
|
||||||
|
snapshot,
|
||||||
|
manifestVersion,
|
||||||
|
entries,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return out, err
|
||||||
|
}
|
||||||
|
out.ResourceRoot = idx.ResourceRoot
|
||||||
|
out.Snapshot = snapshot
|
||||||
|
out.Index = idx
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseCatalogStatus(raw json.RawMessage) (*SnapshotSummary, string, bool) {
|
||||||
|
if len(raw) == 0 || string(raw) == "null" {
|
||||||
|
return nil, "", false
|
||||||
|
}
|
||||||
|
var payload struct {
|
||||||
|
Available bool `json:"available"`
|
||||||
|
AppVersion string `json:"app_version"`
|
||||||
|
BundleVersion string `json:"bundle_version"`
|
||||||
|
ConnectionGroupName string `json:"connection_group_name"`
|
||||||
|
AddressablesRoot string `json:"addressables_root"`
|
||||||
|
LauncherMetadata *LauncherMetadataSummary `json:"launcher_metadata"`
|
||||||
|
GameMainConfig *GameMainConfigSummary `json:"game_main_config"`
|
||||||
|
Version *struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
CompletedUnixSeconds *uint64 `json:"completed_unix_seconds"`
|
||||||
|
ResourceRoot string `json:"resource_root"`
|
||||||
|
} `json:"version"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(raw, &payload); err != nil || !payload.Available {
|
||||||
|
return nil, "", false
|
||||||
|
}
|
||||||
|
snap := &SnapshotSummary{
|
||||||
|
AppVersion: payload.AppVersion,
|
||||||
|
BundleVersion: payload.BundleVersion,
|
||||||
|
ConnectionGroupName: payload.ConnectionGroupName,
|
||||||
|
AddressablesRoot: payload.AddressablesRoot,
|
||||||
|
LauncherMetadata: payload.LauncherMetadata,
|
||||||
|
GameMainConfig: payload.GameMainConfig,
|
||||||
|
}
|
||||||
|
root := ""
|
||||||
|
if payload.Version != nil {
|
||||||
|
snap.VersionID = payload.Version.ID
|
||||||
|
snap.CompletedUnixSeconds = payload.Version.CompletedUnixSeconds
|
||||||
|
root = payload.Version.ResourceRoot
|
||||||
|
}
|
||||||
|
return snap, root, true
|
||||||
|
}
|
||||||
|
|
||||||
|
func fetchAllManifestEntries(ctx context.Context, backend Backend) ([]manifestEntry, int, string, error) {
|
||||||
|
const pageSize = 500
|
||||||
|
offset := 0
|
||||||
|
var all []manifestEntry
|
||||||
|
var version int
|
||||||
|
var root string
|
||||||
|
for {
|
||||||
|
page, err := backend.ResourceManifest(ctx, offset, pageSize)
|
||||||
|
if err != nil {
|
||||||
|
return nil, 0, "", err
|
||||||
|
}
|
||||||
|
if !page.Available {
|
||||||
|
return nil, 0, "", fmt.Errorf("resource.manifest available=false")
|
||||||
|
}
|
||||||
|
if root == "" {
|
||||||
|
root = page.ResourceRoot
|
||||||
|
}
|
||||||
|
if version == 0 {
|
||||||
|
version = page.ManifestVersion
|
||||||
|
}
|
||||||
|
for _, e := range page.Entries {
|
||||||
|
var bytes uint64
|
||||||
|
if e.Bytes != nil {
|
||||||
|
bytes = *e.Bytes
|
||||||
|
}
|
||||||
|
all = append(all, manifestEntry{
|
||||||
|
URL: e.URL,
|
||||||
|
Destination: e.Destination,
|
||||||
|
Bytes: bytes,
|
||||||
|
BLAKE3: e.BLAKE3,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
offset += len(page.Entries)
|
||||||
|
if len(page.Entries) == 0 || offset >= page.TotalEntries {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return all, version, root, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,483 @@
|
|||||||
|
package api
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"log"
|
||||||
|
"net/http"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Server is the bat-api HTTP server for resource bootstrap and distribution.
|
||||||
|
type Server struct {
|
||||||
|
cfg Config
|
||||||
|
backend Backend
|
||||||
|
logger *log.Logger
|
||||||
|
limiter *tokenBucketLimiter
|
||||||
|
|
||||||
|
mu sync.RWMutex
|
||||||
|
idx *ReleaseIndex
|
||||||
|
meta DiscoverResult
|
||||||
|
|
||||||
|
refreshInProgress bool
|
||||||
|
lastRefreshStart time.Time
|
||||||
|
lastRefreshFinish time.Time
|
||||||
|
lastRefreshOK time.Time
|
||||||
|
lastRefreshError string
|
||||||
|
lastRefreshWarns []string
|
||||||
|
lastRefreshDur time.Duration
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewServer constructs a server. Call Refresh before Listen when possible.
|
||||||
|
func NewServer(cfg Config, backend Backend, logger *log.Logger) *Server {
|
||||||
|
if logger == nil {
|
||||||
|
logger = log.Default()
|
||||||
|
}
|
||||||
|
var limiter *tokenBucketLimiter
|
||||||
|
if cfg.RateLimitRPS > 0 {
|
||||||
|
limiter = newTokenBucketLimiter(cfg.RateLimitRPS, cfg.RateLimitBurst)
|
||||||
|
}
|
||||||
|
return &Server{cfg: cfg, backend: backend, logger: logger, limiter: limiter}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Handler returns the root HTTP handler.
|
||||||
|
func (s *Server) Handler() http.Handler {
|
||||||
|
mux := http.NewServeMux()
|
||||||
|
mux.HandleFunc("/healthz", s.handleHealthz)
|
||||||
|
mux.HandleFunc("/readyz", s.handleReadyz)
|
||||||
|
mux.HandleFunc("/v1/bootstrap", s.handleBootstrap)
|
||||||
|
mux.HandleFunc("/v1/launcher/bootstrap", s.handleLauncherBootstrap)
|
||||||
|
mux.HandleFunc("/v1/release", s.handleRelease)
|
||||||
|
mux.HandleFunc("/v1/resources", s.handleResources)
|
||||||
|
mux.HandleFunc("/v1/server-info", s.handleServerInfoDebug)
|
||||||
|
mux.HandleFunc("/api/launcher/game/config", s.handleLauncherGameConfig)
|
||||||
|
mux.HandleFunc("/api/launcher/game/config/json", s.handleLauncherGameConfigJSON)
|
||||||
|
mux.HandleFunc("/api/launcher/advanced/game/download/cdn", s.handleLauncherCdnConfig)
|
||||||
|
mux.HandleFunc(launcherHostPath("/api/launcher/game/config"), s.handleLauncherGameConfig)
|
||||||
|
mux.HandleFunc(launcherHostPath("/api/launcher/game/config/json"), s.handleLauncherGameConfigJSON)
|
||||||
|
mux.HandleFunc(launcherHostPath("/api/launcher/advanced/game/download/cdn"), s.handleLauncherCdnConfig)
|
||||||
|
mux.HandleFunc(launcherHostPath("/api/launcher/resource/bootstrap.json"), s.handleLauncherBootstrap)
|
||||||
|
mux.HandleFunc("/openapi.yaml", s.handleOpenAPI)
|
||||||
|
mux.HandleFunc("/admin/", s.handleAdminIndex)
|
||||||
|
mux.HandleFunc("/"+ServerInfoHost+"/", s.handleServerInfoCDN)
|
||||||
|
mux.HandleFunc("/"+ClientPatchHost+"/", s.serveCDN)
|
||||||
|
// Catch-all for other official host prefixes and 404.
|
||||||
|
mux.HandleFunc("/", s.handleRoot)
|
||||||
|
return s.wrapHandler(mux)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Refresh rebuilds the release index via RPC (and optional resource-root override).
|
||||||
|
func (s *Server) Refresh(ctx context.Context) error {
|
||||||
|
started := s.beginRefresh()
|
||||||
|
result, err := DiscoverAndIndex(ctx, s.backend, s.cfg.ResourceRoot)
|
||||||
|
if err != nil {
|
||||||
|
s.finishRefresh(started, err, nil)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
s.mu.Lock()
|
||||||
|
s.meta = *result
|
||||||
|
s.idx = result.Index
|
||||||
|
s.finishRefreshLocked(started, nil, result.Warnings)
|
||||||
|
s.mu.Unlock()
|
||||||
|
for _, w := range result.Warnings {
|
||||||
|
s.logger.Printf("bat-api discover warning: %s", w)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) index() *ReleaseIndex {
|
||||||
|
s.mu.RLock()
|
||||||
|
defer s.mu.RUnlock()
|
||||||
|
return s.idx
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) discoverMeta() DiscoverResult {
|
||||||
|
s.mu.RLock()
|
||||||
|
defer s.mu.RUnlock()
|
||||||
|
return s.meta
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleRoot(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.URL.Path == "/" {
|
||||||
|
writeNoStoreJSON(w, http.StatusOK, RootResponse{
|
||||||
|
Service: "bat-api",
|
||||||
|
Role: "resource_bootstrap_and_distribution",
|
||||||
|
Note: "resource auto-pull is owned by Rust bat; this service bootstraps and distributes published resources",
|
||||||
|
Endpoints: []string{
|
||||||
|
"/healthz",
|
||||||
|
"/readyz",
|
||||||
|
"/v1/bootstrap",
|
||||||
|
"/v1/launcher/bootstrap",
|
||||||
|
"/v1/release",
|
||||||
|
"/v1/resources",
|
||||||
|
"/v1/server-info",
|
||||||
|
"/api/launcher/game/config",
|
||||||
|
"/api/launcher/game/config/json",
|
||||||
|
"/api/launcher/advanced/game/download/cdn",
|
||||||
|
"/" + LauncherAPIHost + "/api/launcher/game/config",
|
||||||
|
"/" + LauncherAPIHost + "/api/launcher/game/config/json",
|
||||||
|
"/" + LauncherAPIHost + "/api/launcher/advanced/game/download/cdn",
|
||||||
|
"/" + ClientPatchHost + "/...",
|
||||||
|
"/" + ServerInfoHost + "/...",
|
||||||
|
"/openapi.yaml",
|
||||||
|
"/admin/",
|
||||||
|
},
|
||||||
|
})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
// Attempt CDN for known hosts not registered above.
|
||||||
|
if strings.HasPrefix(r.URL.Path, "/"+ClientPatchHost+"/") ||
|
||||||
|
strings.HasPrefix(r.URL.Path, "/"+ServerInfoHost+"/") {
|
||||||
|
if strings.HasPrefix(r.URL.Path, "/"+ServerInfoHost+"/") {
|
||||||
|
s.handleServerInfoCDN(w, r)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.serveCDN(w, r)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
http.NotFound(w, r)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleBootstrap(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != http.MethodGet {
|
||||||
|
writeErrorJSON(w, http.StatusMethodNotAllowed, "method_not_allowed", "method not allowed")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
meta := s.discoverMeta()
|
||||||
|
sum := ReleaseSummary{}
|
||||||
|
if idx := s.index(); idx != nil {
|
||||||
|
sum = idx.Summary()
|
||||||
|
}
|
||||||
|
publicBase := strings.TrimRight(s.cfg.PublicBaseURL, "/")
|
||||||
|
addressablesRoot := ""
|
||||||
|
if sum.Snapshot != nil && sum.Snapshot.AddressablesRoot != "" {
|
||||||
|
if rewritten, ok := rewriteAddressablesRoot(sum.Snapshot.AddressablesRoot, publicBase); ok {
|
||||||
|
addressablesRoot = rewritten
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
status := http.StatusOK
|
||||||
|
if !sum.Ready {
|
||||||
|
status = http.StatusServiceUnavailable
|
||||||
|
}
|
||||||
|
writeNoStoreJSON(w, status, BootstrapResponse{
|
||||||
|
Service: "bat-api",
|
||||||
|
Ready: sum.Ready,
|
||||||
|
Bat: BootstrapBat{
|
||||||
|
Role: "sync_daemon_and_release_producer",
|
||||||
|
Socket: s.cfg.SocketPath,
|
||||||
|
RPCAvailable: meta.RPCAvailable || sum.RPCAvailable,
|
||||||
|
DoctorHealthy: sum.DoctorHealthy,
|
||||||
|
},
|
||||||
|
BatAPI: BootstrapAPI{
|
||||||
|
Role: "resource_bootstrap_and_distribution",
|
||||||
|
PublicBase: publicBase,
|
||||||
|
RefreshIntervalSeconds: int64(s.cfg.RefreshInterval.Seconds()),
|
||||||
|
Refresh: s.refreshSnapshot(),
|
||||||
|
},
|
||||||
|
Resource: BootstrapResource{
|
||||||
|
Release: sum.Snapshot,
|
||||||
|
ResourceRoot: sum.ResourceRoot,
|
||||||
|
Source: sum.Source,
|
||||||
|
ManifestVersion: sum.ManifestVersion,
|
||||||
|
EntryCount: sum.EntryCount,
|
||||||
|
PresentCount: sum.PresentCount,
|
||||||
|
MissingCount: sum.MissingCount,
|
||||||
|
ServerInfoURL: publicBase + "/" + ServerInfoHost + "/server-info.json",
|
||||||
|
ClientPatchBaseURL: publicBase + "/" + ClientPatchHost,
|
||||||
|
AddressablesCatalogURLRoot: addressablesRoot,
|
||||||
|
},
|
||||||
|
Policy: BootstrapPolicy{
|
||||||
|
PullOwner: "rust_bat",
|
||||||
|
ResourceRootDiscovery: "bat_sock_rpc",
|
||||||
|
Deployment: "co_located_with_rust_bat_or_shared_filesystem",
|
||||||
|
ResourceRootOverride: s.cfg.ResourceRoot != "",
|
||||||
|
ServesOnlyPublishedRelease: true,
|
||||||
|
WritesReleaseState: false,
|
||||||
|
FullGameBusinessAPI: false,
|
||||||
|
},
|
||||||
|
Endpoints: []string{
|
||||||
|
"/v1/launcher/bootstrap",
|
||||||
|
"/api/launcher/game/config",
|
||||||
|
"/api/launcher/game/config/json",
|
||||||
|
"/api/launcher/advanced/game/download/cdn",
|
||||||
|
"/v1/server-info",
|
||||||
|
"/" + ServerInfoHost + "/server-info.json",
|
||||||
|
"/" + ClientPatchHost + "/...",
|
||||||
|
},
|
||||||
|
Warnings: meta.Warnings,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleHealthz(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != http.MethodGet {
|
||||||
|
writeErrorJSON(w, http.StatusMethodNotAllowed, "method_not_allowed", "method not allowed")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
meta := s.discoverMeta()
|
||||||
|
sum := ReleaseSummary{}
|
||||||
|
if idx := s.index(); idx != nil {
|
||||||
|
sum = idx.Summary()
|
||||||
|
}
|
||||||
|
writeNoStoreJSON(w, http.StatusOK, map[string]any{
|
||||||
|
"ok": true,
|
||||||
|
"service": "bat-api",
|
||||||
|
"listen": s.cfg.Listen,
|
||||||
|
"socket": s.cfg.SocketPath,
|
||||||
|
"public_base": s.cfg.PublicBaseURL,
|
||||||
|
"rpc_available": meta.RPCAvailable || sum.RPCAvailable,
|
||||||
|
"doctor_healthy": sum.DoctorHealthy,
|
||||||
|
"refresh_interval_seconds": int64(s.cfg.RefreshInterval.Seconds()),
|
||||||
|
"refresh": s.refreshSnapshot(),
|
||||||
|
"resource_root": sum.ResourceRoot,
|
||||||
|
"resource_root_override_configured": s.cfg.ResourceRoot != "",
|
||||||
|
"ready": sum.Ready,
|
||||||
|
"entry_count": sum.EntryCount,
|
||||||
|
"present_count": sum.PresentCount,
|
||||||
|
"missing_count": sum.MissingCount,
|
||||||
|
"source": sum.Source,
|
||||||
|
"warnings": meta.Warnings,
|
||||||
|
// Database/redis are reserved config surface for a normal API process.
|
||||||
|
"database_configured": s.cfg.DatabaseURL != "",
|
||||||
|
"redis_configured": s.cfg.RedisURL != "",
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleReadyz(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != http.MethodGet && r.Method != http.MethodHead {
|
||||||
|
writeErrorJSON(w, http.StatusMethodNotAllowed, "method_not_allowed", "method not allowed")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
meta := s.discoverMeta()
|
||||||
|
sum := ReleaseSummary{}
|
||||||
|
if idx := s.index(); idx != nil {
|
||||||
|
sum = idx.Summary()
|
||||||
|
}
|
||||||
|
status := http.StatusOK
|
||||||
|
if !sum.Ready {
|
||||||
|
status = http.StatusServiceUnavailable
|
||||||
|
}
|
||||||
|
body := map[string]any{
|
||||||
|
"ready": sum.Ready,
|
||||||
|
"service": "bat-api",
|
||||||
|
"rpc_available": meta.RPCAvailable || sum.RPCAvailable,
|
||||||
|
"resource_root": sum.ResourceRoot,
|
||||||
|
"entry_count": sum.EntryCount,
|
||||||
|
"present_count": sum.PresentCount,
|
||||||
|
"missing_count": sum.MissingCount,
|
||||||
|
"source": sum.Source,
|
||||||
|
"refresh": s.refreshSnapshot(),
|
||||||
|
}
|
||||||
|
if r.Method == http.MethodHead {
|
||||||
|
w.WriteHeader(status)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
writeNoStoreJSON(w, status, body)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleRelease(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != http.MethodGet {
|
||||||
|
writeErrorJSON(w, http.StatusMethodNotAllowed, "method_not_allowed", "method not allowed")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
idx := s.index()
|
||||||
|
if idx == nil {
|
||||||
|
writeErrorJSON(w, http.StatusServiceUnavailable, "release_not_ready", "release index not ready")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
writeNoStoreJSON(w, http.StatusOK, idx.Summary())
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleResources(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != http.MethodGet {
|
||||||
|
writeErrorJSON(w, http.StatusMethodNotAllowed, "method_not_allowed", "method not allowed")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
idx := s.index()
|
||||||
|
if idx == nil {
|
||||||
|
writeErrorJSON(w, http.StatusServiceUnavailable, "release_not_ready", "index not ready")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
offset, _ := strconv.Atoi(r.URL.Query().Get("offset"))
|
||||||
|
limit, _ := strconv.Atoi(r.URL.Query().Get("limit"))
|
||||||
|
if limit <= 0 {
|
||||||
|
limit = 100
|
||||||
|
}
|
||||||
|
if limit > s.cfg.MaxResourcePageLimit {
|
||||||
|
limit = s.cfg.MaxResourcePageLimit
|
||||||
|
}
|
||||||
|
items, total := idx.List(offset, limit)
|
||||||
|
writeNoStoreJSON(w, http.StatusOK, ResourceListResponse{
|
||||||
|
Total: total,
|
||||||
|
Offset: offset,
|
||||||
|
Limit: limit,
|
||||||
|
Items: items,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleServerInfoDebug(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != http.MethodGet {
|
||||||
|
writeErrorJSON(w, http.StatusMethodNotAllowed, "method_not_allowed", "method not allowed")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.writeServerInfo(w, r, "")
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleServerInfoCDN(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != http.MethodGet && r.Method != http.MethodHead {
|
||||||
|
writeErrorJSON(w, http.StatusMethodNotAllowed, "method_not_allowed", "method not allowed")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
name := strings.TrimPrefix(r.URL.Path, "/"+ServerInfoHost+"/")
|
||||||
|
name = strings.TrimPrefix(name, "/")
|
||||||
|
s.writeServerInfo(w, r, name)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) writeServerInfo(w http.ResponseWriter, r *http.Request, name string) {
|
||||||
|
raw, err := LoadServerInfoBytes(s.cfg, s.index(), name)
|
||||||
|
if err != nil {
|
||||||
|
writeErrorJSON(w, http.StatusNotFound, "server_info_not_found", err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
rewritten, err := RewriteServerInfoAddressables(raw, s.cfg.PublicBaseURL)
|
||||||
|
if err != nil {
|
||||||
|
writeErrorJSON(w, http.StatusInternalServerError, "server_info_invalid", err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
w.Header().Set("Cache-Control", "no-store")
|
||||||
|
w.Header().Set("Content-Type", "application/json; charset=utf-8")
|
||||||
|
w.Header().Set("Content-Length", strconv.Itoa(len(rewritten)))
|
||||||
|
if r.Method == http.MethodHead {
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
_, _ = w.Write(rewritten)
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeJSON(w http.ResponseWriter, status int, body any) {
|
||||||
|
data, err := json.Marshal(body)
|
||||||
|
if err != nil {
|
||||||
|
w.Header().Set("Content-Type", "application/json; charset=utf-8")
|
||||||
|
w.WriteHeader(http.StatusInternalServerError)
|
||||||
|
fallback, _ := json.Marshal(ErrorResponse{
|
||||||
|
Error: ErrorBody{Code: "internal_error", Message: err.Error(), Status: http.StatusInternalServerError},
|
||||||
|
})
|
||||||
|
_, _ = w.Write(fallback)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
w.Header().Set("Content-Type", "application/json; charset=utf-8")
|
||||||
|
w.WriteHeader(status)
|
||||||
|
_, _ = w.Write(data)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) beginRefresh() time.Time {
|
||||||
|
now := time.Now()
|
||||||
|
s.mu.Lock()
|
||||||
|
s.refreshInProgress = true
|
||||||
|
s.lastRefreshStart = now
|
||||||
|
s.mu.Unlock()
|
||||||
|
return now
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) finishRefresh(started time.Time, err error, warnings []string) {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
s.finishRefreshLocked(started, err, warnings)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) finishRefreshLocked(started time.Time, err error, warnings []string) {
|
||||||
|
now := time.Now()
|
||||||
|
s.refreshInProgress = false
|
||||||
|
s.lastRefreshFinish = now
|
||||||
|
s.lastRefreshDur = now.Sub(started)
|
||||||
|
s.lastRefreshWarns = append([]string(nil), warnings...)
|
||||||
|
if err != nil {
|
||||||
|
s.lastRefreshError = err.Error()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.lastRefreshError = ""
|
||||||
|
s.lastRefreshOK = now
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) refreshSnapshot() RefreshDiagnostics {
|
||||||
|
s.mu.RLock()
|
||||||
|
defer s.mu.RUnlock()
|
||||||
|
warnings := append([]string(nil), s.lastRefreshWarns...)
|
||||||
|
return RefreshDiagnostics{
|
||||||
|
InProgress: s.refreshInProgress,
|
||||||
|
LastAttemptUnixSeconds: unixPtr(s.lastRefreshStart),
|
||||||
|
LastFinishedUnixSeconds: unixPtr(s.lastRefreshFinish),
|
||||||
|
LastSuccessUnixSeconds: unixPtr(s.lastRefreshOK),
|
||||||
|
LastDurationMilliseconds: s.lastRefreshDur.Milliseconds(),
|
||||||
|
LastError: s.lastRefreshError,
|
||||||
|
LastWarningCount: len(warnings),
|
||||||
|
LastWarnings: warnings,
|
||||||
|
RefreshIntervalSeconds: int64(s.cfg.RefreshInterval.Seconds()),
|
||||||
|
ResourceRootOverrideActive: s.cfg.ResourceRoot != "",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func unixPtr(t time.Time) *int64 {
|
||||||
|
if t.IsZero() {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
v := t.Unix()
|
||||||
|
return &v
|
||||||
|
}
|
||||||
|
|
||||||
|
// StartRefreshLoop keeps the in-memory release index aligned with the Rust bat daemon.
|
||||||
|
func (s *Server) StartRefreshLoop(ctx context.Context) {
|
||||||
|
if s.cfg.RefreshInterval <= 0 {
|
||||||
|
s.logger.Printf("bat-api periodic release discovery disabled")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
go func() {
|
||||||
|
ticker := time.NewTicker(s.cfg.RefreshInterval)
|
||||||
|
defer ticker.Stop()
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
case <-ticker.C:
|
||||||
|
refreshCtx, cancel := context.WithTimeout(ctx, s.cfg.RPCTimeout+5*time.Second)
|
||||||
|
if err := s.Refresh(refreshCtx); err != nil {
|
||||||
|
s.logger.Printf("periodic discover failed: %v", err)
|
||||||
|
}
|
||||||
|
cancel()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListenAndServe starts the HTTP server until ctx is cancelled.
|
||||||
|
func (s *Server) ListenAndServe(ctx context.Context) error {
|
||||||
|
httpServer := &http.Server{
|
||||||
|
Addr: s.cfg.Listen,
|
||||||
|
Handler: s.Handler(),
|
||||||
|
ReadHeaderTimeout: 10 * time.Second,
|
||||||
|
ReadTimeout: 30 * time.Second,
|
||||||
|
IdleTimeout: 120 * time.Second,
|
||||||
|
MaxHeaderBytes: 1 << 20,
|
||||||
|
}
|
||||||
|
errCh := make(chan error, 1)
|
||||||
|
go func() {
|
||||||
|
s.logger.Printf("bat-api listening on %s (socket=%s public=%s)", s.cfg.Listen, s.cfg.SocketPath, s.cfg.PublicBaseURL)
|
||||||
|
errCh <- httpServer.ListenAndServe()
|
||||||
|
}()
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
shutdownCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
_ = httpServer.Shutdown(shutdownCtx)
|
||||||
|
return ctx.Err()
|
||||||
|
case err := <-errCh:
|
||||||
|
if err == http.ErrServerClosed {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,125 @@
|
|||||||
|
package api
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"net/url"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// RewriteServerInfoAddressables rewrites AddressablesCatalogUrlRoot values so
|
||||||
|
// clients fetch resources from publicBaseURL while leaving business API URLs
|
||||||
|
// untouched.
|
||||||
|
func RewriteServerInfoAddressables(raw []byte, publicBaseURL string) ([]byte, error) {
|
||||||
|
var doc map[string]any
|
||||||
|
if err := json.Unmarshal(raw, &doc); err != nil {
|
||||||
|
return nil, fmt.Errorf("parse server-info: %w", err)
|
||||||
|
}
|
||||||
|
groups, _ := doc["ConnectionGroups"].([]any)
|
||||||
|
for _, g := range groups {
|
||||||
|
group, ok := g.(map[string]any)
|
||||||
|
if !ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
rewriteGroupAddressables(group, publicBaseURL)
|
||||||
|
if overrides, ok := group["OverrideConnectionGroups"].([]any); ok {
|
||||||
|
for _, o := range overrides {
|
||||||
|
if og, ok := o.(map[string]any); ok {
|
||||||
|
rewriteGroupAddressables(og, publicBaseURL)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return json.Marshal(doc)
|
||||||
|
}
|
||||||
|
|
||||||
|
func rewriteGroupAddressables(group map[string]any, publicBaseURL string) {
|
||||||
|
raw, _ := group["AddressablesCatalogUrlRoot"].(string)
|
||||||
|
if raw == "" {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if rewritten, ok := rewriteAddressablesRoot(raw, publicBaseURL); ok {
|
||||||
|
group["AddressablesCatalogUrlRoot"] = rewritten
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func rewriteAddressablesRoot(original, publicBaseURL string) (string, bool) {
|
||||||
|
original = strings.TrimRight(original, "/")
|
||||||
|
publicBaseURL = strings.TrimRight(publicBaseURL, "/")
|
||||||
|
// Already rewritten to this base.
|
||||||
|
if strings.HasPrefix(original, publicBaseURL+"/"+ClientPatchHost) ||
|
||||||
|
strings.HasPrefix(original, publicBaseURL+"/"+ServerInfoHost) {
|
||||||
|
return original, true
|
||||||
|
}
|
||||||
|
u, err := url.Parse(original)
|
||||||
|
if err != nil || u.Host == "" {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
// Keep host/path so CDN path mapping stays 1:1 with destination_for_url.
|
||||||
|
path := strings.TrimPrefix(u.Path, "/")
|
||||||
|
if path == "" {
|
||||||
|
return publicBaseURL + "/" + u.Host, true
|
||||||
|
}
|
||||||
|
return publicBaseURL + "/" + u.Host + "/" + path, true
|
||||||
|
}
|
||||||
|
|
||||||
|
// LoadServerInfoBytes loads server-info from an explicit file, or from the
|
||||||
|
// release tree under yostar-serverinfo host, or synthesizes a minimal document
|
||||||
|
// from the release snapshot addressables root.
|
||||||
|
func LoadServerInfoBytes(cfg Config, idx *ReleaseIndex, requestName string) ([]byte, error) {
|
||||||
|
if cfg.ServerInfoFile != "" {
|
||||||
|
return os.ReadFile(cfg.ServerInfoFile)
|
||||||
|
}
|
||||||
|
if idx != nil && idx.ResourceRoot != "" {
|
||||||
|
name := requestName
|
||||||
|
if name == "" {
|
||||||
|
name = "server-info.json"
|
||||||
|
}
|
||||||
|
// Official path layout.
|
||||||
|
candidates := []string{
|
||||||
|
filepath.Join(idx.ResourceRoot, ServerInfoHost, name),
|
||||||
|
filepath.Join(idx.ResourceRoot, ServerInfoHost, requestName),
|
||||||
|
}
|
||||||
|
for _, c := range candidates {
|
||||||
|
if c == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if data, err := os.ReadFile(c); err == nil {
|
||||||
|
return data, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Any single json under server-info host.
|
||||||
|
dir := filepath.Join(idx.ResourceRoot, ServerInfoHost)
|
||||||
|
if entries, err := os.ReadDir(dir); err == nil {
|
||||||
|
for _, e := range entries {
|
||||||
|
if e.IsDir() || !strings.HasSuffix(e.Name(), ".json") {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
return os.ReadFile(filepath.Join(dir, e.Name()))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if idx != nil && idx.Snapshot != nil && idx.Snapshot.AddressablesRoot != "" {
|
||||||
|
doc := map[string]any{
|
||||||
|
"ConnectionGroups": []any{
|
||||||
|
map[string]any{
|
||||||
|
"Name": defaultString(idx.Snapshot.ConnectionGroupName, "Prod"),
|
||||||
|
"AddressablesCatalogUrlRoot": idx.Snapshot.AddressablesRoot,
|
||||||
|
"BundleVersion": idx.Snapshot.BundleVersion,
|
||||||
|
"IsLivePublished": true,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
return json.Marshal(doc)
|
||||||
|
}
|
||||||
|
return nil, fmt.Errorf("server-info source not found")
|
||||||
|
}
|
||||||
|
|
||||||
|
func defaultString(v, fallback string) string {
|
||||||
|
if v == "" {
|
||||||
|
return fallback
|
||||||
|
}
|
||||||
|
return v
|
||||||
|
}
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
{
|
||||||
|
"version": 1,
|
||||||
|
"entries": {
|
||||||
|
"https://prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.bytes": {
|
||||||
|
"url": "https://prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.bytes",
|
||||||
|
"destination": "prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.bytes",
|
||||||
|
"bytes": 21,
|
||||||
|
"blake3": "0000000000000000000000000000000000000000000000000000000000000000"
|
||||||
|
},
|
||||||
|
"https://prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.hash": {
|
||||||
|
"url": "https://prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.hash",
|
||||||
|
"destination": "prod-clientpatch.bluearchiveyostar.com/r93_fixture/TableBundles/TableCatalog.hash",
|
||||||
|
"bytes": 10,
|
||||||
|
"blake3": "0000000000000000000000000000000000000000000000000000000000000000"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"snapshot_version": 2,
|
||||||
|
"connection_group_name": "Prod",
|
||||||
|
"app_version": "1.70.0",
|
||||||
|
"bundle_version": "s8tloc7lo3",
|
||||||
|
"addressables_root": "https://prod-clientpatch.bluearchiveyostar.com/r93_fixture",
|
||||||
|
"launcher_metadata": {
|
||||||
|
"launcher_version": "1.7.2",
|
||||||
|
"game_latest_version": "1.70.0",
|
||||||
|
"game_latest_file_path": "prod/ZIP_TEMP/BlueArchive_JP_TEMP/BlueArchive_JP-1.70.436321-game.zip",
|
||||||
|
"game_start_exe_name": "xldr_BlueArchiveOnline_JP_loader_x64",
|
||||||
|
"game_start_params": [
|
||||||
|
"BlueArchive.exe"
|
||||||
|
],
|
||||||
|
"manifest_url": "https://launcher-pkg-ba-jp.yo-star.com/prod/ZIP_TEMP/BlueArchive_JP_TEMP/manifest.json",
|
||||||
|
"manifest_source": "BlueArchive_JP-1.70.436321-game",
|
||||||
|
"manifest_file_count": 2
|
||||||
|
},
|
||||||
|
"game_main_config_bootstrap": {
|
||||||
|
"server_info_data_url": "https://yostar-serverinfo.bluearchiveyostar.com/r93_fixture.json",
|
||||||
|
"default_connection_group": "Prod"
|
||||||
|
},
|
||||||
|
"endpoints": [],
|
||||||
|
"endpoint_markers": []
|
||||||
|
}
|
||||||
+1
@@ -0,0 +1 @@
|
|||||||
|
TABLE_CATALOG_FIXTURE
|
||||||
+1
@@ -0,0 +1 @@
|
|||||||
|
1234567890
|
||||||
+11
@@ -0,0 +1,11 @@
|
|||||||
|
{
|
||||||
|
"ConnectionGroups": [
|
||||||
|
{
|
||||||
|
"Name": "Prod",
|
||||||
|
"AddressablesCatalogUrlRoot": "https://prod-clientpatch.bluearchiveyostar.com/r93_fixture",
|
||||||
|
"BundleVersion": "s8tloc7lo3",
|
||||||
|
"IsLivePublished": true,
|
||||||
|
"ApiUrl": "https://prod-game.example.invalid/"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
+11
@@ -0,0 +1,11 @@
|
|||||||
|
{
|
||||||
|
"ConnectionGroups": [
|
||||||
|
{
|
||||||
|
"Name": "Prod",
|
||||||
|
"AddressablesCatalogUrlRoot": "https://prod-clientpatch.bluearchiveyostar.com/r93_fixture",
|
||||||
|
"BundleVersion": "s8tloc7lo3",
|
||||||
|
"IsLivePublished": true,
|
||||||
|
"ApiUrl": "https://prod-game.example.invalid/"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
# Reserved empty directory
|
||||||
|
|
||||||
|
This path is a monorepo placeholder and is **not implemented**.
|
||||||
|
Do not treat it as a finished module. See `docs/reports/GO_STATUS.md`.
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
# Reserved empty directory
|
||||||
|
|
||||||
|
This path is a monorepo placeholder and is **not implemented**.
|
||||||
|
Do not treat it as a finished module. See `docs/reports/GO_STATUS.md`.
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
# Reserved empty directory
|
||||||
|
|
||||||
|
This path is a monorepo placeholder and is **not implemented**.
|
||||||
|
Do not treat it as a finished module. See `docs/reports/GO_STATUS.md`.
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
# Reserved empty directory
|
||||||
|
|
||||||
|
This path is a monorepo placeholder and is **not implemented**.
|
||||||
|
Do not treat it as a finished module. See `docs/reports/GO_STATUS.md`.
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
# Reserved empty directory
|
||||||
|
|
||||||
|
This path is a monorepo placeholder and is **not implemented**.
|
||||||
|
Do not treat it as a finished module. See `docs/reports/GO_STATUS.md`.
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
# Reserved empty directory
|
||||||
|
|
||||||
|
Placeholder only. **Not implemented.** See `docs/reports/GO_STATUS.md`.
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
# Reserved empty directory
|
||||||
|
|
||||||
|
This path is a monorepo placeholder and is **not implemented**.
|
||||||
|
Do not treat it as a finished module. See `docs/reports/GO_STATUS.md`.
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
# Reserved empty directory
|
||||||
|
|
||||||
|
This path is a monorepo placeholder and is **not implemented**.
|
||||||
|
Do not treat it as a finished module. See `docs/reports/GO_STATUS.md`.
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
# Reserved empty directory
|
||||||
|
|
||||||
|
This path is a monorepo placeholder and is **not implemented**.
|
||||||
|
Do not treat it as a finished module. See `docs/reports/GO_STATUS.md`.
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user