mirror of
https://github.com/Yuyi-Oak/BlueArchiveToolkit.git
synced 2026-09-18 06:34:54 +08:00
feat(assetbundle): 完成已验证结构的重建发布闭环
This commit is contained in:
+4
-4
@@ -113,7 +113,7 @@ TextUnit scope、source history 和 approved review。worker、TM 复用、人
|
||||
|
||||
待完成：
|
||||
|
||||
- `crates/bat-assetbundle` 已具备 UnityFS 容器、对象表、TypeTree 元数据、基础字段读取、TextAsset 和 TextUnit 提取；UnityFS 容器已补充总大小、计数、路径、重复 directory、LZMA 和边界校验，并通过 UnityPy 真实 bundle 隔离回归；已有 TextAsset、TypeTree string field 和 managed-reference string field 的 localized patch 发布闭环，真实复杂版本差异、整体 AssetBundle 重打包和通用 Patch 仍未实现。
|
||||
- `crates/bat-assetbundle` 已具备 UnityFS 容器、对象表、TypeTree 元数据、基础字段读取、TextAsset 和 TextUnit 提取；UnityFS 容器已补充总大小、计数、路径、重复 directory、LZMA 和边界校验，并通过 UnityPy 真实 bundle 隔离回归。对当前真实/合成回归覆盖的结构，TextAsset、TypeTree string field、managed-reference string field 和语义字段已形成 parse→modify→rebuild→reparse 闭环，重建保留已识别压缩/对齐/目录形态并校验未修改对象/字段；整体任意 AssetBundle、未知结构和全部真实版本差异仍未实现。
|
||||
- Addressables parser 已覆盖当前真实形态 fixture/golden 与 `m_Crc`，但仍需继续覆盖二进制/压缩字段组合和更细失败诊断。
|
||||
- 客户端发现、备份、应用补丁流程尚未连接真实实现。
|
||||
|
||||
@@ -162,7 +162,7 @@ TextUnit scope、source history 和 approved review。worker、TM 复用、人
|
||||
|
||||
### `bat-assetbundle`
|
||||
|
||||
状态：**UnityFS 解包、TypeTree 字段读取、TextUnit 提取和受支持 localized patch 发布已可用；复杂结构覆盖与整体 AssetBundle 重打包仍待继续补齐**
|
||||
状态：**已验证 UnityFS 结构的解析、变长修改、重建、重解析和受支持 localized 发布可用；任意复杂结构兼容仍待继续补齐**
|
||||
|
||||
解析扩展当前按路线图和真实 fixture 验收推进。
|
||||
|
||||
@@ -181,7 +181,7 @@ TextUnit scope、source history 和 approved review。worker、TM 复用、人
|
||||
待完成：
|
||||
|
||||
- 真实 MonoBehaviour、ScriptableObject 版本差异、复杂容器结构调整、unknown 字段结构语义和未见样本驱动的完整 managed reference registry / map entry 变体覆盖；TypeTree-covered managed reference 字段与 registry 记录已可结构化解码，常见 full typename 可拆解为 assembly/namespace/class，不做低保真猜测。
|
||||
- 复杂对象整体结构修改后的发布级 AssetBundle 重打包；UnityFS TextAsset、TypeTree string 字段、managed-reference registry payload 字符串、基础语义字段、enum、bit_field、object 字段组合和 TypeTree schema 支撑的 array/vector/map 整体替换的文件级链路已具备重建后校验，受支持 localized patch 已有独立 staging、manifest、current、状态校验和显式 rollback；整体 AssetBundle 发布仍未完成。
|
||||
- 任意复杂对象整体结构和所有真实版本差异的发布级 AssetBundle 重打包；当前已验证的 UnityFS TextAsset、TypeTree string 字段、managed-reference registry payload 字符串、基础语义字段、enum、bit_field、object 字段组合和 TypeTree schema 支撑的 array/vector/map 整体替换已具备变长重建、压缩/对齐保留、未修改对象/字段校验、受支持 localized staging/manifest/current/rollback；带 `archive_entry` 的可验证 ZIP 内 bundle 也会重写外层 ZIP。
|
||||
- 真实资源 fixture 覆盖对象级解析和文本提取。
|
||||
- 详细补全顺序见 `docs/architecture/assetbundle.md`。
|
||||
|
||||
@@ -194,7 +194,7 @@ TextUnit scope、source history 和 approved review。worker、TM 复用、人
|
||||
待完成：
|
||||
|
||||
- 未见样本驱动的 map entry schema 变化、unknown 字段结构语义、完整 managed reference registry 变体驱动字段修改后的语义重打包。
|
||||
- 通用 manifest 驱动的跨类型 patch build/apply/diff 发布；当前 localized 发布仅接受已验证 TextUnit 对应的受支持 UnityFS 文本字段，并不等价于整体 AssetBundle 重打包。
|
||||
- 通用 manifest 驱动的跨类型 patch build/apply/diff 发布；当前 localized 发布仍仅接受已验证 TextUnit 对应的受支持 UnityFS 文本字段，不等价于任意整体 AssetBundle 重打包。
|
||||
- `unityfs.inspect`、复杂 UnityFS 语义编辑和写入型发布工作流仍未开放。
|
||||
|
||||
### `bat-ffi`
|
||||
|
||||
+5
-5
@@ -41,12 +41,12 @@ BlueArchiveToolkit 不是一次性脚本，也不是演示项目。最终交付
|
||||
|
||||
### 仍是骨架或占位
|
||||
|
||||
1. `bat-assetbundle` 已具备 UnityFS 解包和 TextAsset 提取基础能力（header/block info/directory、LZ4/LZMA block info 与数据 block、directory 文件提取、serialized file object table、TypeTree node 元数据、TextAsset bytes、TypeTree-covered managed reference payload TextUnit 上下文），并已有受支持 localized patch 发布能力；MonoBehaviour/ScriptableObject 复杂字段级解析、整体重打包和通用 Patch 仍未完成。
|
||||
2. `bat-patch` 已具备确定性 Binary hunk diff/apply、RFC 6902 JSON Patch apply、UTF-8 Text Patch、通用 Patch manifest、BLAKE3/size 完整性校验和 rollback 元数据；文件级 `patch.apply` RPC / `patch-apply` CLI 与 UnityFS TextAsset / TypeTree string / TypeTree 语义字段写入入口已开放，`bat-assetbundle` + `LocalizedPatchService` 已完成受支持 TextUnit 到 localized patch manifest、独立 staging、发布和 rollback 闭环，通用 manifest 发布与整体 AssetBundle 重打包仍后置。
|
||||
1. `bat-assetbundle` 已具备 UnityFS 解包、TextAsset/TypeTree 字段读取和 TextUnit 提取；对当前真实/合成回归覆盖的结构，UnityFS TextAsset、TypeTree string、managed-reference string 和语义字段已形成 parse→modify→rebuild→reparse 闭环，保留已识别压缩/对齐/目录形态并校验未修改对象/字段；所有真实版本差异、未知字段语义和任意复杂 AssetBundle 兼容仍未完成。
|
||||
2. `bat-patch` 已具备确定性 Binary hunk diff/apply、RFC 6902 JSON Patch apply、UTF-8 Text Patch、通用 Patch manifest、BLAKE3/size 完整性校验和 rollback 元数据；文件级 `patch.apply` RPC / `patch-apply` CLI 与 UnityFS TextAsset / TypeTree string / TypeTree 语义字段写入入口已开放，`bat-assetbundle` + `LocalizedPatchService` 已完成受支持 TextUnit 到 localized patch manifest、独立 staging、发布和 rollback 闭环；ZIP 内 bundle 在 `archive_entry` 可验证时会重写外层 ZIP，通用 manifest 发布与任意 AssetBundle 重打包仍后置。
|
||||
3. Go 侧边界已确定（见 `docs/reports/GO_STATUS.md`）：同步/运维命令行 = Rust `bat`；资源分发和内嵌 dashboard = `cmd/bat-api` MVP；`internal/backendrpc` 完成；`cmd/bat` 仅为试验（`bin/bat-go`）。完整游戏业务 API / 完整 Web 协作后台 / SDK 仍未完成。
|
||||
4. Addressables parser 已覆盖当前真实形态 fixture/golden，但还不是完整 Unity Addressables/SBP catalog 兼容层。
|
||||
5. 官方同步结果可配置为发布后自动导入 CAS + ResourceRepository，并通过 `resource.index` RPC/CLI 查询；Resource metadata 已保存 release、平台、bundle path、parse status、TextAsset 名称和 TextUnit 摘要，资源级查询已覆盖 release、平台、destination、archive entry、parse status 和 TextUnit format；单条 TextUnit 明细和解析错误已持久化到 `official-textunit-index.json`，可通过 `parse.text_units` / `parse.errors` 查询；离线 TextUnit 翻译任务状态和跳过/失败原因可通过 `translation.tasks` 查询，`translation.task.update` 已提供 worker 状态回写 contract，`translation.worker.run` 已提供真实 provider worker 触发、lease/retry 和结果落库 contract，`translation.proofread` 已提供汉化 workflow 人工校对标记 contract，`translation.memory.*` 已提供 Rust-owned TM 摘要、raw source/context 查询、provenance 和显式 confirm contract，Go 侧仅代理。
|
||||
6. 受支持汉化 Patch 发布已具备 UnityFS TextAsset、TypeTree string field 和 managed-reference string field 的 manifest/apply/rollback/完整性校验和 `localized.status` 严格校验；真实 provider worker 与项目级 Translation Memory V1 已接入，翻译记忆到完整汉化文件集合的构建仍未完成。
|
||||
6. 受支持汉化 Patch 发布已具备 UnityFS TextAsset、TypeTree string field 和 managed-reference string field 的 manifest/apply/rollback/完整性校验和 `localized.status` 严格校验；ZIP 内 bundle 在 `archive_entry` 可验证时会重写外层 ZIP。真实 provider worker 与项目级 Translation Memory V1 已接入，翻译记忆到完整汉化文件集合的构建仍未完成。
|
||||
7. 真实官方网络全量下载 smoke 已固化为可重复脚本和 runbook；真实运行记录处于长期运行测试阶段，报告待后续提供。
|
||||
8. 内嵌 dashboard MVP 已实现；完整 Web 协作后台、数据库迁移、插件加载机制尚未实现；`bat-api` 资源 bootstrap/分发/OpenAPI/管理控制面已通过 `/openapi.yaml` 提供，完整游戏业务 API 的 OpenAPI 仍未完成。
|
||||
9. 原 Git 历史未恢复；当前仓库以新初始化基线为准。
|
||||
@@ -213,7 +213,7 @@ BlueArchiveToolkit 不是一次性脚本，也不是演示项目。最终交付
|
||||
|
||||
1. **解析缓存闭环**：官方同步发布后生成 `official-parse-cache.json`，覆盖 manifest 全部条目、直接 bundle、zip 内 bundle、非候选资源和解析失败诊断；未变化文件按 URL、相对路径、size 和 BLAKE3 复用解析结果。
|
||||
2. **Addressables 完整化**：覆盖 Windows/Android JSON、compact JSON 和后续二进制 catalog 入口，解析 provider、internal id、primary key、dependency、bundle name、hash、size、CRC 和资源类型。
|
||||
3. **UnityFS 容器层**：基础目标已完成 header、block info、directory、data block、LZ4/LZMA、alignment、总大小/计数/路径/边界错误、directory 文件提取和 UnityPy 真实样本回归；复杂版本差异和发布级重打包另行推进。
|
||||
3. **UnityFS 容器层**：基础目标已完成 header、block info、directory、data block、LZ4/LZMA、alignment、总大小/计数/路径/边界错误、directory 文件提取和 UnityPy 真实样本回归；当前已验证结构另有压缩/对齐保留的变长发布级重建闭环，复杂版本差异和任意结构重打包另行推进。
|
||||
4. **Serialized file 层**：稳定 Unity serialized file header、type table、TypeTree node、object table、path id、class id 和 raw object bytes 表示。
|
||||
5. **字段级解析层**：实现 TypeTree 字段 reader，支持 bool、integer、float、string、bytes、array、vector/staticvector 嵌套 `Array`、`List<T>` / `HashSet<T>` 集合 alias、map、PPtr、enum `value__` backing field、`LayerMask` / `BitField` 的 `m_Bits` backing field、常见固定 Unity float/int/hash 值类型的 leaf/direct-child 形态、unknown fixed-size raw bytes 保留和同长度替换、TypeTree-covered managed reference / `SerializedReference` alias 和 TypeTree-covered managed reference registry 记录；managed-reference full typename 可拆为 assembly/namespace/class，常见 `m_ManagedReferences` / `RefIds` / verbose type 字段命名、`managedReference*` / `serializedReference*` metadata 和 `data` / `value` / `payload` / `object` / `managedReferencePayload` / `referencePayload` / `serializedReferencePayload` / `managedReferenceValue` / `referenceValue` / `serializedReferenceValue` / `managedReferenceObject` / `referenceObject` / `serializedReferenceObject` / `managedReferenceData` / `referenceData` / `serializedData` / `serializedReferenceData` payload 命名已有回归覆盖，TextUnit 只提取 payload 字符串并按结构化 record、`RefIds[n]` 等记录前缀或子字段保留类型上下文；array/vector/List/HashSet/map 元素与 registry payload 字段保留独立 field path、offset 和 byte size，可支撑字符串元素、managed-reference registry payload 字段、基础语义字段 patch、enum/bit_field 语义 patch、固定值类型 patch、unknown fixed-size bytes patch、object 字段组合和 TypeTree schema 支撑的 array/vector/List/HashSet/map 整体变长替换，`first/second` 与 `key/value` map entry schema 已有回归覆盖；解析模块当前仍有真实版本差异、未见样本驱动的完整 managed reference registry / map entry 变体和 unknown 字段结构语义需要继续推进。
|
||||
6. **文本对象入口**：实现 TextAsset、MonoBehaviour、ScriptableObject 的可扩展提取入口，输出可追溯到 bundle、serialized file、path id 和 field path 的文本定位。
|
||||
@@ -461,7 +461,7 @@ BlueArchiveToolkit 不是一次性脚本，也不是演示项目。最终交付
|
||||
|
||||
按最终目标计算，当前总体完成度不固定写单一百分比，以模块状态、源码、测试和契约为准。
|
||||
|
||||
已完成的是稳定基线、架构骨架、部分接口、CAS V1、Rust 官方资源同步闭环、可配置 CAS/ResourceRepository 导入、TextUnit 明细索引/查询、增量 Crowdin 离线队列、provider worker、Translation Memory V1、Glossary V1、通用 Binary/JSON/Text Patch 基础、受支持 localized patch 发布/rollback，以及 Go `bat-api` 资源分发、内嵌 dashboard 和同机 live 联调。下一阶段的关键是 TM/Glossary 扩展、通用 manifest 发布、复杂 AssetBundle 解析/重打包和官方资源长期运行报告。
|
||||
已完成的是稳定基线、架构骨架、部分接口、CAS V1、Rust 官方资源同步闭环、可配置 CAS/ResourceRepository 导入、TextUnit 明细索引/查询、增量 Crowdin 离线队列、provider worker、Translation Memory V1、Glossary V1、通用 Binary/JSON/Text Patch 基础、已验证结构的 AssetBundle 变长重建、受支持 localized patch 发布/rollback，以及 Go `bat-api` 资源分发、内嵌 dashboard 和同机 live 联调。下一阶段的关键是 TM/Glossary 扩展、通用 manifest 发布、真实版本与复杂 AssetBundle 兼容和官方资源长期运行报告。
|
||||
|
||||
---
|
||||
|
||||
|
||||
+2
-1
@@ -248,6 +248,7 @@ curl -i -H 'Range: bytes=0-1023' \
|
||||
| `--proxy <URL\|auto\|none>` | curl 代理覆盖（默认 `auto`，从环境变量检测）。scheme 支持 http/https/socks4/socks4a/socks5/socks5h |
|
||||
| `--no-proxy` | 强制直连 |
|
||||
| `--unzip <PATH>` | unzip 可执行文件（默认 `unzip`） |
|
||||
| `--zip <PATH>` | zip 可执行文件（默认 `zip`） |
|
||||
| `--dry-run` | 不写同步状态 |
|
||||
| `--plan` | dry-run 时输出计划中的 URL |
|
||||
| `--force` | 强制下载/刷新 |
|
||||
@@ -294,7 +295,7 @@ curl -i -H 'Range: bytes=0-1023' \
|
||||
- `config.toml` 的字段按职责分组：`[runtime]`、`[resource]`、`[localized]`、`[repository]`、`[network]`、`[translation.worker]`。
|
||||
- 现有 `BAT_*` 环境变量仍然有效，可继续覆盖 `config.toml` 中的同名配置。
|
||||
- `BAT_SKIP_ENV_FILE` 已废弃且不再影响启动。
|
||||
- 支持的环境变量：`BAT_OUTPUT`、`BAT_LOCALIZED_OUTPUT`、`BAT_STATE_DIR`、`BAT_AUTO_DISCOVER`、`BAT_WATCH`、`BAT_DAEMON`、`BAT_IMPORT_REPOSITORY`、`BAT_IMPORT_CAS_ROOT`、`BAT_IMPORT_RESOURCE_DB`、`BAT_PROXY`、`BAT_NO_PROXY`、`BAT_INTERVAL_SECONDS`、`BAT_ERROR_RETRY_SECONDS`、`BAT_APP_VERSION`、`BAT_CONNECTION_GROUP`、`BAT_LAUNCHER_VERSION`、`BAT_PLATFORMS`、`BAT_CURL`、`BAT_DOWNLOAD_CONCURRENCY`、`BAT_UNZIP`、`BAT_JSON`、`BAT_QUIET_UP_TO_DATE`、`BAT_TRANSLATION_PROVIDER`、`BAT_TRANSLATION_FIXTURE`、`BAT_TRANSLATION_MEMORY_PATH`、`BAT_GLOSSARY_PATH`、`BAT_TRANSLATION_CONCURRENCY`、`BAT_TRANSLATION_MAX_ATTEMPTS`、`BAT_TRANSLATION_LEASE_SECONDS`、`BAT_TRANSLATION_RETRY_BACKOFF_SECONDS`、`BAT_TRANSLATION_MAX_TASKS`、`BAT_TRANSLATION_WORKER_ID`；也可以直接写 `HTTPS_PROXY` 等通用环境变量（走现有代理自动检测）。布尔值支持 `1/0/true/false/yes/no/on/off`。
|
||||
- 支持的环境变量：`BAT_OUTPUT`、`BAT_LOCALIZED_OUTPUT`、`BAT_STATE_DIR`、`BAT_AUTO_DISCOVER`、`BAT_WATCH`、`BAT_DAEMON`、`BAT_IMPORT_REPOSITORY`、`BAT_IMPORT_CAS_ROOT`、`BAT_IMPORT_RESOURCE_DB`、`BAT_PROXY`、`BAT_NO_PROXY`、`BAT_INTERVAL_SECONDS`、`BAT_ERROR_RETRY_SECONDS`、`BAT_APP_VERSION`、`BAT_CONNECTION_GROUP`、`BAT_LAUNCHER_VERSION`、`BAT_PLATFORMS`、`BAT_CURL`、`BAT_DOWNLOAD_CONCURRENCY`、`BAT_UNZIP`、`BAT_ZIP`、`BAT_JSON`、`BAT_QUIET_UP_TO_DATE`、`BAT_TRANSLATION_PROVIDER`、`BAT_TRANSLATION_FIXTURE`、`BAT_TRANSLATION_MEMORY_PATH`、`BAT_GLOSSARY_PATH`、`BAT_TRANSLATION_CONCURRENCY`、`BAT_TRANSLATION_MAX_ATTEMPTS`、`BAT_TRANSLATION_LEASE_SECONDS`、`BAT_TRANSLATION_RETRY_BACKOFF_SECONDS`、`BAT_TRANSLATION_MAX_TASKS`、`BAT_TRANSLATION_WORKER_ID`；也可以直接写 `HTTPS_PROXY` 等通用环境变量（走现有代理自动检测）。布尔值支持 `1/0/true/false/yes/no/on/off`。
|
||||
- `BAT_WATCH` / `BAT_DAEMON` 只对无子命令的 `bat` 生效（两者同时为 `1` 时 daemon 优先）；命令行显式传入 `--watch` / `--daemon` / `--dry-run` 时运行模式设置让位。`status` / `verify` 等子命令不受它们影响。
|
||||
- 已运行的 daemon 不会热读 `config.toml`；默认 `reload` 只唤醒后台重新发现和刷新。需要应用配置文件变更时，使用带显式启动参数的 `restart`/`reload`，或先 `stop` 再重新启动 daemon。
|
||||
- `BAT_REDIS_URL` / `BAT_REDIS_PASSWORD` 为**预留键**：Redis 任务后端尚未接入，当前任务历史持久化在 `<state-dir>/bat-tasks.json`。
|
||||
|
||||
@@ -17,8 +17,8 @@ pub mod types;
|
||||
pub use error::{AssetBundleError, Result};
|
||||
pub use parser::{compression_from_flags, Parser, UnityFsParser};
|
||||
pub use patch::{
|
||||
patch_unityfs_field, patch_unityfs_string_field, patch_unityfs_text_asset, FieldPatch,
|
||||
StringFieldPatch, TextAssetPatch,
|
||||
patch_unityfs_field, patch_unityfs_string_field, patch_unityfs_text_asset,
|
||||
rebuild_unityfs_bundle, FieldPatch, StringFieldPatch, TextAssetPatch,
|
||||
};
|
||||
pub use serialized::{
|
||||
UnityManagedReferenceMetadata, UnityManagedReferenceRecord, UnitySerializedField,
|
||||
|
||||
@@ -174,6 +174,7 @@ fn parse_unityfs(data: &[u8]) -> Result<UnityFsBundle> {
|
||||
compressed_data_size,
|
||||
uncompressed_data_size,
|
||||
raw_data: data.to_vec(),
|
||||
uncompressed_data,
|
||||
files,
|
||||
serialized_files,
|
||||
text_assets,
|
||||
|
||||
@@ -5,7 +5,7 @@ use crate::parser::UnityFsParser;
|
||||
use crate::serialized::{
|
||||
UnitySerializedField, UnitySerializedReplacementValue, UnitySerializedValue,
|
||||
};
|
||||
use crate::types::UnityFsBundle;
|
||||
use crate::types::{UnityFsBundle, UnityFsCompression};
|
||||
use md5::{Digest, Md5};
|
||||
|
||||
/// One TextAsset replacement inside a serialized UnityFS directory file.
|
||||
@@ -105,9 +105,9 @@ impl FieldPatch {
|
||||
|
||||
/// Patches one TextAsset and rebuilds the UnityFS container.
|
||||
///
|
||||
/// The rebuilt bundle uses a single uncompressed data block. This keeps the
|
||||
/// patch path deterministic and avoids relying on a compressor-specific
|
||||
/// implementation while preserving all directory file paths and metadata.
|
||||
/// The rebuilt bundle retains the parsed block count, compression modes,
|
||||
/// alignment flags and directory metadata while recalculating all variable
|
||||
/// offsets and sizes.
|
||||
pub fn patch_unityfs_text_asset(data: &[u8], patch: &TextAssetPatch) -> Result<Vec<u8>> {
|
||||
let parser = UnityFsParser::new();
|
||||
let mut bundle = parser.parse_bytes(data)?;
|
||||
@@ -142,6 +142,13 @@ pub fn patch_unityfs_text_asset(data: &[u8], patch: &TextAssetPatch) -> Result<V
|
||||
|
||||
let rebuilt = rebuild_unityfs(&bundle)?;
|
||||
let verified = parser.parse_bytes(&rebuilt)?;
|
||||
verify_rebuild_preserves_unmodified_content(
|
||||
&bundle,
|
||||
&verified,
|
||||
&patch.serialized_file_path,
|
||||
patch.path_id,
|
||||
None,
|
||||
)?;
|
||||
let asset = verified
|
||||
.text_assets
|
||||
.iter()
|
||||
@@ -197,6 +204,13 @@ pub fn patch_unityfs_string_field(data: &[u8], patch: &StringFieldPatch) -> Resu
|
||||
|
||||
let rebuilt = rebuild_unityfs(&bundle)?;
|
||||
let verified = parser.parse_bytes(&rebuilt)?;
|
||||
verify_rebuild_preserves_unmodified_content(
|
||||
&bundle,
|
||||
&verified,
|
||||
&patch.serialized_file_path,
|
||||
patch.path_id,
|
||||
Some(&patch.field_path),
|
||||
)?;
|
||||
let serialized = verified
|
||||
.serialized_files
|
||||
.iter()
|
||||
@@ -259,6 +273,13 @@ pub fn patch_unityfs_field(data: &[u8], patch: &FieldPatch) -> Result<Vec<u8>> {
|
||||
|
||||
let rebuilt = rebuild_unityfs(&bundle)?;
|
||||
let verified = parser.parse_bytes(&rebuilt)?;
|
||||
verify_rebuild_preserves_unmodified_content(
|
||||
&bundle,
|
||||
&verified,
|
||||
&patch.serialized_file_path,
|
||||
patch.path_id,
|
||||
Some(&patch.field_path),
|
||||
)?;
|
||||
let serialized = verified
|
||||
.serialized_files
|
||||
.iter()
|
||||
@@ -285,7 +306,12 @@ pub fn patch_unityfs_field(data: &[u8], patch: &FieldPatch) -> Result<Vec<u8>> {
|
||||
Ok(rebuilt)
|
||||
}
|
||||
|
||||
fn rebuild_unityfs(bundle: &UnityFsBundle) -> Result<Vec<u8>> {
|
||||
/// Rebuilds a parsed UnityFS bundle while preserving its container shape.
|
||||
///
|
||||
/// The directory order, path, flags, header version and block compression
|
||||
/// modes are retained. Variable-length file changes are reflected in
|
||||
/// directory offsets and block sizes; no raw offset is patched blindly.
|
||||
pub fn rebuild_unityfs_bundle(bundle: &UnityFsBundle) -> Result<Vec<u8>> {
|
||||
if bundle.files.len() != bundle.directories.len() {
|
||||
return Err(AssetBundleError::Parse(format!(
|
||||
"UnityFS file/directory count mismatch: files={}, directories={}",
|
||||
@@ -293,24 +319,123 @@ fn rebuild_unityfs(bundle: &UnityFsBundle) -> Result<Vec<u8>> {
|
||||
bundle.directories.len()
|
||||
)));
|
||||
}
|
||||
|
||||
let mut uncompressed_data = Vec::new();
|
||||
let mut directory_offsets = Vec::with_capacity(bundle.files.len());
|
||||
for file in &bundle.files {
|
||||
let offset = u64::try_from(uncompressed_data.len())
|
||||
.map_err(|_| AssetBundleError::Parse("UnityFS data offset overflow".to_string()))?;
|
||||
directory_offsets.push(offset);
|
||||
uncompressed_data.extend_from_slice(&file.data);
|
||||
let block_uncompressed_size = bundle
|
||||
.blocks
|
||||
.iter()
|
||||
.try_fold(0u64, |total, block| {
|
||||
total.checked_add(u64::from(block.uncompressed_size))
|
||||
})
|
||||
.ok_or_else(|| AssetBundleError::Parse("UnityFS block size overflow".to_string()))?;
|
||||
let retained_uncompressed_size = u64::try_from(bundle.uncompressed_data.len())
|
||||
.map_err(|_| AssetBundleError::Parse("UnityFS data size does not fit u64".to_string()))?;
|
||||
if block_uncompressed_size != retained_uncompressed_size
|
||||
|| bundle.uncompressed_data_size != retained_uncompressed_size
|
||||
{
|
||||
return Err(AssetBundleError::Parse(format!(
|
||||
"UnityFS block table/data size mismatch: blocks={}, retained={}, declared={}",
|
||||
block_uncompressed_size, retained_uncompressed_size, bundle.uncompressed_data_size
|
||||
)));
|
||||
}
|
||||
|
||||
let mut ordered_directories = bundle.directories.iter().enumerate().collect::<Vec<_>>();
|
||||
ordered_directories.sort_by_key(|(_, directory)| directory.offset);
|
||||
let mut uncompressed_data = Vec::with_capacity(bundle.uncompressed_data.len());
|
||||
let mut directory_offsets = vec![0u64; bundle.files.len()];
|
||||
let mut cursor = 0usize;
|
||||
for (index, directory) in ordered_directories {
|
||||
let file = bundle.files.get(index).ok_or_else(|| {
|
||||
AssetBundleError::Parse(format!(
|
||||
"UnityFS directory {} has no corresponding file",
|
||||
directory.path
|
||||
))
|
||||
})?;
|
||||
if file.path != directory.path {
|
||||
return Err(AssetBundleError::Parse(format!(
|
||||
"UnityFS file/directory path mismatch: file={:?}, directory={:?}",
|
||||
file.path, directory.path
|
||||
)));
|
||||
}
|
||||
let start = usize::try_from(directory.offset).map_err(|_| {
|
||||
AssetBundleError::Parse(format!(
|
||||
"UnityFS directory {} offset does not fit usize",
|
||||
directory.path
|
||||
))
|
||||
})?;
|
||||
let end = start
|
||||
.checked_add(usize::try_from(directory.size).map_err(|_| {
|
||||
AssetBundleError::Parse(format!(
|
||||
"UnityFS directory {} size does not fit usize",
|
||||
directory.path
|
||||
))
|
||||
})?)
|
||||
.ok_or_else(|| {
|
||||
AssetBundleError::Parse(format!(
|
||||
"UnityFS directory {} range overflows usize",
|
||||
directory.path
|
||||
))
|
||||
})?;
|
||||
if start < cursor || end > bundle.uncompressed_data.len() {
|
||||
return Err(AssetBundleError::Parse(format!(
|
||||
"UnityFS directory {} overlaps or exceeds the original data region",
|
||||
directory.path
|
||||
)));
|
||||
}
|
||||
uncompressed_data.extend_from_slice(&bundle.uncompressed_data[cursor..start]);
|
||||
directory_offsets[index] = u64::try_from(uncompressed_data.len()).map_err(|_| {
|
||||
AssetBundleError::Parse("UnityFS directory offset overflow".to_string())
|
||||
})?;
|
||||
uncompressed_data.extend_from_slice(&file.data);
|
||||
cursor = end;
|
||||
}
|
||||
uncompressed_data.extend_from_slice(&bundle.uncompressed_data[cursor..]);
|
||||
|
||||
let block_sizes = repartition_blocks(uncompressed_data.len(), &bundle.blocks)?;
|
||||
let mut compressed_blocks = Vec::with_capacity(bundle.blocks.len());
|
||||
for (index, (block, size)) in bundle.blocks.iter().zip(&block_sizes).enumerate() {
|
||||
let start = block_sizes[..index].iter().sum::<usize>();
|
||||
let end = start.checked_add(*size).ok_or_else(|| {
|
||||
AssetBundleError::Parse(format!("UnityFS rebuilt block {index} range overflows"))
|
||||
})?;
|
||||
let bytes = uncompressed_data.get(start..end).ok_or_else(|| {
|
||||
AssetBundleError::Parse(format!(
|
||||
"UnityFS rebuilt block {index} range {}..{} exceeds data {}",
|
||||
start,
|
||||
end,
|
||||
uncompressed_data.len()
|
||||
))
|
||||
})?;
|
||||
compressed_blocks.push(compress_unityfs_bytes(
|
||||
bytes,
|
||||
block.compression,
|
||||
&format!("UnityFS data block {index}"),
|
||||
)?);
|
||||
}
|
||||
|
||||
let data_size = u32::try_from(uncompressed_data.len()).map_err(|_| {
|
||||
AssetBundleError::Parse("UnityFS rebuilt data exceeds u32 size".to_string())
|
||||
})?;
|
||||
let mut blocks_info_body = Vec::new();
|
||||
push_i32_be(&mut blocks_info_body, 1);
|
||||
push_u32_be(&mut blocks_info_body, data_size);
|
||||
push_u32_be(&mut blocks_info_body, data_size);
|
||||
push_u16_be(&mut blocks_info_body, 0);
|
||||
push_i32_be(
|
||||
&mut blocks_info_body,
|
||||
i32::try_from(bundle.blocks.len())
|
||||
.map_err(|_| AssetBundleError::Parse("UnityFS block count exceeds i32".to_string()))?,
|
||||
);
|
||||
for (block, (bytes, uncompressed_size)) in bundle
|
||||
.blocks
|
||||
.iter()
|
||||
.zip(compressed_blocks.iter().zip(block_sizes.iter()))
|
||||
{
|
||||
push_u32_be(
|
||||
&mut blocks_info_body,
|
||||
u32::try_from(*uncompressed_size).map_err(|_| {
|
||||
AssetBundleError::Parse("UnityFS rebuilt block exceeds u32 size".to_string())
|
||||
})?,
|
||||
);
|
||||
push_u32_be(
|
||||
&mut blocks_info_body,
|
||||
u32::try_from(bytes.len()).map_err(|_| {
|
||||
AssetBundleError::Parse("UnityFS compressed block exceeds u32 size".to_string())
|
||||
})?,
|
||||
);
|
||||
push_u16_be(&mut blocks_info_body, block.flags);
|
||||
}
|
||||
push_i32_be(
|
||||
&mut blocks_info_body,
|
||||
i32::try_from(bundle.files.len()).map_err(|_| {
|
||||
@@ -328,9 +453,14 @@ fn rebuild_unityfs(bundle: &UnityFsBundle) -> Result<Vec<u8>> {
|
||||
push_c_string(&mut blocks_info_body, &file.path);
|
||||
}
|
||||
let digest = Md5::digest(&blocks_info_body);
|
||||
let mut blocks_info = Vec::with_capacity(16 + blocks_info_body.len());
|
||||
blocks_info.extend_from_slice(&digest);
|
||||
blocks_info.extend_from_slice(&blocks_info_body);
|
||||
let mut blocks_info_uncompressed = Vec::with_capacity(16 + blocks_info_body.len());
|
||||
blocks_info_uncompressed.extend_from_slice(&digest);
|
||||
blocks_info_uncompressed.extend_from_slice(&blocks_info_body);
|
||||
let blocks_info = compress_unityfs_bytes(
|
||||
&blocks_info_uncompressed,
|
||||
compression_from_header_flags(bundle.header.flags),
|
||||
"UnityFS block info",
|
||||
)?;
|
||||
|
||||
let mut output = Vec::new();
|
||||
push_c_string(&mut output, "UnityFS");
|
||||
@@ -342,27 +472,395 @@ fn rebuild_unityfs(bundle: &UnityFsBundle) -> Result<Vec<u8>> {
|
||||
push_u32_be(
|
||||
&mut output,
|
||||
u32::try_from(blocks_info.len()).map_err(|_| {
|
||||
AssetBundleError::Parse("UnityFS block info exceeds u32 size".to_string())
|
||||
AssetBundleError::Parse("UnityFS compressed block info exceeds u32 size".to_string())
|
||||
})?,
|
||||
);
|
||||
push_u32_be(
|
||||
&mut output,
|
||||
u32::try_from(blocks_info.len()).map_err(|_| {
|
||||
u32::try_from(blocks_info_uncompressed.len()).map_err(|_| {
|
||||
AssetBundleError::Parse("UnityFS block info exceeds u32 size".to_string())
|
||||
})?,
|
||||
);
|
||||
push_u32_be(&mut output, 0);
|
||||
push_u32_be(&mut output, bundle.header.flags);
|
||||
if bundle.header.format_version >= 7 {
|
||||
align_vec(&mut output, 16);
|
||||
}
|
||||
output.extend_from_slice(&blocks_info);
|
||||
output.extend_from_slice(&uncompressed_data);
|
||||
if block_info_at_end(bundle.header.flags) {
|
||||
if block_data_aligned(bundle.header.flags) {
|
||||
align_vec(&mut output, 16);
|
||||
}
|
||||
for bytes in &compressed_blocks {
|
||||
output.extend_from_slice(bytes);
|
||||
}
|
||||
output.extend_from_slice(&blocks_info);
|
||||
} else {
|
||||
output.extend_from_slice(&blocks_info);
|
||||
if block_data_aligned(bundle.header.flags) {
|
||||
align_vec(&mut output, 16);
|
||||
}
|
||||
for bytes in &compressed_blocks {
|
||||
output.extend_from_slice(bytes);
|
||||
}
|
||||
}
|
||||
let total_size = u64::try_from(output.len())
|
||||
.map_err(|_| AssetBundleError::Parse("UnityFS rebuilt size overflow".to_string()))?;
|
||||
output[total_size_offset..total_size_offset + 8].copy_from_slice(&total_size.to_be_bytes());
|
||||
Ok(output)
|
||||
}
|
||||
|
||||
fn rebuild_unityfs(bundle: &UnityFsBundle) -> Result<Vec<u8>> {
|
||||
rebuild_unityfs_bundle(bundle)
|
||||
}
|
||||
|
||||
fn compression_from_header_flags(flags: u32) -> UnityFsCompression {
|
||||
crate::parser::compression_from_flags((flags & 0x3f) as u16)
|
||||
}
|
||||
|
||||
fn block_info_at_end(flags: u32) -> bool {
|
||||
flags & 0x80 != 0
|
||||
}
|
||||
|
||||
fn block_data_aligned(flags: u32) -> bool {
|
||||
flags & 0x200 != 0
|
||||
}
|
||||
|
||||
fn repartition_blocks(
|
||||
total_size: usize,
|
||||
blocks: &[crate::types::UnityFsBlockInfo],
|
||||
) -> Result<Vec<usize>> {
|
||||
if blocks.is_empty() {
|
||||
return Err(AssetBundleError::Parse(
|
||||
"UnityFS bundle has no blocks to rebuild".to_string(),
|
||||
));
|
||||
}
|
||||
let original_total = blocks
|
||||
.iter()
|
||||
.try_fold(0u64, |total, block| {
|
||||
total.checked_add(u64::from(block.uncompressed_size))
|
||||
})
|
||||
.ok_or_else(|| AssetBundleError::Parse("UnityFS block size overflow".to_string()))?;
|
||||
if original_total == 0 {
|
||||
return Ok(vec![0; blocks.len()]);
|
||||
}
|
||||
let total = u64::try_from(total_size)
|
||||
.map_err(|_| AssetBundleError::Parse("UnityFS data size does not fit u64".to_string()))?;
|
||||
let mut result = Vec::with_capacity(blocks.len());
|
||||
let mut previous = 0u64;
|
||||
let mut cumulative = 0u64;
|
||||
for block in blocks {
|
||||
cumulative = cumulative
|
||||
.checked_add(u64::from(block.uncompressed_size))
|
||||
.ok_or_else(|| {
|
||||
AssetBundleError::Parse("UnityFS block boundary overflow".to_string())
|
||||
})?;
|
||||
let boundary = total.checked_mul(cumulative).ok_or_else(|| {
|
||||
AssetBundleError::Parse("UnityFS block boundary overflow".to_string())
|
||||
})? / original_total;
|
||||
result.push(usize::try_from(boundary - previous).map_err(|_| {
|
||||
AssetBundleError::Parse("UnityFS rebuilt block size does not fit usize".to_string())
|
||||
})?);
|
||||
previous = boundary;
|
||||
}
|
||||
Ok(result)
|
||||
}
|
||||
|
||||
fn compress_unityfs_bytes(
|
||||
data: &[u8],
|
||||
compression: UnityFsCompression,
|
||||
context: &str,
|
||||
) -> Result<Vec<u8>> {
|
||||
match compression {
|
||||
UnityFsCompression::None => Ok(data.to_vec()),
|
||||
UnityFsCompression::Lz4 => {
|
||||
lz4::block::compress(data, Some(lz4::block::CompressionMode::FAST(1)), false).map_err(
|
||||
|error| {
|
||||
AssetBundleError::Parse(format!("Failed to compress {context} as LZ4: {error}"))
|
||||
},
|
||||
)
|
||||
}
|
||||
UnityFsCompression::Lz4Hc => lz4::block::compress(
|
||||
data,
|
||||
Some(lz4::block::CompressionMode::HIGHCOMPRESSION(9)),
|
||||
false,
|
||||
)
|
||||
.map_err(|error| {
|
||||
AssetBundleError::Parse(format!("Failed to compress {context} as LZ4HC: {error}"))
|
||||
}),
|
||||
UnityFsCompression::Lzma => {
|
||||
let mut output = Vec::new();
|
||||
lzma_rs::lzma_compress(&mut std::io::Cursor::new(data), &mut output).map_err(
|
||||
|error| {
|
||||
AssetBundleError::Parse(format!(
|
||||
"Failed to compress {context} as LZMA: {error}"
|
||||
))
|
||||
},
|
||||
)?;
|
||||
Ok(output)
|
||||
}
|
||||
UnityFsCompression::Unknown(value) => Err(AssetBundleError::UnsupportedFormat(format!(
|
||||
"unsupported {context} compression flag: {value}"
|
||||
))),
|
||||
}
|
||||
}
|
||||
|
||||
fn verify_rebuild_preserves_unmodified_content(
|
||||
original: &UnityFsBundle,
|
||||
rebuilt_bytes: &UnityFsBundle,
|
||||
modified_serialized_path: &str,
|
||||
modified_path_id: i64,
|
||||
modified_field_path: Option<&str>,
|
||||
) -> Result<()> {
|
||||
if original.header.format_version != rebuilt_bytes.header.format_version
|
||||
|| original.header.target_version != rebuilt_bytes.header.target_version
|
||||
|| original.header.unity_version != rebuilt_bytes.header.unity_version
|
||||
|| original.header.flags != rebuilt_bytes.header.flags
|
||||
{
|
||||
return Err(AssetBundleError::Parse(
|
||||
"UnityFS rebuild changed immutable header metadata".to_string(),
|
||||
));
|
||||
}
|
||||
if original.blocks.len() != rebuilt_bytes.blocks.len()
|
||||
|| original.directories.len() != rebuilt_bytes.directories.len()
|
||||
{
|
||||
return Err(AssetBundleError::Parse(
|
||||
"UnityFS rebuild changed block or directory count".to_string(),
|
||||
));
|
||||
}
|
||||
if original.serialized_files.len() != rebuilt_bytes.serialized_files.len() {
|
||||
return Err(AssetBundleError::Parse(
|
||||
"UnityFS rebuild changed serialized-file count".to_string(),
|
||||
));
|
||||
}
|
||||
if original.serialized_parse_errors != rebuilt_bytes.serialized_parse_errors {
|
||||
return Err(AssetBundleError::Parse(
|
||||
"UnityFS rebuild changed serialized-file diagnostics".to_string(),
|
||||
));
|
||||
}
|
||||
for (original, rebuilt) in original.blocks.iter().zip(&rebuilt_bytes.blocks) {
|
||||
if original.flags != rebuilt.flags || original.compression != rebuilt.compression {
|
||||
return Err(AssetBundleError::Parse(
|
||||
"UnityFS rebuild changed block compression metadata".to_string(),
|
||||
));
|
||||
}
|
||||
}
|
||||
for (original, rebuilt) in original.directories.iter().zip(&rebuilt_bytes.directories) {
|
||||
if original.path != rebuilt.path || original.flags != rebuilt.flags {
|
||||
return Err(AssetBundleError::Parse(
|
||||
"UnityFS rebuild changed directory path or flags".to_string(),
|
||||
));
|
||||
}
|
||||
}
|
||||
|
||||
for (original_file, rebuilt_file) in original.files.iter().zip(&rebuilt_bytes.files) {
|
||||
if original_file.path != modified_serialized_path
|
||||
&& (original_file.path != rebuilt_file.path || original_file.data != rebuilt_file.data)
|
||||
{
|
||||
return Err(AssetBundleError::Parse(format!(
|
||||
"UnityFS rebuild changed an unmodified directory file: {}",
|
||||
original_file.path
|
||||
)));
|
||||
}
|
||||
}
|
||||
|
||||
for original_file in &original.serialized_files {
|
||||
let rebuilt_file = rebuilt_bytes
|
||||
.serialized_files
|
||||
.iter()
|
||||
.find(|candidate| candidate.source_path == original_file.source_path)
|
||||
.ok_or_else(|| {
|
||||
AssetBundleError::Parse(format!(
|
||||
"UnityFS rebuild lost serialized file {:?}",
|
||||
original_file.source_path
|
||||
))
|
||||
})?;
|
||||
if original_file.source_path != rebuilt_file.source_path
|
||||
|| original_file.version != rebuilt_file.version
|
||||
|| original_file.unity_version != rebuilt_file.unity_version
|
||||
|| original_file.platform != rebuilt_file.platform
|
||||
|| original_file.types != rebuilt_file.types
|
||||
|| original_file.objects.len() != rebuilt_file.objects.len()
|
||||
{
|
||||
return Err(AssetBundleError::Parse(
|
||||
"UnityFS rebuild changed serialized-file metadata".to_string(),
|
||||
));
|
||||
}
|
||||
for (original_object, rebuilt_object) in
|
||||
original_file.objects.iter().zip(&rebuilt_file.objects)
|
||||
{
|
||||
if original_object.path_id != rebuilt_object.path_id
|
||||
|| original_object.type_index != rebuilt_object.type_index
|
||||
|| original_object.class_id != rebuilt_object.class_id
|
||||
{
|
||||
return Err(AssetBundleError::Parse(format!(
|
||||
"UnityFS rebuild changed object table entry path_id {}",
|
||||
original_object.path_id
|
||||
)));
|
||||
}
|
||||
let is_modified_object = original_file.source_path.as_deref()
|
||||
== Some(modified_serialized_path)
|
||||
&& original_object.path_id == modified_path_id;
|
||||
if !is_modified_object && original_object.byte_size != rebuilt_object.byte_size {
|
||||
return Err(AssetBundleError::Parse(format!(
|
||||
"UnityFS rebuild changed unmodified object byte size path_id {}",
|
||||
original_object.path_id
|
||||
)));
|
||||
}
|
||||
if is_modified_object {
|
||||
if let Some(field_path) = modified_field_path {
|
||||
let original_fields = original_file.fields_for_object_entry(original_object)?;
|
||||
let rebuilt_fields = rebuilt_file.fields_for_object_entry(rebuilt_object)?;
|
||||
verify_unmodified_fields(
|
||||
&original_fields,
|
||||
&rebuilt_fields,
|
||||
field_path,
|
||||
original_object.path_id,
|
||||
)?;
|
||||
} else {
|
||||
let original_asset = original_file
|
||||
.text_assets()
|
||||
.iter()
|
||||
.find(|asset| asset.path_id == modified_path_id);
|
||||
let rebuilt_asset = rebuilt_file
|
||||
.text_assets()
|
||||
.iter()
|
||||
.find(|asset| asset.path_id == modified_path_id);
|
||||
if original_asset.map(|asset| &asset.name)
|
||||
!= rebuilt_asset.map(|asset| &asset.name)
|
||||
{
|
||||
return Err(AssetBundleError::Parse(format!(
|
||||
"TextAsset path_id {} name changed while rebuilding",
|
||||
modified_path_id
|
||||
)));
|
||||
}
|
||||
}
|
||||
} else {
|
||||
if original_file.object_bytes(original_object)?
|
||||
!= rebuilt_file.object_bytes(rebuilt_object)?
|
||||
{
|
||||
return Err(AssetBundleError::Parse(format!(
|
||||
"UnityFS rebuild changed unmodified object path_id {}",
|
||||
original_object.path_id
|
||||
)));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn verify_unmodified_fields(
|
||||
original: &[UnitySerializedField],
|
||||
rebuilt: &[UnitySerializedField],
|
||||
modified_field_path: &str,
|
||||
path_id: i64,
|
||||
) -> Result<()> {
|
||||
if original.len() != rebuilt.len() {
|
||||
return Err(AssetBundleError::Parse(format!(
|
||||
"Unity object path_id {path_id} changed field count while rebuilding"
|
||||
)));
|
||||
}
|
||||
for (original, rebuilt) in original.iter().zip(rebuilt) {
|
||||
if original.path != rebuilt.path
|
||||
|| original.name != rebuilt.name
|
||||
|| original.type_name != rebuilt.type_name
|
||||
|| original.type_tree_node_index != rebuilt.type_tree_node_index
|
||||
{
|
||||
return Err(AssetBundleError::Parse(format!(
|
||||
"Unity object path_id {path_id} changed field metadata at {}",
|
||||
original.path
|
||||
)));
|
||||
}
|
||||
if !field_path_contains(&original.path, modified_field_path)
|
||||
&& original.byte_size != rebuilt.byte_size
|
||||
{
|
||||
return Err(AssetBundleError::Parse(format!(
|
||||
"Unity object path_id {path_id} changed unmodified field byte size at {}",
|
||||
original.path
|
||||
)));
|
||||
}
|
||||
if original.path == modified_field_path {
|
||||
continue;
|
||||
}
|
||||
verify_unmodified_values(
|
||||
&original.value,
|
||||
&rebuilt.value,
|
||||
modified_field_path,
|
||||
path_id,
|
||||
)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn field_path_contains(parent: &str, child: &str) -> bool {
|
||||
child == parent
|
||||
|| child
|
||||
.strip_prefix(parent)
|
||||
.is_some_and(|suffix| suffix.starts_with('.') || suffix.starts_with('['))
|
||||
}
|
||||
|
||||
fn verify_unmodified_values(
|
||||
original: &UnitySerializedValue,
|
||||
rebuilt: &UnitySerializedValue,
|
||||
modified_field_path: &str,
|
||||
path_id: i64,
|
||||
) -> Result<()> {
|
||||
match (original, rebuilt) {
|
||||
(UnitySerializedValue::Object(original), UnitySerializedValue::Object(rebuilt))
|
||||
| (UnitySerializedValue::Array(original), UnitySerializedValue::Array(rebuilt))
|
||||
| (UnitySerializedValue::Map(original), UnitySerializedValue::Map(rebuilt)) => {
|
||||
verify_unmodified_fields(original, rebuilt, modified_field_path, path_id)
|
||||
}
|
||||
(
|
||||
UnitySerializedValue::ManagedReference {
|
||||
type_name: original_type,
|
||||
metadata: original_metadata,
|
||||
fields: original_fields,
|
||||
bytes: original_bytes,
|
||||
},
|
||||
UnitySerializedValue::ManagedReference {
|
||||
type_name: rebuilt_type,
|
||||
metadata: rebuilt_metadata,
|
||||
fields: rebuilt_fields,
|
||||
bytes: rebuilt_bytes,
|
||||
},
|
||||
) => {
|
||||
if original_type != rebuilt_type
|
||||
|| original_metadata != rebuilt_metadata
|
||||
|| original_bytes != rebuilt_bytes
|
||||
{
|
||||
return Err(AssetBundleError::Parse(format!(
|
||||
"Unity object path_id {path_id} changed managed-reference metadata"
|
||||
)));
|
||||
}
|
||||
verify_unmodified_fields(
|
||||
original_fields,
|
||||
rebuilt_fields,
|
||||
modified_field_path,
|
||||
path_id,
|
||||
)
|
||||
}
|
||||
(
|
||||
UnitySerializedValue::ManagedReferenceRegistry {
|
||||
references: _,
|
||||
fields: original_fields,
|
||||
},
|
||||
UnitySerializedValue::ManagedReferenceRegistry {
|
||||
references: _,
|
||||
fields: rebuilt_fields,
|
||||
},
|
||||
) => verify_unmodified_fields(
|
||||
original_fields,
|
||||
rebuilt_fields,
|
||||
modified_field_path,
|
||||
path_id,
|
||||
),
|
||||
(original, rebuilt) if original != rebuilt => Err(AssetBundleError::Parse(format!(
|
||||
"Unity object path_id {path_id} changed an unmodified field value"
|
||||
))),
|
||||
_ => Ok(()),
|
||||
}
|
||||
}
|
||||
|
||||
fn find_field_value<'a>(
|
||||
fields: &'a [UnitySerializedField],
|
||||
field_path: &str,
|
||||
@@ -489,6 +987,99 @@ mod tests {
|
||||
data
|
||||
}
|
||||
|
||||
fn synthetic_bundle_with_compressed_blocks(
|
||||
files: &[(&str, &[u8])],
|
||||
block_flags: u16,
|
||||
header_flags: u32,
|
||||
) -> Vec<u8> {
|
||||
let mut uncompressed = Vec::new();
|
||||
let mut directory_offsets = Vec::with_capacity(files.len());
|
||||
for (_, bytes) in files {
|
||||
directory_offsets.push(uncompressed.len() as u64);
|
||||
uncompressed.extend_from_slice(bytes);
|
||||
}
|
||||
let split = uncompressed.len() / 2;
|
||||
let split = if uncompressed.is_empty() {
|
||||
0
|
||||
} else {
|
||||
split.max(1).min(uncompressed.len())
|
||||
};
|
||||
let chunks = [&uncompressed[..split], &uncompressed[split..]];
|
||||
let compressed_chunks = chunks
|
||||
.iter()
|
||||
.map(|chunk| match block_flags & 0x3f {
|
||||
0 => chunk.to_vec(),
|
||||
1 => {
|
||||
let mut output = Vec::new();
|
||||
lzma_rs::lzma_compress(&mut std::io::Cursor::new(chunk), &mut output).unwrap();
|
||||
output
|
||||
}
|
||||
2..=4 => lz4::block::compress(chunk, None, false).unwrap(),
|
||||
value => panic!("unsupported fixture compression {value}"),
|
||||
})
|
||||
.collect::<Vec<_>>();
|
||||
|
||||
let mut block_info_body = Vec::new();
|
||||
push_i32_be(&mut block_info_body, chunks.len() as i32);
|
||||
for (chunk, compressed) in chunks.iter().zip(&compressed_chunks) {
|
||||
push_u32_be(&mut block_info_body, chunk.len() as u32);
|
||||
push_u32_be(&mut block_info_body, compressed.len() as u32);
|
||||
push_u16_be(&mut block_info_body, block_flags);
|
||||
}
|
||||
push_i32_be(&mut block_info_body, files.len() as i32);
|
||||
for ((path, bytes), offset) in files.iter().zip(directory_offsets) {
|
||||
push_u64_be(&mut block_info_body, offset);
|
||||
push_u64_be(&mut block_info_body, bytes.len() as u64);
|
||||
push_u32_be(&mut block_info_body, 0);
|
||||
push_c_string(&mut block_info_body, path);
|
||||
}
|
||||
let mut block_info = vec![0; 16];
|
||||
block_info.extend_from_slice(&block_info_body);
|
||||
let block_info = match header_flags & 0x3f {
|
||||
0 => block_info,
|
||||
1 => {
|
||||
let mut output = Vec::new();
|
||||
lzma_rs::lzma_compress(&mut std::io::Cursor::new(&block_info), &mut output)
|
||||
.unwrap();
|
||||
output
|
||||
}
|
||||
2..=4 => lz4::block::compress(&block_info, None, false).unwrap(),
|
||||
value => panic!("unsupported fixture block-info compression {value}"),
|
||||
};
|
||||
|
||||
let mut data = Vec::new();
|
||||
push_c_string(&mut data, "UnityFS");
|
||||
push_u32_be(&mut data, 8);
|
||||
push_c_string(&mut data, "5.x.x");
|
||||
push_c_string(&mut data, "2021.3.56f2");
|
||||
let total_size_offset = data.len();
|
||||
push_u64_be(&mut data, 0);
|
||||
push_u32_be(&mut data, block_info.len() as u32);
|
||||
push_u32_be(&mut data, (16 + block_info_body.len()) as u32);
|
||||
push_u32_be(&mut data, header_flags);
|
||||
align_vec(&mut data, 16);
|
||||
if header_flags & 0x80 != 0 {
|
||||
if header_flags & 0x200 != 0 {
|
||||
align_vec(&mut data, 16);
|
||||
}
|
||||
for compressed in &compressed_chunks {
|
||||
data.extend_from_slice(compressed);
|
||||
}
|
||||
data.extend_from_slice(&block_info);
|
||||
} else {
|
||||
data.extend_from_slice(&block_info);
|
||||
if header_flags & 0x200 != 0 {
|
||||
align_vec(&mut data, 16);
|
||||
}
|
||||
for compressed in &compressed_chunks {
|
||||
data.extend_from_slice(compressed);
|
||||
}
|
||||
}
|
||||
let total_size = data.len() as u64;
|
||||
data[total_size_offset..total_size_offset + 8].copy_from_slice(&total_size.to_be_bytes());
|
||||
data
|
||||
}
|
||||
|
||||
fn push_i16_le(data: &mut Vec<u8>, value: i16) {
|
||||
data.extend_from_slice(&value.to_le_bytes());
|
||||
}
|
||||
@@ -549,6 +1140,66 @@ mod tests {
|
||||
file
|
||||
}
|
||||
|
||||
fn synthetic_serialized_two_text_assets(first: &[u8], second: &[u8]) -> Vec<u8> {
|
||||
fn text_asset_object(bytes: &[u8]) -> Vec<u8> {
|
||||
let mut object = Vec::new();
|
||||
push_u32_le(&mut object, 8);
|
||||
object.extend_from_slice(b"Scenario");
|
||||
align_vec(&mut object, 4);
|
||||
push_u32_le(&mut object, bytes.len() as u32);
|
||||
object.extend_from_slice(bytes);
|
||||
object
|
||||
}
|
||||
|
||||
let first_object = text_asset_object(first);
|
||||
let second_object = text_asset_object(second);
|
||||
let mut object_data = Vec::new();
|
||||
let first_offset = object_data.len() as u64;
|
||||
object_data.extend_from_slice(&first_object);
|
||||
let first_size = object_data.len() as u64 - first_offset;
|
||||
let second_offset = object_data.len() as u64;
|
||||
object_data.extend_from_slice(&second_object);
|
||||
let second_size = object_data.len() as u64 - second_offset;
|
||||
|
||||
let mut metadata = Vec::new();
|
||||
metadata.extend_from_slice(b"2021.3.56f2\0");
|
||||
push_i32_le(&mut metadata, 19);
|
||||
metadata.push(0);
|
||||
push_i32_le(&mut metadata, 1);
|
||||
push_i32_le(&mut metadata, 49);
|
||||
metadata.push(0);
|
||||
push_i16_le(&mut metadata, 0);
|
||||
metadata.extend_from_slice(&[0; 16]);
|
||||
push_i32_le(&mut metadata, 2);
|
||||
align_vec(&mut metadata, 4);
|
||||
for (path_id, offset, size) in [
|
||||
(1i64, first_offset, first_size),
|
||||
(2, second_offset, second_size),
|
||||
] {
|
||||
metadata.extend_from_slice(&path_id.to_le_bytes());
|
||||
push_u64_le(&mut metadata, offset);
|
||||
push_u32_le(&mut metadata, size as u32);
|
||||
push_i32_le(&mut metadata, 0);
|
||||
}
|
||||
|
||||
let header_len = 48usize;
|
||||
let data_offset = header_len + metadata.len();
|
||||
let file_size = data_offset + object_data.len();
|
||||
let mut file = Vec::new();
|
||||
push_u32_be(&mut file, metadata.len() as u32);
|
||||
push_u32_be(&mut file, file_size as u32);
|
||||
push_u32_be(&mut file, 22);
|
||||
push_u32_be(&mut file, 0);
|
||||
file.extend_from_slice(&[0, 0, 0, 0]);
|
||||
push_u32_be(&mut file, metadata.len() as u32);
|
||||
push_u64_be(&mut file, file_size as u64);
|
||||
push_u64_be(&mut file, data_offset as u64);
|
||||
push_u64_be(&mut file, 0);
|
||||
file.extend_from_slice(&metadata);
|
||||
file.extend_from_slice(&object_data);
|
||||
file
|
||||
}
|
||||
|
||||
fn synthetic_serialized_monobehaviour() -> Vec<u8> {
|
||||
let mut object_data = Vec::new();
|
||||
push_u32_le(&mut object_data, 5);
|
||||
@@ -1391,6 +2042,135 @@ mod tests {
|
||||
assert_eq!(parsed.unity_version, reparsed.unity_version);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rebuild_preserves_compressed_blocks_alignment_and_untouched_files() {
|
||||
let serialized = synthetic_serialized_text_asset(b"old");
|
||||
let source = synthetic_bundle_with_compressed_blocks(
|
||||
&[("CAB-scenario", &serialized), ("CAB-untouched", b"keep")],
|
||||
2,
|
||||
2 | 0x200,
|
||||
);
|
||||
let parsed = UnityFsParser::new().parse_bytes(&source).unwrap();
|
||||
assert_eq!(parsed.blocks.len(), 2);
|
||||
assert_eq!(parsed.blocks[0].compression, UnityFsCompression::Lz4);
|
||||
assert_eq!(parsed.header.flags, 2 | 0x200);
|
||||
|
||||
let patched = patch_unityfs_text_asset(
|
||||
&source,
|
||||
&TextAssetPatch {
|
||||
serialized_file_path: "CAB-scenario".to_string(),
|
||||
path_id: 1,
|
||||
expected_name: Some("Scenario".to_string()),
|
||||
replacement: b"a longer localized payload".to_vec(),
|
||||
},
|
||||
)
|
||||
.unwrap();
|
||||
let reparsed = UnityFsParser::new().parse_bytes(&patched).unwrap();
|
||||
|
||||
assert_eq!(reparsed.blocks.len(), 2);
|
||||
assert_eq!(reparsed.blocks[0].compression, UnityFsCompression::Lz4);
|
||||
assert_eq!(reparsed.blocks[1].compression, UnityFsCompression::Lz4);
|
||||
assert_eq!(reparsed.header.flags, parsed.header.flags);
|
||||
assert_eq!(
|
||||
reparsed
|
||||
.files
|
||||
.iter()
|
||||
.find(|file| file.path == "CAB-untouched")
|
||||
.unwrap()
|
||||
.data,
|
||||
b"keep"
|
||||
);
|
||||
assert_eq!(reparsed.text_assets[0].bytes, b"a longer localized payload");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rebuild_preserves_unmodified_serialized_objects() {
|
||||
let serialized = synthetic_serialized_two_text_assets(b"old", b"keep-object");
|
||||
let source = synthetic_bundle_with_path(&serialized, "CAB-scenario");
|
||||
let patched = patch_unityfs_text_asset(
|
||||
&source,
|
||||
&TextAssetPatch {
|
||||
serialized_file_path: "CAB-scenario".to_string(),
|
||||
path_id: 1,
|
||||
expected_name: Some("Scenario".to_string()),
|
||||
replacement: b"localized".to_vec(),
|
||||
},
|
||||
)
|
||||
.unwrap();
|
||||
let reparsed = UnityFsParser::new().parse_bytes(&patched).unwrap();
|
||||
|
||||
assert_eq!(
|
||||
reparsed
|
||||
.text_assets
|
||||
.iter()
|
||||
.find(|asset| asset.path_id == 1)
|
||||
.unwrap()
|
||||
.bytes,
|
||||
b"localized"
|
||||
);
|
||||
assert_eq!(
|
||||
reparsed
|
||||
.text_assets
|
||||
.iter()
|
||||
.find(|asset| asset.path_id == 2)
|
||||
.unwrap()
|
||||
.bytes,
|
||||
b"keep-object"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rebuild_preserves_block_info_at_end_and_lzma_compression() {
|
||||
let serialized = synthetic_serialized_text_asset(b"old");
|
||||
let source = synthetic_bundle_with_compressed_blocks(
|
||||
&[("CAB-scenario", &serialized)],
|
||||
1,
|
||||
1 | 0x80 | 0x200,
|
||||
);
|
||||
let patched = patch_unityfs_text_asset(
|
||||
&source,
|
||||
&TextAssetPatch {
|
||||
serialized_file_path: "CAB-scenario".to_string(),
|
||||
path_id: 1,
|
||||
expected_name: None,
|
||||
replacement: b"localized".to_vec(),
|
||||
},
|
||||
)
|
||||
.unwrap();
|
||||
let reparsed = UnityFsParser::new().parse_bytes(&patched).unwrap();
|
||||
|
||||
assert_eq!(reparsed.header.flags, 1 | 0x80 | 0x200);
|
||||
assert_eq!(reparsed.blocks[0].compression, UnityFsCompression::Lzma);
|
||||
assert_eq!(reparsed.blocks[1].compression, UnityFsCompression::Lzma);
|
||||
assert_eq!(reparsed.text_assets[0].bytes, b"localized");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rebuild_rejects_unknown_compression_without_guessing() {
|
||||
let source = synthetic_bundle(b"payload");
|
||||
let mut parsed = UnityFsParser::new().parse_bytes(&source).unwrap();
|
||||
parsed.blocks[0].compression = UnityFsCompression::Unknown(63);
|
||||
|
||||
let error = rebuild_unityfs_bundle(&parsed).unwrap_err();
|
||||
|
||||
assert!(matches!(
|
||||
error,
|
||||
AssetBundleError::UnsupportedFormat(message)
|
||||
if message.contains("unsupported UnityFS data block 0 compression flag: 63")
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rebuild_rejects_inconsistent_uncompressed_block_table() {
|
||||
let source = synthetic_bundle(b"payload");
|
||||
let mut parsed = UnityFsParser::new().parse_bytes(&source).unwrap();
|
||||
parsed.blocks[0].uncompressed_size += 1;
|
||||
|
||||
let error = rebuild_unityfs_bundle(&parsed).unwrap_err().to_string();
|
||||
|
||||
assert!(error.contains("block table/data size mismatch"), "{error}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn patches_text_asset_and_verifies_reparsed_payload() {
|
||||
let original_text = "こんにちは".as_bytes();
|
||||
|
||||
@@ -1088,6 +1088,34 @@ impl UnitySerializedFile {
|
||||
}
|
||||
}
|
||||
|
||||
/// Returns the raw payload bytes for one object table entry.
|
||||
///
|
||||
/// The returned slice is still owned by the parsed serialized file. It is
|
||||
/// useful to compare untouched objects across a variable-length rebuild.
|
||||
pub fn object_bytes(&self, object: &UnitySerializedObject) -> Result<&[u8]> {
|
||||
let object_start = self
|
||||
.data_offset
|
||||
.checked_add(usize::try_from(object.byte_start).map_err(|_| {
|
||||
AssetBundleError::Parse(format!(
|
||||
"Unity object path_id {} byte_start does not fit usize",
|
||||
object.path_id
|
||||
))
|
||||
})?)
|
||||
.ok_or_else(|| AssetBundleError::Parse("Unity object offset overflow".to_string()))?;
|
||||
let object_end = object_start
|
||||
.checked_add(object.byte_size as usize)
|
||||
.ok_or_else(|| AssetBundleError::Parse("Unity object size overflow".to_string()))?;
|
||||
self.raw_data.get(object_start..object_end).ok_or_else(|| {
|
||||
AssetBundleError::Parse(format!(
|
||||
"Unity object path_id {} byte range {}..{} exceeds file size {}",
|
||||
object.path_id,
|
||||
object_start,
|
||||
object_end,
|
||||
self.raw_data.len()
|
||||
))
|
||||
})
|
||||
}
|
||||
|
||||
fn rewrite_object_payload(
|
||||
&self,
|
||||
target_index: usize,
|
||||
|
||||
@@ -62,6 +62,9 @@ pub struct UnityFsBundle {
|
||||
pub uncompressed_data_size: u64,
|
||||
/// Original bytes retained for future extraction/serialization.
|
||||
pub raw_data: Vec<u8>,
|
||||
/// Uncompressed data region retained so rebuilds can preserve gaps and
|
||||
/// trailing bytes that are not represented by directory entries.
|
||||
pub(crate) uncompressed_data: Vec<u8>,
|
||||
/// Files extracted from the UnityFS uncompressed data region.
|
||||
pub files: Vec<UnityFsFile>,
|
||||
/// Serialized files parsed from UnityFS directory files.
|
||||
|
||||
@@ -29,10 +29,10 @@
|
||||
| --- | --- | --- | --- |
|
||||
| 官方 seed manifest | `TableCatalog.bytes`、`BundlePackingInfo.bytes`、`MediaCatalog.bytes` | 完整下载 URL、相对路径、hash 校验边界 | 已用于下载计划，仍需沉淀更多结构化字段 |
|
||||
| Addressables catalog | `catalog_*.zip` 内 JSON/bin catalog、`catalog_*.hash` | asset path、provider、dependencies、size、CRC、bundle name | JSON/compact 当前目标字段已覆盖；未知结构返回明确错误 |
|
||||
| UnityFS container | `.bundle`、zip 内 bundle | header、block、directory、解压文件、基础摘要 | 已支持基础解包、LZ4/LZMA、alignment、大小/计数/路径/边界校验 |
|
||||
| UnityFS container | `.bundle`、zip 内 bundle | header、block、directory、解压文件、基础摘要 | 已支持基础解包、LZ4/LZMA、alignment、大小/计数/路径/边界校验；受支持 patch 可保留容器形态重建 |
|
||||
| Serialized file | UnityFS directory 文件 | header、type table、TypeTree node、object table、TextAsset bytes | 已支持基础表结构和 TextAsset bytes |
|
||||
| Unity 对象字段 | TextAsset、MonoBehaviour、ScriptableObject | 可翻译文本单元、上下文、资源定位 | TypeTree 基础字段读取、`SerializedReference` / prefixed managed-reference metadata alias、payload 提取和字符串提取已落地，真实结构覆盖继续扩大 |
|
||||
| Patch 发布 | 已翻译 TextUnit、中间格式、原版资源 | 可验证 localized patch manifest、汉化 release 目录、current/state | TextAsset、TypeTree string field 和 managed-reference string field 的 localized publish/rollback 已落地；整体 AssetBundle 重打包与通用 manifest 发布仍未完成 |
|
||||
| Patch 发布 | 已翻译 TextUnit、中间格式、原版资源 | 可验证 localized patch manifest、汉化 release 目录、current/state | 已支持 TextAsset、TypeTree string field、managed-reference string field 的 UnityFS parse→modify→rebuild→reparse；可验证 ZIP 内 bundle 时会重写外层 ZIP；通用 manifest 发布仍未完成 |
|
||||
|
||||
---
|
||||
|
||||
@@ -51,13 +51,15 @@
|
||||
9. `TextUnitExtractor` 已把 JSON/CSV/TSV/plain TextAsset、TypeTree 字符串字段和 TypeTree-covered managed reference payload 字符串输出为可序列化 TextUnit/JSONL；zip 场景保留 archive entry，TextUnit 明细包含 serialized file、path id、class id、field path、字段 offset/byte size、format、asset name 和上下文。managed-reference 类型元数据保留为 payload context，不进入翻译文本队列；即使 registry 暂时只能走 fallback 字段遍历，`RefIds`、`className`、`namespaceName`、`asmName` 等元数据别名也会被跳过，payload/value/object 家族和 `managedReferenceData` / `referenceData` / `serializedData` 仍按 payload 处理，并按 `RefIds[n]` 等记录前缀或子字段推导 metadata，避免多条 fallback record 混用 managed-reference context。
|
||||
10. `ResourceImportService` 能把 AssetBundle 摘要、TextAsset/Table/Media 分类和 TextUnit 摘要写入导入报告。
|
||||
11. 官方同步后 `OfficialParseCacheService` 能从 `official-download-manifest.json` 遍历所有资源，解析直接 bundle 和 zip 内条目，非候选资源记录为 unsupported，并缓存 TextUnit 数量/格式/诊断摘要，同时写出 `official-textunit-index.json` 供 `parse.text_units` / `parse.errors` 查询。
|
||||
12. `rebuild_unityfs_bundle` 保留当前已验证 bundle 的 header 版本、directory 顺序/路径/flags、block 数量/压缩模式、block-info-at-end 和 block-data alignment；变长目录文件会重新计算目录 offset、block size、block-info hash 和总大小，未知压缩模式明确拒绝。
|
||||
13. 重建后会重解析并校验未修改 directory 文件、serialized object table、未修改 object raw bytes，以及目标对象的未修改 TypeTree 字段；不能证明保真的结构不会发布。
|
||||
|
||||
当前还不能宣称完整：
|
||||
|
||||
1. TypeTree-covered managed reference 字段和 registry 记录已可结构化解码并参与文本提取，常见 registry 命名别名（含 `m_ManagedReferences`、`RefIds`、`m_RefIds`、verbose type 字段）、metadata 命名别名（含 `id`、`typeInfo`）、payload 命名别名（含 `data`、`value`、`payload`、`object`、`managedReferencePayload`、`referencePayload`、`serializedReferencePayload`、`managedReferenceValue`、`referenceValue`、`serializedReferenceValue`、`managedReferenceObject`、`referenceObject`、`serializedReferenceObject`、`managedReferenceData`、`referenceData`、`serializedData`）、full typename 拆解和 payload-only TextUnit 提取已有回归覆盖，多记录 registry 聚合也已有单元回归；fallback 字段遍历会跳过常见 registry 元数据字符串，避免误入翻译队列，并按记录前缀或子字段可推导 metadata 保留 managed-reference TextUnit context。enum `value__` backing field 和 `LayerMask` / `BitField` 的 `m_Bits` backing field 已可语义化解码和替换；`Vector2f/3f/4f`、`Quaternionf`、`ColorRGBA`、`Rectf`、`AABB/Bounds/Ray`、`Matrix4x4f`、`Vector2Int/Vector3Int`、`RectInt`、`BoundsInt`、`RangeInt`、`GUID`、`Hash128` 等固定 Unity 值类型的 leaf 和 direct child TypeTree 形态已可结构化解码和语义替换；array/vector/staticvector/List/HashSet/map 元素与 registry payload 字段已保留独立 field path、offset 和 byte size，可用于字符串元素 patch，managed-reference registry payload 字符串、enum、bit_field、unknown fixed-size raw bytes、object 字段组合和 TypeTree schema 支撑的 array/vector/List/HashSet/map 已可整体变长替换，`first/second` 与 `key/value` map entry schema 已有 serialized 和 UnityFS 重建回归，ScriptableObject `key/value` map 解析、变长替换和 UnityFS 重建已有专门回归，且嵌套 vector `Array`、`List<T>` / `HashSet<T>` 集合 alias、enum、bit_field、unknown fixed-size raw bytes 与 managed-reference payload 字段已有重建回归覆盖；后续仍需继续补齐真实样本驱动的完整 managed reference registry / map entry 变体、unknown 字段结构语义和版本差异。
|
||||
2. Addressables 当前目标 JSON/compact 字段链已补齐；未识别的独立二进制格式仍返回明确错误，不静默降级。
|
||||
3. 官方 release 已可配置导入 CAS + ResourceRepository，并可通过 `resource.index` 查询现有资源索引；Resource metadata 已记录 release、平台、bundle path、parse status、TextAsset 名称和 TextUnit 摘要。
|
||||
4. 不能完成复杂对象字段重打包，也不能从真实 Crowdin 结果自动生成完整汉化文件集合。
|
||||
4. 尚未覆盖所有真实 Unity 版本、未知字段语义和任意复杂 AssetBundle 结构，也不能从真实 Crowdin 结果自动生成完整汉化文件集合；当前仅对已有真实/合成回归覆盖的结构宣称支持。
|
||||
|
||||
---
|
||||
|
||||
@@ -170,10 +172,10 @@ catalog 仍按“明确不支持”处理，不把低保真路径伪装成完整
|
||||
交付：
|
||||
|
||||
1. 已定义 `localized-patch-manifest.json`：目标官方版本、localized release、输出文件、hash、size、byte delta、TextUnit/provider/review trace 和回滚信息。
|
||||
2. 已支持 UnityFS TextAsset、TypeTree string field 和 managed-reference string field 的 localized patch 操作。
|
||||
2. 已支持 UnityFS TextAsset、TypeTree string field 和 managed-reference string field 的 localized patch 操作；普通 ZIP 条目在 `archive_entry` 可验证、内层可重解析时会解包、重建并重写外层 ZIP，路径穿越、symlink、混合直接/ZIP patch 和无效内层 bundle 明确失败。
|
||||
3. MonoBehaviour/ScriptableObject 字段替换必须依赖 P2 字段级解析结果。
|
||||
4. Patch 产物写入配置化汉化发布根下的 `.staging/<id>`，校验通过后发布到 `versions/<id>` 并切换 `current`；rollback 按 manifest 恢复上一 release。
|
||||
5. 成功后发布状态从 `not_localized` 切到 `localized`；`localized.status` 要求 state、current symlink 和 patch manifest 同时匹配当前官方 release。
|
||||
5. 成功后发布状态从 `not_localized` 切到 `localized`；`localized.status` 要求 state、current symlink 和 patch manifest 同时匹配当前官方 release，ZIP 外层文件和内层 UnityFS 也必须通过发布后重解析校验。
|
||||
|
||||
验收：
|
||||
|
||||
|
||||
@@ -244,7 +244,7 @@ trusted 和 release/TextUnit/provider/run provenance；`translation.tasks` 优
|
||||
13. 发布完成后先对比上一完整 release 和当前 release 的 `official-download-manifest.json`，写出 `official-resource-changes.json` 和 `crowdin-translation-handoff.json`。同一 destination 只有 size 或 BLAKE3 变化才算 modified；新增+变更资源进入解析/翻译 handoff，删除资源只进入差异记录。当前只预留 Crowdin 本地 handoff，不发外部 API 请求。
|
||||
14. 随后刷新 active release 下的 `official-parse-cache.json` 和 `official-textunit-index.json`，并从 Added/Modified 资源、parse cache 与 TextUnit 明细索引派生 `official-textunit-tasks.json`、`crowdin-textunit-queue.json` 和版本化的 `translation-tasks.sqlite`；up-to-date 轮询在已有有效解析缓存、TextUnit 明细索引和 TextUnit 队列时只读取摘要，不重复解析，重新同步队列时保留已有 worker 状态。
|
||||
15. 若启用 `--import-repository`，已校验 release 会被导入 CAS + `ResourceRepository`，并可经 `resource.index` 查询。历史 release 候选失效时，已有 CAS 对象会先经过完整性和元数据校验，再增加 release 引用并原子物化；当前 release 在 `official-cas-reuse-references.json` 中记录引用，staging/release 清理时递减，失败则回退网络并保留诊断。
|
||||
16. 官方同步报告默认给出 `localized_release_status=not_localized`，表示原版资源已发布、汉化资源未发布；UnityFS TextAsset patch 发布成功并通过 `localized-patch-manifest.json`、current symlink 和 release ID 校验后，`localized.status` 才返回 `localized`，表示原版和汉化两套资源都已发布。`translation.proofread` 只会把 workflow 标记成 `manual_proofreading` / `translation.manual_proofreading`，不会回退已发布汉化 release 的发布状态。
|
||||
16. 官方同步报告默认给出 `localized_release_status=not_localized`，表示原版资源已发布、汉化资源未发布；受支持的 UnityFS TextAsset、TypeTree string field 和 managed-reference string field patch 发布成功并通过 `localized-patch-manifest.json`、current symlink、release ID 及 ZIP 内层重解析校验后，`localized.status` 才返回 `localized`，表示原版和汉化两套资源都已发布。`translation.proofread` 只会把 workflow 标记成 `manual_proofreading` / `translation.manual_proofreading`，不会回退已发布汉化 release 的发布状态。
|
||||
|
||||
维护期特殊分支：如果官方 launcher/server-info 已经指向新资源根，但 client-patch seed marker 或必需 seed catalog 仍返回 403/404 等未开放状态，`bat` 返回 `waiting_for_official_resources`，保留现有 `current`，不创建失败 staging；若本轮启用 `--auto-discover`，会在 `<output>/official-launcher-bootstrap.pending.json` 写入待处理 launcher bootstrap 证据，供后续排障和自研客户端开发使用。
|
||||
|
||||
@@ -347,7 +347,8 @@ JSON-RPC 2.0 服务，是面向上层服务（Go 层）的**主要跨语言边
|
||||
`resource.state/sync/verify/repair/manifest/list/index`、`parse.status/text_units/errors`、
|
||||
`translation.tasks/handoff/task.update/proofread`、`localized.status`、`catalog.*` 与
|
||||
`task.status/list/cancel/logs` 已实现；文件级 `patch.apply` / `unityfs.patch_*`
|
||||
已实现，发布级 patch 与复杂 UnityFS 语义编辑待引擎；
|
||||
和受支持 localized publish/rollback 已实现，`archive_entry` 可验证时会重写
|
||||
外层 ZIP；通用发布级 patch 与复杂 UnityFS 语义编辑仍待后续；
|
||||
`task.create` 按设计暂不开放通用任务入口；
|
||||
`daemon.restart` 通过 Rust lifecycle controller 复用 CLI restart 路径；
|
||||
`daemon.clean-stable` 仍由 CLI 侧按进程生命周期显式执行。
|
||||
|
||||
@@ -68,7 +68,7 @@ bat parse clear-cache \
|
||||
bat parse repack --repack-spec /tmp/bat-repack.json
|
||||
```
|
||||
|
||||
重打包写入独立的 `target_bundle`，逐个操作后由底层 UnityFS patch 实现重建并校验，不允许 source 和 target 相同。
|
||||
重打包写入独立的 `target_bundle`，逐个操作后由底层 UnityFS patch 实现重建并校验，不允许 source 和 target 相同。重建会保留已识别的 block 压缩、alignment、目录和未修改对象内容；未知压缩模式或无法证明保真的结构会失败。
|
||||
|
||||
## 翻译工作台与发布
|
||||
|
||||
@@ -340,7 +340,8 @@ release，校验工作台与当前 TextUnit 索引的 source/location 一致后
|
||||
范围内的 TextAsset、TypeTree string field 和 managed-reference string field
|
||||
patch。校验通过后才原子切换 `localized/current`，并在 release manifest 中记录
|
||||
源/目标 BLAKE3、字节数、patch kind、TextUnit、provider、review、发布时重新计算的
|
||||
Glossary QA/override 和 rollback 信息。ZIP 内 bundle 不会被静默改写。
|
||||
Glossary QA/override 和 rollback 信息。若 TextUnit 带有 `archive_entry`，发布会在 staging 内验证 ZIP 条目路径，修改并重解析内层 UnityFS 后重写外层 ZIP；路径不安全、内层结构无效或重打包工具失败时不会发布不完整结果。可通过
|
||||
`--unzip <PATH>`、`--zip <PATH>` 或对应的 `BAT_UNZIP`、`BAT_ZIP` 配置工具路径。
|
||||
|
||||
使用人工编辑的工作台发布：
|
||||
|
||||
|
||||
@@ -442,6 +442,12 @@ daemon 重启后仍处于 `queued` 或 `running` 的历史任务会被标记为
|
||||
`data` 会返回 source / patch 或 replacement / target 的 size 与 BLAKE3。`target_path`
|
||||
不能与输入文件相同。
|
||||
|
||||
`localized.publish` 从当前官方 TextUnit/工作台生成受支持的 UnityFS patch
|
||||
operation；当 TextUnit 带有 `archive_entry` 时，发布 manifest 的 operation
|
||||
会记录该可选字段，Rust 发布器会在独立 staging 中校验、重建内层 UnityFS 并
|
||||
重写外层 ZIP。ZIP 路径、内层解析或重打包校验失败时整个发布失败，不会只发布
|
||||
部分结果。
|
||||
|
||||
仍关闭的范围：通用 manifest 驱动的发布级 `patch build` / `patch rollback`、复杂 UnityFS 语义编辑、
|
||||
`unityfs.inspect`、通用 manifest 驱动 release 切换。调用这些规划方法仍返回
|
||||
`BAT-ERR-700003`。
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# 当前实现缺口清单
|
||||
|
||||
- **更新时间**：2026-09-04
|
||||
- **更新时间**：2026-09-10
|
||||
- **文档角色**：只记录尚未完成、仍需验证或仍需设计的工作，不重复维护完整实现状态。
|
||||
- **当前事实**：以源码、测试、稳定契约和 `CURRENT_STATUS.md` 为准。
|
||||
- **Go 进度**：`GO_STATUS.md`
|
||||
@@ -10,25 +10,27 @@
|
||||
|
||||
## 1. 当前工程缺口
|
||||
|
||||
### G-005：AssetBundle 复杂解析仍未完成
|
||||
### G-005：AssetBundle V1 已完成，完整兼容仍未完成
|
||||
|
||||
状态：**部分完成，继续推进**
|
||||
状态：**V1 已完成（仅限已验证结构），继续推进真实版本和复杂结构**
|
||||
|
||||
当前已具备 UnityFS 容器校验、directory 文件提取、serialized file
|
||||
object/type table/TypeTree 元数据、TextAsset、基础 MonoBehaviour 和
|
||||
ScriptableObject 字段读取、TextUnit 提取，以及受支持字段的文件级重建。
|
||||
ScriptableObject 字段读取、TextUnit 提取，以及受支持字段的文件级
|
||||
parse→modify→rebuild→reparse。重建会保留已识别的 block 压缩、alignment、
|
||||
directory 形态和未修改对象/字段，并明确拒绝未知压缩或无法证明保真的输入。
|
||||
|
||||
仍需完成：
|
||||
|
||||
- 用真实资源 fixture 覆盖更多 MonoBehaviour、ScriptableObject、Unity 版本差异、
|
||||
复杂容器和 managed reference registry/map entry 变体。
|
||||
- 为未知字段补充结构语义；不能把低保真猜测当作已支持格式。
|
||||
- 完成发布级复杂对象重打包，并把 bundle、serialized file、path id、class id、
|
||||
field path、offset 和 byte size 的定位信息贯通到稳定发布流程。
|
||||
- 扩大真实 Unity 版本、复杂容器、未知字段和 managed-reference/map 变体覆盖；
|
||||
当前 V1 不等价于任意 AssetBundle 结构的通用重打包。
|
||||
|
||||
现有证据：`crates/bat-assetbundle` 的单元/重建测试、隔离真实 UnityFS 回归和
|
||||
`bat-infrastructure` 的解析缓存测试。新增格式覆盖必须同时补真实 fixture、回归测试
|
||||
和文档。
|
||||
现有证据：`crates/bat-assetbundle` 的单元/压缩/对齐/变长重建测试、隔离真实 UnityFS
|
||||
回归和 `bat-infrastructure` 的解析缓存、ZIP 内 bundle 发布测试。新增格式覆盖必须
|
||||
同时补真实 fixture、回归测试和文档。
|
||||
|
||||
### G-006：通用 Patch 发布仍未完成
|
||||
|
||||
@@ -42,7 +44,8 @@ string field、managed-reference string field 写入及 localized publish/rollba
|
||||
仍需完成：
|
||||
|
||||
- 通用 manifest 驱动的跨类型 patch build/apply/publish/rollback。
|
||||
- 复杂 AssetBundle 重打包和完整翻译文件集合构建。
|
||||
- 任意复杂 AssetBundle 重打包和完整翻译文件集合构建；当前 localized publish 已支持
|
||||
可验证 ZIP 内 bundle 的外层 ZIP 重写，但不扩大 UnityFS 结构支持范围。
|
||||
- 原版 release 与 localized release 双发布后的查询、分发和清理策略。
|
||||
|
||||
所有发布产物必须先进入独立 staging，通过完整性校验后再原子发布；失败不得改变
|
||||
@@ -143,7 +146,7 @@ Rust `bat` 已提供独立项目级 `glossary.sqlite`：term/alias/recommended/a
|
||||
|
||||
## 3. 后续推进顺序
|
||||
|
||||
1. 继续 G-005：真实 AssetBundle 样本、复杂字段解析和发布级重打包。
|
||||
1. 继续 G-005：更多真实 AssetBundle 样本、复杂字段解析、版本差异和任意结构重打包。
|
||||
2. 继续 G-006/G-011D：通用 manifest Patch 和双 release 查询/清理策略。
|
||||
3. 继续 G-011/G-012/G-013/G-014：资源查询、TM/Glossary 扩展和 Provider
|
||||
扩展体系。
|
||||
|
||||
@@ -5098,6 +5098,7 @@ fn sync_command_rpc_method(options: &CliOptions, command_name: &str) -> Option<&
|
||||
&& options.config.curl_command == defaults.curl_command
|
||||
&& options.config.curl_proxy == defaults.curl_proxy
|
||||
&& options.config.unzip_command == defaults.unzip_command
|
||||
&& options.config.zip_command == defaults.zip_command
|
||||
&& !options.config.dry_run
|
||||
&& !options.config.plan
|
||||
&& options.config.audit_local == defaults.audit_local
|
||||
@@ -5351,6 +5352,7 @@ fn build_doctor_report(
|
||||
command_check("curl", &config.curl_command),
|
||||
proxy_check(&config.curl_proxy),
|
||||
command_check("unzip", &config.unzip_command),
|
||||
command_check("zip", &config.zip_command),
|
||||
];
|
||||
|
||||
let pid_path = daemon_pid_path(state_dir);
|
||||
@@ -6246,6 +6248,8 @@ fn daemon_child_args(options: &CliOptions) -> Vec<String> {
|
||||
}
|
||||
args.push("--unzip".to_string());
|
||||
args.push(config.unzip_command.to_string_lossy().to_string());
|
||||
args.push("--zip".to_string());
|
||||
args.push(config.zip_command.to_string_lossy().to_string());
|
||||
if config.force {
|
||||
args.push("--force".to_string());
|
||||
}
|
||||
@@ -6575,6 +6579,9 @@ fn apply_bat_env_overrides(
|
||||
if let Some(v) = value("BAT_UNZIP") {
|
||||
options.config.unzip_command = PathBuf::from(v);
|
||||
}
|
||||
if let Some(v) = value("BAT_ZIP") {
|
||||
options.config.zip_command = PathBuf::from(v);
|
||||
}
|
||||
if let Some(v) = value("BAT_TRANSLATION_PROVIDER") {
|
||||
options.translation_provider = Some(v);
|
||||
}
|
||||
@@ -7077,6 +7084,9 @@ fn parse_args_with_env(
|
||||
"--unzip" => {
|
||||
options.config.unzip_command = PathBuf::from(next_option_value(&mut args, &flag)?);
|
||||
}
|
||||
"--zip" => {
|
||||
options.config.zip_command = PathBuf::from(next_option_value(&mut args, &flag)?);
|
||||
}
|
||||
"--dry-run" => {
|
||||
options.config.dry_run = true;
|
||||
options.sync_option_explicit = true;
|
||||
@@ -7744,7 +7754,7 @@ fn parse_args_with_env(
|
||||
CliCommand::Doctor => {
|
||||
if options.sync_option_explicit {
|
||||
return Err(anyhow::anyhow!(
|
||||
"doctor 只接受 --output、--state-dir、--curl、--proxy 和 --unzip 等诊断参数"
|
||||
"doctor 只接受 --output、--state-dir、--curl、--proxy、--unzip 和 --zip 等诊断参数"
|
||||
));
|
||||
}
|
||||
options.progress = false;
|
||||
@@ -7753,7 +7763,7 @@ fn parse_args_with_env(
|
||||
CliCommand::DoctorCas => {
|
||||
if doctor_cas_has_disallowed_sync_options(&options) {
|
||||
return Err(anyhow::anyhow!(
|
||||
"doctor cas 只接受 --output、--import-repository、--import-cas-root、--import-resource-db、--state-dir、--curl、--proxy 和 --unzip 等诊断参数"
|
||||
"doctor cas 只接受 --output、--import-repository、--import-cas-root、--import-resource-db、--state-dir、--curl、--proxy、--unzip 和 --zip 等诊断参数"
|
||||
));
|
||||
}
|
||||
options.progress = false;
|
||||
@@ -7762,7 +7772,7 @@ fn parse_args_with_env(
|
||||
CliCommand::CleanStable => {
|
||||
if options.sync_option_explicit {
|
||||
return Err(anyhow::anyhow!(
|
||||
"doctor/clean-stable 只接受 --output、--state-dir、--curl、--proxy 和 --unzip 等诊断参数"
|
||||
"doctor/clean-stable 只接受 --output、--state-dir、--curl、--proxy、--unzip 和 --zip 等诊断参数"
|
||||
));
|
||||
}
|
||||
options.progress = false;
|
||||
@@ -8181,6 +8191,7 @@ fn doctor_cas_has_disallowed_sync_options(options: &CliOptions) -> bool {
|
||||
allowed.curl_command = options.config.curl_command.clone();
|
||||
allowed.curl_proxy = options.config.curl_proxy.clone();
|
||||
allowed.unzip_command = options.config.unzip_command.clone();
|
||||
allowed.zip_command = options.config.zip_command.clone();
|
||||
allowed.import_repository = options.config.import_repository;
|
||||
allowed.import_cas_root = options.config.import_cas_root.clone();
|
||||
allowed.import_resource_repository_path =
|
||||
@@ -8412,12 +8423,13 @@ fn ensure_command_not_set(command: CliCommand, next: &str) -> anyhow::Result<()>
|
||||
}
|
||||
}
|
||||
|
||||
/// 判断 curl/代理/unzip 是否偏离基线。基线是 `config.toml` 和环境变量应用后的
|
||||
/// 判断 curl/代理/unzip/zip 是否偏离基线。基线是 `config.toml` 和环境变量应用后的
|
||||
/// 配置快照，因此只有命令行显式传入才算"非默认"。
|
||||
fn tools_are_non_default(config: &OfficialUpdateConfig, baseline: &OfficialUpdateConfig) -> bool {
|
||||
config.curl_command != baseline.curl_command
|
||||
|| config.curl_proxy != baseline.curl_proxy
|
||||
|| config.unzip_command != baseline.unzip_command
|
||||
|| config.zip_command != baseline.zip_command
|
||||
}
|
||||
|
||||
fn translation_memory_has_unsupported_query_filters(options: &CliOptions) -> bool {
|
||||
|
||||
@@ -1537,6 +1537,8 @@ fn parses_explicit_source_and_disable_repair() {
|
||||
"--no-repair",
|
||||
"--unzip",
|
||||
"/usr/bin/unzip",
|
||||
"--zip",
|
||||
"/usr/bin/zip",
|
||||
])
|
||||
.unwrap();
|
||||
let config = options.config;
|
||||
@@ -1547,6 +1549,7 @@ fn parses_explicit_source_and_disable_repair() {
|
||||
assert!(!config.audit_local);
|
||||
assert!(!config.repair);
|
||||
assert_eq!(config.unzip_command, PathBuf::from("/usr/bin/unzip"));
|
||||
assert_eq!(config.zip_command, PathBuf::from("/usr/bin/zip"));
|
||||
assert!(matches!(
|
||||
config.server_info_source,
|
||||
Some(OfficialServerInfoSource::OfficialUrl(ref url))
|
||||
@@ -2226,6 +2229,8 @@ fn daemon_child_args_preserve_sync_options() {
|
||||
"http://127.0.0.1:7890",
|
||||
"--unzip",
|
||||
"/usr/bin/unzip",
|
||||
"--zip",
|
||||
"/usr/bin/zip",
|
||||
"--interval",
|
||||
"30m",
|
||||
"--error-retry",
|
||||
@@ -2242,6 +2247,9 @@ fn daemon_child_args_preserve_sync_options() {
|
||||
assert!(args
|
||||
.windows(2)
|
||||
.any(|pair| pair == ["--output", "/tmp/daemon-output"]));
|
||||
assert!(args
|
||||
.windows(2)
|
||||
.any(|pair| pair == ["--zip", "/usr/bin/zip"]));
|
||||
assert!(args
|
||||
.windows(2)
|
||||
.any(|pair| pair == ["--localized-output", "/tmp/daemon-localized"]));
|
||||
|
||||
@@ -61,6 +61,7 @@ import_resource_repository_path = ''
|
||||
curl_command = 'curl'
|
||||
proxy = 'auto'
|
||||
unzip_command = 'unzip'
|
||||
zip_command = 'zip'
|
||||
download_concurrency = 8
|
||||
|
||||
[translation.worker]
|
||||
@@ -138,6 +139,7 @@ struct NetworkSection {
|
||||
curl_command: Option<PathBuf>,
|
||||
proxy: Option<CurlProxyConfig>,
|
||||
unzip_command: Option<PathBuf>,
|
||||
zip_command: Option<PathBuf>,
|
||||
download_concurrency: Option<usize>,
|
||||
}
|
||||
|
||||
@@ -356,6 +358,9 @@ impl BatConfigFile {
|
||||
if let Some(value) = self.network.unzip_command.as_ref() {
|
||||
options.config.unzip_command = value.clone();
|
||||
}
|
||||
if let Some(value) = self.network.zip_command.as_ref() {
|
||||
options.config.zip_command = value.clone();
|
||||
}
|
||||
if let Some(value) = self.network.download_concurrency {
|
||||
options.config.download_concurrency = value;
|
||||
}
|
||||
@@ -575,6 +580,13 @@ impl BatConfigFile {
|
||||
line_number,
|
||||
)?);
|
||||
}
|
||||
(SectionPath::Network, "zip_command") => {
|
||||
self.network.zip_command = Some(parse_required_path(
|
||||
value,
|
||||
"network.zip_command",
|
||||
line_number,
|
||||
)?);
|
||||
}
|
||||
(SectionPath::Network, "download_concurrency") => {
|
||||
self.network.download_concurrency = Some(parse_download_concurrency(
|
||||
&parse_scalar_text(value, "network.download_concurrency", line_number)?,
|
||||
@@ -1151,6 +1163,7 @@ import_resource_repository_path = '/srv/resources.sqlite'
|
||||
curl_command = '/usr/bin/curl'
|
||||
proxy = 'http://127.0.0.1:7890'
|
||||
unzip_command = '/usr/bin/unzip'
|
||||
zip_command = '/usr/bin/zip'
|
||||
download_concurrency = 16
|
||||
|
||||
[translation.worker]
|
||||
|
||||
@@ -422,6 +422,7 @@ Sync:
|
||||
--proxy <URL|auto|none> curl proxy override (default: auto from env)
|
||||
--no-proxy Force direct curl connections
|
||||
--unzip <PATH> unzip executable (default: unzip)
|
||||
--zip <PATH> zip executable (default: zip)
|
||||
--dry-run Do not write sync state
|
||||
--plan Include planned URLs in dry-run
|
||||
--force Force download/refresh
|
||||
|
||||
@@ -390,6 +390,10 @@ pub(super) fn publish_localized_report(
|
||||
official_release_id,
|
||||
Vec::new(),
|
||||
)
|
||||
.with_archive_commands(
|
||||
options.config.unzip_command.clone(),
|
||||
options.config.zip_command.clone(),
|
||||
)
|
||||
.with_operations(operations)
|
||||
.with_force(options.config.force);
|
||||
if let Some(release_id) = localized_release_id {
|
||||
|
||||
@@ -1,19 +1,21 @@
|
||||
//! Localized release publishing for verified UnityFS text patches.
|
||||
|
||||
use bat_assetbundle::{
|
||||
patch_unityfs_field, patch_unityfs_string_field, patch_unityfs_text_asset, FieldPatch,
|
||||
StringFieldPatch, TextAssetPatch,
|
||||
patch_unityfs_field, patch_unityfs_string_field, patch_unityfs_text_asset, FieldPatch, Parser,
|
||||
StringFieldPatch, TextAssetPatch, UnitySerializedField, UnitySerializedValue,
|
||||
};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::collections::BTreeMap;
|
||||
use std::fs;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::Command;
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
|
||||
use crate::path_security::{
|
||||
ensure_path_within_root, ensure_safe_directory_path, ensure_safe_file_target, lexical_absolute,
|
||||
read_file_no_symlink, write_file_atomic, STATE_FILE_MODE,
|
||||
};
|
||||
use crate::zip_validation::validate_zip_structure;
|
||||
|
||||
/// Atomic current pointer under the localized output root.
|
||||
pub const LOCALIZED_CURRENT_LINK: &str = "current";
|
||||
@@ -39,6 +41,8 @@ pub const LOCALIZED_TRANSLATION_STATUS_MANUAL_PROOFREADING_LABEL: &str = "人工
|
||||
pub struct LocalizedTextAssetPatch {
|
||||
/// Relative path of the UnityFS bundle under the official release.
|
||||
pub bundle_path: String,
|
||||
/// Relative path of the UnityFS bundle inside an outer ZIP archive.
|
||||
pub archive_entry: Option<String>,
|
||||
/// TextAsset replacement inside the bundle.
|
||||
pub text_asset: TextAssetPatch,
|
||||
/// TextUnit/provider metadata recorded in the localized manifest.
|
||||
@@ -50,6 +54,8 @@ pub struct LocalizedTextAssetPatch {
|
||||
pub struct LocalizedStringFieldPatch {
|
||||
/// Relative path of the UnityFS bundle under the official release.
|
||||
pub bundle_path: String,
|
||||
/// Relative path of the UnityFS bundle inside an outer ZIP archive.
|
||||
pub archive_entry: Option<String>,
|
||||
/// String field replacement inside the bundle.
|
||||
pub string_field: StringFieldPatch,
|
||||
/// TextUnit/provider metadata recorded in the localized manifest.
|
||||
@@ -61,6 +67,8 @@ pub struct LocalizedStringFieldPatch {
|
||||
pub struct LocalizedFieldPatch {
|
||||
/// Relative path of the UnityFS bundle under the official release.
|
||||
pub bundle_path: String,
|
||||
/// Relative path of the UnityFS bundle inside an outer ZIP archive.
|
||||
pub archive_entry: Option<String>,
|
||||
/// Semantic field replacement inside the bundle.
|
||||
pub field: FieldPatch,
|
||||
/// TextUnit/provider metadata recorded in the localized manifest.
|
||||
@@ -127,6 +135,10 @@ pub struct LocalizedPatchConfig {
|
||||
pub patches: Vec<LocalizedTextAssetPatch>,
|
||||
/// General UnityFS text/field operations to apply.
|
||||
pub operations: Vec<LocalizedPatchInput>,
|
||||
/// Unzip executable used for ZIP-inner bundle publication.
|
||||
pub unzip_command: PathBuf,
|
||||
/// Zip executable used for ZIP-inner bundle publication.
|
||||
pub zip_command: PathBuf,
|
||||
}
|
||||
|
||||
impl LocalizedPatchConfig {
|
||||
@@ -145,6 +157,8 @@ impl LocalizedPatchConfig {
|
||||
force: false,
|
||||
patches,
|
||||
operations: Vec::new(),
|
||||
unzip_command: PathBuf::from("unzip"),
|
||||
zip_command: PathBuf::from("zip"),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -166,6 +180,17 @@ impl LocalizedPatchConfig {
|
||||
self
|
||||
}
|
||||
|
||||
/// Sets the archive tools used for ZIP-inner bundle publication.
|
||||
pub fn with_archive_commands(
|
||||
mut self,
|
||||
unzip_command: impl Into<PathBuf>,
|
||||
zip_command: impl Into<PathBuf>,
|
||||
) -> Self {
|
||||
self.unzip_command = unzip_command.into();
|
||||
self.zip_command = zip_command.into();
|
||||
self
|
||||
}
|
||||
|
||||
fn published_release_id(&self) -> &str {
|
||||
self.localized_release_id
|
||||
.as_deref()
|
||||
@@ -193,6 +218,14 @@ impl LocalizedPatchInput {
|
||||
}
|
||||
}
|
||||
|
||||
fn archive_entry(&self) -> Option<&str> {
|
||||
match self {
|
||||
Self::TextAsset(operation) => operation.archive_entry.as_deref(),
|
||||
Self::StringField(operation) => operation.archive_entry.as_deref(),
|
||||
Self::Field(operation) => operation.archive_entry.as_deref(),
|
||||
}
|
||||
}
|
||||
|
||||
fn apply(&self, input: &[u8]) -> anyhow::Result<Vec<u8>> {
|
||||
match self {
|
||||
Self::TextAsset(operation) => {
|
||||
@@ -206,7 +239,7 @@ impl LocalizedPatchInput {
|
||||
}
|
||||
|
||||
fn manifest_operation(&self) -> anyhow::Result<LocalizedPatchOperation> {
|
||||
match self {
|
||||
let mut operation = match self {
|
||||
Self::TextAsset(operation) => Ok(LocalizedPatchOperation::from_text_asset_patch(
|
||||
&operation.text_asset,
|
||||
operation.metadata.as_ref(),
|
||||
@@ -219,7 +252,9 @@ impl LocalizedPatchInput {
|
||||
&operation.field,
|
||||
operation.metadata.as_ref(),
|
||||
),
|
||||
}
|
||||
}?;
|
||||
operation.archive_entry = self.archive_entry().map(str::to_string);
|
||||
Ok(operation)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -314,6 +349,9 @@ pub struct LocalizedPatchFile {
|
||||
/// One TextAsset patch operation recorded in the localized patch manifest.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct LocalizedPatchOperation {
|
||||
/// Relative path of the UnityFS bundle inside an outer ZIP archive.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub archive_entry: Option<String>,
|
||||
/// Patch operation kind, for example `unityfs_text_asset`.
|
||||
#[serde(default = "default_patch_operation_kind")]
|
||||
pub patch_kind: String,
|
||||
@@ -499,6 +537,11 @@ impl LocalizedPatchService {
|
||||
.join(&published_release_id);
|
||||
let current_path = config.localized_output_root.join(LOCALIZED_CURRENT_LINK);
|
||||
validate_config(config).map_err(anyhow::Error::msg)?;
|
||||
let state_path = config
|
||||
.localized_output_root
|
||||
.join(LOCALIZED_VERSION_STATE_FILE);
|
||||
let previous_state_bytes =
|
||||
read_file_no_symlink(&state_path, "汉化版本状态").map_err(anyhow::Error::msg)?;
|
||||
let previous_current_target = current_symlink_target(¤t_path)?;
|
||||
if let Some(target) = previous_current_target.as_deref() {
|
||||
validate_previous_current_target(&config.localized_output_root, target)?;
|
||||
@@ -514,6 +557,7 @@ impl LocalizedPatchService {
|
||||
!version_existed_before,
|
||||
¤t_path,
|
||||
previous_current_target.as_ref(),
|
||||
(&state_path, previous_state_bytes.as_deref()),
|
||||
) {
|
||||
return Err(anyhow::anyhow!(
|
||||
"{error}; rollback failed: {rollback_error}"
|
||||
@@ -751,14 +795,35 @@ impl LocalizedPatchService {
|
||||
ensure_safe_file_target(&staging, &target, "汉化 patch 输入")
|
||||
.map_err(anyhow::Error::msg)?;
|
||||
let original = fs::read(&target)?;
|
||||
let mut patched = original.clone();
|
||||
let mut manifest_operations = Vec::with_capacity(operations.len());
|
||||
for operation in operations {
|
||||
patched = operation
|
||||
.apply(&patched)
|
||||
.map_err(|error| anyhow::anyhow!("{bundle_path}: {error}"))?;
|
||||
manifest_operations.push(operation.manifest_operation()?);
|
||||
}
|
||||
let (patched, manifest_operations) = if operations
|
||||
.iter()
|
||||
.any(|operation| operation.archive_entry().is_some())
|
||||
{
|
||||
if operations
|
||||
.iter()
|
||||
.any(|operation| operation.archive_entry().is_none())
|
||||
{
|
||||
return Err(anyhow::anyhow!(
|
||||
"{bundle_path}: 不能混合直接 bundle patch 与 ZIP 内 bundle patch"
|
||||
));
|
||||
}
|
||||
rewrite_zip_bundle(
|
||||
&original,
|
||||
&operations,
|
||||
&config.unzip_command,
|
||||
&config.zip_command,
|
||||
)?
|
||||
} else {
|
||||
let mut patched = original.clone();
|
||||
let mut manifest_operations = Vec::with_capacity(operations.len());
|
||||
for operation in operations {
|
||||
patched = operation
|
||||
.apply(&patched)
|
||||
.map_err(|error| anyhow::anyhow!("{bundle_path}: {error}"))?;
|
||||
manifest_operations.push(operation.manifest_operation()?);
|
||||
}
|
||||
(patched, manifest_operations)
|
||||
};
|
||||
if original == patched {
|
||||
return Err(anyhow::anyhow!("patch produced no change: {bundle_path}"));
|
||||
}
|
||||
@@ -800,7 +865,12 @@ impl LocalizedPatchService {
|
||||
"汉化 patch manifest",
|
||||
)
|
||||
.map_err(anyhow::Error::msg)?;
|
||||
verify_patch_manifest_files(&config.official_release_root, &staging, &manifest)?;
|
||||
verify_patch_manifest_files(
|
||||
&config.official_release_root,
|
||||
&staging,
|
||||
&manifest,
|
||||
&config.unzip_command,
|
||||
)?;
|
||||
fs::create_dir_all(config.localized_output_root.join(LOCALIZED_VERSIONS_DIR))?;
|
||||
fs::rename(&staging, &version_path)?;
|
||||
switch_current_symlink(
|
||||
@@ -828,6 +898,7 @@ impl LocalizedPatchService {
|
||||
&config.official_release_root,
|
||||
&version_path,
|
||||
¤t_path,
|
||||
&config.unzip_command,
|
||||
)?;
|
||||
|
||||
Ok(LocalizedPatchReport {
|
||||
@@ -954,6 +1025,7 @@ impl LocalizedPatchOperation {
|
||||
) -> Self {
|
||||
Self::with_metadata(
|
||||
Self {
|
||||
archive_entry: None,
|
||||
patch_kind: "unityfs_text_asset".to_string(),
|
||||
serialized_file_path: patch.serialized_file_path.clone(),
|
||||
path_id: patch.path_id,
|
||||
@@ -981,6 +1053,7 @@ impl LocalizedPatchOperation {
|
||||
) -> Self {
|
||||
Self::with_metadata(
|
||||
Self {
|
||||
archive_entry: None,
|
||||
patch_kind: "unityfs_string_field".to_string(),
|
||||
serialized_file_path: patch.serialized_file_path.clone(),
|
||||
path_id: patch.path_id,
|
||||
@@ -1011,6 +1084,7 @@ impl LocalizedPatchOperation {
|
||||
let replacement = serde_json::to_vec(&patch.replacement)?;
|
||||
Ok(Self::with_metadata(
|
||||
Self {
|
||||
archive_entry: None,
|
||||
patch_kind: "unityfs_field".to_string(),
|
||||
serialized_file_path: patch.serialized_file_path.clone(),
|
||||
path_id: patch.path_id,
|
||||
@@ -1058,6 +1132,7 @@ fn default_patch_operation_kind() -> String {
|
||||
impl Default for LocalizedPatchOperation {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
archive_entry: None,
|
||||
patch_kind: default_patch_operation_kind(),
|
||||
serialized_file_path: String::new(),
|
||||
path_id: 0,
|
||||
@@ -1082,6 +1157,7 @@ fn verify_published_localized_release(
|
||||
official_release_root: &Path,
|
||||
version_path: &Path,
|
||||
current_path: &Path,
|
||||
unzip_command: &Path,
|
||||
) -> anyhow::Result<LocalizedPatchIntegrity> {
|
||||
let manifest = read_localized_patch_manifest_at(version_path)?.ok_or_else(|| {
|
||||
anyhow::anyhow!(
|
||||
@@ -1089,8 +1165,12 @@ fn verify_published_localized_release(
|
||||
version_path.join(LOCALIZED_PATCH_MANIFEST_FILE).display()
|
||||
)
|
||||
})?;
|
||||
let mut integrity =
|
||||
verify_patch_manifest_files(official_release_root, version_path, &manifest)?;
|
||||
let mut integrity = verify_patch_manifest_files(
|
||||
official_release_root,
|
||||
version_path,
|
||||
&manifest,
|
||||
unzip_command,
|
||||
)?;
|
||||
integrity.current_points_to_release = current_points_to_version(current_path, version_path)?;
|
||||
if !integrity.current_points_to_release {
|
||||
return Err(anyhow::anyhow!(
|
||||
@@ -1106,6 +1186,7 @@ fn verify_patch_manifest_files(
|
||||
official_release_root: &Path,
|
||||
localized_release_root: &Path,
|
||||
manifest: &LocalizedPatchManifest,
|
||||
unzip_command: &Path,
|
||||
) -> anyhow::Result<LocalizedPatchIntegrity> {
|
||||
let mut operation_count = 0usize;
|
||||
for file in &manifest.files {
|
||||
@@ -1151,6 +1232,15 @@ fn verify_patch_manifest_files(
|
||||
file.path
|
||||
));
|
||||
}
|
||||
let archive_operations = file
|
||||
.text_asset_operations
|
||||
.iter()
|
||||
.filter(|operation| operation.archive_entry.is_some())
|
||||
.collect::<Vec<_>>();
|
||||
if !archive_operations.is_empty() {
|
||||
verify_zip_operations(unzip_command, &official_path, &archive_operations, false)?;
|
||||
verify_zip_operations(unzip_command, &localized_path, &archive_operations, true)?;
|
||||
}
|
||||
operation_count += file.text_asset_operations.len();
|
||||
}
|
||||
if manifest.file_count != manifest.files.len() {
|
||||
@@ -1174,6 +1264,316 @@ fn verify_patch_manifest_files(
|
||||
})
|
||||
}
|
||||
|
||||
fn rewrite_zip_bundle(
|
||||
original: &[u8],
|
||||
operations: &[LocalizedPatchInput],
|
||||
unzip_command: &Path,
|
||||
zip_command: &Path,
|
||||
) -> anyhow::Result<(Vec<u8>, Vec<LocalizedPatchOperation>)> {
|
||||
let archive_temp = tempfile::tempdir()?;
|
||||
let archive_path = archive_temp.path().join("source.zip");
|
||||
fs::write(&archive_path, original)?;
|
||||
validate_zip_structure(&archive_path).map_err(anyhow::Error::msg)?;
|
||||
let entries = list_archive_entries(unzip_command, &archive_path)?;
|
||||
let extract_path = archive_temp.path().join("extract");
|
||||
fs::create_dir(&extract_path)?;
|
||||
|
||||
let extract_output = Command::new(unzip_command)
|
||||
.arg("-q")
|
||||
.arg("-o")
|
||||
.arg(&archive_path)
|
||||
.arg("-d")
|
||||
.arg(&extract_path)
|
||||
.output()
|
||||
.map_err(|error| anyhow::anyhow!("启动 unzip 解包 ZIP 失败：{error}"))?;
|
||||
ensure_command_success(&extract_output, "解包 ZIP")?;
|
||||
reject_extracted_symlinks(&extract_path)?;
|
||||
|
||||
let mut manifest_operations = Vec::with_capacity(operations.len());
|
||||
let mut patched_entries = BTreeMap::<String, Vec<u8>>::new();
|
||||
for operation in operations {
|
||||
let archive_entry = operation.archive_entry().ok_or_else(|| {
|
||||
anyhow::anyhow!("ZIP patch 缺少 archive_entry：{}", operation.bundle_path())
|
||||
})?;
|
||||
validate_archive_member_name(archive_entry)?;
|
||||
if !entries.contains(archive_entry) {
|
||||
return Err(anyhow::anyhow!(
|
||||
"ZIP 中不存在 archive entry：{}!{}",
|
||||
operation.bundle_path(),
|
||||
archive_entry
|
||||
));
|
||||
}
|
||||
let entry_path = extract_path.join(archive_entry);
|
||||
ensure_path_within_root(&extract_path, &entry_path).map_err(anyhow::Error::msg)?;
|
||||
ensure_safe_file_target(&extract_path, &entry_path, "ZIP 内 bundle")
|
||||
.map_err(anyhow::Error::msg)?;
|
||||
let current = if let Some(current) = patched_entries.get(archive_entry) {
|
||||
current.clone()
|
||||
} else {
|
||||
fs::read(&entry_path)?
|
||||
};
|
||||
let patched = operation.apply(¤t).map_err(|error| {
|
||||
anyhow::anyhow!("{}!{}: {error}", operation.bundle_path(), archive_entry)
|
||||
})?;
|
||||
fs::write(&entry_path, &patched)?;
|
||||
patched_entries.insert(archive_entry.to_string(), patched);
|
||||
manifest_operations.push(operation.manifest_operation()?);
|
||||
}
|
||||
|
||||
let output_path = archive_temp.path().join("localized.zip");
|
||||
let zip_output = Command::new(zip_command)
|
||||
.current_dir(&extract_path)
|
||||
.arg("-q")
|
||||
.arg("-r")
|
||||
.arg(&output_path)
|
||||
.arg(".")
|
||||
.output()
|
||||
.map_err(|error| anyhow::anyhow!("启动 zip 重打包 ZIP 失败：{error}"))?;
|
||||
ensure_command_success(&zip_output, "重打包 ZIP")?;
|
||||
validate_zip_structure(&output_path).map_err(anyhow::Error::msg)?;
|
||||
let patched = fs::read(output_path)?;
|
||||
Ok((patched, manifest_operations))
|
||||
}
|
||||
|
||||
fn list_archive_entries(
|
||||
unzip_command: &Path,
|
||||
archive_path: &Path,
|
||||
) -> anyhow::Result<std::collections::BTreeSet<String>> {
|
||||
let output = Command::new(unzip_command)
|
||||
.arg("-Z1")
|
||||
.arg(archive_path)
|
||||
.output()
|
||||
.map_err(|error| anyhow::anyhow!("启动 unzip 列出 ZIP 条目失败：{error}"))?;
|
||||
ensure_command_success(&output, "列出 ZIP 条目")?;
|
||||
let entries = std::str::from_utf8(&output.stdout)
|
||||
.map_err(|_| anyhow::anyhow!("ZIP 条目列表不是 UTF-8：{}", archive_path.display()))?
|
||||
.lines()
|
||||
.map(|line| line.trim_end_matches('\r').to_string())
|
||||
.filter(|entry| !entry.is_empty() && !entry.ends_with('/'))
|
||||
.collect::<std::collections::BTreeSet<_>>();
|
||||
for entry in &entries {
|
||||
validate_archive_member_name(entry)?;
|
||||
}
|
||||
Ok(entries)
|
||||
}
|
||||
|
||||
fn verify_zip_operations(
|
||||
unzip_command: &Path,
|
||||
archive_path: &Path,
|
||||
operations: &[&LocalizedPatchOperation],
|
||||
verify_replacements: bool,
|
||||
) -> anyhow::Result<()> {
|
||||
validate_zip_structure(archive_path).map_err(anyhow::Error::msg)?;
|
||||
let entries = list_archive_entries(unzip_command, archive_path)?;
|
||||
for operation in operations {
|
||||
let archive_entry = operation.archive_entry.as_deref().ok_or_else(|| {
|
||||
anyhow::anyhow!(
|
||||
"manifest ZIP operation 缺少 archive_entry：{}",
|
||||
archive_path.display()
|
||||
)
|
||||
})?;
|
||||
validate_archive_member_name(archive_entry)?;
|
||||
if !entries.contains(archive_entry) {
|
||||
return Err(anyhow::anyhow!(
|
||||
"manifest ZIP operation 指向不存在的 archive entry：{}!{}",
|
||||
archive_path.display(),
|
||||
archive_entry
|
||||
));
|
||||
}
|
||||
let output = Command::new(unzip_command)
|
||||
.arg("-p")
|
||||
.arg(archive_path)
|
||||
.arg(archive_entry)
|
||||
.output()
|
||||
.map_err(|error| anyhow::anyhow!("启动 unzip 读取 ZIP 条目失败：{error}"))?;
|
||||
ensure_command_success(&output, "读取 ZIP 条目")?;
|
||||
let parsed = bat_assetbundle::UnityFsParser::new()
|
||||
.parse(&output.stdout)
|
||||
.map_err(|error| {
|
||||
anyhow::anyhow!(
|
||||
"ZIP 内 UnityFS 重解析失败 {}!{}：{error}",
|
||||
archive_path.display(),
|
||||
archive_entry
|
||||
)
|
||||
})?;
|
||||
if verify_replacements {
|
||||
verify_archive_operation_replacement(&parsed, operation, archive_entry)?;
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn verify_archive_operation_replacement(
|
||||
parsed: &bat_assetbundle::ParsedAssetBundle,
|
||||
operation: &LocalizedPatchOperation,
|
||||
archive_entry: &str,
|
||||
) -> anyhow::Result<()> {
|
||||
let Some(serialized_file) = parsed
|
||||
.serialized_files
|
||||
.iter()
|
||||
.find(|file| file.source_path.as_deref() == Some(operation.serialized_file_path.as_str()))
|
||||
else {
|
||||
return Err(anyhow::anyhow!(
|
||||
"ZIP 内 UnityFS manifest serialized file 不存在：{}!{}:{}",
|
||||
archive_entry,
|
||||
operation.serialized_file_path,
|
||||
operation.path_id
|
||||
));
|
||||
};
|
||||
|
||||
let bytes: Vec<u8> = match operation.patch_kind.as_str() {
|
||||
"unityfs_text_asset" => parsed
|
||||
.text_assets
|
||||
.iter()
|
||||
.find(|asset| {
|
||||
asset.source_path.as_deref() == Some(operation.serialized_file_path.as_str())
|
||||
&& asset.path_id == operation.path_id
|
||||
})
|
||||
.map(|asset| asset.bytes.clone())
|
||||
.ok_or_else(|| {
|
||||
anyhow::anyhow!(
|
||||
"ZIP 内 UnityFS manifest TextAsset 不存在：{}!{}:{}",
|
||||
archive_entry,
|
||||
operation.serialized_file_path,
|
||||
operation.path_id
|
||||
)
|
||||
})?,
|
||||
"unityfs_string_field" => {
|
||||
let field_path = operation.field_path.as_deref().ok_or_else(|| {
|
||||
anyhow::anyhow!(
|
||||
"ZIP 内 UnityFS string operation 缺少 field_path：{}!{}:{}",
|
||||
archive_entry,
|
||||
operation.serialized_file_path,
|
||||
operation.path_id
|
||||
)
|
||||
})?;
|
||||
let fields = serialized_file.fields_for_object(operation.path_id)?;
|
||||
let value = find_archive_field_value(&fields, field_path).ok_or_else(|| {
|
||||
anyhow::anyhow!(
|
||||
"ZIP 内 UnityFS manifest 字段不存在：{}!{}:{}:{}",
|
||||
archive_entry,
|
||||
operation.serialized_file_path,
|
||||
operation.path_id,
|
||||
field_path
|
||||
)
|
||||
})?;
|
||||
match value {
|
||||
UnitySerializedValue::String(value) => value.as_bytes().to_vec(),
|
||||
other => {
|
||||
return Err(anyhow::anyhow!(
|
||||
"ZIP 内 UnityFS manifest 字段不是 string：{}!{}:{}:{} ({other:?})",
|
||||
archive_entry,
|
||||
operation.serialized_file_path,
|
||||
operation.path_id,
|
||||
field_path
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
// Semantic field replacements retain structure and are already
|
||||
// verified by the field patch operation before the archive rewrite.
|
||||
_ => return Ok(()),
|
||||
};
|
||||
|
||||
let actual_hash = blake3::hash(&bytes).to_hex().to_string();
|
||||
if actual_hash != operation.replacement_blake3
|
||||
|| bytes.len() as u64 != operation.replacement_bytes
|
||||
{
|
||||
return Err(anyhow::anyhow!(
|
||||
"ZIP 内 UnityFS replacement 校验失败：{}!{}:{}，期望 hash={} bytes={}，实际 hash={} bytes={}",
|
||||
archive_entry,
|
||||
operation.serialized_file_path,
|
||||
operation.path_id,
|
||||
operation.replacement_blake3,
|
||||
operation.replacement_bytes,
|
||||
actual_hash,
|
||||
bytes.len()
|
||||
));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn find_archive_field_value<'a>(
|
||||
fields: &'a [UnitySerializedField],
|
||||
field_path: &str,
|
||||
) -> Option<&'a UnitySerializedValue> {
|
||||
for field in fields {
|
||||
if field.path == field_path {
|
||||
return Some(&field.value);
|
||||
}
|
||||
let children = match &field.value {
|
||||
UnitySerializedValue::Object(children)
|
||||
| UnitySerializedValue::Array(children)
|
||||
| UnitySerializedValue::Map(children)
|
||||
| UnitySerializedValue::ManagedReference {
|
||||
fields: children, ..
|
||||
}
|
||||
| UnitySerializedValue::ManagedReferenceRegistry {
|
||||
fields: children, ..
|
||||
} => children,
|
||||
_ => continue,
|
||||
};
|
||||
if let Some(value) = find_archive_field_value(children, field_path) {
|
||||
return Some(value);
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
fn ensure_command_success(output: &std::process::Output, action: &str) -> anyhow::Result<()> {
|
||||
if output.status.success() {
|
||||
return Ok(());
|
||||
}
|
||||
let stderr = String::from_utf8_lossy(&output.stderr).trim().to_string();
|
||||
let stdout = String::from_utf8_lossy(&output.stdout).trim().to_string();
|
||||
Err(anyhow::anyhow!(
|
||||
"{action}失败：{}{}",
|
||||
stderr,
|
||||
if stdout.is_empty() {
|
||||
String::new()
|
||||
} else {
|
||||
format!("；输出={stdout}")
|
||||
}
|
||||
))
|
||||
}
|
||||
|
||||
fn validate_archive_member_name(name: &str) -> anyhow::Result<()> {
|
||||
let is_windows_absolute = name.as_bytes().get(1) == Some(&b':');
|
||||
if name.is_empty() || name.contains('\0') || name.contains('\\') || is_windows_absolute {
|
||||
return Err(anyhow::anyhow!("ZIP 条目路径无效：{name}"));
|
||||
}
|
||||
let path = Path::new(name);
|
||||
for component in path.components() {
|
||||
match component {
|
||||
std::path::Component::Normal(_) | std::path::Component::CurDir => {}
|
||||
_ => return Err(anyhow::anyhow!("ZIP 条目路径不安全：{name}")),
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn reject_extracted_symlinks(root: &Path) -> anyhow::Result<()> {
|
||||
for entry in fs::read_dir(root)? {
|
||||
let path = entry?.path();
|
||||
let metadata = fs::symlink_metadata(&path)?;
|
||||
if metadata.file_type().is_symlink() {
|
||||
return Err(anyhow::anyhow!(
|
||||
"ZIP 解包结果包含 symlink，拒绝重打包：{}",
|
||||
path.display()
|
||||
));
|
||||
}
|
||||
if metadata.is_dir() {
|
||||
reject_extracted_symlinks(&path)?;
|
||||
} else if !metadata.is_file() {
|
||||
return Err(anyhow::anyhow!(
|
||||
"ZIP 解包结果包含非普通文件：{}",
|
||||
path.display()
|
||||
));
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn current_symlink_target(current_path: &Path) -> anyhow::Result<Option<PathBuf>> {
|
||||
match fs::symlink_metadata(current_path) {
|
||||
Ok(metadata) if metadata.file_type().is_symlink() => Ok(Some(fs::read_link(current_path)?)),
|
||||
@@ -1232,7 +1632,9 @@ fn rollback_failed_publish(
|
||||
remove_version_path: bool,
|
||||
current_path: &Path,
|
||||
previous_current_target: Option<&PathBuf>,
|
||||
state_restore: (&Path, Option<&[u8]>),
|
||||
) -> anyhow::Result<()> {
|
||||
let (state_path, previous_state_bytes) = state_restore;
|
||||
remove_owned_path(staging)?;
|
||||
if remove_version_path {
|
||||
remove_owned_path(version_path)?;
|
||||
@@ -1249,6 +1651,16 @@ fn rollback_failed_publish(
|
||||
}) {
|
||||
restore_current_symlink(localized_output_root, current_path, previous_current_target)?;
|
||||
}
|
||||
remove_owned_path(state_path)?;
|
||||
if let Some(previous_state_bytes) = previous_state_bytes {
|
||||
write_file_atomic(
|
||||
state_path,
|
||||
previous_state_bytes,
|
||||
STATE_FILE_MODE,
|
||||
"恢复汉化版本状态",
|
||||
)
|
||||
.map_err(anyhow::Error::msg)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -1415,6 +1827,110 @@ mod tests {
|
||||
use super::*;
|
||||
use tempfile::TempDir;
|
||||
|
||||
fn push_u32_be(data: &mut Vec<u8>, value: u32) {
|
||||
data.extend_from_slice(&value.to_be_bytes());
|
||||
}
|
||||
|
||||
fn push_u64_be(data: &mut Vec<u8>, value: u64) {
|
||||
data.extend_from_slice(&value.to_be_bytes());
|
||||
}
|
||||
|
||||
fn push_u64_le(data: &mut Vec<u8>, value: u64) {
|
||||
data.extend_from_slice(&value.to_le_bytes());
|
||||
}
|
||||
|
||||
fn push_u32_le(data: &mut Vec<u8>, value: u32) {
|
||||
data.extend_from_slice(&value.to_le_bytes());
|
||||
}
|
||||
|
||||
fn push_i16_le(data: &mut Vec<u8>, value: i16) {
|
||||
data.extend_from_slice(&value.to_le_bytes());
|
||||
}
|
||||
|
||||
fn align_vec(data: &mut Vec<u8>, alignment: usize) {
|
||||
let remainder = data.len() % alignment;
|
||||
if remainder != 0 {
|
||||
data.resize(data.len() + alignment - remainder, 0);
|
||||
}
|
||||
}
|
||||
|
||||
fn push_c_string(data: &mut Vec<u8>, value: &str) {
|
||||
data.extend_from_slice(value.as_bytes());
|
||||
data.push(0);
|
||||
}
|
||||
|
||||
fn synthetic_serialized_text_asset(bytes: &[u8]) -> Vec<u8> {
|
||||
let mut object_data = Vec::new();
|
||||
push_u32_le(&mut object_data, 8);
|
||||
object_data.extend_from_slice(b"Scenario");
|
||||
align_vec(&mut object_data, 4);
|
||||
push_u32_le(&mut object_data, bytes.len() as u32);
|
||||
object_data.extend_from_slice(bytes);
|
||||
|
||||
let mut metadata = Vec::new();
|
||||
metadata.extend_from_slice(b"2021.3.56f2\0");
|
||||
metadata.extend_from_slice(&19i32.to_le_bytes());
|
||||
metadata.push(0);
|
||||
metadata.extend_from_slice(&1i32.to_le_bytes());
|
||||
metadata.extend_from_slice(&49i32.to_le_bytes());
|
||||
metadata.push(0);
|
||||
push_i16_le(&mut metadata, 0);
|
||||
metadata.extend_from_slice(&[0; 16]);
|
||||
metadata.extend_from_slice(&1i32.to_le_bytes());
|
||||
align_vec(&mut metadata, 4);
|
||||
push_u64_le(&mut metadata, 1);
|
||||
push_u64_le(&mut metadata, 0);
|
||||
push_u32_le(&mut metadata, object_data.len() as u32);
|
||||
push_u32_le(&mut metadata, 0);
|
||||
|
||||
let header_len = 48usize;
|
||||
let data_offset = header_len + metadata.len();
|
||||
let file_size = data_offset + object_data.len();
|
||||
let mut file = Vec::new();
|
||||
push_u32_be(&mut file, metadata.len() as u32);
|
||||
push_u32_be(&mut file, file_size as u32);
|
||||
push_u32_be(&mut file, 22);
|
||||
push_u32_be(&mut file, 0);
|
||||
file.extend_from_slice(&[0, 0, 0, 0]);
|
||||
push_u32_be(&mut file, metadata.len() as u32);
|
||||
push_u64_be(&mut file, file_size as u64);
|
||||
push_u64_be(&mut file, data_offset as u64);
|
||||
push_u64_be(&mut file, 0);
|
||||
file.extend_from_slice(&metadata);
|
||||
file.extend_from_slice(&object_data);
|
||||
file
|
||||
}
|
||||
|
||||
fn synthetic_unityfs(payload: &[u8]) -> Vec<u8> {
|
||||
let mut blocks_info = vec![0; 16];
|
||||
blocks_info.extend_from_slice(&1i32.to_be_bytes());
|
||||
push_u32_be(&mut blocks_info, payload.len() as u32);
|
||||
push_u32_be(&mut blocks_info, payload.len() as u32);
|
||||
blocks_info.extend_from_slice(&0u16.to_be_bytes());
|
||||
blocks_info.extend_from_slice(&1i32.to_be_bytes());
|
||||
push_u64_be(&mut blocks_info, 0);
|
||||
push_u64_be(&mut blocks_info, payload.len() as u64);
|
||||
push_u32_be(&mut blocks_info, 0);
|
||||
push_c_string(&mut blocks_info, "CAB-asset");
|
||||
|
||||
let mut file = Vec::new();
|
||||
push_c_string(&mut file, "UnityFS");
|
||||
push_u32_be(&mut file, 8);
|
||||
push_c_string(&mut file, "5.x.x");
|
||||
push_c_string(&mut file, "2021.3.56f2");
|
||||
let total_size_offset = file.len();
|
||||
push_u64_be(&mut file, 0);
|
||||
push_u32_be(&mut file, blocks_info.len() as u32);
|
||||
push_u32_be(&mut file, blocks_info.len() as u32);
|
||||
push_u32_be(&mut file, 0);
|
||||
align_vec(&mut file, 16);
|
||||
file.extend_from_slice(&blocks_info);
|
||||
file.extend_from_slice(payload);
|
||||
let total_size = file.len() as u64;
|
||||
file[total_size_offset..total_size_offset + 8].copy_from_slice(&total_size.to_be_bytes());
|
||||
file
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn localized_manifest_converts_to_generic_patch_manifest() {
|
||||
let source = b"source";
|
||||
@@ -1434,6 +1950,7 @@ mod tests {
|
||||
localized_bytes: target.len() as u64,
|
||||
byte_delta: target.len() as i64 - source.len() as i64,
|
||||
text_asset_operations: vec![LocalizedPatchOperation {
|
||||
archive_entry: None,
|
||||
patch_kind: "unityfs_text_asset".to_string(),
|
||||
serialized_file_path: "CAB-asset".to_string(),
|
||||
path_id: 1,
|
||||
@@ -1489,6 +2006,7 @@ mod tests {
|
||||
};
|
||||
let input = LocalizedPatchInput::TextAsset(LocalizedTextAssetPatch {
|
||||
bundle_path: "Bundles/file.bundle".to_string(),
|
||||
archive_entry: None,
|
||||
text_asset: TextAssetPatch::new("CAB-asset", 1, b"target".to_vec()),
|
||||
metadata: Some(LocalizedPatchOperationMetadata {
|
||||
text_unit_id: "unit-1".to_string(),
|
||||
@@ -1512,6 +2030,69 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
#[test]
|
||||
fn publishes_and_reparses_bundle_nested_in_zip() {
|
||||
let temp = TempDir::new().unwrap();
|
||||
let official = temp.path().join("official-release");
|
||||
let localized = temp.path().join("localized");
|
||||
let zip_source = temp.path().join("zip-source");
|
||||
fs::create_dir_all(zip_source.join("nested")).unwrap();
|
||||
fs::create_dir_all(official.join("Bundles")).unwrap();
|
||||
fs::write(
|
||||
zip_source.join("nested/CAB-asset"),
|
||||
synthetic_unityfs(&synthetic_serialized_text_asset(b"old")),
|
||||
)
|
||||
.unwrap();
|
||||
let archive_path = official.join("Bundles/catalog.zip");
|
||||
let zip_status = std::process::Command::new("zip")
|
||||
.current_dir(&zip_source)
|
||||
.arg("-q")
|
||||
.arg("-r")
|
||||
.arg(&archive_path)
|
||||
.arg(".")
|
||||
.status()
|
||||
.unwrap();
|
||||
assert!(zip_status.success());
|
||||
|
||||
let operation = LocalizedPatchInput::TextAsset(LocalizedTextAssetPatch {
|
||||
bundle_path: "Bundles/catalog.zip".to_string(),
|
||||
archive_entry: Some("nested/CAB-asset".to_string()),
|
||||
text_asset: TextAssetPatch::new("CAB-asset", 1, b"translated".to_vec()),
|
||||
metadata: None,
|
||||
});
|
||||
let report = LocalizedPatchService::new()
|
||||
.publish(
|
||||
&LocalizedPatchConfig::new(&official, &localized, "release-1", Vec::new())
|
||||
.with_archive_commands("unzip", "zip")
|
||||
.with_operations(vec![operation]),
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let localized_archive = report.version_path.join("Bundles/catalog.zip");
|
||||
let output = std::process::Command::new("unzip")
|
||||
.arg("-p")
|
||||
.arg(&localized_archive)
|
||||
.arg("nested/CAB-asset")
|
||||
.output()
|
||||
.unwrap();
|
||||
assert!(output.status.success());
|
||||
let parsed = bat_assetbundle::UnityFsParser::new()
|
||||
.parse(&output.stdout)
|
||||
.unwrap();
|
||||
assert_eq!(parsed.text_assets[0].bytes, b"translated");
|
||||
assert_eq!(
|
||||
report.manifest.files[0].text_asset_operations[0]
|
||||
.archive_entry
|
||||
.as_deref(),
|
||||
Some("nested/CAB-asset")
|
||||
);
|
||||
assert_eq!(
|
||||
fs::read_link(report.current_path).unwrap(),
|
||||
PathBuf::from("versions/release-1")
|
||||
);
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
#[test]
|
||||
fn publishes_a_separate_localized_release_atomically() {
|
||||
@@ -1709,6 +2290,7 @@ mod tests {
|
||||
"release-1",
|
||||
vec![LocalizedTextAssetPatch {
|
||||
bundle_path: "Bundles/bad.bundle".to_string(),
|
||||
archive_entry: None,
|
||||
text_asset: TextAssetPatch::new("CAB-bad", 1, b"replacement".to_vec()),
|
||||
metadata: None,
|
||||
}],
|
||||
@@ -1726,4 +2308,49 @@ mod tests {
|
||||
.exists());
|
||||
assert!(!localized.join(LOCALIZED_CURRENT_LINK).exists());
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
#[test]
|
||||
fn failed_publish_rollback_restores_previous_state_file() {
|
||||
use std::os::unix::fs::symlink;
|
||||
|
||||
let temp = TempDir::new().unwrap();
|
||||
let localized = temp.path().join("localized");
|
||||
let versions = localized.join(LOCALIZED_VERSIONS_DIR);
|
||||
let staging = localized.join(LOCALIZED_STAGING_DIR).join("release-new");
|
||||
let previous_version = versions.join("release-old");
|
||||
let failed_version = versions.join("release-new");
|
||||
let current = localized.join(LOCALIZED_CURRENT_LINK);
|
||||
let state_path = localized.join(LOCALIZED_VERSION_STATE_FILE);
|
||||
let previous_state = br#"{"state_version":1,"status":"localized"}"#;
|
||||
|
||||
fs::create_dir_all(&previous_version).unwrap();
|
||||
fs::create_dir_all(&failed_version).unwrap();
|
||||
fs::create_dir_all(&staging).unwrap();
|
||||
fs::write(&state_path, previous_state).unwrap();
|
||||
symlink(
|
||||
Path::new(LOCALIZED_VERSIONS_DIR).join("release-new"),
|
||||
¤t,
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
rollback_failed_publish(
|
||||
&localized,
|
||||
&staging,
|
||||
&failed_version,
|
||||
true,
|
||||
¤t,
|
||||
Some(&PathBuf::from("versions/release-old")),
|
||||
(&state_path, Some(previous_state)),
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
assert!(!failed_version.exists());
|
||||
assert!(!staging.exists());
|
||||
assert_eq!(
|
||||
fs::read_link(¤t).unwrap(),
|
||||
PathBuf::from("versions/release-old")
|
||||
);
|
||||
assert_eq!(fs::read(&state_path).unwrap(), previous_state);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -126,6 +126,8 @@ pub struct OfficialUpdateConfig {
|
||||
pub download_concurrency: usize,
|
||||
/// Unzip command used when a metadata change requires GameMainConfig parsing.
|
||||
pub unzip_command: PathBuf,
|
||||
/// Zip command used when publishing localized bundles nested in ZIP archives.
|
||||
pub zip_command: PathBuf,
|
||||
/// Dry run reports decisions and optional plan URLs without writing sync state.
|
||||
pub dry_run: bool,
|
||||
/// Include full download URLs when dry-running.
|
||||
@@ -162,6 +164,7 @@ impl Default for OfficialUpdateConfig {
|
||||
curl_proxy: CurlProxyConfig::default(),
|
||||
download_concurrency: DEFAULT_DOWNLOAD_CONCURRENCY,
|
||||
unzip_command: PathBuf::from("unzip"),
|
||||
zip_command: PathBuf::from("zip"),
|
||||
dry_run: false,
|
||||
plan: false,
|
||||
force: false,
|
||||
|
||||
@@ -475,11 +475,10 @@ pub fn validate_translation_workbench_with_glossary_path(
|
||||
}
|
||||
changed_entries += 1;
|
||||
let source_kind = normalized_text_source_kind(entry.text_source_kind.as_deref());
|
||||
let is_publishable = entry.archive_entry.is_none()
|
||||
&& matches!(
|
||||
source_kind.as_deref(),
|
||||
Some("textasset" | "typetreefield" | "managedreferencefield")
|
||||
);
|
||||
let is_publishable = matches!(
|
||||
source_kind.as_deref(),
|
||||
Some("textasset" | "typetreefield" | "managedreferencefield")
|
||||
);
|
||||
if is_publishable {
|
||||
let serialized_file = entry
|
||||
.serialized_file
|
||||
@@ -579,10 +578,7 @@ fn validate_current_glossary_qa(
|
||||
}
|
||||
|
||||
/// Converts reviewed entries to localized patch operations supported by the
|
||||
/// current UnityFS write layer.
|
||||
///
|
||||
/// ZIP-inner bundles are intentionally rejected here because they require a
|
||||
/// separate archive rewrite boundary.
|
||||
/// current UnityFS write layer and its ZIP rewrite boundary.
|
||||
pub fn localized_patch_operations(
|
||||
resource_root: &Path,
|
||||
workbench: &TranslationWorkbench,
|
||||
@@ -654,16 +650,11 @@ pub fn localized_patch_operations_with_glossary_path(
|
||||
entry.id
|
||||
));
|
||||
};
|
||||
if entry.archive_entry.is_some() {
|
||||
return Err(anyhow::anyhow!(
|
||||
"TextUnit {} 位于 zip archive entry，当前 publish-localized 不支持直接修改 zip 内 bundle",
|
||||
entry.id
|
||||
));
|
||||
}
|
||||
let source_kind = normalized_text_source_kind(entry.text_source_kind.as_deref());
|
||||
let field_path = entry.field_path.clone();
|
||||
if !seen.insert((
|
||||
entry.destination.clone(),
|
||||
entry.archive_entry.clone(),
|
||||
serialized_file.clone(),
|
||||
path_id,
|
||||
field_path.clone(),
|
||||
@@ -688,6 +679,7 @@ pub fn localized_patch_operations_with_glossary_path(
|
||||
patch.expected_name = entry.asset_name.clone();
|
||||
operations.push(LocalizedPatchInput::TextAsset(LocalizedTextAssetPatch {
|
||||
bundle_path: entry.destination.clone(),
|
||||
archive_entry: entry.archive_entry.clone(),
|
||||
text_asset: patch,
|
||||
metadata,
|
||||
}));
|
||||
@@ -702,6 +694,7 @@ pub fn localized_patch_operations_with_glossary_path(
|
||||
operations.push(LocalizedPatchInput::StringField(
|
||||
LocalizedStringFieldPatch {
|
||||
bundle_path: entry.destination.clone(),
|
||||
archive_entry: entry.archive_entry.clone(),
|
||||
string_field: StringFieldPatch {
|
||||
serialized_file_path: serialized_file,
|
||||
path_id,
|
||||
@@ -771,12 +764,6 @@ pub fn localized_text_asset_patches(
|
||||
entry.id
|
||||
));
|
||||
};
|
||||
if entry.archive_entry.is_some() {
|
||||
return Err(anyhow::anyhow!(
|
||||
"TextUnit {} 位于 zip archive entry，当前 publish-localized 不支持直接修改 zip 内 bundle",
|
||||
entry.id
|
||||
));
|
||||
}
|
||||
if normalized_text_source_kind(entry.text_source_kind.as_deref()).as_deref()
|
||||
!= Some("textasset")
|
||||
{
|
||||
@@ -785,7 +772,12 @@ pub fn localized_text_asset_patches(
|
||||
entry.id
|
||||
));
|
||||
}
|
||||
if !seen.insert((entry.destination.clone(), serialized_file.clone(), path_id)) {
|
||||
if !seen.insert((
|
||||
entry.destination.clone(),
|
||||
entry.archive_entry.clone(),
|
||||
serialized_file.clone(),
|
||||
path_id,
|
||||
)) {
|
||||
return Err(anyhow::anyhow!(
|
||||
"翻译工作台包含重复 patch 目标：{}",
|
||||
entry.id
|
||||
@@ -811,6 +803,7 @@ pub fn localized_text_asset_patches(
|
||||
}
|
||||
patches.push(LocalizedTextAssetPatch {
|
||||
bundle_path: entry.destination.clone(),
|
||||
archive_entry: entry.archive_entry.clone(),
|
||||
text_asset: patch,
|
||||
metadata: Some(localized_patch_metadata(
|
||||
entry,
|
||||
|
||||
Reference in New Issue
Block a user