feat(addressables): 补全 catalog 可校验字段
bat-rust / Build and test Rust (push) Canceled after 0s
bat-rust / Build and test Go API (push) Canceled after 0s

补充 JSON/compact catalog 的 provider ID、bundle name、资源类型、hash、size、CRC 和依赖字段,并贯通 ResourceEntry、SQLite 资源索引和回归 golden。对 compact extra data 与 resource type index 的损坏返回明确错误,不再把 bundle name 当作 hash fallback。\n\n验证:cargo fmt --check;cargo test -p bat-core --locked;cargo test -p bat-adapters --locked;cargo test -p bat-infrastructure --locked;git diff --check。\n\nClippy 仍受既有 core/src/domain/game_client.rs:141 的 needless-question-mark 和 items-after-test-module 基线问题影响,未混入本 issue 修复。\n\nFixes #2
This commit is contained in:
2026-08-19 23:10:16 +08:00
parent 72c1a879a3
commit 9d4f8d903c
18 changed files with 311 additions and 55 deletions
+2 -2
View File
@@ -97,7 +97,7 @@ job/unit/provider run 状态。当前下载实现使用默认 8 个独立 worker
- Unity adapter trait、注册表、Unity 2021.3 adapter 基础解析与校验。
- Manifest driver trait、Addressables driver、注册表。
- Addressables JSON catalog 的 path、hash、size、address、dependencies、metadata 解析。
- Addressables JSON/compact catalog 的 path、hash、size、address、dependencies、provider ID、bundle name、CRC、metadata 解析。
- 真实形态 Addressables fixture/golden 测试。
- 当前 catalog、上一个版本 catalog、结构变化 catalog 的离线回归 fixture。
- 官方日服 `server-info`、URL 规则、平台 discovery 和 inventory 枚举;`MediaCatalog.bytes` 使用官方相对路径生成媒体 URL,覆盖 `GameData/``Prologue/` 下的 zip/mp4/png/jpg/ogg/wav 等媒体资源,避免把叶子文件名误拼到媒体根目录。
@@ -295,7 +295,7 @@ GitHub issue 状态:#1 已关闭;#17 的历史决定不代表当前下载实
2. Go 侧:进度见 `docs/reports/GO_STATUS.md`。G-008 已关闭;`bat-api` 资源 bootstrap/分发 MVP 已落地,已含 `/v1/bootstrap``/v1/launcher/bootstrap`、launcher 资源 metadata 兼容、HTTP 鉴权/限流/日志/反代适配、动态 JSON no-store、OpenAPI、管理控制白名单、CDN Range/缓存头、RPC 周期刷新和 USERGUIDE 基础章节;仓库内 Rust/Go snapshot contract fixture 已落地,剩余为远程服务器全量 release 联调、refresh mtime/size 增量缓存和可选持久化。
3. 文本提取 / 翻译队列 / Patch 输入:`official-textunit-index.json``official-textunit-tasks.json``crowdin-textunit-queue.json` 已生成;TextUnit 明细、解析错误和离线翻译任务状态/失败原因已可查询,`translation.task.update` 已提供 Rust CLI、Go typed helper 和 bat-api dashboard 状态回写入口,`translation.proofread` 已提供人工校对状态标记入口。剩余为真实 Crowdin/provider worker、翻译记忆和 Patch 构建。
4. Issue #3P1):AssetBundle UnityFS 基础解析校验已具备离线和隔离真实样本覆盖;对象级解析继续跟踪 G-005。
5. Issue #2P1):继续逆向 Addressables catalog提取 bundle hash/size/CRC 等可校验字段
5. Issue #2已完成当前目标):Addressables JSON/compact catalog提取 bundle provider、name、hash/size/CRC、资源类型和依赖关系,并贯通 ResourceEntry、SQLite 与 resource.index 输出;独立二进制格式仍明确拒绝
6. 通用 Binary/JSON/Text Patch 基础已落地;复杂 AssetBundle 重打包和真实翻译系统仍应后置,UnityFS TextAsset patch 发布前置已具备回归测试。
非阻塞跟踪项:官方同步长期运行测试正在进行,运行报告将在后续提供。
+1 -1
View File
@@ -173,7 +173,7 @@ BlueArchiveToolkit 不是一次性脚本,也不是演示项目。最终交付
交付物:
1. Addressables Catalog 真实字段解析:**部分完成**。当前已覆盖 path、hash、size、address、dependencies、metadata 和真实形态 fixture/golden;仍需继续覆盖更多官方 catalog 结构变体
1. Addressables Catalog 目标字段解析:**当前目标完成**。JSON/compact 已覆盖 path、hash、size、address、dependencies、provider ID、bundle name、CRC、metadata,并通过 fixture/golden 与 SQLite 迁移回归;独立二进制 catalog 仍明确拒绝
2. 资源版本、区域、渠道、远端 URL、Hash、大小、依赖关系模型:**部分完成**。`Resource` 和官方 endpoint/snapshot 模型已扩展;仍需冻结 Go CLI/API 可见模型。
3. Rust 官方下载器:**已完成当前生产入口需要的核心能力**。包含官方 URL 校验、`.part` 续传、重试、本地 manifest size+BLAKE3 校验、官方 seed `.hash` 校验、repair,以及默认 8、范围 `1..=256` 的有界并发 scheduler;worker 动态领取任务,进度按完成数单调上报,report 保持 plan 顺序。
4. Rust 自动更新入口:**已完成当前生产入口**。`bat` 支持 snapshot、marker diff、bootstrap cache、one-shot、`--watch``--daemon`、默认 1 小时间隔、北京时间固定强制刷新,以及 Unix socket JSON-RPC 后台运维命令返回。
+166 -37
View File
@@ -256,6 +256,22 @@ impl AddressablesCatalogDriver {
}
let address = Self::string_field(value, &["address", "Address", "m_Address", "key", "Key"]);
let provider_id = Self::catalog_string_field(
value,
&[
"provider_id",
"providerId",
"ProviderId",
"provider",
"Provider",
"m_ProviderId",
"m_Provider",
],
);
let bundle_name = Self::catalog_string_field(
value,
&["bundle_name", "bundleName", "BundleName", "m_BundleName"],
);
let hash = value
.get("hash")
.or_else(|| value.get("Hash"))
@@ -264,28 +280,32 @@ impl AddressablesCatalogDriver {
.map(ToOwned::to_owned)
.unwrap_or_else(|| format!("addressable_{}", index));
let size = value
.get("size")
.or_else(|| value.get("Size"))
.or_else(|| value.get("m_Size"))
.and_then(|value| value.as_u64())
.unwrap_or_default();
let size = Self::u64_field(value, &["size", "Size", "m_Size"]).unwrap_or_default();
let dependencies = Self::dependencies_from_entry(value);
let crc = value
.get("crc")
.or_else(|| value.get("Crc"))
.or_else(|| value.get("m_Crc"))
.and_then(|value| value.as_u64())
.and_then(|value| u32::try_from(value).ok());
let crc = Self::u32_field(value, &["crc", "Crc", "m_Crc"]);
let resource_type_name = Self::type_name_field(
value,
&[
"resource_type",
"resourceType",
"ResourceType",
"m_ResourceType",
],
);
Some(ResourceEntry {
path: path.to_string(),
hash,
size,
resource_type: Self::resource_type_for_path(path),
resource_type: Self::resource_type_for_compact_entry(
path,
resource_type_name.as_deref(),
),
address,
dependencies,
provider_id,
bundle_name,
crc,
})
}
@@ -299,6 +319,42 @@ impl AddressablesCatalogDriver {
})
}
fn catalog_string_field(value: &Value, fields: &[&str]) -> Option<String> {
Self::string_field(value, fields).or_else(|| {
["extra_data", "ExtraData", "m_ExtraData", "data", "Data"]
.iter()
.find_map(|field| value.get(*field))
.and_then(|extra| Self::string_field(extra, fields))
})
}
fn type_name_field(value: &Value, fields: &[&str]) -> Option<String> {
fields.iter().find_map(|field| {
let value = value.get(*field)?;
value.as_str().map(ToOwned::to_owned).or_else(|| {
value
.get("m_ClassName")
.or_else(|| value.get("ClassName"))
.or_else(|| value.get("class_name"))
.and_then(|name| name.as_str())
.map(ToOwned::to_owned)
})
})
}
fn u64_field(value: &Value, fields: &[&str]) -> Option<u64> {
fields.iter().find_map(|field| {
let value = value.get(*field)?;
value
.as_u64()
.or_else(|| value.as_str()?.parse::<u64>().ok())
})
}
fn u32_field(value: &Value, fields: &[&str]) -> Option<u32> {
Self::u64_field(value, fields).and_then(|value| u32::try_from(value).ok())
}
fn string_array_field(value: &Value, fields: &[&str]) -> Vec<String> {
for field in fields {
if let Some(array) = value.get(field).and_then(|value| value.as_array()) {
@@ -383,6 +439,8 @@ impl AddressablesCatalogDriver {
resource_type: Self::resource_type_for_path(path),
address: None,
dependencies: Vec::new(),
provider_id: None,
bundle_name: None,
crc: None,
})
})
@@ -421,6 +479,8 @@ impl AddressablesCatalogDriver {
resource_type,
address: None,
dependencies: Vec::new(),
provider_id: None,
bundle_name: None,
crc: None,
});
}
@@ -479,8 +539,9 @@ impl AddressablesCatalogDriver {
internal_ids.len()
)
})?;
provider_ids
let provider_id = provider_ids
.get(record.provider_index as usize)
.cloned()
.ok_or_else(|| {
format!(
"compact entry {index} provider_index {} out of range {}",
@@ -499,13 +560,12 @@ impl AddressablesCatalogDriver {
)
})?;
let path = Self::normalize_internal_id(&internal_id_prefixes, internal_id);
let extra = Self::extra_data_at(&extra_data, record.data_index);
let resource_type_name = Self::resource_type_name(json, record.resource_type_index);
let extra = Self::extra_data_at(&extra_data, record.data_index)?;
let resource_type_name = Self::resource_type_name(json, record.resource_type_index)?;
let dependencies = Self::compact_dependencies(record, &entry_records, &buckets, &keys);
let hash = extra
.hash
.filter(|value| !value.is_empty())
.or_else(|| extra.bundle_name.filter(|value| !value.is_empty()))
.unwrap_or_else(|| format!("addressable_{}", index));
resources.push(ResourceEntry {
@@ -522,6 +582,8 @@ impl AddressablesCatalogDriver {
Some(primary_key)
},
dependencies,
provider_id: Some(provider_id),
bundle_name: extra.bundle_name,
crc: extra.crc,
});
}
@@ -632,25 +694,34 @@ impl AddressablesCatalogDriver {
.collect()
}
fn extra_data_at(extra_data: &[u8], data_index: i32) -> AddressablesExtraData {
fn extra_data_at(extra_data: &[u8], data_index: i32) -> Result<AddressablesExtraData, String> {
if data_index < 0 {
return AddressablesExtraData::default();
return Ok(AddressablesExtraData::default());
}
let Some((object, _)) = Self::read_serialized_object(extra_data, data_index as usize)
else {
return AddressablesExtraData::default();
return Err(format!(
"compact entry extra data index {} is not decodable",
data_index
));
};
let AddressablesObject::JsonObject { json, .. } = object else {
return AddressablesExtraData::default();
return Err(format!(
"compact entry extra data index {} is not a JSON object",
data_index
));
};
let Some(json) = json else {
return AddressablesExtraData::default();
return Err(format!(
"compact entry extra data index {} contains invalid JSON",
data_index
));
};
AddressablesExtraData {
Ok(AddressablesExtraData {
hash: json
.get("m_Hash")
.and_then(|value| value.as_str())
@@ -659,26 +730,31 @@ impl AddressablesCatalogDriver {
.get("m_BundleName")
.and_then(|value| value.as_str())
.map(ToOwned::to_owned),
bundle_size: json.get("m_BundleSize").and_then(|value| value.as_u64()),
bundle_size: Self::u64_field(&json, &["m_BundleSize", "bundle_size", "size"]),
// m_Crc 是 bundle 的 IEEE CRC-320 表示不做 CRC 校验,忠实保留原值。
crc: json
.get("m_Crc")
.and_then(|value| value.as_u64())
.and_then(|value| u32::try_from(value).ok()),
}
crc: Self::u32_field(&json, &["m_Crc", "crc", "Crc"]),
})
}
fn resource_type_name(json: &Value, index: i32) -> Option<String> {
fn resource_type_name(json: &Value, index: i32) -> Result<Option<String>, String> {
if index < 0 {
return None;
return Ok(None);
}
json.get("m_resourceTypes")?
.as_array()?
.get(index as usize)?
.get("m_ClassName")?
.as_str()
.map(ToOwned::to_owned)
let resource_types = json
.get("m_resourceTypes")
.and_then(|value| value.as_array())
.ok_or_else(|| "compact catalog missing m_resourceTypes array".to_string())?;
let value = resource_types.get(index as usize).ok_or_else(|| {
format!(
"compact resource_type_index {} out of range {}",
index,
resource_types.len()
)
})?;
Self::type_name_field(value, &["m_ClassName", "ClassName", "class_name"])
.ok_or_else(|| format!("compact resource type {} has no class name", index))
.map(Some)
}
fn normalize_internal_id(prefixes: &[String], internal_id: &str) -> String {
@@ -869,6 +945,25 @@ impl AddressablesCatalogDriver {
fallback_hash_count.to_string(),
);
}
let provider_count = resources
.iter()
.filter(|entry| entry.provider_id.is_some())
.count();
if provider_count > 0 {
extra.insert("provider_id_count".to_string(), provider_count.to_string());
}
let bundle_name_count = resources
.iter()
.filter(|entry| entry.bundle_name.is_some())
.count();
if bundle_name_count > 0 {
extra.insert(
"bundle_name_count".to_string(),
bundle_name_count.to_string(),
);
}
}
}
@@ -1146,6 +1241,8 @@ mod tests {
"hash": "synthetic-entry-hash",
"size": 119,
"crc": 3735928559,
"provider_id": "synthetic-provider",
"bundle_name": "synthetic-bundle",
"address": "Character_001",
"dependencies": ["synthetic/shared.bundle"]
},
@@ -1163,6 +1260,14 @@ mod tests {
assert_eq!(manifest.resources[0].path, "synthetic/minimal.bundle");
assert_eq!(manifest.resources[0].hash, "synthetic-entry-hash");
assert_eq!(manifest.resources[0].size, 119);
assert_eq!(
manifest.resources[0].provider_id.as_deref(),
Some("synthetic-provider")
);
assert_eq!(
manifest.resources[0].bundle_name.as_deref(),
Some("synthetic-bundle")
);
// m_Crc(此处 0xDEADBEEF)应被提取;缺该字段的条目为 None。
assert_eq!(manifest.resources[0].crc, Some(0xDEAD_BEEF));
assert_eq!(manifest.resources[1].crc, None);
@@ -1207,6 +1312,14 @@ mod tests {
manifest.metadata.extra.get("dependency_count"),
Some(&"1".to_string())
);
assert_eq!(
manifest.metadata.extra.get("provider_id_count"),
Some(&"1".to_string())
);
assert_eq!(
manifest.metadata.extra.get("bundle_name_count"),
Some(&"1".to_string())
);
}
#[tokio::test]
@@ -1243,6 +1356,14 @@ mod tests {
assert_eq!(resource.hash, "hash-compact");
assert_eq!(resource.size, 42);
assert_eq!(resource.crc, Some(0x1234_5678));
assert_eq!(
resource.provider_id.as_deref(),
Some("UnityEngine.ResourceManagement.ResourceProviders.AssetBundleProvider")
);
assert_eq!(
resource.bundle_name.as_deref(),
Some("synthetic-bundle-name")
);
assert_eq!(resource.resource_type, ResourceType::AssetBundle);
assert_eq!(resource.address.as_deref(), Some("synthetic.bundle"));
assert_eq!(
@@ -1261,6 +1382,14 @@ mod tests {
manifest.metadata.extra.get("declared_crc_count"),
Some(&"1".to_string())
);
assert_eq!(
manifest.metadata.extra.get("provider_id_count"),
Some(&"1".to_string())
);
assert_eq!(
manifest.metadata.extra.get("bundle_name_count"),
Some(&"1".to_string())
);
}
#[tokio::test]
+25
View File
@@ -21,6 +21,19 @@ async fn parses_current_catalog_fixture_with_resource_categories() {
manifest.resources[3].dependencies,
vec!["shared_dependencies.bundle".to_string()]
);
assert_eq!(
manifest.resources[0].provider_id.as_deref(),
Some("provider-table")
);
assert_eq!(
manifest.resources[0].bundle_name.as_deref(),
Some("table-bundle")
);
assert_eq!(manifest.resources[0].crc, Some(0x1234_5678));
assert_eq!(
manifest.resources[3].provider_id.as_deref(),
Some("provider-bundle")
);
assert_eq!(
manifest.metadata.cdn_prefixes,
vec!["https://fixture.invalid/current/".to_string()]
@@ -63,6 +76,18 @@ async fn parses_catalog_structure_change_with_alias_fields() {
manifest.resources[0].dependencies,
vec!["shared_assets_current.bundle".to_string()]
);
assert_eq!(
manifest.resources[0].provider_id.as_deref(),
Some("provider-android")
);
assert_eq!(
manifest.resources[0].bundle_name.as_deref(),
Some("title-android-bundle")
);
assert_eq!(manifest.resources[1].resource_type, ResourceType::TextAsset);
assert_eq!(manifest.resources[1].address.as_deref(), Some("lesson"));
assert_eq!(
manifest.resources[1].provider_id.as_deref(),
Some("provider-text")
);
}
@@ -8,6 +8,9 @@
"internal_id": "TableBundles/ExcelDB.db",
"hash": "current-table-hash",
"size": 4096,
"provider_id": "provider-table",
"bundle_name": "table-bundle",
"crc": "305419896",
"address": "ExcelDB",
"dependencies": []
},
@@ -15,6 +18,8 @@
"internal_id": "MediaResources-Windows/voice/title.acb",
"hash": "current-media-hash",
"size": 2048,
"m_ProviderId": "provider-media",
"m_BundleName": "media-bundle",
"address": "title",
"dependencies": []
},
@@ -22,6 +27,8 @@
"internal_id": "TextAssets/dialogue.csv",
"hash": "current-text-hash",
"size": 128,
"Provider": "provider-text",
"BundleName": "text-bundle",
"address": "dialogue",
"dependencies": []
},
@@ -29,6 +36,8 @@
"internal_id": "shared_assets_current.bundle",
"hash": "current-bundle-hash",
"size": 8192,
"provider": "provider-bundle",
"bundleName": "shared-bundle",
"address": "shared_assets_current",
"dependencies": [
"shared_dependencies.bundle"
@@ -9,6 +9,8 @@
"InternalId": "MediaResources-Android/voice/title.awb",
"Hash": "changed-media-hash",
"Size": 65536,
"ProviderId": "provider-android",
"BundleName": "title-android-bundle",
"Address": "title-android",
"m_Dependencies": [
"shared_assets_current.bundle"
@@ -18,6 +20,8 @@
"Path": "TextAssets/lesson.json",
"Hash": "changed-text-hash",
"Size": 512,
"provider_id": "provider-text",
"bundle_name": "lesson-bundle",
"Key": "lesson"
}
]
@@ -13,6 +13,8 @@
"dependencies": [
"shared_assets_all_123.bundle"
],
"provider_id": "UnityEngine.ResourceManagement.ResourceProviders.AssetBundleProvider",
"bundle_name": "bundle-main",
"crc": 0
},
{
@@ -22,6 +24,8 @@
"resource_type": "AssetBundle",
"address": "academy-_mxload-prefabs-2025-08-26_assets_all_1581352935.bundle",
"dependencies": [],
"provider_id": "UnityEngine.ResourceManagement.ResourceProviders.AssetBundleProvider",
"bundle_name": "bundle-second",
"crc": 0
},
{
@@ -31,16 +35,20 @@
"resource_type": "AssetBundle",
"address": "shared_assets_all_123.bundle",
"dependencies": [],
"provider_id": "UnityEngine.ResourceManagement.ResourceProviders.AssetBundleProvider",
"bundle_name": "bundle-shared",
"crc": 0
}
],
"metadata": {
"asset_bundle_count": "3",
"declared_size_count": "3",
"bundle_name_count": "3",
"dependency_count": "1",
"internal_id_count": "3",
"resource_count": "3",
"resource_type_count": "1",
"provider_id_count": "3",
"key_object_count": "4",
"bucket_record_count": "4",
"entry_record_count": "3",
@@ -22,6 +22,8 @@ async fn parses_real_shape_addressables_catalog_against_golden() {
"resource_type": format!("{:?}", resource.resource_type),
"address": resource.address,
"dependencies": resource.dependencies,
"provider_id": resource.provider_id,
"bundle_name": resource.bundle_name,
"crc": resource.crc,
})
}).collect::<Vec<_>>(),
+12
View File
@@ -34,6 +34,16 @@ pub struct ResourceEntry {
pub address: Option<String>,
/// 该资源依赖的其他资源标识
pub dependencies: Vec<String>,
/// Addressables provider ID。
///
/// 旧的 manifest 和资源索引没有该字段,缺省时保持 `None`。
#[serde(default, skip_serializing_if = "Option::is_none")]
pub provider_id: Option<String>,
/// Addressables bundle name。
///
/// 该值是定位/诊断字段,不作为资源 hash 的替代值。
#[serde(default, skip_serializing_if = "Option::is_none")]
pub bundle_name: Option<String>,
/// Addressables bundle 的 CRC32catalog 中的 `m_Crc`)。
///
/// `None` 表示 catalog 未提供该字段;Unity 用 `0` 表示「不做 CRC 校验」,
@@ -202,6 +212,8 @@ mod tests {
resource_type: ResourceType::AssetBundle,
address: None,
dependencies: Vec::new(),
provider_id: None,
bundle_name: None,
crc,
}
}
+8 -3
View File
@@ -1,6 +1,6 @@
# AssetBundle 与资源解析路线图
- **更新时间**2026-07-26
- **更新时间**2026-08-19
- **适用范围**:Rust 解析引擎、官方同步后的解析缓存、CAS/ResourceRepository 接入、后续文本提取和 Patch 发布。
- **权威关联**`PROJECT_PLAN.md` Milestone 3/4/5/8`docs/reports/CURRENT_GAPS.md` G-005/G-007/G-011/G-011D。
- **维护冻结**:解析扩展遵循 [`docs/reports/PARSER_FREEZE.md`](../reports/PARSER_FREEZE.md),冻结期只接受稳定性、诊断、真实回归和文档一致性修复。
@@ -28,7 +28,7 @@
| 层级 | 输入 | 输出 | 当前状态 |
| --- | --- | --- | --- |
| 官方 seed manifest | `TableCatalog.bytes``BundlePackingInfo.bytes``MediaCatalog.bytes` | 完整下载 URL、相对路径、hash 校验边界 | 已用于下载计划,仍需沉淀更多结构化字段 |
| Addressables catalog | `catalog_*.zip` 内 JSON/bin catalog、`catalog_*.hash` | asset path、provider、dependencies、size、CRC、bundle name | JSON/compact 当前样本已覆盖,仍需更多真实结构变体 |
| Addressables catalog | `catalog_*.zip` 内 JSON/bin catalog、`catalog_*.hash` | asset path、provider、dependencies、size、CRC、bundle name | JSON/compact 当前目标字段已覆盖;未知结构返回明确错误 |
| UnityFS container | `.bundle`、zip 内 bundle | header、block、directory、解压文件、基础摘要 | 已支持基础解包、LZ4/LZMA、边界校验 |
| Serialized file | UnityFS directory 文件 | header、type table、TypeTree node、object table、TextAsset bytes | 已支持基础表结构和 TextAsset bytes |
| Unity 对象字段 | TextAsset、MonoBehaviour、ScriptableObject | 可翻译文本单元、上下文、资源定位 | TypeTree 基础字段读取、`SerializedReference` / prefixed managed-reference metadata alias、payload 提取和字符串提取已落地,真实结构覆盖继续扩大 |
@@ -54,7 +54,7 @@
当前还不能宣称完整:
1. TypeTree-covered managed reference 字段和 registry 记录已可结构化解码并参与文本提取,常见 registry 命名别名(含 `m_ManagedReferences``RefIds``m_RefIds`、verbose type 字段)、metadata 命名别名(含 `id``typeInfo`)、payload 命名别名(含 `data``value``payload``object``managedReferencePayload``referencePayload``serializedReferencePayload``managedReferenceValue``referenceValue``serializedReferenceValue``managedReferenceObject``referenceObject``serializedReferenceObject``managedReferenceData``referenceData``serializedData`)、full typename 拆解和 payload-only TextUnit 提取已有回归覆盖,多记录 registry 聚合也已有单元回归;fallback 字段遍历会跳过常见 registry 元数据字符串,避免误入翻译队列,并按记录前缀或子字段可推导 metadata 保留 managed-reference TextUnit context。enum `value__` backing field 和 `LayerMask` / `BitField``m_Bits` backing field 已可语义化解码和替换;`Vector2f/3f/4f``Quaternionf``ColorRGBA``Rectf``AABB/Bounds/Ray``Matrix4x4f``Vector2Int/Vector3Int``RectInt``BoundsInt``RangeInt``GUID``Hash128` 等固定 Unity 值类型的 leaf 和 direct child TypeTree 形态已可结构化解码和语义替换;array/vector/staticvector/List/HashSet/map 元素与 registry payload 字段已保留独立 field path、offset 和 byte size,可用于字符串元素 patchmanaged-reference registry payload 字符串、enum、bit_field、unknown fixed-size raw bytes、object 字段组合和 TypeTree schema 支撑的 array/vector/List/HashSet/map 已可整体变长替换,`first/second``key/value` map entry schema 已有 serialized 和 UnityFS 重建回归,ScriptableObject `key/value` map 解析、变长替换和 UnityFS 重建已有专门回归,且嵌套 vector `Array``List<T>` / `HashSet<T>` 集合 alias、enum、bit_field、unknown fixed-size raw bytes 与 managed-reference payload 字段已有重建回归覆盖;解析模块当前处于维护冻结,未见样本驱动的完整 managed reference registry / map entry 变体、unknown 字段结构语义和版本差异冻结后再推进。
2. Addressables bin/compact catalog 结构变体仍需真实样本驱动补齐
2. Addressables 当前目标 JSON/compact 字段链已补齐;未识别的独立二进制格式仍返回明确错误,不静默降级
3. 官方 release 已可配置导入 CAS + ResourceRepository,并可通过 `resource.index` 查询现有资源索引;Resource metadata 已记录 release、平台、bundle path、parse status、TextAsset 名称和 TextUnit 摘要。
4. 不能完成复杂对象字段重打包,也不能从真实 Crowdin 结果自动生成完整汉化文件集合。
@@ -98,6 +98,11 @@
4. 将 Windows/Android catalog 样本拆成可复现 fixture,不把大文件纳入 Git。
5. 对未知结构返回明确错误或保真 raw metadata,不静默丢字段。
本次 issue #2 交付已完成上述 JSON/compact 字段链:provider ID、bundle name、
primary/dependency key、resource type、hash、size 和 CRC 会进入 `ResourceEntry`
并通过 SQLite `ResourceRepository` 持久化;旧索引会按列迁移继续可读。独立二进制
catalog 仍按“明确不支持”处理,不把低保真路径伪装成完整解析。
验收:
1. 当前目标版本 Windows/Android catalog 样本集合解析通过。
+1
View File
@@ -271,6 +271,7 @@ worker 状态、parse status、TextUnit format、队列 reason 和 provider fail
`translation-handoff.json` 与 SQLite 状态,返回 job、unit、provider run 的完整交接
视图;
`resource-index` 返回的资源 JSON 包含 release、平台、bundle path、TextAsset 和 TextUnit metadata
以及 Addressables entry 的 provider ID、bundle name、hash、size、CRC 和依赖关系;
并可按 release、平台、destination、bundle path、archive entry、parse status 和 TextUnit format 做资源级过滤;
`localized-status` 只有在 `localized-version-state.json``current` symlink 和
`localized-patch-manifest.json` 都匹配当前官方 release 时才返回 `localized`
+3 -2
View File
@@ -123,8 +123,9 @@ SQLite `ResourceRepository`,索引不存在时返回 `ok=true` 且
`entry` 外会包含 `metadata``official_release_id``platform`
`bundle_path``archive_entries``parse_statuses``unity_versions`
`text_assets``text_unit_count``text_unit_formats`
`text_unit_error_count` 等字段。旧索引库会通过 `metadata_json` 迁移列得到
默认空 metadata
`text_unit_error_count` 等字段。`entry` 还会保留 Addressables 的
`provider_id``bundle_name``hash``size``crc``dependencies`
旧索引库会通过 `metadata_json` 以及资源字段兼容迁移得到默认空值。
`resource.state``catalog.status``parse.status``localized.status`
都会返回当前观察面的短状态 `status` 与稳定状态码 `status_code``status_code`
+5 -5
View File
@@ -180,15 +180,15 @@ issue 43 例外说明:新增的 `parse repack` 只编排已有 TextAsset、Typ
### G-007Addressables Catalog 解析不完整
状态:**部分关闭**
状态:**当前目标字段已关闭,独立二进制格式待后续**
现象:
- `AddressablesCatalogDriver` 已能解析当前真实形态 JSON catalog fixture/golden。
- 已输出 path、hash、size、resource_type、address、dependencies、metadata,并已提取 `m_Crc``crc` 字段。
- 已输出 path、hash、size、resource_type、address、dependencies、provider ID、bundle name、metadata,并已提取 `m_Crc``crc` 字段。
- compact catalog 解析已补充 hash/size/CRC 的非 0 回归、资源计数 metadata,以及 blob 解码失败时的明确错误;不再在 compact 字段损坏时静默退回低保真 `m_InternalIds`
- `bat-core` 已提供 `crc32_ieee``ResourceEntry::verify_downloaded_bytes`SQLite `ResourceRepository` 已有 `crc`迁移。
- 仍需覆盖更多官方 catalog 结构变体、provider/bundle name 持久化字段,以及真实 Windows/Android catalog 样本集合
- `bat-core` 已提供 `crc32_ieee``ResourceEntry::verify_downloaded_bytes`SQLite `ResourceRepository` 已有 `crc`、provider_id、bundle_name 兼容迁移。
- 当前 JSON/compact 目标字段已通过 expanded、alias、compact 和真实形态 golden 回归;独立二进制 catalog 仍明确拒绝,不静默丢字段
影响:
@@ -197,7 +197,7 @@ issue 43 例外说明:新增的 `parse repack` 只编排已有 TextAsset、Typ
验收:
- 能解析项目目标版本的真实 Catalog 样本集合。
- 解析结果包含资源 key、provider、dependency、hash、size、path、CRC。
- 解析结果包含资源 key、provider、dependency、hash、size、path、bundle name、CRC。
- 对不支持的 catalog 结构返回明确错误,而不是静默丢字段。
- 解析结果能反查 bundle 文件、依赖链和本地下载 manifest 条目。
- Windows/Android 样本集合需要覆盖 JSON、compact JSON 和后续二进制 catalog 入口。
+12
View File
@@ -37,6 +37,18 @@
该例外不解冻解析器,不新增 UnityFS/AssetBundle/Addressables/TypeTree 解析类型、字段覆盖、catalog 结构或合成 fixture 能力。后续任何扩大解析覆盖的变更仍需单独解冻授权。
## issue 2/3 的开发例外
授权时间:2026-08-19
用户明确授权将 issue #2/#3 作为解析开发进展继续推进。本次例外允许:
- Addressables JSON/compact catalog 的 provider、bundle name、hash、size、CRC、资源类型和依赖关系字段补全,以及对应 SQLite/fixture 回归;
- UnityFS 已有 header、block、directory、压缩和 alignment 能力的校验加固,以及隔离真实 bundle 回归;
- 更新解析路线图、状态和 RPC/CLI 资源索引字段说明。
本次例外不包含发布级复杂对象重打包、完整 Unity 版本兼容承诺或新的汉化发布控制面;这些仍按后续 Patch/发布路线单独验收。
## 禁止变更
冻结期禁止以下解析相关变更:
+2
View File
@@ -3611,6 +3611,8 @@ fn test_resource(id: &str, path: &str, resource_type: ResourceType) -> Resource
resource_type,
address: None,
dependencies: Vec::new(),
provider_id: None,
bundle_name: None,
crc: None,
},
metadata,
+16
View File
@@ -560,6 +560,8 @@ mod tests {
resource_type: ResourceType::AssetBundle,
address: None,
dependencies: Vec::new(),
provider_id: None,
bundle_name: None,
crc: None,
},
ResourceEntry {
@@ -569,6 +571,8 @@ mod tests {
resource_type: ResourceType::Manifest,
address: None,
dependencies: Vec::new(),
provider_id: None,
bundle_name: None,
crc: None,
},
ResourceEntry {
@@ -578,6 +582,8 @@ mod tests {
resource_type: ResourceType::TextAsset,
address: Some("dialogue".to_string()),
dependencies: Vec::new(),
provider_id: None,
bundle_name: None,
crc: None,
},
ResourceEntry {
@@ -587,6 +593,8 @@ mod tests {
resource_type: ResourceType::TableBundle,
address: None,
dependencies: Vec::new(),
provider_id: None,
bundle_name: None,
crc: None,
},
ResourceEntry {
@@ -596,6 +604,8 @@ mod tests {
resource_type: ResourceType::Media,
address: None,
dependencies: Vec::new(),
provider_id: None,
bundle_name: None,
crc: None,
},
],
@@ -627,6 +637,8 @@ mod tests {
resource_type: ResourceType::TextAsset,
address: None,
dependencies: Vec::new(),
provider_id: None,
bundle_name: None,
crc: None,
}
}
@@ -648,6 +660,8 @@ mod tests {
resource_type: ResourceType::AssetBundle,
address: None,
dependencies: Vec::new(),
provider_id: None,
bundle_name: None,
crc: None,
},
]);
@@ -801,6 +815,8 @@ mod tests {
resource_type: ResourceType::AssetBundle,
address: None,
dependencies: Vec::new(),
provider_id: None,
bundle_name: None,
crc: None,
}]);
@@ -211,6 +211,8 @@ impl<'a> OfficialReleaseImportService<'a> {
resource_type,
address: None,
dependencies: Vec::new(),
provider_id: None,
bundle_name: None,
crc: None,
},
metadata,
+33 -5
View File
@@ -129,6 +129,8 @@ impl SqliteResourceRepository {
resource_type TEXT NOT NULL,
local_path TEXT NOT NULL,
address TEXT,
provider_id TEXT,
bundle_name TEXT,
dependencies_json TEXT NOT NULL DEFAULT '[]',
crc INTEGER,
metadata_json TEXT NOT NULL DEFAULT '{}'
@@ -148,6 +150,8 @@ impl SqliteResourceRepository {
"TEXT NOT NULL DEFAULT '{}'",
)
.await?;
Self::ensure_column(&self.pool, "resources", "provider_id", "TEXT").await?;
Self::ensure_column(&self.pool, "resources", "bundle_name", "TEXT").await?;
Self::execute_query(
&self.pool,
@@ -280,6 +284,8 @@ impl SqliteResourceRepository {
resource_type,
local_path,
address,
provider_id,
bundle_name,
dependencies_json,
crc,
metadata_json,
@@ -294,6 +300,8 @@ impl SqliteResourceRepository {
resource_type: Self::resource_type_from_str(&resource_type)?,
address,
dependencies: Self::dependencies_from_json(&dependencies_json)?,
provider_id,
bundle_name,
crc: crc.and_then(|value| u32::try_from(value).ok()),
},
metadata: Self::metadata_from_json(&metadata_json)?,
@@ -341,7 +349,7 @@ impl SqliteResourceRepository {
limit: Option<usize>,
) -> bat_core::Result<Vec<Resource>> {
let mut builder = QueryBuilder::<Sqlite>::new(
"SELECT id, path, hash, size, resource_type, local_path, address, dependencies_json, crc, metadata_json FROM resources",
"SELECT id, path, hash, size, resource_type, local_path, address, provider_id, bundle_name, dependencies_json, crc, metadata_json FROM resources",
);
Self::apply_filters(&mut builder, query)?;
builder.push(" ORDER BY id");
@@ -392,9 +400,9 @@ impl ResourceRepository for SqliteResourceRepository {
sqlx::query(
r#"
INSERT INTO resources (
id, path, hash, size, resource_type, local_path, address, dependencies_json, crc, metadata_json
id, path, hash, size, resource_type, local_path, address, provider_id, bundle_name, dependencies_json, crc, metadata_json
)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12)
ON CONFLICT(id) DO UPDATE SET
path = excluded.path,
hash = excluded.hash,
@@ -402,6 +410,8 @@ impl ResourceRepository for SqliteResourceRepository {
resource_type = excluded.resource_type,
local_path = excluded.local_path,
address = excluded.address,
provider_id = excluded.provider_id,
bundle_name = excluded.bundle_name,
dependencies_json = excluded.dependencies_json,
crc = excluded.crc,
metadata_json = excluded.metadata_json
@@ -414,6 +424,8 @@ impl ResourceRepository for SqliteResourceRepository {
.bind(Self::resource_type_to_str(resource.entry.resource_type))
.bind(resource.local_path.to_string_lossy().to_string())
.bind(resource.entry.address.clone())
.bind(resource.entry.provider_id.clone())
.bind(resource.entry.bundle_name.clone())
.bind(dependencies)
.bind(resource.entry.crc.map(i64::from))
.bind(metadata),
@@ -426,7 +438,7 @@ impl ResourceRepository for SqliteResourceRepository {
async fn find_by_id(&self, id: &str) -> bat_core::Result<Resource> {
let row: Option<ResourceRow> = sqlx::query_as(
r#"
SELECT id, path, hash, size, resource_type, local_path, address, dependencies_json, crc, metadata_json
SELECT id, path, hash, size, resource_type, local_path, address, provider_id, bundle_name, dependencies_json, crc, metadata_json
FROM resources
WHERE id = ?1
"#,
@@ -444,7 +456,7 @@ impl ResourceRepository for SqliteResourceRepository {
async fn find_by_hash(&self, hash: &str) -> bat_core::Result<Resource> {
let row: Option<ResourceRow> = sqlx::query_as(
r#"
SELECT id, path, hash, size, resource_type, local_path, address, dependencies_json, crc, metadata_json
SELECT id, path, hash, size, resource_type, local_path, address, provider_id, bundle_name, dependencies_json, crc, metadata_json
FROM resources
WHERE hash = ?1
ORDER BY id
@@ -505,6 +517,8 @@ type ResourceRow = (
String,
String,
Option<String>,
Option<String>,
Option<String>,
String,
Option<i64>,
String,
@@ -652,6 +666,8 @@ mod tests {
resource_type,
address: None,
dependencies: Vec::new(),
provider_id: None,
bundle_name: None,
crc: None,
},
metadata: ResourceMetadata::default(),
@@ -796,6 +812,10 @@ mod tests {
.entry
.dependencies
.push("assets/shared.bundle".to_string());
resource.entry.provider_id = Some(
"UnityEngine.ResourceManagement.ResourceProviders.AssetBundleProvider".to_string(),
);
resource.entry.bundle_name = Some("assets/model.bundle".to_string());
resource.metadata.official_release_id = Some("release-1".to_string());
resource.metadata.platform = Some("windows".to_string());
resource.metadata.bundle_path = Some("assets/model.bundle".to_string());
@@ -813,6 +833,14 @@ mod tests {
by_id.entry.dependencies,
vec!["assets/shared.bundle".to_string()]
);
assert_eq!(
by_id.entry.provider_id.as_deref(),
Some("UnityEngine.ResourceManagement.ResourceProviders.AssetBundleProvider")
);
assert_eq!(
by_id.entry.bundle_name.as_deref(),
Some("assets/model.bundle")
);
assert_eq!(
by_id.metadata.official_release_id.as_deref(),
Some("release-1")