mirror of
https://github.com/Yuyi-Oak/BlueArchiveToolkit.git
synced 2026-07-22 01:15:14 +08:00
feat: validate official zip structure
This commit is contained in:
+2
-2
@@ -18,7 +18,7 @@ Rust 侧官方日服资源链路已经从实验验证推进到正式入口:
|
||||
2. 不安装、不启动、不依赖已安装官方启动器。
|
||||
3. 默认平台为 `Windows + Android`。
|
||||
4. 能生成官方全量 pull plan,执行真实下载,维护 `official-download-manifest.json`。
|
||||
5. 下载后使用本地 manifest 的 size + BLAKE3 校验复用文件;官方 seed `.hash` 使用 `xxHash32(seed=0)` 强校验。
|
||||
5. 下载后使用本地 manifest 的 size + BLAKE3 校验复用文件;所有 `.zip` 在下载验收、复用、本地 audit/verify 时做 ZIP 结构校验;官方 seed `.hash` 使用 `xxHash32(seed=0)` 强校验。
|
||||
6. 支持 `.part` 断点续传、失败后 clean retry、本地 manifest audit/repair。
|
||||
7. `bat --watch` 可常驻运行,`bat --daemon` 可后台运行并用 `bat status` / `bat stop` / `bat restart` / `bat reload` / `bat logs` 管理;daemon 使用 `bat.sock` Unix socket JSON-RPC 作为 live 控制通道,PID/状态/日志文件作为快照和 fallback;正常检查默认每 1 小时一次;远端和本地一致时默认静默,失败后默认 60 秒快速重试;CLI 默认向 stdout 输出人类可读摘要,向 stderr 输出 ASCII banner 和 progress log,需要机器输出时使用 `--json --no-progress`。
|
||||
8. 远端 snapshot 未变化但输出目录为空时,会按首次运行执行全量拉取;官方 seed `.hash` 校验失败时会清理对应 manifest 条目,避免失败产物被后续本地 audit 误判为可复用。
|
||||
@@ -125,7 +125,7 @@ Rust 侧官方日服资源链路已经从实验验证推进到正式入口:
|
||||
- `InMemoryResourceRepository`。
|
||||
- `SqliteResourceRepository`。
|
||||
- 官方 pull plan 构建。
|
||||
- `OfficialResourcePullService`:官方 URL 拒绝策略、目标路径映射、下载 manifest、`.part` 续传、重试、官方 seed `.hash` 校验、本地全量 verify。
|
||||
- `OfficialResourcePullService`:官方 URL 拒绝策略、目标路径映射、下载 manifest、`.part` 续传、重试、ZIP 结构校验、官方 seed `.hash` 校验、本地全量 verify。
|
||||
- `OfficialUpdateService`:官方 metadata auto-discover、bootstrap cache、snapshot diff、marker diff、本地 audit/repair。
|
||||
- `bat`:正式 CLI binary,支持 one-shot、`--watch`、`--daemon`、`status`、`stop`、`restart`、`reload`、`refresh`、`logs`、`verify`、`repair`、`doctor` 和 `clean-stable`。
|
||||
|
||||
|
||||
@@ -13,7 +13,7 @@
|
||||
- `bat-adapters` Unity、Manifest、Client 集成框架,以及当前真实形态 Addressables catalog 解析覆盖。
|
||||
- `bat-cas-engine` CAS V1:原子写入、BLAKE3 校验、引用计数、GC、并发写入测试、损坏检测。
|
||||
- `bat-infrastructure` CAS 适配层、SQLite Resource Repository、官方资源 pull/update 服务。
|
||||
- `bat`:官方资源自动发现、全量拉取、本地 manifest audit/repair、`.part` 断点续传、官方 seed `.hash` 校验、snapshot/cache、`--watch` 常驻更新、`--daemon` 后台运行,以及 Unix socket JSON-RPC 后台控制命令 `status/stop/restart/reload/refresh/logs/verify/repair/doctor/clean-stable`。
|
||||
- `bat`:官方资源自动发现、全量拉取、本地 manifest audit/repair、`.part` 断点续传、ZIP 结构校验、官方 seed `.hash` 校验、snapshot/cache、`--watch` 常驻更新、`--daemon` 后台运行,以及 Unix socket JSON-RPC 后台控制命令 `status/stop/restart/reload/refresh/logs/verify/repair/doctor/clean-stable`。
|
||||
- `bat-ffi` 粗粒度 JSON 接口:Manifest inspect 和官方 sync plan。
|
||||
- 文档路线图、当前状态、缺口清单、官方资源运行指南。
|
||||
|
||||
@@ -105,7 +105,7 @@ cargo run -p bat-infrastructure --bin bat -- doctor
|
||||
cargo run -p bat-infrastructure --bin bat -- clean-stable
|
||||
```
|
||||
|
||||
`verify` 会以只读方式检查当前官方计划、本地 download manifest 的 size+BLAKE3,以及本地已有官方 seed `.bytes/.hash` 对的 xxHash32;发现缺失、远端变化或本地损坏会返回非 0。`repair` 会在异常资源存在时复用当前同步链路重新下载必要文件。`clean-stable` 只清理 `.part`、临时状态文件和失效锁/PID,不删除正式资源。
|
||||
`verify` 会以只读方式检查当前官方计划、本地 download manifest 的 size+BLAKE3、ZIP 结构,以及本地已有官方 seed `.bytes/.hash` 对的 xxHash32;发现缺失、远端变化或本地损坏会返回非 0。`repair` 会在异常资源存在时复用当前同步链路重新下载必要文件。`clean-stable` 只清理 `.part`、临时状态文件和失效锁/PID,不删除正式资源。
|
||||
|
||||
CLI 默认启动时会向 stderr 打印 `BlueArchiveToolkit` ASCII banner,并继续把阶段进度日志写到 stderr,例如自动发现、拉取 catalog、audit、下载第 N/总数个 URL 等;命令结果默认以人类可读摘要写到 stdout。需要给上层程序保留稳定结构化输出时加 `--json --no-progress`,只想关闭横幅但保留日志时可加 `--no-banner`。
|
||||
|
||||
|
||||
@@ -122,13 +122,13 @@
|
||||
1. 检查 URL 是否属于官方 host。
|
||||
2. 把 URL 映射到本地输出路径。
|
||||
3. 读取输出目录下的 `official-download-manifest.json`。
|
||||
4. 已存在目标文件只有在本地清单中的相对路径、size、BLAKE3 都匹配时才跳过。
|
||||
5. 缺少清单、清单不匹配或文件损坏时重新下载。
|
||||
4. 已存在目标文件只有在本地清单中的相对路径、size、BLAKE3 都匹配时才跳过;`.zip` 文件还必须通过 ZIP central directory / local header 结构校验。
|
||||
5. 缺少清单、清单不匹配、ZIP 结构无效或文件损坏时重新下载。
|
||||
6. `TableCatalog.bytes`、`BundlePackingInfo.bytes`、`MediaCatalog.bytes` 总是刷新并用官方 `.hash` 强校验;该 `.hash` 是 `xxHash32(seed=0)` 的十进制文本。
|
||||
7. `catalog_*.hash` 当前只作为 Addressables catalog 变更标记,不作为 zip/JSON 内容校验算法;Unity Addressables/SBP builder 对 JSON/bin catalog 使用 `HashingMethods.Calculate` 生成 `Hash128` 文本,运行时用它判断 remote catalog cache 是否过期,它不能套用 seed catalog 的 `xxHash32` 规则。
|
||||
8. 官方 seed `.hash` 校验失败会让当前下载失败,并移除对应 data/hash URL 的本地 manifest 条目,避免失败产物在下一轮被本地 BLAKE3 audit 误判为健康缓存。
|
||||
9. 存在 `.part` 临时文件时通过 `curl --continue-at -` 尝试断点续传。
|
||||
10. 新下载写入 `.part`,成功后原子 rename 到最终路径。
|
||||
10. 新下载写入 `.part`,成功并通过必要校验后原子 rename 到最终路径;断点续传后的 `.zip` 如果结构无效,会删除 `.part` 并重新全量下载。
|
||||
11. 成功下载后更新本地下载清单。
|
||||
12. 记录最终文件大小、本次传输字节数、官方 hash 校验数和执行状态。
|
||||
13. 非官方 URL 直接拒绝。
|
||||
@@ -189,7 +189,7 @@
|
||||
5. 使用 `OfficialSyncPlan` 和扩展 snapshot diff 判断是否需要下载;URL 未变但 `.hash` / marker 内容变化也会触发更新。
|
||||
6. 每轮都会基于最新 seed catalog 构建当前 pull plan,并检查输出目录是否已有当前 plan 的 manifest 条目或目标文件。
|
||||
7. 如果远端 snapshot 未变化但输出目录没有任何当前 plan 的本地资源,仍按首次运行处理并执行全量拉取。
|
||||
8. 远端无变化且本地已有资源时执行 download manifest audit,检查路径、size 和 BLAKE3。
|
||||
8. 远端无变化且本地已有资源时执行 download manifest audit,检查路径、size、BLAKE3 和 ZIP 结构。
|
||||
9. 远端变化、本地 audit 发现 repair_needed,或首次空目录运行时,下载并校验官方 URL。
|
||||
10. 下载成功后写回新的 snapshot。
|
||||
|
||||
@@ -233,7 +233,7 @@ Linux 生产路径:
|
||||
- `OfficialUpdateService` 能持久化 v2 snapshot,并在远端 marker 内容变化时触发下载决策
|
||||
- `bat` 默认向 stderr 输出 `BlueArchiveToolkit` ASCII banner 和 progress log,stdout 默认输出人类可读摘要;支持 `--json` 输出稳定 JSON,支持 `--no-progress` 关闭进度日志,支持 `--no-banner` 只关闭横幅,支持 `--watch --interval 1h --error-retry 60s` 常驻运行,支持 `--daemon` Unix socket JSON-RPC 控制、`status`、`stop`、`restart`、`reload`、`logs`、`refresh --force`、`verify`、`repair`、`doctor`、`clean-stable`,非 dry-run 使用 `.official-sync.lock` 防止并发写资源目录
|
||||
- `OfficialUpdateService` 能读写 `official-bootstrap-cache.json`,并支持默认开启的 `audit_local` / `repair` CLI 行为
|
||||
- 下载层能在本地文件 size/BLAKE3/path 或 manifest 不匹配时重新下载
|
||||
- 下载层能在本地文件 size/BLAKE3/path、ZIP 结构或 manifest 不匹配时重新下载
|
||||
- 官方 seed `.hash` mismatch 会导致下载失败,而不是降级为本地 BLAKE3 猜测
|
||||
- 非官方 URL 会在 fetch/download 入口被拒绝
|
||||
- 尚未在仓库中记录真实官方网络全量下载 smoke test
|
||||
|
||||
@@ -136,11 +136,11 @@ cargo run -p bat-infrastructure --example official_pull_plan -- \
|
||||
- 只接受官方 URL
|
||||
- 下载完整官方资源集合
|
||||
- 下载成功后在输出目录维护 `official-download-manifest.json`
|
||||
- 已存在目标文件只有在本地下载清单中的 size 和 BLAKE3 校验通过时才会跳过
|
||||
- 已存在目标文件只有在本地下载清单中的 size 和 BLAKE3 校验通过时才会跳过;`.zip` 文件还必须通过 ZIP central directory / local header 结构校验
|
||||
- `TableCatalog.bytes`、`BundlePackingInfo.bytes`、`MediaCatalog.bytes` 会用官方 `.hash` 做强校验;该 `.hash` 是 `xxHash32(seed=0)` 的十进制文本
|
||||
- `catalog_*.hash` 是 Unity Addressables/SBP 写出的 catalog 变更标记,来源是 `HashingMethods.Calculate(json/bin catalog)` 生成的 `Hash128` 文本;它不是 seed catalog 的 `xxHash32` 规则,目前不作为 zip/JSON 内容强校验
|
||||
- 存在 `.part` 临时文件时会尝试断点续传
|
||||
- 新下载先写 `.part`,成功后再替换为最终文件
|
||||
- 新下载先写 `.part`,成功并通过必要校验后再替换为最终文件;如果断点续传后的 `.zip` 结构校验失败,会删除 `.part` 并重新全量下载
|
||||
- 把结果写入 `--output`
|
||||
|
||||
## 5. 自动更新检查
|
||||
@@ -151,7 +151,7 @@ cargo run -p bat-infrastructure --example official_pull_plan -- \
|
||||
- URL 没变但 `.hash` / marker 内容变更时,也会判定为需要更新。
|
||||
- 即使远端 snapshot 未变化,只要输出目录没有任何当前 pull plan 的本地 manifest 条目或目标文件,也会按首次运行处理并执行全量拉取。
|
||||
- 远端无变化时,默认执行本地 `official-download-manifest.json` audit。
|
||||
- 本地文件缺失、路径不一致、size 不一致或 BLAKE3 不一致时,默认进入 repair 并重下必要文件。
|
||||
- 本地文件缺失、路径不一致、size 不一致、BLAKE3 不一致或 ZIP 结构无效时,默认进入 repair 并重下必要文件。
|
||||
- 官方 seed `.hash` 校验失败会让本轮失败,并清理对应本地 manifest 条目;下一轮会继续把这类文件视为需要 repair,而不是把失败产物当作健康缓存复用。
|
||||
- 远端和本地都一致:单次模式输出 `update_status=up_to_date`,watch 模式默认静默并等待下次检查。
|
||||
- 有远端变化或本地 repair:生成 pull plan,下载完整官方资源,成功后更新 snapshot。
|
||||
@@ -159,7 +159,7 @@ cargo run -p bat-infrastructure --example official_pull_plan -- \
|
||||
- `--dry-run --plan`:除更新判断外,还会解析 seed catalog 并打印完整下载 URL。
|
||||
- 真实更新会输出 `downloaded_count`、`resumed_count`、`skipped_count`、`transferred_bytes`、`official_seed_hash_verified_count`。
|
||||
- 校验报告分层输出 `official_seed_hash_verified_count`、`local_manifest_verified_count`、`addressables_marker_checked_count`、`unverified_marker_count`。
|
||||
- 下载阶段复用同一套本地清单和 `.part` 续传逻辑;没有清单或校验不匹配的文件会重新下载。
|
||||
- 下载阶段复用同一套本地清单、ZIP 结构校验和 `.part` 续传逻辑;没有清单或校验不匹配的文件会重新下载。
|
||||
|
||||
资源同步状态文件默认位于 `--output` 下:
|
||||
|
||||
@@ -241,7 +241,7 @@ cargo run -p bat-infrastructure --bin bat -- clean-stable
|
||||
|
||||
- `refresh` 执行一次同步检查;`refresh --force` 强制刷新并重新匹配当前官方资源。
|
||||
- 如果后台 daemon 正在运行,并且 `refresh` 没有显式指定另一套资源目录、server-info、connection-group、app-version、platforms、snapshot、curl 或 unzip 参数,`refresh` / `refresh --force` 会通过 RPC 唤醒或排队后台进程执行;否则作为一次性前台同步运行。
|
||||
- `verify` 只读验证当前官方计划、本地 manifest size+BLAKE3,以及本地已有官方 seed `.bytes/.hash` 对的 xxHash32;资源缺失、远端变化或本地损坏时返回非 0。
|
||||
- `verify` 只读验证当前官方计划、本地 manifest size+BLAKE3、ZIP 结构,以及本地已有官方 seed `.bytes/.hash` 对的 xxHash32;资源缺失、远端变化或本地损坏时返回非 0。
|
||||
- `repair` 在有异常资源时尝试重新下载并修复,成功后重新写 manifest 和 snapshot。
|
||||
- `doctor` 检查输出目录、后台状态目录、curl/unzip、后台 PID、RPC socket 和资源锁。
|
||||
- `clean-stable` 只能在后台未运行且 RPC socket 不可连接时清理 `.part`、`.tmp`、失效 PID、失效 socket 和失效锁,不删除正式资源文件。
|
||||
|
||||
@@ -1781,6 +1781,7 @@ struct VerificationItemReport {
|
||||
actual_bytes: Option<u64>,
|
||||
expected_blake3: Option<String>,
|
||||
actual_blake3: Option<String>,
|
||||
zip_error: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
@@ -1830,6 +1831,7 @@ fn run_verify_command(options: &CliOptions) -> anyhow::Result<bool> {
|
||||
actual_bytes: item.actual_bytes,
|
||||
expected_blake3: item.expected_blake3.clone(),
|
||||
actual_blake3: item.actual_blake3.clone(),
|
||||
zip_error: item.zip_error.clone(),
|
||||
})
|
||||
.collect::<Vec<_>>();
|
||||
let healthy = update_report.update_status == OfficialUpdateStatus::UpToDate
|
||||
|
||||
@@ -19,6 +19,7 @@ pub mod official_pull;
|
||||
pub mod official_sync;
|
||||
pub mod official_update;
|
||||
pub mod resources;
|
||||
mod zip_validation;
|
||||
|
||||
pub use cas::FileSystemCasRepository;
|
||||
pub use import::{BundleSource, ImportedResource, ResourceImportReport, ResourceImportService};
|
||||
|
||||
@@ -1,6 +1,9 @@
|
||||
//! Official JP resource download execution.
|
||||
|
||||
use crate::official_pull::OfficialResourcePullPlan;
|
||||
use crate::zip_validation::{
|
||||
path_has_zip_extension, url_or_path_has_zip_extension, validate_zip_structure,
|
||||
};
|
||||
use bat_adapters::official::yostar_jp::{is_official_yostar_jp_url, YostarJpResourceEndpointKind};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::collections::{BTreeMap, HashSet};
|
||||
@@ -219,6 +222,9 @@ pub enum OfficialLocalManifestAuditStatus {
|
||||
SizeMismatch,
|
||||
/// The local file BLAKE3 differs from the manifest entry.
|
||||
Blake3Mismatch,
|
||||
/// The local file has a `.zip` URL or destination but its ZIP structure is
|
||||
/// invalid.
|
||||
ZipStructureInvalid,
|
||||
}
|
||||
|
||||
impl OfficialLocalManifestAuditStatus {
|
||||
@@ -237,6 +243,7 @@ impl OfficialLocalManifestAuditStatus {
|
||||
Self::MissingFile => "missing_file",
|
||||
Self::SizeMismatch => "size_mismatch",
|
||||
Self::Blake3Mismatch => "blake3_mismatch",
|
||||
Self::ZipStructureInvalid => "zip_structure_invalid",
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -260,6 +267,9 @@ pub struct OfficialLocalManifestAuditItem {
|
||||
pub expected_blake3: Option<String>,
|
||||
/// Actual local BLAKE3, when it could be computed.
|
||||
pub actual_blake3: Option<String>,
|
||||
/// ZIP structure validation error, when the URL or destination is a ZIP
|
||||
/// and the local file did not pass ZIP structure validation.
|
||||
pub zip_error: Option<String>,
|
||||
}
|
||||
|
||||
/// Local download-manifest audit result for a pull plan.
|
||||
@@ -636,6 +646,11 @@ impl OfficialResourcePullService {
|
||||
let partial = partial_path_for(destination);
|
||||
let partial_len_before = file_len_if_exists(&partial)?.unwrap_or_default();
|
||||
let resumed = partial_len_before > 0;
|
||||
let mut status = if resumed {
|
||||
OfficialResourcePullStatus::Resumed
|
||||
} else {
|
||||
OfficialResourcePullStatus::Downloaded
|
||||
};
|
||||
|
||||
if resumed {
|
||||
if let Err(error) = self.download_one(url, &partial, true) {
|
||||
@@ -644,10 +659,22 @@ impl OfficialResourcePullService {
|
||||
.map_err(|retry_error| {
|
||||
format!("续传失败 {url}:{error};清理后重新下载也失败:{retry_error}")
|
||||
})?;
|
||||
status = OfficialResourcePullStatus::Downloaded;
|
||||
} else if let Err(error) = self.validate_zip_if_needed(url, &partial) {
|
||||
let _ = fs::remove_file(&partial);
|
||||
self.download_one(url, &partial, false)
|
||||
.map_err(|retry_error| {
|
||||
format!("续传后的 ZIP 结构校验失败 {url}:{error};清理后重新下载也失败:{retry_error}")
|
||||
})?;
|
||||
status = OfficialResourcePullStatus::Downloaded;
|
||||
}
|
||||
} else {
|
||||
self.download_one(url, &partial, false)?;
|
||||
}
|
||||
if let Err(error) = self.validate_zip_if_needed(url, &partial) {
|
||||
let _ = fs::remove_file(&partial);
|
||||
return Err(error);
|
||||
}
|
||||
|
||||
if file_len_if_exists(destination)?.is_some() {
|
||||
fs::remove_file(destination).map_err(|error| {
|
||||
@@ -669,12 +696,12 @@ impl OfficialResourcePullService {
|
||||
|
||||
Ok(PullOneResult {
|
||||
bytes,
|
||||
transferred_bytes: bytes.saturating_sub(partial_len_before),
|
||||
status: if resumed {
|
||||
OfficialResourcePullStatus::Resumed
|
||||
transferred_bytes: if status == OfficialResourcePullStatus::Resumed {
|
||||
bytes.saturating_sub(partial_len_before)
|
||||
} else {
|
||||
OfficialResourcePullStatus::Downloaded
|
||||
bytes
|
||||
},
|
||||
status,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -914,6 +941,10 @@ impl OfficialResourcePullService {
|
||||
return Ok(None);
|
||||
}
|
||||
|
||||
if let Err(_error) = self.validate_zip_if_needed(url, destination) {
|
||||
return Ok(None);
|
||||
}
|
||||
|
||||
Ok(Some(PullOneResult {
|
||||
bytes,
|
||||
transferred_bytes: 0,
|
||||
@@ -927,6 +958,7 @@ impl OfficialResourcePullService {
|
||||
url: &str,
|
||||
destination: &Path,
|
||||
) -> Result<(), String> {
|
||||
self.validate_zip_if_needed(url, destination)?;
|
||||
let bytes = fs::metadata(destination)
|
||||
.map_err(|error| format!("读取已下载文件信息失败 {}:{error}", destination.display()))?
|
||||
.len();
|
||||
@@ -946,6 +978,14 @@ impl OfficialResourcePullService {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn validate_zip_if_needed(&self, url: &str, destination: &Path) -> Result<(), String> {
|
||||
if !url_or_path_has_zip_extension(url) && !path_has_zip_extension(destination) {
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
validate_zip_structure(destination).map(|_report| ())
|
||||
}
|
||||
|
||||
fn relative_destination(&self, destination: &Path) -> Result<String, String> {
|
||||
let relative = destination
|
||||
.strip_prefix(&self.output_root)
|
||||
@@ -977,6 +1017,7 @@ impl OfficialResourcePullService {
|
||||
actual_bytes,
|
||||
expected_blake3: None,
|
||||
actual_blake3: None,
|
||||
zip_error: None,
|
||||
});
|
||||
};
|
||||
|
||||
@@ -994,6 +1035,7 @@ impl OfficialResourcePullService {
|
||||
actual_bytes: None,
|
||||
expected_blake3,
|
||||
actual_blake3: None,
|
||||
zip_error: None,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -1008,6 +1050,7 @@ impl OfficialResourcePullService {
|
||||
actual_bytes: None,
|
||||
expected_blake3,
|
||||
actual_blake3: None,
|
||||
zip_error: None,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -1021,6 +1064,7 @@ impl OfficialResourcePullService {
|
||||
actual_bytes: None,
|
||||
expected_blake3,
|
||||
actual_blake3: None,
|
||||
zip_error: None,
|
||||
});
|
||||
};
|
||||
|
||||
@@ -1034,6 +1078,7 @@ impl OfficialResourcePullService {
|
||||
actual_bytes: Some(actual_bytes),
|
||||
expected_blake3,
|
||||
actual_blake3: None,
|
||||
zip_error: None,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -1048,6 +1093,21 @@ impl OfficialResourcePullService {
|
||||
actual_bytes: Some(actual_bytes),
|
||||
expected_blake3,
|
||||
actual_blake3: Some(actual_blake3),
|
||||
zip_error: None,
|
||||
});
|
||||
}
|
||||
|
||||
if let Err(error) = self.validate_zip_if_needed(url, &destination) {
|
||||
return Ok(OfficialLocalManifestAuditItem {
|
||||
url: url.to_string(),
|
||||
destination,
|
||||
status: OfficialLocalManifestAuditStatus::ZipStructureInvalid,
|
||||
manifest_destination,
|
||||
expected_bytes,
|
||||
actual_bytes: Some(actual_bytes),
|
||||
expected_blake3,
|
||||
actual_blake3: Some(actual_blake3),
|
||||
zip_error: Some(error),
|
||||
});
|
||||
}
|
||||
|
||||
@@ -1060,6 +1120,7 @@ impl OfficialResourcePullService {
|
||||
actual_bytes: Some(actual_bytes),
|
||||
expected_blake3,
|
||||
actual_blake3: Some(actual_blake3),
|
||||
zip_error: None,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -1477,6 +1538,9 @@ if [ -n "$out" ]; then
|
||||
*/MediaResources/Catalog/MediaCatalog.bytes)
|
||||
printf '%s' 'JP_Airi_Android.zip' > "$out"
|
||||
;;
|
||||
*.zip)
|
||||
printf '\120\113\003\004\024\000\000\000\000\000\000\000\000\000\000\000\000\000\002\000\000\000\002\000\000\000\010\000\000\000file.txtok\120\113\001\002\024\000\024\000\000\000\000\000\000\000\000\000\000\000\000\000\002\000\000\000\002\000\000\000\010\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000file.txt\120\113\005\006\000\000\000\000\001\000\001\000\066\000\000\000\050\000\000\000\000\000' > "$out"
|
||||
;;
|
||||
*)
|
||||
printf '%s' "$url" > "$out"
|
||||
;;
|
||||
@@ -1610,6 +1674,35 @@ fi
|
||||
"#
|
||||
}
|
||||
|
||||
fn fake_bad_zip_curl_script() -> &'static str {
|
||||
r#"#!/bin/sh
|
||||
set -eu
|
||||
out=""
|
||||
url=""
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
--output)
|
||||
out="$2"
|
||||
shift 2
|
||||
;;
|
||||
--url)
|
||||
url="$2"
|
||||
shift 2
|
||||
;;
|
||||
*)
|
||||
shift
|
||||
;;
|
||||
esac
|
||||
done
|
||||
if [ -n "$out" ]; then
|
||||
mkdir -p "$(dirname "$out")"
|
||||
printf '%s' 'not a zip archive' > "$out"
|
||||
else
|
||||
printf '%s' "$url"
|
||||
fi
|
||||
"#
|
||||
}
|
||||
|
||||
fn write_shell_script(path: &Path, script: &str) {
|
||||
fs::write(path, script).unwrap();
|
||||
#[cfg(unix)]
|
||||
@@ -1637,6 +1730,58 @@ fi
|
||||
write_shell_script(path, fake_bad_hash_curl_script());
|
||||
}
|
||||
|
||||
fn write_fake_bad_zip_curl(path: &Path) {
|
||||
write_shell_script(path, fake_bad_zip_curl_script());
|
||||
}
|
||||
|
||||
fn one_file_zip(name: &[u8], data: &[u8]) -> Vec<u8> {
|
||||
let mut bytes = Vec::new();
|
||||
bytes.extend_from_slice(&0x0403_4b50u32.to_le_bytes());
|
||||
bytes.extend_from_slice(&20u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u32.to_le_bytes());
|
||||
bytes.extend_from_slice(&(data.len() as u32).to_le_bytes());
|
||||
bytes.extend_from_slice(&(data.len() as u32).to_le_bytes());
|
||||
bytes.extend_from_slice(&(name.len() as u16).to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(name);
|
||||
bytes.extend_from_slice(data);
|
||||
|
||||
let central_offset = bytes.len() as u32;
|
||||
bytes.extend_from_slice(&0x0201_4b50u32.to_le_bytes());
|
||||
bytes.extend_from_slice(&20u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&20u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u32.to_le_bytes());
|
||||
bytes.extend_from_slice(&(data.len() as u32).to_le_bytes());
|
||||
bytes.extend_from_slice(&(data.len() as u32).to_le_bytes());
|
||||
bytes.extend_from_slice(&(name.len() as u16).to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u32.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u32.to_le_bytes());
|
||||
bytes.extend_from_slice(name);
|
||||
let central_size = bytes.len() as u32 - central_offset;
|
||||
|
||||
bytes.extend_from_slice(&0x0605_4b50u32.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&1u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&1u16.to_le_bytes());
|
||||
bytes.extend_from_slice(¢ral_size.to_le_bytes());
|
||||
bytes.extend_from_slice(¢ral_offset.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn downloads_verified_pull_plan_to_disk() {
|
||||
let out_dir = TempDir::new().unwrap();
|
||||
@@ -1764,11 +1909,13 @@ fi
|
||||
let destination = service.destination_for_url(&url).unwrap();
|
||||
fs::create_dir_all(destination.parent().unwrap()).unwrap();
|
||||
let bytes = if url.ends_with("/TableBundles/TableCatalog.bytes") {
|
||||
b"ExcelDB.db".as_slice()
|
||||
b"ExcelDB.db".to_vec()
|
||||
} else if url.ends_with("/TableBundles/TableCatalog.hash") {
|
||||
b"3223500437".as_slice()
|
||||
b"3223500437".to_vec()
|
||||
} else if url.ends_with(".zip") {
|
||||
one_file_zip(b"fixture.txt", b"ok")
|
||||
} else {
|
||||
url.as_bytes()
|
||||
url.as_bytes().to_vec()
|
||||
};
|
||||
fs::write(&destination, bytes).unwrap();
|
||||
service
|
||||
@@ -1955,6 +2102,99 @@ fi
|
||||
assert_eq!(xxhash32(b"JP_Airi_Win.zip"), 876690720);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_downloaded_zip_when_structure_is_invalid() {
|
||||
let out_dir = TempDir::new().unwrap();
|
||||
let bin_dir = TempDir::new().unwrap();
|
||||
let curl_path = bin_dir.path().join("curl");
|
||||
write_fake_bad_zip_curl(&curl_path);
|
||||
|
||||
let service = OfficialResourcePullService::with_curl_command(out_dir.path(), &curl_path);
|
||||
let zip_url =
|
||||
"https://prod-clientpatch.bluearchiveyostar.com/r93_token/Windows_PatchPack/Broken.zip";
|
||||
let plan = OfficialResourcePullPlan {
|
||||
discovery: YostarJpResourceDiscoveryPlan {
|
||||
connection_group_name: "Prod-Audit".to_string(),
|
||||
app_version: "1.70.0".to_string(),
|
||||
bundle_version: None,
|
||||
addressables_root: "https://prod-clientpatch.bluearchiveyostar.com/r93_token"
|
||||
.to_string(),
|
||||
endpoints: vec![YostarJpResourceEndpoint {
|
||||
kind: YostarJpResourceEndpointKind::AddressablesCatalog,
|
||||
platform: Some(PatchPlatform::Windows),
|
||||
url: zip_url.to_string(),
|
||||
}],
|
||||
},
|
||||
inventory: YostarJpPlatformDownloadInventory::from_shared_inventory(
|
||||
YostarJpDownloadInventory::from_catalog_bytes(b"", b"", b""),
|
||||
&[],
|
||||
),
|
||||
platforms: Vec::new(),
|
||||
};
|
||||
|
||||
let error = service.pull(&plan).unwrap_err();
|
||||
|
||||
assert!(error.contains("ZIP"));
|
||||
assert!(!partial_path_for(&service.destination_for_url(zip_url).unwrap()).exists());
|
||||
assert!(service.read_download_manifest().unwrap().entries.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn local_manifest_audit_rejects_zip_with_matching_size_and_blake3_but_bad_structure() {
|
||||
let out_dir = TempDir::new().unwrap();
|
||||
let service = OfficialResourcePullService::new(out_dir.path());
|
||||
let zip_url =
|
||||
"https://prod-clientpatch.bluearchiveyostar.com/r93_token/Windows_PatchPack/Broken.zip";
|
||||
let destination = service.destination_for_url(zip_url).unwrap();
|
||||
fs::create_dir_all(destination.parent().unwrap()).unwrap();
|
||||
fs::write(&destination, b"not a zip archive").unwrap();
|
||||
|
||||
let mut manifest = OfficialDownloadManifest::default();
|
||||
let bytes = fs::metadata(&destination).unwrap().len();
|
||||
let blake3 = blake3_file_hex(&destination).unwrap();
|
||||
manifest.entries.insert(
|
||||
zip_url.to_string(),
|
||||
OfficialDownloadManifestEntry {
|
||||
url: zip_url.to_string(),
|
||||
destination: service.relative_destination(&destination).unwrap(),
|
||||
bytes,
|
||||
blake3,
|
||||
},
|
||||
);
|
||||
service.write_download_manifest(&manifest).unwrap();
|
||||
|
||||
let plan = OfficialResourcePullPlan {
|
||||
discovery: YostarJpResourceDiscoveryPlan {
|
||||
connection_group_name: "Prod-Audit".to_string(),
|
||||
app_version: "1.70.0".to_string(),
|
||||
bundle_version: None,
|
||||
addressables_root: "https://prod-clientpatch.bluearchiveyostar.com/r93_token"
|
||||
.to_string(),
|
||||
endpoints: vec![YostarJpResourceEndpoint {
|
||||
kind: YostarJpResourceEndpointKind::AddressablesCatalog,
|
||||
platform: Some(PatchPlatform::Windows),
|
||||
url: zip_url.to_string(),
|
||||
}],
|
||||
},
|
||||
inventory: YostarJpPlatformDownloadInventory::from_shared_inventory(
|
||||
YostarJpDownloadInventory::from_catalog_bytes(b"", b"", b""),
|
||||
&[],
|
||||
),
|
||||
platforms: Vec::new(),
|
||||
};
|
||||
|
||||
let audit = service.audit_local_manifest(&plan).unwrap();
|
||||
let verification = service.verify_local_download_manifest().unwrap();
|
||||
|
||||
assert_eq!(
|
||||
audit.items[0].status,
|
||||
OfficialLocalManifestAuditStatus::ZipStructureInvalid
|
||||
);
|
||||
assert!(audit.items[0].zip_error.as_ref().unwrap().contains("ZIP"));
|
||||
assert!(!verification.is_clean());
|
||||
assert_eq!(verification.failure_count(), 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn resumes_partial_files_before_atomic_completion() {
|
||||
let out_dir = TempDir::new().unwrap();
|
||||
|
||||
@@ -9,6 +9,7 @@ use crate::official_launcher::OfficialLauncherBootstrapService;
|
||||
use crate::official_launcher::{
|
||||
YostarJpLauncherCdnConfig, YostarJpLauncherGameConfig, YostarJpLauncherRemoteManifest,
|
||||
};
|
||||
use crate::zip_validation::validate_zip_structure;
|
||||
use bat_adapters::official::game_main_config::YostarJpGameMainConfig;
|
||||
use bat_adapters::official::launcher::YostarJpLauncherManifestFile;
|
||||
use std::fs;
|
||||
@@ -192,6 +193,7 @@ impl OfficialGameMainConfigBootstrapService {
|
||||
package_path,
|
||||
&archive_path,
|
||||
)?;
|
||||
validate_zip_structure(&archive_path)?;
|
||||
let extract_root = temp_root.join("extract");
|
||||
fs::create_dir_all(&extract_root)
|
||||
.map_err(|error| format!("创建解压目录失败:{error}"))?;
|
||||
|
||||
@@ -0,0 +1,688 @@
|
||||
//! ZIP structure validation helpers.
|
||||
|
||||
use std::fs::{self, File};
|
||||
use std::io::{Read, Seek, SeekFrom};
|
||||
use std::path::Path;
|
||||
|
||||
const EOCD_SIGNATURE: u32 = 0x0605_4b50;
|
||||
const ZIP64_EOCD_SIGNATURE: u32 = 0x0606_4b50;
|
||||
const ZIP64_EOCD_LOCATOR_SIGNATURE: u32 = 0x0706_4b50;
|
||||
const CENTRAL_DIRECTORY_SIGNATURE: u32 = 0x0201_4b50;
|
||||
const LOCAL_FILE_HEADER_SIGNATURE: u32 = 0x0403_4b50;
|
||||
const EOCD_MIN_LEN: u64 = 22;
|
||||
const EOCD_SEARCH_WINDOW: u64 = EOCD_MIN_LEN + 65_535;
|
||||
const ZIP64_EOCD_LOCATOR_LEN: u64 = 20;
|
||||
const ZIP64_EOCD_FIXED_LEN: usize = 56;
|
||||
const CENTRAL_DIRECTORY_FIXED_LEN: u64 = 46;
|
||||
const LOCAL_FILE_HEADER_FIXED_LEN: u64 = 30;
|
||||
const ZIP64_EXTENDED_INFORMATION_EXTRA_ID: u16 = 0x0001;
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub(crate) struct ZipStructureReport {
|
||||
pub(crate) entry_count: u64,
|
||||
pub(crate) uses_zip64: bool,
|
||||
}
|
||||
|
||||
pub(crate) fn path_has_zip_extension(path: &Path) -> bool {
|
||||
path.extension()
|
||||
.and_then(|extension| extension.to_str())
|
||||
.is_some_and(|extension| extension.eq_ignore_ascii_case("zip"))
|
||||
}
|
||||
|
||||
pub(crate) fn url_or_path_has_zip_extension(value: &str) -> bool {
|
||||
let value = value.split(['?', '#']).next().unwrap_or(value);
|
||||
value
|
||||
.rsplit('/')
|
||||
.next()
|
||||
.is_some_and(|name| name.to_ascii_lowercase().ends_with(".zip"))
|
||||
}
|
||||
|
||||
pub(crate) fn validate_zip_structure(path: &Path) -> Result<ZipStructureReport, String> {
|
||||
let metadata = fs::metadata(path)
|
||||
.map_err(|error| format!("读取 ZIP 文件信息失败 {}:{error}", path.display()))?;
|
||||
if !metadata.is_file() {
|
||||
return Err(format!("ZIP 路径不是文件:{}", path.display()));
|
||||
}
|
||||
|
||||
let file_len = metadata.len();
|
||||
if file_len < EOCD_MIN_LEN {
|
||||
return Err(format!(
|
||||
"ZIP 文件过短,找不到 EOCD 记录 {}:{} bytes",
|
||||
path.display(),
|
||||
file_len
|
||||
));
|
||||
}
|
||||
|
||||
let mut file = File::open(path)
|
||||
.map_err(|error| format!("打开 ZIP 文件失败 {}:{error}", path.display()))?;
|
||||
let eocd = find_eocd(&mut file, file_len, path)?;
|
||||
let central_directory = read_central_directory_descriptor(&mut file, &eocd, file_len, path)?;
|
||||
validate_central_directory(&mut file, ¢ral_directory, file_len, path)?;
|
||||
|
||||
Ok(ZipStructureReport {
|
||||
entry_count: central_directory.entry_count,
|
||||
uses_zip64: central_directory.uses_zip64,
|
||||
})
|
||||
}
|
||||
|
||||
fn find_eocd(file: &mut File, file_len: u64, path: &Path) -> Result<EocdRecord, String> {
|
||||
let tail_len = file_len.min(EOCD_SEARCH_WINDOW);
|
||||
let tail_start = file_len - tail_len;
|
||||
let mut tail = vec![0u8; tail_len as usize];
|
||||
file.seek(SeekFrom::Start(tail_start))
|
||||
.map_err(|error| format!("定位 ZIP EOCD 搜索窗口失败 {}:{error}", path.display()))?;
|
||||
file.read_exact(&mut tail)
|
||||
.map_err(|error| format!("读取 ZIP EOCD 搜索窗口失败 {}:{error}", path.display()))?;
|
||||
|
||||
let min_offset = tail.len().saturating_sub(EOCD_MIN_LEN as usize);
|
||||
for offset in (0..=min_offset).rev() {
|
||||
if read_u32_le(&tail, offset)? != EOCD_SIGNATURE {
|
||||
continue;
|
||||
}
|
||||
|
||||
let comment_len = read_u16_le(&tail, offset + 20)? as u64;
|
||||
let absolute_offset = tail_start + offset as u64;
|
||||
if absolute_offset + EOCD_MIN_LEN + comment_len != file_len {
|
||||
continue;
|
||||
}
|
||||
|
||||
return Ok(EocdRecord {
|
||||
offset: absolute_offset,
|
||||
disk_number: read_u16_le(&tail, offset + 4)?,
|
||||
central_directory_disk: read_u16_le(&tail, offset + 6)?,
|
||||
disk_entry_count: read_u16_le(&tail, offset + 8)?,
|
||||
total_entry_count: read_u16_le(&tail, offset + 10)?,
|
||||
central_directory_size: read_u32_le(&tail, offset + 12)?,
|
||||
central_directory_offset: read_u32_le(&tail, offset + 16)?,
|
||||
});
|
||||
}
|
||||
|
||||
Err(format!("ZIP 文件缺少有效 EOCD 记录:{}", path.display()))
|
||||
}
|
||||
|
||||
fn read_central_directory_descriptor(
|
||||
file: &mut File,
|
||||
eocd: &EocdRecord,
|
||||
file_len: u64,
|
||||
path: &Path,
|
||||
) -> Result<CentralDirectoryDescriptor, String> {
|
||||
if eocd.disk_number != 0 || eocd.central_directory_disk != 0 {
|
||||
return Err(format!("不支持多卷 ZIP 文件:{}", path.display()));
|
||||
}
|
||||
|
||||
let requires_zip64 = eocd.disk_entry_count == u16::MAX
|
||||
|| eocd.total_entry_count == u16::MAX
|
||||
|| eocd.central_directory_size == u32::MAX
|
||||
|| eocd.central_directory_offset == u32::MAX;
|
||||
|
||||
if !requires_zip64 {
|
||||
if eocd.disk_entry_count != eocd.total_entry_count {
|
||||
return Err(format!(
|
||||
"ZIP EOCD 条目计数不一致 {}:disk={} total={}",
|
||||
path.display(),
|
||||
eocd.disk_entry_count,
|
||||
eocd.total_entry_count
|
||||
));
|
||||
}
|
||||
|
||||
return Ok(CentralDirectoryDescriptor {
|
||||
entry_count: eocd.total_entry_count as u64,
|
||||
size: eocd.central_directory_size as u64,
|
||||
offset: eocd.central_directory_offset as u64,
|
||||
suffix_offset: eocd.offset,
|
||||
uses_zip64: false,
|
||||
});
|
||||
}
|
||||
|
||||
if eocd.offset < ZIP64_EOCD_LOCATOR_LEN {
|
||||
return Err(format!("ZIP64 文件缺少 locator:{}", path.display()));
|
||||
}
|
||||
|
||||
let locator_offset = eocd.offset - ZIP64_EOCD_LOCATOR_LEN;
|
||||
let locator = read_exact_at(file, locator_offset, ZIP64_EOCD_LOCATOR_LEN as usize, path)?;
|
||||
if read_u32_le(&locator, 0)? != ZIP64_EOCD_LOCATOR_SIGNATURE {
|
||||
return Err(format!("ZIP64 文件缺少有效 locator:{}", path.display()));
|
||||
}
|
||||
|
||||
let eocd_disk = read_u32_le(&locator, 4)?;
|
||||
let zip64_eocd_offset = read_u64_le(&locator, 8)?;
|
||||
let total_disks = read_u32_le(&locator, 16)?;
|
||||
if eocd_disk != 0 || total_disks != 1 {
|
||||
return Err(format!("不支持多卷 ZIP64 文件:{}", path.display()));
|
||||
}
|
||||
|
||||
if zip64_eocd_offset >= file_len {
|
||||
return Err(format!(
|
||||
"ZIP64 EOCD 偏移越界 {}:offset={}",
|
||||
path.display(),
|
||||
zip64_eocd_offset
|
||||
));
|
||||
}
|
||||
|
||||
let record = read_exact_at(file, zip64_eocd_offset, ZIP64_EOCD_FIXED_LEN, path)?;
|
||||
if read_u32_le(&record, 0)? != ZIP64_EOCD_SIGNATURE {
|
||||
return Err(format!("ZIP64 EOCD 签名无效:{}", path.display()));
|
||||
}
|
||||
|
||||
let record_size = read_u64_le(&record, 4)?;
|
||||
if record_size < 44 {
|
||||
return Err(format!(
|
||||
"ZIP64 EOCD 长度无效 {}:{}",
|
||||
path.display(),
|
||||
record_size
|
||||
));
|
||||
}
|
||||
|
||||
let disk_number = read_u32_le(&record, 16)?;
|
||||
let central_directory_disk = read_u32_le(&record, 20)?;
|
||||
let disk_entry_count = read_u64_le(&record, 24)?;
|
||||
let total_entry_count = read_u64_le(&record, 32)?;
|
||||
if disk_number != 0 || central_directory_disk != 0 || disk_entry_count != total_entry_count {
|
||||
return Err(format!("ZIP64 EOCD 多卷或条目计数无效:{}", path.display()));
|
||||
}
|
||||
|
||||
let central_directory_size = read_u64_le(&record, 40)?;
|
||||
let central_directory_offset = read_u64_le(&record, 48)?;
|
||||
|
||||
Ok(CentralDirectoryDescriptor {
|
||||
entry_count: total_entry_count,
|
||||
size: central_directory_size,
|
||||
offset: central_directory_offset,
|
||||
suffix_offset: zip64_eocd_offset,
|
||||
uses_zip64: true,
|
||||
})
|
||||
}
|
||||
|
||||
fn validate_central_directory(
|
||||
file: &mut File,
|
||||
central_directory: &CentralDirectoryDescriptor,
|
||||
file_len: u64,
|
||||
path: &Path,
|
||||
) -> Result<(), String> {
|
||||
let central_end = checked_add(
|
||||
central_directory.offset,
|
||||
central_directory.size,
|
||||
"ZIP central directory 边界溢出",
|
||||
path,
|
||||
)?;
|
||||
if central_end > central_directory.suffix_offset || central_end > file_len {
|
||||
return Err(format!(
|
||||
"ZIP central directory 越界 {}:offset={} size={} file_size={}",
|
||||
path.display(),
|
||||
central_directory.offset,
|
||||
central_directory.size,
|
||||
file_len
|
||||
));
|
||||
}
|
||||
|
||||
let mut cursor = central_directory.offset;
|
||||
for index in 0..central_directory.entry_count {
|
||||
if checked_add(
|
||||
cursor,
|
||||
CENTRAL_DIRECTORY_FIXED_LEN,
|
||||
"ZIP central header 边界溢出",
|
||||
path,
|
||||
)? > central_end
|
||||
{
|
||||
return Err(format!(
|
||||
"ZIP central directory 条目不足 {}:index={}",
|
||||
path.display(),
|
||||
index
|
||||
));
|
||||
}
|
||||
|
||||
let header = read_exact_at(file, cursor, CENTRAL_DIRECTORY_FIXED_LEN as usize, path)?;
|
||||
if read_u32_le(&header, 0)? != CENTRAL_DIRECTORY_SIGNATURE {
|
||||
return Err(format!(
|
||||
"ZIP central header 签名无效 {}:offset={}",
|
||||
path.display(),
|
||||
cursor
|
||||
));
|
||||
}
|
||||
|
||||
let compressed_size_32 = read_u32_le(&header, 20)?;
|
||||
let uncompressed_size_32 = read_u32_le(&header, 24)?;
|
||||
let file_name_len = read_u16_le(&header, 28)? as u64;
|
||||
let extra_len = read_u16_le(&header, 30)? as u64;
|
||||
let comment_len = read_u16_le(&header, 32)? as u64;
|
||||
let disk_start_16 = read_u16_le(&header, 34)?;
|
||||
let local_header_offset_32 = read_u32_le(&header, 42)?;
|
||||
let entry_end = checked_add(
|
||||
checked_add(
|
||||
checked_add(
|
||||
cursor,
|
||||
CENTRAL_DIRECTORY_FIXED_LEN,
|
||||
"ZIP central header 文件名边界溢出",
|
||||
path,
|
||||
)?,
|
||||
file_name_len,
|
||||
"ZIP central header extra 边界溢出",
|
||||
path,
|
||||
)?,
|
||||
extra_len + comment_len,
|
||||
"ZIP central header comment 边界溢出",
|
||||
path,
|
||||
)?;
|
||||
if entry_end > central_end {
|
||||
return Err(format!(
|
||||
"ZIP central header 条目越界 {}:offset={}",
|
||||
path.display(),
|
||||
cursor
|
||||
));
|
||||
}
|
||||
if file_name_len == 0 {
|
||||
return Err(format!(
|
||||
"ZIP central header 文件名为空 {}:offset={}",
|
||||
path.display(),
|
||||
cursor
|
||||
));
|
||||
}
|
||||
|
||||
let name = read_exact_at(
|
||||
file,
|
||||
cursor + CENTRAL_DIRECTORY_FIXED_LEN,
|
||||
file_name_len as usize,
|
||||
path,
|
||||
)?;
|
||||
let extra = read_exact_at(
|
||||
file,
|
||||
cursor + CENTRAL_DIRECTORY_FIXED_LEN + file_name_len,
|
||||
extra_len as usize,
|
||||
path,
|
||||
)?;
|
||||
|
||||
let zip64 = parse_zip64_extra(
|
||||
&extra,
|
||||
uncompressed_size_32 == u32::MAX,
|
||||
compressed_size_32 == u32::MAX,
|
||||
local_header_offset_32 == u32::MAX,
|
||||
disk_start_16 == u16::MAX,
|
||||
path,
|
||||
)?;
|
||||
let compressed_size = zip64.compressed_size.unwrap_or(compressed_size_32 as u64);
|
||||
let local_header_offset = zip64
|
||||
.local_header_offset
|
||||
.unwrap_or(local_header_offset_32 as u64);
|
||||
let disk_start = zip64.disk_start.unwrap_or(disk_start_16 as u32);
|
||||
if disk_start != 0 {
|
||||
return Err(format!("不支持多卷 ZIP 条目:{}", path.display()));
|
||||
}
|
||||
|
||||
validate_local_file_header(
|
||||
file,
|
||||
path,
|
||||
&name,
|
||||
local_header_offset,
|
||||
compressed_size,
|
||||
central_directory.offset,
|
||||
)?;
|
||||
cursor = entry_end;
|
||||
}
|
||||
|
||||
if cursor != central_end {
|
||||
return Err(format!(
|
||||
"ZIP central directory 大小与条目不匹配 {}:parsed_end={} expected_end={}",
|
||||
path.display(),
|
||||
cursor,
|
||||
central_end
|
||||
));
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn validate_local_file_header(
|
||||
file: &mut File,
|
||||
path: &Path,
|
||||
central_name: &[u8],
|
||||
local_header_offset: u64,
|
||||
compressed_size: u64,
|
||||
central_directory_offset: u64,
|
||||
) -> Result<(), String> {
|
||||
if local_header_offset >= central_directory_offset {
|
||||
return Err(format!(
|
||||
"ZIP local header 偏移越界 {}:offset={}",
|
||||
path.display(),
|
||||
local_header_offset
|
||||
));
|
||||
}
|
||||
|
||||
let header = read_exact_at(
|
||||
file,
|
||||
local_header_offset,
|
||||
LOCAL_FILE_HEADER_FIXED_LEN as usize,
|
||||
path,
|
||||
)?;
|
||||
if read_u32_le(&header, 0)? != LOCAL_FILE_HEADER_SIGNATURE {
|
||||
return Err(format!(
|
||||
"ZIP local header 签名无效 {}:offset={}",
|
||||
path.display(),
|
||||
local_header_offset
|
||||
));
|
||||
}
|
||||
|
||||
let file_name_len = read_u16_le(&header, 26)? as u64;
|
||||
let extra_len = read_u16_le(&header, 28)? as u64;
|
||||
let data_start = checked_add(
|
||||
checked_add(
|
||||
checked_add(
|
||||
local_header_offset,
|
||||
LOCAL_FILE_HEADER_FIXED_LEN,
|
||||
"ZIP local header 文件名边界溢出",
|
||||
path,
|
||||
)?,
|
||||
file_name_len,
|
||||
"ZIP local header extra 边界溢出",
|
||||
path,
|
||||
)?,
|
||||
extra_len,
|
||||
"ZIP local data 边界溢出",
|
||||
path,
|
||||
)?;
|
||||
let data_end = checked_add(
|
||||
data_start,
|
||||
compressed_size,
|
||||
"ZIP local data 压缩数据边界溢出",
|
||||
path,
|
||||
)?;
|
||||
if data_end > central_directory_offset {
|
||||
return Err(format!(
|
||||
"ZIP local data 越界 {}:data_end={} central_directory_offset={}",
|
||||
path.display(),
|
||||
data_end,
|
||||
central_directory_offset
|
||||
));
|
||||
}
|
||||
|
||||
let local_name = read_exact_at(
|
||||
file,
|
||||
local_header_offset + LOCAL_FILE_HEADER_FIXED_LEN,
|
||||
file_name_len as usize,
|
||||
path,
|
||||
)?;
|
||||
if local_name != central_name {
|
||||
return Err(format!(
|
||||
"ZIP local header 文件名与 central directory 不一致 {}:offset={}",
|
||||
path.display(),
|
||||
local_header_offset
|
||||
));
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn parse_zip64_extra(
|
||||
extra: &[u8],
|
||||
need_uncompressed_size: bool,
|
||||
need_compressed_size: bool,
|
||||
need_local_header_offset: bool,
|
||||
need_disk_start: bool,
|
||||
path: &Path,
|
||||
) -> Result<Zip64ExtraValues, String> {
|
||||
let mut cursor = 0usize;
|
||||
while cursor + 4 <= extra.len() {
|
||||
let header_id = read_u16_le(extra, cursor)?;
|
||||
let data_size = read_u16_le(extra, cursor + 2)? as usize;
|
||||
cursor += 4;
|
||||
if cursor + data_size > extra.len() {
|
||||
return Err(format!("ZIP extra field 边界无效:{}", path.display()));
|
||||
}
|
||||
|
||||
if header_id == ZIP64_EXTENDED_INFORMATION_EXTRA_ID {
|
||||
let data = &extra[cursor..cursor + data_size];
|
||||
let values = parse_zip64_extended_information(
|
||||
data,
|
||||
need_uncompressed_size,
|
||||
need_compressed_size,
|
||||
need_local_header_offset,
|
||||
need_disk_start,
|
||||
path,
|
||||
)?;
|
||||
return Ok(values);
|
||||
}
|
||||
|
||||
cursor += data_size;
|
||||
}
|
||||
|
||||
if need_uncompressed_size || need_compressed_size || need_local_header_offset || need_disk_start
|
||||
{
|
||||
return Err(format!(
|
||||
"ZIP64 条目缺少 extended information extra:{}",
|
||||
path.display()
|
||||
));
|
||||
}
|
||||
|
||||
Ok(Zip64ExtraValues::default())
|
||||
}
|
||||
|
||||
fn parse_zip64_extended_information(
|
||||
data: &[u8],
|
||||
need_uncompressed_size: bool,
|
||||
need_compressed_size: bool,
|
||||
need_local_header_offset: bool,
|
||||
need_disk_start: bool,
|
||||
path: &Path,
|
||||
) -> Result<Zip64ExtraValues, String> {
|
||||
let mut cursor = 0usize;
|
||||
let mut values = Zip64ExtraValues::default();
|
||||
|
||||
if need_uncompressed_size {
|
||||
values.uncompressed_size = Some(read_zip64_u64(data, &mut cursor, path)?);
|
||||
}
|
||||
if need_compressed_size {
|
||||
values.compressed_size = Some(read_zip64_u64(data, &mut cursor, path)?);
|
||||
}
|
||||
if need_local_header_offset {
|
||||
values.local_header_offset = Some(read_zip64_u64(data, &mut cursor, path)?);
|
||||
}
|
||||
if need_disk_start {
|
||||
if cursor + 4 > data.len() {
|
||||
return Err(format!("ZIP64 extra 缺少 disk start:{}", path.display()));
|
||||
}
|
||||
values.disk_start = Some(read_u32_le(data, cursor)?);
|
||||
}
|
||||
|
||||
Ok(values)
|
||||
}
|
||||
|
||||
fn read_zip64_u64(data: &[u8], cursor: &mut usize, path: &Path) -> Result<u64, String> {
|
||||
if *cursor + 8 > data.len() {
|
||||
return Err(format!("ZIP64 extra 字段长度不足:{}", path.display()));
|
||||
}
|
||||
let value = read_u64_le(data, *cursor)?;
|
||||
*cursor += 8;
|
||||
Ok(value)
|
||||
}
|
||||
|
||||
fn read_exact_at(file: &mut File, offset: u64, len: usize, path: &Path) -> Result<Vec<u8>, String> {
|
||||
file.seek(SeekFrom::Start(offset)).map_err(|error| {
|
||||
format!(
|
||||
"定位 ZIP 文件失败 {}:offset={offset}:{error}",
|
||||
path.display()
|
||||
)
|
||||
})?;
|
||||
let mut data = vec![0u8; len];
|
||||
file.read_exact(&mut data).map_err(|error| {
|
||||
format!(
|
||||
"读取 ZIP 文件失败 {}:offset={offset} len={len}:{error}",
|
||||
path.display()
|
||||
)
|
||||
})?;
|
||||
Ok(data)
|
||||
}
|
||||
|
||||
fn checked_add(left: u64, right: u64, label: &str, path: &Path) -> Result<u64, String> {
|
||||
left.checked_add(right)
|
||||
.ok_or_else(|| format!("{label}:{}", path.display()))
|
||||
}
|
||||
|
||||
fn read_u16_le(bytes: &[u8], offset: usize) -> Result<u16, String> {
|
||||
let end = offset
|
||||
.checked_add(2)
|
||||
.ok_or_else(|| "读取 u16 偏移溢出".to_string())?;
|
||||
let slice = bytes
|
||||
.get(offset..end)
|
||||
.ok_or_else(|| "读取 u16 越界".to_string())?;
|
||||
Ok(u16::from_le_bytes([slice[0], slice[1]]))
|
||||
}
|
||||
|
||||
fn read_u32_le(bytes: &[u8], offset: usize) -> Result<u32, String> {
|
||||
let end = offset
|
||||
.checked_add(4)
|
||||
.ok_or_else(|| "读取 u32 偏移溢出".to_string())?;
|
||||
let slice = bytes
|
||||
.get(offset..end)
|
||||
.ok_or_else(|| "读取 u32 越界".to_string())?;
|
||||
Ok(u32::from_le_bytes([slice[0], slice[1], slice[2], slice[3]]))
|
||||
}
|
||||
|
||||
fn read_u64_le(bytes: &[u8], offset: usize) -> Result<u64, String> {
|
||||
let end = offset
|
||||
.checked_add(8)
|
||||
.ok_or_else(|| "读取 u64 偏移溢出".to_string())?;
|
||||
let slice = bytes
|
||||
.get(offset..end)
|
||||
.ok_or_else(|| "读取 u64 越界".to_string())?;
|
||||
Ok(u64::from_le_bytes([
|
||||
slice[0], slice[1], slice[2], slice[3], slice[4], slice[5], slice[6], slice[7],
|
||||
]))
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
struct EocdRecord {
|
||||
offset: u64,
|
||||
disk_number: u16,
|
||||
central_directory_disk: u16,
|
||||
disk_entry_count: u16,
|
||||
total_entry_count: u16,
|
||||
central_directory_size: u32,
|
||||
central_directory_offset: u32,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
struct CentralDirectoryDescriptor {
|
||||
entry_count: u64,
|
||||
size: u64,
|
||||
offset: u64,
|
||||
suffix_offset: u64,
|
||||
uses_zip64: bool,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
|
||||
struct Zip64ExtraValues {
|
||||
uncompressed_size: Option<u64>,
|
||||
compressed_size: Option<u64>,
|
||||
local_header_offset: Option<u64>,
|
||||
disk_start: Option<u32>,
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use tempfile::TempDir;
|
||||
|
||||
fn empty_zip() -> Vec<u8> {
|
||||
vec![
|
||||
0x50, 0x4b, 0x05, 0x06, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
]
|
||||
}
|
||||
|
||||
fn one_file_zip(name: &[u8], data: &[u8]) -> Vec<u8> {
|
||||
let mut bytes = Vec::new();
|
||||
bytes.extend_from_slice(&LOCAL_FILE_HEADER_SIGNATURE.to_le_bytes());
|
||||
bytes.extend_from_slice(&20u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u32.to_le_bytes());
|
||||
bytes.extend_from_slice(&(data.len() as u32).to_le_bytes());
|
||||
bytes.extend_from_slice(&(data.len() as u32).to_le_bytes());
|
||||
bytes.extend_from_slice(&(name.len() as u16).to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(name);
|
||||
bytes.extend_from_slice(data);
|
||||
|
||||
let central_offset = bytes.len() as u32;
|
||||
bytes.extend_from_slice(&CENTRAL_DIRECTORY_SIGNATURE.to_le_bytes());
|
||||
bytes.extend_from_slice(&20u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&20u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u32.to_le_bytes());
|
||||
bytes.extend_from_slice(&(data.len() as u32).to_le_bytes());
|
||||
bytes.extend_from_slice(&(data.len() as u32).to_le_bytes());
|
||||
bytes.extend_from_slice(&(name.len() as u16).to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u32.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u32.to_le_bytes());
|
||||
bytes.extend_from_slice(name);
|
||||
let central_size = bytes.len() as u32 - central_offset;
|
||||
|
||||
bytes.extend_from_slice(&EOCD_SIGNATURE.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&1u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&1u16.to_le_bytes());
|
||||
bytes.extend_from_slice(¢ral_size.to_le_bytes());
|
||||
bytes.extend_from_slice(¢ral_offset.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn accepts_empty_zip() {
|
||||
let temp = TempDir::new().unwrap();
|
||||
let path = temp.path().join("empty.zip");
|
||||
fs::write(&path, empty_zip()).unwrap();
|
||||
|
||||
let report = validate_zip_structure(&path).unwrap();
|
||||
|
||||
assert_eq!(report.entry_count, 0);
|
||||
assert!(!report.uses_zip64);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn accepts_one_file_zip() {
|
||||
let temp = TempDir::new().unwrap();
|
||||
let path = temp.path().join("one.zip");
|
||||
fs::write(&path, one_file_zip(b"nested/file.txt", b"hello")).unwrap();
|
||||
|
||||
let report = validate_zip_structure(&path).unwrap();
|
||||
|
||||
assert_eq!(report.entry_count, 1);
|
||||
assert!(!report.uses_zip64);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_truncated_zip() {
|
||||
let temp = TempDir::new().unwrap();
|
||||
let path = temp.path().join("bad.zip");
|
||||
let mut bytes = one_file_zip(b"file.txt", b"hello");
|
||||
bytes.truncate(bytes.len() - 8);
|
||||
fs::write(&path, bytes).unwrap();
|
||||
|
||||
let error = validate_zip_structure(&path).unwrap_err();
|
||||
|
||||
assert!(error.contains("EOCD") || error.contains("越界"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_local_name_mismatch() {
|
||||
let temp = TempDir::new().unwrap();
|
||||
let path = temp.path().join("bad-name.zip");
|
||||
let mut bytes = one_file_zip(b"file.txt", b"hello");
|
||||
let central_name_offset = 30 + b"file.txt".len() + b"hello".len() + 46;
|
||||
bytes[central_name_offset] = b'X';
|
||||
fs::write(&path, bytes).unwrap();
|
||||
|
||||
let error = validate_zip_structure(&path).unwrap_err();
|
||||
|
||||
assert!(error.contains("文件名"));
|
||||
}
|
||||
}
|
||||
@@ -257,7 +257,7 @@ impl TestHarness {
|
||||
fs::write(&resources_assets_path, &resources_assets).unwrap();
|
||||
|
||||
let zip_fixture = temp.path().join("downloaded.zip");
|
||||
fs::write(&zip_fixture, b"dummy zip content").unwrap();
|
||||
fs::write(&zip_fixture, one_file_zip(b"fixture.txt", b"ok")).unwrap();
|
||||
|
||||
let curl_log = temp.path().join("curl.log");
|
||||
let curl_script = temp.path().join("fake-curl");
|
||||
@@ -331,6 +331,54 @@ fn write_executable(path: &Path, content: &str) {
|
||||
fs::set_permissions(path, permissions).unwrap();
|
||||
}
|
||||
|
||||
fn one_file_zip(name: &[u8], data: &[u8]) -> Vec<u8> {
|
||||
let mut bytes = Vec::new();
|
||||
bytes.extend_from_slice(&0x0403_4b50u32.to_le_bytes());
|
||||
bytes.extend_from_slice(&20u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u32.to_le_bytes());
|
||||
bytes.extend_from_slice(&(data.len() as u32).to_le_bytes());
|
||||
bytes.extend_from_slice(&(data.len() as u32).to_le_bytes());
|
||||
bytes.extend_from_slice(&(name.len() as u16).to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(name);
|
||||
bytes.extend_from_slice(data);
|
||||
|
||||
let central_offset = bytes.len() as u32;
|
||||
bytes.extend_from_slice(&0x0201_4b50u32.to_le_bytes());
|
||||
bytes.extend_from_slice(&20u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&20u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u32.to_le_bytes());
|
||||
bytes.extend_from_slice(&(data.len() as u32).to_le_bytes());
|
||||
bytes.extend_from_slice(&(data.len() as u32).to_le_bytes());
|
||||
bytes.extend_from_slice(&(name.len() as u16).to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u32.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u32.to_le_bytes());
|
||||
bytes.extend_from_slice(name);
|
||||
let central_size = bytes.len() as u32 - central_offset;
|
||||
|
||||
bytes.extend_from_slice(&0x0605_4b50u32.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&1u16.to_le_bytes());
|
||||
bytes.extend_from_slice(&1u16.to_le_bytes());
|
||||
bytes.extend_from_slice(¢ral_size.to_le_bytes());
|
||||
bytes.extend_from_slice(¢ral_offset.to_le_bytes());
|
||||
bytes.extend_from_slice(&0u16.to_le_bytes());
|
||||
bytes
|
||||
}
|
||||
|
||||
fn official_curl_script(
|
||||
curl_log: &Path,
|
||||
zip_fixture: &Path,
|
||||
|
||||
Reference in New Issue
Block a user