mirror of
https://github.com/Yuyi-Oak/BlueArchiveToolkit.git
synced 2026-09-18 07:24:55 +08:00
fix(sqlite): 收口长期状态数据库版本化迁移契约
This commit is contained in:
@@ -46,6 +46,14 @@ TextUnit scope、source history 和 approved review。worker、TM 复用、人
|
|||||||
按当前 QA 精确校验 identity。`translation.glossary.*` 已通过
|
按当前 QA 精确校验 identity。`translation.glossary.*` 已通过
|
||||||
`bat.sock` 暴露,Go `bat-api` 仅做鉴权 typed forwarding。
|
`bat.sock` 暴露,Go `bat-api` 仅做鉴权 typed forwarding。
|
||||||
|
|
||||||
|
三个长期 SQLite owner 现在统一使用只读 schema preflight、精确 component
|
||||||
|
fingerprint 和 `BEGIN IMMEDIATE` writer transaction:Translation Tasks 从 V1
|
||||||
|
按显式 `v1 -> v2` step 迁移,当前版本为 V2;Translation Memory 和 Glossary
|
||||||
|
当前版本均为 V1。future、未知或版本与结构不一致的数据库在任何 schema/data
|
||||||
|
mutation 前 fail closed;migration 失败会 rollback,已知无
|
||||||
|
`schema_migrations` 表的历史 fingerprint 可安全补建版本表后重试。正式 schema
|
||||||
|
路径不再使用 `ensure_column` 隐式补列。
|
||||||
|
|
||||||
`localized.status` 现将 generic manifest schema/contract 与已发布 artifact integrity
|
`localized.status` 现将 generic manifest schema/contract 与已发布 artifact integrity
|
||||||
分开报告;current、state 和 identity 存在但文件被截断或手工修改时返回
|
分开报告;current、state 和 identity 存在但文件被截断或手工修改时返回
|
||||||
`localized.degraded`,只读检查不会自动回滚、删除或修复。双 release 的
|
`localized.degraded`,只读检查不会自动回滚、删除或修复。双 release 的
|
||||||
|
|||||||
@@ -236,7 +236,7 @@ T02 → T13
|
|||||||
# T03 — SQLite Schema Migration 体系化
|
# T03 — SQLite Schema Migration 体系化
|
||||||
|
|
||||||
**类型:** P2
|
**类型:** P2
|
||||||
**状态:** Ready
|
**状态:** Done
|
||||||
|
|
||||||
## 问题
|
## 问题
|
||||||
|
|
||||||
@@ -311,12 +311,23 @@ T03 → T15
|
|||||||
T03 → T16
|
T03 → T16
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## 完成记录
|
||||||
|
|
||||||
|
已为 Translation Tasks、Translation Memory 和 Glossary 收口统一的 SQLite
|
||||||
|
migration contract:现有数据库先以只读方式读取 schema、component version 和结构
|
||||||
|
fingerprint,future/unknown/mismatch 在任何写入前 fail closed;确认可迁移后在
|
||||||
|
`BEGIN IMMEDIATE` 事务内执行显式版本步骤,重读并校验结构后写入版本并提交。Translation
|
||||||
|
Tasks 按真实历史从 V1 迁移到 V2,Translation Memory 和 Glossary 保持真实的 V1
|
||||||
|
contract,没有凭空引入版本;已覆盖缺失 schema bookkeeping、业务数据保留、失败回滚
|
||||||
|
重试、并发打开和当前版本 reopen no-op。生产路径不再依赖 `ensure_column` 隐式升级,
|
||||||
|
相关状态、架构、RPC 参考和缺口文档已同步。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# T04 — Translation Memory Trusted 唯一性与 Supersede 治理
|
# T04 — Translation Memory Trusted 唯一性与 Supersede 治理
|
||||||
|
|
||||||
**类型:** P2
|
**类型:** P2
|
||||||
**状态:** Blocked by T03
|
**状态:** Ready
|
||||||
|
|
||||||
## 问题
|
## 问题
|
||||||
|
|
||||||
@@ -1047,16 +1058,7 @@ Stable Resource / Release / AssetBundle Platform
|
|||||||
|
|
||||||
## In Progress
|
## In Progress
|
||||||
|
|
||||||
建议当前只放:
|
当前无。
|
||||||
|
|
||||||
```text
|
|
||||||
T01 — bat-api 发布完整性与分发健康契约
|
|
||||||
T08 — CI Gate 与质量门禁整理
|
|
||||||
```
|
|
||||||
|
|
||||||
T01 是当前唯一 P1。
|
|
||||||
|
|
||||||
T08 可以与其并行,不涉及核心业务状态机。
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -1064,7 +1066,7 @@ T08 可以与其并行,不涉及核心业务状态机。
|
|||||||
|
|
||||||
```text
|
```text
|
||||||
T02 — ResourceRepository 查询契约统一
|
T02 — ResourceRepository 查询契约统一
|
||||||
T03 — SQLite Schema Migration 体系化
|
T04 — Translation Memory Trusted 唯一性与 Supersede 治理
|
||||||
T05 — bat.sock 本地 IPC 安全与资源边界
|
T05 — bat.sock 本地 IPC 安全与资源边界
|
||||||
T06 — AssetBundle / ZIP Parser Resource Budget
|
T06 — AssetBundle / ZIP Parser Resource Budget
|
||||||
T07 — Durable Atomic State Write
|
T07 — Durable Atomic State Write
|
||||||
@@ -1078,8 +1080,6 @@ T11 可以现在就开始采集,但 T01 后的数据才作为新的正式 dist
|
|||||||
## Blocked / Planned
|
## Blocked / Planned
|
||||||
|
|
||||||
```text
|
```text
|
||||||
T04 ← T03
|
|
||||||
|
|
||||||
T09 ← T06
|
T09 ← T06
|
||||||
T10 ← T09
|
T10 ← T09
|
||||||
|
|
||||||
|
|||||||
@@ -179,7 +179,7 @@
|
|||||||
`resource.index` RPC / CLI 只读查询现有 SQLite 索引;索引不存在时返回
|
`resource.index` RPC / CLI 只读查询现有 SQLite 索引;索引不存在时返回
|
||||||
`available=false`,不会因为查询创建空库。发布后的 TextUnit 队列还会在当前
|
`available=false`,不会因为查询创建空库。发布后的 TextUnit 队列还会在当前
|
||||||
release 根目录写入 `translation-tasks.sqlite`,由版本化 `schema_migrations`
|
release 根目录写入 `translation-tasks.sqlite`,由版本化 `schema_migrations`
|
||||||
管理 queued/running/failed/completed/skipped、provider run、lease、失败分类、
|
管理 V2 queued/running/failed/completed/skipped、provider run、lease、失败分类、
|
||||||
重试计划和 TextUnit 级译文结果。跨 release 的 Translation Memory V1 独立存储在
|
重试计划和 TextUnit 级译文结果。跨 release 的 Translation Memory V1 独立存储在
|
||||||
`<output>/translation-memory.sqlite`,记录 raw source/hash、完整 context、candidate/
|
`<output>/translation-memory.sqlite`,记录 raw source/hash、完整 context、candidate/
|
||||||
trusted 和 release/TextUnit/provider/run provenance;`translation.tasks` 优先查询这份状态库,
|
trusted 和 release/TextUnit/provider/run provenance;`translation.tasks` 优先查询这份状态库,
|
||||||
|
|||||||
@@ -165,18 +165,24 @@ count,任何一页不一致都会丢弃整个候选快照。
|
|||||||
- `translation-tasks.sqlite`:当前 release 的可变 worker 状态库,记录
|
- `translation-tasks.sqlite`:当前 release 的可变 worker 状态库,记录
|
||||||
queued / running / failed / completed / skipped、attempt count、provider run
|
queued / running / failed / completed / skipped、attempt count、provider run
|
||||||
ID、provider、TextUnit 级译文结果、lease、失败分类、可重试标记和
|
ID、provider、TextUnit 级译文结果、lease、失败分类、可重试标记和
|
||||||
next attempt;schema 由 `schema_migrations` 版本表管理。
|
next attempt;当前 schema version 为 V2,schema 由 `schema_migrations` 版本表
|
||||||
|
管理,并通过只读 fingerprint、`BEGIN IMMEDIATE` 和显式 V1 → V2 migration
|
||||||
|
保证 future/未知 schema fail closed。
|
||||||
- `translation-handoff.json`:当前 release 的版本化 job/unit/provider run 交接
|
- `translation-handoff.json`:当前 release 的版本化 job/unit/provider run 交接
|
||||||
快照;worker 更新后的实时状态仍以 `translation-tasks.sqlite` 为准。
|
快照;worker 更新后的实时状态仍以 `translation-tasks.sqlite` 为准。
|
||||||
- `translation-memory.sqlite`:跨 release 的项目级 Translation Memory,不位于
|
- `translation-memory.sqlite`:跨 release 的项目级 Translation Memory,不位于
|
||||||
`versions/<id>`,也不与 `translation-tasks.sqlite` 共用;记录 raw source/hash、完整
|
`versions/<id>`,也不与 `translation-tasks.sqlite` 共用;记录 raw source/hash、完整
|
||||||
TextUnit context、candidate/trusted、translation 和 release/TextUnit/provider/run
|
TextUnit context、candidate/trusted、translation 和 release/TextUnit/provider/run
|
||||||
provenance。默认路径为 `<output>/translation-memory.sqlite`,可由
|
provenance;当前 schema version 为 V1,打开时先进行只读 fingerprint preflight,
|
||||||
|
再在 writer transaction 内补齐 `schema_migrations` 版本记录。默认路径为
|
||||||
|
`<output>/translation-memory.sqlite`,可由
|
||||||
`BAT_TRANSLATION_MEMORY_PATH`、`[translation.worker].translation_memory_path` 或 CLI
|
`BAT_TRANSLATION_MEMORY_PATH`、`[translation.worker].translation_memory_path` 或 CLI
|
||||||
覆盖。
|
覆盖。
|
||||||
- `glossary.sqlite`:跨 release 的项目级 Glossary,不位于 `versions/<id>`,也不与
|
- `glossary.sqlite`:跨 release 的项目级 Glossary,不位于 `versions/<id>`,也不与
|
||||||
`translation-tasks.sqlite` 或 TM 共用;记录 term、alias、推荐/允许译法、scope、
|
`translation-tasks.sqlite` 或 TM 共用;记录 term、alias、推荐/允许译法、scope、
|
||||||
priority、review 状态、source provenance 和完整 source/review history。默认路径为
|
priority、review 状态、source provenance 和完整 source/review history;当前
|
||||||
|
schema version 为 V1,正式 schema evolution 不再通过 `ensure_column` 隐式修复。
|
||||||
|
默认路径为
|
||||||
`<output>/glossary.sqlite`,可由 `BAT_GLOSSARY_PATH`、`[translation.worker].glossary_path`
|
`<output>/glossary.sqlite`,可由 `BAT_GLOSSARY_PATH`、`[translation.worker].glossary_path`
|
||||||
或 CLI 覆盖。
|
或 CLI 覆盖。
|
||||||
|
|
||||||
|
|||||||
@@ -154,11 +154,26 @@ official/localized distribution 均被阻断,普通查询不会自动重建。
|
|||||||
|
|
||||||
### G-012:Translation Memory V1 已实现,扩展能力仍缺失
|
### G-012:Translation Memory V1 已实现,扩展能力仍缺失
|
||||||
|
|
||||||
Rust `bat` 已提供独立项目级 SQLite TM,记录 raw source/hash、完整 context、release/TextUnit/provider/run provenance,区分 candidate/trusted,只有显式 confirm 才能建立 trusted 记录;worker 只自动复用 trusted 的 raw source + 完整 context exact match,并在复用前执行已批准 Glossary 的确定性 QA。Go `bat-api` 已提供鉴权的 summary/query 只读接口和 confirm 转发,但 Go 不持有 TM 状态。仍缺少模糊匹配和更丰富的导入导出历史能力。
|
Rust `bat` 已提供独立项目级 SQLite TM,当前 schema version 为 V1;schema 打开遵守
|
||||||
|
只读 preflight、fingerprint、transaction rollback 和 future/unknown fail-closed
|
||||||
|
契约。它记录 raw source/hash、完整 context、release/TextUnit/provider/run provenance,
|
||||||
|
区分 candidate/trusted,只有显式 confirm 才能建立 trusted 记录;worker 只自动复用
|
||||||
|
trusted 的 raw source + 完整 context exact match,并在复用前执行已批准 Glossary 的
|
||||||
|
确定性 QA。Go `bat-api` 已提供鉴权的 summary/query 只读接口和 confirm 转发,但 Go
|
||||||
|
不持有 TM 状态。仍缺少模糊匹配和更丰富的导入导出历史能力。
|
||||||
|
|
||||||
### G-013:Glossary V1 已实现,协作视图仍缺失
|
### G-013:Glossary V1 已实现,协作视图仍缺失
|
||||||
|
|
||||||
Rust `bat` 已提供独立项目级 `glossary.sqlite`:term/alias/recommended/allowed/category/priority、全局与 TextUnit scope、source history、approved review、冲突诊断、provider-neutral constraints 和确定性 QA 均由 Rust 持有。trusted TM 复用会先经过 Glossary QA;provider、TM、人工 task/workbench 结果都记录 QA,blocking deviation 必须显式提交与当前 QA 精确绑定的 `qa_identity` 及 reviewer/reason/provenance。localized publish 会把发布时重算的 QA 写入 manifest。`translation.glossary.*` 已通过 `bat.sock` 暴露,Go 仅提供鉴权后的 typed forwarding。剩余缺口是完整 Web 术语协作视图和更丰富的导入/搜索能力。
|
Rust `bat` 已提供独立项目级 `glossary.sqlite`,当前 schema version 为 V1;打开
|
||||||
|
遵守只读 fingerprint preflight、writer transaction、rollback 和 future/unknown
|
||||||
|
fail-closed 契约,不再使用隐式 `ensure_column` 修复结构。term/alias/recommended/
|
||||||
|
allowed/category/priority、全局与 TextUnit scope、source history、approved review、
|
||||||
|
冲突诊断、provider-neutral constraints 和确定性 QA 均由 Rust 持有。trusted TM 复用会
|
||||||
|
先经过 Glossary QA;provider、TM、人工 task/workbench 结果都记录 QA,blocking
|
||||||
|
deviation 必须显式提交与当前 QA 精确绑定的 `qa_identity` 及 reviewer/reason/provenance。
|
||||||
|
localized publish 会把发布时重算的 QA 写入 manifest。`translation.glossary.*` 已通过
|
||||||
|
`bat.sock` 暴露,Go 仅提供鉴权后的 typed forwarding。剩余缺口是完整 Web 术语协作视图
|
||||||
|
和更丰富的导入/搜索能力。
|
||||||
|
|
||||||
### G-014:完整 Provider 扩展体系未实现
|
### G-014:完整 Provider 扩展体系未实现
|
||||||
|
|
||||||
|
|||||||
+675
-129
@@ -3,6 +3,9 @@
|
|||||||
use crate::path_security::{
|
use crate::path_security::{
|
||||||
ensure_safe_directory_path, lexical_absolute, set_file_mode, STATE_FILE_MODE,
|
ensure_safe_directory_path, lexical_absolute, set_file_mode, STATE_FILE_MODE,
|
||||||
};
|
};
|
||||||
|
use crate::sqlite_migration::{
|
||||||
|
self, ExpectedColumn, ExpectedIndex, ExpectedTable, SqliteSchemaSnapshot,
|
||||||
|
};
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
use bat_core::domain::{
|
use bat_core::domain::{
|
||||||
evaluate_glossary, validate_glossary_draft, GlossaryEvaluation, GlossaryHistoryRecord,
|
evaluate_glossary, validate_glossary_draft, GlossaryEvaluation, GlossaryHistoryRecord,
|
||||||
@@ -12,7 +15,7 @@ use bat_core::domain::{
|
|||||||
use bat_core::repositories::GlossaryRepository;
|
use bat_core::repositories::GlossaryRepository;
|
||||||
use bat_core::{Error, Result};
|
use bat_core::{Error, Result};
|
||||||
use serde::de::DeserializeOwned;
|
use serde::de::DeserializeOwned;
|
||||||
use sqlx::sqlite::{SqliteConnectOptions, SqliteJournalMode, SqlitePoolOptions};
|
use sqlx::sqlite::{SqliteConnectOptions, SqliteJournalMode};
|
||||||
use sqlx::{Row, SqlitePool};
|
use sqlx::{Row, SqlitePool};
|
||||||
use std::path::{Path, PathBuf};
|
use std::path::{Path, PathBuf};
|
||||||
use std::str::FromStr;
|
use std::str::FromStr;
|
||||||
@@ -76,15 +79,20 @@ impl SqliteGlossaryRepository {
|
|||||||
absolute.display()
|
absolute.display()
|
||||||
)));
|
)));
|
||||||
}
|
}
|
||||||
|
if metadata.len() > 0 {
|
||||||
|
let snapshot =
|
||||||
|
sqlite_migration::read_only_preflight(&absolute, GLOSSARY_SCHEMA_COMPONENT)
|
||||||
|
.await
|
||||||
|
.map_err(db_error)?;
|
||||||
|
classify_glossary_schema(&snapshot)?;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
let options = SqliteConnectOptions::from_str(&format!("sqlite://{}", absolute.display()))
|
let options = SqliteConnectOptions::from_str(&format!("sqlite://{}", absolute.display()))
|
||||||
.map_err(|error| Error::Other(error.into()))?
|
.map_err(|error| Error::Other(error.into()))?
|
||||||
.create_if_missing(create_if_missing)
|
.create_if_missing(create_if_missing)
|
||||||
.journal_mode(SqliteJournalMode::Wal)
|
.journal_mode(SqliteJournalMode::Wal)
|
||||||
.busy_timeout(Duration::from_secs(30));
|
.busy_timeout(Duration::from_secs(30));
|
||||||
let pool = SqlitePoolOptions::new()
|
let pool = sqlite_migration::connect_writable_pool(options)
|
||||||
.max_connections(1)
|
|
||||||
.connect_with(options)
|
|
||||||
.await
|
.await
|
||||||
.map_err(db_error)?;
|
.map_err(db_error)?;
|
||||||
if create_if_missing {
|
if create_if_missing {
|
||||||
@@ -97,119 +105,86 @@ impl SqliteGlossaryRepository {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async fn init_schema(&self) -> Result<()> {
|
async fn init_schema(&self) -> Result<()> {
|
||||||
sqlx::query(
|
self.init_schema_with_failure(None).await
|
||||||
"CREATE TABLE IF NOT EXISTS schema_migrations (
|
}
|
||||||
component TEXT PRIMARY KEY NOT NULL,
|
|
||||||
version INTEGER NOT NULL CHECK(version >= 1)
|
#[cfg(test)]
|
||||||
)",
|
async fn init_schema_with_test_failure(&self, fail_after_step: usize) -> Result<()> {
|
||||||
)
|
self.init_schema_with_failure(Some(fail_after_step)).await
|
||||||
.execute(&self.pool)
|
}
|
||||||
|
|
||||||
|
async fn init_schema_with_failure(&self, fail_after_step: Option<usize>) -> Result<()> {
|
||||||
|
let mut transaction = sqlite_migration::begin_immediate(&self.pool)
|
||||||
.await
|
.await
|
||||||
.map_err(db_error)?;
|
.map_err(db_error)?;
|
||||||
sqlx::query(
|
let result = self
|
||||||
"CREATE TABLE IF NOT EXISTS glossary_terms (
|
.migrate_in_transaction(&mut transaction, fail_after_step)
|
||||||
term_id TEXT PRIMARY KEY NOT NULL,
|
.await;
|
||||||
source_term TEXT NOT NULL,
|
match result {
|
||||||
aliases_json TEXT NOT NULL,
|
Ok(()) => transaction.commit().await.map_err(db_error),
|
||||||
recommended_translation TEXT NOT NULL,
|
Err(error) => {
|
||||||
allowed_translations_json TEXT NOT NULL,
|
let _ = transaction.rollback().await;
|
||||||
source_language TEXT,
|
Err(error)
|
||||||
target_language TEXT,
|
}
|
||||||
category TEXT,
|
}
|
||||||
priority INTEGER NOT NULL,
|
}
|
||||||
scope_json TEXT NOT NULL,
|
|
||||||
review_status TEXT NOT NULL,
|
async fn migrate_in_transaction(
|
||||||
source_kind TEXT NOT NULL,
|
&self,
|
||||||
source_ref TEXT,
|
transaction: &mut sqlx::Transaction<'_, sqlx::Sqlite>,
|
||||||
source_author TEXT,
|
fail_after_step: Option<usize>,
|
||||||
source_note TEXT,
|
) -> Result<()> {
|
||||||
source_observed_unix_seconds INTEGER NOT NULL,
|
let snapshot =
|
||||||
created_unix_seconds INTEGER NOT NULL,
|
sqlite_migration::snapshot_connection(transaction.as_mut(), GLOSSARY_SCHEMA_COMPONENT)
|
||||||
updated_unix_seconds INTEGER NOT NULL,
|
|
||||||
CHECK(length(term_id) > 0),
|
|
||||||
CHECK(length(source_term) > 0),
|
|
||||||
CHECK(length(recommended_translation) > 0),
|
|
||||||
CHECK(review_status IN ('draft', 'approved', 'deprecated', 'rejected')),
|
|
||||||
CHECK(source_kind IN ('manual', 'imported'))
|
|
||||||
)",
|
|
||||||
)
|
|
||||||
.execute(&self.pool)
|
|
||||||
.await
|
.await
|
||||||
.map_err(db_error)?;
|
.map_err(db_error)?;
|
||||||
ensure_column(
|
match classify_glossary_schema(&snapshot)? {
|
||||||
&self.pool,
|
GlossarySchemaState::Empty => {
|
||||||
"glossary_terms",
|
create_glossary_schema(transaction, fail_after_step).await?;
|
||||||
"source_observed_unix_seconds",
|
sqlite_migration::write_component_version(
|
||||||
"INTEGER NOT NULL DEFAULT 1",
|
transaction,
|
||||||
|
GLOSSARY_SCHEMA_COMPONENT,
|
||||||
|
GLOSSARY_SCHEMA_VERSION,
|
||||||
)
|
)
|
||||||
.await?;
|
|
||||||
sqlx::query(
|
|
||||||
"CREATE TABLE IF NOT EXISTS glossary_term_history (
|
|
||||||
history_id TEXT PRIMARY KEY NOT NULL,
|
|
||||||
term_id TEXT NOT NULL,
|
|
||||||
action TEXT NOT NULL,
|
|
||||||
reviewer TEXT,
|
|
||||||
reason TEXT,
|
|
||||||
source_json TEXT NOT NULL,
|
|
||||||
review_status TEXT NOT NULL,
|
|
||||||
snapshot_json TEXT NOT NULL,
|
|
||||||
observed_unix_seconds INTEGER NOT NULL,
|
|
||||||
FOREIGN KEY(term_id) REFERENCES glossary_terms(term_id)
|
|
||||||
)",
|
|
||||||
)
|
|
||||||
.execute(&self.pool)
|
|
||||||
.await
|
.await
|
||||||
.map_err(db_error)?;
|
.map_err(db_error)?;
|
||||||
sqlx::query(
|
}
|
||||||
"CREATE TABLE IF NOT EXISTS glossary_term_deletions (
|
GlossarySchemaState::Version(version) if version == GLOSSARY_SCHEMA_VERSION => {
|
||||||
deletion_id TEXT PRIMARY KEY NOT NULL,
|
if snapshot.component_version.is_none() {
|
||||||
term_id TEXT NOT NULL,
|
if !snapshot
|
||||||
reviewer TEXT NOT NULL,
|
.tables
|
||||||
reason TEXT NOT NULL,
|
.contains_key(sqlite_migration::SCHEMA_MIGRATIONS_TABLE)
|
||||||
source_json TEXT NOT NULL,
|
{
|
||||||
snapshot_json TEXT NOT NULL,
|
sqlite_migration::create_schema_migrations_table(transaction)
|
||||||
history_json TEXT NOT NULL,
|
.await
|
||||||
observed_unix_seconds INTEGER NOT NULL
|
.map_err(db_error)?;
|
||||||
)",
|
}
|
||||||
|
sqlite_migration::write_component_version(
|
||||||
|
transaction,
|
||||||
|
GLOSSARY_SCHEMA_COMPONENT,
|
||||||
|
GLOSSARY_SCHEMA_VERSION,
|
||||||
)
|
)
|
||||||
.execute(&self.pool)
|
|
||||||
.await
|
.await
|
||||||
.map_err(db_error)?;
|
.map_err(db_error)?;
|
||||||
sqlx::query(
|
}
|
||||||
"CREATE INDEX IF NOT EXISTS idx_glossary_status
|
}
|
||||||
ON glossary_terms(review_status, priority DESC, term_id)",
|
GlossarySchemaState::Version(version) => {
|
||||||
)
|
return Err(invalid_glossary_schema(format!(
|
||||||
.execute(&self.pool)
|
"内部不支持的迁移起点 {version}"
|
||||||
.await
|
|
||||||
.map_err(db_error)?;
|
|
||||||
sqlx::query(
|
|
||||||
"CREATE INDEX IF NOT EXISTS idx_glossary_source_term
|
|
||||||
ON glossary_terms(source_term)",
|
|
||||||
)
|
|
||||||
.execute(&self.pool)
|
|
||||||
.await
|
|
||||||
.map_err(db_error)?;
|
|
||||||
let current: Option<i64> =
|
|
||||||
sqlx::query_scalar("SELECT version FROM schema_migrations WHERE component = ?1")
|
|
||||||
.bind(GLOSSARY_SCHEMA_COMPONENT)
|
|
||||||
.fetch_optional(&self.pool)
|
|
||||||
.await
|
|
||||||
.map_err(db_error)?;
|
|
||||||
if current.is_some_and(|version| version > i64::from(GLOSSARY_SCHEMA_VERSION)) {
|
|
||||||
return Err(Error::InvalidArgument(format!(
|
|
||||||
"不支持的 Glossary schema 版本:{}",
|
|
||||||
current.unwrap_or_default()
|
|
||||||
)));
|
)));
|
||||||
}
|
}
|
||||||
sqlx::query(
|
}
|
||||||
"INSERT INTO schema_migrations(component, version) VALUES (?1, ?2)
|
let final_snapshot =
|
||||||
ON CONFLICT(component) DO UPDATE SET version = excluded.version",
|
sqlite_migration::snapshot_connection(transaction.as_mut(), GLOSSARY_SCHEMA_COMPONENT)
|
||||||
)
|
|
||||||
.bind(GLOSSARY_SCHEMA_COMPONENT)
|
|
||||||
.bind(i64::from(GLOSSARY_SCHEMA_VERSION))
|
|
||||||
.execute(&self.pool)
|
|
||||||
.await
|
.await
|
||||||
.map_err(db_error)?;
|
.map_err(db_error)?;
|
||||||
|
if final_snapshot.component_version != Some(i64::from(GLOSSARY_SCHEMA_VERSION))
|
||||||
|
|| !matches_glossary_fingerprint(&final_snapshot)
|
||||||
|
{
|
||||||
|
return Err(invalid_glossary_schema(
|
||||||
|
"migration 结果与当前 schema fingerprint 不一致".to_string(),
|
||||||
|
));
|
||||||
|
}
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -743,6 +718,450 @@ fn row_to_history(row: sqlx::sqlite::SqliteRow) -> Result<GlossaryHistoryRecord>
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
|
enum GlossarySchemaState {
|
||||||
|
Empty,
|
||||||
|
Version(u32),
|
||||||
|
}
|
||||||
|
|
||||||
|
fn classify_glossary_schema(snapshot: &SqliteSchemaSnapshot) -> Result<GlossarySchemaState> {
|
||||||
|
if snapshot.is_empty() {
|
||||||
|
return Ok(GlossarySchemaState::Empty);
|
||||||
|
}
|
||||||
|
if let Some(observed) = snapshot.component_version {
|
||||||
|
if observed > i64::from(GLOSSARY_SCHEMA_VERSION) {
|
||||||
|
return Err(invalid_glossary_schema(format!(
|
||||||
|
"不支持的 Glossary schema 版本:observed={observed}, supported={GLOSSARY_SCHEMA_VERSION}"
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
if observed < 1 {
|
||||||
|
return Err(invalid_glossary_schema(format!(
|
||||||
|
"schema version 无效:observed={observed}, supported=1..={GLOSSARY_SCHEMA_VERSION}"
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (matches_glossary_fingerprint(snapshot)
|
||||||
|
|| (snapshot.component_version.is_none()
|
||||||
|
&& matches_glossary_component_fingerprint(snapshot)))
|
||||||
|
&& snapshot
|
||||||
|
.component_version
|
||||||
|
.is_none_or(|observed| observed == i64::from(GLOSSARY_SCHEMA_VERSION))
|
||||||
|
{
|
||||||
|
return Ok(GlossarySchemaState::Version(GLOSSARY_SCHEMA_VERSION));
|
||||||
|
}
|
||||||
|
match snapshot.component_version {
|
||||||
|
Some(observed) => Err(invalid_glossary_schema(format!(
|
||||||
|
"schema version 与实际结构不一致:observed={observed}, supported={GLOSSARY_SCHEMA_VERSION}"
|
||||||
|
))),
|
||||||
|
None => Err(invalid_glossary_schema(
|
||||||
|
"未识别的 legacy schema,拒绝静默修复".to_string(),
|
||||||
|
)),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn invalid_glossary_schema(detail: String) -> Error {
|
||||||
|
Error::InvalidArgument(format!("Glossary schema 无效:{detail}"))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn matches_glossary_fingerprint(snapshot: &SqliteSchemaSnapshot) -> bool {
|
||||||
|
let tables = [
|
||||||
|
sqlite_migration::schema_migrations_table(),
|
||||||
|
ExpectedTable {
|
||||||
|
name: "glossary_terms",
|
||||||
|
columns: &GLOSSARY_TERM_COLUMNS,
|
||||||
|
},
|
||||||
|
ExpectedTable {
|
||||||
|
name: "glossary_term_history",
|
||||||
|
columns: &GLOSSARY_HISTORY_COLUMNS,
|
||||||
|
},
|
||||||
|
ExpectedTable {
|
||||||
|
name: "glossary_term_deletions",
|
||||||
|
columns: &GLOSSARY_DELETION_COLUMNS,
|
||||||
|
},
|
||||||
|
];
|
||||||
|
let indexes = [
|
||||||
|
ExpectedIndex {
|
||||||
|
table: "glossary_terms",
|
||||||
|
name: "idx_glossary_status",
|
||||||
|
columns: &["review_status", "priority", "term_id"],
|
||||||
|
},
|
||||||
|
ExpectedIndex {
|
||||||
|
table: "glossary_terms",
|
||||||
|
name: "idx_glossary_source_term",
|
||||||
|
columns: &["source_term"],
|
||||||
|
},
|
||||||
|
];
|
||||||
|
matches_glossary_fingerprint_with_tables(snapshot, &tables, &indexes)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn matches_glossary_component_fingerprint(snapshot: &SqliteSchemaSnapshot) -> bool {
|
||||||
|
let tables = [
|
||||||
|
ExpectedTable {
|
||||||
|
name: "glossary_terms",
|
||||||
|
columns: &GLOSSARY_TERM_COLUMNS,
|
||||||
|
},
|
||||||
|
ExpectedTable {
|
||||||
|
name: "glossary_term_history",
|
||||||
|
columns: &GLOSSARY_HISTORY_COLUMNS,
|
||||||
|
},
|
||||||
|
ExpectedTable {
|
||||||
|
name: "glossary_term_deletions",
|
||||||
|
columns: &GLOSSARY_DELETION_COLUMNS,
|
||||||
|
},
|
||||||
|
];
|
||||||
|
let indexes = [
|
||||||
|
ExpectedIndex {
|
||||||
|
table: "glossary_terms",
|
||||||
|
name: "idx_glossary_status",
|
||||||
|
columns: &["review_status", "priority", "term_id"],
|
||||||
|
},
|
||||||
|
ExpectedIndex {
|
||||||
|
table: "glossary_terms",
|
||||||
|
name: "idx_glossary_source_term",
|
||||||
|
columns: &["source_term"],
|
||||||
|
},
|
||||||
|
];
|
||||||
|
matches_glossary_fingerprint_with_tables(snapshot, &tables, &indexes)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn matches_glossary_fingerprint_with_tables(
|
||||||
|
snapshot: &SqliteSchemaSnapshot,
|
||||||
|
tables: &[ExpectedTable<'_>],
|
||||||
|
indexes: &[ExpectedIndex<'_>],
|
||||||
|
) -> bool {
|
||||||
|
sqlite_migration::matches_fingerprint(snapshot, tables, indexes)
|
||||||
|
}
|
||||||
|
|
||||||
|
const GLOSSARY_TERM_COLUMNS: [ExpectedColumn<'static>; 18] = [
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "term_id",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: true,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "source_term",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "aliases_json",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "recommended_translation",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "allowed_translations_json",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "source_language",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: false,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "target_language",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: false,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "category",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: false,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "priority",
|
||||||
|
data_type: "INTEGER",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "scope_json",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "review_status",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "source_kind",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "source_ref",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: false,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "source_author",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: false,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "source_note",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: false,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "source_observed_unix_seconds",
|
||||||
|
data_type: "INTEGER",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "created_unix_seconds",
|
||||||
|
data_type: "INTEGER",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "updated_unix_seconds",
|
||||||
|
data_type: "INTEGER",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
const GLOSSARY_HISTORY_COLUMNS: [ExpectedColumn<'static>; 9] = [
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "history_id",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: true,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "term_id",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "action",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "reviewer",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: false,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "reason",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: false,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "source_json",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "review_status",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "snapshot_json",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "observed_unix_seconds",
|
||||||
|
data_type: "INTEGER",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
const GLOSSARY_DELETION_COLUMNS: [ExpectedColumn<'static>; 8] = [
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "deletion_id",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: true,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "term_id",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "reviewer",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "reason",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "source_json",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "snapshot_json",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "history_json",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "observed_unix_seconds",
|
||||||
|
data_type: "INTEGER",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
async fn create_glossary_schema(
|
||||||
|
transaction: &mut sqlx::Transaction<'_, sqlx::Sqlite>,
|
||||||
|
fail_after_step: Option<usize>,
|
||||||
|
) -> Result<()> {
|
||||||
|
let steps = [
|
||||||
|
"CREATE TABLE schema_migrations (
|
||||||
|
component TEXT PRIMARY KEY NOT NULL,
|
||||||
|
version INTEGER NOT NULL CHECK(version >= 1)
|
||||||
|
)",
|
||||||
|
"CREATE TABLE glossary_terms (
|
||||||
|
term_id TEXT PRIMARY KEY NOT NULL,
|
||||||
|
source_term TEXT NOT NULL,
|
||||||
|
aliases_json TEXT NOT NULL,
|
||||||
|
recommended_translation TEXT NOT NULL,
|
||||||
|
allowed_translations_json TEXT NOT NULL,
|
||||||
|
source_language TEXT,
|
||||||
|
target_language TEXT,
|
||||||
|
category TEXT,
|
||||||
|
priority INTEGER NOT NULL,
|
||||||
|
scope_json TEXT NOT NULL,
|
||||||
|
review_status TEXT NOT NULL,
|
||||||
|
source_kind TEXT NOT NULL,
|
||||||
|
source_ref TEXT,
|
||||||
|
source_author TEXT,
|
||||||
|
source_note TEXT,
|
||||||
|
source_observed_unix_seconds INTEGER NOT NULL,
|
||||||
|
created_unix_seconds INTEGER NOT NULL,
|
||||||
|
updated_unix_seconds INTEGER NOT NULL,
|
||||||
|
CHECK(length(term_id) > 0),
|
||||||
|
CHECK(length(source_term) > 0),
|
||||||
|
CHECK(length(recommended_translation) > 0),
|
||||||
|
CHECK(review_status IN ('draft', 'approved', 'deprecated', 'rejected')),
|
||||||
|
CHECK(source_kind IN ('manual', 'imported'))
|
||||||
|
)",
|
||||||
|
"CREATE TABLE glossary_term_history (
|
||||||
|
history_id TEXT PRIMARY KEY NOT NULL,
|
||||||
|
term_id TEXT NOT NULL,
|
||||||
|
action TEXT NOT NULL,
|
||||||
|
reviewer TEXT,
|
||||||
|
reason TEXT,
|
||||||
|
source_json TEXT NOT NULL,
|
||||||
|
review_status TEXT NOT NULL,
|
||||||
|
snapshot_json TEXT NOT NULL,
|
||||||
|
observed_unix_seconds INTEGER NOT NULL,
|
||||||
|
FOREIGN KEY(term_id) REFERENCES glossary_terms(term_id)
|
||||||
|
)",
|
||||||
|
"CREATE TABLE glossary_term_deletions (
|
||||||
|
deletion_id TEXT PRIMARY KEY NOT NULL,
|
||||||
|
term_id TEXT NOT NULL,
|
||||||
|
reviewer TEXT NOT NULL,
|
||||||
|
reason TEXT NOT NULL,
|
||||||
|
source_json TEXT NOT NULL,
|
||||||
|
snapshot_json TEXT NOT NULL,
|
||||||
|
history_json TEXT NOT NULL,
|
||||||
|
observed_unix_seconds INTEGER NOT NULL
|
||||||
|
)",
|
||||||
|
"CREATE INDEX idx_glossary_status
|
||||||
|
ON glossary_terms(review_status, priority DESC, term_id)",
|
||||||
|
"CREATE INDEX idx_glossary_source_term
|
||||||
|
ON glossary_terms(source_term)",
|
||||||
|
];
|
||||||
|
for (index, statement) in steps.iter().enumerate() {
|
||||||
|
sqlx::query(statement)
|
||||||
|
.execute(&mut **transaction)
|
||||||
|
.await
|
||||||
|
.map_err(db_error)?;
|
||||||
|
if fail_after_step == Some(index + 1) {
|
||||||
|
return Err(Error::Other(anyhow::anyhow!(
|
||||||
|
"Glossary migration failed after step {}",
|
||||||
|
index + 1
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
fn parse_json<T: DeserializeOwned>(value: String) -> Result<T> {
|
fn parse_json<T: DeserializeOwned>(value: String) -> Result<T> {
|
||||||
serde_json::from_str(&value).map_err(|error| Error::Serialization(error.to_string()))
|
serde_json::from_str(&value).map_err(|error| Error::Serialization(error.to_string()))
|
||||||
}
|
}
|
||||||
@@ -769,35 +1188,10 @@ fn db_error(error: sqlx::Error) -> Error {
|
|||||||
Error::Other(error.into())
|
Error::Other(error.into())
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn ensure_column(
|
|
||||||
pool: &SqlitePool,
|
|
||||||
table: &str,
|
|
||||||
column: &str,
|
|
||||||
definition: &str,
|
|
||||||
) -> Result<()> {
|
|
||||||
let columns = sqlx::query(&format!("PRAGMA table_info({table})"))
|
|
||||||
.fetch_all(pool)
|
|
||||||
.await
|
|
||||||
.map_err(db_error)?;
|
|
||||||
let exists = columns.iter().any(|row| {
|
|
||||||
row.try_get::<String, _>("name")
|
|
||||||
.map(|name| name == column)
|
|
||||||
.unwrap_or(false)
|
|
||||||
});
|
|
||||||
if !exists {
|
|
||||||
sqlx::query(&format!(
|
|
||||||
"ALTER TABLE {table} ADD COLUMN {column} {definition}"
|
|
||||||
))
|
|
||||||
.execute(pool)
|
|
||||||
.await
|
|
||||||
.map_err(db_error)?;
|
|
||||||
}
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
use sqlx::sqlite::SqlitePoolOptions;
|
||||||
use std::collections::BTreeMap;
|
use std::collections::BTreeMap;
|
||||||
|
|
||||||
fn draft(status: GlossaryReviewStatus) -> GlossaryTermDraft {
|
fn draft(status: GlossaryReviewStatus) -> GlossaryTermDraft {
|
||||||
@@ -825,6 +1219,158 @@ mod tests {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async fn raw_repository(
|
||||||
|
path: &std::path::Path,
|
||||||
|
create_if_missing: bool,
|
||||||
|
) -> SqliteGlossaryRepository {
|
||||||
|
let options = SqliteConnectOptions::from_str(&format!("sqlite://{}", path.display()))
|
||||||
|
.unwrap()
|
||||||
|
.create_if_missing(create_if_missing);
|
||||||
|
let pool = SqlitePoolOptions::new()
|
||||||
|
.max_connections(1)
|
||||||
|
.connect_with(options)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
SqliteGlossaryRepository { pool }
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn sqlite_glossary_preserves_data_across_reopen_and_missing_row() {
|
||||||
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
let path = temp.path().join(GLOSSARY_REPOSITORY_FILE);
|
||||||
|
let repository = SqliteGlossaryRepository::new(&path).await.unwrap();
|
||||||
|
repository
|
||||||
|
.add(draft(GlossaryReviewStatus::Draft))
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
repository
|
||||||
|
.review(
|
||||||
|
"term-sensei",
|
||||||
|
GlossaryReviewStatus::Approved,
|
||||||
|
"reviewer",
|
||||||
|
Some("accepted".to_string()),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
sqlx::query("DROP TABLE schema_migrations")
|
||||||
|
.execute(&repository.pool)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
repository.pool.close().await;
|
||||||
|
|
||||||
|
let reopened = SqliteGlossaryRepository::open(&path).await.unwrap();
|
||||||
|
let term = reopened.find("term-sensei").await.unwrap();
|
||||||
|
assert_eq!(term.definition.recommended_translation, "老师");
|
||||||
|
assert_eq!(term.history.len(), 2);
|
||||||
|
assert_eq!(term.history[1].action, "approved");
|
||||||
|
assert_eq!(
|
||||||
|
reopened.summary().await.unwrap().schema_version,
|
||||||
|
GLOSSARY_SCHEMA_VERSION
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn sqlite_glossary_future_schema_is_read_only_failure() {
|
||||||
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
let path = temp.path().join(GLOSSARY_REPOSITORY_FILE);
|
||||||
|
let repository = SqliteGlossaryRepository::new(&path).await.unwrap();
|
||||||
|
sqlx::query("UPDATE schema_migrations SET version = ?2 WHERE component = ?1")
|
||||||
|
.bind(GLOSSARY_SCHEMA_COMPONENT)
|
||||||
|
.bind(i64::from(GLOSSARY_SCHEMA_VERSION) + 1)
|
||||||
|
.execute(&repository.pool)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
repository.pool.close().await;
|
||||||
|
let before = std::fs::read(&path).unwrap();
|
||||||
|
let wal_path = std::path::PathBuf::from(format!("{}-wal", path.display()));
|
||||||
|
let shm_path = std::path::PathBuf::from(format!("{}-shm", path.display()));
|
||||||
|
let wal_before = std::fs::read(&wal_path).ok();
|
||||||
|
let shm_before = std::fs::read(&shm_path).ok();
|
||||||
|
let error = SqliteGlossaryRepository::new(&path).await.unwrap_err();
|
||||||
|
assert!(error.to_string().contains("不支持的 Glossary schema"));
|
||||||
|
assert_eq!(std::fs::read(&path).unwrap(), before);
|
||||||
|
assert_eq!(std::fs::read(&wal_path).ok(), wal_before);
|
||||||
|
assert_eq!(std::fs::read(&shm_path).ok(), shm_before);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn sqlite_glossary_unknown_schema_fails_closed() {
|
||||||
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
let path = temp.path().join(GLOSSARY_REPOSITORY_FILE);
|
||||||
|
let repository = raw_repository(&path, true).await;
|
||||||
|
sqlx::query(
|
||||||
|
"CREATE TABLE schema_migrations (
|
||||||
|
component TEXT PRIMARY KEY NOT NULL,
|
||||||
|
version INTEGER NOT NULL CHECK(version >= 1)
|
||||||
|
)",
|
||||||
|
)
|
||||||
|
.execute(&repository.pool)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
sqlx::query("INSERT INTO schema_migrations(component, version) VALUES (?1, 1)")
|
||||||
|
.bind(GLOSSARY_SCHEMA_COMPONENT)
|
||||||
|
.execute(&repository.pool)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
sqlx::query("CREATE TABLE glossary_terms (term_id TEXT PRIMARY KEY)")
|
||||||
|
.execute(&repository.pool)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
repository.pool.close().await;
|
||||||
|
let before = std::fs::read(&path).unwrap();
|
||||||
|
let error = SqliteGlossaryRepository::open(&path).await.unwrap_err();
|
||||||
|
assert!(error
|
||||||
|
.to_string()
|
||||||
|
.contains("schema version 与实际结构不一致"));
|
||||||
|
assert_eq!(std::fs::read(&path).unwrap(), before);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn sqlite_glossary_failed_new_schema_rolls_back_and_retries() {
|
||||||
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
let path = temp.path().join(GLOSSARY_REPOSITORY_FILE);
|
||||||
|
let repository = raw_repository(&path, true).await;
|
||||||
|
assert!(repository.init_schema_with_test_failure(3).await.is_err());
|
||||||
|
let table_count: i64 =
|
||||||
|
sqlx::query_scalar("SELECT COUNT(*) FROM sqlite_master WHERE type = 'table'")
|
||||||
|
.fetch_one(&repository.pool)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(table_count, 0);
|
||||||
|
repository.init_schema().await.unwrap();
|
||||||
|
let version: i64 =
|
||||||
|
sqlx::query_scalar("SELECT version FROM schema_migrations WHERE component = ?1")
|
||||||
|
.bind(GLOSSARY_SCHEMA_COMPONENT)
|
||||||
|
.fetch_one(&repository.pool)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(version, 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn sqlite_glossary_concurrent_new_open_has_one_current_schema() {
|
||||||
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
let path = temp.path().join(GLOSSARY_REPOSITORY_FILE);
|
||||||
|
let (left, right) = tokio::join!(
|
||||||
|
SqliteGlossaryRepository::new(&path),
|
||||||
|
SqliteGlossaryRepository::new(&path)
|
||||||
|
);
|
||||||
|
assert!(left.is_ok(), "left open failed: {left:?}");
|
||||||
|
assert!(right.is_ok(), "right open failed: {right:?}");
|
||||||
|
let repository = left.unwrap();
|
||||||
|
drop(right);
|
||||||
|
assert_eq!(
|
||||||
|
repository.summary().await.unwrap().schema_version,
|
||||||
|
GLOSSARY_SCHEMA_VERSION
|
||||||
|
);
|
||||||
|
repository.pool.close().await;
|
||||||
|
let reopened = SqliteGlossaryRepository::open(&path).await.unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
reopened.summary().await.unwrap().schema_version,
|
||||||
|
GLOSSARY_SCHEMA_VERSION
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
#[tokio::test]
|
#[tokio::test]
|
||||||
async fn sqlite_glossary_preserves_history_and_only_approved_terms_match() {
|
async fn sqlite_glossary_preserves_history_and_only_approved_terms_match() {
|
||||||
let temp = tempfile::TempDir::new().unwrap();
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
|||||||
@@ -31,6 +31,7 @@ pub mod path_security;
|
|||||||
pub mod release_flow;
|
pub mod release_flow;
|
||||||
pub mod release_ops;
|
pub mod release_ops;
|
||||||
pub mod resources;
|
pub mod resources;
|
||||||
|
mod sqlite_migration;
|
||||||
pub mod translation_memory;
|
pub mod translation_memory;
|
||||||
pub mod translation_tasks;
|
pub mod translation_tasks;
|
||||||
pub mod translation_worker;
|
pub mod translation_worker;
|
||||||
|
|||||||
@@ -0,0 +1,377 @@
|
|||||||
|
//! Shared, deliberately small SQLite schema-migration primitives.
|
||||||
|
//!
|
||||||
|
//! Component owners still define their own schema fingerprints and migration
|
||||||
|
//! steps. This module only owns the read-only preflight, schema snapshot, and
|
||||||
|
//! writer-lock mechanics shared by the long-lived SQLite stores.
|
||||||
|
|
||||||
|
use sqlx::sqlite::{SqliteConnectOptions, SqlitePoolOptions};
|
||||||
|
use sqlx::{Row, SqliteConnection, SqlitePool};
|
||||||
|
use std::collections::{BTreeMap, BTreeSet};
|
||||||
|
use std::path::Path;
|
||||||
|
use std::str::FromStr;
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
|
pub const SCHEMA_MIGRATIONS_TABLE: &str = "schema_migrations";
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub struct SqliteColumn {
|
||||||
|
pub data_type: String,
|
||||||
|
pub not_null: bool,
|
||||||
|
pub default_value: Option<String>,
|
||||||
|
pub primary_key: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub struct SqliteSchemaSnapshot {
|
||||||
|
/// Non-internal SQLite objects, including tables, indexes, views, and
|
||||||
|
/// triggers. Internal `sqlite_autoindex_*` objects are omitted.
|
||||||
|
pub objects: BTreeSet<(String, String)>,
|
||||||
|
pub tables: BTreeMap<String, BTreeMap<String, SqliteColumn>>,
|
||||||
|
pub indexes: BTreeMap<String, BTreeMap<String, Vec<String>>>,
|
||||||
|
pub component_version: Option<i64>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl SqliteSchemaSnapshot {
|
||||||
|
pub fn is_empty(&self) -> bool {
|
||||||
|
self.objects.is_empty()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Copy)]
|
||||||
|
pub struct ExpectedColumn<'a> {
|
||||||
|
pub name: &'a str,
|
||||||
|
pub data_type: &'a str,
|
||||||
|
pub not_null: bool,
|
||||||
|
pub default_value: Option<&'a str>,
|
||||||
|
pub primary_key: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Copy)]
|
||||||
|
pub struct ExpectedTable<'a> {
|
||||||
|
pub name: &'a str,
|
||||||
|
pub columns: &'a [ExpectedColumn<'a>],
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Copy)]
|
||||||
|
pub struct ExpectedIndex<'a> {
|
||||||
|
pub table: &'a str,
|
||||||
|
pub name: &'a str,
|
||||||
|
pub columns: &'a [&'a str],
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Opens an existing database with SQLite's read-only flag and snapshots its
|
||||||
|
/// schema before a writable connection can perform any mutation.
|
||||||
|
pub async fn read_only_preflight(
|
||||||
|
path: &Path,
|
||||||
|
component: &str,
|
||||||
|
) -> Result<SqliteSchemaSnapshot, sqlx::Error> {
|
||||||
|
let has_wal_sidecar =
|
||||||
|
sidecar_path(path, "-wal").exists() || sidecar_path(path, "-shm").exists();
|
||||||
|
let options = SqliteConnectOptions::from_str(&format!("sqlite://{}", path.display()))?
|
||||||
|
.read_only(true)
|
||||||
|
.create_if_missing(false)
|
||||||
|
// A cleanly closed WAL database has all committed pages in the main
|
||||||
|
// file. Immutable read-only mode prevents SQLite from creating a new
|
||||||
|
// `-shm` sidecar during future-schema rejection. Live WAL sidecars
|
||||||
|
// must remain visible to the preflight reader.
|
||||||
|
.immutable(!has_wal_sidecar)
|
||||||
|
.busy_timeout(Duration::from_secs(30));
|
||||||
|
let pool = SqlitePoolOptions::new()
|
||||||
|
.max_connections(1)
|
||||||
|
.connect_with(options)
|
||||||
|
.await?;
|
||||||
|
let snapshot = {
|
||||||
|
let mut connection = pool.acquire().await?;
|
||||||
|
snapshot_connection(&mut connection, component).await
|
||||||
|
};
|
||||||
|
pool.close().await;
|
||||||
|
snapshot
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Connects a writable single-connection pool, retrying the SQLite-specific
|
||||||
|
/// exclusive lock needed when a connection switches an existing database to
|
||||||
|
/// WAL mode. SQLite's busy timeout cannot wait for that PRAGMA, so the retry
|
||||||
|
/// belongs around connection establishment rather than only around writes.
|
||||||
|
pub async fn connect_writable_pool(
|
||||||
|
options: SqliteConnectOptions,
|
||||||
|
) -> Result<SqlitePool, sqlx::Error> {
|
||||||
|
const MAX_ATTEMPTS: usize = 32;
|
||||||
|
|
||||||
|
for attempt in 0..=MAX_ATTEMPTS {
|
||||||
|
match SqlitePoolOptions::new()
|
||||||
|
.max_connections(1)
|
||||||
|
.connect_with(options.clone())
|
||||||
|
.await
|
||||||
|
{
|
||||||
|
Ok(pool) => return Ok(pool),
|
||||||
|
Err(error) if attempt < MAX_ATTEMPTS && is_sqlite_lock_error(&error) => {
|
||||||
|
let delay_millis = (25 * (attempt as u64 + 1)).min(250);
|
||||||
|
tokio::time::sleep(Duration::from_millis(delay_millis)).await;
|
||||||
|
}
|
||||||
|
Err(error) => return Err(error),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
unreachable!("SQLite connection retry loop always returns")
|
||||||
|
}
|
||||||
|
|
||||||
|
fn is_sqlite_lock_error(error: &sqlx::Error) -> bool {
|
||||||
|
error.to_string().contains("database is locked")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Snapshots the schema using an already-open connection. The caller may use
|
||||||
|
/// this both for read-only preflight and inside the migration transaction.
|
||||||
|
pub async fn snapshot_connection(
|
||||||
|
connection: &mut SqliteConnection,
|
||||||
|
component: &str,
|
||||||
|
) -> Result<SqliteSchemaSnapshot, sqlx::Error> {
|
||||||
|
let object_rows = sqlx::query(
|
||||||
|
"SELECT type, name FROM sqlite_master
|
||||||
|
WHERE name NOT LIKE 'sqlite_%'
|
||||||
|
ORDER BY type, name",
|
||||||
|
)
|
||||||
|
.fetch_all(&mut *connection)
|
||||||
|
.await?;
|
||||||
|
|
||||||
|
let mut objects = BTreeSet::new();
|
||||||
|
let mut table_names = BTreeSet::new();
|
||||||
|
for row in object_rows {
|
||||||
|
let object_type: String = row.try_get("type")?;
|
||||||
|
let name: String = row.try_get("name")?;
|
||||||
|
if object_type == "table" {
|
||||||
|
table_names.insert(name.clone());
|
||||||
|
}
|
||||||
|
objects.insert((object_type, name));
|
||||||
|
}
|
||||||
|
|
||||||
|
let mut tables = BTreeMap::new();
|
||||||
|
let mut indexes = BTreeMap::new();
|
||||||
|
for table in table_names {
|
||||||
|
let quoted_table = quote_identifier(&table);
|
||||||
|
let column_rows = sqlx::query(&format!("PRAGMA table_info({quoted_table})"))
|
||||||
|
.fetch_all(&mut *connection)
|
||||||
|
.await?;
|
||||||
|
let mut columns = BTreeMap::new();
|
||||||
|
for row in column_rows {
|
||||||
|
let name: String = row.try_get("name")?;
|
||||||
|
let data_type: String = row.try_get("type")?;
|
||||||
|
let not_null: i64 = row.try_get("notnull")?;
|
||||||
|
let default_value: Option<String> = row.try_get("dflt_value")?;
|
||||||
|
let primary_key: i64 = row.try_get("pk")?;
|
||||||
|
columns.insert(
|
||||||
|
name,
|
||||||
|
SqliteColumn {
|
||||||
|
data_type,
|
||||||
|
not_null: not_null != 0,
|
||||||
|
default_value,
|
||||||
|
primary_key: primary_key != 0,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
}
|
||||||
|
tables.insert(table.clone(), columns);
|
||||||
|
|
||||||
|
let index_rows = sqlx::query(&format!("PRAGMA index_list({quoted_table})"))
|
||||||
|
.fetch_all(&mut *connection)
|
||||||
|
.await?;
|
||||||
|
let mut table_indexes = BTreeMap::new();
|
||||||
|
for row in index_rows {
|
||||||
|
let index_name: String = row.try_get("name")?;
|
||||||
|
if index_name.starts_with("sqlite_autoindex_") {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let quoted_index = quote_identifier(&index_name);
|
||||||
|
let index_columns = sqlx::query(&format!("PRAGMA index_info({quoted_index})"))
|
||||||
|
.fetch_all(&mut *connection)
|
||||||
|
.await?;
|
||||||
|
let mut columns = Vec::new();
|
||||||
|
for index_column in index_columns {
|
||||||
|
let sequence: i64 = index_column.try_get("seqno")?;
|
||||||
|
let name: Option<String> = index_column.try_get("name")?;
|
||||||
|
if sequence < 0 {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let name = name.ok_or_else(|| {
|
||||||
|
sqlx::Error::Protocol(format!(
|
||||||
|
"SQLite index {index_name} has an unnamed column"
|
||||||
|
))
|
||||||
|
})?;
|
||||||
|
columns.push((sequence, name));
|
||||||
|
}
|
||||||
|
columns.sort_by_key(|(sequence, _)| *sequence);
|
||||||
|
table_indexes.insert(
|
||||||
|
index_name,
|
||||||
|
columns
|
||||||
|
.into_iter()
|
||||||
|
.map(|(_, name)| name)
|
||||||
|
.collect::<Vec<_>>(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if !table_indexes.is_empty() {
|
||||||
|
indexes.insert(table, table_indexes);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let component_version = if tables.contains_key(SCHEMA_MIGRATIONS_TABLE) {
|
||||||
|
sqlx::query_scalar("SELECT version FROM schema_migrations WHERE component = ?1")
|
||||||
|
.bind(component)
|
||||||
|
.fetch_optional(&mut *connection)
|
||||||
|
.await?
|
||||||
|
} else {
|
||||||
|
None
|
||||||
|
};
|
||||||
|
|
||||||
|
Ok(SqliteSchemaSnapshot {
|
||||||
|
objects,
|
||||||
|
tables,
|
||||||
|
indexes,
|
||||||
|
component_version,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Starts a real SQLite writer transaction. `BEGIN IMMEDIATE` serializes DDL
|
||||||
|
/// migration writers instead of allowing two preflight results to race.
|
||||||
|
pub async fn begin_immediate(
|
||||||
|
pool: &SqlitePool,
|
||||||
|
) -> Result<sqlx::Transaction<'static, sqlx::Sqlite>, sqlx::Error> {
|
||||||
|
pool.begin_with("BEGIN IMMEDIATE").await
|
||||||
|
}
|
||||||
|
|
||||||
|
pub async fn write_component_version(
|
||||||
|
transaction: &mut sqlx::Transaction<'_, sqlx::Sqlite>,
|
||||||
|
component: &str,
|
||||||
|
version: u32,
|
||||||
|
) -> Result<(), sqlx::Error> {
|
||||||
|
sqlx::query(
|
||||||
|
"INSERT INTO schema_migrations(component, version) VALUES (?1, ?2)
|
||||||
|
ON CONFLICT(component) DO UPDATE SET version = excluded.version",
|
||||||
|
)
|
||||||
|
.bind(component)
|
||||||
|
.bind(i64::from(version))
|
||||||
|
.execute(&mut **transaction)
|
||||||
|
.await?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
pub async fn create_schema_migrations_table(
|
||||||
|
transaction: &mut sqlx::Transaction<'_, sqlx::Sqlite>,
|
||||||
|
) -> Result<(), sqlx::Error> {
|
||||||
|
sqlx::query(
|
||||||
|
"CREATE TABLE IF NOT EXISTS schema_migrations (
|
||||||
|
component TEXT PRIMARY KEY NOT NULL,
|
||||||
|
version INTEGER NOT NULL CHECK(version >= 1)
|
||||||
|
)",
|
||||||
|
)
|
||||||
|
.execute(&mut **transaction)
|
||||||
|
.await?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn matches_fingerprint(
|
||||||
|
snapshot: &SqliteSchemaSnapshot,
|
||||||
|
expected_tables: &[ExpectedTable<'_>],
|
||||||
|
expected_indexes: &[ExpectedIndex<'_>],
|
||||||
|
) -> bool {
|
||||||
|
let expected_table_names = expected_tables
|
||||||
|
.iter()
|
||||||
|
.map(|table| table.name)
|
||||||
|
.collect::<BTreeSet<_>>();
|
||||||
|
let actual_table_names: BTreeSet<&str> = snapshot.tables.keys().map(String::as_str).collect();
|
||||||
|
if actual_table_names != expected_table_names {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
let expected_object_names = expected_tables
|
||||||
|
.iter()
|
||||||
|
.map(|table| ("table", table.name))
|
||||||
|
.chain(expected_indexes.iter().map(|index| ("index", index.name)))
|
||||||
|
.collect::<BTreeSet<_>>();
|
||||||
|
let actual_object_names = snapshot
|
||||||
|
.objects
|
||||||
|
.iter()
|
||||||
|
.map(|(object_type, name)| (object_type.as_str(), name.as_str()))
|
||||||
|
.collect::<BTreeSet<_>>();
|
||||||
|
if actual_object_names != expected_object_names {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
for table in expected_tables {
|
||||||
|
let Some(actual_columns) = snapshot.tables.get(table.name) else {
|
||||||
|
return false;
|
||||||
|
};
|
||||||
|
if actual_columns.len() != table.columns.len() {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
for expected in table.columns {
|
||||||
|
let Some(actual) = actual_columns.get(expected.name) else {
|
||||||
|
return false;
|
||||||
|
};
|
||||||
|
if actual.data_type.to_ascii_uppercase() != expected.data_type
|
||||||
|
|| actual.not_null != expected.not_null
|
||||||
|
|| actual.primary_key != expected.primary_key
|
||||||
|
|| normalize_default(actual.default_value.as_deref())
|
||||||
|
!= normalize_default(expected.default_value)
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let expected_indexes = expected_indexes
|
||||||
|
.iter()
|
||||||
|
.map(|index| {
|
||||||
|
(
|
||||||
|
index.table.to_string(),
|
||||||
|
index.name.to_string(),
|
||||||
|
index
|
||||||
|
.columns
|
||||||
|
.iter()
|
||||||
|
.map(|column| (*column).to_string())
|
||||||
|
.collect::<Vec<_>>(),
|
||||||
|
)
|
||||||
|
})
|
||||||
|
.collect::<BTreeSet<_>>();
|
||||||
|
let actual_indexes = snapshot
|
||||||
|
.indexes
|
||||||
|
.iter()
|
||||||
|
.flat_map(|(table, indexes)| {
|
||||||
|
indexes
|
||||||
|
.iter()
|
||||||
|
.map(|(name, columns)| (table.clone(), name.clone(), columns.clone()))
|
||||||
|
})
|
||||||
|
.collect::<BTreeSet<_>>();
|
||||||
|
actual_indexes == expected_indexes
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn schema_migrations_table() -> ExpectedTable<'static> {
|
||||||
|
ExpectedTable {
|
||||||
|
name: SCHEMA_MIGRATIONS_TABLE,
|
||||||
|
columns: &[
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "component",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: true,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "version",
|
||||||
|
data_type: "INTEGER",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn normalize_default(value: Option<&str>) -> Option<String> {
|
||||||
|
value.map(|value| value.trim().to_ascii_lowercase())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn quote_identifier(value: &str) -> String {
|
||||||
|
format!("\"{}\"", value.replace('"', "\"\""))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sidecar_path(path: &Path, suffix: &str) -> std::path::PathBuf {
|
||||||
|
std::path::PathBuf::from(format!("{}{}", path.display(), suffix))
|
||||||
|
}
|
||||||
@@ -1,6 +1,9 @@
|
|||||||
//! 跨 official release 的 Translation Memory SQLite 仓储。
|
//! 跨 official release 的 Translation Memory SQLite 仓储。
|
||||||
|
|
||||||
use crate::path_security::{set_file_mode, STATE_FILE_MODE};
|
use crate::path_security::{set_file_mode, STATE_FILE_MODE};
|
||||||
|
use crate::sqlite_migration::{
|
||||||
|
self, ExpectedColumn, ExpectedIndex, ExpectedTable, SqliteSchemaSnapshot,
|
||||||
|
};
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
use bat_core::domain::{
|
use bat_core::domain::{
|
||||||
TranslationMemoryContext, TranslationMemoryDraft, TranslationMemoryEntry,
|
TranslationMemoryContext, TranslationMemoryDraft, TranslationMemoryEntry,
|
||||||
@@ -10,7 +13,7 @@ use bat_core::domain::{
|
|||||||
use bat_core::repositories::TranslationMemoryRepository;
|
use bat_core::repositories::TranslationMemoryRepository;
|
||||||
use bat_core::{Error, Result};
|
use bat_core::{Error, Result};
|
||||||
use serde::de::DeserializeOwned;
|
use serde::de::DeserializeOwned;
|
||||||
use sqlx::sqlite::{SqliteConnectOptions, SqliteJournalMode, SqlitePoolOptions};
|
use sqlx::sqlite::{SqliteConnectOptions, SqliteJournalMode};
|
||||||
use sqlx::{Row, SqlitePool};
|
use sqlx::{Row, SqlitePool};
|
||||||
use std::fs;
|
use std::fs;
|
||||||
use std::path::{Path, PathBuf};
|
use std::path::{Path, PathBuf};
|
||||||
@@ -85,14 +88,24 @@ impl SqliteTranslationMemoryRepository {
|
|||||||
return Err(Error::NotFound(absolute.display().to_string()));
|
return Err(Error::NotFound(absolute.display().to_string()));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if let Ok(metadata) = fs::symlink_metadata(&absolute) {
|
||||||
|
if metadata.len() > 0 {
|
||||||
|
let snapshot = sqlite_migration::read_only_preflight(
|
||||||
|
&absolute,
|
||||||
|
TRANSLATION_MEMORY_SCHEMA_COMPONENT,
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.map_err(db_error)?;
|
||||||
|
classify_translation_memory_schema(&snapshot)?;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
let options = SqliteConnectOptions::from_str(&format!("sqlite://{}", absolute.display()))
|
let options = SqliteConnectOptions::from_str(&format!("sqlite://{}", absolute.display()))
|
||||||
.map_err(|error| Error::Other(error.into()))?
|
.map_err(|error| Error::Other(error.into()))?
|
||||||
.create_if_missing(create_if_missing)
|
.create_if_missing(create_if_missing)
|
||||||
.journal_mode(SqliteJournalMode::Wal)
|
.journal_mode(SqliteJournalMode::Wal)
|
||||||
.busy_timeout(Duration::from_secs(30));
|
.busy_timeout(Duration::from_secs(30));
|
||||||
let pool = SqlitePoolOptions::new()
|
let pool = sqlite_migration::connect_writable_pool(options)
|
||||||
.max_connections(1)
|
|
||||||
.connect_with(options)
|
|
||||||
.await
|
.await
|
||||||
.map_err(db_error)?;
|
.map_err(db_error)?;
|
||||||
set_file_mode(&absolute, STATE_FILE_MODE, "Translation Memory 数据库")
|
set_file_mode(&absolute, STATE_FILE_MODE, "Translation Memory 数据库")
|
||||||
@@ -103,20 +116,359 @@ impl SqliteTranslationMemoryRepository {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async fn init_schema(&self) -> Result<()> {
|
async fn init_schema(&self) -> Result<()> {
|
||||||
sqlx::query(
|
self.init_schema_with_failure(None).await
|
||||||
r#"
|
}
|
||||||
CREATE TABLE IF NOT EXISTS schema_migrations (
|
|
||||||
component TEXT PRIMARY KEY NOT NULL,
|
#[cfg(test)]
|
||||||
version INTEGER NOT NULL CHECK(version >= 1)
|
async fn init_schema_with_test_failure(&self, fail_after_step: usize) -> Result<()> {
|
||||||
)
|
self.init_schema_with_failure(Some(fail_after_step)).await
|
||||||
"#,
|
}
|
||||||
)
|
|
||||||
.execute(&self.pool)
|
async fn init_schema_with_failure(&self, fail_after_step: Option<usize>) -> Result<()> {
|
||||||
|
let mut transaction = sqlite_migration::begin_immediate(&self.pool)
|
||||||
.await
|
.await
|
||||||
.map_err(db_error)?;
|
.map_err(db_error)?;
|
||||||
sqlx::query(
|
let result = self
|
||||||
|
.migrate_in_transaction(&mut transaction, fail_after_step)
|
||||||
|
.await;
|
||||||
|
match result {
|
||||||
|
Ok(()) => transaction.commit().await.map_err(db_error),
|
||||||
|
Err(error) => {
|
||||||
|
let _ = transaction.rollback().await;
|
||||||
|
Err(error)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn migrate_in_transaction(
|
||||||
|
&self,
|
||||||
|
transaction: &mut sqlx::Transaction<'_, sqlx::Sqlite>,
|
||||||
|
fail_after_step: Option<usize>,
|
||||||
|
) -> Result<()> {
|
||||||
|
let snapshot = sqlite_migration::snapshot_connection(
|
||||||
|
transaction.as_mut(),
|
||||||
|
TRANSLATION_MEMORY_SCHEMA_COMPONENT,
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.map_err(db_error)?;
|
||||||
|
match classify_translation_memory_schema(&snapshot)? {
|
||||||
|
TranslationMemorySchemaState::Empty => {
|
||||||
|
create_translation_memory_schema(transaction, fail_after_step).await?;
|
||||||
|
sqlite_migration::write_component_version(
|
||||||
|
transaction,
|
||||||
|
TRANSLATION_MEMORY_SCHEMA_COMPONENT,
|
||||||
|
TRANSLATION_MEMORY_SCHEMA_VERSION,
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.map_err(db_error)?;
|
||||||
|
}
|
||||||
|
TranslationMemorySchemaState::Version(version)
|
||||||
|
if version == TRANSLATION_MEMORY_SCHEMA_VERSION =>
|
||||||
|
{
|
||||||
|
if snapshot.component_version.is_none() {
|
||||||
|
if !snapshot
|
||||||
|
.tables
|
||||||
|
.contains_key(sqlite_migration::SCHEMA_MIGRATIONS_TABLE)
|
||||||
|
{
|
||||||
|
sqlite_migration::create_schema_migrations_table(transaction)
|
||||||
|
.await
|
||||||
|
.map_err(db_error)?;
|
||||||
|
}
|
||||||
|
sqlite_migration::write_component_version(
|
||||||
|
transaction,
|
||||||
|
TRANSLATION_MEMORY_SCHEMA_COMPONENT,
|
||||||
|
TRANSLATION_MEMORY_SCHEMA_VERSION,
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.map_err(db_error)?;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
TranslationMemorySchemaState::Version(version) => {
|
||||||
|
return Err(invalid_translation_memory_schema(format!(
|
||||||
|
"内部不支持的迁移起点 {version}"
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let final_snapshot = sqlite_migration::snapshot_connection(
|
||||||
|
transaction.as_mut(),
|
||||||
|
TRANSLATION_MEMORY_SCHEMA_COMPONENT,
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.map_err(db_error)?;
|
||||||
|
if final_snapshot.component_version != Some(i64::from(TRANSLATION_MEMORY_SCHEMA_VERSION))
|
||||||
|
|| !matches_translation_memory_fingerprint(&final_snapshot)
|
||||||
|
{
|
||||||
|
return Err(invalid_translation_memory_schema(
|
||||||
|
"migration 结果与当前 schema fingerprint 不一致".to_string(),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn find_optional(&self, record_id: &str) -> Result<Option<TranslationMemoryEntry>> {
|
||||||
|
let row = sqlx::query(
|
||||||
r#"
|
r#"
|
||||||
CREATE TABLE IF NOT EXISTS translation_memory (
|
SELECT record_id, source_text, source_hash, normalized_source_text,
|
||||||
|
source_context_json, source_context_hash, translated_text,
|
||||||
|
translation_source_kind, trust_status, official_release_id,
|
||||||
|
source_trace_json, provider, provider_run_id,
|
||||||
|
created_unix_seconds, updated_unix_seconds, trusted_unix_seconds,
|
||||||
|
trusted_by, trusted_reason, supersedes_record_id, superseded_by_record_id
|
||||||
|
FROM translation_memory
|
||||||
|
WHERE record_id = ?1
|
||||||
|
"#,
|
||||||
|
)
|
||||||
|
.bind(record_id)
|
||||||
|
.fetch_optional(&self.pool)
|
||||||
|
.await
|
||||||
|
.map_err(db_error)?;
|
||||||
|
row.map(row_to_entry).transpose()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
|
enum TranslationMemorySchemaState {
|
||||||
|
Empty,
|
||||||
|
Version(u32),
|
||||||
|
}
|
||||||
|
|
||||||
|
fn classify_translation_memory_schema(
|
||||||
|
snapshot: &SqliteSchemaSnapshot,
|
||||||
|
) -> Result<TranslationMemorySchemaState> {
|
||||||
|
if snapshot.is_empty() {
|
||||||
|
return Ok(TranslationMemorySchemaState::Empty);
|
||||||
|
}
|
||||||
|
if let Some(observed) = snapshot.component_version {
|
||||||
|
if observed > i64::from(TRANSLATION_MEMORY_SCHEMA_VERSION) {
|
||||||
|
return Err(invalid_translation_memory_schema(format!(
|
||||||
|
"不支持的 Translation Memory schema 版本:observed={observed}, supported={TRANSLATION_MEMORY_SCHEMA_VERSION}"
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
if observed < 1 {
|
||||||
|
return Err(invalid_translation_memory_schema(format!(
|
||||||
|
"schema version 无效:observed={observed}, supported=1..={TRANSLATION_MEMORY_SCHEMA_VERSION}"
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (matches_translation_memory_fingerprint(snapshot)
|
||||||
|
|| (snapshot.component_version.is_none()
|
||||||
|
&& matches_translation_memory_component_fingerprint(snapshot)))
|
||||||
|
&& snapshot
|
||||||
|
.component_version
|
||||||
|
.is_none_or(|observed| observed == i64::from(TRANSLATION_MEMORY_SCHEMA_VERSION))
|
||||||
|
{
|
||||||
|
return Ok(TranslationMemorySchemaState::Version(
|
||||||
|
TRANSLATION_MEMORY_SCHEMA_VERSION,
|
||||||
|
));
|
||||||
|
}
|
||||||
|
match snapshot.component_version {
|
||||||
|
Some(observed) => Err(invalid_translation_memory_schema(format!(
|
||||||
|
"schema version 与实际结构不一致:observed={observed}, supported={TRANSLATION_MEMORY_SCHEMA_VERSION}"
|
||||||
|
))),
|
||||||
|
None => Err(invalid_translation_memory_schema(
|
||||||
|
"未识别的 legacy schema,拒绝静默修复".to_string(),
|
||||||
|
)),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn invalid_translation_memory_schema(detail: String) -> Error {
|
||||||
|
Error::InvalidArgument(format!("Translation Memory schema 无效:{detail}"))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn matches_translation_memory_fingerprint(snapshot: &SqliteSchemaSnapshot) -> bool {
|
||||||
|
matches_translation_memory_fingerprint_with_migrations(snapshot, true)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn matches_translation_memory_component_fingerprint(snapshot: &SqliteSchemaSnapshot) -> bool {
|
||||||
|
matches_translation_memory_fingerprint_with_migrations(snapshot, false)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn matches_translation_memory_fingerprint_with_migrations(
|
||||||
|
snapshot: &SqliteSchemaSnapshot,
|
||||||
|
include_migrations: bool,
|
||||||
|
) -> bool {
|
||||||
|
let tables = [ExpectedTable {
|
||||||
|
name: "translation_memory",
|
||||||
|
columns: &TRANSLATION_MEMORY_COLUMNS,
|
||||||
|
}];
|
||||||
|
let indexes = [
|
||||||
|
ExpectedIndex {
|
||||||
|
table: "translation_memory",
|
||||||
|
name: "idx_translation_memory_source_hash",
|
||||||
|
columns: &["source_hash"],
|
||||||
|
},
|
||||||
|
ExpectedIndex {
|
||||||
|
table: "translation_memory",
|
||||||
|
name: "idx_translation_memory_normalized_source",
|
||||||
|
columns: &["normalized_source_text"],
|
||||||
|
},
|
||||||
|
ExpectedIndex {
|
||||||
|
table: "translation_memory",
|
||||||
|
name: "idx_translation_memory_context",
|
||||||
|
columns: &["source_hash", "source_context_hash"],
|
||||||
|
},
|
||||||
|
];
|
||||||
|
if include_migrations {
|
||||||
|
let tables = [sqlite_migration::schema_migrations_table(), tables[0]];
|
||||||
|
return sqlite_migration::matches_fingerprint(snapshot, &tables, &indexes);
|
||||||
|
}
|
||||||
|
sqlite_migration::matches_fingerprint(snapshot, &tables, &indexes)
|
||||||
|
}
|
||||||
|
|
||||||
|
const TRANSLATION_MEMORY_COLUMNS: [ExpectedColumn<'static>; 20] = [
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "record_id",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: true,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "source_text",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "source_hash",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "normalized_source_text",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "source_context_json",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "source_context_hash",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "translated_text",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "translation_source_kind",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "trust_status",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "official_release_id",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "source_trace_json",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "provider",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: false,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "provider_run_id",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: false,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "created_unix_seconds",
|
||||||
|
data_type: "INTEGER",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "updated_unix_seconds",
|
||||||
|
data_type: "INTEGER",
|
||||||
|
not_null: true,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "trusted_unix_seconds",
|
||||||
|
data_type: "INTEGER",
|
||||||
|
not_null: false,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "trusted_by",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: false,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "trusted_reason",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: false,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "supersedes_record_id",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: false,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
ExpectedColumn {
|
||||||
|
name: "superseded_by_record_id",
|
||||||
|
data_type: "TEXT",
|
||||||
|
not_null: false,
|
||||||
|
default_value: None,
|
||||||
|
primary_key: false,
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
async fn create_translation_memory_schema(
|
||||||
|
transaction: &mut sqlx::Transaction<'_, sqlx::Sqlite>,
|
||||||
|
fail_after_step: Option<usize>,
|
||||||
|
) -> Result<()> {
|
||||||
|
let steps = [
|
||||||
|
"CREATE TABLE schema_migrations (
|
||||||
|
component TEXT PRIMARY KEY NOT NULL,
|
||||||
|
version INTEGER NOT NULL CHECK(version >= 1)
|
||||||
|
)",
|
||||||
|
"CREATE TABLE translation_memory (
|
||||||
record_id TEXT PRIMARY KEY NOT NULL,
|
record_id TEXT PRIMARY KEY NOT NULL,
|
||||||
source_text TEXT NOT NULL,
|
source_text TEXT NOT NULL,
|
||||||
source_hash TEXT NOT NULL,
|
source_hash TEXT NOT NULL,
|
||||||
@@ -143,80 +495,27 @@ impl SqliteTranslationMemoryRepository {
|
|||||||
CHECK (length(official_release_id) > 0),
|
CHECK (length(official_release_id) > 0),
|
||||||
CHECK (translation_source_kind IN ('provider', 'manual', 'imported')),
|
CHECK (translation_source_kind IN ('provider', 'manual', 'imported')),
|
||||||
CHECK (trust_status IN ('candidate', 'trusted', 'superseded', 'rejected'))
|
CHECK (trust_status IN ('candidate', 'trusted', 'superseded', 'rejected'))
|
||||||
)
|
)",
|
||||||
"#,
|
"CREATE INDEX idx_translation_memory_source_hash
|
||||||
)
|
|
||||||
.execute(&self.pool)
|
|
||||||
.await
|
|
||||||
.map_err(db_error)?;
|
|
||||||
sqlx::query(
|
|
||||||
"CREATE INDEX IF NOT EXISTS idx_translation_memory_source_hash \
|
|
||||||
ON translation_memory(source_hash)",
|
ON translation_memory(source_hash)",
|
||||||
)
|
"CREATE INDEX idx_translation_memory_normalized_source
|
||||||
.execute(&self.pool)
|
|
||||||
.await
|
|
||||||
.map_err(db_error)?;
|
|
||||||
sqlx::query(
|
|
||||||
"CREATE INDEX IF NOT EXISTS idx_translation_memory_normalized_source \
|
|
||||||
ON translation_memory(normalized_source_text)",
|
ON translation_memory(normalized_source_text)",
|
||||||
)
|
"CREATE INDEX idx_translation_memory_context
|
||||||
.execute(&self.pool)
|
|
||||||
.await
|
|
||||||
.map_err(db_error)?;
|
|
||||||
sqlx::query(
|
|
||||||
"CREATE INDEX IF NOT EXISTS idx_translation_memory_context \
|
|
||||||
ON translation_memory(source_hash, source_context_hash)",
|
ON translation_memory(source_hash, source_context_hash)",
|
||||||
)
|
];
|
||||||
.execute(&self.pool)
|
for (index, statement) in steps.iter().enumerate() {
|
||||||
|
sqlx::query(statement)
|
||||||
|
.execute(&mut **transaction)
|
||||||
.await
|
.await
|
||||||
.map_err(db_error)?;
|
.map_err(db_error)?;
|
||||||
|
if fail_after_step == Some(index + 1) {
|
||||||
let current: Option<i64> =
|
return Err(Error::Other(anyhow::anyhow!(
|
||||||
sqlx::query_scalar("SELECT version FROM schema_migrations WHERE component = ?1")
|
"Translation Memory migration failed after step {}",
|
||||||
.bind(TRANSLATION_MEMORY_SCHEMA_COMPONENT)
|
index + 1
|
||||||
.fetch_optional(&self.pool)
|
|
||||||
.await
|
|
||||||
.map_err(db_error)?;
|
|
||||||
if current.is_some_and(|version| version > i64::from(TRANSLATION_MEMORY_SCHEMA_VERSION)) {
|
|
||||||
return Err(Error::InvalidArgument(format!(
|
|
||||||
"不支持的 Translation Memory schema 版本:{}",
|
|
||||||
current.unwrap_or_default()
|
|
||||||
)));
|
)));
|
||||||
}
|
}
|
||||||
sqlx::query(
|
}
|
||||||
r#"
|
|
||||||
INSERT INTO schema_migrations(component, version)
|
|
||||||
VALUES (?1, ?2)
|
|
||||||
ON CONFLICT(component) DO UPDATE SET version = excluded.version
|
|
||||||
"#,
|
|
||||||
)
|
|
||||||
.bind(TRANSLATION_MEMORY_SCHEMA_COMPONENT)
|
|
||||||
.bind(i64::from(TRANSLATION_MEMORY_SCHEMA_VERSION))
|
|
||||||
.execute(&self.pool)
|
|
||||||
.await
|
|
||||||
.map_err(db_error)?;
|
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
|
||||||
|
|
||||||
async fn find_optional(&self, record_id: &str) -> Result<Option<TranslationMemoryEntry>> {
|
|
||||||
let row = sqlx::query(
|
|
||||||
r#"
|
|
||||||
SELECT record_id, source_text, source_hash, normalized_source_text,
|
|
||||||
source_context_json, source_context_hash, translated_text,
|
|
||||||
translation_source_kind, trust_status, official_release_id,
|
|
||||||
source_trace_json, provider, provider_run_id,
|
|
||||||
created_unix_seconds, updated_unix_seconds, trusted_unix_seconds,
|
|
||||||
trusted_by, trusted_reason, supersedes_record_id, superseded_by_record_id
|
|
||||||
FROM translation_memory
|
|
||||||
WHERE record_id = ?1
|
|
||||||
"#,
|
|
||||||
)
|
|
||||||
.bind(record_id)
|
|
||||||
.fetch_optional(&self.pool)
|
|
||||||
.await
|
|
||||||
.map_err(db_error)?;
|
|
||||||
row.map(row_to_entry).transpose()
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#[async_trait]
|
#[async_trait]
|
||||||
@@ -721,6 +1020,7 @@ fn ensure_safe_tm_parent(parent: &Path) -> Result<()> {
|
|||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
use bat_core::domain::TranslationMemorySourceTrace;
|
use bat_core::domain::TranslationMemorySourceTrace;
|
||||||
|
use sqlx::sqlite::SqlitePoolOptions;
|
||||||
|
|
||||||
fn draft(release: &str, source: &str, translated: &str) -> TranslationMemoryDraft {
|
fn draft(release: &str, source: &str, translated: &str) -> TranslationMemoryDraft {
|
||||||
let source_trace = TranslationMemorySourceTrace {
|
let source_trace = TranslationMemorySourceTrace {
|
||||||
@@ -762,6 +1062,163 @@ mod tests {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async fn raw_repository(
|
||||||
|
path: &std::path::Path,
|
||||||
|
create_if_missing: bool,
|
||||||
|
) -> SqliteTranslationMemoryRepository {
|
||||||
|
let options = SqliteConnectOptions::from_str(&format!("sqlite://{}", path.display()))
|
||||||
|
.unwrap()
|
||||||
|
.create_if_missing(create_if_missing);
|
||||||
|
let pool = SqlitePoolOptions::new()
|
||||||
|
.max_connections(1)
|
||||||
|
.connect_with(options)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
SqliteTranslationMemoryRepository { pool }
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn sqlite_translation_memory_preserves_data_across_reopen_and_missing_row() {
|
||||||
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
let path = temp.path().join(TRANSLATION_MEMORY_REPOSITORY_FILE);
|
||||||
|
let repository = SqliteTranslationMemoryRepository::new(&path).await.unwrap();
|
||||||
|
let entry = repository
|
||||||
|
.upsert_candidate(draft("release-1", "Hello", "你好"))
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
let trusted = repository
|
||||||
|
.confirm(&entry.record_id, "reviewer", Some("accepted".to_string()))
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
sqlx::query("DROP TABLE schema_migrations")
|
||||||
|
.execute(&repository.pool)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
repository.pool.close().await;
|
||||||
|
|
||||||
|
let reopened = SqliteTranslationMemoryRepository::open(&path)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
let restored = reopened.find(&trusted.record_id).await.unwrap();
|
||||||
|
assert_eq!(restored.translated_text, "你好");
|
||||||
|
assert_eq!(restored.trust_status, TranslationMemoryTrustStatus::Trusted);
|
||||||
|
assert_eq!(restored.source_trace.official_release_id, "release-1");
|
||||||
|
assert_eq!(
|
||||||
|
reopened.summary().await.unwrap().schema_version,
|
||||||
|
TRANSLATION_MEMORY_SCHEMA_VERSION
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn sqlite_translation_memory_future_schema_is_read_only_failure() {
|
||||||
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
let path = temp.path().join(TRANSLATION_MEMORY_REPOSITORY_FILE);
|
||||||
|
let repository = SqliteTranslationMemoryRepository::new(&path).await.unwrap();
|
||||||
|
sqlx::query("UPDATE schema_migrations SET version = ?2 WHERE component = ?1")
|
||||||
|
.bind(TRANSLATION_MEMORY_SCHEMA_COMPONENT)
|
||||||
|
.bind(i64::from(TRANSLATION_MEMORY_SCHEMA_VERSION) + 1)
|
||||||
|
.execute(&repository.pool)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
repository.pool.close().await;
|
||||||
|
let before = std::fs::read(&path).unwrap();
|
||||||
|
let wal_path = std::path::PathBuf::from(format!("{}-wal", path.display()));
|
||||||
|
let shm_path = std::path::PathBuf::from(format!("{}-shm", path.display()));
|
||||||
|
let wal_before = std::fs::read(&wal_path).ok();
|
||||||
|
let shm_before = std::fs::read(&shm_path).ok();
|
||||||
|
let error = SqliteTranslationMemoryRepository::new(&path)
|
||||||
|
.await
|
||||||
|
.unwrap_err();
|
||||||
|
assert!(error
|
||||||
|
.to_string()
|
||||||
|
.contains("不支持的 Translation Memory schema"));
|
||||||
|
assert_eq!(std::fs::read(&path).unwrap(), before);
|
||||||
|
assert_eq!(std::fs::read(&wal_path).ok(), wal_before);
|
||||||
|
assert_eq!(std::fs::read(&shm_path).ok(), shm_before);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn sqlite_translation_memory_unknown_schema_fails_closed() {
|
||||||
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
let path = temp.path().join(TRANSLATION_MEMORY_REPOSITORY_FILE);
|
||||||
|
let repository = raw_repository(&path, true).await;
|
||||||
|
sqlx::query(
|
||||||
|
"CREATE TABLE schema_migrations (
|
||||||
|
component TEXT PRIMARY KEY NOT NULL,
|
||||||
|
version INTEGER NOT NULL CHECK(version >= 1)
|
||||||
|
)",
|
||||||
|
)
|
||||||
|
.execute(&repository.pool)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
sqlx::query("INSERT INTO schema_migrations(component, version) VALUES (?1, 1)")
|
||||||
|
.bind(TRANSLATION_MEMORY_SCHEMA_COMPONENT)
|
||||||
|
.execute(&repository.pool)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
sqlx::query("CREATE TABLE translation_memory (record_id TEXT PRIMARY KEY)")
|
||||||
|
.execute(&repository.pool)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
repository.pool.close().await;
|
||||||
|
let before = std::fs::read(&path).unwrap();
|
||||||
|
let error = SqliteTranslationMemoryRepository::open(&path)
|
||||||
|
.await
|
||||||
|
.unwrap_err();
|
||||||
|
assert!(error
|
||||||
|
.to_string()
|
||||||
|
.contains("schema version 与实际结构不一致"));
|
||||||
|
assert_eq!(std::fs::read(&path).unwrap(), before);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn sqlite_translation_memory_failed_new_schema_rolls_back_and_retries() {
|
||||||
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
let path = temp.path().join(TRANSLATION_MEMORY_REPOSITORY_FILE);
|
||||||
|
let repository = raw_repository(&path, true).await;
|
||||||
|
assert!(repository.init_schema_with_test_failure(2).await.is_err());
|
||||||
|
let table_count: i64 =
|
||||||
|
sqlx::query_scalar("SELECT COUNT(*) FROM sqlite_master WHERE type = 'table'")
|
||||||
|
.fetch_one(&repository.pool)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(table_count, 0);
|
||||||
|
repository.init_schema().await.unwrap();
|
||||||
|
let version: i64 =
|
||||||
|
sqlx::query_scalar("SELECT version FROM schema_migrations WHERE component = ?1")
|
||||||
|
.bind(TRANSLATION_MEMORY_SCHEMA_COMPONENT)
|
||||||
|
.fetch_one(&repository.pool)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(version, 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn sqlite_translation_memory_concurrent_new_open_has_one_current_schema() {
|
||||||
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
let path = temp.path().join(TRANSLATION_MEMORY_REPOSITORY_FILE);
|
||||||
|
let (left, right) = tokio::join!(
|
||||||
|
SqliteTranslationMemoryRepository::new(&path),
|
||||||
|
SqliteTranslationMemoryRepository::new(&path)
|
||||||
|
);
|
||||||
|
assert!(left.is_ok(), "left open failed: {left:?}");
|
||||||
|
assert!(right.is_ok(), "right open failed: {right:?}");
|
||||||
|
let repository = left.unwrap();
|
||||||
|
drop(right);
|
||||||
|
assert_eq!(
|
||||||
|
repository.summary().await.unwrap().schema_version,
|
||||||
|
TRANSLATION_MEMORY_SCHEMA_VERSION
|
||||||
|
);
|
||||||
|
repository.pool.close().await;
|
||||||
|
let reopened = SqliteTranslationMemoryRepository::open(&path)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
reopened.summary().await.unwrap().schema_version,
|
||||||
|
TRANSLATION_MEMORY_SCHEMA_VERSION
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
#[tokio::test]
|
#[tokio::test]
|
||||||
async fn initializes_schema_and_reuses_trusted_entry_across_releases() {
|
async fn initializes_schema_and_reuses_trusted_entry_across_releases() {
|
||||||
let temp = tempfile::TempDir::new().unwrap();
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user