读书笔记 · 约 20,598 字 · 预计推送 3 部分

PostgreSQL 事务与 MVCC:隔离级别、索引、JSONB、备份恢复(主题精读) 未开始

目录

题记与说明


0. TL;DR

PostgreSQL 的并发模型一句话可以说完:每条 SQL 看到的是数据在过去某个时刻的快照,写操作不覆盖旧版本而是追加新版本。这就是 MVCC(多版本并发控制)。官方给它的卖点是:

"in MVCC locks acquired for querying (reading) data do not conflict with locks acquired for writing data, and so reading never blocks writing and writing never blocks reading."(§13.1)

收益是读写互不阻塞(甚至在最严格的 Serializable 下依然成立),代价则散落在主题的另外四个角落:隔离级别要理解"四档请求、三档实现"与序列化失败重试;表与索引要为 UPDATE = insert + delete 留下的死元组付 VACUUM 的账;JSONB 之所以能建 GIN 索引,正是因为它把文档分解成了可索引的二进制项;备份恢复的 PITR 体系(基础备份 + WAL 归档)则是把 MVCC 背后的 WAL 从容灾材料再利用了一遍。四块知识共享同一个内核:版本、快照、WAL。


1. 核心问题

这组材料共同回答的问题是:一个多用户 OLTP 数据库,如何在不加读锁的情况下保证事务隔离,并且这个选择的后果如何波及索引设计、JSON 存储与备份策略? 拆成五个子问题:

  1. MVCC 机制:PostgreSQL 用什么结构保存"多个版本"?读快照如何判定哪个版本对当前事务可见?
  2. 隔离级别:ANSI 四档隔离级别 PostgreSQL 如何实现?为什么 Read Uncommitted 形同虚设、Repeatable Read 强于标准要求、Serializable 靠 SSI 达成?
  3. 版本堆积的代价:UPDATE 为什么等价于 insert + delete?死元组、表/索引膨胀、VACUUM 的分工是什么?
  4. 索引与 JSONB:六种索引类型各自适配什么数据形态?json 与 jsonb 的本质区别是什么?GIN 的两个操作类如何取舍?
  5. 备份恢复:pg_dump、文件系统快照、PITR 三条路线的能力边界在哪?为什么官方说高可靠场景首选连续归档?

2. 主要主张

2.1 MVCC:把"UNDO 日志"塞进每张表里

2.2 隔离级别:四档请求、三档实现,可串行化靠 SSI

官方总纲(原文):

"In PostgreSQL, you can request any of the four standard transaction isolation levels, but internally only three distinct isolation levels are implemented, i.e., PostgreSQL's Read Uncommitted mode behaves like Read Committed."

理由:"This is because it is the only sensible way to map the standard isolation levels to PostgreSQL's multiversion concurrency control architecture."(§13.2)

Table 13.1(异常 × 级别矩阵,注意两处 "but not in PG"):

级别脏读不可重复读幻读序列化异常
Read uncommittedAllowed, but not in PGPossiblePossiblePossible
Read committedNot possiblePossiblePossiblePossible
Repeatable readNot possibleNot possibleAllowed, but not in PGPossible
SerializableNot possibleNot possibleNot possibleNot possible
ERROR: could not serialize access due to concurrent update

应用侧义务:"must be prepared to retry transactions due to serialization failures"——只有只读事务永不冲突。作者注:9.1 之前 PostgreSQL 的 Serializable 其实就是这一档。

ERROR: could not serialize access due to read/write dependencies among transactions

官方示例:事务 A SELECT SUM(value) FROM mytab WHERE class = 1; 然后插入 class=2 的行;事务 B 对 class=2 求和再插入 class=1 的行——两事务单独串行执行都不会产生双方提交后的结果,故一方必须回滚。序列化失败一律返回 SQLSTATE '40001'(应用重试的判据)。SERIALIZABLE READ ONLY DEFERRABLE 是唯一"Serializable 会阻塞而 Repeatable Read 不会"的情形。

2.3 死元组与 VACUUM:MVCC 每月寄来的账单

"Running a VACUUM is a non-blocking operation. It never causes exclusive locks on tables."
"VACUUM FULL is not an ONLINE operation. It is a blocking operation."

期间读写全停。在线重表的替代工具是 pg_repack(评论区提及,Percona 承诺另文)。

2.4 索引:六种类型,按数据形态选型(§11.2)

官方给出六种内建索引类型:B-tree、Hash、GiST、SP-GiST、GIN、BRIN(另有 bloom 扩展)。每种原文定义:

类型一句话能力(原文引述)适配场景
B-tree"B-trees can handle equality and range queries on data that can be sorted into some ordering."等值/范围/排序,< <= = >= >、BETWEEN、IN、IS NULL;锚定前缀的 col LIKE 'foo%'(不是 '%bar');还能按序取数
Hash"Hash indexes store a 32-bit hash code derived from the value of the indexed column. Hence, such indexes can only handle simple equality comparisons."纯等值
GiST"GiST indexes are not a single kind of index, but rather an infrastructure within which many different indexing strategies can be implemented."几何/二维空间、KNN 最近邻(ORDER BY location <-> point '(101,456)' LIMIT 10)
SP-GiST"SP-GiST permits implementation of a wide range of different non-balanced disk-based data structures, such as quadtrees, k-d trees, and radix trees (tries)."空间分区树、前缀类结构
GIN"GIN indexes are 'inverted indexes' which are appropriate for data values that contain multiple component values, such as arrays."数组/JSONB/全文检索等复合值
BRIN"BRIN indexes ... store summaries about the values stored in consecutive physical block ranges of a table","most effective for columns whose values are well-correlated with the physical order of the table rows."超大表、值与物理顺序强相关(如时序),索引体积极小

选型总纲:"By default, the CREATE INDEX command creates B-tree indexes, which fit the most common situations."——默认 B-tree,特殊数据形态(几何、数组、JSON、时序大表)才换 USING gist/gin/brin/...。

2.5 JSONB:二进制分解换来可索引性(§8.14)

json 与 jsonb 的本质区别(原文整段,值得背下来):

"The json data type stores an exact copy of the input text, which processing functions must reparse on each execution; while jsonb data is stored in a decomposed binary format that makes it slightly slower to input due to added conversion overhead, but significantly faster to process, since no reparsing is needed. jsonb also supports indexing, which can be a significant advantage."

行为差异:json 保留空格、键序、重复键(处理函数视最后一个为有效);jsonb 不保留三者,重复键只留最后值。jsonb 的输出会重排键({"bar": "baz", "balance": 7.77, "active":false} 输出为按序的 {"bar": "baz", "active": false, "balance": 7.77})。jsonb 把 JSON 原始类型映射到 PostgreSQL 原生类型:string→text、number→numeric、boolean(只认小写 true/false),因此拒绝 \u0000、拒绝超出 numeric 范围的数字。

两个杀手级操作符:

GIN 的两个操作类:

"A jsonb_path_ops index is usually much smaller than a jsonb_ops index over the same data, and the specificity of searches is better, particularly when queries contain keys that appear frequently in the data."

代价:"it produces no index entries for JSON structures not containing any values, such as {"a": {}}",查空结构要全索引扫描。

表达式索引补位:WHERE jdoc -> 'tags' ? 'qui' 对表达式求值,默认 GIN 索引帮不上忙;官方给的正解是建表达式索引 CREATE INDEX idxgintags ON api USING GIN ((jdoc -> 'tags'));,或者改写成可索引的包含查询 WHERE jdoc @> '{"tags": ["qui"]}'(代价是索引里存下全部键值)。

设计建议(三条官方原话):

  1. "In general, most applications should prefer to store JSON data as jsonb, unless there are quite specialized needs, such as legacy assumptions about ordering of object keys."
  2. "even for applications where maximal flexibility is desired, it is still recommended that JSON documents have a somewhat fixed structure"——半固定结构的文档让汇总查询好写,JSON 与关系模型"can co-exist and complement each other"。
  3. "any update acquires a row-level lock on the whole row. Consider limiting JSON documents to a manageable size in order to decrease lock contention"——理想情况"JSON documents should each represent an atomic datum that business rules dictate cannot reasonably be further subdivided into smaller datums"。

另有实用细节:对象比数组更适合做包含/存在查询("because unlike arrays they are internally optimized for searching");支持下标读写 jsonb_field['key'] = '1';对 NULL 源做下标 UPDATE 会把它变成 {"a": 1}。

2.6 备份恢复:pg_dump 与 PITR 是两个物种(§25.3)

路线盘点:

路线能力边界
pg_dump 逻辑转储灵活、可单库单表、可跨版本"pg_dump and pg_dumpall do not produce file-system-level backups and cannot be used as part of a continuous-archiving solution. Such dumps are logical and do not contain enough information to be used by WAL replay."
文件系统级备份快(冷拷贝)只能整簇恢复;需要文件系统快照工具保证一致性
连续归档 + PITR基础备份 + WAL 回放,可恢复到任意时间点只能整簇恢复;恢复目标点必须晚于基础备份结束时间

PITR 的四大收益(官方原文):

  1. "We do not need a perfectly consistent file system backup as the starting point. Any internal inconsistency in the backup will be corrected by log replay… So we do not need a file system snapshot capability, just tar or a similar archiving tool."
  2. "Since we can combine an indefinitely long sequence of WAL files for replay, continuous backup can be achieved simply by continuing to archive the WAL files. This is particularly valuable for large databases, where it might not be convenient to take a full backup frequently."
  3. "It is not necessary to replay the WAL entries all the way to the end. We could stop the replay at any point and have a consistent snapshot of the database as it was at that time."——误删表后可回放到删表前一秒,这是 pg_dump 给不了的能力。
  4. "If we continuously feed the series of WAL files to another machine that has been loaded with the same base backup file, we have a warm standby system."

机制:WAL 段文件"normally 16MB apiece",按序编号(段名前 8 位即 timeline ID,如 0000000100001234000055CD)。归档三件套:wal_level = replica+、archive_mode = on、archive_command(或 archive_library),如 archive_command = 'test ! -f /mnt/server/archivedir/%f && cp %p /mnt/server/archivedir/%f'(%p 待归档文件路径、%f 仅文件名)。三条硬规则:

基础备份:"The easiest way to perform a base backup is to use the pg_basebackup tool.";低层 API 是 pg_backup_start(label => 'label', fast => false) → 复制数据目录(用 tar/cpio,不是 pg_dump)→ pg_backup_stop(wait_for_archive => true),并把后者的第 2/3 个返回值逐字节写入 backup_label/tablespace_map("byte for byte without modification")。备份期间 "full_page_writes is effectively forced on"(性能略降但"not necessary to be concerned about the amount of time it takes"),可并发多个备份,也无需停库。当前版本文档还描述了 --incremental 增量备份:依赖 backup manifest 与 pg_wal/summaries 中的 WAL 摘要,恢复时需 pg_combinebackup 合成全量(增量备份能力自 PostgreSQL 17 起提供——此版本号为本笔记基于已有知识的补充,本次提取的文档页未标注引入版本)。

恢复七步(文档顺序):停库 → 把旧数据目录挪走("you should at least save the contents of the cluster's pg_wal subdirectory",未归档段可能救命)→ 清空数据目录与表空间根 → 恢复基础备份(属主要对——"the database system user, not root!";增量备份先过 pg_combinebackup)→ 清空 pg_wal/ 后把保留的未归档段放回 → 在 postgresql.conf 配 restore_command = 'cp /mnt/server/archivedir/%f %p' 并 "create a file recovery.signal in the cluster data directory"(PG 12+ 用信号文件取代 recovery.conf)→ 启动。注意 restore_command 对"找不到文件"应返回非零——"This is not an error condition",正常的恢复流程就是以一次"file not found"收尾的。

恢复目标与 timeline:可按时间(recovery_target_time)、命名还原点或事务 ID(xid)指定停止点;硬约束:"The stop point must be after the ending time of the base backup… You cannot use a base backup to recover to a time when that backup was in progress."。每次归档恢复完成都会开启新 timeline:"Whenever an archive recovery completes, a new timeline is created to identify the series of WAL records generated after that recovery.";.history 文件"cheap and appropriate to keep… around indefinitely";且 "You cannot recover into timelines that branched off earlier than the base backup."——做演练时选错 timeline 会扑空。


3. 数据与证据

4. 本地实证说明

本次未执行任何本地验证:which psql pg_ctl pg_dump 均未命中(PostgreSQL 未安装);docker images/docker ps 因 daemon 未运行失败(failed to connect to the docker API ... The system cannot find the file specified)。理论上可做的验证(启动 postgres 容器后:双会话演示 RC/RR 差异、xmin/xmax 观察、jsonb @> 语义、pg_basebackup 演练)均未进行,已列入局限。


5. 结论

  1. MVCC 把并发控制降维成"版本可见性判定":读路径只做快照过滤,不加锁;写路径追加新版本。由此换来读写并存的并发模型,也由此产生 PostgreSQL 特有的两笔开销——表内自持旧版本(没有 UNDO 段,也就没有 snapshot too old)与死元组回收(VACUUM 家族)。
  2. 隔离级别的"表面四档、内核三档"是理解 PostgreSQL 事务的钥匙:RU≈RC(每语句快照)、RR=快照隔离(每事务快照 + 首更新者获胜)、Serializable=SSI(快照隔离 + 谓词锁 + 读写依赖检测)。RR 比 SQL 标准强(防幻读),RU 比 SQL 标准弱(无脏读也谈不上)。应用层的义务是把 40001 重试写成默认习惯。
  3. 膨胀治理要看删除形态:VACUUM 只把空间还给表内部复用,只有高水位线后的空页才还给 OS;散布删除的表 VACUUM 后页数不变,要缩表只有 VACUUM FULL(阻塞)或 pg_repack(在线)。索引膨胀与堆膨胀并存。
  4. 索引选型是"数据形态 → 索引类型"的映射:可排序标量用 B-tree(默认),纯等值 Hash,几何 GiST/SP-GiST,复合值(数组/JSONB/全文)GIN,物理序相关的大表 BRIN。JSONB 场景还要在 jsonb_ops / jsonb_path_ops / 表达式索引之间按查询模式取舍。
  5. 备份策略的判定线是"要不要 PITR":要"恢复到误操作前一秒"或热备,只有连续归档一条路;pg_dump 是逻辑备份,逻辑上就不可能参与 WAL 回放。PITR 的韧性来自"基础备份不必完全一致 + WAL 回放修正",其纪律来自归档命令的零退出约定与 pg_wal 空间监控。

6. 局限


7. 对我的启发

  1. 和已有的 SQLite WAL 笔记接上:SQLite 的"读者记住 WAL end-mark 就是 MVCC 的最小实现",PostgreSQL 是它的服务器级放大版——xmin/xmax 判可见性、快照即"活跃事务集合"、WAL 从日志升级为备份/复制的原材料。两个系统一起看,"多版本 + 追加日志 + 定期回收"这个三件套就是所有现代存储的公共骨架。
  2. 把 40001 重试写进默认模板:官方明确要求 RR/Serializable 应用"must be prepared to retry transactions due to serialization failures"。以后写任何涉及 PostgreSQL 事务的代码(包括 Agent 工具里的写操作),冲突重试(指数退避 + 上限)应当是模板的一部分,而不是出了 40001 再补。
  3. 默认 Read Committed ≠ 默认正确:UPDATE ... SET hits = hits + 1 这类"读-改-写"在 RC 下要靠 EvalPlanQual 兜底,复杂条件下("read-modify-write 跨语句")就该上 RR + 重试或 SELECT ... FOR UPDATE。选隔离级别的依据是并发异常对业务的代价,不是"默认的最安全"。
  4. 长事务是 MVCC 系统共同之敌:SQLite 的 checkpoint starvation 与 PostgreSQL 的死元组堆积/谓词锁膨胀是同一个病的两副面孔——旧版本因快照而不可回收。生产 checklist 应有:idle_in_transaction_session_timeout、监控 pg_stat_user_tables.n_dead_tup 与 WAL 目录体积。
  5. JSONB 用于 Agent 记忆/事件存储的边界:jsonb + GIN 很适合"半固定结构 + 偶发按任意键查询"的场景(正是我之前事件溯源/Agent 记忆调研里的需求),但要记住官方警告:整行级锁意味着大文档高频更新是反模式——把"可变状态"和"追加型事实"分表存放,文档只做原子事实的载体。
  6. 备份恢复接上 321 原则笔记:之前"数据备份与恢复演练-321原则"那篇讲的是通用纪律,PostgreSQL PITR 给出数据库版的落地形态——基础备份(全量层)+ WAL 归档(增量层)+ 恢复演练(recovery.signal 七步)。值得吸收的硬规则:归档命令"零退出当且仅当成功"、恢复目标必须晚于基础备份结束时刻、演练时盯 timeline。
  7. "读实现之前先读文档的例外框":官方文档里"Allowed, but not in PG"、"sequence 变更不回滚"、"EvalPlanQual"这些加框/加粗的例外,恰恰是二手教程最常省略、生产事故最常发生的地方。精读官方文档的价值密度高于泛读博客。

附:来源链接汇总

  1. https://www.postgresql.org/docs/current/mvcc-intro.html (§13.1 MVCC 引言:定义与"读不阻塞写")
  2. https://www.postgresql.org/docs/current/transaction-iso.html (§13.2 事务隔离:四档矩阵、RC/RR/Serializable 行为、SSI 调优清单)
  3. https://www.postgresql.org/docs/current/datatype-json.html (§8.14 JSON 类型:json vs jsonb、@>/? 语义、GIN 两个操作类)
  4. https://www.postgresql.org/docs/current/indexes-types.html (§11.2 索引类型:六类索引的定义与适配)
  5. https://www.postgresql.org/docs/current/continuous-archiving.html (§25.3 连续归档与 PITR:归档配置、基础备份、恢复七步、timeline)
  6. https://www.percona.com/blog/basic-understanding-bloat-vacuum-postgresql-mvcc (Percona:xmin/xmax 实测、死元组、VACUUM/VACUUM FULL 对比、索引膨胀演示)
← 资料库今日播报 →