sched discussion
[PATCH v2] sched/numa: avoid doubling scan period for remote private faults
LLM 分析
sched/numa:避免 remote private fault 让扫描周期被错误地翻倍
系列概况
- 标题:[PATCH v2] sched/numa: avoid doubling scan period for remote private faults
- 作者:Hongling Zeng zenghongling@kylinos.cn;Suggested-by: Zhan Xusheng zhanxusheng@xiaomi.com
- 版本:v2,单 patch(无后续回复)
- 规模:1 file changed, 10 insertions(+), 7 deletions(-)
- 修改文件:kernel/sched/fair.c
- 代码统计:
update_task_scan_period()内 3 处实质改动(第 3505、3529、3538 行附近)+ 两段注释修订 - Message-ID:20260804061515.640924-1-zenghongling@kylinos.cn
- 完整性:commit message、Suggested-by、Signed-off-by、Changes from v1 齐全,签名链完整
补丁目的
NUMA 自动平衡为每个 task 维护 numa_scan_period,按最近一段时间的 fault 分布动态调整扫描节奏。旧实现用 local + shared == 0 判定 "无 relevant fault",对只产生 remote private fault 的进程(local=0, shared=0, remote>0)会错误判为完全空闲,从而触发 numa_scan_period << 1,扫描被强制放慢一倍。本 patch 把 no-fault 判定改成 local + remote == 0,让 remote private fault 继续走到 locality-ratio 计算,同时纠正 ps_ratio、lr_ratio 两个分支注释与代码语义相反的问题。
旧流程的问题
- 判定口径错:把 "local=0 且 shared=0" 当成 "无 fault",忽略 remote 字段,把大量 "远程私有访问为主" 的负载打入空闲档。
- 注释/代码语义错位:
ps_ratio分支注释写的是 "shared 多 → 没必要继续 fast scan",但ps_ratio对应的正是 "private 多" 的场景;lr_ratio分支同样语义错置。 - 两处错误同时出现在同一函数里,维护者很容易误读 NUMA 调整策略。
新流程
- no-fault 条件:
local + remote == 0,或p->numa_faults_locality[2](迁移失败过)→ 触发numa_scan_period << 1。 - ps_ratio 路径:专门处理 "大多访问是 private",注释同步改正,扫描频率按 NUMA_PERIOD_SLOTS 节奏递增。
- lr_ratio 路径:专门处理 "大多访问是 local",注释去掉过时的 "shared memory" 表述。
- 行为差异:纯 remote private 负载不再被无条件翻倍;最终周期交给 locality-ratio 决定。
关键实现
/* kernel/sched/fair.c, update_task_scan_period() */
@@ update_task_scan_period(struct task_struct *p, ...)
- if (local + shared == 0 || p->numa_faults_locality[2]) {
+ if (local + remote == 0 || p->numa_faults_locality[2]) {
/* Completely idle or all activity is in areas that
* are not of interest to automatic numa balancing.
* Related to that, if there were failed migration then
* it implies we are migrating too quickly or the local
* node is overloaded. In either case, scan slower. */
p->numa_scan_period = min(p->numa_scan_period_max,
p->numa_scan_period << 1);
}
if (ps_ratio >= NUMA_PERIOD_THRESHOLD) {
- /* Most memory accesses are shared with other tasks.
- * There is no point in continuing fast NUMA scanning,
- * since other tasks may just move the memory elsewhere. */
+ /* Most memory accesses are private. Slow down NUMA scanning
+ * since there is little shared memory to rebalance. */
...
} else if (lr_ratio >= NUMA_PERIOD_THRESHOLD) {
- /* Most memory accesses are local. There is no need to
- * do fast NUMA scanning, since memory is already local. */
+ /* Most memory accesses are local. There is no need to
+ * do fast NUMA scanning, since memory is already local. */
...
}
Patch 概览
本系列只有一个 patch,聚焦 update_task_scan_period() 内的判定逻辑与两段错置注释,没有触及 locality-ratio 计算本身、阈值(NUMA_PERIOD_SLOTS、NUMA_PERIOD_THRESHOLD)或 numa_faults_locality[2] 的语义。
类比
把 NUMA 扫描节奏想成 "体检频率",把 fault 类型想成 "运动场景":
local fault= 在家门口跑步;shared fault= 在公司会议室开会;remote fault= 出差去外地跑业务。
旧规则只看 "家门口 + 公司" 的步数,于是出差日被误判为 "整天没动",下次体检直接拉长一倍。新规则改成 "家门口 + 外地",出差日也算动过,按正常节奏安排体检;当你几乎只出差、不怎么开共享会议时,仍会因 "私事为主" 适当放慢扫描——但绝不会突然从 "出差日" 直接跳到 "完全不查"。
Highlight:风险与注意点
- CPU 开销变化面:no-fault 判定改动会让 "纯 remote private" 负载的扫描更密,SCAN 路径 CPU 开销可能略增;建议跑 numactl/perf 基准实测。
- 作者已声明保留语义:"this does not necessarily make the scan period shorter"——纯 private 负载最终仍会因
ps_ratio涨到NUMA_PERIOD_SLOTS而变慢,不再被强制<< 1,但仍属调整。 - 失败迁移仍会触发翻倍:
numa_faults_locality[2]分支未改动,若一次失败 migration 后又被新逻辑识别为有 fault,仍可能走后续调整路径,需要继续观察真实工作负载。 - 注释错位可能影响后续 reviewer:
ps_ratio/lr_ratio注释颠倒的问题仅在表达层修复;若有维护者按 v1 注释做过行为推断,需要回看前面是否有要被追溯修正的判断。 - 未触及 numa_group/shared 权衡:随着 "remote shared" 概念未来可能引入,这里以
local + remote == 0作为新基线,仍需要在 v3 或后续补丁里重新审视。 - 单 patch、零回复:本邮件列表只有 1 封 v2 提交,没有 reviewer 反馈;上游对 "remote shared" 是否存在的进一步澄清,要在落地后看 reply 走向。
版本变化(v1 → v2)
- 修订 commit message,去掉错误的 "speed up" 措辞。
- patch 标题改为 "avoid doubling scan period for remote private faults",更准确描述行为。
- 移除
lr_ratio分支中关于 shared memory 的过时注释。 - 增加 "scan period comparison for clarity",方便复核到底该不该进入翻倍分支。
一句话总结
把 update_task_scan_period() 的 no-fault 判定从 local + shared 改成 local + remote,避免只产生远程私有 fault 的 NUMA 负载被无条件翻倍降速;同时对齐 ps_ratio / lr_ratio 两个分支的注释语义。
+-----------------------------------------------------------+
| old: local + shared == 0 vs new: local + remote == 0 |
+-----------------------------------------------------------+
stats: local=0, shared=0, remote=N
|
v
+-------+----------------------------+ +----------------------------+
| old | treat as fully idle | | new: treat as has remote |
| | / no relevant fault | | private fault |
+-------+----------------------------+ +----------------------------+
| |
v v
period = min(period_max, period << 1) continue into ps_ratio
(unconditional 2x slower) / lr_ratio calculation
| (locality-driven adjust)
v |
| v
must wait full scan window NUMA_PERIOD_SLOTS converges
before re-evaluate naturally toward steady state
+-----------------------------------------------------+
| ps_ratio branch: comment fix |
+-----------------------------------------------------+
- old (wrong): "Most accesses are shared with other tasks"
+ new (correct): "Most accesses are private. Slow down
+ NUMA scanning ... little shared memory
+ to rebalance."
+-----------------------------------------------------+
| lr_ratio branch: comment fix |
+-----------------------------------------------------+
- old: stale "shared memory" wording
+ new: keep correct intent, remove stale shared wording.