Mount Mayhem at Netflix: Scaling Containers on Modern CPUs

Netflix explains how migrating to kubelet + containerd with per-container user namespaces triggered severe mount lock contention on r5.metal nodes.

• •100 containers with 50-layer images generate 20,200 mount operations, all competing for a global kernel VFS lock.
• •Flamegraph showed containerd spending nearly all time acquiring the lock via open_tree(), mount_setattr(), and move_mount() calls.
• •Intel TMA revealed 95.5% of pipeline slots stalled on contested accesses, with 57% from false sharing.
• •NUMA (dual-socket r5.metal) worsened contention via remote memory latency; disabling hyperthreading improved launch times 20-30%.

This summary was automatically generated by AI based on the original article and may not be fully accurate.