Chris Down explains how swap’s main role is being the missing backing store for anonymous (i.e. allocated by
malloc) pages. While all other kinds of data (e.g. paged in files) can be reclaimed easily and later reloaded, because their “source of truth” is elsewhere. There’s no such source for anonymous pages hence these pages can “never” be reclaimed unless there’s swap space available (even if those pages aren’t “hot”).
Linux has historically had poor swap (and by extension OOM) handling with few and imprecise means for configuration. Chris describes the behavior of a machine with and without swap in different scenarios of memory contention. He thinks that poor swap performance is caused by having a poor measure “memory pressure.” He explains how work on cgroup v2 might give the kernel (and thus admins) better measures for memory pressure and knobs for dealing with it.