Felix Eberhardt, Andreas Grapentin and Andreas Polze
Beyond machine boundaries: A field report on memory disaggregation
With coherent memory interconnects such as CXL, OpenCAPI, CCEX and GenZ, the shared memory programming model could be applied beyond machine boundaries. Until recently, these technologies have been mostly unavailable, but prototypes are becoming available.
First processors with CXL support are announced to arrive in 2022, and IBM has announced the integration of memory inception technology in its Power10 line of processors.
Like storage, main memory could be disaggregated, combined into large pools and shared among different machines, potentially making memory disaggregation a staple of the future datacenter.
In this talk, we will outline the base concept of memory disaggregation, as well as our experiences with a prototypical precursor of the memory inception technology, the ThymesisFlow project.
We discuss our experimental setup using two IC922 Power9 machines and their OpenCAPI based interconnect, and we will present some projects currently running in our lab that use this technology.
We will also present some early latency measurements and discuss what kinds of workloads could be suited to make use of this new memory paradigm, and outline a first way of how to gauge the impact of the wildly different latencies in a new world of super-NUMA datacenters.