Lecture: How to Tame the Memory Zoo in Database Systems

Kai-Uwe Sattler (TU Ilmenau)

Lecture slides

For many years, database systems worked successfully with a simple abstraction: data is stored persistently in pages on disk and is loaded into volatile memory for processing. Because disks are much slower than memory, a bufferpool handles the efficient management of the active set of pages by implementing sophisticated eviction strategies.

Today, modern memory and storage technologies have radically changed the landscape. In-memory database systems allow keeping most of the data in memory and have fostered the design and development of memory- and cache-optimized data structures, new technologies such as non-volatile memory (NVM) and processing-in-memory (PIM) open up new opportunities not only for database processing. Together with cloud technologies, new interconnects, and paradigms such as disaggregated memory the traditional memory hierarchy of computing systems evolves towards a memory zoo.

In this lecture, we discuss the implications of this development on data-intensive systems from a database perspective. We survey recent technologies, discuss the challenges of leveraging these technologies in database systems, and present suitable approaches, particularly for NVM, PIM as well as ideas on how to cope with different memory and storage devices in one system.

Prof. Dr.-Ing. habil. Kai-Uwe Sattler (TU Ilmenau)

Kai-Uwe Sattler leads the Database and Information Systems group at the Department of Computer Science and Automation of the TU Ilmenau, Germany. He received his Diploma (M.Sc.) in Computer Science from the University of Magdeburg, Germany. In 1998, he received his Ph.D. in Computer Science (magna cum laude) from the same university. From 1998 to September 2003 he was a member of the Database research group at the University of Magdeburg. From October 2001 until March 2002, he worked as a visiting assistant professor at UC Davis, CA.
His research interests include query processing, database architectures for modern hardware as well as database support for data science and big data. He is the author of 8 textbooks and more than 300 publications. Currently, he is also the coordinator of the priority program "Scalable Data Management for Future Hardware (SPP 2037)" funded by the DFG.