Back to Search View Original Cite This Article

Abstract

<jats:p>This thesis studies how relational database systems can process queries whose temporary intermediate data exceed main memory without suffering abrupt performance collapse. Modern systems are highly optimized for in-memory execution, while traditional larger-than-memory, or external, processing can be orders of magnitude slower once memory limits are crossed. The thesis aims for graceful degradation while preserving in-memory speed. Because query operators differ substantially, the work addresses external execution operator by operator, focusing on blocking operators that must store intermediate relational data. For sorting, it shows that row-oriented layouts outperform column-oriented layouts and can be adapted so data can be written to and read from storage without full serialization or deserialization. For grouped aggregation, it proposes paged memory for intermediates, unifying temporary and persistent memory management, and introduces a buffer page layout that removes additional serialization costs. For joins, it extends these storage techniques and tackles plan-wide memory management, since multiple join operators may be active simultaneously. A dynamic memory assignment strategy maximizes throughput across active joins. Together, these techniques enable robust larger-than-memory query processing on typical hardware without sacrificing in-memory performance.</jats:p>

Show More

Keywords

memory data inmemory operators thesis

Related Articles

PORE

About

Connect