Beyond SWIRL: Scalable Database Indexing for Dynamic Workloads

Table of Links

Abstract and 1. Introduction

1. Related Works

   2.1 Traditional Index Selection Approaches

   2.2 RL-based Index Selection Approaches

2. Index Selection Problem

3. Methodology

   4.1 Formulation of the DRL Problem

   4.2 Instance-Aware Deep Reinforcement Learning for Efficient Index Selection

4. System Framework of IA2

   5.1 Preprocessing Phase

   5.2 RL Training and Application Phase

5. Experiments

   6.1 Experimental Setting

   6.2 Experimental Results

   6.3 End-to-End Performance Comparison

   6.4 Key Insights

6. Conclusion and Future Work, and References

6.3 End-to-End Performance Comparison

reveals IA2’s distinctive advantages in terms of performance, adaptability, and learning efficiency. This section delves into the comparative analysis across three critical dimensions: storage budget optimization, workload diversity, and training length, highlighting IA2’s outperformance. Across various benchmarks, IA2 consistently outperforms other index selection methodologies by an average margin of 15-20%. This performance differential is not only significant but also indicative of IA2’s robust and efficient algorithmic design, which is finely tuned to optimize database query execution times.

\ Storage Budget Efficiency: IA2 demonstrates a remarkable performance gain, with a 61% improvement over the runtime without indexes (shown in Figure 6 (a)). This is a significant enhancement compared to other methods, notably SWIRL, which peaks at about 64%. The key differentiation for

\

\ IA2 lies in its storage-aware RL agent design. By efficiently utilizing the available storage budget, IA2 optimizes index configurations to achieve superior performance gains. Such efficiency is pivotal in scenarios where storage resources are limited, making IA2 a preferred solution for database performance optimization.

\ Workload Changes: Figure 6 (b) underscores IA2’s exceptional adaptability, consistently delivering high performance across complex workloads (W3-W6) and achieving notable improvements in previously unseen scenarios like W7. This demonstrates its robustness and crucial adaptability for dynamic real-world applications.

\ Conversely, SWIRL’s performance improvements on simpler workloads (W1 and W2) and its ability to adapt to the unseen W7 are significantly aided by its intricate workload model, benefiting from its detailed approach to centralized patterns that facilitate action pruning. Nonetheless, these strengths are largely attributed to its elaborate designs and the substantial training resources it consumes. Despite these advantages, IA2 distinguishes itself with superior adaptability and efficiency across a broader spectrum of workloads, affirming its suitability for dynamic environments.

\ Training Efficiency: IA2’s training efficiency is a hallmark of its design, achieving optimal performance with fewer training episodes (shown in Figure 6 (c)). This rapid convergence to peak efficiency is indicative of an efficient learning process, crucial in fast-paced environments where swift adaptation is key. In comparison, SWIRL’s performance with limited training underlines the effectiveness of IA2’s learning mechanism, which not only conserves time but also computational resources, enhancing cost efficiency.

\ IA2’s training efficiency results in significant operational savings, making it a compelling choice for database optimization, where minimizing training costs without sacrificing performance is crucial. To summarize, IA2 excels in learning efficiency, cost-effectiveness, and adaptability, leveraging storage budgets effectively to boost database performance in environments with limited storage, varied workloads, and a need for swift adaptation, establishing it as a vital tool for database administrators and architects.

\

:::info Authors:

(1) Taiyi Wang, University of Cambridge, Cambridge, United Kingdom (Taiyi.Wang@cl.cam.ac.uk);

(2) Eiko Yoneki, University of Cambridge, Cambridge, United Kingdom (eiko.yoneki@cl.cam.ac.uk).

:::

:::info This paper is available on arxiv under CC BY-NC-SA 4.0 Deed (Attribution-Noncommercial-Sharelike 4.0 International) license.

:::

\