To Partition, or Not to Partition, That is the Join Question in a Real System

An efficient implementation of a hash join has been a highly researched problem for decades. Recently, the radix join has been shown to have superior performance than the alternatives (e.g., the non-partitioned hash join), albeit on synthetic microbenchmarks. So, it is not clear whether one can simply replace the hash join in an RDBMS or use the radix join as a performance booster for selected queries. If the latter, it is still unknown when one should rely on the radix join to improve the performance.In this paper, we address these questions, show how to integrate the radix join in a code-generating DBMS, and make it competitive for selective queries by introducing a bloom filter based semi-join reducer. We then evaluate how well it runs when used in queries from more representative workloads like TPC-H. Surprisingly, the radix join brings a noticeable improvement only in one out of all the 59 joins in TPC-H. Thus, with an extensive range of microbenchmarks, we isolate the effects of the most important workload factors and synthesize the range of values, where partitioning the data for the radix join pays off. Our analysis shows that the benefit of data partitioning quickly diminishes as soon as we deviate from the optimal parameters, and does not compensate for the added materialization overhead. We, thus, conclude that integrating the radix join within a database rarely justifies the increase in code and optimizer complexity and advise against it.