Beaconnect: continuous web performance A/B testing at scale

Content delivery networks (CDNs) are critical for minimizing access latency in the Web as they efficiently distribute online resources across the globe. But since CDNs can only be enabled on the scope of entire websites (and not for individual users or user groups), the effects of page speed acceleration are often quantified with potentially skewed before-after comparisons rather than statistically sound A/B tests. We introduce the system Beaconnect for collecting and analyzing Web performance data without being subject to these limitations. Our contributions are threefold. First, Beaconnect is natively compatible with A/B testing Web performance as it is built for a custom browser-based acceleration approach and thus does not rely on traditional CDN technology. Second, we present our continuous aggregation pipeline that achieves sub-minute end-to-end latency. Third, we describe and evaluate a scheme for continuous real-time reporting that is especially efficient for large customers and processes data from over 100 million monthly users at Baqend.