FedCTR: Federated Native Ad CTR Prediction with Cross-platform User Behavior Data

Native ad is a special kind of display ad that is similar to the native content displayed in online websites [25]. CTR prediction for display ads has been extensively studied [5, 10, 19, 21, 23, 37, 44, 45]. Different from search ads where the search query triggering ad impressions can provide clear user intent, in display ads there is no explicit user intent [45]. Thus, it is very important for display ad CTR prediction to model user interest from users’ historical behaviors. Many existing CTR prediction methods rely on handcrafted features to represent users and ads, and they focus on capturing the interactions between these features to estimate the relevance between users and ads [3, 5, 10, 15, 20, 28, 37, 45]. For example, Cheng et al. [5] proposed a Wide&Deep model that integrates a wide linear channel with cross-product and a deep neural network channel to capture feature interactions for CTR prediction. They represented users’ interest with their demographic features, device features and the features extracted from their historical impressions. Guo et al. [10] proposed a DeepFM model that combines factorization machines and deep neural networks to model feature interactions. They used ID and category features to represent ads, and used the feature collections of historical clicked items to represent users. However, the design of handcrafted features used in these methods usually requires massive domain knowledge, and handcrafted features may not be optimal in modeling user interest. There are also several methods for display ads CTR prediction that use deep learning techniques to learn user-interest representations from their behaviors on e-commerce platforms [9, 18, 31]. For example, Zhou et al. [45] proposed a deep interest network (DIN) that learns representations of users from the items they have interacted with on the e-commerce platform based on the relatedness between those items and the candidate ads. In Reference [44], an improved version of DIN named deep interest evolution network (DIEN) was proposed, which models users from their historical behaviors on items via a GRU network with attentional update gate. These deep learning-based CTR prediction methods for display ads on e-commerce platform usually rely on the user behaviors on the same platform to model user interest. However, besides the e-commerce platform, native ads are widely displayed on many other online platforms such as news websites [1]. The user behaviors on these platforms may have insufficient clues to infer user interest in ads. Thus, these CTR prediction methods designed for display ads on e-commerce platform may be not optimal for native ads.

There are only a few studies for native ad CTR prediction [1, 29]. For example, An et al. [1] proposed to model user interest for native ad CTR prediction from their search queries and browsed webpages. These studies found that incorporating multi-platform user behaviors can model user interest more accurately than using single-platform user behaviors. These methods usually require centralized storage of multi-platform user behaviors. However, user behavior data is highly privacy-sensitive, and cannot be directly aggregated across platforms due to privacy concerns and user data protection regulations like GDPR. Different from these methods, our proposed FedCTR method can exploit user behaviors on different platforms for native ad CTR prediction via federated learning, which can eliminate the centralized storage of user behavior data and achieve better user privacy protection.

The learning of user representation in our proposed FedCTR method with multi-platform user behavior data is based on federated learning [26]. Federated learning is a recently proposed machine learning technique, which can learn a shared model from the private data of massive users in a privacy-preserving way [8, 11, 22, 27, 42]. Instead of directly uploading the private user data to a central server for model training, in federated learning the user data is locally stored on different user devices such as smartphones and personal computers. Each user device has a copy of the local model and computes the model updates based on local user interactions. The model updates from a large number of users are uploaded to the server and aggregated into a single one for global model update [26]. Then the new global model is delivered to user devices for local model update, and this process iterates for multiple rounds. Since the model updates contain much less information than the raw user data, federated learning can provide an effective way to exploit the private data of different users and protect their privacy at the same time [26]. Based on the idea of federated learning, Jiang et al. [14] proposed a federated topic modeling approach to train topic models from the corpus owned by different parties. In these federated learning methods, the samples for model training are distributed on different clients, and each client shares the same feature space, aka horizontal federated learning [42]. In horizontal federated learning, the models are usually fully or partially shared across different clients, and the training sample sets on different clients are usually different. The server and clients communicate the updates of the local models. However, different from horizontal federated learning, in the task of native ad CTR prediction with multi-platform user behaviors, the user behavior data of each sample is distributed on different platforms. These platforms may contain the same sample but they can only see part of the user feature space, aka vertical federated learning [42]. In vertical federated learning, the models on different clients are usually independent, and the feature sets of each sample are kept by different clients in a decentralized manner. Since the features on different clients cannot be directly aggregated, the clients need to exchange the intermediate results inferred by local models as well as their gradients to synthesize the final prediction results and train models, respectively. Note that different from horizontal federated learning, in vertical federated learning the clients are usually service providing platforms that store certain kinds of data of many users rather than individual user devices. Thus, the problem studied in this work is quite different from the existing federated learning methods. To our best knowledge, this is the first work that applies federated learning to privacy-preserving native ad CTR prediction with multi-platform user behaviors, and FedCTR is the first approach to protect the privacy information in user embeddings at both training and inference stages.

Existing CTR prediction methods usually train the models in a centralized way, where both the training data and the model parameters are located in the same place. In the proposed FedCTR method for native ad CTR prediction, the training data is distributed on multiple platforms. For example, the ad information and the users’ click and non-click behaviors on ads that can serve as labels for model training are located in the ad platform, while users’ behaviors on many online platforms are located in other platforms, which cannot be centralized due to privacy constraints. In addition, FedCTR contains multiple models such as user models, ad model, and aggregator model, which are distributed on different platforms. Thus, it is a non-trivial task to train the models of FedCTR without violating the privacy protection requirement. Motivate by Reference [26], in this section, we present a privacy-preserving framework to train the models for FedCTR where each platform learns the model on it in a federated way, and only the gradients of user embeddings (rather than raw user behaviors or models) are communicated across different platforms. The framework for model training is shown in Figure 4, and the data communications among different participants are summarized in Table 2. The details of model training are introduced as follows.

At the model training stage, for a randomly selected user behavior on the ad platform denoted as \((u, d, y, t),\) which means at timestamp t an ad d is displayed to user u and her click behavior is y (1 for click and 0 for non-click), we first use the FedCTR framework to learn the local user embeddings \([\mathbf {u}_1, \mathbf {u}_2,\ldots , \mathbf {u}_K]\) and the aggregated user embedding \(\mathbf {u}\) at timestamp t using the current user and aggregator models (steps 1–3). We also use the current ad model to learn an embedding \(\mathbf {d}\) of this ad. Then, we use the current CTR predictor model to compute the predicted click probability score \(\hat{y}\). By comparing \(\hat{y}\) with y, we can compute the loss of the current FedCTR models on this training sample (step 4). In our model training framework cross-entropy loss is used, and loss of this sample can be formulated aswhere \(\Theta _U = [\Theta _{U_1},\Theta _{U_2},\ldots ,\Theta _{U_K}]\) is the parameter set of user models on different platforms.

Then, we compute the model gradients for model update (steps 5–9). According to the loss \(\mathcal {L}\), we compute the gradient of \(\hat{y}\), denoted as \(\nabla \hat{y}\). We input \(\nabla \hat{y}\) to the CTR predictor. Based on \(\nabla \hat{y}\) and the CTR prediction function \(\hat{y}=f_{CTR}(\mathbf {u},\mathbf {d};\Theta _C)\), we can compute the gradient of parameters \(\Theta _C\) in CTR predictor (denoted as \(\nabla \Theta _C\)), the gradient of user embedding \(\mathbf {u}\) (denoted as \(\nabla \mathbf {u}\)) and the gradient of ad embedding \(\mathbf {d}\) (denoted as \(\nabla \mathbf {d}\)). The model parameters of the CTR predictor \(\Theta _C\) can be updated using \(\nabla \Theta _C\) following the SGD [2] algorithm, i.e., \(\Theta _C = \Theta _C - \eta \nabla \Theta _C\), where \(\eta\) is the learning rate.

The gradient of ad embedding \(\nabla \mathbf {d}\) is then sent to the ad model. Based on \(\nabla \mathbf {d}\) and the function \(\mathbf {d}=f_{ad}(ID, text; \Theta _D)\) that summarizes the neural architecture of ad model, we can compute the gradient of the ad model, denoted as \(\nabla \Theta _D\). We use \(\nabla \Theta _D\) to update the model parameters of ad model \(\Theta _D\), i.e., \(\Theta _D = \Theta _D - \eta \nabla \Theta _D\).

The user embedding gradient \(\nabla \mathbf {u}\) is distributed to the aggregator in the user server. Based on the aggregator model function \(\mathbf {u}=f_{agg}(\mathbf {U}; \Theta _A)\) and \(\nabla \mathbf {u}\), we compute the gradients of the aggregator model (denoted as \(\nabla \Theta _A\)) as well as the gradient of each local user embedding (denoted as \(\nabla \mathbf {u}_i, i=1,\ldots ,K\)). The model parameters of the aggregator can be updated as \(\Theta _A = \Theta _A - \eta \nabla \Theta _A\). The gradients of the local embeddings, i.e., \(\nabla \mathbf {u}_i,i=1,\ldots ,K\), are sent to the corresponding behavior platform, respectively.

On each behavior platform, when it receives the gradient of the local user embedding, the platform will combine the local user embedding gradient, the input user behaviors and the current user model based on the function \(\mathbf {u}_i=f_{user}^i(behaviors; \Theta _{U_i})\) to compute the gradient of the user model, which is denoted as \(\nabla \Theta _{U_i}\) on the i-th platform. Then the model parameters of the local user models are updated as \(\Theta _{U_i} = \Theta _{U_i} - \eta \nabla \Theta _{U_i}\). Above model training process is conducted on different training samples for multiple rounds until models converge. We summarize the process of our FedCTR method in Algorithm 1.

In our federated model training framework, the user behaviors on different platforms (e.g., the ad platform and the multiple behavior platforms for user-interest modeling) never leave the local platforms, and only the model gradients are distributed from the ad platform to user server, and from user server to each user behavior platform. Since model gradients usually contain much less private information than the raw user behaviors [26], user privacy can be protected at the training stage of the proposed FedCTR method.

Since there is no publicly available dataset for native ad CTR prediction, we constructed one by collecting the logs of 100,000 users on the native ads displayed on the Bing Ads platform from 11/06/2019 to 02/06/2020. The logs in the last week were reserved for testing, and the remaining logs were used for model training. We randomly selected 10% of the samples in training set for validation. We also collected the search logs and webpage browsing behavior logs of users recorded by a commercial search engine during the same period for user-interest modeling. The detailed statistics of the dataset are shown in Table 3. In addition, we show the length distributions of different types of texts and user behavior histories in Figures 5 and 6, respectively. We assume that different kinds of user behavior data come from independent platforms and they cannot be directly aggregated. In this way, the behavior platforms include (1) the search engine that keeps users’ search behaviors, (2) the web browser platform that keeps users’ webpage behaviors, and (3) the ad display platform that records users’ ad click behaviors (the ad platform itself also serves as a behavior platform).

In our experiments, the word embeddings in the user and ad models were initialized by the pre-trained Glove [30] embeddings. The ad ID embeddings are randomly initialized. In the CTR predictor, following Reference [1], we used dot-product as the CTR prediction function. Each self-attention networks had 16 heads, and the output from each head was 16-dimensional, which means that the total hidden dimension is 256. The query vectors in attention networks were also 256-dimensional. Adam [16] was used as the model optimizer (with a learning rate of 1e-3). The batch size was set to 32. The number of epoch is 2. The dropout [38] ratio after each layer was 20%. The strength of Laplace noise \(\lambda _{LDP}\) in the LDP modules was 0.01, and \(\lambda _{DP}\) in the DP module was 0.005. Hyperparamters were tuned on the validation set. To evaluate the model performance, on the Ads dataset, we used AUC and AP as the metrics. We reported the average results of 10 independent experiments.

We compare the performance of FedCTR with several baseline methods, include:

We report the performance of baseline methods based on user behaviors (i.e., ad clicks) on the ads platform only and their ideal performance using the centralized storage of behavior data from different platforms. The results under different ratios of training data of different methods are shown in Table 4. According to the results, we find the methods using neural networks to learn user and ad representations (e.g., FedCTR) perform better than those using handcrafted features to represent users and ads (e.g., LR and FM). It implies that the representations learned by neural networks are more suitable than handcrafted features in modeling users and ads. In addition, compared with the methods solely based on ad click behaviors on the ad platform for user-interest modeling, the methods that consider multi-platform behaviors in user modeling can achieve better performance. This is because the user behavior data on the ad platform may be sparse, which is insufficient to infer user interest accurately. Since user behaviors on different platforms can provide rich clues for inferring user interest in different aspects. Unfortunately, user behavior data is highly privacy-sensitive and usually cannot be centrally stored or exchanged among platforms due to the constraints of data protection regulations like GDPR and the privacy concerns from users. Thus, in many situations the methods that need the centralized storage of multi-platform user behavior data may not achieve the ideal performance in Table 4. Besides, our FedCTR method consistently outperforms other baseline methods. This is because our framework is more effective in leveraging multi-platform user behaviors for user-interest modeling than other baseline methods, and the multi-head self-attention networks in the ad and user models also have greater ability in learning ad and user representations than other architectures like FM, CNN, and RNN. Moreover, in our approach the raw behavior data never leaves the local platforms, and only user embeddings and model gradients are communicated among different platforms. Thus, our approach can model user interest in a privacy-preserving manner.

Next, we explore the effectiveness of incorporating user behavior data from multiple platforms for CTR prediction. We first study the influence of the number of behavior platforms for user modeling. The average results of FedCTR with different numbers of platforms are shown in Figure 7(a). From the results, we find that the performance improves as number of platforms increases. This is probably because user behaviors on different platforms can provide complementary information to help cover user interest more comprehensively. It shows that incorporating multi-platform user behaviors can effectively enhance user-interest modeling for CTR prediction.

We also study the influence of the number of user behaviors on each platform on the CTR prediction performance. We vary the ratios of user behaviors for user-interest modeling from 20% to 100%, and the results are shown in Figure 7(b). From the results, we find that the performance of FedCTR declines when the number of behaviors decreases. It indicates that it is more difficult to infer user interest when user behaviors are scarce. In addition, we find that the FedCTR method consistently outperforms its variants with single-platform user behaviors, and the advantage becomes larger when user behaviors are scarcer. It shows that utilizing the user behavior data decentralized on different platforms can help model user interest more accurately, especially when user behaviors on a single platform are sparse.

In this section, we verify the effectiveness of our FedCTR method in privacy protection when exploiting multi-platform user behavior data for user modeling. Since user embeddings learned from different platforms may contain private information that can be used to infer the raw behavior data [24], we use LDP and DP to protect the local and aggregated user embeddings, respectively. In our experiments, we observe that the sensitivity of local user embeddings is about 0.05, which means that their privacy budget \(\epsilon\) is 5. The privacy budget of the aggregated user embedding is smaller, because more noise is added. To empirically evaluate the privacy protection performance of these embeddings, we use a behavior prediction task by predicting the raw user behaviors from the local and aggregated user embeddings to indicate their privacy protection ability. The evaluation framework is shown in Figure 8. More specifically, for each user, we randomly sample a real behavior of this user (regarded as a positive sample) and \(P-1\) behaviors that do not belong to this user (regarded as negative samples).⁶ The goal is to infer the real behavior from these P candidate behaviors by measuring their similarities to the user embedding. We perform dot product between the embeddings of each user-behavior pair to compute relevance scores \([z_1, z_2,\ldots , z_P]\), then all candidate behaviors are ranked according to the relevance scores.⁷ We use AUC as the metric to evaluate the privacy protection performance, and lower scores mean better privacy protection.

There are two key hyperparameters in the LDP and DP modules, i.e., \(\lambda _{LDP}\) and \(\lambda _{DP}\), which control the strength of the Laplacian noise added to user embeddings. We first vary the value of \(\lambda _{LDP}\) to explore its influence on privacy protection and CTR prediction, and the results are shown in Figures 9(a) and 10(a), respectively.⁸ We find that although the CTR prediction performance is slightly better if the local user embeddings are not protected by LDP, the raw user behaviors can be inferred to a certain extent, which indicates that the private information of local user embeddings is not fully protected. Thus, it is important to use LDP to protect the local embeddings learned from different behavior platforms. In addition, if \(\lambda _{LDP}\) is too large, then the CTR performance declines significantly, and the improvement on privacy protection is marginal. Thus, we choose a moderate \(\lambda _{LDP}\) (i.e., 0.01) to achieve a trade-off between CTR prediction and privacy protection. Then, we explore the influence of \(\lambda _{DP}\) on the performance of FedCTR in privacy protection and CTR prediction (under \(\lambda _{LDP}=0.01\)), and the results are, respectively, shown in Figures 9(b) and 10(b). We find it is also important to set a moderate value for \(\lambda _{DP}\) (e.g., 0.005) to balance the performance of CTR prediction and privacy protection. Besides, comparing the privacy protecting performance on local and aggregated embeddings, we find that it is more difficult to infer the raw user behaviors on a specific platform from the aggregated user embedding than from local user embeddings. We hypothesise this may be because the aggregated user embedding is a summarization of local embeddings, thus the private information is more difficult to be recovered.

In this section, we present some analysis of the computational and communication cost of FedCTR. Since the self-attention network has quadratic complexity with respect to the input sequence length, the total theoretical computational cost of CTR prediction on each platform is \(O(M^2+MN^2)\), where N is the number of words in a user behavior and M is the number of the behaviors of a user on this platform. In practice, the local training time takes around 2,300 s per epoch (two epochs in total). In our FedCTR model, there are 88.56M parameters in total. Among them, the word embeddings on different platforms take 84.92M parameters, and the Ad ID embeddings have 2.07 parameters. In the model training process, there are 21,580 rounds of communications among the user server and behavior platforms. The overall communication cost in the model training process is 168.59 MB (all parameters are in 32-bit float format), which is mainly handled by the user server. The size of communicated data is quite acceptable for the behavior platforms and user server, which usually have adequate communication bandwidths.

We also verify the effectiveness of the aggregator and CTR predictor models. First, we compare different CTR prediction models like factorization machine (FM) [10], dense layers, outer product [12] and dot product [1], and the results are shown in Figure 11(a). From Figure 11(a), we find the performance of FM is not optimal. It shows that FM may not be suitable for modeling the similarity between the user and ad representations learned by neural networks. In addition, we find that using a dense layer is also not optimal. This may be because dense layers compute the click scores based on the concatenation of ad and user representations while difficult to model their interactions, which is also validated by Reference [36]. Besides, it is interesting that dot product achieves the best performance. This may be because dot product simultaneously models the distance between two vectors as well as their lengths, which can effectively measure the relevance of user and ad representations for CTR prediction. Thus, we prefer dot product for its effectiveness and simplicity.

Then, we compare different models for user embedding aggregation, including attention network, average pooling, max pooling and concatenation. The results are shown in Figure 11(b). We find that average pooling is sub-optimal for aggregation, since it cannot distinguish the informativeness of different local user embeddings. In addition, max pooling is also not optimal, since it only keeps the most salient features. Moreover, although concatenating user embeddings can keep more information, it is inferior to using attention mechanism due to its lack of informativeness modeling. Thus, we use attention networks to implement the aggregator in the user server.

Finally, we study the influence of model size on the CTR prediction performance. We change the model size by varying the hidden dimension in the ad and user models, and the corresponding model performance is shown in Figure 12. We find that the model performance first improves with the increase of hidden dimension of ad and user models, while the performance does not significantly improve or even declines if the hidden dimension is larger than 256. Thus, in FedCTR, we set the hidden dimension to 256 to achieve the best performance.