A Survey on Federated Unlearning: Challenges, Methods, and Future Directions

Existing MU research papers predominantly rely on the following unlearning principles to make the distribution of the model \(\mathcal {M}(D)\) identical to the distribution of the model \(\mathcal {M}(D_r)\) [6].

Retraining. is a process training from a model free from the influence of data from \(D_u\) on the dataset \(D_r\), essentially starting from scratch. In this method, a newly trained model \(\mathcal {M}(D_r)\) does not have any information about \(D_u\). However, this process is both time-consuming and resource-intensive because it discards the model \(\mathcal {M}(D)\) on \(D\) containing the contribution of \(D_r\) dataset.

Fine-tuning. uses the remaining dataset \(D_r\) to optimize the model \(\mathcal {M}(D)\) and reduce the impact of data from \(D_u\). However, this process involves multiple iterations, leading to increased computational and communication costs.

Gradient ascent. represents a reverse learning process. In ML, the model \(\mathcal {M}(D)\) is trained by minimizing the loss using gradient descent. Conversely, the unlearning process involves the application of gradient ascent to maximize the loss. However, this method can easily lead to catastrophic forgetting. As a result, many studies introduce constraints to preserve memory.

Multi-task unlearning seeks to not only eliminate the influence of \(D_u\) but also to reinforce the acquisition of knowledge from the remaining data \(D_r\). In the course of these endeavors, most studies aim at striking a balance between the erasure effect and the retention effect.

Model scrubbing. applies a “scrubbing” transformation \(\mathscr{H}\) to the model \(\mathcal {M}(D)\) to ensure that the unlearned model closely approximates the perfectly retrained model with only \(D_r\), as expressed by \(\mathscr{H}(\mathcal {M}(D)) \approx \mathcal {M}(D_r)\) [30]. When defining the scrubbing method \(\mathscr{H}\), most approaches rely on a quadratic approximation of the loss function. Specifically, for model parameters \(\theta\) and \(\phi\), the gradient of the loss function of a given data point \(D_x\) satisfieswhere \(\mathcal {H}_{D_x}(\theta)\) is positive semi-define. The scrubbed model becomes the new optimum by setting \(\nabla f_{D_r}(\mathscr{H}_{D_r}(\theta)) = 0\), yielding the equation:\(\mathscr{H}\) can perform a Newton step and can be derived under various theoretical assumptions [25] [31]. However, the challenge of this method lies in computing the Hessian matrix, which is infeasible for high-dimensional models. Therefore, some approaches aim at computing an approximation of the Hessian.

Synthetic data. is a method that replaces certain data with synthetic data to help the model “forget” specific information. An example of this approach involves generating synthetic labels for the data within \(D_u\) and then combining them with the data in \(D_u\) for training to accomplish unlearning. This method disentangles the impact of certain data from the model, helping to eliminate the influence of specific information while retaining the model’s overall performance.

Verification methods aim at confirming whether data intended for deletion has indeed been effectively unlearned. Currently, these methods can be classified as outlined below:

Model performance. The most straightforward approach is to evaluate the model performance on the target client’s data and test data to assess how effectively the data has been unlearned and how robustly the unlearned model is maintained. The evaluation metrics encompass accuracy, loss, and statistical errors.

Model discrepancy. Another approach to assess unlearning performance is by evaluating the discrepancy between the trained model and the unlearned model. This discrepancy can be measured using metrics such as Euclidean distance, KL-divergence, L2 distance, Wasserstein distance, and angle-based distance.

Execution efficiency. In addition, the time taken for the unlearning process, measured in terms of rounds, runtime, or speed-up ratio compared with a baseline, as well as memory consumption, can be used to evaluate the efficiency of the unlearning algorithm.

Attack performance. As introduced in Section 3.4, membership inference attacks can be used to determine whether a particular data was used during the training of a model. Therefore, by executing MIA on the unlearned model over unlearned data, the attack success rate (ASR) can be used to evaluate how effectively the data has been unlearned. Poorer performance by the MIA indicates that the influence of the unlearned data on the global model has diminished. Similarly, in the context of backdoor attacks, by injecting backdoors into the unlearned data and following the unlearning procedure, effective unlearning should disrupt the relationship between the trigger pattern and the backdoor class. The ASR can also be used to evaluate how effectively the backdoor is removed by unlearning. An empirical study on these metrics can be referred to in Reference [94].

The participants involved in federated learning [56, 92, 139] can be categorized into two categories: (i) a set of \(n\) clients denoted as \(\mathcal {U}={u_1,u_2,\ldots , u_n}\), where each client \(u_i \in \mathcal {U}\) possesses its local dataset \(\mathcal {D}_i\), and (ii) a central server represented as \(S\). A typical FL scheme works by repeating the following steps until training is stopped [56]. (i) Local model training: each FL client \(u_i\) trains its model \(\mathcal {M}_i\) using the local dataset \(\mathcal {D}_i\). (ii) Model uploading: each FL client \(u_i\) uploads its locally trained model \(\mathcal {M}_i\) to the central server \(S\). (iii) Model aggregation: the central server \(S\) collects and aggregates clients’ models to update the global model \(\mathcal {M}\). (iv) Model updating: the central server \(S\) updates the global model \(\mathcal {M}\) and distributes it to all FL clients.

The revelation of a participant’s local model poses a direct threat to the fundamental privacy guarantee of standard federated learning [155]. Thus, privacy-preserving aggregation protocols [4, 5, 40, 84] are essential to maintain the security and privacy of the model aggregation process in Step iii of FL. Additionally, FL is susceptible to poisoning attacks [89] (see Section 3.4 for more details). In these attacks, malicious clients manipulate the global model by sending poisoned model updates to the server during Step ii, to affect global model performance or inject backdoors. Therefore, malicious-client detection mechanisms [7, 59, 109, 149] are imperative to differentiate between malicious and benign clients.

First proposed by Shokri et al. [110], the fundamental idea behind MIA is to determine whether a particular record was used during the training of a target model. This is predicated on the observation that data samples present in the training set will lead the model to produce outputs with higher confidence scores. Consequently, an adversary can train a separate model for binary classification, designating outputs as either “member” (indicating that the data was part of the training set) or “non-member” (indicating that the data was not part of the training set). This potential to distinguish between member and non-member records poses a threat to data privacy. Remarkably, MIA does not require knowledge of the target model’s specific architecture or the distribution of its training data. Relying on the shadow models, a series of shadow training datasets \(D^{\prime }_1, \ldots , D^{\prime }_k\) and disjointed shadow test datasets \(T^{\prime }_1, \ldots , T^{\prime }_k\) can be synthesized to mimic the behavior of the target model so as to train the attack model. Evaluating the unlearning effectiveness can be achieved by performing MIA on the unlearned model with the unlearned data. The ASR serves as an indicator of how well the data has been unlearned. A decrease in the MIA’s performance suggests that the influence of the unlearned data on the global model has been successfully reduced.

Backdoor attacks embed a distinct pattern or “trigger” into portions of the training data [65]. The trigger can be a small patch or sticker that is visible to humans [39, 73], or the value perturbation of benign samples indistinguishable from human inspection [2, 61, 104]. When the model is subsequently trained or fine-tuned on this, its behavior remains typical for standard inputs. Yet, upon detecting an input that contains this covert trigger, the model will yield malicious behaviors that align with the attacker’s intentions. Backdoor attacks are particularly concerning because they can remain dormant and undetected until the attacker chooses to exploit them. In the context of unlearning, injecting backdoors into the unlearned data and then applying the unlearning procedure should effectively disrupt the relationship between the trigger pattern and the backdoor class. The ASR can be utilized to assess the effectiveness of the unlearning process in removing the backdoor. A lower ASR would indicate that the backdoor has been successfully eliminated.

As previously mentioned, standalone server unlearning typically depends on the utilization of stored historical data, which may include gradients, global models [41, 49, 54, 129, 130, 146], contribution information [146], or intermediate information necessary for constructing a random forest [75]. This category necessitates a significant amount of memory on the server, potentially limiting its practical application in large-scale FL systems with complex ML models.

In the case of FedRecovery [146], the server retains historical data from all clients and quantifies their contributions in each round based on gradient residuals. When a target client requests to leave, the server systematically removes its contributions from all FL rounds through a fine-tuning process. Based on FedRecovery [146], Crab [54] achieves a more efficient recovery based on (i) selective historical information rather than all historical information and (ii) a historical model that has not been significantly affected by malicious clients rather than the initial model. Additional constraints can be introduced to further guide the recovery process, such as a penalty term based on projected gradients [26, 107], randomly initialized degradation models [152], and estimated skew [49]. The approach of eliminating the contribution of the target client is more straightforward in [129, 130], where the server directly averages the models of the remaining clients. Strategic retraining based on the change of sampling probability is adopted for fast and efficient recovery [113]. To mitigate the potential decrease in accuracy due to the averaging process in the averaged model, knowledge distillation is employed. This technique facilitates the transfer of information from the trained model to the unlearned model, helping to preserve performance. Consequently, these designs adhere to a multi-task unlearning approach. In VERIFI [28], after receiving the gradients from all clients, including those from the target client, are uploaded to the server. The server then amplifies the gradients from the remaining clients and diminishes the gradients of the target client, to reduce the impact of target clients, hence achieving unlearning.

Apart from the above-mentioned works, RevFRF [75], as shown in Figure 6, focuses on federated random forest training. To remove a client, the server first identifies nodes affected by the target client and subsequently eliminates these affected nodes until reaching the leaf node. Subsequently, the server reconstructs the affected branches using previously stored intermediate information, rather than instructing the remaining clients to undergo retraining. In particular, in the worst-case scenario where the revoked node is the root node, the server has to reconstruct the entire random decision tree. Unlearning within a federated clustering setting is explored in SCMA [97], where each client maintains a vector to denote its local clustering result. These vectors are then aggregated by the server to form a global clustering outcome. Eliminating a client is straightforward by assigning a zero vector to the unlearned client and then re-aggregating all vectors.

Limitations. Server-standalone unlearning, which relies on historical data stored on the server, lacks real-time input from remaining clients during the unlearning process. This limitation may result in slightly lower unlearning performance compared with client-aided unlearning. This characteristic could impede the applicability of server-standalone unlearning in complex ML models or Non-IID FL settings, where there is a notable bias, affecting the overall efficacy of the unlearning process. Furthermore, server-standalone unlearning may lack responsiveness to changes in data and client behavior, as it solely relies on historical information. This can limit its adaptability in dynamic environments where real-time data and client interactions are crucial.

Unlearning performed by the server and remaining clients typically offers greater potential compared with standalone server unlearning. This is because the remaining clients contribute valuable information about the remaining data, which enables the server to enhance its unlearning process. In this context, the server may or may not have access to historical information.

This research direction is arguably pioneered by the design of FedEraser [71], as shown in Figure 7. The core concept of FedEraser is that the current global model can be reconstructed using only the initial model and historical clients’ gradients at each round. Consequently, unlearning boils down to eliminating the influence of the target client on the historical gradients, i.e., calibrating historical gradients. To achieve this goal, for a historical FL round \(i\) with the stored gradients \(G_i\) and a calibrated global model \(M_{i-1}\) for the previous round, each remaining client \(u_j\) calculates a local calibration direction \(c_i^j\) based on its local data \(D_j\) and \(M_{i-1}\). The local calibration directions are then aggregated by the server to derive a global calibration direction \(c_i\), which enables the server to calculate calibrated historical gradients \(G_i^{\prime }\) and to obtain a calibrated global model \(M_i\) via \(G_i^{\prime }\). This iterative process continues round-by-round until all historical gradients are successfully calibrated, resulting in the server obtaining a final calibrated global model, eliminating the influence of the target client. To enhance unlearning efficiency, the server stores clients’ gradients at intervals of every \(\Delta t\) rounds, leading to a tradeoff between unlearning performance and resource consumption in terms of memory and computation. Improve upon FedEraser, Crab [54] and Fast-FedUL [49] optimize storage efficiency by selectively storing important gradients, while Sharding Eraser [67] compress storage using coding-based techniques. A similar idea is adopted in [120] focusing on ranking tasks instead of classification tasks. Building upon the unlearning concept introduced in FedEraser, an efficiency-enhancing technique is employed in FRU [142] for federated recommendations. In FRU, only the important updates to clients’ item embeddings are stored. In line with FedEraser [71] and FRU [142], FedRecover [8] also entails the storage of historical gradients and global models. In FedRecover, to prevent the remaining clients from computing exact model updates for fine-tuning, which can lead to significant computational overhead, the server calculates updates for the remaining clients using historical gradients and global models, as described below:where \(\mathbf {H}_t^i=\int _0^1 \mathbf {H}\left(\overline{\boldsymbol {w}}_t+z\left(\hat{\boldsymbol {w}}_t-\overline{\boldsymbol {w}}_t\right)\right) d z\) is an integrated Hessian matrix for the \(i\)th client in the \(t\)th round. Denote the global-model difference in the \(t\)th round as \(\Delta \boldsymbol {w}_t=\hat{\boldsymbol {w}}_t-\overline{\boldsymbol {w}}_t\) and the model-update difference of the \(i\)th client in the \(t\)th round as \(\Delta \boldsymbol {g}_t^i=\boldsymbol {g}_t^i-\overline{\boldsymbol {g}}_t^i\). The Hessian matrix \(\tilde{\boldsymbol {H}}_t^i\) for the \(i\)th client in the \(t\)th round can be approximated aswhere \(\Delta \boldsymbol {W}_t=\left[\Delta \boldsymbol {w}_{b_1}, \Delta \boldsymbol {w}_{b_2}, \ldots , \Delta \boldsymbol {w}_{b_s}\right]\) and \(\Delta \boldsymbol {G}_t^i=\left[\Delta \boldsymbol {g}_{b_1}^i, \Delta \boldsymbol {g}_{b_2}^i, \ldots , \Delta \boldsymbol {g}_{b_s}^i\right]\) are L-BFGS buffers [96] maintained by the server. Nonetheless, these approximations introduce estimation errors over rounds. Therefore, the remaining clients are periodically tasked with computing their exact model updates to correct these approximations, based on an adaptive abnormality threshold. A more straightforward retraining-based method is employed in SIFU [25]. In SIFU, the fundamental concept of unlearning is to identify the most recent global model using a bounded sensitivity metric calculated from historical contributions. Subsequently, the remaining clients retrain based on the identified model.

Other approaches concentrate on unlearning without reliance on historical updates. In SFU [58], upon receiving an unlearning request, the server refines the global model using gradient information provided by the target client and representation matrix information provided by other clients. In KNOT [112], clients are grouped into clusters based on their training time and model sparsity. Clients within the same cluster collectively aggregate and update the global model asynchronously. When a client requests to leave, only clients within the same cluster require retraining. A similar structure is adopted in FedCIO [100] where clients are clustered according to their data distribution. Differing from conventional FL systems, HDUS [141] operates without a central server, as shown in Figure 8. Instead, each client possesses their own neighboring distilled models, referred to as seed models. When a client requests to leave, the adjacent clients simply delete the seed model of the unlearned client. For predictions, an ensemble learning method is employed to combine the outputs of the primary model with those of the seed models. Incentive mechanisms along with game theoretical analysis in FU systems is presented in References [19, 20, 68].

Client-aided unlearning inherently depends on the involvement of remaining clients and their updates, which can be a vulnerability in dynamic environments where client participation fluctuates. Additionally, this unlearning method can be slow, as it relies on all remaining clients, often resource-constrained devices, and is limited by the bandwidth of the FL system. This could lead to inefficiencies, particularly in cross-device scenarios with frequent client turnover or limited system resources.

To unlearn the partial data of the target client, retraining is the most straightforward approach. To mitigate the computational cost of starting from scratch with retraining, one solution is to roll back the global model to a state where it has not been significantly influenced by the target client. From this point, all FL clients can conduct the retraining process. For instance, in Exact-Fun [134], where FL models are quantized, when a client requests to leave, the client calculates a new model based on the remaining data. If the original model matches the new quantized model, signifying that the removal has no impact, the FL model remains unchanged. Otherwise, retraining is required to eliminate the influence of the unlearned data. In ViFLa [23], which is essentially a machine unlearning scheme, training samples are segmented into different groups, with each group representing an FL client. Hence, ViFLa can simulate an FU process in this context. The local model is trained using ring-based SQ-learning for LSTM, and weighted aggregation is determined by KL-attention scores. The historical model parameters represented by states over a ring are stored. To unlearn partial data, each client removes unlearned data and computes the new updates. Based on these new updates, the server identifies a previous state from which the remaining clients continue their training. A similar concept of identifying the optimal previous state for retraining is also present in SIFU [25], as discussed earlier in Section 4.1.2. In SCMA [97], a straightforward approach to unlearning partial data involves naive retraining. Each client maintains a vector representing its local clustering result, and these vectors are aggregated by the server to create a global clustering outcome. To unlearn partial data, SCMA entails each client calculating a new local vector, i.e., retraining, and then re-aggregating all vectors.

Fine-tuning and multi-task learning are popular approaches for FU as well. As an example, in FRAMU [105], the server aggregates fine-tuned local models and attention scores. Using these scores, it filters out irrelevant data points and updates the global model. The attention scores are acquired through local reinforcement learning applied to dynamic data. In FedLU [156], designed for FL over knowledge graphs where embeddings are aggregated instead of gradients as in standard FL, the unlearning of partial data is accomplished through iterative optimization of local embeddings. This process follows the multi-task unlearning concept, involving unlearning over the local model and learning over the global model. In FedME\(^2\) [133], clients engage in multi-task learning to optimize the loss of the local model, the loss from an MIA-like evaluation model, and a penalty term that accounts for the difference between the local model and the global model. In [115], unlearning is conducted by optimizing model performance on the remaining dataset while considering bias caused by the unlearned data.

Model scrubbing-based methods are commonly employed for unlearning partial data. In [76], the model scrubbing technique is applied to the target client to locally unlearn the partial data, involving Hessian matrix computations, with enhanced computational efficiency through an approximate diagonal empirical Fisher Information Matrix (FIM). In Forsaken [74], dummy gradients are computed to align the confidence vector of the unlearned model with that of a perfectly unlearned model. Forsaken+ [91] minimizes the distance between the posteriors of the data to be forgotten and those of non-member data for unlearning. FedAU [38] relies on the linear combination to approximate the unlearned model utilizing a pre-computed auxiliary model during the learning process. Reference [37] focus on feature unlearning by minimizing local feature sensitivity through model scrubbing. A similar approach of local unlearning followed by aggregation is described in CONFUSE [93] for multi-task unlearning at different levels. FFMU [9] treats data removals as perturbations on the dataset, employing random smoothing (RS) [15] to obtain a smoother model to simulate an unlearning process. In particular, FFMU aligns with the fundamental idea presented in PCMU [150], which involves randomized gradient smoothing combined with gradient quantization as follows.where \(\mathcal {D}=\mathcal {N}\left(0, \sigma ^2 I\right)\) is a Gaussian distribution, \(Q\) is a gradient quantization to map each dimension of the continuous gradient \(G(x, y) \in \mathbb {R}^T\) over a discrete three-class space \(\lbrace -1,0,1\rbrace\), for mimicking the classification in the randomized smoothing for certified robustness. \(S\) is a smooth version of \(Q\), and returns whichever gradient classes \(Q^t\) is most likely to return when \(\bar{G}\) is perturbed by noise \(\varepsilon\). Extending FFMU from PCMU poses a challenge due to FL’s privacy requirements, limiting server access to clients’ local training data and affecting the certified data removal radius and budget in the global model. Therefore, by leveraging the theory of nonlinear functional analysis, the local MU models \(g(x;q)\) in FFMU are reformulated as output functions of a Nemytskii operator \(O(q)(x)\) where \(q=Q(\bar{G})+\varepsilon\). In this way, the global unlearned model with bounded errors can maintain the certified radius and budget of data removals of the local unlearned models within a distance \(\frac{(K-1)Cd}{\sqrt {2\pi }K\delta }\) (see Reference [9] for more details).

Some FU approaches are proposed with the use of synthetic data and gradient ascent principles. For instance, in UKRL[137], unlearning is conducted by training on perturbed unlearned data. In FedAF [63], shown in Figure 9, synthetic labels are generated for the data to be unlearned. A trusted third party creates random teacher models, and ensemble predictions from these models to provide synthetic labels for the unlearned data. Training is then conducted using this data with synthetic labels to achieve unlearning. Note that a multi-task approach is also employed in FedAF, where an additional task is introduced to retain the memory of the remaining data. In another work [1], which focuses on how adversaries can stealthily perform backdoor unlearning to evade server detection, the gradient ascent method is employed. The loss from the local benign dataset is utilized to constrain unbounded losses during the unlearning process based on gradient ascent.

Unlearning is also applicable in Bayesian federated learning systems. In Forget-SVGD [34], when a client requests to leave, the target client leverages the remaining data to compute the posterior probability, approximated through variational inference (VI), and performs an extra FL round for model updating. Similar unlearning methods are employed in [33, 36]. BVIFU [35] shares a core concept with Forget-SVGD, employing exponential family distributions in VI to approximate posterior probability. Unlike retraining-based unlearning in Bayesian FL, BFU [122] introduces a multi-task unlearning approach. It employs a parameter self-sharing method to balance between forgetting the unlearned data and remembering the knowledge learned by the original model, where probability distributions are approximated by a neural network.

In addition to the aforementioned approaches, the work in Reference [116] specializes in unlearning a specific type of partial data, particularly focusing on a category within the training dataset, i.e., a class. Typically initiated by the server, this unlearning process involves the application of quantization and pruning techniques. Specifically, the locally trained CNN model takes private images as input and produces feature map scores that assess the relationship between each channel and category. These scores are transmitted to the central server and aggregated into global feature map scores. The server utilizes TF-IDF to evaluate the relevance scores between channels and categories and creates a pruner to perform selective pruning on the most distinguishing channels of the target category. Subsequently, normal federated training proceeds with the exclusion of training data associated with the target category.

Limitations. Unlearning partial data through active unlearning is notably complex among all unlearning targets. It requires the removal of specific data while maintaining model performance on the remaining data. This complexity often results in more intricate algorithms, leading to increased computational and communication costs. Furthermore, this setting is susceptible to attacks based on over-unlearning, as identified in [44], where adversaries can exploit the unlearning process to enhance the effects of remaining poisoned data, thus facilitating poisoning attacks. This vulnerability underscores the need for careful consideration in implementing active partial data unlearning.

As mentioned earlier, unlearning through gradient ascent is a versatile approach suitable for both partial data and target client unlearning. As detailed in Reference [42], the target client employs gradient ascent to maximize the local loss, subject to constraints determined by the reference model provided by the remaining clients. Specifically, during FL training, a client’s objective is to address the following optimization problem:where \(\mathcal {L}(w;(x_i,y_i))\) represents the loss function, which calculates the prediction error for an individual example \((x_i,y_i)\) from the dataset \(D\), using the model parameters \(w\). The unlearning method designed in Reference [42] is to reverse this learning process. That is, during unlearning, instead of learning model parameters that minimize the empirical loss, the client \(i\) strives to learn the model parameters to maximize the loss. Additionally, to prevent the process of gradient ascent from producing an arbitrary model similar to a random model, the average of the other clients’ models, i.e., \(w_{ref}=\frac{1}{N-1}\sum \nolimits _{i \ne j}w_j\), is used as a reference model, and an \(\ell _2\)-norm ball around the reference model is employed to limit the unbounded loss. Thus, during unlearning, the client solves the following optimization problem:

Similarly to constraints, concurrent boost training on the remaining data [117] and reference models [3] can also be adopted as mitigation methods. To reduce the computation overhead, unlearning is conducted based on gradient ascent on distilled dataset [17]. Apart from the unlearning principle based on gradient ascent, 2F2L [55] and Appro-Fun [134] adopt model scrubbing methods similar to previous works such as [76] and [8]. To enhance computational efficiency, the complex task of Hessian matrix inversions is approximated using a pre-trained deep neural network and Taloy expansion.

Limitations. Unlearning an entire client through active learning poses limitations in scenarios where the unlearning request comes from the client to be unlearned. In such cases, the client must remain in the system for unlearning, conflicting with the common “request then leave” behavior in FL systems. Moreover, this method necessitates the unlearned client’s continued participation alongside remaining clients, which is not cost-effective, especially in large-scale and resource-constrained FL systems.

To ensure the “right to verify,” it is imperative that federated unlearning schemes provide clients with the ability to confirm the successful unlearning of their data. Regrettably, this aspect has received limited attention in existing FU literature. In EMA [47], the target client employs various metrics, including correctness, confidence, and negative entropy, to assess the performance of the audited dataset concerning the global model. These metrics are then ensembled to determine whether they meet a predefined threshold, serving as an indicator of whether the target client’s data has been effectively unlearned.

Another noteworthy contribution in this domain is VERIFI [28]. VERIFI introduces two non-invasive verification methods that distinguish themselves from invasive techniques involving the injection of backdoors or watermarks, which manipulate the original data. In contrast, the verification methods proposed in VERIFI operate without modifying the data itself. These methods revolve around tracking a subset of unlearned data known as “markers,” selected based on specific criteria. The criteria encompass two primary categories: (i) Forgettable Memory, where markers are identified as a representative subset incurring a high variance of local training loss, and (ii) Erroneous Memory, which designates markers as incorrectly predicted samples labeled as erroneous. By actively monitoring the unlearned model’s performance on these markers, clients can effectively verify whether the unlearning process has successfully removed the unlearned data.

The outcome of the verification conducted by the server plays an important role in determining when to stop the unlearning process within the FU system. For instance, FRAMU [105] terminates unlearning when the difference between two consecutive global models becomes smaller than a predefined parameter. KNOT [112] concludes unlearning based on the required validation accuracy and the standard deviation across a recent history of such validation accuracies. ViFLa [23] and SCMA [97] end unlearning when the model converges. FedLU [156] relies on prediction results derived from knowledge, while FFMU [9] assesses whether data removals exceed a certified budget.

While the remaining FU works reviewed in this survey do not explicitly introduce verification mechanisms, the verification metrics employed in these works to assess unlearning performance could provide valuable insights for future research. We have summarized these metrics adopted in reviewed FU works in Table 3.

Accuracy-based metrics over unlearned data are the most commonly utilized metrics in the reviewed FU works. For example, they are used in works such as [1, 8, 25, 42, 55, 63, 71, 76, 122, 130, 133, 146, 156]. Metrics based on running time are employed to assess the efficiency of unlearning algorithms, as demonstrated in References [8, 25, 63, 71, 74, 76, 91, 97, 122, 133, 146]. Furthermore, some works rely on verification through backdoor attacks, as in [1, 8, 42, 58, 91, 130, 142], while others use membership inference attacks, as seen in References [55, 71, 91, 146]. The difference between the unlearned model and the retrained model is adopted to evaluate unlearning performance in References [36, 91, 122].

By surveying the existing FU literature, we can make several observations regarding verification as follows:

These observations indicate the need for research into “who-verify” and the development of efficient and robust verification methods conducted by different participants, especially client-verify methods. For instance, when considering MLaaS with unlearning services, clients must be allowed to verify if their data has been unlearned and its impact on the FL model has been removed. Only when the data removal adheres strictly to the specified unlearning request can the trustworthiness of the federated unlearning system be maintained.

In many FU works, historical information is essential for facilitating the unlearning process. Nevertheless, this implies that the server needs to store a substantial volume of data, leading to significant memory consumption. This historical information can be gradients and global models [8, 25, 42, 71, 130, 142], gradient residuals [146], specific state [105], or some intermediate results [75].

Approaches to mitigation. Memory consumption reduction can be achieved by selectively storing historical information. For instance, in FedEraser [71], the server stores clients’ gradients at specific intervals of FL rounds or based on the importance of gradients [54]. Similarly, in FRU [142], only important updates to clients’ item embeddings are stored. In addition, by adopting coding-based techniques, the storage can be further compressed as demonstrated in Reference [67].

Unlearning in the FU system typically necessitates FL clients to transmit extra information, such as gradients, to the server to facilitate the process. For example, in FRAMU [105], clients additionally send attention scores to the server. In FedLU [156], loss information is transferred for mutual knowledge distillation between the server and clients. In SFU [58], the remaining clients need to send their representation matrix to the server. In Reference [42], the remaining clients’ models are sent to the target client as references.

Approaches to mitigation. Reducing the communication cost can be achieved by minimizing the size of the model to be transferred, which may involve methods like quantization [134], and by reducing the number of unlearning rounds. For retraining-based unlearning, the FU system may roll back the global model to a state where it has not been significantly influenced by the target client. From this point, all FL clients can conduct the retraining process [23, 25, 134], thus reducing the unlearning rounds and enhancing communication efficiency. Besides, clustering is used to divide FL users into groups, each with its own model. The final inference is determined by a majority vote from these sub-models. This method confines unlearning processes to individual clusters, eliminating the need for participation from all users, thus improving efficiency [83, 100, 112, 124].

FU often requires clients to engage in additional computational tasks compared with standard FL. These tasks can involve generating dummy gradients [74], conducting online reinforcement learning [105], seed model generation [141], or computing Hessian matrices [8, 55, 63, 76]. It’s important to note that some of these computational tasks can be resource-intensive, which could pose challenges for deploying unlearning mechanisms in FL systems.

Approaches to mitigation. To enhance computational efficiency, approximation methods are commonly employed to accelerate certain components of FU algorithms. For example, the computation on the Hessian matrix can be approximated using techniques like a pre-trained deep neural network with Taylor expansion [55], the L-BFGS algorithm [8], or the FIM [63, 76]. Additionally, other optimizations are based on different FU structures. For instance, some works conduct training and unlearning simultaneously [9], similar to the approach used in Reference [150] in an MU setting. Orthogonally, computation tasks can be outsourced to a trusted third party, as demonstrated in Reference [63]. Another approach is transferring the majority of tasks to the server for estimation while executing only a small part of computation tasks for calibration [8], or reducing the computational tasks in the unlearning process by involving some pre-computation during the learning phase [38]. In Reference [17], dataset distillation is adopted to compress the size of the dataset while preserving the unlearning performance, hence reducing the computational overhead.