Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

ABT-SVDD: : A method for uncertainty handling in domain adaptation using belief function theory

Published: 01 November 2023 Publication History

Abstract

Domain adaptation involves adapting a model trained on one domain to work effectively on another, which can have different statistical properties, such as distributions, correlations, and relationships between features. These heterogeneities can lead to uncertainty, impacting the model’s performance. Despite many studies that have been done on domain adaptation, most have ignored the adverse impact of uncertain and noisy data on adaptation and classification. To address this issue, the proposed method, Adaptive Belief-based Twin Support Vector Data Description (ABT-SVDD), extends the one-class support vector data description (SVDD) to an adaptive twin classifier and integrates it with a belief-based sample weighting approach. Also, it utilizes a combination of Hermite polynomial and Gaussian kernels to enhance the computational power of the linear objective function of the SVDD classifier while improving the generalization capability. The effectiveness of ABT-SVDD has been compared to the state-of-the-art methods on several tasks taken from two benchmark datasets. The experimental results demonstrate that ABT-SVDD significantly improves classification accuracy on various tasks with varying amounts of labeled data in the target domain. Specifically, in normal situations, ABT-SVDD outperforms competing methods by 6.33% to 9.08%, while in noisy situations, it achieves a more significant improvement of 9.87% compared to the competing methods. Besides, the Wilcoxon statistical test demonstrates the superiority of ABT-SVDD over state-of-the-art ones in terms of classification accuracy and computational time.

Highlights

A new dual-domain adaptation method is introduced that can be used in noisy and uncertain environments.
By incorporating a belief theory, a weighting strategy has been introduced to quantify the uncertainty in the domains.
Noise robustness has been evaluated to demonstrate the performance of the proposed method.
The effectiveness of the proposed method has been compared to the state-of-the-art methods on fifteen different datasets.
The experimental results demonstrate the superiority of the proposed method over state-of-the-art ones.

References

[1]
He C., Zheng L., Tan T., Fan X., Ye Z., Multi-attention representation network partial domain adaptation for COVID-19 diagnosis, Appl. Soft Comput. 125 (2022).
[2]
Hong J., Yu S.C.-H., Chen W., Unsupervised domain adaptation for cross-modality liver segmentation via joint adversarial learning and self-learning, Appl. Soft Comput. 121 (2022).
[3]
Ma W., Liu R., Guo J., Wang Z., Ma L., A collaborative central domain adaptation approach with multi-order graph embedding for bearing fault diagnosis under few-shot samples, Appl. Soft Comput. 140 (2023).
[4]
Liu H., Liu Q., Li P., Zhao P., Wu X., A deep selective learning network for cross-domain recommendation, Appl. Soft Comput. 125 (2022).
[5]
Pan T., Chen J., Ye Z., Li A., A multi-head attention network with adaptive meta-transfer learning for RUL prediction of rocket engines, Reliab. Eng. Syst. Saf. (2022).
[6]
Zhao P., Gao H., Lu Y., Wu T., A cross-media heterogeneous transfer learning for preventing over-adaption, Appl. Soft Comput. 85 (2019).
[7]
Pan S.J., Yang Q., A survey on transfer learning, IEEE Trans. Knowl. Data Eng. 22 (2010) 1345–1359.
[8]
Singhal P., Walambe R., Ramanna S., Kotecha K., Domain adaptation: Challenges, methods, datasets, and applications, IEEE Access (2023).
[9]
Zhang L., Gao X., Transfer adaptation learning: A decade survey, IEEE Trans. Neural Netw. Learn. Syst. (2022).
[10]
Y. Zhang, D.-Y. Yeung, Transfer metric learning by learning task relationships, in: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2010, pp. 1199–1208.
[11]
Xu Y., Pan S.J., Xiong H., Wu Q., Luo R., Min H., Song H., A unified framework for metric transfer learning, IEEE Trans. Knowl. Data Eng. 29 (2017) 1158–1171.
[12]
Chang H., Zhang F., Ma S., Gao G., Zheng H., Chen Y., Unsupervised domain adaptation based on cluster matching and Fisher criterion for image classification, Comput. Electr. Eng. 91 (2021).
[13]
Chen S., Han L., Liu X., He Z., Yang X., Subspace distribution adaptation frameworks for domain adaptation, IEEE Trans. Neural Netw. Learn. Syst. 31 (2020) 5204–5218.
[14]
Li S., Song S., Huang G., Prediction reweighting for domain adaptation, IEEE Trans. Neural Netw. Learn. Syst. 28 (2016) 1682–1695.
[15]
Khalighi S., Ribeiro B., Nunes U.J., Importance weighted import vector machine for unsupervised domain adaptation, IEEE Trans. Cybern. 47 (2016) 3280–3292.
[16]
J.N. Kundu, N. Venkat, R.V. Babu, Universal source-free domain adaptation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 4544–4553.
[17]
Liang J., Hu D., Feng J., Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation, in: International Conference on Machine Learning, PMLR, 2020, pp. 6028–6039.
[18]
J.N. Kundu, N. Venkat, A. Revanur, R.V. Babu, Towards inheritable models for open-set domain adaptation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 12376–12385.
[19]
R. Li, Q. Jiao, W. Cao, H.-S. Wong, S. Wu, Model adaptation: Unsupervised domain adaptation without source data, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 9641–9650.
[20]
J. Yang, R. Yan, A.G. Hauptmann, Cross-domain video concept detection using adaptive svms, in: Proceedings of the 15th ACM International Conference on Multimedia, Association for Computing Machinery, Augsburg, Germany, 2007, pp. 188–197.
[21]
J. Yang, R. Yan, A.G. Hauptmann, Adapting SVM Classifiers to Data with Shifted Distributions, in: 17th IEEE International Conference on Data Mining Workshops, ICDMW 2007, 2007, pp. 69–76.
[22]
J. Wei, E. Zavesky, C. Shih-Fu, A. Loui, Cross-domain learning methods for high-level visual concept classification, in: 15th IEEE International Conference on Image Processing, 2008, pp. 161–164.
[23]
Y. Aytar, A. Zisserman, Tabula rasa: Model transfer for object category detection, in: International Conference on Computer Vision, 2011, pp. 2252–2259.
[24]
Matasci G., Tuia D., Kanevski M., SVM-based boosting of active learning strategies for efficient domain adaptation, IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 5 (2012) 1335–1343.
[25]
Sun Z., Wang C., Wang H., Li J., Learn multiple-kernel SVMs for domain adaptation in hyperspectral data, IEEE Geosci. Remote Sens. Lett. 10 (2013) 1224–1228.
[26]
B. Sun, J. Feng, K. Saenko, Return of frustratingly easy domain adaptation, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2016.
[27]
Xue Y., Beauseroy P., Transfer learning for one class SVM adaptation to limited data distribution change, Pattern Recognit. Lett. 100 (2017) 117–123.
[28]
Lv Y., Zhang B., Zou G., Yue X., Xu Z., Li H., Domain adaptation with data uncertainty measure based on evidence theory, Entropy 24 (2022) 966.
[29]
Jayadeva, Khemchandani R., Chandra S., Twin support vector machines for pattern classification, IEEE Trans. Pattern Anal. Mach. Intell. 29 (2007) 905–910.
[30]
Ding S., Zhang N., Zhang X., Wu F., Twin support vector machine: theory, algorithm and applications, Neural Comput. Appl. 28 (2017) 3119–3130.
[31]
Li Y., Sun H., Yan W., Domain adaptive twin support vector machine learning using privileged information, Neurocomputing 469 (2022) 13–27.
[32]
Maddox W.J., Izmailov P., Garipov T., Vetrov D.P., Wilson A.G., A simple baseline for bayesian uncertainty in deep learning, Adv. Neural Inf. Process. Syst. 32 (2019).
[33]
Pan L., Deng Y., A new belief entropy to measure uncertainty of basic probability assignments based on belief function and plausibility function, Entropy 20 (2018).
[34]
Jiroušek R., Shenoy P.P., A new definition of entropy of belief functions in the Dempster–Shafer theory, Internat. J. Approx. Reason. 92 (2018) 49–65.
[35]
Fu C., Chang W., Liu W., Yang S., Data-driven group decision making for diagnosis of thyroid nodule, Sci. China Inf. Sci. 62 (2019).
[36]
Liu Z.G., Pan Q., Dezert J., Martin A., Combination of classifiers with optimal weight based on evidential reasoning, IEEE Trans. Fuzzy Syst. 26 (2018) 1217–1230.
[37]
Bossman A., Umar Z., Agyei S.K., Junior P.O., A new ICEEMDAN-based transfer entropy quantifying information flow between real estate and policy uncertainty, Res. Econ. (2022).
[38]
Zadeh L.A., Fuzzy sets, Inf. Control 8 (1965) 338–353.
[39]
Xiao F., Ding W., Divergence measure of pythagorean fuzzy sets and its application in medical diagnosis, Appl. Soft Comput. 79 (2019) 254–267.
[40]
Kang B., Deng Y., Hewage K., Sadiq R., A method of measuring uncertainty for Z-number, IEEE Trans. Fuzzy Syst. 27 (2018) 731–738.
[41]
Deng X., Jiang W., D number theory based game-theoretic framework in adversarial decision making under a fuzzy environment, Internat. J. Approx. Reason. 106 (2019) 194–213.
[42]
Fei L., Wang Y., An optimization model for rescuer assignments under an uncertain environment by using Dempster–Shafer theory, Knowl.-Based Syst. 255 (2022).
[43]
Xiao F., A new divergence measure for belief functions in D–S evidence theory for multisensor data fusion, Inform. Sci. 514 (2020) 462–483.
[44]
Karimian M., Beigy H., Concept drift handling: A domain adaptation perspective, Expert Syst. Appl. 224 (2023).
[45]
Tax D.M., Duin R.P., Support vector data description, Mach. Learn. 54 (2004) 45–66.
[46]
Xie X., Sun S., Chen H., Qian J., Domain adaptation with twin support vector machines, Neural Process. Lett. 48 (2018) 1213–1226.
[47]
Mozafari A.S., Jamzad M., A SVM-based model-transferring method for heterogeneous domain adaptation, Pattern Recognit. 56 (2016) 142–158.
[48]
Gao P., Wu W., Li J., Multi-source fast transfer learning algorithm based on support vector machine, Appl. Intell. (2021) 1–15.
[49]
E. Tzeng, J. Hoffman, K. Saenko, T. Darrell, Adversarial discriminative domain adaptation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 7167–7176.
[50]
Yang Y., Zhou J., Ai J., Bin Y., Hanjalic A., Shen H.T., Ji Y., Video captioning by adversarial LSTM, IEEE Trans. Image Process. 27 (2018) 5600–5611.
[51]
Hong C., Zeng Z., Xie R., Zhuang W., Wang X., Domain adaptation with low-rank alignment for weakly supervised hand pose recovery, Signal Process. 142 (2018) 223–230.
[52]
S. Motiian, M. Piccirilli, D.A. Adjeroh, G. Doretto, Unified deep supervised domain adaptation and generalization, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 5715–5725.
[53]
X. Xu, X. Zhou, R. Venkatesan, G. Swaminathan, O. Majumder, d-sne: Domain adaptation using stochastic neighborhood embedding, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 2497–2506.
[54]
Wang Z., Du B., Guo Y., Domain adaptation with neural embedding matching, IEEE Trans. Neural Netw. Learn. Syst. 31 (2019) 2387–2397.
[55]
S. Yang, Y. Wang, J. van de Weijer, L. Herranz, S. Jui, Generalized source-free domain adaptation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 8978–8987.
[56]
Pan S.J., Tsang I.W., Kwok J.T., Yang Q., Domain adaptation via transfer component analysis, IEEE Trans. Neural Netw. 22 (2010) 199–210.
[57]
J. Gao, W. Fan, J. Jiang, J. Han, Knowledge transfer via multiple model local structure mapping, in: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2008, pp. 283–291.
[58]
Ghifary M., Balduzzi D., Kleijn W.B., Zhang M., Scatter component analysis: A unified framework for domain adaptation and domain generalization, IEEE Trans. Pattern Anal. Mach. Intell. 39 (2016) 1414–1430.
[59]
Chen Y., Song S., Li S., Wu C., A graph embedding framework for maximum mean discrepancy-based domain adaptation algorithms, IEEE Trans. Image Process. 29 (2019) 199–213.
[60]
Chen S., Harandi M., Jin X., Yang X., Domain adaptation by joint distribution invariant projections, IEEE Trans. Image Process. 29 (2020) 8264–8277.
[61]
Long M., Cao Z., Wang J., Jordan M.I., Conditional adversarial domain adaptation, Adv. Neural Inf. Process. Syst. 31 (2018).
[62]
Liu H., Shao M., Ding Z., Fu Y., Structure-preserved unsupervised domain adaptation, IEEE Trans. Knowl. Data Eng. 31 (2018) 799–812.
[63]
Huang Z., Sheng K., Li K., Liang J., Yao T., Dong W., Zhou D., Sun X., Reciprocal normalization for domain adaptation, Pattern Recognit. 140 (2023).
[64]
Thangavelu S., Hermite and Laguerre semigroups: some recent developments, 2006.
[65]
Moghaddam V.H., Hamidzadeh J., New Hermite orthogonal polynomial kernel and combined kernels in Support Vector Machine classifier, Pattern Recognit. 60 (2016) 921–935.
[66]
Shafer G., A Mathematical Theory of Evidence, Princeton University Press, 1976.
[67]
Smets P., The combination of evidence in the transferable belief model, IEEE Trans. Pattern Anal. Mach. Intell. 12 (1990) 447–458.
[68]
K. Hw, T. Aw, Nonlinear programming, pp. 481–492.
[69]
Karush W., Minima of Functions of Several Variables with Inequalities as Side Constraints, (M. Sc. dissertation) Dept. of Mathematics, Univ. of Chicago, 1939.
[70]
Kjeldsen T.H., A contextualized historical analysis of the kuhn–tucker theorem in nonlinear programming: the impact of world war II, Historia Math. 27 (2000) 331–361.
[71]
Mikolov T., Chen K., Corrado G., Dean J., Efficient estimation of word representations in vector space, 2013, arXiv preprint arXiv:1301.3781.
[72]
S. Arora, Y. Liang, T. Ma, A simple but tough-to-beat baseline for sentence embeddings, in: International Conference on Learning Representations, 2017.
[73]
Sheskin D.J., Handbook of Parametric and Nonparametric Statistical Procedures, Chapman & Hall/CRC, 2007.

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Applied Soft Computing
Applied Soft Computing  Volume 147, Issue C
Nov 2023
1683 pages

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 01 November 2023

Author Tags

  1. Belief theory
  2. Domain adaptation
  3. Hermite orthogonal polynomials
  4. Robust-to-noise
  5. Support Vector Data Description
  6. Twin classifier
  7. Uncertainty

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 0
    Total Downloads
  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 22 Dec 2024

Other Metrics

Citations

View Options

View options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media