research-article

DC-FUDA: : Improving deep clustering via fully unsupervised domain adaptation

Authors:

Lifang HeAuthors Info & Claims

Volume 526, Issue C

Pages 109 - 120

https://doi.org/10.1016/j.neucom.2023.01.058

Published: 14 March 2023 Publication History

Abstract

By transferring knowledge from a source domain, the performance of deep clustering on an unlabeled target domain can be greatly improved. When achieving this, traditional approaches assume that an adequate amount of labeled data are available in the source domain. However, this assumption is not always satisfied in practice. First, it cannot be guaranteed that rich labeled samples are readily available in the selected source domain. Second, the noisy data in the source domain may lead to negative transferring. In this paper, we propose a novel transfer learning framework to improve deep clustering via fully unsupervised domain adaptation, called DC-FUDA. Specifically, to select reliable instances in the source domain for transferring, we propose a novel adaptive threshold algorithm to select low entropy instances. To transfer important features of the selected instances, we propose a feature-level domain adaptation network (FeatureDA) that cancels an unstable instance generation process. With extensive experiments, we validate that our method effectively improves deep clustering. Besides, without using any labeled data in the source domain, our method achieves competitive results, compared to the state-of-the-art methods using labeled data in the source domain.

References

[1]

P. Arbelaez, M. Maire, C. Fowlkes, J. Malik, Contour detection and hierarchical image segmentation, TPAMI 33 (2010) 898–916.

[2]

A.A. Baffour, Z. Qin, J. Geng, Y. Ding, F. Deng, Z. Qin, Generic network for domain adaptation based on self-supervised learning and deep clustering, Neurocomputing 476 (2022) 126–136.

Digital Library

[3]

K. Bousmalis, N. Silberman, D. Dohan, D. Erhan, D. Krishnan, Unsupervised pixel-level domain adaptation with generative adversarial networks, in: CVPR, 2017, pp. 3722–3731.

[4]

D. Calandriello, G. Niu, M. Sugiyama, Semi-supervised information-maximization clustering, Neural Netw. 57 (2014) 103–111.

[5]

F.M. Carlucci, L. Porzi, B. Caputo, E. Ricci, S.R. Bulò, Just dial: Domain alignment layers for unsupervised domain adaptation, in: Battiato, S., Gallo, G., Schettini, R., Stanco, F. (Eds.), ICIAP, 2017, pp. 357–369.

[6]

M. Caron, P. Bojanowski, A. Joulin, M. Douze, Deep clustering for unsupervised learning of visual features, in: ECCV, 2018.

[7]

F.J. Castellanos, A.J. Gallego, J. Calvo-Zaragoza, Unsupervised neural domain adaptation for document image binarization, Pattern Recogn. 119 (2021).

[8]

W. Chen, H. Hu, Generative attention adversarial classification network for unsupervised domain adaptation, Pattern Recogn. 107 (2020).

[9]

H. Chi, F. Liu, W. Yang, L. Lan, T. Liu, B. Han, W. Cheung, J. Kwok, Tohan: A one-step approach towards few-shot hypothesis adaptation, NeurIPS 34 (2021).

[10]

Y. Dong, Z. Wang, J. Du, W. Fang, L. Li, Attention-based hierarchical denoised deep clustering network, World Wide Web (2022) 1–19.

[11]

J. Duan, J. Zhou, Y. Li, C. Huang, Privacy-preserving and verifiable deep learning inference based on secret sharing, Neurocomputing 483 (2022) 221–234.

[12]

H. Elahi, A. Castiglione, G. Wang, O. Geman, A human-centered artificial intelligence approach for privacy protection of elderly app users in smart cities, Neurocomputing 444 (2021) 189–202.

[13]

X. Fang, H. Bai, Z. Guo, B. Shen, S. Hoi, Z. Xu, Dart: domain-adversarial residual-transfer networks for unsupervised cross-domain image classification, Neural Netw. 127 (2020) 182–192.

[14]

Y. Ganin, V. Lempitsky, Unsupervised domain adaptation by backpropagation, in: ICML, PMLR, 2015, pp. 1180–1189.

[15]

Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, V. Lempitsky, Domain-adversarial training of neural networks, JMLR 17 (2016) 2030–2096.

[16]

P. Ge, C.X. Ren, D.Q. Dai, J. Feng, S. Yan, Dual adversarial autoencoders for clustering, TNNLS 31 (2019) 1417–1424.

[17]

K. Ghasedi, X. Wang, C. Deng, H. Huang, Balanced self-paced learning for generative adversarial clustering network, in: CVPR, 2019, pp. 4391–4400.

[18]

L. Gligic, A. Kormilitzin, P. Goldberg, A. Nevado-Holgado, Named entity recognition in electronic health records using transfer learning bootstrapped neural networks, Neural Netw. 121 (2020) 132–139.

Digital Library

[19]

B. Gong, Y. Shi, F. Sha, K. Grauman, Geodesic flow kernel for unsupervised domain adaptation, in: CVPR, 2012, pp. 2066–2073.

[20]

I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial nets, 2014, pp. 2672–2680.

[21]

A. Gretton, K.M. Borgwardt, M.J. Rasch, B. Schölkopf, A. Smola, A kernel two-sample test, J. Mach. Learn. Res. 13 (2012) 723–773.

Digital Library

[22]

E. Guizzo, T. Weyde, G. Tarroni, Anti-transfer learning for task invariance in convolutional neural networks for speech processing, Neural Netw. 142 (2021) 238–251.

[23]

X. Guo, L. Gao, X. Liu, J. Yin, Improved deep embedded clustering with local structure preservation, in: IJCAI, 2017, pp. 1753–1759.

[24]

X. Guo, X. Liu, E. Zhu, X. Zhu, M. Li, X. Xu, J. Yin, Adaptive self-paced deep clustering with data augmentation, TKDE 32 (2019) 1680–1693.

[25]

X. Guo, E. Zhu, X. Liu, J. Yin, Deep embedded clustering with data augmentation, in: ACML, 2018, pp. 550–565.

[26]

B.D. Haeffele, C. You, R. Vidal, A critique of self-expressive deep subspace clustering, in: ICLR, 2021.

[27]

J.A. Hartigan, M.A. Wong, Algorithm as 136: A k-means clustering algorithm, J. R. Stat. Soc. Ser. C-Appl. Stat. 28 (1979) 100–108.

[28]

J.R. Hershey, Z. Chen, J. Le Roux, S. Watanabe, Deep clustering: Discriminative embeddings for segmentation and separation, in: IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, 2016, pp. 31–35.

[29]

P. Huang, Y. Huang, W. Wang, L. Wang, Deep embedding network for clustering, in: ICPR, 2014, pp. 1532–1537.

[30]

Z. Huang, Y. Ren, X. Pu, L. Pan, D. Yao, G. Yu, Dual self-paced multi-view clustering, Neural Netw. 140 (2021) 184–192.

[31]

J.J. Hull, A database for handwritten text recognition research, TPAMI 16 (1994) 550–554.

[32]

J. Hwang, H. Kim, Variational deep clustering of wafer map patterns, IEEE Trans. Semicond. Manuf. 33 (2020) 466–475.

[33]

G. Kang, L. Jiang, Y. Yang, A.G. Hauptmann, Contrastive adaptation network for unsupervised domain adaptation, in: CVPR, 2019, pp. 4893–4902.

[34]

J. Kim, J. Kwon Lee, K. Mu Lee, Deeply-recursive convolutional network for image super-resolution, in: CVPR, 2016, pp. 1637–1645.

[35]

Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proc. IEEE 86 (1998) 2278–2324.

[36]

R. Li, W. Cao, S. Wu, H.S. Wong, Generating target image-label pairs for unsupervised domain adaptation, TIP 29 (2020) 7997–8011.

[37]

X. Li, W. Chen, D. Xie, S. Yang, P. Yuan, S. Pu, Y. Zhuang, A free lunch for unsupervised domain adaptive object detection without source data, 2020b. arXiv preprint arXiv:2012.05400.

[38]

Y. Li, P. Hu, Z. Liu, D. Peng, J.T. Zhou, X. Peng, Contrastive clustering, 2020c. arXiv preprint arXiv:2009.09687.

[39]

J. Liang, D. Hu, J. Feng, Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation, in: ICML, 2020, pp. 6028–6039.

[40]

F. Liu, G. Zhang, J. Lu, Heterogeneous domain adaptation: An unsupervised approach, TNNLS 31 (2020) 5588–5602.

[41]

M.Y. Liu, O. Tuzel, Coupled generative adversarial networks, in: NeurIPS, 2016, pp. 469–477.

[42]

M. Long, H. Zhu, J. Wang, M.I. Jordan, Deep transfer learning with joint adaptation networks, in: ICML, 2017, pp. 2208–2217.

[43]

Y.W. Luo, C.X. Ren, D.Q. Dai, H. Yan, Unsupervised domain adaptation via discriminative manifold propagation, 2020.

[44]

L.v.d. Maaten, G. Hinton, Visualizing data using t-sne, JMLR 9 (2008) 2579–2605.

[45]

M. Mancini, L. Porzi, S.R. Bulo, B. Caputo, E. Ricci, Inferring latent domains for unsupervised deep domain adaptation, TPAMI 43 (2019) 485–498.

[46]

S. Motiian, M. Piccirilli, D.A. Adjeroh, G. Doretto, Unified deep supervised domain adaptation and generalization, in: ICCV, 2017, pp. 5715–5725.

[47]

N. Mrabah, M. Bouguessa, R. Ksantini, Adversarial deep embedded clustering: on a better trade-off between feature randomness and feature drift, TKDE, 2020.

[48]

J. Na, H. Jung, H.J. Chang, W. Hwang, Fixbi: Bridging domain spaces for unsupervised domain adaptation, in: CVPR, 2021, pp. 1094–1103.

[49]

H. Ng, S. Ong, K. Foong, P.S. Goh, W. Nowinski, Medical image segmentation using k-means clustering and improved watershed algorithm, in: 2006 IEEE southwest symposium on image analysis and interpretation, IEEE, 2006, pp. 61–65.

[50]

M. Ning, D. Lu, D. Wei, C. Bian, C. Yuan, S. Yu, K. Ma, Y. Zheng, Multi-anchor active domain adaptation for semantic segmentation, in: CVPR, 2021, pp. 9112–9122.

[51]

X. Peng, S. Xiao, J. Feng, W.Y. Yau, Z. Yi, Deep subspace clustering with sparsity prior, in: IJCAI, 2016, pp. 1925–1931.

[52]

F. Pizzati, R.d. Charette, M. Zaccaria, P. Cerri, Domain bridge for unpaired image-to-image translation and unsupervised domain adaptation, in: WACV, 2020, pp. 2990–2998.

[53]

C.X. Ren, Y.H. Liu, X.W. Zhang, K.K. Huang, Multi-source unsupervised domain adaptation via pseudo target domain, TIP 31 (2022) 2122–2135.

[54]

Y. Ren, K. Hu, X. Dai, L. Pan, S.C. Hoi, Z. Xu, Semi-supervised deep embedded clustering, Neurocomputing 325 (2019) 121–130.

[55]

Y. Ren, S. Huang, P. Zhao, M. Han, Z. Xu, Self-paced and auto-weighted multi-view clustering, Neurocomputing 383 (2020) 248–256.

Digital Library

[56]

Y. Ren, N. Wang, M. Li, Z. Xu, Deep density-based image clustering, Knowl.-Based Syst. 197 (2020).

[57]

A. Saha, P. Rai, H. Daumé, S. Venkatasubramanian, S.L. DuVall, Active supervised domain adaptation, in: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Springer, 2011, pp. 97–112.

[58]

K. Saito, D. Kim, S. Sclaroff, T. Darrell, K. Saenko, Semi-supervised domain adaptation via minimax entropy, in: ICCV, 2019, pp. 8050–8058.

[59]

K. Saito, K. Watanabe, Y. Ushiku, T. Harada, Maximum classifier discrepancy for unsupervised domain adaptation, in: CVPR, 2018, pp. 3723–3732.

[60]

C. Song, F. Liu, Y. Huang, L. Wang, T. Tan, Auto-encoder based data clustering, in: ICPR, Springer, 2013, pp. 117–124.

[61]

V. Stephanie, M. Chamikara, I. Khalil, M. Atiquzzaman, Privacy-preserving location data stream clustering on mobile edge computing and cloud, Inf. Syst. (2021).

[62]

Z. Sun, H. Sun, Stacked denoising autoencoder with density-grid based clustering method for detecting outlier of wind turbine components, IEEE Access 7 (2019) 13078–13091.

[63]

C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna, Rethinking the inception architecture for computer vision, in: CVPR, 2016, pp. 2818–2826.

[64]

L. Van Der Maaten, Learning a parametric embedding by preserving local structure, in: JMLR, 2009, pp. 384–391.

[65]

P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, P.A. Manzagol, L. Bottou, Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion, JMLR 11 (2010) 3371–3408.

[66]

Q. Wang, T. Breckon, Unsupervised domain adaptation via structured prediction based selective pseudo-labeling, in: AAAI, 2020, pp. 6243–6250.

[67]

Z. Wang, B. Du, Y. Guo, Domain adaptation with neural embedding matching, TNNLS 31 (2019) 2387–2397.

[68]

J. Xie, R. Girshick, A. Farhadi, Unsupervised deep embedding for clustering analysis, in: ICML, 2016, pp. 478–487.

[69]

C. Xing, L. Ma, X. Yang, Stacked denoise autoencoder based feature extraction and classification for hyperspectral images, J. Sens. (2016).

[70]

J. Xu, Y. Ren, G. Li, L. Pan, C. Zhu, Z. Xu, Deep embedded multi-view clustering with collaborative training, Inf. Sci. 573 (2021) 279–290.

[71]

J. Xu, Y. Ren, H. Tang, Z. Yang, L. Pan, Y. Yang, X. Pu, S.Y. Philip, L. He, Self-supervised discriminative feature learning for deep multi-view clustering, TKDE, 2022.

[72]

H. Yan, Y. Ding, P. Li, Q. Wang, Y. Xu, W. Zuo, Mind the class weight bias: Weighted maximum mean discrepancy for unsupervised domain adaptation, in: CVPR, 2017, pp. 2272–2281.

[73]

J. Yang, D. Parikh, D. Batra, Joint unsupervised learning of deep representations and image clusters, in: CVPR, 2016, pp. 5147–5156.

[74]

Y. Yang, R. Wang, C. Feng, Level set formulation for automatic medical image segmentation based on fuzzy clustering, Signal Process.: Image Commun. 87 (2020).

[75]

T. Yao, Y. Pan, C.W. Ngo, H. Li, T. Mei, Semi-supervised domain adaptation with subspace learning for visual recognition, in: CVPR, 2015, pp. 2142–2150.

[76]

Z.X. Yong, T.T. Torrent, Semi-supervised deep embedded clustering with anomaly detection for semantic frame induction, in: Proceedings of The 12th Language Resources and Evaluation Conference, 2020, pp. 3509–3519.

[77]

Y. Zhang, F. Liu, Z. Fang, B. Yuan, G. Zhang, J. Lu, Learning from a complementary-label source domain: theory and algorithms, TNNLS, 2021.

[78]

J. Zhao, D. Lu, K. Ma, Y. Zhang, Y. Zheng, Deep image clustering with category-style representation, in: ECCV, 2020, pp. 54–70.

[79]

S. Zhao, B. Li, X. Yue, Y. Gu, P. Xu, R. Hu, H. Chai, K. Keutzer, Multi-source domain adaptation for semantic segmentation, in: NeurIPS, 2019.

[80]

P. Zhou, L. Du, X. Liu, Y.D. Shen, M. Fan, X. Li, Self-paced clustering ensemble, TNNLS 32 (2020) 1497–1511.

[81]

Y. Zhu, W. Li, M. Zhang, Y. Pang, R. Tao, Q. Du, Joint feature extraction for multi-source data using similar double-concentrated network, Neurocomputing 450 (2021) 70–79.

[82]

L. Zong, X. Zhang, L. Zhao, H. Yu, Q. Zhao, Multi-view clustering via multi-manifold regularized non-negative matrix factorization, Neural Netw. 88 (2017) 74–89.

Index Terms

DC-FUDA: Improving deep clustering via fully unsupervised domain adaptation
1. Information systems

Index terms have been assigned to the content through auto-classification.

Recommendations

AVATAR: Adversarial self-superVised domain Adaptation network for TARget domain
Abstract
This paper introduces AVATAR (adversarial self-supervised domain adaptation network for target domain), a novel unsupervised domain adaptation method designed to address the challenge of transferring knowledge from a well-labeled source domain to ...
Highlights
- AVATAR excels in complex domain adaptation tasks.
- Enhances target discrimination with deep clustering.
- Iteratively refines features for robust adaptation.
- AVATAR achieves notable performance in imbalanced adaptations.
Transfer alignment network for blind unsupervised domain adaptation
Abstract
How can we transfer the knowledge from a source domain to a target domain when each side cannot observe the data in the other side? Recent transfer learning methods show significant performance in classification tasks by leveraging both source and ...
Joint Feature and Labeling Function Adaptation for Unsupervised Domain Adaptation
Advances in Knowledge Discovery and Data Mining
Abstract
Unsupervised domain adaptation aims to transfer knowledge from a labeled source domain to an unlabeled target domain. Although having achieved remarkable progress, most existing methods only focus on learning domain-invariant features and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Neurocomputing

Neurocomputing Volume 526, Issue C

Mar 2023

192 pages

ISSN:0925-2312

Issue’s Table of Contents

Elsevier B.V.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 14 March 2023

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Nov 2024

Other Metrics

View Author Metrics

Citations

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents