research-article

Data Driven Water Pipe Failure Prediction: A Bayesian Nonparametric Approach

Authors:

Fang ChenAuthors Info & Claims

CIKM '15: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management

Pages 193 - 202

https://doi.org/10.1145/2806416.2806509

Published: 17 October 2015 Publication History

Abstract

Water pipe failures can cause significant economic and social costs, hence have become the primary challenge to water utilities. In this paper, we propose a Bayesian nonparametric approach, namely the Dirichlet process mixture of hierarchical beta process model, for water pipe failure prediction. It can select high-risk pipes for physical condition assessment, thereby preventing disastrous failures proactively.

The proposed method is adaptable to the diversity of failure patterns. Its model structure and complexity can automatically adjust according to observed data. Additionally, the sparse failure data problem that often occurs in real-world data is tackled by the proposed method via flexible pipe grouping and failure data sharing. An approximated yet computational efficient Metropolis-within-Gibbs sampling method is developed with the exploitation of the failure data sparsity for model parameter inference.

The proposed method has been applied to a metropolitan water supply network. The details of the application context are also presented for demonstrating its real-life impact. The comparison experiments conducted on the metropolitan water pipe data show that the proposed approach significantly outperforms the state-of-the-art prediction methods, and it is capable of bringing enormous economic and social savings to water utilities.

References

[1]

D. J. Aldous. Exchangeability and related topics. Springer, 1985.

[2]

A. G. Constantine. Pipeline reliability: Stochastic models in engineering technology and management. Singapore: World Scientifi, 1996.

[3]

D. R. Cox. Regression models and life-tables. In Journal of the Royal Statistical Society, pages 187--220. Series B Methodological, 1972.

[4]

T. S. Ferguson. A bayesian analysis of some nonparametric problems. The annals of statistics, pages 209--230, 1973.

[5]

N. L. Hjort. Nonparametric bayes estimators based on beta processes in models for life history data. The Annals of Statistics, pages 1259--1294, 1990.

[6]

M. D. Hoffan, D. M. Blei, and P. R. Cook. Content-based musical similarity computation using the hierarchical dirichlet process. In ISMIR, pages 349--354, 2008.

[7]

J. P. Huelsenbeck, S. Jain, S. W. Frost, and S. L. K. Pond. A Dirichlet process model for detecting positive selection in protein-coding dna sequences. Proceedings of the National Academy of Sciences, 103(16):6263--6268, 2006.

[8]

J. G. Ibrahim, M.-H. Chen, and D. Sinha. Bayesian survival analysis. Wiley Online Library, 2005.

[9]

A. Kettler and I. Goulter. An analysis of pipe breakage in urban water distribution networks. Canadian Journal of Civil Engineering, 12(2):286--293, 1985.

[10]

B. Li, B. Zhang, Z. Li, Y. Wang, F. Chen, and D. Vitanage. Prioritising water pipes for condition assessment with data analytics. OzWater, 2015.

[11]

Z. Li, B. Zhang, Y. Wang, F. Chen, R. Taib, V. Whiffi and Y. Wang. Water pipe condition assessment: a hierarchical beta process approach for sparse incident data. Machine learning, 95(1):11--26, 2014.

Digital Library

[12]

K. Mavin. Predicting the failure performance of individual water mains. Urban Water Research Association of Australia, (114), 1996.

[13]

J. Paisley and L. Carin. Nonparametric factor analysis with beta process priors. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 777--784. ACM, 2009.

Digital Library

[14]

B. Rajani and Y. Kleiner. Comprehensive review of structural deterioration of water mains: physically based models. Urban water, 3(3):151--164, 2001.

[15]

U. Shamir and C. Howard. An analytic approach to scheduling pipe replacement. pages 71(5), 248--258, 1979.

[16]

Y. W. Teh, M. I. Jordan, M. J. Beal, and D. M. Blei. Hierarchical dirichlet processes. Journal of the american statistical association, 101(476), 2006.

[17]

R. Thibaux and M. I. Jordan. Hierarchical beta processes and the indian bufft process. In International conference on artifiial intelligence and statistics, pages 564--571, 2007.

[18]

R. Wang, W. Dong, Y. Wang, K. Tang, and X. Yao. Pipe failure prediction: A data mining method. In Data Engineering (ICDE), 2013 IEEE 29th International Conference on, pages 1208--1218. IEEE, 2013.

Digital Library

[19]

O. Yakhnenko and V. Honavar. Annotating images and image objects using a hierarchical Dirichlet process model. In Proceedings of the 9th International Workshop on Multimedia Data Mining: held in conjunction with the ACM SIGKDD 2008, pages 1--7. ACM, 2008.

Digital Library

[20]

M. Zhou, H. Chen, L. Ren, G. Sapiro, L. Carin, and J. W. Paisley. Non-parametric bayesian dictionary learning for sparse image representations. In Advances in neural information processing systems, pages 2295--2303, 2009.

[21]

M. Zhou, H. Yang, G. Sapiro, D. B. Dunson, and L. Carin. Dependent hierarchical beta process for image interpolation and denoising. In International conference on artificial intelligence and statistics, pages 883--891, 2011.

Cited By

Xiong CWang JGao WHuang XTao T(2023)Leakage Risk Assessment of Urban Water Distribution Network Based on Unascertained Measure Theory and Game Theory Weighting MethodWater10.3390/w1524429415:24(4294)Online publication date: 16-Dec-2023
https://doi.org/10.3390/w15244294
Taiwo RBen Seghier MZayed T(2023)Toward Sustainable Water Infrastructure: The State‐Of‐The‐Art for Modeling the Failure Probability of Water PipesWater Resources Research10.1029/2022WR03325659:4Online publication date: 30-Mar-2023
https://doi.org/10.1029/2022WR033256
Akbarkhiavi SImteaz M(2021)A novel ‘pressure index’ for predicting number of pipe bursts in water distribution systemProceedings of the Institution of Civil Engineers - Water Management10.1680/jwama.20.00076174:6(278-290)Online publication date: Dec-2021
https://doi.org/10.1680/jwama.20.00076
Show More Cited By

Recommendations

Failure Prediction for Large-scale Water Pipe Networks Using GNN and Temporal Failure Series
CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge Management

Pipe failure prediction in the water industry aims to prioritize the pipes that are at high risk of failure for proactive maintenance. However, existing statistical or machine learning models that rely on historical failures and asset attributes can ...
Multitask Learning for Sparse Failure Prediction
Advances in Knowledge Discovery and Data Mining
Abstract
Sparsity is a problem which occurs inherently in many real-world datasets. Sparsity induces an imbalance in data, which has an adverse effect on machine learning and hence reducing the predictability. Previously, strong assumptions were made by ...
Pipe failure prediction: A data mining method
ICDE '13: Proceedings of the 2013 IEEE International Conference on Data Engineering (ICDE 2013)

Pipe breaks in urban water distribution network lead to significant economical and social costs, putting the service quality as well as the profit of water utilities at risk. To cope with such a situation, scheduled preventive maintenance is desired, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '15: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management

October 2015

1998 pages

ISBN:9781450337946

DOI:10.1145/2806416

General Chairs:
James Bailey
The University of Melbourne
,
Alistair Moffat
The University of Melbourne
,
Program Chairs:
Charu C. Aggarwal
IBM
,
Maarten de Rijke
University of Amsterdam
,
Ravi Kumar
Google
,
Vanessa Murdock
Microsoft
,
Timos Sellis
RMIT University
,
Jeffrey Xu Yu
Chinese University of Hong Kong

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CIKM'15

Sponsor:

CIKM'15: 24th ACM International Conference on Information and Knowledge Management

October 18 - 23, 2015

Melbourne, Australia

Acceptance Rates

CIKM '15 Paper Acceptance Rate 165 of 646 submissions, 26%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
481
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)2

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Xiong CWang JGao WHuang XTao T(2023)Leakage Risk Assessment of Urban Water Distribution Network Based on Unascertained Measure Theory and Game Theory Weighting MethodWater10.3390/w1524429415:24(4294)Online publication date: 16-Dec-2023
https://doi.org/10.3390/w15244294
Taiwo RBen Seghier MZayed T(2023)Toward Sustainable Water Infrastructure: The State‐Of‐The‐Art for Modeling the Failure Probability of Water PipesWater Resources Research10.1029/2022WR03325659:4Online publication date: 30-Mar-2023
https://doi.org/10.1029/2022WR033256
Akbarkhiavi SImteaz M(2021)A novel ‘pressure index’ for predicting number of pipe bursts in water distribution systemProceedings of the Institution of Civil Engineers - Water Management10.1680/jwama.20.00076174:6(278-290)Online publication date: Dec-2021
https://doi.org/10.1680/jwama.20.00076
Liang SLi ZLiang BDing YWang YChen FDemartini GZuccon GCulpepper JHuang ZTong H(2021)Failure Prediction for Large-scale Water Pipe Networks Using GNN and Temporal Failure SeriesProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3481918(3955-3964)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3481918
Snider BMcBean E(2021)Combining Machine Learning and Survival Statistics to Predict Remaining Service Life of WatermainsJournal of Infrastructure Systems10.1061/(ASCE)IS.1943-555X.000062927:3Online publication date: Sep-2021
https://doi.org/10.1061/(ASCE)IS.1943-555X.0000629
Meng ZXu JLi ZWang YChen FWang Z(2021)A Multi-task Kernel Learning Algorithm for Survival AnalysisAdvances in Knowledge Discovery and Data Mining10.1007/978-3-030-75768-7_24(298-311)Online publication date: 11-May-2021
https://dl.acm.org/doi/10.1007/978-3-030-75768-7_24
Snider BMcBean E(2020)Improving Urban Water Security through Pipe-Break Prediction Models: Machine Learning or Survival AnalysisJournal of Environmental Engineering10.1061/(ASCE)EE.1943-7870.0001657146:3Online publication date: Mar-2020
https://doi.org/10.1061/(ASCE)EE.1943-7870.0001657
Luo SChu VLi ZWang YZhou JChen FWong R(2020)Multi-task learning by hierarchical Dirichlet mixture model for sparse failure predictionInternational Journal of Data Science and Analytics10.1007/s41060-020-00219-z12:1(15-29)Online publication date: 21-May-2020
https://doi.org/10.1007/s41060-020-00219-z
Zhang LZhang BGuo TChen FRuncie PCameron BRooney R(2019)Linking Complex Urban Systems: Insights from Cross-Domain Urban Data AnalysisOpen Cities | Open Data10.1007/978-981-13-6605-5_10(221-239)Online publication date: 27-Sep-2019
https://doi.org/10.1007/978-981-13-6605-5_10
Luo SChu VLi ZWang YZhou JChen FWong R(2019)Multitask Learning for Sparse Failure PredictionAdvances in Knowledge Discovery and Data Mining10.1007/978-3-030-16148-4_1(3-14)Online publication date: 22-Mar-2019
https://doi.org/10.1007/978-3-030-16148-4_1
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents