Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/2588555.2593674acmconferencesArticle/Chapter ViewAbstractPublication PagesmodConference Proceedingsconference-collections
research-article

Fusing data with correlations

Published: 18 June 2014 Publication History

Abstract

Many applications rely on Web data and extraction systems to accomplish knowledge-driven tasks. Web information is not curated, so many sources provide inaccurate, or conflicting information. Moreover, extraction systems introduce additional noise to the data. We wish to automatically distinguish correct data and erroneous data for creating a cleaner set of integrated data. Previous work has shown that a naive voting strategy that trusts data provided by the majority or at least a certain number of sources may not work well in the presence of copying between the sources. However, correlation between sources can be much broader than copying: sources may provide data from complementary domains (negative correlation), extractors may focus on different types of information (negative correlation), and extractors may apply common rules in extraction (positive correlation, without copying). In this paper we present novel techniques modeling correlations between sources and applying it in truth finding. We provide a comprehensive evaluation of our approach on three real-world datasets with different characteristics, as well as on synthetic data, showing that our algorithms outperform the existing state-of-the-art techniques.

References

[1]
Amazon EC2 Instances. http://aws.amazon.com/ec2/instance-types/.
[2]
L. Berti-Equille, A. D. Sarma, X. Dong, A. Marian, and D. Srivastava. Sailing the information ocean with awareness of currents: Discovery and application of source dependence. In CIDR, 2009.
[3]
L. Blanco, V. Crescenzi, P. Merialdo, and P. Papotti. Probabilistic models to reconcile complex data from inaccurate data sources. In CAiSE, 2010.
[4]
J. Bleiholder, K. Draba, and F. Naumann. FuSem-exploring different semantics of data fusion. In VLDB, pages 1350--1353, 2007.
[5]
X. L. Dong, L. Berti-Equille, Y. Hu, and D. Srivastava. Global detection of complex copying relationships between sources. PVLDB, 3(1-2):1358--1369, Sept. 2010.
[6]
X. L. Dong, L. Berti-Equille, and D. Srivastava. Integrating conflicting data: The role of source dependence. PVLDB, 2(1):550--561, 2009.
[7]
X. L. Dong, L. Berti-Equille, and D. Srivastava. Truth discovery and copying detection in a dynamic world. PVLDB, 2(1), 2009.
[8]
X. L. Dong and F. Naumann. Data fusion-resolving data conflicts for integration. PVLDB, 2009.
[9]
X. L. Dong, B. Saha, and D. Srivastava. Less is more: Selecting sources wisely for integration. PVLDB, 6(2):37--48, Dec. 2012.
[10]
X. L. Dong and D. Srivastava. Large-scale copying detection. In SIG-MOD (Tutorial), 2011.
[11]
A. Fader, S. Soderland, and O. Etzioni. Identifying relations for open information extraction. In EMNLP, 2011.
[12]
J. Fleiss. Statistical methods for rates and proportions. John Wiley and Sons, 1981.
[13]
A. Galland, S. Abiteboul, A. Marian, and P. Senellart. Corroborating information from disagreeing views. In WSDM, 2010.
[14]
J. M. Kleinberg. Authoritative sources in a hyperlinked environment. In SODA, 1998.
[15]
X. Li, X. L. Dong, K. B. Lyons, W. Meng, and D. Srivastava. Truth finding on the deep web: Is the problem solved? PVLDB, 6(2), 2013.
[16]
X. Liu, X. L. Dong, B.-C. Ooi, and D. Srivastava. Online data fusion. PVLDB, 4(12), 2011.
[17]
A. Marian and M. Wu. Corroborating information from web sources. IEEE Data Eng. Bull., 34(3):11--17, 2011.
[18]
M. Mintz, S. Bills, R. Snow, and D. Jurafsky. Distant supervision for relation extraction without labeled data. In ACL, 2009.
[19]
J. Pasternack and D. Roth. Knowing what to believe (when you already know something). In COLING, pages 877--885, 2010.
[20]
J. Pasternack and D. Roth. Making better informed trust decisions with generalized fact-finding. In IJCAI, pages 2324--2329, 2011.
[21]
J. Pasternack and D. Roth. Latent credibility analysis. In WWW, 2013.
[22]
G.-J. Qi, C. Aggarwal, J. Han, and T. Huang. Mining collective intelligence in groups. In WWW, 2013.
[23]
X. Yin, J. Han, and P. S. Yu. Truth discovery with multiple conflicting information providers on the web. In SIGKDD, 2007.
[24]
X. Yin and W. Tan. Semi-supervised truth discovery. In WWW, 2011.
[25]
B. Zhao, B. I. P. Rubinstein, J. Gemmell, and J. Han. A Bayesian approach to discovering truth from conflicting sources for data integration. PVLDB, 5(6):550--561, 2012.

Cited By

View all
  • (2024)Stochastic Fusion Techniques for State EstimationComputation10.3390/computation1210020912:10(209)Online publication date: 17-Oct-2024
  • (2024)Money Never Sleeps: Maximizing Liquidity Mining Yields in Decentralized FinanceProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671942(2248-2259)Online publication date: 25-Aug-2024
  • (2024)Generalizing truth discovery by incorporating multi-truth featuresComputing10.1007/s00607-024-01288-9106:5(1557-1583)Online publication date: 22-Apr-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGMOD '14: Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data
June 2014
1645 pages
ISBN:9781450323765
DOI:10.1145/2588555
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 June 2014

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. correlated sources
  2. data fusion
  3. integration

Qualifiers

  • Research-article

Funding Sources

Conference

SIGMOD/PODS'14
Sponsor:

Acceptance Rates

SIGMOD '14 Paper Acceptance Rate 107 of 421 submissions, 25%;
Overall Acceptance Rate 785 of 4,003 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)17
  • Downloads (Last 6 weeks)2
Reflects downloads up to 30 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Stochastic Fusion Techniques for State EstimationComputation10.3390/computation1210020912:10(209)Online publication date: 17-Oct-2024
  • (2024)Money Never Sleeps: Maximizing Liquidity Mining Yields in Decentralized FinanceProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671942(2248-2259)Online publication date: 25-Aug-2024
  • (2024)Generalizing truth discovery by incorporating multi-truth featuresComputing10.1007/s00607-024-01288-9106:5(1557-1583)Online publication date: 22-Apr-2024
  • (2023)Decision Aggregation with Reliability PropagationSSRN Electronic Journal10.2139/ssrn.4383255Online publication date: 2023
  • (2023)TIRA: Truth Inference via Reliability Aggregation on Object-Source GraphIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2022.322530835:11(11967-11981)Online publication date: 1-Nov-2023
  • (2023)Multi-Truth Discovery While Being Aware of Unbalanced Data Distribution2023 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN54540.2023.10191906(1-10)Online publication date: 18-Jun-2023
  • (2023)A Multi-truth Discovery Approach Based on Confidence Interval Estimation of TruthsAdvanced Data Mining and Applications10.1007/978-3-031-46677-9_41(599-615)Online publication date: 5-Nov-2023
  • (2023)HIT - An Effective Approach to Build a Dynamic Financial Knowledge BaseDatabase Systems for Advanced Applications10.1007/978-3-031-30672-3_48(716-731)Online publication date: 14-Apr-2023
  • (2022)Selecting Workers Wisely for Crowdsourcing When Copiers and Domain Experts Co-existFuture Internet10.3390/fi1402003714:2(37)Online publication date: 24-Jan-2022
  • (2022)A Probabilistic Data Fusion Modeling Approach for Extracting True Values from Uncertain and Conflicting AttributesBig Data and Cognitive Computing10.3390/bdcc60401146:4(114)Online publication date: 13-Oct-2022
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media