research-article

Linear scale semantic mining algorithms in microsoft SQL server's semantic platform

Authors:

Kunal Mukerjee,

Sorin GhermanAuthors Info & Claims

KDD '11: Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining

Pages 213 - 221

https://doi.org/10.1145/2020408.2020447

Published: 21 August 2011 Publication History

Abstract

This paper describes three linear scale, incremental, and fully automatic semantic mining algorithms that are at the foundation of the new Semantic Platform being released in the next version of SQL Server. The target workload is large (10 -- 100 million) Enterprise document corpuses. At these scales, anything short of linear scale and incremental is costly to deploy. These three algorithms give rise to three weighted physical indexes: Tag Index (top keywords in each document); Document Similarity Index (top closely related documents given any document); and Semantic Phrase Similarity Index (top semantically related phrases, given any phrase), which are then query-able through the SQL interface. The need for specifically creating these three indexes was motivated by observing typical stages of document research, and gap analysis, given current tools and technology at the Enterprise. We describe the mining algorithms and architecture, and also outline some compelling user experiences that are enabled by the indexes.

References

[1]

Baeza-Yates, R., and Ribeiro-Neto, B., Modern Information Retrieval, Addison-Wesley, 1999.

Digital Library

[2]

Blei, D. M., Ng, A. Y., and Jordan, M., Latent Dirichlet allocation, Journal of Machine Learning Research No. 3, pp. 993--1022.

Digital Library

[3]

Cohen, J. D., Highlights: Language- and domain independent automatic indexing terms for abstracting, Journal of the American Society of Information Science, Volume 46, Issue 3, pp. 162--174, April 1995.

Digital Library

[4]

Damashek, M., Gauging similarity with N-grams: Language-independent categorization of text, Science 267, Feb. 1995.

[5]

Deerwester, S., et al., Improving Information Retrieval with Latent Semantic Indexing, Proceedings of the 51st Annual Meeting of the American Society for Information Science 25, 1988, pp. 36--40.

[6]

Full Text Search Overview (http://msdn.microsoft.com/en-us/library/ms142571.aspx).

[7]

Gabrilovich, E., Markovitch, S., Computing semantic relatedness using Wikipedia-based explicit semantic analysis, in Proceedings of the 20th International Joint Conference on Artificial Intelligence, 2007.

Digital Library

[8]

Hammouda, K., M., Kamel, M., S., Document Similarity using a Phrase Indexing Graph Model, Journal of Knowledge and Information Systems, Vol. 6, Issue 6, November 2004.

Digital Library

[9]

Jiang, X., Hu, Y., Li, H., A Ranking Approach to Keyphrase Extraction, Microsoft Research Technical Report, 2009.

[10]

McNabb, K., Moore, C., and Levitt, D., Open Text Leads ECM Suite Pure Plays, The Forrester Wave Vendor Summary, Q4 2007.

[11]

Salton, G., Wong, A., Yang, C. S., "A Vector Space Model for Automatic Indexing", Communications of the ACM, vol. 18, nr. 11, pages 613--620, 1975.

Digital Library

[12]

Silberschatz, A., Stonebraker, M., and Ullman, J., Database Research: Achievements and Opportunities into the 21st Century. Technical Report, Stanford, 1996.

Digital Library

[13]

Tan, P-N., Steinbach, M., and Kumar, V., Introduction to Data Mining, 2005.

Digital Library

[14]

Witten, I.H., Paynter, G. W., Frank, E., Gutwin, C., and Nevill-Manning, C., G., KEA: Practical automatic keyphrase extraction, Proc. DL '99, pp. 254--256.

Digital Library

[15]

Zamir, O., and Etzioni, O., Web Document Clustering: A Feasibility Demonstration, in Proc. ACM SIGIR'98, 1998.

Digital Library

Cited By

Cerone A(2015)Process Mining as a Modelling ToolRevised Selected Papers of the SEFM 2015 Collocated Workshops on Software Engineering and Formal Methods - Volume 950910.1007/978-3-662-49224-6_12(139-144)Online publication date: 7-Sep-2015
https://dl.acm.org/doi/10.1007/978-3-662-49224-6_12
Mehrbod AZutshi AGrilo A(2014)A Vector Space Model Approach for Searching and Matching Product E-CataloguesProceedings of the Eighth International Conference on Management Science and Engineering Management10.1007/978-3-642-55122-2_71(833-842)Online publication date: 7-May-2014
https://doi.org/10.1007/978-3-642-55122-2_71

Index Terms

Linear scale semantic mining algorithms in microsoft SQL server's semantic platform
1. Applied computing
  1. Physical sciences and engineering
    1. Mathematics and statistics
2. Information systems
  1. Information systems applications
    1. Data mining
    2. Decision support systems
      1. Data analytics

Recommendations

Procedure Based on Semantic Similarity for Merging Ontologies by Non-Redundant Knowledge Enrichment

The merging procedures of two ontologies are mostly related to the enrichment of one of the input ontologies, i.e. the knowledge of the aligned concepts from one ontology are copied into the other ontology. As a consequence, the resulting new ontology ...
Document Similarity by Word Clustering with Semantic Distance
Hybrid Artificial Intelligent Systems
Abstract
In information retrieval, Latent Semantic Analysis (LSA) is a method to handle large and sparse document vectors. LSA reduces the dimension of document vectors by producing a set of topics related to the documents and terms statistically. ...
Semantic interoperability between relational databases in large-scale environments

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '11: Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining

August 2011

1446 pages

ISBN:9781450308137

DOI:10.1145/2020408

General Chair:
Chid Apte
IBM Research
,
Program Chairs:
Joydeep Ghosh
UT Austin
,
Padhraic Smyth
UC Irvine

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 August 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

KDD '11

Sponsor:

KDD '11: The 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

August 21 - 24, 2011

California, San Diego, USA

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
343
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 20 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cerone A(2015)Process Mining as a Modelling ToolRevised Selected Papers of the SEFM 2015 Collocated Workshops on Software Engineering and Formal Methods - Volume 950910.1007/978-3-662-49224-6_12(139-144)Online publication date: 7-Sep-2015
https://dl.acm.org/doi/10.1007/978-3-662-49224-6_12
Mehrbod AZutshi AGrilo A(2014)A Vector Space Model Approach for Searching and Matching Product E-CataloguesProceedings of the Eighth International Conference on Management Science and Engineering Management10.1007/978-3-642-55122-2_71(833-842)Online publication date: 7-May-2014
https://doi.org/10.1007/978-3-642-55122-2_71

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents