Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/2840728.2840753acmconferencesArticle/Chapter ViewAbstractPublication PagesitcsConference Proceedingsconference-collections
research-article
Public Access

On Sketching Quadratic Forms

Published: 14 January 2016 Publication History

Abstract

We undertake a systematic study of sketching a quadratic form: given an n x n matrix A, create a succinct sketch sk(A) which can produce (without further access to A) a multiplicative (1+ε)-approximation to xT A x for any desired query xRn. While a general matrix does not admit non-trivial sketches, positive semi-definite (PSD) matrices admit sketches of size θ(ε{-2 n), via the Johnson-Lindenstrauss lemma, achieving the "for each" guarantee, namely, for each query x, with a constant probability the sketch succeeds. (For the stronger "for all" guarantee, where the sketch succeeds for all x's simultaneously, again there are no non-trivial sketches.)
We design significantly better sketches for the important subclass of graph Laplacian matrices, which we also extend to symmetric diagonally dominant matrices. A sequence of work culminating in that of Batson, Spielman, and Srivastava (SIAM Review, 2014), shows that by choosing and reweighting O{-2 n) edges in a graph, one achieves the "for all" guarantee. Our main results advance this front.
For the "for all" guarantee, we prove that Batson et al.'s bound is optimal even when we restrict to "cut queries" x ∈ (0,1}n. Specifically, an arbitrary sketch that can (1+ε)-estimate the weight of all cuts (S, bar S) in an n-vertex graph must be of size Ω(ε{-2 n) bits. Furthermore, if the sketch is a cut-sparsifier (i.e., itself a weighted graph and the estimate is the weight of the corresponding cut in this graph), then the sketch must have Ω(ε{-2 n) edges.
In contrast, previous lower bounds showed the bound only for spectral-sparsifiers.
For the "for each" guarantee, we design a sketch of size Õ(ε{-1 n) bits for "cut queries" x ∈{0,1}n. We apply this sketch to design an algorithm for the distributed minimum cut problem. We prove a nearly-matching lower bound of Ω(ε{-1 n) bits. For general queries xRn, we construct sketches of size Õ(ε{-1.6 n) bits.
Our results provide the first separation between the sketch size needed for the "for all" and "for each" guarantees for Laplacian matrices.

References

[1]
K. J. Ahn, S. Guha, and A. McGregor. Analyzing graph structure via linear measurements. In 23rd Annual ACM-SIAM Symposium on Discrete Algorithms, pages 459--467, 2012.
[2]
K. J. Ahn, S. Guha, and A. McGregor. Graph sketches: sparsification, spanners, and subgraphs. In 31st ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, pages 5--14, 2012.
[3]
S. Arora, E. Hazan, and S. Kale. Fast algorithms for approximate semidefinite programming using the multiplicative weights update method. In 46th Annual IEEE Symposium on Foundations of Computer Science, FOCS '05, pages 339--348. IEEE Computer Society, 2005.href http://dx.doi.org/10.1109/SFCS.2005.35pathdoi:10.1109/SFCS.2005.35.
[4]
N. Alon. On the edge-expansion of graphs. Comb. Probab. Comput., 6(2):145--152, June 1997.href http://dx.doi.org/10.1017/S096354839700299Xpathdoi:10.1017/S096354839700299X.
[5]
S. Arora, S. Rao, and U. Vazirani. Expander flows, geometric embeddings and graph partitioning. J. ACM, 56(2):1--37, 2009.href http://dx.doi.org/10.1145/1502793.1502794pathdoi:10.1145/1502793.1502794.
[6]
J. Blocki, A. Blum, A. Datta, and O. Sheffet. Differentially private data analysis of social networks via restricted sensitivity. In Innovations in Theoretical Computer Science, ITCS 2013, pages 87--96, 2013.
[7]
A. A. Benczúr and D. R. Karger. Approximating $\rm s$-$\rm t$ minimum cuts in Õ(n2) time. In 28th Annual ACM Symposium on Theory of Computing, pages 47--55. ACM, 1996.href http://dx.doi.org/10.1145/237814.237827pathdoi:10.1145/237814.237827.
[8]
J. D. Batson, D. A. Spielman, and N. Srivastava. Twice-ramanujan sparsifiers. SIAM Review, 56(2):315--334, 2014.href http://dx.doi.org/10.1137/130949117pathdoi:10.1137/130949117.
[9]
K. L. Clarkson and D. P. Woodruff. Numerical linear algebra in the streaming model. In Proceedings of the 41st Annual ACM Symposium on Theory of Computing, STOC 2009, pages 205--214, 2009.
[10]
W. S. Fung, R. Hariharan, N. J. Harvey, and D. Panigrahi. A general framework for graph sparsification. In Proceedings of the Symposium on Theory of Computing (STOC), pages 71--80. ACM, 2011.href http://dx.doi.org/10.1145/1993636.1993647pathdoi:10.1145/1993636.1993647.
[11]
A. Gupta, A. Roth, and J. Ullman. Iterative constructions and private data release. In 9th International Conference on Theory of Cryptography, TCC'12, pages 339--356. Springer-Verlag, 2012.href http://dx.doi.org/10.1007/978-3-642-28914-9_19pathdoi:10.1007/978--3--642--28914--9_19.
[12]
M. R. Henzinger and D. P. Williamson. On the number of small cuts in a graph. Inf. Process. Lett., 59(1):41--44, 1996.
[13]
P. Jain and A. Thakurta. Mirror descent based database privacy. In 15th International Workshop, APPROX 2012, and 16th International Workshop, RANDOM 2012, pages 579--590, 2012.
[14]
D. R. Karger. Minimum cuts in near-linear time. J. ACM, 47(1):46--76, 2000.href http://dx.doi.org/10.1145/331605.331608pathdoi:10.1145/331605.331608.
[15]
M. Kapralov, Y. T. Lee, C. Musco, C. Musco, and A. Sidford. Single pass spectral sparsification in dynamic streams. In 55th Annual Symposium on Foundations of Computer Science, FOCS '14, pages 561--570. IEEE Computer Society, 2014.href http://arxiv.org/abs/1407.1289patharXiv:1407.1289,href http://dx.doi.org/10.1109/FOCS.2014.66pathdoi:10.1109/FOCS.2014.66.
[16]
H. Klauck, D. Nanongkai, G. Pandurangan, and P. Robinson. Distributed computation of large-scale graph problems. In 26th Annual ACM-SIAM Symposium on Discrete Algorithms, SODA '15, pages 391--410. SIAM, 2015.href http://arxiv.org/abs/1311.6209patharXiv:1311.6209,href http://dx.doi.org/10.1137/1.9781611973730.28pathdoi:10.1137/1.9781611973730.28.
[17]
M. Kapralov and R. Panigrahy. Spectral sparsification via random spanners. In 3rd Innovations in Theoretical Computer Science Conference, pages 393--398. ACM, 2012.href http://dx.doi.org/10.1145/2090236.2090267pathdoi:10.1145/2090236.2090267.
[18]
A. Madry. Fast approximation algorithms for cut-based problems in undirected graphs. In Proceedings of the Symposium on Foundations of Computer Science (FOCS), pages 245--254. IEEE, 2010.
[19]
A. McGregor. Graph stream algorithms: a survey. SIGMOD Record, 43(1):9--20, 2014.
[20]
A. Nilli. On the second eigenvalue of a graph. Discrete Math, 91:207--210, 1991.href http://dx.doi.org/10.1016/0012-365X(91)90112-Fpathdoi:10.1016/0012-365X(91)90112-F.
[21]
J. Sherman. Breaking the multicommodity flow barrier for O(√log n)-approximations to sparsest cut. In Proceedings of the Symposium on Foundations of Computer Science (FOCS), pages 363--372, 2009.
[22]
D. A. Spielman and N. Srivastava. Graph sparsification by effective resistances. SIAM J. Comput., 40(6):1913--1926, December 2011.href http://dx.doi.org/10.1137/080734029pathdoi:10.1137/080734029.
[23]
D. A. Spielman and S.-H. Teng. Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems. In Proceedings of the Symposium on Theory of Computing (STOC), pages 81--90. ACM, 2004.href http://dx.doi.org/10.1145/1007352.1007372pathdoi:10.1145/1007352.1007372.
[24]
D. A. Spielman and S.-H. Teng. Spectral sparsification of graphs. SIAM J. Comput., 40(4):981--1025, 2011.href http://dx.doi.org/10.1137/08074489Xpathdoi:10.1137/08074489X.
[25]
J. Upadhyay. Random projections, graph sparsification, and differential privacy. In 19th International Conference on Advances in Cryptology, ASIACRYPT 2013, pages 276--295. Springer-Verlag, 2013.href http://dx.doi.org/10.1007/978-3-642-42033-7_15pathdoi:10.1007/978-3-642-42033-7_15.
[26]
J. Upadhyay. Circulant matrices and differential privacy. CoRR, abs/1410.2470, 2014.href http://arxiv.org/abs/1410.2470patharXiv:1410.2470.
[27]
D. P. Woodruff and Q. Zhang. When distributed computation is communication expensive. In Distributed Computing - 27th International Symposium, DISC 2013, pages 16--30, 2013.href http://dx.doi.org/10.1007/978-3-642-41527-2_2pathdoi:10.1007/978-3-642-41527-2_2.

Cited By

View all
  • (2024)Tight Lower Bounds for Directed Cut Sparsification and Distributed Min-CutProceedings of the ACM on Management of Data10.1145/36511482:2(1-18)Online publication date: 14-May-2024
  • (2024)Approximating Small Sparse CutsProceedings of the 56th Annual ACM Symposium on Theory of Computing10.1145/3618260.3649747(319-330)Online publication date: 10-Jun-2024
  • (2023)Additive Sparsification of CSPsACM Transactions on Algorithms10.1145/362582420:1(1-18)Online publication date: 13-Nov-2023
  • Show More Cited By

Index Terms

  1. On Sketching Quadratic Forms

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ITCS '16: Proceedings of the 2016 ACM Conference on Innovations in Theoretical Computer Science
    January 2016
    422 pages
    ISBN:9781450340571
    DOI:10.1145/2840728
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 14 January 2016

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. graph sparsification
    2. lower bound
    3. quadratic forms
    4. sketching

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    ITCS'16
    Sponsor:
    ITCS'16: Innovations in Theoretical Computer Science
    January 14 - 17, 2016
    Massachusetts, Cambridge, USA

    Acceptance Rates

    ITCS '16 Paper Acceptance Rate 40 of 145 submissions, 28%;
    Overall Acceptance Rate 172 of 513 submissions, 34%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)78
    • Downloads (Last 6 weeks)6
    Reflects downloads up to 29 Nov 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Tight Lower Bounds for Directed Cut Sparsification and Distributed Min-CutProceedings of the ACM on Management of Data10.1145/36511482:2(1-18)Online publication date: 14-May-2024
    • (2024)Approximating Small Sparse CutsProceedings of the 56th Annual ACM Symposium on Theory of Computing10.1145/3618260.3649747(319-330)Online publication date: 10-Jun-2024
    • (2023)Additive Sparsification of CSPsACM Transactions on Algorithms10.1145/362582420:1(1-18)Online publication date: 13-Nov-2023
    • (2023)Streaming Euclidean MST to a Constant FactorProceedings of the 55th Annual ACM Symposium on Theory of Computing10.1145/3564246.3585168(156-169)Online publication date: 2-Jun-2023
    • (2022)Spectral Hypergraph Sparsifiers of Nearly Linear Size2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS)10.1109/FOCS52979.2021.00114(1159-1170)Online publication date: Feb-2022
    • (2022)A note on distance-preserving graph sparsificationInformation Processing Letters10.1016/j.ipl.2021.106205174(106205)Online publication date: Mar-2022
    • (2022)The algebraic structure of the densification and the sparsification tasks for CSPsConstraints10.1007/s10601-022-09340-128:1(13-44)Online publication date: 8-Dec-2022
    • (2022)Faster Cut Sparsification of Weighted GraphsAlgorithmica10.1007/s00453-022-01053-485:4(929-964)Online publication date: 1-Nov-2022
    • (2021)The expander hierarchy and its applications to dynamic graph algorithmsProceedings of the Thirty-Second Annual ACM-SIAM Symposium on Discrete Algorithms10.5555/3458064.3458196(2212-2228)Online publication date: 10-Jan-2021
    • (2021)Towards tight bounds for spectral sparsification of hypergraphsProceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing10.1145/3406325.3451061(598-611)Online publication date: 15-Jun-2021
    • Show More Cited By

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media