Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Balanced content space partitioning for pub/sub: a study on impact of varying partitioning granularity

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The increasing volume of live content poses new challenges to publish/subscribe services at the cloud scale. Providing efficient publish/subscribe services for live content is a complex task because most subscriptions occupy only a small portion of the entire subscription space, i.e., use limited live content. Thus, the real-world workload of a publish/subscribe service for live content becomes skewed, and the distribution of subscriptions becomes seriously imbalanced, causing an inefficient processing of events. We present a correlation-based balanced content space partitioning technique for a publish/subscribe service. Our proposed technique reduces the degree of imbalance from a skewed subscription workload in a content-based publish/subscribe service, using the correlation coefficient between attributes to build dimension groups. We assign attributes of low correlation to the same dimension group to balance the subscription workloads. Moreover, we present our analysis on the load balance impacts of varying partitioning granularity for efficient message processing. We conducted empirical experiments evaluating the effectiveness of our partitioning technique and measuring the impact of varying partitioning granularity. The results show that the proposed technique outperforms conventional partitioning techniques by evaluating the ways in which subscriptions are evenly distributed among brokers. Moreover, the results show that the load balance can be improved by increasing the partitioning granularity with an adjustment of two degrees, i.e., the segment and dimension group degrees.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Cheung AKY, Jacobsen HA (2010) Load balancing content-based publish/subscribe systems. ACM Trans Comput Syst (TOCS) 28(4):1–55

    Article  Google Scholar 

  2. Cormen TH, Leiserson CE, Rivest RL, Stein C (2009) Introduction to algorithms. MIT press, New York

    MATH  Google Scholar 

  3. Ellison NB, Steinfield C, Lampe C (2007) The benefits of facebook “friends:’’ social capital and college students’ use of online social network sites. J Comput Med Commun 12(4):1143–1168

    Article  Google Scholar 

  4. Esposito C, Ficco M, Palmieri F, Castiglione A (2015) A knowledge-based platform for big data analytics based on publish/subscribe services and stream processing. Knowl Based Syst 79:3–17

    Article  Google Scholar 

  5. Feldmann M, Kolb C, Scheideler C, Strothmann T (2018) Self-stabilizing supervised publish-subscribe systems. In: 2018 IEEE international parallel and distributed processing symposium (IPDPS). IEEE, pp 1050–1059

  6. Gascon-Samson J, Garcia FP, Kemme B, Kienzle J (2015) Dynamoth: a scalable pub/sub middleware for latency-constrained applications in the cloud. In: 2015 IEEE 35th International Conference on Distributed Computing Systems. IEEE, pp 486–496

  7. Google protocol buffers. http://code.google.com/p/protobuf/ (2008)

  8. Gupta A, Sahin OD, Agrawal D, El Abbadi A (2004) Meghdoot: content-based publish/subscribe over p2p networks. In: ACM/IFIP/USENIX International Conference on Distributed Systems Platforms and Open Distributed Processing. Springer, pp 254–273

  9. Kreps J, Narkhede N, Rao J et al (2011) Kafka: a distributed messaging system for log processing. Proc NetDB 11:1–7

    Google Scholar 

  10. Langford E, Schwertman N, Owens M (2001) Is the property of being positively correlated transitive? Am Stat 55(4):322–325

    Article  MathSciNet  Google Scholar 

  11. Li M, Ye F, Kim M, Chen H, Lei H (2011) A scalable and elastic publish/subscribe service. In: 2011 IEEE international parallel and distributed processing symposium. IEEE, pp 1254–1265

  12. Ma X, Wang Y, Pei X (2014) A scalable and reliable matching service for content-based publish/subscribe systems. IEEE Trans Coud Comput 3(1):1–13

    Google Scholar 

  13. Ma X, Wang Y, Qiu Q, Sun W, Pei X (2014) Scalable and elastic event matching for attribute-based publish/subscribe systems. Future Gener Comput Syst 36:102–119

    Article  Google Scholar 

  14. Majumder A, Shrivastava N, Rastogi R, Srinivasan A (2009) Scalable content-based routing in pub/sub systems. In: IEEE INFOCOM 2009. IEEE, pp 567–575

  15. Murmurhash. http://burtleburtle.net/bob/hash/doobs.html/ (2014)

  16. Ross SM (2014) Introduction to probability and statistics for engineers and scientists. Academic Press, London

    MATH  Google Scholar 

  17. Tariq MA, Koldehofe B, Koch GG, Khan I, Rothermel K (2011) Meeting subscriber-defined qos constraints in publish/subscribe systems. Concurr Comput Pract Exp 23(17):2140–2153

    Article  Google Scholar 

  18. Voulgaris S, Riviere E, Kermarrec AM, Van Steen M et al. (2006) Sub-2-sub: Self-organizing content-based publish subscribe for dynamic large scale collaborative networks. In: IPTPS, vol. 141. Citeseer

  19. Wang Y, Ma X (2014) A general scalable and elastic content-based publish/subscribe service. IEEE Trans Parallel Distrib Syst 26(8):2100–2113

    Article  Google Scholar 

Download references

Acknowledgements

This research was jointly supported by the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2020-2018-0-01431) supervised by the IITP (Institute for Information & communications Technology Promotion), the Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education (NRF-2018R1D1A1B07043858), and National Supercomputing Center with supercomputing resources including technical support (KSC-2019-CRE-0105).

Author information

Authors and Affiliations

  1. Ajou University, Suwon-si, Gyeonggi-do, Republic of Korea

    Daegun Yoon & Sangyoon Oh

  2. Xiangtan University, Xiangtan, Hunan, China

    Zhetao Li

Authors
  1. Daegun Yoon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhetao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sangyoon Oh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sangyoon Oh.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yoon, D., Li, Z. & Oh, S. Balanced content space partitioning for pub/sub: a study on impact of varying partitioning granularity. J Supercomput 77, 13676–13702 (2021). https://doi.org/10.1007/s11227-021-03821-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-021-03821-5

Keywords

Search

Navigation