Nothing Special   »   [go: up one dir, main page]

Halas, 2017 - Google Patents

Performance Measurement of Stream Data Processing in Apache Spark

Halas, 2017

View PDF
Document ID
12745864001249793185
Author
Halas F
Publication year

External Links

Snippet

The primary goal of this thesis was to implement application for measuring stream processing performance of Apache Spark on network traffic data and to perform performance benchmark in a distributed environment. The results of the benchmark are …
Continue reading at is.muni.cz (PDF) (other versions)

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for programme control, e.g. control unit
    • G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466—Performance evaluation by tracing or monitoring
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for programme control, e.g. control unit
    • G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogramme communication; Intertask communication
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30286—Information retrieval; Database structures therefor; File system structures therefor in structured data stores
    • G06F17/30386—Retrieval requests
    • G06F17/30424—Query processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/362—Software debugging
    • G06F11/3636—Software debugging by tracing the execution of the program
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/3668—Software testing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F15/00—Digital computers in general; Data processing equipment in general
    • G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a programme unit and a register, e.g. for a simultaneous processing of several programmes
    • G06F15/163—Interprocessor communication
    • G06F15/173—Interprocessor communication using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/86—Event-based monitoring
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/875—Monitoring of systems including the internet
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance or administration or management of packet switching networks
    • H04L41/14—Arrangements for maintenance or administration or management of packet switching networks involving network analysis or design, e.g. simulation, network model or planning

Similar Documents

Publication Publication Date Title
US9152532B2 (en) System and method for configuring a cloud computing system with a synthetic test workload
US9262231B2 (en) System and method for modifying a hardware configuration of a cloud computing system
US8887056B2 (en) System and method for configuring cloud computing systems
US9658895B2 (en) System and method for configuring boot-time parameters of nodes of a cloud computing system
US8762951B1 (en) Apparatus and method for profiling system events in a fine grain multi-threaded multi-core processor
US10560544B2 (en) Data caching in a collaborative file sharing system
US20140047342A1 (en) System and method for allocating a cluster of nodes for a cloud computing system based on hardware characteristics
US20140047095A1 (en) System and method for tuning a cloud computing system
US20140047079A1 (en) System and method for emulating a desired network configuration in a cloud computing system
WO2014025584A1 (en) System and method for tuning a cloud computing system
Enes et al. BDWatchdog: Real-time monitoring and profiling of Big Data applications and frameworks
Wang Stream processing systems benchmark: Streambench
Bader Comparison of time series databases
Bensien Scalability benchmarking of stream processing engines with Apache Beam
Halas Performance Measurement of Stream Data Processing in Apache Spark
Zhang et al. uqSim: Scalable and Validated Simulation of Cloud Microservices
Vonheiden Empirical scalability evaluation of window aggregation methods in distributed stream processing
Anghel et al. Quantifying communication in graph analytics
CN110046319A (en) Social media information acquisition method, device, system, equipment and storage medium
Simonet et al. Active data: a data-centric approach to data life-cycle management
Higashino Complex event processing as a service in multi-cloud environments
Carias Knowledge-based performance monitoring for large scale HPC architectures
Khan Microservices metrics visualization
Formentin Design and implementation of a software pipeline for machine learning on streaming data
Bakayov et al. Blockchain Evolution