Nothing Special   »   [go: up one dir, main page]

Zhou et al., 2019 - Google Patents

Cost-aware partitioning for efficient large graph processing in geo-distributed datacenters

Zhou et al., 2019

View PDF
Document ID
18010219357301456406
Author
Zhou A
Shen B
Xiao Y
Ibrahim S
He B
Publication year
Publication venue
IEEE Transactions on Parallel and Distributed Systems

External Links

Snippet

Graph processing is an emerging computation model for a wide range of applications and graph partitioning is important for optimizing the cost and performance of graph processing jobs. Recently, many graph applications store their data on geo-distributed datacenters …
Continue reading at inria.hal.science (PDF) (other versions)

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for programme control, e.g. control unit
    • G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for programme control, e.g. control unit
    • G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for programme control, e.g. control unit
    • G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Programme initiating; Programme switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30943—Information retrieval; Database structures therefor; File system structures therefor details of database functions independent of the retrieved data type
    • G06F17/30946—Information retrieval; Database structures therefor; File system structures therefor details of database functions independent of the retrieved data type indexing structures
    • G06F17/30958—Graphs; Linked lists
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F15/00—Digital computers in general; Data processing equipment in general
    • G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a programme unit and a register, e.g. for a simultaneous processing of several programmes
    • G06F15/163—Interprocessor communication
    • G06F15/173—Interprocessor communication using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30286—Information retrieval; Database structures therefor; File system structures therefor in structured data stores
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/50—Computer-aided design
    • G06F17/5009—Computer-aided design using simulation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network-specific arrangements or communication protocols supporting networked applications
    • H04L67/10—Network-specific arrangements or communication protocols supporting networked applications in which an application is distributed across nodes in the network
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/54—Store-and-forward switching systems
    • H04L12/56—Packet switching systems

Similar Documents

Publication Publication Date Title
Zhou et al. Cost-aware partitioning for efficient large graph processing in geo-distributed datacenters
Cheng et al. Network-aware locality scheduling for distributed data operators in data centers
Khayyat et al. Mizan: a system for dynamic load balancing in large-scale graph processing
Kumar et al. ARPS: An autonomic resource provisioning and scheduling framework for cloud platforms
Ke et al. On traffic-aware partition and aggregation in mapreduce for big data applications
Zhou et al. On achieving efficient data transfer for graph processing in geo-distributed datacenters
Xiao et al. Cost-aware big data processing across geo-distributed datacenters
Zhang et al. Accelerate large-scale iterative computation through asynchronous accumulative updates
Fu et al. An optimal locality-aware task scheduling algorithm based on bipartite graph modelling for spark applications
Ilkhechi et al. Network-aware virtual machine placement in cloud data centers with multiple traffic-intensive components
Kumar et al. Graph partitioning for parallel applications in heterogeneous grid environments
Wu et al. Optimizing the performance of big data workflows in multi-cloud environments under budget constraint
Mayer et al. Graph: Traffic-aware graph processing
Wesolowski et al. Tram: Optimizing fine-grained communication with topological routing and aggregation of messages
Liu et al. Scalable and adaptive data replica placement for geo-distributed cloud storages
Chen et al. Tology-aware optimal data placement algorithm for network traffic optimization
Teli et al. Big data migration between data centers in online cloud environment
Supreeth et al. An Efficient Policy‐Based Scheduling and Allocation of Virtual Machines in Cloud Computing Environment
Li et al. Wide-area spark streaming: Automated routing and batch sizing
Alkaff et al. Cross-layer scheduling in cloud systems
Ke et al. Aggregation on the fly: Reducing traffic for big data in the cloud
Shabeera et al. Optimising virtual machine allocation in MapReduce cloud for improved data locality
Veeravalli et al. Suboptimal solutions using integer approximation techniques for scheduling divisible loads on distributed bus networks
Ahmad et al. A semi distributed task allocation strategy for large hypercube supercomputers
WO2015055502A2 (en) Method of partitioning storage in a distributed data storage system and corresponding device