Nothing Special   »   [go: up one dir, main page]

Du et al., 2019 - Google Patents

OctopusKing: A TCT-aware task scheduling on spark platform

Du et al., 2019

Document ID
6319780336501772902
Author
Du H
Zhang K
Huang S
Publication year
Publication venue
2019 IEEE 25th International Conference on Parallel and Distributed Systems (ICPADS)

External Links

Snippet

The majority of large-scale data intensive applications executed by data centers are based on Map-Reduce or its open-source implementation, Spark. However, the native Spark's data locality has significant side effects. Spark prioritizes high-priority tasks. But tasks at the next …
Continue reading at ieeexplore.ieee.org (other versions)

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for programme control, e.g. control unit
    • G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Programme initiating; Programme switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for programme control, e.g. control unit
    • G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for programme control, e.g. control unit
    • G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for programme control, e.g. control unit
    • G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083—Techniques for rebalancing the load in a distributed system
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/50—Computer-aided design
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06N—COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N99/00—Subject matter not provided for in other groups of this subclass
    • G06N99/005—Learning machines, i.e. computer in which a programme is changed according to experience gained by the machine itself during a complete run
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06N—COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computer systems based on biological models
    • G06N3/02—Computer systems based on biological models using neural network models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06Q—DATA PROCESSING SYSTEMS OR METHODS, SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORY OR FORECASTING PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORY OR FORECASTING PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management, e.g. organising, planning, scheduling or allocating time, human or machine resources; Enterprise planning; Organisational models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F15/00—Digital computers in general; Data processing equipment in general
    • G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a programme unit and a register, e.g. for a simultaneous processing of several programmes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06N—COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computer systems utilising knowledge based models

Similar Documents

Publication Publication Date Title
CN110389820B (en) Private cloud task scheduling method for resource prediction based on v-TGRU model
CN107404523A (en) Cloud platform adaptive resource dispatches system and method
CN105046327B (en) A kind of intelligent grid information system and method based on machine learning techniques
Zhang et al. Virtual machine placement strategy using cluster-based genetic algorithm
CN111597043A (en) Method, device and system for calculating edge of whole scene
Dogani et al. K-agrued: A container autoscaling technique for cloud-based web applications in kubernetes using attention-based gru encoder-decoder
Wang et al. A computing resources prediction approach based on ensemble learning for complex system simulation in cloud environment
CN115794341A (en) Task scheduling method, device, equipment and storage medium based on artificial intelligence
Chu et al. LPG-model: A novel model for throughput prediction in stream processing, using a light gradient boosting machine, incremental principal component analysis, and deep gated recurrent unit network
Wang et al. A rhombic dodecahedron topology for human-centric banking big data
Chai Task scheduling based on swarm intelligence algorithms in high performance computing environment
Swain et al. An Intelligent Straggler Traffic Management Framework for Sustainable Cloud Environments
CN107025141B (en) Scheduling method based on big data mixed operation model
Du et al. OctopusKing: A TCT-aware task scheduling on spark platform
WO2023207630A1 (en) Task solving method and apparatus therefor
Konovalov et al. Job control in heterogeneous computing systems
Li Research on cloud computing resource scheduling based on machine learning
Zhuang et al. Overview of cloud computing resource allocation and management technology
Ghannane et al. Diviml: A module-based heuristic for mapping neural networks onto heterogeneous platforms
Jayanthi et al. Experimental Setup of Apache Spark Application Execution in a Standalone Cluster Environment using Default Scheduling Mode
Wu et al. Application of Improved Feature Pre-processing Method in Prevention and Control of Electricity Charge Risk
Hong et al. A complex task scheduling scheme for big data platforms based on Boolean satisfiability problem
Jiang et al. Energy optimization oriented three-way clustering algorithm for cloud tasks
Cérin et al. Accelerating the computation of multi-objectives scheduling solutions for cloud computing
Zhang et al. Design of Grid Resource Management System Based on Information Service.