Nothing Special   »   [go: up one dir, main page]

WO2016187831A1 - 存取文件的方法、装置和存储系统 - Google Patents

存取文件的方法、装置和存储系统 Download PDF

Info

Publication number
WO2016187831A1
WO2016187831A1 PCT/CN2015/079894 CN2015079894W WO2016187831A1 WO 2016187831 A1 WO2016187831 A1 WO 2016187831A1 CN 2015079894 W CN2015079894 W CN 2015079894W WO 2016187831 A1 WO2016187831 A1 WO 2016187831A1
Authority
WO
WIPO (PCT)
Prior art keywords
file
call request
information
request message
request messages
Prior art date
Application number
PCT/CN2015/079894
Other languages
English (en)
French (fr)
Inventor
施国军
李志刚
肖福洲
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to CN201580014438.8A priority Critical patent/CN106537321B/zh
Priority to EP15892912.5A priority patent/EP3293625B1/en
Priority to PCT/CN2015/079894 priority patent/WO2016187831A1/zh
Priority to US15/576,637 priority patent/US10846265B2/en
Priority to KR1020177037049A priority patent/KR102063281B1/ko
Priority to JP2017561340A priority patent/JP6584529B2/ja
Publication of WO2016187831A1 publication Critical patent/WO2016187831A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
    • G06F9/38—Concurrent instruction execution, e.g. pipeline or look ahead
    • G06F9/3854—Instruction completion, e.g. retiring, committing or graduating
    • G06F9/3856—Reordering of instructions, e.g. using queues or age tags
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/13—File access structures, e.g. distributed indices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/06—Addressing a physical block of locations, e.g. base addressing, module addressing, memory dedication
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/1724—Details of de-fragmentation performed by the file system
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/182—Distributed file systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G06F3/0613—Improving I/O performance in relation to throughput

Definitions

  • the present invention relates to the field of information technology, and in particular, to a method, an apparatus, and a storage system for accessing files.
  • the file system usually allocates an inode (information node) for each file, allocates a data block data block for the actual data corresponding to each file, and records the number of the data block to the inode.
  • the data block size supported in the file system is 1K, 2K, 4K, and so on. The size of the data block is fixed during formatting, and each data block is numbered to facilitate the recording of the inode.
  • the invention provides a method, device and storage system for accessing files, which can improve the I/O efficiency of the file system as a whole.
  • a method for accessing a file comprising: determining L first file call request messages, each first file call request message in the L first file call request messages being used for request Calling a first file, the size of the first file is less than or equal to a threshold, and each first file call request message includes physical sector information, where the physical sector information includes an address of a physical sector storing the first file L is an integer greater than 1; according to the physical sector information, the L first file call request messages are respectively added to the M first file scheduling queues corresponding to the L first file call request messages, M a positive integer; processing K first file call request messages included in each first file scheduling queue of the M first file scheduling queues, and obtaining a call request message of each first file scheduling queue, The call request message of each first file scheduling queue is used to request to call the first file corresponding to the K first file call request messages, where K is a positive integer; accessing the first file corresponding to the K first file call request messages according to the call request message of each first file scheduling
  • the method further includes: determining, on a physical sector of the storage file, N first file storage space groups, where the N first file storage space groups are used For storing the first file, N is a positive integer; when at least two first file storage space groups of the N first file storage space groups have available storage space, the at least two first file storage space groups are included The first file is merged; when at least one of the N first file storage space groups does not store the file, the at least one first file storage space group is released; when the N first files are stored If the available storage space is not present in the space group, the number of storage space groups of the N first files is increased; and the K first file calls are accessed according to the call request message of each first file scheduling queue.
  • the first file corresponding to the request message includes: accessing the K first file call request messages from the N first file storage space groups according to the call request message of each first file scheduling queue Corresponding first file.
  • the L first file call request messages are respectively added to the L first files according to the physical sector information.
  • the M first file scheduling queues corresponding to the request message include: adding, according to the physical sector information, at least two first file call request messages in the L first file call request messages to the same first In the file scheduling queue, the addresses of the physical sectors of the first file requested by the at least two first file call request messages are consecutive.
  • each of the first file call request messages further includes access path information, where the access path information is used to indicate that the access is performed.
  • the first file calls the path of the physical sector of the first file corresponding to the message.
  • the determining the L first file invocation request messages includes: determining L first file information nodes, the L first files Each first file information node in the information node is configured to store the access path information of the first file invoked by each of the first file call request message requests; and each of the first file information nodes is determined according to the first file information node The first file invokes the access path information of the first file called by the request message request.
  • determining the L first file information nodes includes: determining, in the information node linked list, whether the L first file information nodes exist The i-th first file information node; when the information node list does not exist The i-th first file information node, the i-th first file information node is created in the information node linked list, and the access path information of the i-th first file is stored to the i-th first In the file information node, when the i-th first file information node exists in the information node linked list, the access path information of the i-th first file is determined according to the i-th first file information node.
  • a second aspect provides an apparatus for accessing a file, the apparatus comprising: a first determining module, configured to determine L first file call request messages, each of the first one of the L first file call request messages The file call request message is used to request to call the first file, the size of the first file is less than or equal to a threshold, and each first file call request message includes physical sector information, and the physical sector information includes storing the first file.
  • the address of the physical sector of the file, L is an integer greater than 1; an allocation module, configured to add the L first file call request messages determined by the first determining module to the L according to the physical sector information
  • M is a positive integer
  • the processing module is configured to schedule each first file in the M first file scheduling queues allocated to the allocation module
  • the K first file call request messages included in the queue are processed to obtain an invocation request message of each first file scheduling queue, and the call request message of each first file scheduling queue is used for Requesting to call the first file corresponding to the K first file call request message, K is a positive integer;
  • the calling module is configured to access the K according to the call request message of each first file scheduling queue obtained by the processing module
  • the first file calls the first file corresponding to the request message.
  • the apparatus further includes: a second determining module, configured to determine N first file storage space groups on the physical sector of the storage file, the N The first file storage space group is configured to store the first file, and when at least two first file storage space groups of the N first file storage space groups have available storage space, the at least two first file storage spaces are The first file included in the group is merged, or when at least one of the first file storage space groups of the N first file storage spaces does not store the file, the at least one first file storage space group is released, or when the N When the storage space group of the first file does not have the available storage space, the number of storage space groups of the N first files is increased; the calling module is further configured to: according to the call request of each first file scheduling queue The message accesses the first file corresponding to the K first file call request messages from the N first file storage space groups.
  • a second determining module configured to determine N first file storage space groups on the physical sector of the storage file, the N The first file storage space group is configured to store the first file, and when at least two
  • the processing module is specifically configured to: according to the physical sector information, at least two of the L first file call request messages The first file call request message is added to the same first file dispatch queue, At least two first file call request messages request that the address of the physical sector of the first file called is consecutive.
  • each of the first file call request messages further includes access path information, where the access path information is used to indicate that the access is performed.
  • the first file calls the path of the physical sector of the first file corresponding to the message.
  • the determining module is specifically configured to: determine L first file information nodes, each of the L first file information nodes The first file information node is configured to store the access path information of the first file invoked by each of the first file call request message requests; and determine each of the first file call request messages according to each of the first file information nodes The access path information of the first file requested to be called.
  • the determining module is specifically configured to: determine, in the information node linked list, whether the i-th number of the L first file information nodes exists a file information node; when the i-th first file information node does not exist in the information node linked list, the i-th first file information node is created in the information node linked list, and the i-th first file is The access path information is stored in the i-th first file information node; when the i-th first file information node exists in the information node linked list, determining the i-th according to the i-th first file information node The access path information of the first file.
  • a storage system comprising: a memory for storing files and programs; and a processor for executing the program to: determine L first file call request messages, the L Each first file call request message in the first file call request message is used to request to call a first file, the size of the first file is less than or equal to a threshold, and each first file call request message includes a physical sector Information, the physical sector information includes an address of a physical sector storing the first file, L is an integer greater than 1; and the L first file call request messages are respectively added to the L according to the physical sector information
  • M is a positive integer
  • the K first file calling requests included in each first file scheduling queue in the M first file scheduling queues The message is processed to obtain the call request message of each first file scheduling queue, and the call request message of each first file scheduling queue is used to request to call the K first file A first file corresponding to the request message, K is a positive integer;
  • the processor is further configured to: The processor is further configured to: determine, on a physical sector of the storage file, N first file storage space groups, where the N first file storage space groups are used to store the first file, when the N first file storages When at least two first file storage space groups in the space group have available storage space, merge the first files included in the at least two first file storage space groups, or at least the N first file storage space groups When the first file storage space group does not store the file, release the at least one first file storage space group, or when the storage space group of the N first files does not have the available storage space, increase the N first The number of storage space groups of the file; accessing the first file corresponding to the K first file call request messages from the N first file storage space groups according to the call request message of each first file scheduling queue .
  • the processor is specifically configured to: according to the physical sector information, at least two of the L first file call request messages The first file call request message is added to the same first file dispatch queue, and the addresses of the physical sectors of the first file requested by the at least two first file call request messages are consecutive.
  • each of the first file call request messages further includes access path information, where the access path information is used to indicate access to the The first file calls the path of the physical sector of the first file corresponding to the message.
  • the processor is specifically configured to: determine L first file information nodes, each of the L first file information nodes The first file information node is configured to store the access path information of the first file invoked by each of the first file call request message requests; and determine each of the first file call request messages according to each of the first file information nodes The access path information of the first file requested to be called.
  • the processor is specifically configured to: determine, in the information node linked list, whether the i-th number of the L first file information nodes exists a file information node; when the i-th first file information node does not exist in the information node linked list, the i-th first file information node is created in the information node linked list, and the i-th first file is The access path information is stored in the i-th first file information node; when the i-th first file information node exists in the information node linked list, determining the i-th according to the i-th first file information node The access path information of the first file.
  • the method, device, and storage system for accessing a file according to the foregoing technical solution, by determining a call request message of a small file, and allocating a corresponding first file scheduling queue for the call request message of the small file, thereby Combine call request messages for multiple small files and / Or delete processing, reduce the number of calls to small files, can avoid the number of I / O increased by the small file call request and lead to excessive memory resources, can reduce the number of system calls, and increase the throughput of I / O requests, In turn, the overall I/O efficiency of the file system is improved.
  • FIG. 1 is a schematic flowchart of a method of accessing a file according to an embodiment of the present invention.
  • FIG. 2 is another schematic flowchart of a method of accessing a file according to an embodiment of the present invention.
  • FIG. 3 is a schematic block diagram of an apparatus 200 for accessing a file in accordance with an embodiment of the present invention.
  • FIG. 4 is another schematic block diagram of an apparatus 200 for accessing a file in accordance with an embodiment of the present invention.
  • FIG. 5 is a schematic block diagram of a memory system 300 in accordance with an embodiment of the present invention.
  • the Linux system consists of tens of thousands of files, even reaching a million.
  • the file system organizes the storage of data using a directory structure and associates its original information with actual data.
  • Linux supports many different file systems, such as ext4, F2FS, Brtfs, Vfat, etc.
  • the Linux kernel associates different underlying file systems with an additional software layer VFS.
  • the system can divide the system into five modules: User Space, File System Layer, Block Device Layer, Multimedia Card (MMC) driver layer, and hardware.
  • MMC Multimedia Card
  • the User Space contains all the applications and standard libraries that handle I/O requests in user space.
  • the file system layer, the block device layer, and the MMC driver layer run in the kernel space.
  • the hardware refers to the MMC host controller, which mainly depends on the JEDEC standard and the hardware of each manufacturer. Implementation is not the focus of this patent discussion.
  • FIG. 1 shows a schematic flow diagram of a method 100 of accessing a file, which may be performed by a block device layer within a file system, in accordance with an embodiment of the present invention. As shown in FIG. 1, the method 100 includes:
  • each first file call request message in the L first file call request messages is used to request to call a first file, where the size of the first file is less than or equal to a threshold.
  • a value the each first file call request message includes physical sector information, the physical sector information including an address of a physical sector storing the first file, and L is an integer greater than one;
  • S120 Add, according to the physical sector information, the L first file call request messages to the M first file scheduling queues corresponding to the L first file call request messages, where M is a positive integer;
  • the file request message sent by the user equipment determines L first file call request messages, where L is an integer greater than 1, and the first file call request message is used to invoke the first file, the first file That is, a small file, each of the first file call request messages includes physical sector information, and the physical sector information includes an address of a physical sector storing the corresponding small file.
  • M is a positive integer
  • calling the L first first file call request messages according to physical sector information in each first file call request message Adding to the M first file scheduling queues respectively, processing the K first file call request messages allocated in each first file scheduling queue, for example, performing merge and/or delete processing, thereby obtaining each
  • the call request message corresponding to the first file scheduling queue is called according to the call request message of each first file scheduling queue, and the first file corresponding to the K first file call request message is called, for example, the first access may be performed. file.
  • the method for accessing a file in the embodiment of the present invention can determine a call request message of a small file, and allocate a corresponding first file scheduling queue for the call request message of the small file, so that a call request of multiple small files can be requested. Messages are merged and/or deleted to reduce the invocation of small files The number of times, the number of I/Os that can be increased by the small file call request can be avoided, and the excessive memory resources are occupied, which can reduce the number of system calls and increase the throughput of I/O requests, thereby improving the overall I/O of the file system. effectiveness.
  • the file system may determine L first file call request messages from the received file call request message, where L is a positive integer greater than one.
  • L is a positive integer greater than one.
  • Each first file request message in the L first file call request messages is used to request to call a corresponding first file.
  • the first file is a small file (Micro File, referred to as “MF”).
  • a threshold may be set, and a file whose file size is less than or equal to the threshold is determined as a small file, that is,
  • the first file for example, the threshold value may be set to 1K, that is, the file whose file size is less than or equal to 1K is limited to a small file, and the threshold value may also be set to 2M, 4M, etc., and the present invention is not limited thereto, and the file system
  • the call request message of the small file may be determined from the received file call request message, and the call request message of the small file is used to request the file system to call the corresponding small file.
  • the file system determines L first file call request messages, may receive a first file call request message at a time, or may receive multiple first file call request messages at a time, for example, the file system may receive at one time.
  • the three first files invoke the request message, and the file system determines L first file call request messages, that is, determines the received three first file call request messages. It is also possible to receive a random first file call request message, and when accumulating to L, determine L first file call request messages, the value of L may be determined according to an actual application, or may be set according to an empirical value, and the present invention Not limited to this.
  • each first file call request message may include physical sector information, where the physical sector information includes storing the first file that is requested by each of the first file call request message requests.
  • the address of the physical sector may specifically include: a start address of the first file that is requested to be invoked, a size of the first file to be called, and related information of a device that issues the first file call request message. at least one.
  • the file system may determine the first file to be invoked based on the first file call request message including the physical sector information.
  • the first file call request message may further include access path information, where the access path information includes path information of a physical sector storing the first file, according to the path information.
  • the access path information may be found in the file system to store the physical sector of the corresponding first file, that is, the physical sector in which the small file is stored.
  • the kernel of the file system allocates a file descriptor and returns it to the user program, and the pointer in the file descriptor table item points to the newly opened file.
  • the user program passes the file descriptor to the kernel, and the kernel finds the corresponding entry according to the file descriptor, and then finds the corresponding file by using the pointer in the entry.
  • Each file structure has a pointer to the dentry (directory entry) structure.
  • the kernel In order to reduce the number of disk reads, the kernel generally caches the tree structure of the directory, called dentry cache, where each inode node is a dentry structure, as long as the dentry search along each part of the path, dentry cache only Save the recently accessed directory entry. If there is no hit, read it from disk to memory.
  • the file system may store the information node MF-inode of the small file in the form of a linked list, and the information nodes of the small files respectively store the access path information of the corresponding small file, which may be according to the information node.
  • the access path information finds a small file.
  • the file system When the file system receives the ith first file call request message, searching for the information node of the corresponding i-th first file in the stored MF-inode linked list, if the i-th first file exists in the linked list And the information node determines, according to the information node of the i-th first file, access path information for indicating a storage path of the first file, where the access path information may be stored in the first i-th file call In the request message.
  • the information node of the corresponding i-th first file cannot be found in the MF-inode linked list, apply for a new inode corresponding to the i-th first file, and add the new inode to the MF-inode list. And saving the storage path and the file name corresponding to the first file to the new inode, recording the member variable of the inode, and storing the access path information in the ith first file call request message. .
  • the large file is truncated to a small file; the enlarged file can also be removed from the MF-inode list of the small file, and a normal inode can be assigned to it, which has no effect on performance. keep status quo.
  • the small file can be added to the MF-inode list. Alternatively, it can be done by the defragment daemon when the system is idle.
  • the Linux I/O scheduler is the main component of the block device I/O subsystem, which is between the common block layer and the block device driver, when the Linux kernel component wants to read and write some data. Instead of the request being issued, the kernel executes the request immediately, but defers it. Latency allocation is a key mechanism for block device performance. When a new block of data is transmitted, the kernel checks whether the new request can be satisfied by extending the previous request that has been pending.
  • each block device program has a request queue associated with it.
  • the request queue is allocated and initialized when the block device is initialized.
  • the system default scheduling algorithm is used.
  • the default scheduling algorithm prioritizes the most recent request with the previous processing request, and therefore ignores a request for a long time, which is the elevator algorithm.
  • a timeout timer is started to avoid a situation where a request is never processed.
  • read requests will generally take precedence over write requests, since read requests typically block the requesting process.
  • the algorithm may be used to process the call request message of the small file, and optionally, the following I/O scheduling algorithm may be used for scheduling.
  • the L first file call request messages may be respectively added to the M first file scheduling queues (MF dispatch Queues) according to the physical sector information in the L first file call request messages, where M is Integer.
  • the file system allocates a corresponding first file scheduling queue for the call request message of the small file, and the first file call request message received by the file system may be based on the physical sector information in each call request message.
  • a file call request message is respectively added to the M first file scheduling queues, so that at least two first file call request messages having physical sector addresses consecutive are added to the same first file scheduling queue, wherein each The first file scheduling queue includes K first file call request messages, and both K and M are positive integers.
  • the physical sector information in the three first file call request messages in the first file call request message indicates that the physical sector address is continuous and the other three are discontinuous.
  • Putting six first file call request messages into the first file dispatch queue, and merging three first file call request messages with consecutive physical sector addresses, that is, K 3 at this time; and the other three
  • the first file call request message with the physical sector address discontinuity is temporarily not processed.
  • the physical sector address is continuously judged.
  • whether the merging process is performed may be determined whether the physical sector address is continuous, and the merging of the first file calling request information may be performed according to the size of the first file that is invoked, for example, when two first files invoke the request message.
  • the sum of the sizes of the two first files requested to be written may constitute a block of data, and the two first file call request messages may also be merged, and the corresponding first request file may be written. Write in the same physical sector, the same Among the data blocks, the present invention is not limited to this.
  • the physical sector information in the three first file call request messages indicates that their physical sector addresses are consecutive and the other three are discontinuous.
  • the first file call request message with three physical sector addresses consecutively is placed in the same first file scheduling queue, for example, the three first file call request messages are put into the first first file scheduling queue.
  • the other three first file invocation requests that are not consecutive to each other can be placed in the other three first file scheduling queues, and one of the first file scheduling queues is temporarily empty.
  • the three first file call request messages in the first file scheduling queue may be merged.
  • the three first scheduling queues each having a first file call request message may not be merged first, and continue to wait for allocation of other first file scheduling request messages, when there are multiple first file scheduling queues.
  • the first file that can be merged invokes the request message, similarly, as with the first first file scheduling queue, a plurality of first file call request messages in the same first file scheduling queue are merged.
  • whether the merging process is performed may be determined whether the physical sector address is continuous, and the merging of the first file calling request information may be performed according to the size of the first file that is invoked, for example, when two first files invoke the request message.
  • the sum of the sizes of the two first files requested to be written may constitute a block of data, and the two first file call request messages may also be merged, and the corresponding first request file may be written.
  • the present invention is not limited to the same data block written in the same physical sector.
  • the physical sector information in the three first file call request messages indicates that the physical sector addresses are consecutive, and the other three are discontinuous, first the first file call request message consecutive to the three physical sector addresses is put into the same
  • the three first file call request messages are put into the first first file scheduling queue, and for the other three physical addresses that are not consecutive, the request message is invoked, and they can be Allocate according to size or randomly assigned to three first file scheduling queues; or they may be assigned to three first file scheduling queues according to the relevance of the storage addresses of the first files they request to call; They are randomly assigned to three first file dispatch queues.
  • the file invocation request message is merged and merged into a call request message of a first file scheduling queue.
  • the other two first file scheduling queues may not be processed first, or if the first file scheduling queue in the first first file scheduling queue has a merged first file call request message, it is temporarily not processed.
  • the request message for merging there are multiple first file call request messages that can be merged in the first file scheduling queue, similarly, like the first first file scheduling queue, multiple firsts in the same first file scheduling queue The file calls the request message for merging.
  • whether the merging process is performed may be determined whether the physical sector address is continuous, and the merging of the first file calling request information may be performed according to the size of the first file that is invoked, for example, when two first files invoke the request message.
  • the sum of the sizes of the two first files requested to be written may constitute a block of data, and the two first file call request messages may also be merged, and the corresponding first request file may be written.
  • the present invention is not limited to the same data block written in the same physical sector.
  • the call request messages received by the file system may be grouped according to different process IDs (PIDs) of the L first file call request messages. And storing the first file call request message in the grouped call request message to the first file scheduling queue, where the first file scheduling queue may include M scheduling queues.
  • a structure may be added to the file system to determine which is the first file call request message. When the first file call request message is determined in S110, the first file call request message may be identified. The first file call request message is distinguished by the identifier, and the first file call request message is allocated to the corresponding M first file dispatch queues, and each first file schedule corresponding may include K first file call request messages. .
  • each of the first file scheduling queues of the M first file scheduling queues in the file system may process the K first file call request messages included in the file system, and obtain the first file scheduling queue of each of the first file scheduling queues. Invoke the request message. Specifically, the L first file call request messages are added to the M first file scheduling queues, and for each first file scheduling queue, when at least two first files are included in the call request message, there is a continuous physical fan. a zone (bi_sector) address, the at least two first file call request messages may be merged, so that the at least two first file call request messages may be combined into one call request message, that is, a call request of the first file scheduling queue Messages, which reduces the number of calls.
  • the first file scheduling queue includes K first file call request messages, and the K pieces may be obtained according to the physical sector information included in the K first file call request message.
  • the first file invokes the request message for processing, and generates a call request message of the first file scheduling queue, the first The call request message of the file scheduling queue is used to call the first file corresponding to the K first file call request messages.
  • the file system may also delete the first file call request message, delete unnecessary call requests, and reduce the number of calls.
  • the temporary call request message may be temporarily The first file call request message is not processed.
  • the merge process is performed, and the present invention is not limited thereto.
  • the file system may access the corresponding first according to the obtained call request message of the first file scheduling queue. file. Specifically, since the access path information may be included in each first file call request message, after the first file scheduling queue is merged, the call request message of each first file scheduling queue still has corresponding access path information. Therefore, the first file can be accessed according to the access path information.
  • the call request message of the first file scheduling queue may be sent to a storage medium request queue (mmc_request), the mmc_request processes the call request message of the first file scheduling queue, and sets a device controller, which is mainly used to generate data and transmit When the data transmission is completed, the current call request message may be deleted from the corresponding scheduling queue, and the data transmission corresponding to the next request is started.
  • mmc_request storage medium request queue
  • the size of the sequence numbers of the above processes does not mean the order of execution, and the order of execution of each process should be determined by its function and internal logic, and should not be taken to the embodiments of the present invention.
  • the implementation process constitutes any limitation.
  • the method for accessing a file in the embodiment of the present invention can determine a call request message of a small file, and allocate a corresponding first file scheduling queue for the call request message of the small file, so that a call request of multiple small files can be requested.
  • the message is merged and/or deleted to reduce the number of calls to small files. It can avoid the increase of I/O times caused by small file call requests and cause excessive memory resources. It can reduce the number of system calls and increase I/O. The throughput of the request, which in turn improves the I/O efficiency of the file system as a whole.
  • the method 100 for accessing a file according to an embodiment of the present invention further includes:
  • S150 Determine, on a physical sector of the storage file, N first file storage space groups, where the N first file storage space groups are used to store the first file.
  • the file system may set a separate physical sector for accessing a small file, where the physical sector includes a plurality of individual regions, and the region includes a plurality of first file storage space groups (MF Group), the size of the area can be adjusted and optimized according to the time slot application, and the number of MF Groups included in the area can be dynamically adjusted according to requirements.
  • each MF Group may include a plurality of date blocks, and each date block may store one or more small files, and the present invention is not limited thereto.
  • the defragment daemon can be used to complete the merging and/or deleting of multiple MF groups when the system is idle. For example, multiple MF Groups can be combined to ensure that multiple first files are stored on consecutive physical sectors, thereby making better use of the characteristics of the eMMC.
  • the space for storing files in some MF Groups is left, that is, most of the space in the MF Group is added to the list to be deleted, and these MFs can be added.
  • the group merges and stores the data in at least two MF Groups into one or more MF Groups, thereby freeing up more space and improving space utilization.
  • the MF Groups may also be deleted. For example, when some small files in the MF Group become large files, for example, beyond the range of 4M, the MF Group where the enlarged file is located can be found for marking, and then the defragement daemon traverses the list to be deleted and finds the MF. The first file included in the group is added to the deletion list. When the file does not exist in the MF Group, that is, when the MF Groups is empty, the deletion is performed, and the MF Group is deleted to release the resource.
  • the MF group may also be added.
  • the defragment daemon when a small file is newly added, for example, when a large file becomes a small file, the defragment daemon first searches for multiple MF Groups for the MF Group with the remaining space, and if there is available storage space, the small file can be used. Saved to the MF Group, the newly added small file is stored in the MF Group; if there is no MF Groups with free space, the defragement daemon will add a new MF Group to store the small file and other new additions. Small file.
  • the file system may search for the first file that needs to be accessed in the MF Group of the stored file according to the call request message of the at least one first file scheduling queue.
  • the size of the sequence numbers of the above processes does not mean the order of execution, and the order of execution of each process should be determined by its function and internal logic, and should not be taken to the embodiments of the present invention.
  • the implementation process constitutes any limitation.
  • the method for accessing a file in the embodiment of the present invention can determine a call request message of a small file, and allocate a corresponding first file scheduling queue for the call request message of the small file, so that a call request of multiple small files can be requested.
  • the message is merged and/or deleted, which can avoid the increase of the number of I/Os caused by the small file call request and cause excessive memory resources, which can reduce the number of system calls and increase the throughput of I/O requests. Improve the I/O efficiency of the file system.
  • a corresponding dynamic storage area can be allocated for small files, and different small files can be aggregated and stored, which can reduce the waste of storage space and improve the utilization of space.
  • an apparatus 200 for accessing a file includes:
  • a first determining module 210 configured to determine L first file call request messages, each first file call request message in the L first file call request messages is used to request to call a first file, where the first file The size is less than or equal to the threshold value, and each of the first file call request messages includes physical sector information, where the physical sector information includes an address of a physical sector storing the first file, and L is an integer greater than one;
  • the allocating module 220 is configured to add, according to the physical sector information, the L first file call request messages determined by the first determining module to the M first files corresponding to the L first file call request messages.
  • M is a positive integer
  • the processing module 230 is configured to process K first file call request messages included in each first file scheduling queue of the M first file scheduling queues allocated by the allocation module, to obtain each first file scheduling The call request message of the queue, the call request message of each first file dispatch queue is used to request to call the first file corresponding to the K first file call request messages, where K is a positive integer;
  • the calling module 240 is configured to access the first file corresponding to the K first file call request messages according to the call request message of each of the first file scheduling queues obtained by the processing module.
  • the determining module 210 determines, from the call request message that the user equipment sends a file, an L request message of the first file, where L is an integer greater than 1, and the first file call request message is used to invoke the first file.
  • the first file is a small file, and the call request message of each small file may include physical sector information including an address of a physical sector in which the small file is stored.
  • the allocation module 220 allocates M first file scheduling queues for the L first file invocation request messages determined by the determining module 210, where M is a positive integer, and according to each first file, the request message is invoked.
  • the call request messages of the L first files are respectively added to the M first file scheduling queues, and the processing module 230 allocates K firsts to each of the first file scheduling queues.
  • the file invocation request message is processed, for example, a merge and/or delete process may be performed to obtain a call request message corresponding to each first file scheduling queue, and the calling module 240 may perform a scheduling request message according to each first file scheduling queue.
  • the K first file calls the first file corresponding to the request message to be called, for example, the first file can be accessed.
  • the apparatus for accessing a file in the embodiment of the present invention can determine a call request message of a small file and allocate a corresponding first file scheduling queue for the call request message of the small file, so that a call request of multiple small files can be requested.
  • the message is merged and/or deleted to reduce the number of calls to small files. It can avoid the increase of I/O times caused by small file call requests and cause excessive memory resources. It can reduce the number of system calls and increase I/O. The throughput of the request, which in turn improves the I/O efficiency of the file system as a whole.
  • the determining module 210 of the file system accessing the file may determine L first file call request messages from the received file call request message, and each of the L first file call request messages The first file request message is used to request to call a corresponding first file.
  • the first file is a small file (Micro File, referred to as “MF”).
  • a threshold may be set, and a file whose file size is less than or equal to the threshold is determined as a small file, that is,
  • the first file for example, the threshold value may be set to 1K, that is, the file whose file size is less than or equal to 1K is limited to a small file, and the threshold value may also be set to 2M, 4M, etc., and the present invention is not limited thereto, and the file system
  • the call request message of the small file may be determined from the received file call request message, and the call request message of the small file is used to request the file system to call the corresponding small file.
  • the file system determines L first file call request messages, may receive a first file call request message at a time, or may receive multiple first file call request messages at a time, for example, the file system may receive at one time.
  • the three first files invoke the request message, and the file system determines L first file call request messages, that is, determines the received three first file call request messages. It is also possible to receive a random first file call request message, and when accumulating to L, determine L first file call request messages, the value of L may be determined according to an actual application, or may be set according to an empirical value, and the present invention Not limited to this.
  • each of the first file call request messages in the L first file call request messages determined by the determining module 210 of the device 200 for accessing the file may include Physical sector information including an address of a physical sector of the first file in which the call request message requesting each of the first files is stored.
  • the physical sector information may specifically include a start address of the first file requested to be invoked, a size of the first file to be called, and related information of a device that issues the first file call request message.
  • the file system may determine the first file to be invoked based on the first file call request message including the physical sector information.
  • the first file call request message determined by the determining module 210 may further include access path information, where the access path information includes a corresponding physical sector storing the first file. Path information, according to which the physical sector storing the corresponding small file can be found in the file system.
  • the kernel of the file system allocates a file descriptor and returns it to the user program, and the pointer in the file descriptor table item points to the newly opened file.
  • the user program passes the file descriptor to the kernel, and the kernel finds the corresponding entry according to the file descriptor, and then finds the corresponding file by using the pointer in the entry.
  • Each file structure has a pointer to the dentry (directory entry) structure. We can find the inode node of the file according to the path.
  • the kernel In order to reduce the number of disk reads, the kernel generally caches the tree structure of the directory, called dentry cache, where each inode node is a dentry structure, as long as the dentry search along each part of the path, dentry cache only Save the recently accessed directory entry. If there is no hit, read it from disk to memory.
  • the file system may store the information node MF-inode of the small file in the form of a linked list, and the information nodes of the small files respectively store the access path information of the corresponding small file, which may be according to the information node.
  • the access path information finds a small file.
  • the file system When the file system receives the ith first file call request message, searching for the information node of the corresponding i-th first file in the stored MF-inode linked list, if the i-th first file exists in the linked list And the information node determines, according to the information node of the i-th first file, access path information for indicating a storage path of the first file, where the access path information may be stored in the first i-th file call In the request message.
  • the information node of the corresponding i-th first file cannot be found in the MF-inode linked list, apply for a new inode corresponding to the i-th first file, and add the new inode to the MF-inode list. And saving the storage path and the file name corresponding to the first file to the new inode, recording the member variable of the inode, and storing the access path information in the ith first file call request message. .
  • the large file can be truncated to a small file by calling the truncation (O_TRUNC) operation; Large files are removed from the MF-inode list of small files, and ordinary inodes are assigned to them. Since there is no effect on performance, the status can be maintained. Conversely, when a large file becomes a small file, the small file can be added to the MF-inode list. Alternatively, it can be done by the defragement daemon when the system is idle.
  • the Linux I/O scheduler is the main component of the block device I/O subsystem, which is between the common block layer and the block device driver, when the Linux kernel component wants to read and write some data. Instead of the request being issued, the kernel executes the request immediately, but defers it. Latency allocation is a key mechanism for block device performance. When a new block of data is transmitted, the kernel checks whether the new request can be satisfied by extending the previous request that has been pending.
  • each block device program has a request queue associated with it.
  • the request queue is allocated and initialized when the block device is initialized.
  • the system default scheduling algorithm is used.
  • the default scheduling algorithm prioritizes the most recent request with the previous processing request, and therefore ignores a request for a long time, which is the elevator algorithm.
  • a timeout timer is started to avoid a situation where a request is never processed.
  • read requests will generally take precedence over write requests, since read requests typically block the requesting process.
  • the algorithm may be used to process the call request message of the small file, and optionally, the following I/O scheduling algorithm may be used for scheduling.
  • the L first first file call request messages may be respectively added to the M first file scheduling queues by the allocation module 220 according to the physical sector information in the L first file call request messages. Dispatch Queue).
  • the file system allocates a corresponding first file scheduling queue for the call request message of the small file, and the first file call request message received by the file system may be based on the physical sector information in each call request message.
  • a file call request message is respectively added to the M first file scheduling queues, so that at least two first file call request messages having physical sector addresses consecutive are added to the same first file scheduling queue, wherein each The first file scheduling queue includes K first file call request messages, and both K and M are positive integers.
  • the information shows that its physical sector address is continuous and the other three are not consecutive.
  • Putting six first file call request messages into the first file dispatch queue, and merging three first file call request messages with consecutive physical sector addresses, that is, K 3 at this time; and the other three
  • the first file call request message with the physical sector address discontinuity is temporarily not processed.
  • the physical sector address is continuously judged.
  • whether the merging process is performed may be determined whether the physical sector address is continuous, and the merging of the first file calling request information may be performed according to the size of the first file that is invoked, for example, when two first files invoke the request message.
  • the sum of the sizes of the two first files requested to be written may constitute a block of data, and the two first file call request messages may also be merged, and the corresponding first request file may be written.
  • the present invention is not limited to the same data block written in the same physical sector.
  • the physical sector information in the three first file call request messages indicates that their physical sector addresses are consecutive and the other three are discontinuous.
  • the first file call request message with three physical sector addresses consecutively is placed in the same first file scheduling queue, for example, the three first file call request messages are put into the first first file scheduling queue.
  • the other three first file invocation requests that are not consecutive to each other can be placed in the other three first file scheduling queues, and one of the first file scheduling queues is temporarily empty.
  • the three first file call request messages in the first file scheduling queue may be merged.
  • the three first scheduling queues each having a first file call request message may not be merged first, and continue to wait for allocation of other first file scheduling request messages, when there are multiple first file scheduling queues.
  • the first file that can be merged invokes the request message, similarly, as with the first first file scheduling queue, a plurality of first file call request messages in the same first file scheduling queue are merged.
  • whether the merging process is performed may be determined whether the physical sector address is continuous, and the merging of the first file calling request information may be performed according to the size of the first file that is invoked, for example, when two first files invoke the request message.
  • the sum of the sizes of the two first files requested to be written may constitute a block of data.
  • the two first file call request messages may be merged, and the corresponding first file to be written is written in the same physical sector, and the present invention is not limited thereto.
  • the physical sector information in the three first file call request messages indicates that the physical sector addresses are consecutive, and the other three are discontinuous, first the first file call request message consecutive to the three physical sector addresses is put into the same
  • the three first file call request messages are put into the first first file scheduling queue, and for the other three physical addresses that are not consecutive, the request message is invoked, and they can be Allocate according to size or randomly assigned to three first file scheduling queues; or they may be assigned to three first file scheduling queues according to the relevance of the storage addresses of the first files they request to call; They are randomly assigned to three first file dispatch queues.
  • the three first file call request messages consecutive to the physical sector information in the first first file scheduling queue are merged and merged into a call request message of the first file scheduling queue.
  • the other two first file scheduling queues may not be processed first, or if the first file scheduling queue in the first first file scheduling queue has a merged first file call request message, it is temporarily not processed.
  • the request message for merging there are multiple first file call request messages that can be merged in the first file scheduling queue, similarly, like the first first file scheduling queue, multiple firsts in the same first file scheduling queue The file calls the request message for merging.
  • whether the merging process is performed may be determined whether the physical sector address is continuous, and the merging of the first file calling request information may be performed according to the size of the first file that is invoked, for example, when two first files invoke the request message.
  • the sum of the sizes of the two first files requested to be written may constitute a block of data, and the two first file call request messages may also be merged, and the corresponding first request file may be written.
  • the present invention is not limited to the same data block written in the same physical sector.
  • the allocating module 220 may group the call request messages received by the file system according to different process IDs (PIDs) of the L first file call request messages, and then the first file in the grouped call request message.
  • the call request message is stored in a scheduling queue of the first file, and the first file scheduling queue may include M scheduling queues.
  • a structure may be added to the file system to determine which is the first file call request message. When the first file call request message is determined in S110, the first file call request message may be identified. Distinguishing the first file call request message by the identifier, and assigning the first file call request message to the corresponding M In the first file scheduling queue, each first file scheduling corresponding may include K first file call request messages.
  • the processing module 230 in the file system processes the queue for each first file in the M first file scheduling queues, and may process the K first file call request messages included in the first file scheduling queue.
  • the L first file call request messages are added to the M first file scheduling queues, and for each first file scheduling queue, when at least two first files are included in the call request message, there is a continuous physical fan. a zone (bi_sector) address, the at least two first file call request messages may be merged, so that the at least two first file call request messages may be combined into one call request message, that is, a call request of the first file scheduling queue Messages, which reduces the number of calls.
  • the first file scheduling queue includes K first file call request messages, and the K pieces may be obtained according to the physical sector information included in the K first file call request message.
  • the first file invokes the request message for processing, and generates a call request message of the first file dispatch queue, where the call request message of the first file dispatch queue is used to invoke the first file corresponding to the K first file call request messages.
  • the file system may also delete the first file call request message, delete unnecessary call requests, and reduce the number of calls.
  • the temporary call request message may be temporarily The first file call request message is not processed.
  • the merge process is performed, and the present invention is not limited thereto.
  • the calling module 240 may schedule the queue according to the obtained first file.
  • the request message is invoked to access the corresponding first file.
  • the access path information may be included in each first file call request message, after the first file scheduling queue is merged, the call request message of each first file scheduling queue still has corresponding access path information. Therefore, the first file can be accessed according to the access path information.
  • the call request message of the first file scheduling queue may be sent to a storage medium request queue (mmc_request), the mmc_request processes the call request message of the first file scheduling queue, and sets a device controller, which is mainly used to generate data and transmit After the data transmission is completed, the current call request message may be deleted from the corresponding scheduling queue, and the next request corresponding to the next request is started. data transmission.
  • mmc_request storage medium request queue
  • the apparatus for accessing a file in the embodiment of the present invention can determine a call request message of a small file and allocate a corresponding first file scheduling queue for the call request message of the small file, so that a call request of multiple small files can be requested.
  • the message is merged and/or deleted to reduce the number of calls to small files. It can avoid the increase of I/O times caused by small file call requests and cause excessive memory resources. It can reduce the number of system calls and increase I/O. The throughput of the request, which in turn improves the I/O efficiency of the file system as a whole.
  • the device 200 for accessing a file further includes:
  • the second determining module 250 is configured to determine N first file storage space groups on the physical sector of the storage file, where the N first file storage space groups are used to store the first file.
  • the file system may set a separate physical sector for accessing a small file, where the physical sector includes a plurality of individual regions, and the region includes a plurality of first file storage space groups (MF Group), the size of the area can be adjusted and optimized according to the time slot application, and the number of MF Groups included in the area can be dynamically adjusted according to requirements.
  • each MF Group may include a plurality of date blocks, and each date block may store one or more small files, and the present invention is not limited thereto.
  • the second determining module 250 may complete the merging and/or deleting of the plurality of MF groups when the system is idle through a defragement daemon. For example, multiple MF Groups can be combined to ensure that multiple first files are stored on consecutive physical sectors, thereby making better use of the characteristics of the eMMC.
  • the space for storing files in some MF Groups is left, that is, most of the space in the MF Group is added to the list to be deleted, and these MFs can be added.
  • the group merges and stores the data in at least two MF Groups into one or more MF Groups, thereby freeing up more space and improving space utilization.
  • the MF Groups may also be deleted. For example, when some small files in the MF Group become large files, for example, beyond the range of 4M, the MF Group where the enlarged file is located can be found for marking, and then the defragement daemon traverses the list to be deleted and finds the MF. The first file included in the group is added to the deletion list. When the file does not exist in the MF Group, that is, when the MF Groups is empty, the deletion is performed, and the MF Group is deleted to release the resource.
  • the MF group may also be added.
  • the defragment daemon when a small file is newly added, for example, when a large file becomes a small file, the defragment daemon first searches for multiple MF Groups for the MF Group with the remaining space, and if there is available storage space, the small file can be used. Saved to the MF Group, the newly added small file is stored in the MF Group; if there is no MF Groups with free space, the defragement daemon will add a new MF Group to store the small file and other new additions. Small file.
  • the calling module 240 of the file system may search for the first file that needs to be accessed in the MF Group of the stored file according to the call request message of the at least one first file scheduling queue.
  • apparatus 200 for accessing files in accordance with embodiments of the present invention may correspond to performing method 100 in embodiments of the present invention, and that the above and other operations and/or functions of the various modules in apparatus 200 for accessing files are respectively The corresponding processes of the various methods in FIG. 1 and FIG. 2 are implemented, and are not described herein for brevity.
  • the apparatus for accessing a file in the embodiment of the present invention can determine a call request message of a small file and allocate a corresponding first file scheduling queue for the call request message of the small file, so that a call request of multiple small files can be requested.
  • the message is merged and/or deleted, which can avoid the increase of the number of I/Os caused by the small file call request and cause excessive memory resources, which can reduce the number of system calls and increase the throughput of I/O requests. Improve the I/O efficiency of the file system.
  • a corresponding dynamic storage area can be allocated for small files, and different small files can be aggregated and stored, which can reduce the waste of storage space and improve the utilization of space.
  • an embodiment of the present invention further provides a storage system 300, including a processor 310, a memory 320, and a bus system 330.
  • the processor 310 and the memory 320 are connected by a bus system 330 for storing instructions for executing instructions stored by the memory 320.
  • the memory 320 stores the program code, and the processor 310 can call the program code stored in the memory 320 to perform the following operations: determining L first file call request messages, each of the first files in the L first file call request messages The call request message is used to request to call the first file, the size of the first file is less than or equal to a threshold, and each first file call request message includes physical sector information, and the physical sector information includes storing the first file.
  • the address of the physical sector, L is an integer greater than 1; according to the physical sector information, the L first file call request messages are respectively added to the M first corresponding to the L first file call request messages
  • M is a positive integer; scheduling each first file in the M first file scheduling queues
  • the K first file calling request message included in the queue is processed to obtain the call request message of each first file scheduling queue, and the call request message of each first file scheduling queue is used to request to call the K first files.
  • K is a positive integer; accessing the first file corresponding to the K first file call request messages according to the call request message of each first file scheduling queue.
  • the storage system of the embodiment of the present invention can combine the call request messages of multiple small files by determining the call request message of the small file and allocating the corresponding first file scheduling queue for the call request messages of the small files. And / or delete processing, can avoid the number of I / O increased by the small file call request and lead to excessive memory resources, can reduce the number of system calls, and increase the throughput of I / O requests, thereby improving the overall file system I/O efficiency.
  • a corresponding dynamic storage area can be allocated for small files, and different small files can be aggregated and stored, which can reduce the waste of storage space and improve the utilization of space.
  • the processor 310 may be a central processing unit ("CPU"), and the processor 310 may also be other general-purpose processors, digital signal processors (DSPs). , an application specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, and the like.
  • the general purpose processor may be a microprocessor or the processor or any conventional processor or the like.
  • the memory 320 can include read only memory and random access memory and provides instructions and data to the processor 310. A portion of the memory 320 may also include a non-volatile random access memory. For example, the memory 320 can also store information of the device type.
  • the bus system 330 may include a power bus, a control bus, a status signal bus, and the like in addition to the data bus. However, for clarity of description, various buses are labeled as bus system 330 in the figure.
  • each step of the foregoing method may be completed by an integrated logic circuit of hardware in the processor 310 or an instruction in a form of software.
  • the steps of the method disclosed in the embodiments of the present invention may be directly implemented as a hardware processor, or may be performed by a combination of hardware and software modules in the processor.
  • the software module can be located in a conventional storage medium such as random access memory, flash memory, read only memory, programmable read only memory or electrically erasable programmable memory, registers, and the like.
  • the storage medium is located in the memory 320, and the processor 310 reads the information in the memory 320 and combines the hardware to perform the steps of the above method. To avoid repetition, it will not be described in detail here.
  • the processor 310 may call the process stored in the memory 320.
  • the sequence code performs the following operations: determining, on a physical sector of the storage file, N first file storage space groups, the N first file storage space groups for storing the first file, N being a positive integer; when the N When at least two first file storage space groups in the first file storage space group have available storage space, merge the first files included in the at least two first file storage space groups; when the N first file storage spaces When at least one first file storage space group in the group does not store the file, release the at least one first file storage space group; when the storage space group of the N first files does not have the available storage space, increase the N The number of storage space groups of the first file; accessing the first of the K first file call request messages from the N first file storage space groups according to the call request message of each first file scheduling queue a file.
  • the processor 310 may invoke the program code stored in the memory 320 to perform an operation of: calling at least two first files in the L first file in the request message according to the physical sector information.
  • the call request message is added to the same first file dispatch queue, and the addresses of the physical sectors of the first file requested by the at least two first file call request messages are consecutive.
  • each of the first file call request messages further includes access path information, where the access path information is used to indicate access to the physicality of the first file corresponding to each of the first file call messages.
  • the path of the sector is used to indicate access to the physicality of the first file corresponding to each of the first file call messages.
  • the processor 310 may invoke the program code stored in the memory 320 to perform the following operations: determining L first file information nodes, each of the first file information nodes The node is configured to store the access path information of the first file invoked by each of the first file call request message requests; and determine, according to each of the first file information nodes, the first call request message request The access path information of a file.
  • the processor 310 may call the program code stored in the memory 320 to perform the following operations: determining, in the information node linked list, whether the i-th first file information node of the L first file information nodes exists When the i-th first file information node does not exist in the information node linked list, the i-th first file information node is created in the information node linked list, and the access path of the i-th first file is created.
  • the information is stored in the i-th first file information node; when the i-th first file information node exists in the information node linked list, determining the i-th first file according to the i-th first file information node The access path information.
  • the storage system of the embodiment of the present invention can determine the call request message of the small file and allocate the corresponding first file scheduling queue for the call request message of the small file, thereby A small file call request message is merged and/or deleted, which can avoid the increase of I/O times caused by the small file call request and cause excessive memory resources, which can reduce the number of system calls and increase the I/O request. Throughput, which in turn improves the I/O efficiency of the file system as a whole.
  • a corresponding dynamic storage area can be allocated for small files, and different small files can be aggregated and stored, which can reduce the waste of storage space and improve the utilization of space.
  • the disclosed systems, devices, and methods may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the functions may be stored in a computer readable storage medium if implemented in the form of a software functional unit and sold or used as a standalone product. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, or a part of the technical solution, may be embodied in the form of a software product stored in a storage medium. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本发明实施例涉及存取文件的方法、装置和存储系统，该方法包括：确定L个第一文件调用请求消息；根据该L个第一文件调用请求消息中的物理扇区信息，将该L个第一文件调用请求消息分别添加到M个第一文件调度队列中；对每个第一文件调度队列包括的第一文件调用请求消息进行处理，获得每个第一文件调度队列的调用请求消息并用来存取第一文件。本发明实施例的存取文件的方法、装置和存储系统，通过为小文件的调用请求消息分配对应的调度队列，从而可以将多个小文件的调用请求消息进行合并处理，减少小文件的调用次数，可以避免小文件的调用请求而增加的I/O次数并导致的占用过多内存资源，并增加I/O请求的吞吐量，进而整体提升文件系统的I/O效率。

Description

存取文件的方法、装置和存储系统 技术领域
本发明涉及信息技术领域，尤其涉及存取文件的方法、装置和存储系统。
背景技术
文件系统通常为每一个文件分配一个inode(information node，信息节点)，为每个文件对应的实际数据分配数据块data block，并将data block的号码记录到inode中。文件系统中所支持的data block大小有1K，2K，4K等。在格式化时data block的大小就固定了，且每个data block都有编号，以方便inode的记录。
由于现有技术中每个data block仅能容纳一个文件的数据，而文件系统中往往存在大量的小文件，比如几百个字节或者更小。如果为这样的小文件分配单独的data block，便会引起内部碎片，浪费磁盘空间。而且，频繁的小文件读写还会增加输入(Input，I)/输出(Output，O)次数，产生过多的系统调用，系统调用也会引起上下文切换，占用过多的CPU和内存资源，从而降低文件系统的整体I/O效率。
发明内容
本发明提供了一种存取文件的方法、装置和存储系统，能够整体提升文件系统的I/O效率。
第一方面，提供了一种存取文件的方法，该方法包括：确定L个第一文件调用请求消息，该L个第一文件调用请求消息中的每个第一文件调用请求消息用于请求调用第一文件，该第一文件的大小小于或等于门限值，该每个第一文件调用请求消息包括物理扇区信息，该物理扇区信息包括存储该第一文件的物理扇区的地址，L为大于1的整数；根据该物理扇区信息，将该L个第一文件调用请求消息分别添加到与该L个第一文件调用请求消息对应的M个第一文件调度队列中，M为正整数；对该M个第一文件调度队列中的每个第一文件调度队列包括的K个第一文件调用请求消息进行处理，获得该每个第一文件调度队列的调用请求消息，该每个第一文件调度队列的调用请求消息用于请求调用该K个第一文件调用请求消息对应的第一文件，K为 正整数；根据该每个第一文件调度队列的调用请求消息，存取该K个第一文件调用请求消息对应的第一文件。
结合第一方面，在第一方面的一种实现方式中，该方法还包括：在存储文件的物理扇区上，确定N个第一文件存储空间组，该N个第一文件存储空间组用于存储第一文件，N为正整数；当该N个第一文件存储空间组中至少两个第一文件存储空间组存在可用存储空间时，将该至少两个第一文件存储空间组包括的第一文件进行合并；当该N个第一文件存储空间组中至少一个第一文件存储空间组不存储文件时，释放该至少一个第一文件存储空间组；当该N个第一文件的存储空间组都不存在该可用存储空间时，增加该N个第一文件的存储空间组的个数；该根据该每个第一文件调度队列的调用请求消息，存取该K个第一文件调用请求消息对应的第一文件，包括：根据该每个第一文件调度队列的调用请求消息，从该N个第一文件存储空间组中存取该K个第一文件调用请求消息对应的第一文件。
结合第一方面及其上述实现方式，在第一方面的另一种实现方式中，该根据该物理扇区信息，将该L个第一文件调用请求消息分别添加到与该L个第一文件调用请求消息对应的M个第一文件调度队列中，包括：根据该物理扇区信息，将该L个第一文件调用请求消息中的至少两个第一文件调用请求消息添加到同一个第一文件调度队列中，该至少两个第一文件调用请求消息请求调用的第一文件的物理扇区的地址连续。
结合第一方面及其上述实现方式，在第一方面的另一种实现方式中，该每个第一文件调用请求消息还包括存取路径信息，该存取路径信息用于指示存取该每个第一文件调用消息对应的第一文件的物理扇区的路径。
结合第一方面及其上述实现方式，在第一方面的另一种实现方式中，该确定L个第一文件调用请求消息，包括：确定L个第一文件信息节点，该L个第一文件信息节点中的每个第一文件信息节点用于存储该每个第一文件调用请求消息请求调用的第一文件的该存取路径信息；根据该每个第一文件信息节点，确定该每个第一文件调用请求消息请求调用的第一文件的该存取路径信息。
结合第一方面及其上述实现方式，在第一方面的另一种实现方式中，该确定L个第一文件信息节点，包括：在信息节点链表中确定是否存在该L个第一文件信息节点中第i个第一文件信息节点；当该信息节点链表中不存在 该第i个第一文件信息节点时，在该信息节点链表中创建该第i个第一文件信息节点，并将第i个第一文件的该存取路径信息存储到该第i个第一文件信息节点中；当该信息节点链表中存在该第i个第一文件信息节点时，根据该第i个第一文件信息节点确定该第i个第一文件的该存取路径信息。
第二方面，提供了一种存取文件的装置，该装置包括：第一确定模块，用于确定L个第一文件调用请求消息，该L个第一文件调用请求消息中的每个第一文件调用请求消息用于请求调用第一文件，该第一文件的大小小于或等于门限值，该每个第一文件调用请求消息包括物理扇区信息，该物理扇区信息包括存储该第一文件的物理扇区的地址，L为大于1的整数；分配模块，用于根据该物理扇区信息，将该第一确定模块确定的该L个第一文件调用请求消息分别添加到与该L个第一文件调用请求消息对应的M个第一文件调度队列中，M为正整数；处理模块，用于对该分配模块分配的该M个第一文件调度队列中的每个第一文件调度队列包括的K个第一文件调用请求消息进行处理，获得该每个第一文件调度队列的调用请求消息，该每个第一文件调度队列的调用请求消息用于请求调用该K个第一文件调用请求消息对应的第一文件，K为正整数；调用模块，用于根据该处理模块获得的该每个第一文件调度队列的调用请求消息，存取该K个第一文件调用请求消息对应的第一文件。
结合第二方面，在第二方面的一种实现方式中，该装置还包括：第二确定模块，用于在存储文件的物理扇区上，确定N个第一文件存储空间组，该N个第一文件存储空间组用于存储该第一文件，当该N个第一文件存储空间组中至少两个第一文件存储空间组存在可用存储空间时，将该至少两个第一文件存储空间组包括的第一文件进行合并，或当该N个第一文件存储空间组中至少一个第一文件存储空间组不存储文件时，释放该至少一个第一文件存储空间组，或当该N个第一文件的存储空间组都不存在该可用存储空间时，增加该N个第一文件的存储空间组的个数；该调用模块还用于：根据该每个第一文件调度队列的调用请求消息，从该N个第一文件存储空间组中存取该K个第一文件调用请求消息对应的第一文件。
结合第二方面及其上述实现方式，在第二方面的另一种实现方式中，该处理模块具体用于：根据该物理扇区信息，将该L个第一文件调用请求消息中的至少两个第一文件调用请求消息添加到同一个第一文件调度队列中，该 至少两个第一文件调用请求消息请求调用的第一文件的物理扇区的地址连续。
结合第二方面及其上述实现方式，在第二方面的另一种实现方式中，该每个第一文件调用请求消息还包括存取路径信息，该存取路径信息用于指示存取该每个第一文件调用消息对应的第一文件的物理扇区的路径。
结合第二方面及其上述实现方式，在第二方面的另一种实现方式中，该确定模块具体用于：确定L个第一文件信息节点，该L个第一文件信息节点中的每个第一文件信息节点用于存储该每个第一文件调用请求消息请求调用的第一文件的该存取路径信息；根据该每个第一文件信息节点，确定该每个第一文件调用请求消息请求调用的第一文件的该存取路径信息。
结合第二方面及其上述实现方式，在第二方面的另一种实现方式中，该确定模块具体用于：在信息节点链表中确定是否存在该L个第一文件信息节点中第i个第一文件信息节点；当该信息节点链表中不存在该第i个第一文件信息节点时，在该信息节点链表中创建该第i个第一文件信息节点，并将第i个第一文件的该存取路径信息存储到该第i个第一文件信息节点中；当该信息节点链表中存在该第i个第一文件信息节点时，根据该第i个第一文件信息节点确定该第i个第一文件的该存取路径信息。
第三方面，提供了一种存储系统，该存储系统包括：存储器，用于存储文件和程序；处理器，用于执行该程序，以实现：确定L个第一文件调用请求消息，该L个第一文件调用请求消息中的每个第一文件调用请求消息用于请求调用第一文件，该第一文件的大小小于或等于门限值，该每个第一文件调用请求消息包括物理扇区信息，该物理扇区信息包括存储该第一文件的物理扇区的地址，L为大于1的整数；根据该物理扇区信息，将该L个第一文件调用请求消息分别添加到与该L个第一文件调用请求消息对应的M个第一文件调度队列中，M为正整数；对该M个第一文件调度队列中的每个第一文件调度队列包括的K个第一文件调用请求消息进行处理，获得该每个第一文件调度队列的调用请求消息，该每个第一文件调度队列的调用请求消息用于请求调用该K个第一文件调用请求消息对应的第一文件，K为正整数；根据该每个第一文件调度队列的调用请求消息，存取该K个第一文件调用请求消息对应的第一文件。
结合第三方面，在第三方面的一种实现方式中，该处理器还用于：该处 理器还用于：在存储文件的物理扇区上，确定N个第一文件存储空间组，该N个第一文件存储空间组用于存储该第一文件，当该N个第一文件存储空间组中至少两个第一文件存储空间组存在可用存储空间时，将该至少两个第一文件存储空间组包括的第一文件进行合并，或当该N个第一文件存储空间组中至少一个第一文件存储空间组不存储文件时，释放该至少一个第一文件存储空间组，或当该N个第一文件的存储空间组都不存在该可用存储空间时，增加该N个第一文件的存储空间组的个数；根据该每个第一文件调度队列的调用请求消息，从该N个第一文件存储空间组中存取该K个第一文件调用请求消息对应的第一文件。
结合第三方面及其上述实现方式，在第三方面的另一种实现方式中，该处理器具体用于：根据该物理扇区信息，将该L个第一文件调用请求消息中的至少两个第一文件调用请求消息添加到同一个第一文件调度队列中，该至少两个第一文件调用请求消息请求调用的第一文件的物理扇区的地址连续。
结合第三方面及其上述实现方式，在第三方面的另一种实现方式中，该每个第一文件调用请求消息还包括存取路径信息，该存取路径信息用于指示存取该每个第一文件调用消息对应的第一文件的物理扇区的路径。
结合第三方面及其上述实现方式，在第三方面的另一种实现方式中，该处理器具体用于：确定L个第一文件信息节点，该L个第一文件信息节点中的每个第一文件信息节点用于存储该每个第一文件调用请求消息请求调用的第一文件的该存取路径信息；根据该每个第一文件信息节点，确定该每个第一文件调用请求消息请求调用的第一文件的该存取路径信息。
结合第三方面及其上述实现方式，在第三方面的另一种实现方式中，该处理器具体用于：在信息节点链表中确定是否存在该L个第一文件信息节点中第i个第一文件信息节点；当该信息节点链表中不存在该第i个第一文件信息节点时，在该信息节点链表中创建该第i个第一文件信息节点，并将第i个第一文件的该存取路径信息存储到该第i个第一文件信息节点中；当该信息节点链表中存在该第i个第一文件信息节点时，根据该第i个第一文件信息节点确定该第i个第一文件的该存取路径信息。
基于上述技术方案，本发明实施例的存取文件的方法、装置和存储系统，通过确定小文件的调用请求消息，并为这些小文件的调用请求消息分配对应的第一文件调度队列，从而可以将多个小文件的调用请求消息进行合并和/ 或删除处理，减少小文件的调用次数，可以避免小文件的调用请求而增加的I/O次数并导致的占用过多内存资源，可以减少系统调用次数，并增加I/O请求的吞吐量，进而整体提升文件系统的I/O效率。
附图说明
为了更清楚地说明本发明实施例的技术方案，下面将对本发明实施例中所需要使用的附图作简单地介绍，显而易见地，下面所描述的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是根据本发明实施例的存取文件的方法的示意性流程图。
图2是根据本发明实施例的存取文件的方法的另一示意性流程图。
图3是根据本发明实施例的存取文件的装置200的示意性框图。
图4是根据本发明实施例的存取文件的装置200的另一示意性框图。
图5是根据本发明实施例的存储系统300的示意性框图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明的一部分实施例，而不是全部实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都应属于本发明保护的范围。
经初步统计，linux系统由数以万记的文件组成，甚至达到百万级。文件系统使用目录结构组织存储数据，并将其原信息和实际数据关联起来。linux支持许多不同的文件系统，例如：ext4，F2FS，Brtfs，Vfat等。linux内核通过一个额外的软件层VFS将不同的底层文件系统关联起来。
根据I/O访问的流程，我们可以将系统分为五大模块：用户层(User Space)，文件系统层，块设备层，多媒体卡(Multimedia Card，简称“MMC”)驱动层，硬件。
这里的User Space包含所有在用户空间处理I/O请求的应用程序及标准库。
其中，文件系统层，块设备层，MMC驱动层运行在内核空间。
硬件是指MMC主机控制器，主要依赖于JEDEC标准和各厂家的硬件 实现，不作为本专利讨论的重点。
图1示出了根据本发明实施例的存取文件的方法100的示意性流程图，该方法100可以由文件系统内的块设备层执行。如图1所示，该方法100包括：
S110，确定L个第一文件调用请求消息，该L个第一文件调用请求消息中的每个第一文件调用请求消息用于请求调用第一文件，该第一文件的大小小于或等于门限值，该每个第一文件调用请求消息包括物理扇区信息，该物理扇区信息包括存储该第一文件的物理扇区的地址，L为大于1的整数；
S120，根据该物理扇区信息，将该L个第一文件调用请求消息分别添加到与该L个第一文件调用请求消息对应的M个第一文件调度队列中，M为正整数；
S130，对该M个第一文件调度队列中的每个第一文件调度队列包括的K个第一文件调用请求消息进行处理，获得该每个第一文件调度队列的调用请求消息，该每个第一文件调度队列的调用请求消息用于请求调用该K个第一文件调用请求消息对应的第一文件，K为正整数；
S140，根据该每个第一文件调度队列的调用请求消息，存取该K个第一文件调用请求消息对应的第一文件。
具体地，从用户设备发送的文件调用请求消息中，确定出L个第一文件调用请求消息，L为大于1的整数，该第一文件调用请求消息用于调用第一文件，该第一文件即为小文件，每个第一文件调用请求消息中都包括物理扇区信息，该物理扇区信息包括存储对应的小文件的物理扇区的地址。为L个第一文件调用请求消息分配M个第一文件调度队列，M为正整数，并根据每个第一文件调用请求消息中的物理扇区信息，将该L个第一文件调用请求消息分别添加到M个第一文件调度队列中，将每个第一文件调度队列中被分配到的K个第一文件调用请求消息进行处理，例如可以进行合并和/或删除处理，从而获得每个第一文件调度队列对应的调用请求消息，根据每个第一文件调度队列的调用请求消息，对K个第一文件调用请求消息所对应的第一文件进行调用，例如可以进行存取该第一文件。
因此，本发明实施例的存取文件的方法，通过确定小文件的调用请求消息，并为这些小文件的调用请求消息分配对应的第一文件调度队列，从而可以将多个小文件的调用请求消息进行合并和/或删除处理，减少小文件的调用 次数，可以避免小文件的调用请求而增加的I/O次数并导致的占用过多内存资源，可以减少系统调用次数，并增加I/O请求的吞吐量，进而整体提升文件系统的I/O效率。
在S110中，文件系统可以从接收到的文件调用请求消息中确定L个第一文件调用请求消息，L为大于1的正整数。该L个第一文件调用请求消息中每个第一文件请求消息用于请求调用对应的一个第一文件。可选地，该第一文件为小文件(Micro File，简称“MF”)，根据实际应用，可以设置门限值，将文件的大小小于或等于该门限值的文件确定为小文件，即第一文件，例如，门限值可以设置为1K，即将文件大小小于或等于1K的文件限定为小文件，门限值也可以设置为2M、4M等，本发明并不限于此，则文件系统可以从接收到的文件调用请求消息中确定小文件的调用请求消息，这些小文件的调用请求消息用于向文件系统请求调用对应的小文件。
在本发明实施例中，文件系统确定L个第一文件调用请求消息，可以一次接受一个第一文件调用请求消息，也可以一次接收多个第一文件调用请求消息，例如，文件系统可以一次接收3个第一文件调用请求消息，则文件系统确定L个第一文件调用请求消息，即确定接收到的3个第一文件调用请求消息。也可以接收随机的第一文件调用请求消息，当积累到L个时，即确定L个第一文件调用请求消息，该L的值可以根据实际应用确定，也可以根据经验值设置，本发明并不限于此。
可选地，在本发明实施例中，每个第一文件调用请求消息中可以包括物理扇区信息，该物理扇区信息包括存储该每个第一文件调用请求消息请求调用的第一文件的物理扇区的地址。可选地，该物理扇区信息中具体可以包括请求调用的第一文件的起始地址、被调用的第一文件的大小、以及下发该第一文件调用请求消息的设备的相关信息中的至少一个。文件系统可以根据包括该物理扇区信息的第一文件调用请求消息确定要调用的第一文件。
可选地，在本发明实施例中，该第一文件调用请求消息中还可以包括存取路径信息，该存取路径信息包括对应的存储该第一文件的物理扇区的路径信息，根据该存取路径信息可以在文件系统中找到存储对应的第一文件的物理扇区，即存储该小文件的物理扇区。具体地，当打开open或关闭close一个文件或创建一个新文件时，文件系统的内核会分配一个文件描述符并返回给用户程序，该文件描述符表项中的指针指向新打开的文件。在存取文件时， 用户程序把文件描述符传给内核，内核根据文件描述符找到相应的表项，再通过表项中的指针找到相应的文件。每个文件结构体都有一个指向dentry(directory entry，目录项)结构体的指针，我们可以根据路径找到文件的inode节点。为了减少读盘次数，内核一般会缓存了目录的树状结构，称为dentry cache，其中每个inode节点都是是一个dentry结构体，只要沿着路径各部分的dentry搜索即可，dentry cache只保存最近访问过的目录项，如果要没有命中，就要从磁盘读到内存中。
具体地，对于小文件，文件系统可以将小文件的信息节点MF-inode以链表的形式进行存储，这些小文件的信息节点分别存储了对应的小文件的存取路径信息，可以根据信息节点中的存取路径信息查找小文件。当文件系统接收到第i个第一文件调用请求消息时，在存储的MF-inode链表中查找对应的第i个第一文件的信息节点，若该链表中存在该第i个第一文件的信息节点，则根据该第i个第一文件的信息节点，确定用于指示该第一文件的存储路径的存取路径信息，可以将该存取路径信息存储到该第i个第一文件调用请求消息中。当在MF-inode链表中无法查找到对应的第i个第一文件的信息节点，则申请该第i个第一文件对应的新的inode，并将该新的inode加入到MF-inode链表中，同时将该第一文件对应的存储路径和文件名保存到该新的inode中，记录为该inode的成员变量，也将该存取路径信息存储到该第i个第一文件调用请求消息中。
可选地，对于小文件对应的MF-inode链表，当其中包括的inode对应的文件由于文件数据的不断写入，从小文件变为大文件时，例如当该小文件的大小超过4M时，可以通过调用截断(O_TRUNC)操作，使得大文件被截断为小文件；也可以将变大的文件移除小文件的MF-inode链表中，为其分配普通inode，由于性能上没有什么影响，也可以保持现状。相反地，当大文件变为小文件时，可以将该小文件添加到MF-inode链表，可选地，可以通过磁盘碎片整理程序(defragement daemon)在系统空闲的时候完成。
在本发明实施例中，Linux的I/O调度程序是块设备I/O子系统的主要组件，它介于通用块层和块设备驱动程序之间，当Linux内核组件要读写一些数据时，并不是请求一发出，内核便立即执行该请求，而是将其推迟执行。延迟分配是块设备性能的关键机制，当传输一个新数据块时，内核检查能否通过扩展前一个一直处于等待状态的请求而满足新请求。
具体地，每个块设备程序都有一个请求队列与之关联。在块设备初始化时，会分配并初始化请求队列。在这个时候，我们便可以为块设备驱动程序指定特定的I/O调度算法，默认情况下是使用系统默认的调度算法。默认的调度算法会优先处理与上一个处理请求最近的请求，因而就会对某个请求忽略很长一段时间，即为电梯算法。为了避免这种情况，当请求被传给电梯算法是，就会启动超时定时器，从而来避免某个请求永远得不到处理的情况。为了提升用户体验，读请求将一般优先于写请求，因为读请求通常阻塞发出请求的进程。
在本发明实施例中，可以采用这种算法对小文件的调用请求消息进行处理，可选地，还可以通过下面的I/O调度(scheduler)算法方法进行调度。在S120中，可以根据L个第一文件调用请求消息中的物理扇区信息，将该L个第一文件调用请求消息分别添加到M个第一文件调度队列(MF dispatch Queue)中，M为整数。文件系统为小文件的调用请求消息分配了对应的第一文件调度队列，对于文件系统接收到的第一文件调用请求消息，可以根据每个调用请求消息中的物理扇区信息，将L个第一文件调用请求消息分别添加到M个第一文件调度队列中，使得具有物理扇区地址连续的至少两个第一文件调用请求消息被添加到同一个第一文件调度队列中，其中，每个第一文件调度队列包括K个第一文件调用请求消息，K和M均为正整数。
例如，在本发明实施例中，文件系统确定了6个第一文件调用请求消息，即L＝6，当只有一个第一文件调度队列，即小文件的调度队列时，M＝1，而6个第一文件调用请求消息中存在三个第一文件调用请求消息中的物理扇区信息显示其物理扇区地址连续，另外三个不连续。将6个第一文件调用请求消息均放入该第一文件调度队列中，并将物理扇区地址连续的三个第一文件调用请求消息进行合并，即此时K＝3；而另外三个物理扇区地址不连续的第一文件调用请求消息暂时不做处理，待有新的第一文件调用请求消息时，再进行物理扇区地址是否连续的判断，若连续，则进行合并，如果不连续，则仍然不做处理。可选地，可以通过物理扇区地址是否连续来判断是否进行合并处理，还可以根据调用的第一文件的大小进行第一文件调用请求信息的合并，例如，当两个第一文件调用请求消息请求写入的两个第一文件的大小之和，恰好可以构成一个数据块(block)，则也可以将这两个第一文件调用请求消息合并，并将对应的要求写入的第一文件写入同一个物理扇区内，同一 个数据块中，本发明并不限于此。
可选地，还例如，文件系统确定了6个第一文件调用请求消息，即L＝6，且将第一文件调度队列分为5个，即小文件的调度队列有5个，M＝5时，而6个第一文件调用请求消息中存在三个第一文件调用请求消息中的物理扇区信息显示其物理扇区地址连续，另外三个不连续。则将该三个物理扇区地址连续的第一文件调用请求消息放入同一个第一文件调度队列中，例如将该三个第一文件调用请求消息放入第一个第一文件调度队列中，另外三个互相之间不连续的第一文件调用请求消息则可以放入另外三个第一文件调度队列中，而还有一个第一文件调度队列暂时为空。此时，第一个第一文件调度队列中存在3个第一文件调用请求消息，即K＝3，则可以对该第一文件调度队列中的三个第一文件调用请求消息进行合并处理，合并为一个第一文件调度队列的调用请求消息。而另外各有一个第一文件调用请求消息的三个第一调度队列则可以先不进行合并处理，继续等待分配其它第一文件调度请求消息，当这几个第一文件调度队列中存在多个可以合并的第一文件调用请求消息时，则同样地，与第一个第一文件调度队列一样，将在同一个第一文件调度队列中的多个第一文件调用请求消息进行合并。可选地，可以通过物理扇区地址是否连续来判断是否进行合并处理，还可以根据调用的第一文件的大小进行第一文件调用请求信息的合并，例如，当两个第一文件调用请求消息请求写入的两个第一文件的大小之和，恰好可以构成一个数据块(block)，则也可以将这两个第一文件调用请求消息合并，并将对应的要求写入的第一文件写入同一个物理扇区内，同一个数据块中，本发明并不限于此。
可选地，又例如，文件系统确定了6个第一文件调用请求消息，即L＝6，如果存在3个第一文件调度队列，即M＝3，并且6个第一文件调用请求消息中三个第一文件调用请求消息中的物理扇区信息显示其物理扇区地址连续，另外三个不连续，则先将该三个物理扇区地址连续的第一文件调用请求消息放入同一个第一文件调度队列中，例如将该三个第一文件调用请求消息放入第一个第一文件调度队列中，而对于另外三个物理地址不连续的第一文件调用请求消息，可以将他们按照大小分配或随机分配到三个第一文件调度队列中；或者也可以根据他们请求调用的第一文件的存储地址的相关性，将他们分配到三个第一文件调度队列中；还可以将他们随机分配到三个第一文件调度队列中。然后对第一个第一文件调度队列中物理扇区信息连续的三个第一 文件调用请求消息进行合并处理，合并为一个第一文件调度队列的调用请求消息。而另外的两个第一文件调度队列可以先不进行处理，或者第一个第一文件调度队列中如果还有没有被合并的第一文件调用请求消息，也暂时不进行处理，当这几个第一文件调度队列中存在多个可以合并的第一文件调用请求消息时，则同样地，与第一个第一文件调度队列一样，将在同一个第一文件调度队列中的多个第一文件调用请求消息进行合并。可选地，可以通过物理扇区地址是否连续来判断是否进行合并处理，还可以根据调用的第一文件的大小进行第一文件调用请求信息的合并，例如，当两个第一文件调用请求消息请求写入的两个第一文件的大小之和，恰好可以构成一个数据块(block)，则也可以将这两个第一文件调用请求消息合并，并将对应的要求写入的第一文件写入同一个物理扇区内，同一个数据块中，本发明并不限于此。
可选地，可以按照L个第一文件调用请求消息的进程号(PID)的不同，将文件系统接收的调用请求消息进行分组。再将分组后的调用请求消息中的第一文件调用请求消息存储到第一文件调度队列，该第一文件调度队列可以包括M个调度队列。可选地，可以在文件系统中新增一个结构体，用于判断哪些是第一文件调用请求消息；也可以在S110中确定第一文件调用请求消息时，将第一文件调用请求消息进行标识，通过该标识区分第一文件调用请求消息，并将第一文件调用请求消息分配到对应的M个第一文件调度队列中，每个第一文件调度对应可以包括K个第一文件调用请求消息。
在S130中，文件系统中的M个第一文件调度队列中的每个第一文件调度队列可以对其包括的K个第一文件调用请求消息进行处理，获得该每个第一文件调度队列的调用请求消息。具体地，将L个第一文件调用请求消息添加至M个第一文件调度队列，对于每个第一文件调度队列，当其中包括的至少两个第一文件调用请求消息，具有连续的物理扇区(bi_sector)地址，可以将该至少两个第一文件调用请求消息进行合并处理，使得该至少两个第一文件调用请求消息可以合成为一个调用请求消息，即第一文件调度队列的调用请求消息，从而减少了调用的次数。例如，对于任意一个第一文件调度队列，该第一文件调度队列包括K个第一文件调用请求消息，根据该K个第一文件调用请求消息中包括的物理扇区信息，可以对该K个第一文件调用请求消息进行处理，生成一个该第一文件调度队列的调用请求消息，该第一 文件调度队列的调用请求消息用于调用K个第一文件调用请求消息对应的第一文件。可选地，文件系统还可以对第一文件调用请求消息进行删除处理，删除不必要的调用请求，也可以减少调用次数。
可选地，当某个第一文件调度队列只包括一个第一文件调用请求消息时，即不存在与该调用请求消息中物理扇区地址相连续的另一个调用请求消息时，可以暂时对该第一文件调用请求消息不作处理，当该第一文件调度队列中包括至少两个第一文件调用请求消息的物理扇区地址连续时，再进行合并处理，本发明并不限于此。
在S140中，文件系统将第一文件调度队列中包括的K个第一文件调用请求消息进行合并或删除处理后，可以根据获得的第一文件调度队列的调用请求消息，存取相应的第一文件。具体地，由于每个第一文件调用请求消息中可以包括存取路径信息，在第一文件调度队列进行合并后，每个第一文件调度队列的调用请求消息仍然具有对应的存取路径信息，因此可以根据该存取路径信息，对应的对第一文件进行存取处理。可选地，第一文件调度队列的调用请求消息可以发送至存储介质请求队列(mmc_request)中，mmc_request处理第一文件调度队列的调用请求消息，并设置设备控制器，主要用于产生数据并传送，当数据传输完成后可以从对应的调度队列中删除当前的调用请求消息，并开始下一个请求对应的数据传输。
应理解，在本发明的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本发明实施例的实施过程构成任何限定。
因此，本发明实施例的存取文件的方法，通过确定小文件的调用请求消息，并为这些小文件的调用请求消息分配对应的第一文件调度队列，从而可以将多个小文件的调用请求消息进行合并和/或删除处理，减少小文件的调用次数，可以避免小文件的调用请求而增加的I/O次数并导致的占用过多内存资源，可以减少系统调用次数，并增加I/O请求的吞吐量，进而整体提升文件系统的I/O效率。
可选地，作为一个实施例，根据本发明实施例的存取文件的方法100还包括：
S150，在存储文件的物理扇区上，确定N个第一文件存储空间组，该N个第一文件存储空间组用于存储第一文件。
在本发明实施例中，具体地，文件系统可以为小文件的存取设置单独的物理扇区，该物理扇区中包括多个单独的区域，该区域包括多个第一文件存储空间组(MF Group)，该区域的大小可以根据时隙应用进行调整及优化，该区域内包括的MF Groups的个数可以根据需求动态调整。可选地，对于单独存放小文件的多个MF Group，每个MF Group可以包括多个数据库(date block)，每个date block可以存储一个或多个小文件，本发明并不限于此。
具体地，可以通过磁盘碎片整理程序(defragement daemon)在系统空闲时完成多个MF Group的合并和/或删除。例如，可以将多个MF Group进行合并，保证多个第一文件存储在连续的物理扇区上，进而可以更好地利用eMMC的特性。在系统的运行的过程中，由于文件的删除操作，使得某些MF Group中用于存储文件的空间有剩余，即该MF Group中大部分空间被加入了待删除的列表中，可以将这些MF Group进行合并，将至少两个MF Group中的数据合并存储到为一个或多个MF Group中，从而释放出更多的空间，提升空间的利用率。
可选地，作为一个实施例，也可以对MF Groups进行删除处理。例如，当MF Group中存在某些小文件变成大文件时，例如超过了4M的范围，可以找到该变大的文件所在的MF Group进行标记，然后当defragement daemon遍历待删除的列表，发现该MF Group中包括的第一文件均被加入到删除列表，使得该MF Group中已经不存在文件时，即该MF Groups为空时，会进行删除，删除该MF Group来释放资源。
可选地，作为一个实施例，也可以对MF Group进行增加处理。例如，当新增加了小文件时，例如：某个大文件变成小文件时，defragement daemon首先会在多个MF Group中查找存在剩余空间的MF Group，如果存在可用存储空间可以将该小文件存到该MF Group中，则将新增加的小文件存储到该MF Group中；如果不存在有剩余空间的MF Groups，defragement daemon会增加新的MF Group，用来存放该小文件以及其它新增加的小文件。
在本发明实施例，文件系统可以该根据该至少一个第一文件调度队列的调用请求消息，在存储文件的MF Group中查找需要存取调用的第一文件。
应理解，在本发明的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本发明实施例的实施过程构成任何限定。
因此，本发明实施例的存取文件的方法，通过确定小文件的调用请求消息，并为这些小文件的调用请求消息分配对应的第一文件调度队列，从而可以将多个小文件的调用请求消息进行合并和/或删除处理，可以避免小文件的调用请求而增加的I/O次数并导致的占用过多内存资源，可以减少系统调用次数，并增加I/O请求的吞吐量，进而整体提升文件系统的I/O效率。另外，还可以为小文件分配相应的动态存储区域，将不同的小文件聚合在一起存放，可以减少存储空间的浪费，提升空间的利用率。
上文中结合图1和图2，详细描述了根据本发明实施例的存取文件的方法，下面将结合图3和图4，描述根据本发明实施例的存取文件的装置200。
如图3所示，根据本发明实施例的存取文件的装置200包括：
第一确定模块210，用于确定L个第一文件调用请求消息，该L个第一文件调用请求消息中的每个第一文件调用请求消息用于请求调用第一文件，该第一文件的大小小于或等于门限值，该每个第一文件调用请求消息包括物理扇区信息，该物理扇区信息包括存储该第一文件的物理扇区的地址，L为大于1的整数；
分配模块220，用于根据该物理扇区信息，将该第一确定模块确定的该L个第一文件调用请求消息分别添加到与该L个第一文件调用请求消息对应的M个第一文件调度队列中，M为正整数；
处理模块230，用于对该分配模块分配的该M个第一文件调度队列中的每个第一文件调度队列包括的K个第一文件调用请求消息进行处理，获得该每个第一文件调度队列的调用请求消息，该每个第一文件调度队列的调用请求消息用于请求调用该K个第一文件调用请求消息对应的第一文件，K为正整数；
调用模块240，用于根据该处理模块获得的该每个第一文件调度队列的调用请求消息，存取该K个第一文件调用请求消息对应的第一文件。
具体地，确定模块210从用户设备发送文件的调用请求消息中，确定出L个第一文件的调用请求消息，L为大于1的整数，该第一文件调用请求消息用于调用第一文件，该第一文件即为小文件，每个小文件的调用请求消息可以包括物理扇区信息，该物理扇区信息包括存储该小文件的物理扇区的地址。分配模块220为确定模块210确定的L个第一文件调用请求消息分配M个第一文件调度队列，M为正整数，并根据每个第一文件调用请求消息中的 物理扇区信息，将该L个第一文件的调用请求消息分别添加到M个第一文件调度队列中，处理模块230针对每个第一文件调度队列，将被分配到其中的K个第一文件调用请求消息进行处理，例如可以进行合并和/或删除处理，从而获得每个第一文件调度队列对应的调用请求消息，调用模块240可以根据每个第一文件调度队列的调度请求消息，对K个第一文件调用请求消息所对应的第一文件进行调用，例如可以进行存取该第一文件。
因此，本发明实施例的存取文件的装置，通过确定小文件的调用请求消息，并为这些小文件的调用请求消息分配对应的第一文件调度队列，从而可以将多个小文件的调用请求消息进行合并和/或删除处理，减少小文件的调用次数，可以避免小文件的调用请求而增加的I/O次数并导致的占用过多内存资源，可以减少系统调用次数，并增加I/O请求的吞吐量，进而整体提升文件系统的I/O效率。
在本发明实施例中，文件系统存取文件的装置200的确定模块210可以从接收到的文件调用请求消息中确定L个第一文件调用请求消息，该L个第一文件调用请求消息中每个第一文件请求消息用于请求调用对应的一个第一文件。可选地，该第一文件为小文件(Micro File，简称“MF”)，根据实际应用，可以设置门限值，将文件的大小小于或等于该门限值的文件确定为小文件，即第一文件，例如，门限值可以设置为1K，即将文件大小小于或等于1K的文件限定为小文件，门限值也可以设置为2M、4M等，本发明并不限于此，则文件系统可以从接收到的文件调用请求消息中确定小文件的调用请求消息，这些小文件的调用请求消息用于向文件系统请求调用对应的小文件。
在本发明实施例中，文件系统确定L个第一文件调用请求消息，可以一次接受一个第一文件调用请求消息，也可以一次接收多个第一文件调用请求消息，例如，文件系统可以一次接收3个第一文件调用请求消息，则文件系统确定L个第一文件调用请求消息，即确定接收到的3个第一文件调用请求消息。也可以接收随机的第一文件调用请求消息，当积累到L个时，即确定L个第一文件调用请求消息，该L的值可以根据实际应用确定，也可以根据经验值设置，本发明并不限于此。
可选地，在本发明实施例中，存取文件的装置200的确定模块210确定的该L个第一文件调用请求消息中每个第一文件调用请求消息中可以包括 物理扇区信息，该物理扇区信息包括存储该每个第一文件的调用请求消息请求调用的第一文件的物理扇区的地址。可选地，该物理扇区信息中具体可以包括请求调用的第一文件的起始地址、被调用的第一文件的大小以及下发该第一文件调用请求消息的设备的相关信息等。文件系统可以根据包括该物理扇区信息的第一文件调用请求消息确定要调用的第一文件。
可选地，在本发明实施例中，确定模块210确定的该第一文件调用请求消息中还可以包括存取路径信息，该存取路径信息包括对应的存储该第一文件的物理扇区的路径信息，根据该存取路径信息可以在文件系统中找到存储对应的小文件的物理扇区。具体地，当打开open或关闭close一个文件或创建一个新文件时，文件系统的内核会分配一个文件描述符并返回给用户程序，该文件描述符表项中的指针指向新打开的文件。在存取文件时，用户程序把文件描述符传给内核，内核根据文件描述符找到相应的表项，再通过表项中的指针找到相应的文件。每个文件结构体都有一个指向dentry(directory entry，目录项)结构体的指针，我们可以根据路径找到文件的inode节点。为了减少读盘次数，内核一般会缓存了目录的树状结构，称为dentry cache，其中每个inode节点都是是一个dentry结构体，只要沿着路径各部分的dentry搜索即可，dentry cache只保存最近访问过的目录项，如果要没有命中，就要从磁盘读到内存中。
具体地，对于小文件，文件系统可以将小文件的信息节点MF-inode以链表的形式进行存储，这些小文件的信息节点分别存储了对应的小文件的存取路径信息，可以根据信息节点中的存取路径信息查找小文件。当文件系统接收到第i个第一文件调用请求消息时，在存储的MF-inode链表中查找对应的第i个第一文件的信息节点，若该链表中存在该第i个第一文件的信息节点，则根据该第i个第一文件的信息节点，确定用于指示该第一文件的存储路径的存取路径信息，可以将该存取路径信息存储到该第i个第一文件调用请求消息中。当在MF-inode链表中无法查找到对应的第i个第一文件的信息节点，则申请该第i个第一文件对应的新的inode，并将该新的inode加入到MF-inode链表中，同时将该第一文件对应的存储路径和文件名保存到该新的inode中，记录为该inode的成员变量，也将该存取路径信息存储到该第i个第一文件调用请求消息中。
可选地，对于小文件对应的MF-inode链表，当其中包括的inode对应的 文件由于文件数据的不断写入，从小文件变为大文件时，例如当该小文件的大小超过4M时，可以通过调用截断(O_TRUNC)操作，使得大文件被截断为小文件；也可以将变大的文件移除小文件的MF-inode链表中，为其分配普通inode，由于性能上没有什么影响，也可以保持现状。相反地，当大文件变为小文件时，可以将该小文件添加到MF-inode链表，可选地，可以通过磁盘碎片整理程序(defragement daemon)在系统空闲的时候完成。
在本发明实施例中，Linux的I/O调度程序是块设备I/O子系统的主要组件，它介于通用块层和块设备驱动程序之间，当Linux内核组件要读写一些数据时，并不是请求一发出，内核便立即执行该请求，而是将其推迟执行。延迟分配是块设备性能的关键机制，当传输一个新数据块时，内核检查能否通过扩展前一个一直处于等待状态的请求而满足新请求。
具体地，每个块设备程序都有一个请求队列与之关联。在块设备初始化时，会分配并初始化请求队列。在这个时候，我们便可以为块设备驱动程序指定特定的I/O调度算法，默认情况下是使用系统默认的调度算法。默认的调度算法会优先处理与上一个处理请求最近的请求，因而就会对某个请求忽略很长一段时间，即为电梯算法。为了避免这种情况，当请求被传给电梯算法是，就会启动超时定时器，从而来避免某个请求永远得不到处理的情况。为了提升用户体验，读请求将一般优先于写请求，因为读请求通常阻塞发出请求的进程。
在本发明实施例中，可以采用这种算法对小文件的调用请求消息进行处理，可选地，还可以通过下面的I/O调度(scheduler)算法方法进行调度。在本发明实施例中，可以通过分配模块220根据L个第一文件调用请求消息中的物理扇区信息，将该L个第一文件调用请求消息分别添加到M个第一文件调度队列(MF dispatch Queue)中。文件系统为小文件的调用请求消息分配了对应的第一文件调度队列，对于文件系统接收到的第一文件调用请求消息，可以根据每个调用请求消息中的物理扇区信息，将L个第一文件调用请求消息分别添加到M个第一文件调度队列中，使得具有物理扇区地址连续的至少两个第一文件调用请求消息被添加到同一个第一文件调度队列中，其中，每个第一文件调度队列包括K个第一文件调用请求消息，K和M均为正整数。
例如，在本发明实施例中，文件系统确定了6个第一文件调用请求消息， 即L＝6，当只有一个第一文件调度队列，即小文件的调度队列时，M＝1，而6个第一文件调用请求消息中存在三个第一文件调用请求消息中的物理扇区信息显示其物理扇区地址连续，另外三个不连续。将6个第一文件调用请求消息均放入该第一文件调度队列中，并将物理扇区地址连续的三个第一文件调用请求消息进行合并，即此时K＝3；而另外三个物理扇区地址不连续的第一文件调用请求消息暂时不做处理，待有新的第一文件调用请求消息时，再进行物理扇区地址是否连续的判断，若连续，则进行合并，如果不连续，则仍然不做处理。可选地，可以通过物理扇区地址是否连续来判断是否进行合并处理，还可以根据调用的第一文件的大小进行第一文件调用请求信息的合并，例如，当两个第一文件调用请求消息请求写入的两个第一文件的大小之和，恰好可以构成一个数据块(block)，则也可以将这两个第一文件调用请求消息合并，并将对应的要求写入的第一文件写入同一个物理扇区内，同一个数据块中，本发明并不限于此。
可选地，还例如，文件系统确定了6个第一文件调用请求消息，即L＝6，且将第一文件调度队列分为5个，即小文件的调度队列有5个，M＝5时，而6个第一文件调用请求消息中存在三个第一文件调用请求消息中的物理扇区信息显示其物理扇区地址连续，另外三个不连续。则将该三个物理扇区地址连续的第一文件调用请求消息放入同一个第一文件调度队列中，例如将该三个第一文件调用请求消息放入第一个第一文件调度队列中，另外三个互相之间不连续的第一文件调用请求消息则可以放入另外三个第一文件调度队列中，而还有一个第一文件调度队列暂时为空。此时，第一个第一文件调度队列中存在3个第一文件调用请求消息，即K＝3，则可以对该第一文件调度队列中的三个第一文件调用请求消息进行合并处理，合并为一个第一文件调度队列的调用请求消息。而另外各有一个第一文件调用请求消息的三个第一调度队列则可以先不进行合并处理，继续等待分配其它第一文件调度请求消息，当这几个第一文件调度队列中存在多个可以合并的第一文件调用请求消息时，则同样地，与第一个第一文件调度队列一样，将在同一个第一文件调度队列中的多个第一文件调用请求消息进行合并。可选地，可以通过物理扇区地址是否连续来判断是否进行合并处理，还可以根据调用的第一文件的大小进行第一文件调用请求信息的合并，例如，当两个第一文件调用请求消息请求写入的两个第一文件的大小之和，恰好可以构成一个数据块(block)， 则也可以将这两个第一文件调用请求消息合并，并将对应的要求写入的第一文件写入同一个物理扇区内，同一个数据块中，本发明并不限于此。
可选地，又例如，文件系统确定了6个第一文件调用请求消息，即L＝6，如果存在3个第一文件调度队列，即M＝3，并且6个第一文件调用请求消息中三个第一文件调用请求消息中的物理扇区信息显示其物理扇区地址连续，另外三个不连续，则先将该三个物理扇区地址连续的第一文件调用请求消息放入同一个第一文件调度队列中，例如将该三个第一文件调用请求消息放入第一个第一文件调度队列中，而对于另外三个物理地址不连续的第一文件调用请求消息，可以将他们按照大小分配或随机分配到三个第一文件调度队列中；或者也可以根据他们请求调用的第一文件的存储地址的相关性，将他们分配到三个第一文件调度队列中；还可以将他们随机分配到三个第一文件调度队列中。然后对第一个第一文件调度队列中物理扇区信息连续的三个第一文件调用请求消息进行合并处理，合并为一个第一文件调度队列的调用请求消息。而另外的两个第一文件调度队列可以先不进行处理，或者第一个第一文件调度队列中如果还有没有被合并的第一文件调用请求消息，也暂时不进行处理，当这几个第一文件调度队列中存在多个可以合并的第一文件调用请求消息时，则同样地，与第一个第一文件调度队列一样，将在同一个第一文件调度队列中的多个第一文件调用请求消息进行合并。可选地，可以通过物理扇区地址是否连续来判断是否进行合并处理，还可以根据调用的第一文件的大小进行第一文件调用请求信息的合并，例如，当两个第一文件调用请求消息请求写入的两个第一文件的大小之和，恰好可以构成一个数据块(block)，则也可以将这两个第一文件调用请求消息合并，并将对应的要求写入的第一文件写入同一个物理扇区内，同一个数据块中，本发明并不限于此。
可选地，分配模块220可以按照L个第一文件调用请求消息的进程号(PID)的不同，将文件系统接收的调用请求消息进行分组，再将分组后的调用请求消息中的第一文件调用请求消息存储到第一文件的调度队列，该第一文件调度队列可以包括M个调度队列。可选地，可以在文件系统中新增一个结构体，用于判断哪些是第一文件调用请求消息；也可以在S110中确定第一文件调用请求消息时，将第一文件调用请求消息进行标识，通过该标识区分第一文件调用请求消息，并将第一文件调用请求消息分配到对应的M 个第一文件调度队列中，每个第一文件调度对应可以包括K个第一文件调用请求消息。
在本发明实施例中，文件系统中的处理模块230针对M个第一文件调度队列中的每个第一文件调度队列，可以对其包括的K个第一文件调用请求消息进行处理，获得该每个第一文件调度队列的调用请求消息。具体地，将L个第一文件调用请求消息添加至M个第一文件调度队列，对于每个第一文件调度队列，当其中包括的至少两个第一文件调用请求消息，具有连续的物理扇区(bi_sector)地址，可以将该至少两个第一文件调用请求消息进行合并处理，使得该至少两个第一文件调用请求消息可以合成为一个调用请求消息，即第一文件调度队列的调用请求消息，从而减少了调用的次数。例如，对于任意一个第一文件调度队列，该第一文件调度队列包括K个第一文件调用请求消息，根据该K个第一文件调用请求消息中包括的物理扇区信息，可以对该K个第一文件调用请求消息进行处理，生成一个该第一文件调度队列的调用请求消息，该第一文件调度队列的调用请求消息用于调用K个第一文件调用请求消息对应的第一文件。可选地，文件系统还可以对第一文件调用请求消息进行删除处理，删除不必要的调用请求，也可以减少调用次数。
可选地，当某个第一文件调度队列只包括一个第一文件调用请求消息时，即不存在与该调用请求消息中物理扇区地址相连续的另一个调用请求消息时，可以暂时对该第一文件调用请求消息不作处理，当该第一文件的调度队列中包括至少两个第一文件调用请求消息的物理扇区地址连续时，再进行合并处理，本发明并不限于此。
在本发明实施例中，文件系统的处理模块230将第一文件调度队列中包括的K个第一文件调用请求消息进行合并或删除处理后，调用模块240可以根据获得的第一文件调度队列的调用请求消息，存取相应的第一文件。具体地，由于每个第一文件调用请求消息中可以包括存取路径信息，在第一文件调度队列进行合并后，每个第一文件调度队列的调用请求消息仍然具有对应的存取路径信息，因此可以根据该存取路径信息，对应的对第一文件进行存取处理。可选地，第一文件调度队列的调用请求消息可以发送至存储介质请求队列(mmc_request)中，mmc_request处理第一文件调度队列的调用请求消息，并设置设备控制器，主要用于产生数据并传送，当数据传输完成后可以从对应的调度队列中删除当前的调用请求消息，并开始下一个请求对应的 数据传输。
因此，本发明实施例的存取文件的装置，通过确定小文件的调用请求消息，并为这些小文件的调用请求消息分配对应的第一文件调度队列，从而可以将多个小文件的调用请求消息进行合并和/或删除处理，减少小文件的调用次数，可以避免小文件的调用请求而增加的I/O次数并导致的占用过多内存资源，可以减少系统调用次数，并增加I/O请求的吞吐量，进而整体提升文件系统的I/O效率。
可选地，作为一个实施例，根据本发明实施例的存取文件的装置200还包括：
第二确定模块250，用于在存储文件的物理扇区上，确定N个第一文件存储空间组，该N个第一文件存储空间组用于存储第一文件。
在本发明实施例中，具体地，文件系统可以为小文件的存取设置单独的物理扇区，该物理扇区中包括多个单独的区域，该区域包括多个第一文件存储空间组(MF Group)，该区域的大小可以根据时隙应用进行调整及优化，该区域内包括的MF Groups的个数可以根据需求动态调整。可选地，对于单独存放小文件的多个MF Group，每个MF Group可以包括多个数据库(date block)，每个date block可以存储一个或多个小文件，本发明并不限于此。
具体地，第二确定模块250可以通过磁盘碎片整理程序(defragement daemon)在系统空闲时完成多个MF Group的合并和/或删除。例如，可以将多个MF Group进行合并，保证多个第一文件存储在连续的物理扇区上，进而可以更好地利用eMMC的特性。在系统的运行的过程中，由于文件的删除操作，使得某些MF Group中用于存储文件的空间有剩余，即该MF Group中大部分空间被加入了待删除的列表中，可以将这些MF Group进行合并，将至少两个MF Group中的数据合并存储到为一个或多个MF Group中，从而释放出更多的空间，提升空间的利用率。
可选地，作为一个实施例，也可以对MF Groups进行删除处理。例如，当MF Group中存在某些小文件变成大文件时，例如超过了4M的范围，可以找到该变大的文件所在的MF Group进行标记，然后当defragement daemon遍历待删除的列表，发现该MF Group中包括的第一文件均被加入到删除列表，使得该MF Group中已经不存在文件时，即该MF Groups为空时，会进行删除，删除该MF Group来释放资源。
可选地，作为一个实施例，也可以对MF Group进行增加处理。例如，当新增加了小文件时，例如：某个大文件变成小文件时，defragement daemon首先会在多个MF Group中查找存在剩余空间的MF Group，如果存在可用存储空间可以将该小文件存到该MF Group中，则将新增加的小文件存储到该MF Group中；如果不存在有剩余空间的MF Groups，defragement daemon会增加新的MF Group，用来存放该小文件以及其它新增加的小文件。
在本发明实施例，文件系统的调用模块240可以该根据该至少一个第一文件调度队列的调用请求消息，在存储文件的MF Group中查找需要存取调用的第一文件。
应理解，根据本发明实施例的存取文件的装置200可对应于执行本发明实施例中的方法100，并且存取文件的装置200中的各个模块的上述和其它操作和/或功能分别为了实现图1和图2中的各个方法的相应流程，为了简洁，在此不再赘述。
因此，本发明实施例的存取文件的装置，通过确定小文件的调用请求消息，并为这些小文件的调用请求消息分配对应的第一文件调度队列，从而可以将多个小文件的调用请求消息进行合并和/或删除处理，可以避免小文件的调用请求而增加的I/O次数并导致的占用过多内存资源，可以减少系统调用次数，并增加I/O请求的吞吐量，进而整体提升文件系统的I/O效率。另外，还可以为小文件分配相应的动态存储区域，将不同的小文件聚合在一起存放，可以减少存储空间的浪费，提升空间的利用率。
如图5所示，本发明实施例还提供了一种存储系统300，包括处理器310、存储器320和总线系统330。其中，处理器310和存储器320通过总线系统330相连，该存储器320用于存储指令，该处理器310用于执行该存储器320存储的指令。该存储器320存储程序代码，且处理器310可以调用存储器320中存储的程序代码执行以下操作：确定L个第一文件调用请求消息，该L个第一文件调用请求消息中的每个第一文件调用请求消息用于请求调用第一文件，该第一文件的大小小于或等于门限值，该每个第一文件调用请求消息包括物理扇区信息，该物理扇区信息包括存储该第一文件的物理扇区的地址，L为大于1的整数；根据该物理扇区信息，将该L个第一文件调用请求消息分别添加到与该L个第一文件调用请求消息对应的M个第一文件调度队列中，M为正整数；对该M个第一文件调度队列中的每个第一文件调度 队列包括的K个第一文件调用请求消息进行处理，获得该每个第一文件调度队列的调用请求消息，该每个第一文件调度队列的调用请求消息用于请求调用该K个第一文件调用请求消息对应的第一文件，K为正整数；根据该每个第一文件调度队列的调用请求消息，存取该K个第一文件调用请求消息对应的第一文件。
因此，本发明实施例的存储系统，通过确定小文件的调用请求消息，并为这些小文件的调用请求消息分配对应的第一文件调度队列，从而可以将多个小文件的调用请求消息进行合并和/或删除处理，可以避免小文件的调用请求而增加的I/O次数并导致的占用过多内存资源，可以减少系统调用次数，并增加I/O请求的吞吐量，进而整体提升文件系统的I/O效率。另外，还可以为小文件分配相应的动态存储区域，将不同的小文件聚合在一起存放，可以减少存储空间的浪费，提升空间的利用率。
应理解，在本发明实施例中，该处理器310可以是中央处理单元(Central Processing Unit，简称为“CPU”)，该处理器310还可以是其他通用处理器、数字信号处理器(DSP)、专用集成电路(ASIC)、现成可编程门阵列(FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
该存储器320可以包括只读存储器和随机存取存储器，并向处理器310提供指令和数据。存储器320的一部分还可以包括非易失性随机存取存储器。例如，存储器320还可以存储设备类型的信息。
该总线系统330除包括数据总线之外，还可以包括电源总线、控制总线和状态信号总线等。但是为了清楚说明起见，在图中将各种总线都标为总线系统330。
在实现过程中，上述方法的各步骤可以通过处理器310中的硬件的集成逻辑电路或者软件形式的指令完成。结合本发明实施例所公开的方法的步骤可以直接体现为硬件处理器执行完成，或者用处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器320，处理器310读取存储器320中的信息，结合其硬件完成上述方法的步骤。为避免重复，这里不再详细描述。
可选地，作为一个实施例，处理器310可以调用存储器320中存储的程 序代码执行以下操作：在存储文件的物理扇区上，确定N个第一文件存储空间组，该N个第一文件存储空间组用于存储第一文件，N为正整数；当该N个第一文件存储空间组中至少两个第一文件存储空间组存在可用存储空间时，将该至少两个第一文件存储空间组包括的第一文件进行合并；当该N个第一文件存储空间组中至少一个第一文件存储空间组不存储文件时，释放该至少一个第一文件存储空间组；当该N个第一文件的存储空间组都不存在该可用存储空间时，增加该N个第一文件的存储空间组的个数；根据该每个第一文件调度队列的调用请求消息，从该N个第一文件存储空间组中存取该K个第一文件调用请求消息对应的第一文件。
可选地，作为一个实施例，处理器310可以调用存储器320中存储的程序代码执行以下操作：根据该物理扇区信息，将该L个第一文件调用请求消息中的至少两个第一文件调用请求消息添加到同一个第一文件调度队列中，该至少两个第一文件调用请求消息请求调用的第一文件的物理扇区的地址连续。
可选地，作为一个实施例，该每个第一文件调用请求消息还包括存取路径信息，该存取路径信息用于指示存取该每个第一文件调用消息对应的第一文件的物理扇区的路径。
可选地，作为一个实施例，处理器310可以调用存储器320中存储的程序代码执行以下操作：确定L个第一文件信息节点，该L个第一文件信息节点中的每个第一文件信息节点用于存储该每个第一文件调用请求消息请求调用的第一文件的该存取路径信息；根据该每个第一文件信息节点，确定该每个第一文件调用请求消息请求调用的第一文件的该存取路径信息。
可选地，作为一个实施例，处理器310可以调用存储器320中存储的程序代码执行以下操作：在信息节点链表中确定是否存在该L个第一文件信息节点中第i个第一文件信息节点；当该信息节点链表中不存在该第i个第一文件信息节点时，在该信息节点链表中创建该第i个第一文件信息节点，并将第i个第一文件的该存取路径信息存储到该第i个第一文件信息节点中；当该信息节点链表中存在该第i个第一文件信息节点时，根据该第i个第一文件信息节点确定该第i个第一文件的该存取路径信息。
因此，本发明实施例的存储系统，通过确定小文件的调用请求消息，并为这些小文件的调用请求消息分配对应的第一文件调度队列，从而可以将多 个小文件的调用请求消息进行合并和/或删除处理，可以避免小文件的调用请求而增加的I/O次数并导致的占用过多内存资源，可以减少系统调用次数，并增加I/O请求的吞吐量，进而整体提升文件系统的I/O效率。另外，还可以为小文件分配相应的动态存储区域，将不同的小文件聚合在一起存放，可以减少存储空间的浪费，提升空间的利用率。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质 中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应所述以权利要求的保护范围为准。

Claims (18)

  1. 一种存取文件的方法，其特征在于，包括：
    确定L个第一文件调用请求消息，所述L个第一文件调用请求消息中的每个第一文件调用请求消息用于请求调用第一文件，所述第一文件的大小小于或等于门限值，所述每个第一文件调用请求消息包括物理扇区信息，所述物理扇区信息包括存储所述第一文件的物理扇区的地址，L为大于1的整数；
    根据所述物理扇区信息，将所述L个第一文件调用请求消息分别添加到与所述L个第一文件调用请求消息对应的M个第一文件调度队列中，M为正整数；
    对所述M个第一文件调度队列中的每个第一文件调度队列包括的K个第一文件调用请求消息进行处理，获得所述每个第一文件调度队列的调用请求消息，所述每个第一文件调度队列的调用请求消息用于请求调用所述K个第一文件调用请求消息对应的第一文件，K为正整数；
    根据所述每个第一文件调度队列的调用请求消息，存取所述K个第一文件调用请求消息对应的第一文件。
  2. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    在存储文件的物理扇区上，确定N个第一文件存储空间组，所述N个第一文件存储空间组用于存储第一文件，N为正整数；
    当所述N个第一文件存储空间组中至少两个第一文件存储空间组存在可用存储空间时，将所述至少两个第一文件存储空间组包括的第一文件进行合并；
    当所述N个第一文件存储空间组中至少一个第一文件存储空间组不存储文件时，释放所述至少一个第一文件存储空间组；
    当所述N个第一文件的存储空间组都不存在所述可用存储空间时，增加所述N个第一文件的存储空间组的个数；
    所述根据所述每个第一文件调度队列的调用请求消息，存取所述K个第一文件调用请求消息对应的第一文件，包括：
    根据所述每个第一文件调度队列的调用请求消息，从所述N个第一文件存储空间组中存取所述K个第一文件调用请求消息对应的第一文件。
  3. 根据权利要求1或2所述的方法，其特征在于，所述根据所述物理扇区信息，将所述L个第一文件调用请求消息分别添加到与所述L个第一文 件调用请求消息对应的M个第一文件调度队列中，包括：
    根据所述物理扇区信息，将所述L个第一文件调用请求消息中的至少两个第一文件调用请求消息添加到同一个第一文件调度队列中，所述至少两个第一文件调用请求消息请求调用的第一文件的物理扇区的地址连续。
  4. 根据权利要求1至3中任一项所述的方法，其特征在于，所述每个第一文件调用请求消息还包括存取路径信息，所述存取路径信息用于指示存取所述每个第一文件调用消息对应的第一文件的物理扇区的路径。
  5. 根据权利要求4所述的方法，其特征在于，所述确定L个第一文件调用请求消息，包括：
    确定L个第一文件信息节点，所述L个第一文件信息节点中的每个第一文件信息节点用于存储所述每个第一文件调用请求消息请求调用的第一文件的所述存取路径信息；
    根据所述每个第一文件信息节点，确定所述每个第一文件调用请求消息请求调用的第一文件的所述存取路径信息。
  6. 根据权利要求5所述的方法，其特征在于，所述确定L个第一文件信息节点，包括：
    在信息节点链表中确定是否存在所述L个第一文件信息节点中第i个第一文件信息节点；
    当所述信息节点链表中不存在所述第i个第一文件信息节点时，在所述信息节点链表中创建所述第i个第一文件信息节点，并将第i个第一文件的所述存取路径信息存储到所述第i个第一文件信息节点中；
    当所述信息节点链表中存在所述第i个第一文件信息节点时，根据所述第i个第一文件信息节点确定所述第i个第一文件的所述存取路径信息。
  7. 一种存取文件的装置，其特征在于，包括：
    第一确定模块，用于确定L个第一文件调用请求消息，所述L个第一文件调用请求消息中的每个第一文件调用请求消息用于请求调用第一文件，所述第一文件的大小小于或等于门限值，所述每个第一文件调用请求消息包括物理扇区信息，所述物理扇区信息包括存储所述第一文件的物理扇区的地址，L为大于1的整数；
    分配模块，用于根据所述物理扇区信息，将所述第一确定模块确定的所述L个第一文件调用请求消息分别添加到与所述L个第一文件调用请求消息 对应的M个第一文件调度队列中，M为正整数；
    处理模块，用于对所述分配模块分配的所述M个第一文件调度队列中的每个第一文件调度队列包括的K个第一文件调用请求消息进行处理，获得所述每个第一文件调度队列的调用请求消息，所述每个第一文件调度队列的调用请求消息用于请求调用所述K个第一文件调用请求消息对应的第一文件，K为正整数；
    调用模块，用于根据所述处理模块获得的所述每个第一文件调度队列的调用请求消息，存取所述K个第一文件调用请求消息对应的第一文件。
  8. 根据权利要求7所述的装置，其特征在于，所述装置还包括：
    第二确定模块，用于在存储文件的物理扇区上，确定N个第一文件存储空间组，所述N个第一文件存储空间组用于存储所述第一文件，
    当所述N个第一文件存储空间组中至少两个第一文件存储空间组存在可用存储空间时，将所述至少两个第一文件存储空间组包括的第一文件进行合并，或
    当所述N个第一文件存储空间组中至少一个第一文件存储空间组不存储文件时，释放所述至少一个第一文件存储空间组，或
    当所述N个第一文件的存储空间组都不存在所述可用存储空间时，增加所述N个第一文件的存储空间组的个数；
    所述调用模块还用于：
    根据所述每个第一文件调度队列的调用请求消息，从所述N个第一文件存储空间组中存取所述K个第一文件调用请求消息对应的第一文件。
  9. 根据权利要求7或8所述的装置，其特征在于，所述处理模块具体用于：
    根据所述物理扇区信息，将所述L个第一文件调用请求消息中的至少两个第一文件调用请求消息添加到同一个第一文件调度队列中，所述至少两个第一文件调用请求消息请求调用的第一文件的物理扇区的地址连续。
  10. 根据权利要求7至9中任一项所述的装置，其特征在于，所述每个第一文件调用请求消息还包括存取路径信息，所述存取路径信息用于指示存取所述每个第一文件调用消息对应的第一文件的物理扇区的路径。
  11. 根据权利要求10所述的装置，其特征在于，所述确定模块具体用于：
    确定L个第一文件信息节点，所述L个第一文件信息节点中的每个第一文件信息节点用于存储所述每个第一文件调用请求消息请求调用的第一文件的所述存取路径信息；
    根据所述每个第一文件信息节点，确定所述每个第一文件调用请求消息请求调用的第一文件的所述存取路径信息。
  12. 根据权利要求11所述的装置，其特征在于，所述确定模块具体用于：
    在信息节点链表中确定是否存在所述L个第一文件信息节点中第i个第一文件信息节点；
    当所述信息节点链表中不存在所述第i个第一文件信息节点时，在所述信息节点链表中创建所述第i个第一文件信息节点，并将第i个第一文件的所述存取路径信息存储到所述第i个第一文件信息节点中；
    当所述信息节点链表中存在所述第i个第一文件信息节点时，根据所述第i个第一文件信息节点确定所述第i个第一文件的所述存取路径信息。
  13. 一种存储系统，其特征在于，包括：
    存储器，用于存储文件和程序；
    处理器，用于执行所述程序，以实现：
    确定L个第一文件调用请求消息，所述L个第一文件调用请求消息中的每个第一文件调用请求消息用于请求调用第一文件，所述第一文件的大小小于或等于门限值，所述每个第一文件调用请求消息包括物理扇区信息，所述物理扇区信息包括存储所述第一文件的物理扇区的地址，L为大于1的整数；
    根据所述物理扇区信息，将所述L个第一文件调用请求消息分别添加到与所述L个第一文件调用请求消息对应的M个第一文件调度队列中，M为正整数；
    对所述M个第一文件调度队列中的每个第一文件调度队列包括的K个第一文件调用请求消息进行处理，获得所述每个第一文件调度队列的调用请求消息，所述每个第一文件调度队列的调用请求消息用于请求调用所述K个第一文件调用请求消息对应的第一文件，K为正整数；
    根据所述每个第一文件调度队列的调用请求消息，存取所述K个第一文件调用请求消息对应的第一文件。
  14. 根据权利要求13所述的存储系统，其特征在于，所述处理器还用 于：
    在存储文件的物理扇区上，确定N个第一文件存储空间组，所述N个第一文件存储空间组用于存储所述第一文件，
    当所述N个第一文件存储空间组中至少两个第一文件存储空间组存在可用存储空间时，将所述至少两个第一文件存储空间组包括的第一文件进行合并，或
    当所述N个第一文件存储空间组中至少一个第一文件存储空间组不存储文件时，释放所述至少一个第一文件存储空间组，或
    当所述N个第一文件的存储空间组都不存在所述可用存储空间时，增加所述N个第一文件的存储空间组的个数；
    根据所述每个第一文件调度队列的调用请求消息，从所述N个第一文件存储空间组中存取所述K个第一文件调用请求消息对应的第一文件
  15. 根据权利要求13或14所述的存储系统，其特征在于，所述处理器具体用于：
    根据所述物理扇区信息，将所述L个第一文件调用请求消息中的至少两个第一文件调用请求消息添加到同一个第一文件调度队列中，所述至少两个第一文件调用请求消息请求调用的第一文件的物理扇区的地址连续。
  16. 根据权利要求13至15中任一项所述的存储系统，其特征在于，所述每个第一文件调用请求消息还包括存取路径信息，所述存取路径信息用于指示存取所述每个第一文件调用消息对应的第一文件的物理扇区的路径。
  17. 根据权利要求16所述的存储系统，其特征在于，所述处理器具体用于：
    确定L个第一文件信息节点，所述L个第一文件信息节点中的每个第一文件信息节点用于存储所述每个第一文件调用请求消息请求调用的第一文件的所述存取路径信息；
    根据所述每个第一文件信息节点，确定所述每个第一文件调用请求消息请求调用的第一文件的所述存取路径信息。
  18. 根据权利要求17所述的存储系统，其特征在于，所述处理器具体用于：
    在信息节点链表中确定是否存在所述L个第一文件信息节点中第i个第一文件信息节点；
    当所述信息节点链表中不存在所述第i个第一文件信息节点时，在所述信息节点链表中创建所述第i个第一文件信息节点，并将第i个第一文件的所述存取路径信息存储到所述第i个第一文件信息节点中；
    当所述信息节点链表中存在所述第i个第一文件信息节点时，根据所述第i个第一文件信息节点确定所述第i个第一文件的所述存取路径信息。
PCT/CN2015/079894 2015-05-27 2015-05-27 存取文件的方法、装置和存储系统 WO2016187831A1 (zh)

Priority Applications (6)

Application Number Priority Date Filing Date Title
CN201580014438.8A CN106537321B (zh) 2015-05-27 2015-05-27 存取文件的方法、装置和存储系统
EP15892912.5A EP3293625B1 (en) 2015-05-27 2015-05-27 Method and device for accessing file, and storage system
PCT/CN2015/079894 WO2016187831A1 (zh) 2015-05-27 2015-05-27 存取文件的方法、装置和存储系统
US15/576,637 US10846265B2 (en) 2015-05-27 2015-05-27 Method and apparatus for accessing file, and storage system
KR1020177037049A KR102063281B1 (ko) 2015-05-27 2015-05-27 파일에 접근하기 위한 방법과 장치, 및 저장 시스템
JP2017561340A JP6584529B2 (ja) 2015-05-27 2015-05-27 ファイルにアクセスするための方法および装置、ならびに記憶システム

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2015/079894 WO2016187831A1 (zh) 2015-05-27 2015-05-27 存取文件的方法、装置和存储系统

Publications (1)

Publication Number Publication Date
WO2016187831A1 true WO2016187831A1 (zh) 2016-12-01

Family

ID=57393766

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/079894 WO2016187831A1 (zh) 2015-05-27 2015-05-27 存取文件的方法、装置和存储系统

Country Status (6)

Country Link
US (1) US10846265B2 (zh)
EP (1) EP3293625B1 (zh)
JP (1) JP6584529B2 (zh)
KR (1) KR102063281B1 (zh)
CN (1) CN106537321B (zh)
WO (1) WO2016187831A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107509085A (zh) * 2017-08-24 2017-12-22 山东浪潮商用系统有限公司 一种流媒体文件的存储方法及装置
CN112148683A (zh) * 2019-06-26 2020-12-29 中国移动通信集团江西有限公司 文件采集方法和装置

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111176566B (zh) * 2019-12-25 2023-09-19 山东方寸微电子科技有限公司 一种支持queue命令的eMMC读写控制方法及存储介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104536959A (zh) * 2014-10-16 2015-04-22 南京邮电大学 一种Hadoop存取海量小文件的优化方法
CN104571952A (zh) * 2014-12-25 2015-04-29 华中科技大学 一种分开处理数据与元数据读写请求的方法
CN104571931A (zh) * 2013-10-10 2015-04-29 中国科学院声学研究所 基于系统资源的i/o请求合并调度系统与方法

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB8829919D0 (en) * 1988-12-22 1989-02-15 Int Computer Limited File system
JP3851723B2 (ja) 1998-04-30 2006-11-29 株式会社東芝 ディスク記憶装置及び同装置におけるセグメントキャッシュ制御方法
US6931501B1 (en) * 2001-10-26 2005-08-16 Adaptec, Inc. Method and apparatus for merging contiguous like commands
US7996903B2 (en) * 2006-07-07 2011-08-09 Webroot Software, Inc. Method and system for detecting and removing hidden pestware files
JP4331220B2 (ja) * 2007-03-07 2009-09-16 株式会社東芝 未使用物理領域自律管理機能を持つストレージ装置
JP5247192B2 (ja) 2008-03-14 2013-07-24 三菱電機株式会社 周期更新データ管理システム
KR101338282B1 (ko) 2009-04-24 2014-01-02 레벨 3 커뮤니케이션즈 엘엘씨 미디어 자원 저장 및 관리
US20110035557A1 (en) * 2009-08-05 2011-02-10 Nokia Corporation Fragmentation reduction using virtual sectors for static data
US8578126B1 (en) * 2009-10-29 2013-11-05 Netapp, Inc. Mapping of logical start addresses to physical start addresses in a system having misalignment between logical and physical data blocks
US8768980B2 (en) * 2009-11-02 2014-07-01 Stg Interactive S.A. Process for optimizing file storage systems
US9009388B2 (en) * 2010-11-30 2015-04-14 Red Hat, Inc. Performing discard commands on RAID storage devices
US20150143021A1 (en) * 2012-12-26 2015-05-21 Unisys Corporation Equalizing wear on storage devices through file system controls
US20140351485A1 (en) * 2013-05-23 2014-11-27 Spansion Llc Differential File System for Computer Memory
JP6095539B2 (ja) 2013-09-20 2017-03-15 株式会社日立製作所 Ｉ／ｏ処理制御装置及びｉ／ｏ処理制御方法
CN104866428B (zh) * 2014-02-21 2018-08-31 联想(北京)有限公司 数据存取方法和数据存取装置
CN103914522A (zh) 2014-03-20 2014-07-09 电子科技大学 一种应用于云存储重复数据删除的数据块合并方法
US10793713B2 (en) * 2015-11-17 2020-10-06 University Of Houston System Biorenewable blends of polylactide and acrylated epoxidized soybean oil compatibilized by a polylactide star polymer

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104571931A (zh) * 2013-10-10 2015-04-29 中国科学院声学研究所 基于系统资源的i/o请求合并调度系统与方法
CN104536959A (zh) * 2014-10-16 2015-04-22 南京邮电大学 一种Hadoop存取海量小文件的优化方法
CN104571952A (zh) * 2014-12-25 2015-04-29 华中科技大学 一种分开处理数据与元数据读写请求的方法

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107509085A (zh) * 2017-08-24 2017-12-22 山东浪潮商用系统有限公司 一种流媒体文件的存储方法及装置
CN107509085B (zh) * 2017-08-24 2019-12-24 山东浪潮商用系统有限公司 一种流媒体文件的存储方法及装置
CN112148683A (zh) * 2019-06-26 2020-12-29 中国移动通信集团江西有限公司 文件采集方法和装置

Also Published As

Publication number Publication date
CN106537321A (zh) 2017-03-22
EP3293625A4 (en) 2018-05-23
US10846265B2 (en) 2020-11-24
KR20180011255A (ko) 2018-01-31
EP3293625B1 (en) 2021-07-07
CN106537321B (zh) 2020-04-28
JP2018515859A (ja) 2018-06-14
JP6584529B2 (ja) 2019-10-02
EP3293625A1 (en) 2018-03-14
US20180129672A1 (en) 2018-05-10
KR102063281B1 (ko) 2020-01-07

Similar Documents

Publication Publication Date Title
US10248175B2 (en) Off-line affinity-aware parallel zeroing of memory in non-uniform memory access (NUMA) servers
CN111324427B (zh) 一种基于dsp的任务调度方法及装置
CN115658277B (zh) 一种任务调度方法、装置及电子设备和存储介质
CN108038062B (zh) 嵌入式系统的内存管理方法和装置
US20190286582A1 (en) Method for processing client requests in a cluster system, a method and an apparatus for processing i/o according to the client requests
US20080244118A1 (en) Method and apparatus for sharing buffers
WO2016187831A1 (zh) 存取文件的方法、装置和存储系统
US20150026447A1 (en) Apparatus for configuring operating system and method therefor
US10430233B1 (en) Scheduling computational tasks among multiple classes of storage resources based on job classification
EP4239462A1 (en) Systems and methods for heterogeneous storage systems
CN113076180B (zh) 上行数据通路的构建方法及数据处理系统
CN113076189B (zh) 具有多数据通路的数据处理系统及用多数据通路构建虚拟电子设备
WO2023116910A1 (zh) 一种计算资源和缓存资源调度方法、装置及系统
JP5847313B2 (ja) 情報処理装置
CN113535377A (zh) 数据通路、其资源管理方法及其信息处理设备
CN113535345A (zh) 下行数据通路的构建方法及设备
JP2013210812A (ja) ストレージ装置、ストレージシステム、データ格納方法およびデータ格納プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15892912

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 15576637

Country of ref document: US

ENP Entry into the national phase

Ref document number: 2017561340

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2015892912

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 20177037049

Country of ref document: KR

Kind code of ref document: A