CN117560408A - File system remote access method and device for DPU - Google Patents
File system remote access method and device for DPU Download PDFInfo
- Publication number
- CN117560408A CN117560408A CN202311466587.0A CN202311466587A CN117560408A CN 117560408 A CN117560408 A CN 117560408A CN 202311466587 A CN202311466587 A CN 202311466587A CN 117560408 A CN117560408 A CN 117560408A
- Authority
- CN
- China
- Prior art keywords
- host
- file system
- dpu
- file
- vsock
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000004891 communication Methods 0.000 claims abstract description 59
- 238000004458 analytical method Methods 0.000 claims abstract description 8
- 238000012217 deletion Methods 0.000 claims description 6
- 230000037430 deletion Effects 0.000 claims description 6
- 238000012986 modification Methods 0.000 claims description 6
- 230000004048 modification Effects 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 abstract description 9
- 238000012545 processing Methods 0.000 description 12
- 230000005540 biological transmission Effects 0.000 description 6
- 238000013461 design Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000012546 transfer Methods 0.000 description 4
- 230000001133 acceleration Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000017525 heat dissipation Effects 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 238000007792 addition Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000007630 basic procedure Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
- H04L67/125—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks involving control of end-device applications over a network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
- H04L67/141—Setup of application sessions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45595—Network integration; Enabling network access in virtual machine instances
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a file system remote access method and a device for a DPU, which are applied to the DPU, wherein the DPU is arranged on a main board of a physical server; the HOST system is also installed in the physical server; a client kernel module of a self-defined file system is installed in the DPU, and a network client component for remote access of the file system is installed in the DPU; the HOST is provided with a server kernel module and a network server component for remote access of a file system, and the method comprises the following steps: sending a connection establishment request to HOST, and establishing a vsock communication channel based on a PCI address with HOST; mounting a file system on HOST to obtain a mounting catalog; responding to a received target operation instruction acting on the mounting catalog, and sending the target operation instruction to HOST through a vsock communication channel, so that the HOST executes after receiving the analysis of the target operation instruction and returns an operation result corresponding to the target operation instruction; the problem of high network delay in the process of remotely accessing the file system is solved.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for remote access to a file system of a DPU.
Background
With the continuous expansion of the scale of internet services, the service types are continuously increased, the consumption of resources of cloud computing data centers for bearing various service applications is also increasingly greater, and the improvement of the utilization rate of the resources such as computing, storage and the like of the data centers becomes a problem to be solved. In the application scenario of the data processing unit (Data Processing Unit, DPU), after the control plane of various application services running on the data center server is unloaded onto the DPU, the resource consumption of the data center server on the control plane service can be reduced, so that the saved resources are allocated to the application services of the user for use, and the marketable computing resources can be improved. When the DPU performs control plane unloading, it is necessary to be able to access the file system on the virtual machine that is the HOST on the DPU, and further to execute commands of the control plane on the HOST and files in the file system related to the command parameters on the DPU, which access requires fast, low-latency data transmission and processing in order to respond to the operation of the control plane in real time.
The traditional file system remote access method comprises the following steps: access to the file system is allowed by the remote client over the network in a manner that is similar to the local file system through the network file system (NetworkFile System, NFS).
However, NFS protocol is essentially a network-based file sharing protocol, whose design goal is primarily to provide reliable file transfer, rather than low latency and high throughput data transfer. Therefore, NFS typically introduces a high latency in file read and write operations, which is a problem with high network latency in remotely accessing the file system.
Disclosure of Invention
In view of the foregoing, embodiments of the present invention provide a method and apparatus for remote access to a file system of a DPU, which obviate or mitigate one or more disadvantages in the prior art. The problem of high network delay in the process of remotely accessing the file system is solved.
One aspect of the present invention provides a file system remote access method for a DPU, which is applied to a DPU, wherein the DPU is installed on a motherboard of a physical server; the HOST system is also installed in the physical server; the DPU is provided with a kernel module of a self-defined file system and a network client component for remote access of the file system; the HOST runs a kernel module and a network server component for remote access of the file system; the method comprises the following steps:
sending a connection establishment request to HOST, and establishing a vsock communication channel based on a PCI address with HOST to access HOST through the vsock communication channel;
mounting a file system on HOST to obtain a mounting catalog;
and responding to the received target operation instruction acting on the mounting catalog, and sending the target operation instruction to HOST through a vsock communication channel, so that the HOST executes after receiving the analysis of the target operation instruction and returns an operation result corresponding to the target operation instruction.
Optionally, implementing logic for vsock-based network connection and implementing logic for file system mount are encapsulated in the file system
Optionally, a client network connection component is running in the DPU; a server network connection component is operated in HOST; HOST is in a connection request monitoring state after starting a network server component; sending a connection establishment request to HOST, and establishing a vsock communication channel based on a PCI address with HOST, comprising:
a connection setup request is sent to the HOST through the network client component to establish a vsock communication channel with the HOST.
Optionally, the DPU and HOST are configured with the same vsock identification; the method comprises the steps of sending a connection establishment request to HOST, and before establishing a vsock communication channel based on a PCI address with HOST, further comprising:
responding to the received identifier configuration instruction, and configuring the vsock identifier corresponding to the DPU into a vsock identifier indicated by the identifier configuration instruction; the vsock identifier indicated by the identifier configuration instruction is the same as the vsock identifier of HOST; the vsock identification of the HOST configuration is used to uniquely indicate HOST.
Optionally, the target operation instruction includes a file creation instruction, a file deletion instruction, a file reading instruction, a file directory creation instruction, a file directory deletion instruction, a file directory modification instruction, or a file writing instruction.
Optionally, in the case that the target operation instruction is a file reading instruction, the operation result further includes a file operator; after sending the target operation instruction to HOST through the vsock communication channel, the method further comprises:
accessing the target file indicated by the file reading instruction through the file operator; the file operator is an operator obtained after the HOST opens the target file through an operation interface provided by the virtual file system protocol.
Optionally, the network server component includes a file system whitelist configuration file, where the file system whitelist configuration file is used to configure the specified path and files allowed to be accessed under the specified path.
Optionally, the file system is a custom file system based on a Linux virtual file system.
Another aspect of the present invention provides a file system remote access apparatus for a DPU, comprising a processor and a memory, the memory having stored therein computer instructions, the processor being operable to execute the computer instructions stored in the memory, the apparatus implementing the steps of a file system remote access method for a DPU as described above when the computer instructions are executed by the processor.
Another aspect of the invention provides a computer readable storage medium having stored thereon a computer program, characterized in that the program when executed by a processor implements the steps of the above-described remote access method based on a file system for a DPU.
According to the file system remote access method for the DPU, a connection establishment request is sent to HOST, and a vsock communication channel based on a PCI address is established with the HOST, so that the HOST is accessed through the vsock communication channel; mounting a file system on HOST to obtain a mounting catalog; responding to a received target operation instruction acting on the mounting catalog, and sending the target operation instruction to HOST through a vsock communication channel, so that the HOST executes after receiving the analysis of the operation instruction and returns an operation result corresponding to the target operation instruction; the problem of higher network delay in the process of remotely accessing the file system is solved; by installing a client kernel module and a network client component of a self-defined file system in a DPU, installing a server kernel module and a network server component in HOST, establishing a vsock communication channel between the DPU and the HOST, transmitting a target operation instruction sent by the DPU to the HOST server kernel module for execution, realizing access to the file system on the HSOT side at the DPU side without using a network file system, thereby reducing network delay in the process of remotely accessing the file system and improving the remote access performance of the file system.
Further, through the custom virtual file system, the design and implementation of the file system can be simplified according to actual needs, the complexity of the file system is reduced, meanwhile, the installation difficulty and the use difficulty are reduced, a great amount of relevant knowledge is not required to be learned by a user, the learning cost of the user is reduced, and the remote access efficiency of the file system is improved.
Furthermore, the user-defined virtual file system can be more easily adapted to the requirements of a specific operating system or environment, the coordination and compatibility requirements between the DPU and the HOST can be reduced, the problem of compatibility possibly existing in the network file system when the network file system is crossed or different versions of the network file system are used is solved, and the stability of remote access of the file system is improved.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and drawings.
It will be appreciated by those skilled in the art that the objects and advantages that can be achieved with the present invention are not limited to the above-described specific ones, and that the above and other objects that can be achieved with the present invention will be more clearly understood from the following detailed description.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate and together with the description serve to explain the invention.
FIG. 1 is a flow chart of a method for remote access of a file system for a DPU according to one embodiment of the invention;
FIG. 2 is a schematic diagram of a physical server according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a user mode and a kernel mode according to an embodiment of the present invention;
fig. 4 is a block diagram of a file system remote access apparatus for a DPU according to one embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the following embodiments and the accompanying drawings, in order to make the objects, technical solutions and advantages of the present invention more apparent. The exemplary embodiments of the present invention and the descriptions thereof are used herein to explain the present invention, but are not intended to limit the invention.
It should be noted here that, in order to avoid obscuring the present invention due to unnecessary details, only structures and/or processing steps closely related to the solution according to the present invention are shown in the drawings, while other details not greatly related to the present invention are omitted.
It should be emphasized that the term "comprises/comprising" when used herein is taken to specify the presence of stated features, elements, steps or components, but does not preclude the presence or addition of one or more other features, elements, steps or components.
It is also noted herein that the term "coupled" may refer to not only a direct connection, but also an indirect connection in which an intermediate is present, unless otherwise specified.
Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.
First, several terms related to the embodiments of the present application will be described.
Data processing unit (Data Processing Unit, DPU): a DPU is a specially designed hardware unit for data processing, the main function of which is to achieve efficient, fast and reliable data processing and analysis. DPU plays an important role in various application scenes, and can provide excellent performance and efficiency from a data center to Internet of things equipment and from edge calculation to deep learning reasoning. The DPU is taken as a special data processing unit, is one of key technologies indispensable in the modern digital age, and has important significance for promoting technical innovation and application development in various fields.
Virtual Machine (VM): is a virtual operating system running on a computer. It is typically created by virtualization software that abstracts computer hardware so that multiple virtual machine instances can be run on one physical computer.
Virtual Socket communication protocol (vsock): is a communication protocol used in a Linux virtual machine environment. It provides a high performance, low latency way of communicating, allowing communication between a host and a client. Virtual socket communication protocol is a secure way of communication because it uses unique address formats and socket families, which can only be used in virtualized environments. Such protocols may be used to transfer files between a host and a client, make remote procedure calls, and implement network stack acceleration in a virtual environment. The virtual socket communications protocol is implemented by a kernel module loaded into the host and client operating system kernels. When using a virtual socket communication protocol, a specific vsock address is specified when the host and client communicate with each other. Once the connection is established, the communication can be performed as on a local computer without processing through a network protocol stack, thereby realizing efficient communication.
PCIe (Peripheral Component Interconnect Express) is a computer bus standard for high-speed data transfer between a connection expansion card, external devices, and a motherboard in a computer system.
PCIe is based on a point-to-point connection architecture, where each device is directly connected to one or more PCIe slots on the motherboard. It provides high bandwidth and low latency data transmission by using multiple data channels (called lanes), each capable of supporting a certain data throughput. Common PCIe versions include PCIe 1.0, PCIe 2.0, PCIe 3.0, PCIe 4.0, and PCIe 5.0, each of which increases data transmission speed and functional characteristics.
PCIe is widely used for connection of high-performance expansion cards such as graphics cards, memory controllers, network adapters, and the like. PCIe has higher bandwidth and faster transmission speed, supports hot plug and hot swap, and provides better scalability than conventional PCI buses. In addition, PCIe may also perform device management and communication through configuration space and messaging mechanisms.
Network file system (Network File System, NFS): the shared file system and directory can be accessed between different hosts through a network. The main working principle of NFS is:
1. the server side exports the specified file directory and exposes it to the network in the form of a shared file system.
2. The client can mount the server-side derived shared directory and map it to a local mount point.
3. When the client performs read-write operation on the mounted directory, a request is sent to the server through the NFS protocol.
4. After receiving the request, the server performs actual file operation on the shared directory, and then returns the result to the client.
5. And after the client obtains the result returned by the server, the client completes the access to the file system.
Basic procedure using NFS:
1. NFS services are installed and configured at the server side, specifying the file directories to be exported.
2. NFS client software is installed at the client and the mount command is used to mount the shared directory at the server to the local directory.
3. By accessing the local mount directory, the client can access the shared file system of the server.
4. If access needs to be canceled, the umount command can be used to cancel the mounting of the catalog.
NFS has evolved and evolved into the most powerful and widely used network file system of the UNIX system. NFS allows sharing a common file system among multiple users and provides advantages in data sets to minimize the storage space required. The NFS protocol implementation is primarily implemented using RPC (remote procedure call) mechanisms, providing a set of operations to access remote files independent of machine, operating system, and low-level transport protocols. The RPC employs the support of XDR, which is a machine-independent protocol for data description encoding, to encode and decode data transmitted over the network in a format independent of any machine architecture, supporting the transmission of data between heterogeneous systems.
The virtual file system (Virtual File System, VFS) is an abstraction layer in the operating system kernel that manages access and operation of different types of file systems. It provides a unified interface so that applications can read, write, create, delete, etc. files without concern for implementation details of a particular file system.
The file system remote access method for the DPU provided in the present application is described in detail below.
As shown in fig. 1, an embodiment of the present application provides a remote file system access method for a DPU, where the embodiment is described by taking the method as an example in the DPU shown in fig. 2, where the DPU is installed on a motherboard of a physical server, and an HOST system is also installed in the physical server; the method at least comprises the steps of S101 to S103:
step S101, a connection establishment request is sent to the HOST, and a vsock communication channel based on the PCI address is established with the HOST to access the HOST through the vsock communication channel.
In this embodiment, a kernel module of a user-defined file system is running in the DPU, and a network client component for remote access of the file system is running in the DPU; accordingly, HOST runs with a kernel module and a network server component for remote access to the file system. Virtual sockets (vsocks) are communication mechanisms that can be used in virtualized environments that can communicate using different physical channels. In this embodiment, PCIe is used as the communication channel.
The file system is a self-defined file system based on a Linux virtual file system. The file system is encapsulated with vsock-based network connection realization logic and file system mount realization logic.
In this embodiment, the implementing logic of the vsock-based network connection refers to the implementing logic for establishing a vsock communication channel between the DPU and the HOST. Based on this, the kernel module of the file system installed in the DPU and the kernel module installed in the HOST both support vsock and PCIe communications; meanwhile, PCIe drivers are installed in the DPU and the HOST to ensure that a vsock communication channel between the DPU and the HOST can be established. In applications on the DPU and HOST, the vsock and the API of the virtual file system are used for business operations.
HOST enters a connection request listening state after starting the network server component to listen for connection setup requests from the DPU. After the network client component is started, the DPU sends a connection setup request to HOST to establish a PCI address-based vsock communication channel with HOST.
Specifically, sending a connection establishment request to the HOST, and establishing a vsock communication channel based on a PCI address with the HOST, including: a connection establishment request is sent to the HOST through the network client component to establish a vsock communication channel with the HOST.
In this embodiment, at least one HOST is running on the physical server, and at least one DPU is installed on the motherboard of the physical server. The communication between the DPU and HOST requires the use of a PCI address for indicating the PCIE card slot in which the network card is installed; and configuring the PCI address corresponding to the vsock module in the physical server through the configuration file in HOST so that the vsock module can use the PCI address to determine the network card for establishing the vsock communication connection through the PCI address.
At the same time, communication between the DPU and HOST will also use vsock identification. In the same physical server, the vsock identifiers corresponding to different HOSTs are different, and the vsock identifiers of HOST configuration are used for uniquely indicating HOSTs. In order to enable the DPU to establish a vsock communication channel with a designated HOST, the DPU is configured with the same vsock identification as the HOST for which the vsock communication channel is to be established.
Specifically, the method sends a connection establishment request to HOST, and before establishing a vsock communication channel based on PCI address with HOST, the method further comprises: responding to the received identifier configuration instruction, and configuring the vsock identifier corresponding to the DPU into a vsock identifier indicated by the identifier configuration instruction; the vsock identifier indicated by the identifier configuration instruction is the same as the vsock identifier of HOST; the vsock identification of the HOST configuration is used to uniquely indicate HOST.
After completing configuration of the vsock identifier and the PCI address, a PCI address-based vsock communication channel between the DPU and the HOST is established through which the DPU and the HOST communicate.
Step S102, mounting the file system on HOST to obtain a mounting catalog.
In this embodiment, the implementation logic of the file system mount on HOST is encapsulated in the file system. After the vsock communication channel between the DPU and the T is successfully established, the DPU is allowed to access the file system of the HOST through configuration in the file system white list configuration file of the HOST, and after the configuration of the white list is completed, the mounting sharing of the file system of the HOST can be carried out on the DPU side, so that a mounting catalog is obtained.
Specifically, in this embodiment, the network server component includes a file system whitelist configuration file, where the file system whitelist configuration file is used to configure the mount directory and files allowed to be accessed under the mount directory. The directory that can be shared with the DPU and the files under the directory are configured through the file system whitelist configuration file to enable the DPU to access and operate on the HOST side file system using the operating instructions and vsock communication channels encapsulated in the network client component.
Step S103, in response to the received target operation instruction acting on the mount directory, the target operation instruction is sent to HOST through the vsock channel, so that the HOST executes after receiving the analysis of the target operation instruction and returns an operation result corresponding to the target operation instruction.
The target operation instruction is packaged in the client kernel module and the server kernel module. The target operation instruction includes a file creation instruction, a file deletion instruction, a file reading instruction, a file directory creation instruction, a file directory deletion instruction, a file directory modification instruction, or a file writing instruction. In actual implementation, the target operation instruction further includes a file copy instruction, a file move instruction, and the like, and the implementation of the target operation instruction is not limited in this embodiment.
After receiving a file operation instruction sent by a user, the DPU uses a target operation instruction corresponding to the file operation instruction to process a file, and the DPU comprises the following steps: and sending the target operation instruction to the HOST through the vsock communication channel, so that the HOST executes the operation instruction through the kernel module after receiving the target operation instruction, and the operation on the file system in the HOST is realized on the DPU.
After HOST analyzes and executes the operation instruction, the operation result corresponding to the target operation instruction is returned to the DPU through the vsock communication channel, so that a user can conveniently check or carry out subsequent file operation. For example, in the case where the target operation instruction is a file read instruction, the HOST returns a file operator corresponding to the target file to the DPU, so that the user can access the target file through the file operator later, and after the access is completed, the file program operator is ended and closed.
Specifically, in the case where the target operation instruction is a file reading instruction, the operation result further includes a file operator; after sending the target operation instruction to HOST through the vsock communication channel, the method further comprises: accessing the target file indicated by the file reading instruction through the file operator; the file operator is an operator obtained after the HOST opens the target file through an operation interface provided by the virtual file system protocol.
Fig. 3 is a schematic diagram of a user mode and a kernel mode of the DPU and the HOST in the present embodiment.
As shown in fig. 3, in this embodiment, the DPU side is installed with a network client component yudurfs-client of a custom file system, and the network client component includes a client kernel module yudurfs.ko and a client network connection component yudurfs-proxy-client. Wherein yudurfs is a self-defined file system, and the module code is compiled into yudurfs.ko file and then is loaded to the client kernel module by using insmod yudurfs.ko.
And a network server component yudurfs-server of the file system is installed on the HOST side, and comprises a server kernel module yudurfs-server. Ko and a server network connection component yudurfs-proxy-server.
Configuration on HOST side the path through the file system whitelist profile, i.e. filesystem whitelist profile, is configured that needs to be mapped to the file system on the DPU. On the DPU side, the file system is shared by mounting.
Such as: in the HOST side/opt/yudurs/yudurfs/whistelist file, the configuration allows the DPU side to access the HOST side file system/opt/yudurs/yudurfs/whistelist, the configuration is completed by the following instructions:
Path=/var/lib/libvirt/qemu;/home/vms
after the HOST side is configured, the DPU side carries out mounting sharing of the file system, and the configuration is completed through the following instructions:
mount-t yusurfs/home/vms/home/vms
and then, starting a client network connection component yudurfs-proxy-client at the DPU side and starting a server network connection component yudurfs-proxy-server at the HOST side respectively, and establishing a vsock communication channel between the DPU and the HOST.
Before starting the client network connection component and starting the server network connection component, the vsock identifier of the DPU and the HOST needs to be configured.
Such as: taking vsock designation as 2 for example, in HOST, the setting is made by the following commands:
Yusurfs-server vsock_cid=2vsock_port=12345log_level=WARN
wherein vsock_cid is a vsock identifier, and vsock_port is a vsock port number.
In the DPU, the setting is made by the following commands:
Yusurfs-client vsock_cid=2vsock_port=12345log_level=WARN
wherein vsock_cid is a vsock identifier, and vsock_port is a vsock port number.
And then, after the DPU side uses the Linux mount command to mount, the content of the file system on HOST can be accessed on the DPU.
Such as: taking the path of the file system on HOST as/home/vms as an example, after the DPU uses Linux mount command mount-t qtfs/home/vms/DPU/vms to mount, the content on HOST/home/vms can be accessed in the/home/DPU/vms directory on the DPU.
According to the file system remote access method for the DPU, a connection establishment request is sent to HOST, and a vsock communication channel based on a PCI address is established with the HOST, so that the HOST is accessed through the vsock communication channel; mounting a file system on HOST to obtain a mounting catalog; responding to a received target operation instruction acting on the mounting catalog, and sending the target operation instruction to HOST through a vsock communication channel, so that the HOST executes after receiving the analysis of the operation instruction and returns an operation result corresponding to the target operation instruction; the problem of higher network delay in the process of remotely accessing the file system is solved; by installing a client kernel module and a network client component of a self-defined file system in a DPU, installing a server kernel module and a network server component in HOST, establishing a vsock communication channel between the DPU and the HOST, transmitting a target operation instruction sent by the DPU to the HOST server kernel module for execution, realizing access to the file system on the HSOT side at the DPU side without using a network file system, thereby reducing network delay in the process of remotely accessing the file system and improving the remote access performance of the file system.
Further, through the custom virtual file system, the design and implementation of the file system can be simplified according to actual needs, the complexity of the file system is reduced, meanwhile, the installation difficulty and the use difficulty are reduced, a great amount of relevant knowledge is not required to be learned by a user, the learning cost of the user is reduced, and the remote access efficiency of the file system is improved.
Furthermore, the user-defined virtual file system can be more easily adapted to the requirements of a specific operating system or environment, the coordination and compatibility requirements between the DPU and the HOST can be reduced, the problem of compatibility possibly existing in the network file system when the network file system is crossed or different versions of the network file system are used is solved, and the stability of remote access of the file system is improved.
The present embodiment provides a file system remote access apparatus for a DPU, as shown in fig. 4. The present embodiment is applied to the DPU shown in fig. 2 by the apparatus, and the apparatus includes at least the following modules: a request sending module 410, a file mounting module 420, and an instruction sending module 430.
A request sending module 410, configured to send a connection establishment request to the HOST, and establish a vsock communication channel with the HOST based on the PCI address, so as to access the HOST through the vsock communication channel.
File mount module 420 is configured to mount the file system on HOST to obtain a mount directory.
The instruction sending module 430 is configured to send, in response to a received target operation instruction acting on the mount directory, the target operation instruction to the HOST through the vsock channel, so that the HOST executes and returns an operation result corresponding to the target operation instruction after receiving the analysis of the target operation instruction.
For relevant details reference is made to the above-described method and system embodiments.
It should be noted that: in the file system remote access device for a DPU provided in the foregoing embodiment, when performing file system remote access, only the division of the foregoing functional modules is used as an example, in practical application, the foregoing functional allocation may be performed by different functional modules, that is, the internal structure of the file system remote access device for a DPU is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the file system remote access device for the DPU provided in the foregoing embodiment and the file system remote access method embodiment for the DPU belong to the same concept, and detailed implementation processes of the device are referred to as method embodiments, which are not repeated herein.
Correspondingly, the invention also provides a file system remote access device for a DPU, which comprises a computer device, wherein the computer device comprises a processor and a memory, the memory is stored with computer instructions, the processor is used for executing the computer instructions stored in the memory, and the device realizes the steps of the file system remote access method for the DPU when the computer instructions are executed by the processor.
Correspondingly, the invention also provides electronic equipment comprising: the system comprises a computing component 1, an accelerating component 2, a memory expanding component 3 and a heat radiating component 4;
(1) The computing assembly 1 includes: a central processing unit CPU, a double rate synchronous dynamic random access memory DDR and a power supply;
(2) The acceleration assembly 2 comprises: the carrier board and the accelerator card interconnect switch; wherein, the acceleration card includes: one or more of an image processor GPU, a processor decentralized processing unit DPU and a neural network processor NPU;
(3) The memory expansion unit 3 includes: the carrier plate further comprises one or more of a memory expansion chip, a dual in-line memory module DIMM and a memory level storage medium SCM medium;
(4) The heat dissipation assembly 4 includes: one or more of air-cooled heat dissipation and liquid-cooled heat dissipation.
The embodiments of the present invention also provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the aforementioned file system remote access method for a DPU. The computer readable storage medium may be a tangible storage medium such as Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disk, a removable memory disk, a CD-ROM, or any other form of storage medium known in the art.
Those of ordinary skill in the art will appreciate that the various illustrative components, systems, and methods described in connection with the embodiments disclosed herein can be implemented as hardware, software, or a combination of both. The particular implementation is hardware or software dependent on the specific application of the solution and the design constraints. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, a plug-in, a function card, or the like. When implemented in software, the elements of the invention are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine readable medium or transmitted over transmission media or communication links by a data signal carried in a carrier wave.
It should be understood that the invention is not limited to the particular arrangements and instrumentality described above and shown in the drawings. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present invention are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after appreciating the spirit of the present invention.
In this disclosure, features that are described and/or illustrated with respect to one embodiment may be used in the same way or in a similar way in one or more other embodiments and/or in combination with or instead of the features of the other embodiments.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, but various modifications and variations can be made to the embodiment of the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. The file system remote access method for the DPU is characterized by being applied to the DPU, wherein the DPU is installed on a main board of a physical server; a HOST system is also installed in the physical server; a client kernel module of a self-defined file system is installed in the DPU, and a network client component for remote access of the file system is installed in the DPU; the HOST is provided with a server-side kernel module of the file system and a network server-side component for remote access of the file system; the method comprises the following steps:
sending a connection establishment request to the HOST, and establishing a vsock communication channel based on a PCI address with the HOST to access the HOST through the vsock communication channel;
mounting the file system on the HOST to obtain a mounting catalog;
responding to a received target operation instruction acting on the mount catalog, and sending the target operation instruction to the HOST through the vsock channel so that the HOST executes after receiving the analysis of the target operation instruction and returns an operation result corresponding to the target operation instruction; and the target operation instruction is packaged in the client kernel module and the server kernel module.
2. The method for remote access to a file system of a DPU of claim 1, wherein said file system has encapsulated therein vsock based network connection implementation logic and said file system mount implementation logic.
3. The file system remote access method for a DPU of claim 1, wherein a client network connection component is running in the DPU; a server network connection component is operated in the HOST; the HOST is in a connection request monitoring state after the server-side network connection component is started; the sending a connection establishment request to the HOST, and establishing a vsock communication channel based on a PCI address with the HOST, including:
and sending a connection establishment request to the HOST through the client network connection component to establish the vsock communication channel with the HOST.
4. A file system remote access method for a DPU as claimed in claim 3, wherein said DPU and said HOST are configured with the same vsock identification; the method further comprises the steps of, before sending a connection establishment request to the HOST and establishing a vsock communication channel based on a PCI address with the HOST:
responding to a received identifier configuration instruction, and configuring a vsock word identifier corresponding to the DPU as a vsock identifier indicated by the identifier configuration instruction; the vsock identifier indicated by the identifier configuration instruction is the same as the vsock identifier of the HOST; the vsock identification of the HOST configuration is used to uniquely indicate the HOST.
5. The file system remote access method for a DPU of claim 1, wherein the target operation instructions comprise file creation instructions, file deletion instructions, file reading instructions, file directory creation instructions, file directory deletion instructions, file directory modification instructions, or file writing instructions.
6. The file system remote access method for a DPU according to claim 5, wherein in the case where the target operation instruction is the file read instruction, the operation result further includes a file operator; after the target operation instruction is sent to the HOST through the vsock communication channel, the method further includes:
accessing a target file indicated by the file reading instruction through the file operator; the file operator is an operator obtained after the HOST opens the target file through an operation interface provided by a virtual file system protocol.
7. The method of claim 1, wherein the network server component includes a file system whitelist configuration file, the file system whitelist configuration file being used to configure a mount directory and files under the mount directory that are allowed to be accessed.
8. The method for remote access to a file system of a DPU of claim 1, wherein the file system is a custom file system based on a Linux virtual file system.
9. A file system remote access device for a DPU, comprising a processor and a memory, wherein the memory has stored therein computer instructions for executing the computer instructions stored in the memory, which device, when executed by the processor, implements the steps of the method according to any of claims 1 to 8.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the steps of the method according to any one of claims 1 to 8.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202311466587.0A CN117560408B (en) | 2023-11-06 | 2023-11-06 | File system remote access method and device for DPU |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202311466587.0A CN117560408B (en) | 2023-11-06 | 2023-11-06 | File system remote access method and device for DPU |
Publications (2)
Publication Number | Publication Date |
---|---|
CN117560408A true CN117560408A (en) | 2024-02-13 |
CN117560408B CN117560408B (en) | 2024-10-18 |
Family
ID=89822457
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202311466587.0A Active CN117560408B (en) | 2023-11-06 | 2023-11-06 | File system remote access method and device for DPU |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN117560408B (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN118245252A (en) * | 2024-05-20 | 2024-06-25 | 珠海星云智联科技有限公司 | Remote procedure call method, system, equipment and cluster |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20170091209A1 (en) * | 2009-04-16 | 2017-03-30 | International Business Machines Corporation | System that maintains objects created and modified within designated directories on a per-client basis in a network filesystem |
US20190258810A1 (en) * | 2018-02-20 | 2019-08-22 | Unisys Corporation | Method and system for low overhead control/status handshake for remote shared file server |
US20190332684A1 (en) * | 2018-04-25 | 2019-10-31 | International Business Machines Corporation | Optimized Network File System Client for Read-Only Exports/Mounts |
CN113032099A (en) * | 2021-03-26 | 2021-06-25 | 阿里巴巴新加坡控股有限公司 | Cloud computing node, file management method and device |
US20220334989A1 (en) * | 2021-04-19 | 2022-10-20 | Mellanox Technologies, Ltd. | Apparatus, method and computer program product for efficient software-defined network accelerated processing using storage devices which are local relative to a host |
CN115309545A (en) * | 2022-07-27 | 2022-11-08 | 阿里巴巴(中国)有限公司 | Access processing method and computer equipment |
US20230325220A1 (en) * | 2022-04-07 | 2023-10-12 | Vmware, Inc. | Hosting dpu management operating system using dpu software stack |
-
2023
- 2023-11-06 CN CN202311466587.0A patent/CN117560408B/en active Active
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20170091209A1 (en) * | 2009-04-16 | 2017-03-30 | International Business Machines Corporation | System that maintains objects created and modified within designated directories on a per-client basis in a network filesystem |
US20190258810A1 (en) * | 2018-02-20 | 2019-08-22 | Unisys Corporation | Method and system for low overhead control/status handshake for remote shared file server |
US20190332684A1 (en) * | 2018-04-25 | 2019-10-31 | International Business Machines Corporation | Optimized Network File System Client for Read-Only Exports/Mounts |
CN113032099A (en) * | 2021-03-26 | 2021-06-25 | 阿里巴巴新加坡控股有限公司 | Cloud computing node, file management method and device |
US20220334989A1 (en) * | 2021-04-19 | 2022-10-20 | Mellanox Technologies, Ltd. | Apparatus, method and computer program product for efficient software-defined network accelerated processing using storage devices which are local relative to a host |
US20230325220A1 (en) * | 2022-04-07 | 2023-10-12 | Vmware, Inc. | Hosting dpu management operating system using dpu software stack |
CN115309545A (en) * | 2022-07-27 | 2022-11-08 | 阿里巴巴(中国)有限公司 | Access processing method and computer equipment |
Non-Patent Citations (1)
Title |
---|
PETER-JAN GOOTZEN: "DPFS: DPU-Powered File System Virtualization", PROCEEDINGS OF THE 16TH ACM INTERNATIONAL CONFERENCE ON SYSTEMS AND STORAGE, 22 June 2023 (2023-06-22) * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN118245252A (en) * | 2024-05-20 | 2024-06-25 | 珠海星云智联科技有限公司 | Remote procedure call method, system, equipment and cluster |
CN118245252B (en) * | 2024-05-20 | 2024-08-23 | 珠海星云智联科技有限公司 | Remote procedure call method, system, equipment and cluster |
Also Published As
Publication number | Publication date |
---|---|
CN117560408B (en) | 2024-10-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2017066944A1 (en) | Method, apparatus and system for accessing storage device | |
US20240193106A1 (en) | Cross Address-Space Bridging | |
CN112540941B (en) | Data forwarding chip and server | |
CN117560408B (en) | File system remote access method and device for DPU | |
WO2024041412A1 (en) | Storage system and method, and hardware offload card | |
US10296369B2 (en) | Systems and methods for protocol termination in a host system driver in a virtualized software defined storage architecture | |
CN115599502B (en) | RTOS network sharing method based on virtual machine monitor | |
US20220309019A1 (en) | Storage Protocol Emulation in a Peripheral Device | |
US11741039B2 (en) | Peripheral component interconnect express device and method of operating the same | |
CN112463307A (en) | Data transmission method, device, equipment and readable storage medium | |
US10235195B2 (en) | Systems and methods for discovering private devices coupled to a hardware accelerator | |
CN115129625A (en) | Enhanced storage protocol emulation in a peripheral device | |
US20240241729A1 (en) | Server system starting method, apparatus, electronic device and storage medium | |
EP4407952A1 (en) | Network storage method, storage system, data processing unit, and computer system | |
WO2024217333A1 (en) | Io access method and apparatus based on block storage, and electronic device and medium | |
CN109656675B (en) | Bus equipment, computer equipment and method for realizing physical host cloud storage | |
CN117135189A (en) | Server access method and device, storage medium and electronic equipment | |
CN110659143A (en) | Communication method and device between containers and electronic equipment | |
CN111666579B (en) | Computer device, access control method thereof and computer readable medium | |
EP3913488B1 (en) | Data processing method and device | |
CN115549858A (en) | Data transmission method and device | |
CN116418848A (en) | Method and device for processing configuration and access requests of network nodes | |
CN113691465B (en) | Data transmission method, intelligent network card, computing device and storage medium | |
Zhang et al. | NVMe-over-RPMsg: A Virtual Storage Device Model Applied to Heterogeneous Multi-Core SoCs | |
CN101441661A (en) | System and method for sharing file resource between multiple embedded systems |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |