JP2021530813A - Integrated address space for multiple hardware accelerators with dedicated low latency links - Google Patents

Abstract

The system has a host processor (105) connected to the communication bus, a first hardware accelerator (135-1) that is communicably linked to the host processor (105) through the communication bus, and a host processor (135-1) through the communication bus. It may include a second hardware accelerator (135-2) communicably linked to 105). The first hardware accelerator (135-1) and the second hardware accelerator (135-2) are directly connected through an accelerator link independent of the communication bus. The host processor (105) is configured to initiate data transfer between the first hardware accelerator (135-1) and the second hardware accelerator (135-2) through the accelerator link directly. NS. [Selection diagram] Fig. 1

Description

The present disclosure relates to hardware acceleration, and more particularly to enabling the use of multiple hardware accelerators through an integrated address space and low latency communication links.

A heterogeneous computing platform (HCP) refers to a data processing system that includes a host processor connected to one or more other devices through an interface circuit. Devices are generally architecturally different from host processors. The host processor can offload tasks to the device. The device is capable of performing that task and producing results that are available to the host processor. As an exemplary example, the host processor is typically implemented as a central processing unit and the device is implemented as a graphics processing unit (GPU) and / or digital signal processor (DSP).

In other HCPs, one or more of the devices performing tasks offloaded from the host processor include devices adapted for hardware acceleration (called "hardware accelerators"). Hardware accelerators include circuits that are capable of performing tasks offloaded from the host, as opposed to executing software or program code to perform the tasks. The circuit of a hardware accelerator is functionally equivalent to running software, but in general it is possible to complete a task in less time.

Examples of hardware accelerators include programmable ICs such as field programmable gate arrays (FPGAs), partially programmable integrated circuits (ICs), and application specific integrated circuits (ASICs). Obviously, the HCP may include a combination of different devices, one or more adapted to execute program code and one or more others adapted for hardware acceleration.

In one or more embodiments, the system can communicate to the host processor through the communication bus with a host processor connected to the communication bus and a first hardware accelerator linked to the host processor so that it can communicate through the communication bus. It may include a second hardware accelerator linked to. The first hardware accelerator and the second hardware accelerator are directly connected through an accelerator link independent of the communication bus. The host processor is configured to initiate data transfer between the first hardware accelerator and the second hardware accelerator directly through the accelerator link.

In one or more embodiments, the hardware accelerator is an endpoint configured to communicate with the host processor over a communication bus, a memory controller connected to memory local to the hardware accelerator, and an endpoint. And a link circuit connected to the memory controller. The link circuit is configured to establish an accelerator link with a target hardware accelerator that is also connected to the communication bus. Accelerator links are direct connections between a hardware accelerator and a target hardware accelerator, independent of the communication bus.

In one or more embodiments, the method is to receive an instruction sent from a host processor over a communication bus and a target address for data transfer within the first hardware accelerator, and the first. In response to the hardware accelerator comparing the target address with the upper bound of the address range corresponding to the first hardware accelerator and determining that the target address exceeds the address range based on the comparison. A first hardware accelerator performs a transaction with a second hardware accelerator in order to perform a data transfer using an accelerator link that directly connects the first hardware accelerator and the second hardware accelerator. It can include starting.

The overview section of the invention is provided only to introduce some concepts, not to identify important or essential features of the claimed subject matter. Other features of the configuration of the present invention will become apparent from the accompanying drawings and the embodiments for carrying out the following inventions.

The configuration of the present invention is shown as an example in the accompanying drawings. However, the drawings should not be construed as limiting the configuration of the invention to the particular implementations shown. Various aspects and advantages will become apparent when examining embodiments for carrying out the following inventions and referring to the drawings.

It is a figure which shows an example of the system which has a plurality of hardware accelerators. It is a figure which shows an exemplary implementation form of a hardware accelerator. It is a figure which shows an example of a retransmission engine (RTE: retransmit engine). FIG. 5 illustrates an exemplary method of operation for a system with multiple hardware accelerators. FIG. 5 shows an example of a system with a plurality of hardware accelerators and one or more additional devices. FIG. 5 illustrates an exemplary architecture for an integrated circuit (IC).

The present disclosure concludes with claims that define new features, but the various features described within this disclosure are believed to be better understood by considering their description along with the drawings. The (s) processes, (s) machines, (s) products and any variants thereof described herein are provided for illustration purposes. The particular structural and functional details described within this disclosure should not be construed as limiting, but merely as a basis for the claims and in almost all well-detailed structures. It should be interpreted as a representative basis for teaching those skilled in the art to adopt various features described. Moreover, the terms and phrases used in this disclosure are not limiting, but rather provide an understandable description of the features being described.

The present disclosure relates to hardware acceleration, and more particularly to enabling the use of multiple hardware accelerators through an integrated address space and low latency communication links. Using a hardware accelerator with a data processing system has become an effective technique for offloading tasks from the host processor, thereby reducing the workload on the host processor. Hardware accelerators are typically mounted on the host processor through the bus. For example, a hardware accelerator can be mounted on a circuit board inserted into an available bus slot in a host system. Generally, each hardware accelerator is mounted on the corresponding circuit board. Adding additional hardware accelerators to the system usually involves inserting additional circuit boards with hardware accelerators into the available bus slots.

Within a traditional system, applications run by the host processor must be updated and / or rewritten, especially to access any newly added hardware accelerator (by hardware address, for example). It doesn't become. In addition, to transfer data from one hardware accelerator to another, the data is moved from the source hardware accelerator to the host processor and then from the host processor to the target hardware accelerator. Data travels via the bus through the host processor to and from each hardware accelerator. Therefore, each additional hardware accelerator added to the system increases the number of devices on the bus, thereby creating a competition for bandwidth on the bus. As the complexity, number, and / or size of tasks performed by hardware accelerators (or other devices) increases, the available bandwidth on the bus is further constrained.

The configurations of the invention described within the present disclosure provide an integrated address space for the device. Further provided herein are direct communication links between hardware accelerators that can operate independently of the bus, called "accelerator links". The runtime libraries and drivers run by the host have been integrated so that applications run by the host processor can run without directly referencing (for example, addressing) specific hardware accelerators in the system. It is possible to utilize the address space. The run-time library can determine the appropriate address to use to achieve data transfer between hardware accelerators. Therefore, the application does not need to be modified to access additional hardware accelerators that may be added to the system. In addition, data transfer can be performed over the accelerator link, which allows data to be transferred directly from one hardware accelerator to another without going through the host processor. Effectively bypass the bus. Therefore, the bandwidth used by the hardware accelerators on the bus can be significantly reduced, thereby increasing the overall system performance.

As mentioned, additional hardware accelerators are added to the system using the existing address space without the need for corresponding changes or modifications to the program code (for example, applications) executed by the host processor. Can be done. This is, at least in part, the implementation of an auto-discovery process for hardware accelerator boards, and the implementation of adding such boards to the system, the use of remote buffer flags vs. local buffer flags, at least in some cases. Supported through automatic switching to accelerator links for data transfer, as well as automatic address translation for remote buffers.

Further aspects of the configuration of the present invention will be described in more detail below with reference to the figures. For the sake of simplicity and clarity, the elements shown in the figure are not always drawn to a constant scale. For example, some dimensions of an element may be exaggerated relative to other elements for clarity. In addition, reference numbers are repeated between figures to indicate corresponding, similar, or similar features where deemed appropriate.

FIG. 1 shows an example of a system 100 having a plurality of hardware accelerators. System 100 is an example of computer hardware that can be used to implement a computer, server, or other data processing system. System 100 is also an example of a heterogeneous computing system. As depicted, system 100 includes at least one host processor 105 connected to host memory 110 through interface circuit 115.

System 100 also includes a plurality of hardware accelerators 135. In the example of FIG. 1, system 100 includes three hardware accelerators 135-1, 135-2, and 135-3. Although the example in FIG. 1 shows three hardware accelerators, it should be appreciated that system 100 may include less than three hardware accelerators or four or more hardware accelerators. In addition, system 100 may include one or more other devices, such as a graphics processing unit (GPU) or digital signal processor (DSP).

The system 100 can store computer-readable instructions (also referred to as "program code") in the host memory 110. The host memory 110 is an example of a computer-readable storage medium. The host processor 105 can execute the program code accessed from the host memory 110 via the interface circuit 115. In one or more embodiments, the host processor 105 communicates with the host memory 110 through a memory controller (not shown).

The host memory 110 may include one or more physical memory devices, such as local memory and bulk storage devices. Local memory generally refers to non-persistent memory devices (s) used during the actual execution of program code. Examples of local memory include random access memory (RAM) and / or any of various types of RAM suitable for use by a processor running program code, such as DRAM, SRAM, DDR SDRAM. .. Bulk storage device refers to a persistent data storage device. Examples of bulk storage devices are, but are not limited to, hard disk drives (HDD), solid state drives (SSD), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable. Includes read-only memory (EEPROM), or other suitable memory. System 100 also includes one or more cache memories (not shown) that temporarily store at least some program code in order to reduce the number of times the program code must be retrieved from the bulk storage device during execution. obtain.

The host memory 110 can store program code and / or data. For example, the host memory 110 may store the operating system 120, the instructions 125, and the data 130. In the example of FIG. 1, instruction 125 is associated with one or more applications 170, a runtime library 172 (referred to herein as the "runtime"), and a driver 174 capable of communicating with the hardware accelerator 135. May include. Runtime 172 can handle completion events, manage command queues, and provide notifications to application 170 (s). Data 130 may include buffer objects such as buffer objects 176 and 178 that allow direct data transfer between hardware accelerators 135, among other types of data items. The buffer object 176 includes the remote flag 180 and the buffer object 178 includes the remote flag 182. For illustrative purposes, the remote flag 180 is not set and the remote flag 182 is set. System 100, such as the host processor 105, is capable of executing operating system 120 and instructions 125 to perform the operations described herein.

Examples of the interface circuit 115 include, but are not limited to, a system bus and an input / output (I / O) bus. The interface circuit 115 can be implemented using any of a variety of bus architectures. Examples of bus architectures are, but are not limited to, Extended Industry Standard Architecture (EISA) Buses, Accelerated Graphics Ports (AGP), Video Electronics Standards Association (VESA) Local Buses, Universal Serial Buses (USB), and Peripheral Components. It may include an interconnected express (PCIe) bus. The host processor 105 may be connected to the host memory 110 through a different interface circuit than that used to couple to the hardware accelerator 135. For illustrative purposes, no endpoint is shown for the interface circuit 115 through which the host processor 105 communicates with other devices.

The system 100 may further include one or more other I / O devices (not shown) connected to the interface circuit 115. The I / O device can be connected to the system 100, eg, the interface circuit 115, either directly or through an intervening I / O controller. Examples of I / O devices include, but are not limited to, keyboards, display devices, pointing devices, one or more communication ports, and network adapters. A network adapter refers to a circuit that allows a system 100 to be connected to other systems, computer systems, remote printers, and / or remote storage devices through an intervening private or public network. Modems, cable modems, Ethernet cards, and wireless transceivers are examples of different types of network adapters that can be used with System 100.

In the example of FIG. 1, each of the hardware accelerators 135-1, 135-2, and 135-3 is connected to memories 140-1, 140-2, and 140-3, respectively. The memories 140-1, 140-2, and 140-3 are generally implemented as RAM as described for the host memory 110. In one or more embodiments, each hardware accelerator 135 is implemented as an IC. The IC can be a programmable IC. An example of a programmable IC is a field programmable gate array (FPGA).

In the example of FIG. 1, each of the hardware accelerators 135 includes an endpoint 145, a link circuit 150, a memory controller 155 (abbreviated as "MC" in FIG. 1), and an interconnect circuit 168. Each hardware accelerator 135 also includes one or more calculation units (abbreviated as "CU" in FIG. 1). The calculation unit is a circuit capable of performing tasks offloaded from the host processor 105. For purposes of illustration, each of the hardware accelerators 135 is shown to include a calculation unit 160 and a calculation unit 165. It should be appreciated that the hardware accelerator 135 may contain fewer or more computing units than shown.

In one example, each of the endpoints 145 is implemented as a PCIe endpoint. It should be appreciated that the endpoint 145 can be implemented as any type of endpoint suitable for communicating over a particular type or implementation of the interface circuit 115 used by the system 100. Each of the memory controllers 155 is connected to the respective memory 140 to allow access (eg, read and write) of the memory 140 by the hardware accelerator 135.

In one or more embodiments, the hardware accelerator 135-1 and the memory 140-1 are mounted on a first circuit board (not shown), and the hardware accelerator 135-2 and the memory 140-2 are attached. , Attached to a second circuit board (not shown), hardware accelerator 135-3 and memory 140-3 are attached to a third circuit board (not shown). Each of these circuit boards may include a suitable connector for coupling to a bus port or slot. For example, each of the circuit boards may have a connector configured for insertion into the available PCIe slots (or other bus / interface connectors) of the system 100.

Each of the link circuits 150 is capable of establishing an accelerator link with at least one other, eg, adjacent, link circuit 150. As used herein, "accelerator link" refers to a communication link that directly connects two hardware accelerators. For example, each of the circuit boards with the hardware accelerator 135 may be connected through a wire that connects to the link circuit 150. The link circuit 150 may establish an accelerator link via a wire.

In certain embodiments, the link circuit 150 is communicably linked using a ring topology. Data sent over the (s) accelerator links established by the link circuit 150 is mastered from left to right as indicated by the directional arrows. For example, referring to the example of FIG. 1, the left link circuit (eg, link circuit 150-1) can act as a master and the right adjacent link circuit (eg, link circuit 150-2) can act as a slave. obtain. Similarly, the link circuit 150-2 may act as a master with respect to the link circuit 150-3. The link circuit 150-3 may act as a master with respect to the link circuit 150-1.

In one or more embodiments, each link circuit 150 includes a table or register that specifies the amount (or size) of memory 140 for each hardware accelerator (eg, on each board). Using the table, each link circuit 150 can modify the address specified in the transaction for the purpose of exchanging information using the accelerator link. In certain embodiments, the table or register is static. In one or more other embodiments, the driver can dynamically read and / or update the information stored in a table or register, eg, at runtime.

For illustrative purposes, the operation of hardware accelerator 135-2 will be described. It should be appreciated that components with similar numbers in each respective hardware accelerator can operate in the same or similar manner. Therefore, referring to hardware accelerator 135-2, link circuit 150-2 can receive transactions from any of a variety of different sources or initiators and route transactions to any of a variety of targets. It is possible. For example, link circuit 150-2 may include transactions from endpoint 145-2 (eg, originating from host processor 105), transactions from compute unit 160-2, transactions from compute unit 165-2, hardware accelerator 135. A transaction from -1 via the link circuit 150-1, or a transaction flowing from the hardware accelerator 135-3 via the link circuit 150-3 to the link circuit 150-1 and then onto the link circuit 150-2. It is possible to receive. The link circuit 150-2 has a transaction to the endpoint 145-2 (for example, to the host processor 105), a transaction to the calculation unit 160-2, a transaction to the calculation unit 165-2, and a memory controller 155-2. Any transaction, such as a transaction, a transaction to the hardware accelerator 135-1 via the link circuit 150-3 over the link circuit 150-1, or a transaction to the hardware accelerator 135-3 via the link circuit 150-3. It is possible to route a transaction to a target, where the target is different from the source or initiator.

For example, the host processor 105 can access any location in memory 140-1, memory 140-2, and / or memory 140-3 as part of the integrated address space. However, when accessing such memory, the host processor 105 was selected using the selected hardware accelerator, eg, hardware accelerator 135-2, and then using the accelerator link. Such memory can be accessed by reaching any target, such as memory 140-1, memory 140-2, or memory 140-3, through a hardware accelerator.

As exemplary and non-limiting example, host processor 105 may initiate data transfer involving hardware accelerators 135-2 and 135-3. Hardware accelerator 135-2 can be an initiator. In this example, the host processor 105, eg, the runtime 172 and / or the driver 174, creates a buffer object 176 corresponding to the hardware accelerator 135-2 and a buffer object 178 corresponding to the hardware accelerator 135-3. Host processor 105 indicates that the target address for data transfer (in hardware accelerator 135-3) is remote to the starting hardware accelerator (hardware accelerator 135-2) with remote flag 182. To set.

The endpoint 145-2 is capable of receiving tasks offloaded from the host processor 105 via the interface circuit 115. In one or more embodiments, the host processor 105 can consider the hardware accelerator 135 as an integrated address space through running runtime 172 and driver 174. The endpoint 145-2 may provide a task (eg, data) to calculation unit 160-2. The task may specify a target address in memory 140-3 from which calculation unit 160-2 should retrieve data for performing the offloaded task. The hardware accelerator 135-2 uses the link circuit 150-2 and directly with the hardware accelerator 135-3 via the accelerator link established between the link circuit 150-2 and the link circuit 150-3. , It is possible to start and carry out data transfer.

The data transfer is initiated by the host processor 105, but the data transfer is performed using the link circuit 150 and without involvement of the host processor 105, the host memory 110, or the interface circuit 115. Data transfer takes place directly between hardware accelerators. In conventional systems, data transfer involves the host processor 105 retrieving data from hardware accelerator 135-3 via interface circuit 115 and then transferring data to hardware accelerator 135-2 via interface circuit 115. It will be done by providing.

The ability of the hardware accelerator 135 to read and write data between the hardware accelerator 135 itself without moving that data through the host processor 105 is received via the interface circuit 115 (eg, the PCIe bus). Significantly reduce the amount of data passed. This saves considerable bandwidth of the interface circuit 115 for use in transmitting data between the host processor 105 and the other hardware accelerator 135. In addition, the speed of operation of the system 100 can be increased by reducing the time required for the hardware accelerator 135 to share data.

The system 100 may include fewer components than the components shown, or additional components not shown in FIG. 1, depending on the device to be mounted and / or the particular type of system. In addition, the particular operating system, application (s), and / or I / O devices included may vary based on the system type. In addition, one or more of the exemplary components may be incorporated into another component or otherwise form part of another component. For example, the processor may include at least some memory. System 100 can be used to implement a single computer or multiple networked or interconnected computers, each implemented using the architecture of FIG. 1 or similar.

FIG. 2 shows an exemplary implementation of the hardware accelerator 135-2 of FIG. An exemplary implementation of the link circuit 150-2 is provided in FIG. It should be appreciated that the architecture shown for link circuit 150-2 in FIG. 2 can be used to implement any of the link circuits 150 shown in FIG.

In one or more embodiments, the link circuit 150-2 converts a transaction that should be sent to another hardware accelerator into a data stream-based packet, providing an accelerator link established between the link circuits 150. It is possible to route packets through. In certain embodiments, the link circuit 150-2 is capable of converting an AMBA Extensible Interface (AXI) compliant memory-mapped transaction into an AXI data stream for transmission. Within this disclosure, AXI is used as an exemplary communication protocol. Please understand that other communication protocols may be used. In this regard, the use of AXI is for illustrative purposes only. The link circuit 150-2 handles incoming packets from other hardware accelerators (eg, hardware accelerators 135-1 and 135-3), converts the packets into memory-mapped transactions, and hardens the data. It is also possible to route locally within the hardware accelerator 135-2. In addition, the link circuit 150-2 converts the received packet into a memory-mapped transaction, modifies the transaction, converts the memory-mapped transaction into a packet, and passes the packet to the next hardware accelerator. Is possible. Data received over the accelerator link can be routed internally within the hardware accelerator 135-2 as a memory-mapped transaction.

In the example of FIG. 2, the link circuit 150-2 includes transceivers 202 and 204, retransmission engines (RTE) 206 and 208, and memory map-stream (MM-stream) mappers 210 and 212. The MM-stream mappers 210 and 212 are connected to the interconnect circuit 214.

As depicted, transceiver 202 may be connected to the corresponding transceiver in hardware accelerator 135-1 and transceiver 204 may be connected to the corresponding transceiver in hardware accelerator 135-3. Transceivers 202 and 204 implement the physical layer of the accelerator link established with other hardware accelerators. Each of the transceivers 202 and 204 can implement a lightweight serial communication protocol for multi-Gigabit communication links. In one or more embodiments, each of the transceivers 202 and 204 can implement a bidirectional interface to the transceiver in adjacent ICs. Transceivers 202 and 204 are capable of automatically initializing accelerator links with other hardware accelerators. In general, transceivers 202 and 204 are capable of bidirectional communication for implementing low level signaling and low PHY level protocols related to flow control. However, data flow can be implemented using the ring topology and master-to-slave flow (eg, in a single direction around the ring) as previously described.

For example, transceiver 202 is capable of bidirectionally communicating with the corresponding transceiver in the link circuit 150-1 of hardware accelerator 135-1. Transceiver 204 is capable of bidirectionally communicating with the corresponding transceiver in the link circuit 150-3 of hardware accelerator 135-3. Each of the transceivers 202 and 204 can use a data stream, eg, an AXI data stream, to communicate with an adjacent transceiver.

In certain embodiments, transceivers 202 and 204 are capable of sending and receiving data to adjacent hardware accelerators using 8B / 10B coding rules. Each of the transceivers 202 and 204 can detect single-bit errors and most multi-bit errors using 8B / 10B coding rules.

In one or more embodiments, each of the transceivers 202 and 204 is implemented as an Aurora 8B / 10B IP core, which is described by Xilinx, Inc., San Jose, Calif. It is available from. However, please understand that the particular core mentioned is provided for illustrative purposes and is not intended as a limitation. Other transceivers capable of operating as described herein may be used.

The transceiver 202 is connected to the RTE 206. The transceiver 202 and RTE206 can communicate through a plurality of data streams moving in each direction that support bidirectional communication. Transceiver 204 is connected to RTE208. Transceiver 204 and RTE 208 can communicate through multiple data streams moving in each direction that support bidirectional communication.

RTE206 and 208 are capable of managing transactions. In one or more embodiments, the RTE 206 and RTE 208 each implement an additional layer of communication protocol on top of those implemented by transceivers 202 and 204, respectively. For example, RTE206 and RTE208 implement a transaction layer (TL) / link layer (LL) and a user layer, respectively. These additional layers provide additional assurance of data integrity. After initialization, the application can pass data over the accelerator link as a stream of data. Additional data integrity measures are particularly useful when converting memory-mapped transactions to stream data, as the control signals are merged with the data. Data integrity issues can result in corrupted control signals. On-chip interconnects and / or buses cannot tolerate data loss on control signals.

TL / LL implements token-based flow control to ensure lossless data communication. In one or more embodiments, the communication channel between adjacent transceivers and the communication channel between the transceiver and the RTE are 128 bits wide. When sending data, each RTE buffers enough to receive the entire transaction before the receive link circuit in the target hardware accelerator actually sends the transaction to be sent to the physical layer implemented by the transceiver. It is possible to check that you have a resource (eg, a token). For example, the RTE 206 is sufficient for the receive link circuit 150-1 in the hardware accelerator 135-1 to receive the data before it provides the data to the transceiver 202 (within the link circuit 150-2) to send. It can be inspected to have a good buffer resource.

RTE206 and 208 are capable of detecting data corruption. For example, each of the RTEs 206 and 208 can verify packet length information, packet sequence information, and / or cyclic redundancy check (CRC) checksums for each packet received. When an RTE slave (eg, a receiving RTE) detects a packet error, the RTE may enter error abort mode. In error abort mode, the RTE drops the packet with the error as a failed packet. The RTE further drops all subsequent packets of the transaction. In certain embodiments, the start of error abort mode causes the RTE to trigger a link retry sequence. If the link retry sequence is successful, the link master (eg, the sending RTE) can restore the transmission by starting from the point of failure.

The RTE 206 is connected to the MM stream mapper 210. The RTE 206 is capable of communicating with the MM stream mapper 210 via a plurality of data streams moving in each direction that support bidirectional communication. The RTE 208 is connected to the MM stream mapper 212. The RTE 208 can communicate with the MM stream mapper 212 via a plurality of data streams moving in each direction that support bidirectional communication.

Each of the MM stream mapper 210 and the MM stream mapper 212 is connected to the interconnection circuit 214. The interconnect circuit 214 is capable of routing data between the MM stream mappers 210 and 212 and other master and / or slave circuits of the hardware accelerator 135-2 connected to them. The interconnect circuit 214 may be implemented as one or more on-chip interconnects. An example of on-chip interconnection is the AXI bus. The AXI bus is an embedded microcontroller bus interface for use in establishing on-chip connections between circuit blocks and / or systems. Other exemplary implementations of interconnect circuits may include, but are not limited to, other buses, crossbars, network on chip (NoC), and the like.

The MM stream mappers 210 and 212 are capable of converting the received data streams from the RTEs 206 and 208 into memory-mapped transactions that may be provided to the interconnect circuit block 214, respectively. In this regard, the data stream can be demultiplexed into multiple channels that support memory-mapped transactions. The MM stream mappers 210 and 212 can also convert the received memory-mapped transactions from the interconnect circuit block 214 into stream data that can be provided to RTE 206 and 208, respectively. The MM stream mappers 210 and 212 multiplex multiple channels that support memory-mapped transactions (including, for example, the control signals described) into a single data stream for sending to RTEs 206 and 208, respectively. It is possible.

In one or more embodiments, each of the MM stream mappers 210 and 212 is capable of adjusting the target address received in the transaction. When the MM stream mapper 210 receives a transaction from the hardware accelerator 135-1 via the accelerator link, for example, the address range for the hardware accelerator 135-2 from the target address of the transaction (for example, memory 140-). The upper limit of the address range of 2) can be subtracted. As the transaction passes through the link circuit 150, the transaction can be directed from one hardware accelerator to another via the accelerator link by adjusting the target address. Further details relating to address behavior when using accelerator links are described in more detail with respect to FIG.

For illustrative purposes, other parts of the hardware accelerator 135-2 are described with respect to the link circuit 150-2. In the example of FIG. 2, the interconnect circuit 214 is directly connected to the memory access (DMA) master circuit 216. The DMA master circuit 216 includes, for example, a memory-mapped interface for communicating with the interconnect circuit block 214. The DMA master circuit 216 is connected to the PCIe endpoint 218. The PCIe endpoint 218, which is an exemplary implementation of the endpoint 145-2 of FIG. 1, is communicably linked to the host processor 105.

In the example of FIG. 2, the interconnect circuit 214 is also connected to one or more calculation unit masters 220-1 to 220-N. Each calculation unit master 220 provides a bidirectional interface between the calculation unit mounted in the hardware accelerator 135-2 and the interconnect circuit block 214. Each calculation unit master 220 further includes a memory-mapped interface for communicating with the interconnect circuit block 214. Each of the calculation unit 160-2 and the calculation unit 165-2 may be connected to the interconnect circuit 214 via a slave interface (not shown).

In the example of FIG. 2, the interconnect circuit 214 is also connected to one or more memory controller slave circuits 225-1 to 225-N. Each memory controller slave circuit 225 enables read and write operations for memory 140-2. Memory 140-2 may be implemented as one or more off-chip memories accessible by hardware accelerator 135-2. Each of the memory controllers 225-1 to 225-N further includes a memory-mapped interface for communicating with the interconnect circuit block 214.

FIG. 3 shows an exemplary implementation of the RTE 206. The exemplary architecture described with reference to FIG. 3 implements a credit-based flow control / retransmission control scheme using a flow control unit (FLIT). The RTE 206 is capable of translating between FLIT-based protocols and / or interfaces that are used internally for protocols and / or interfaces that may be used by the application.

The RTE 206 includes a transmission channel 330. Transmission channel 330 is capable of decapsulating a data (eg, AXI) stream into a FLIT-based transaction. In the example of FIG. 3, the transmit channel 330 includes a transmit (TX) packet cyclic redundancy check (CRC) generator 302, a retry pointer return command (PRET) packet / initial retry command (IRTRY) packet generator, and a return retry. Trial pointer (RRP) embedding device 304, token return (TRET) packet generator and sequence (SEQ) number / forward retry pointer (FRP) / return token count (RTC) embedding device 306, flow control circuit 308, and Includes an output buffer 310. The TRET generator and SEQ / FRP / RTC implanter 306 are also connected to the retry buffer 312.

The RTE 206 includes a receive channel 340. The receive channel 340 can encapsulate a FLIT-based interface and convert that interface into a data (eg, AXI) stream. In the example of FIG. 3, the receive channel 340 includes a packet boundary detector 316, a receive (RX) packet CRC circuit 318, an RX packet processor 320, and an input buffer 322. The Rx packet processor 320 is connected to the error handler 324 and the retry sequence circuit 314.

RTE206 is provided for purposes of illustration, but not limitation. Please understand that other architectures suitable for implementing credit-based flow control / retransmission control schemes may be used. The architecture described with respect to FIG. 3 can also be used to implement the RTE 208 of FIG. 2 which has an inverted or reversed orientation with respect to the data flow.

FIG. 4 shows an exemplary method of operation 400 for a system with multiple hardware accelerators. Method 400 illustrates an example of direct data transfer between hardware accelerators. Method 400 may be performed by the same or similar system as system 100 described with respect to FIG. Method 400 shows how insufficient bandwidth on the bus connecting the host processor and the hardware accelerator can be mitigated. Data transfers that would normally take place on the bus can be directed to the accelerator link, thereby freeing up bandwidth on the bus for other operations.

At block 405, the system can automatically discover the hardware accelerator sequence. In one or more embodiments, the hardware accelerator, eg, the board of the hardware accelerator, is configured in a ring topology within the system. The host processor is aware of the existing PCIe topology, and thus the number of hardware accelerators present in the system connected to the PCIe bus. In addition, the host processor is aware of the particular circuit (eg, image or configuration bitstream) loaded into each hardware accelerator, for example, through the runtime. Therefore, the host processor is aware that hardware accelerators support accelerator links as described herein. The host processor still has to determine the sequence of hardware accelerators. The driver can, for example, perform automatic discovery of the hardware accelerator sequence described. This auto-discovery capability supports the addition of new and / or additional hardware accelerators to the system without the need to modify the applications run by the host processor.

Each hardware accelerator may have known and the same address range. For example, each hardware accelerator can be assumed to have an address range of 16 GB corresponding to 16 GB of memory 140. In one or more embodiments, the host processor is capable of writing a unique value to the memory address at 16 GB intervals. The host processor can then read the values again to determine the sequence of hardware accelerators within the ring topology based on the write and read values.

At block 410, the host processor can create a buffer on each hardware accelerator at startup. For example, the runtime run by the host processor can communicate with each hardware accelerator to create a buffer in the memory of each hardware accelerator. Referring to FIG. 1, hardware accelerator 135-1 creates a buffer in memory 140-1. Hardware accelerator 135-2 creates a buffer in memory 140-2. Hardware accelerator 135-3 creates a buffer in memory 140-3.

At block 415, the host processor initiates data transfer between hardware accelerators. Data transfer can be, for example, part of a task that should be offloaded from the host processor to the hardware accelerator. As an exemplary and non-limiting example, the host processor 105 may offload tasks for an application to compute unit 160-1 of hardware accelerator 135-1. The task may include an instruction and a target address from which calculation unit 160-1 should obtain data for the task. The target address in this example is in hardware accelerator 135-2 (eg, in memory 140-2). Therefore, in order to perform a task offloaded from the host processor, compute unit 160-1 must retrieve data from the target address in memory 140-2.

At block 420, the runtime may request data transfer between hardware accelerator 135-1 and hardware accelerator 135-2. For example, the runtime may request a read of hardware accelerator 135-2 by or from hardware accelerator 135-1.

At block 425, the driver can create a buffer object in host memory corresponding to hardware accelerator 135-2 and a buffer object in host memory corresponding to hardware accelerator 135-1. A buffer object is a shadow data structure implemented in host memory. Each buffer object can correspond to or represent a device in the system. The buffer object may contain data that supports the management functions performed by the runtime executed by the host processor.

In one or more embodiments, the buffer object created in host memory may include a remote flag. The remote flag can be set to indicate that the buffer object is remote in terms of the hardware accelerator initiating the transaction. In this example, hardware accelerator 135-1 is reading data from hardware accelerator 135-2. Therefore, hardware accelerator 135-1 is initiating a transaction. The driver sets the remote flag in the buffer object corresponding to hardware accelerator 135-2 at creation time.

At block 430, the runtime library initiates access to the buffer object (eg, the remote buffer object) by the start hardware accelerator. The runtime library initiates access to the buffer object corresponding to hardware accelerator 135-2 from hardware accelerator 135-1. For example, the runtime determines that the remote flag is set in the buffer object for hardware accelerator 135-2. In response to determining that the remote flag is set, the runtime library schedules the transfer using the accelerator link established by the link circuit. When scheduling a transfer using an accelerator link between hardware accelerators, the runtime determines the address that should be used by the hardware accelerator 135-1 to access the data from the hardware accelerator 135-2. ..

For illustrative purposes, consider an example in which each of the hardware accelerators 135 has an address range of 1 to 1000. In such an example, the runtime corresponds to the data to be retrieved from hardware accelerator 135-2 by hardware accelerator 135-1 at address 500 corresponding to hardware accelerator 135-2 (eg, memory 140-2). Can be determined to be in the buffer (at address 500). In this example, the runtime adds 1000 to the target address to yield 1500 addresses, which are the hardware accelerator 135 as the target address for reading data to operate for offloaded tasks. Provided in -1.

As another example, if data is stored at address 500 in memory 140-3, the runtime will run an address range of 1 to 1000 for each of the hardware accelerator 135 in order for the transaction to reach hardware accelerator 135-3. 2000 will be added, assuming that In general, as is known, return route data can be tracked through the on-chip bus interconnects used (eg, AXI interconnects). When a read request is issued from the master, for example, the read request interconnects with a series of address decryption and / or address shifts (implemented by the mm stream mapper) as the read request traverses each hardware accelerator. Routed to the slave through. Each individual interconnect can track which master has an open transaction to each slave. When the read data is returned, the read data can be sent back through the correct interface (s). In some cases, an identifier (ID) bit may be used to associate the read data with a particular master in order to return the particular read data.

At block 435, the starting hardware accelerator (eg, the first hardware accelerator) receives the task from the host processor. The endpoint 145-1 may, for example, receive a task and provide the task to calculation unit 160-1. The task specifies that the data to be the target of the operation by the calculation unit 160-1 is located at the target address, and the target address is 1500 in this example. The calculation unit 160-1 may have, for example, a control port in which the target address can be stored. When attempting to access data located at address 1500, compute unit 160-1 recognizes that the address is not within range of hardware accelerator 135-1. For example, the calculation unit 160-1 can compare the address with the upper bound of the 1000 address range and determine that the address exceeds the upper bound. In this example, compute unit 160-1 is capable of initiating a read transaction from address 1500. For example, compute unit 160-1 may initiate a read transaction as a memory-mapped transaction sent over interconnect 214.

At block 440, the starting hardware accelerator accesses the target hardware accelerator (eg, a second hardware accelerator) via the accelerator link. For example, link circuit 150-1 can convert a memory-mapped transaction initiated by compute unit 160-1 (eg, using an MM stream mapper) into a stream-based packet. The link circuit 150-1 can further encode packets with additional data that support data integrity checking, retransmission, initialization, and error reporting (eg, using RPE). be. The ring topology can be mastered from left to right. Therefore, the packet can be output to the link circuit 150-2 by the transceiver of the link circuit 150-1.

The link circuit 150-2 receives the data stream on the transceiver 202 and processes the transaction on the RTE 206. The MM stream mapper 210 can perform various operations in response to receiving a packet of the stream database. The MM stream mapper 210 can, for example, convert stream-based packets into memory-mapped transactions. Further, the MM stream mapper 210 can decrement the target address of 1500 by the upper limit of the address range of the hardware accelerator 135-2. As mentioned, the upper limit can be stored in a table or register in the link circuit 150-2, eg, in the MM stream mapper 210. In this example, the MM stream mapper 210 subtracts 1500 target addresses by 1000 to produce 500 target addresses. Since the target address is local to hardware accelerator 135-2, hardware accelerator 135-2 can act on received transactions. In this example, the MM stream mapper 210 provides memory-mapped transactions to interconnect 214. The memory-mapped transaction may be provided to the memory controller 155-2 (eg, through the memory controller slave) to perform a read transaction. In this way, the hardware accelerator 135-1 can read data from the hardware accelerator 135-2 (or write data to the hardware accelerator 135-2). The requested data can be provided from memory 140-2 to the requester using the same route used to send the read request. For example, data read from memory 140-2 traverses forward through the ring topology to hardware accelerator 135-3 and then to hardware accelerator 135-1 without having to traverse to hardware accelerator 135-1. 2 is sent to the hardware accelerator 135-1.

For example, if the target address is 2500, the result of the decrement is 1500. In that case, the MM stream mapper 210 determines that the target address is not in the hardware accelerator 135-2 because the target address is greater than the upper limit of the address range for the hardware accelerator 135-2 (eg, 1000). In that case, the MM stream mapper 210 may send a transaction to the MM stream mapper 212 through an interconnect circuit for forwarding on the next hardware accelerator.

At block 445, the calculation unit 160-1 in the hardware accelerator 135-1 can generate an interrupt to the host processor that informs the host processor that the data transfer between the hardware accelerators is complete. At block 450, the runtime can provide the application with the necessary notification that the data transfer is complete. The runtime can, for example, handle completion events, command queues, and notifications to the application.

In one or more embodiments, the PCIe endpoint and DMA master can write to target addresses in different hardware accelerators. As an exemplary and non-limiting example, the host processor may send data to hardware accelerator 135-1 with a target address in hardware accelerator 135-2. In that case, the DMA master can recognize that the target address is in a different hardware accelerator and schedule data transfer over the accelerator link. For example, the DMA master may compare the target address with the upper bound of the address range for hardware accelerator 135-1. In response to determining that the target address exceeds the upper bound, the DMA master MM in the link circuit 150-1 via the interconnect circuit to send to the hardware accelerator 135-2 over the accelerator link. It is possible to initiate a memory-mapped transaction to the stream mapper 212.

In one or more embodiments, the host processor can use accelerator links for load balancing purposes. For example, the host processor to determine the status of a DMA channel (eg, DMA master) in a selected hardware accelerator for which data should be provided or tasks should be offloaded. , It is possible to use the runtime. In response to the DMA master determining that it is busy or operating above the activity threshold, the host processor may send data over the bus to different hardware accelerators. The data may specify a target address within the selected hardware accelerator. When the DMA master in the receiving hardware accelerator receives the data from the host processor, it can forward the data to the selected hardware accelerator via the accelerator link (s). In certain embodiments, the host processor selects the receive hardware accelerator based on the determination that the DMA master in the receive hardware accelerator is not busy or is operating below the activity threshold. It is possible.

For illustrative purposes, an example of a write transaction from hardware accelerator 135-1 to hardware accelerator 135-3 is generally described as being initiated by the host processor. The host processor sets the remote flag for the target hardware accelerator via the runtime and driver (using the previous example, where the desired address is located at address 500 in the hardware accelerator 135-3). Determine the address of 2500. The host processor provides an instruction to the hardware accelerator 135-1 to write to address 2500. Within the hardware accelerator 135-1, a transaction with 2500 addresses is presented to interconnect 214. Since the address exceeds the upper limit of the hardware accelerator 135-1, the interconnect 214 sends a transaction to the link circuit 150-1. The link circuit 150-1 sends a transaction to the link circuit 150-2. The MM stream mapper in the hardware accelerator 135-2 decrements the address by 1000, resulting in 1500 new addresses. The new address is still remote as 1500 exceeds the upper address limit of hardware accelerator 135-2. Therefore, the transaction is forwarded to hardware accelerator 135-3.

The MM stream mapper in the hardware accelerator 135-3 decrements the address, yielding 500 new addresses. The transaction is then provided to the memory controller via interconnect 214 in hardware accelerator 135-3 and data is written to memory 140-3. In the example described, the address indicates where the transaction can be serviced by the hardware accelerator and, if the transaction can be serviced by the hardware accelerator, where the transaction is internally (eg, a memory controller or other circuit block). Used by each hardware accelerator to determine whether it should be routed or forwarded to the next hardware accelerator. In certain embodiments, the address is different from the actual address at which the data is written into memory. The write acknowledgment is sent from hardware accelerator 135-3 through hardware accelerator 135-2 to hardware accelerator 135-1 as described.

For illustrative purposes, another example of a read transaction to hardware accelerator 135-3 initiated by hardware accelerator 135-1 is generally described as being initiated by the host processor. The host processor sets the remote flag for the target hardware accelerator via the runtime and driver (using the previous example, where the desired address is located at address 500 in the hardware accelerator 135-3). Determine the address of 2500. The host processor provides instructions to the hardware accelerator 135-1 to read from address 2500. Within the hardware accelerator 135-1, a transaction with 2500 addresses is presented to interconnect 214. Since the address exceeds the upper limit of the hardware accelerator 135-1, the interconnect 214 sends a transaction to the link circuit 150-1. The link circuit 150-1 sends a transaction to the link circuit 150-2. The MM stream mapper in the hardware accelerator 135-2 decrements the address by 1000, resulting in 1500 new addresses. The new address is still remote as 1500 exceeds the upper address limit of hardware accelerator 135-2. Therefore, the transaction is forwarded to hardware accelerator 135-3.

The MM stream mapper in the hardware accelerator 135-3 decrements the address, yielding 500 new addresses. The transaction is then provided to the memory controller via interconnect 214 in hardware accelerator 135-3 and data is read from memory 140-3. In the example described, the address indicates whether the transaction can be serviced by a hardware accelerator, and where the transaction should be routed internally if the transaction can be serviced by a hardware accelerator, or the next hardware accelerator. Used by each hardware accelerator to determine if it should be forwarded to. In certain embodiments, the address is different from the actual address from which the data is read from memory. The read data is sent from hardware accelerator 135-3 through hardware accelerator 135-2 to hardware accelerator 135-1 as described.

FIG. 5 shows an example of a system that includes a hardware accelerator and one or more additional devices. In the example of FIG. 5, hardware accelerators 135-1 and 135-2 are shown and are connected by accelerator links using the link circuitry in each hardware accelerator. For illustrative purposes, hardware accelerators 135-3 are not shown. The system also includes a GPU 515, which is connected to a memory 520 and an I / O device 525.

In the example of FIG. 5, the GPU 515 may write data to or read data from hardware accelerator 135-2. In this example, the host processor (not shown) provides the GPU 515 with handles 505-N. In certain embodiments, the handle can be implemented as a file descriptor. Handles 505-N may point to buffer object 510-N, which corresponds to hardware accelerator 135-2. When the GPU 515 uses handles 505-N for read or write operations, the host processor initiates an action on the buffer object corresponding to handle 505-N, eg, buffer object 510-N. The host processor determines whether the buffer object 510-N is local or remote. Since the remote flag in the buffer object 510-N is not set, the host processor may retrieve data from memory 140-2 via PCIe and provide data to GPU 515 via PCIe.

In one or more other embodiments, the host processor may initiate the retrieval of data from memory 140-2 by accessing different hardware accelerators. For example, the host processor may initiate communication with the hardware accelerator 135-1 via PCIe to retrieve data from memory 140-2. In that case, the hardware accelerator 135-1 may communicate directly with the hardware accelerator 135-2 using a link circuit to retrieve data from memory 140-2. The hardware accelerator 135-1 may then provide the data to the host processor, which provides the data to the GPU 515 via PCIe.

In another example, an I / O device 525, such as a camera, may write data to the hardware accelerator 135-1. In that case, the host processor can provide handle 505-1 to the I / O device 525. Handle 505-1 may point to buffer object 510-1, and buffer object 510-1 corresponds to hardware accelerator 135-1. When the I / O device 525 uses the handle 505-1 for the write operation, the host processor initiates an action on the buffer object corresponding to the handle 505-1, eg, the buffer object 510-1. The host processor determines whether the buffer object 510-1 is local or remote. Since the remote flag in buffer object 510-1 is not set, the host processor receives data from the I / O device 525 and sends the PCIe for writing to memory 140-1 and / or for further processing. Such data may be provided to the hardware accelerator 135-1 via.

In one or more embodiments, the driver may set the remote flag in the buffer object only for data transfers between hardware accelerators where accelerator links can be used as described. It is possible. FIG. 5 shows that other types of devices can be used with hardware accelerators, but data transfer between such other devices and the hardware accelerators takes place over the bus and involves a host processor. Is shown.

FIG. 6 shows an exemplary architecture 600 for ICs. In one aspect, the architecture 600 may be implemented within a programmable IC. For example, architecture 600 can be used to implement a field programmable gate array (FPGA). Architecture 600 may also represent a system-on-chip (SOC) type of IC. An SOC is an IC that includes a processor that executes program code and one or more other circuits. Other circuits may be implemented as hard-wired circuits, programmable circuits, and / or combinations thereof. The circuits may operate with each other and / or in cooperation with the processor.

As shown, the architecture 600 includes several different types of programmable circuits, such as logic, blocks. For example, architecture 600 includes a multi-gigabit transceiver (MGT) 601, configurable logic block (CLB) 602, random access memory block (BRAM) 603, input / output block (IOB) 604, configuration and clocking logic. (CONFIG / CLOCKS) 605, Digital Signal Processing Block (DSP) 606, Special I / O Block 607 (eg, Configuration Ports and Clock Ports), and Others such as Digital Clock Managers, Analog Digital Converters, System Monitoring Logic, etc. It may include a number of different programmable tiles, including programmable logic 608.

In some ICs, each programmable tile contains a programmable interconnect element (INT) 611, which has a standardized connection to and from the corresponding INT 611 in each adjacent tile. Therefore, INT611, in summary, implements a programmable interconnect structure for the ICs shown. Each INT611 also includes a connection between programmable logic elements within the same tile, as shown by the example contained at the top of FIG.

For example, CLB602 may include a configurable logic element (CLE) 612, which may be programmed to implement user logic, and a single INT611. The BRAM 603 may include a BRAM logical element (BRL) 613 in addition to the one or more INT 611s. Generally, the number of INT611s contained in a tile depends on the height of the tile. As depicted, BRAM tiles have the same height as 5 CLBs, but other numbers (eg, 4) may be used. The DSP tile 606 may include a DSP logical element (DSPL) 614 in addition to an appropriate number of INT611s. The IOB604 may include, for example, one instance of INT611 plus two instances of the I / O logical element (IOL) 615. The actual I / O pad connected to the IOL 615 may not be limited to the area of the IOL 615.

In the example depicted in FIG. 6, a columnar area near the center of the die, for example formed from regions 605, 607, and 608, is used for configuration, clock, and other control logic. obtain. A horizontal area 609 extending from this row can be used to distribute the clock and set signals across the width of the programmable IC.

Some ICs utilizing the architecture shown in FIG. 6 include additional logical blocks that compromise the regular sequence structure that makes up most of the ICs. Additional logic blocks can be programmable blocks and / or dedicated circuits. For example, the processor block, designated as PROC610, spans several columns of CLB and BRAM.

In one aspect, the PROC610 can be implemented as a dedicated circuit made as part of a die that implements a programmable circuit of an IC, for example as a hardwired processor. The PROC610 ranges in complexity from individual processors, such as a single core capable of executing program code, to an entire processor system with one or more cores, modules, coprocessors, interfaces, and so on. It can represent any of a variety of different processor types and / or systems.

In another aspect, PROC610 can be omitted from architecture 600 and replaced with one or more of the other types of programmable blocks described. In addition, such blocks form a "soft processor" in that various blocks of programmable circuits can be used to form a processor capable of executing program code as in the case of PROC610. Can be used for.

The phrase "programmable circuit" refers to programmable circuit elements within an IC, such as the various programmable or configurable circuit blocks or tiles described herein, as well as the various circuit blocks according to the configuration data loaded into the IC. Refers to an interconnect circuit that selectively connects tiles and / or elements. For example, the circuit blocks shown in FIG. 6 outside the PROC610, such as CLB602 and BRAM603, are considered as programmable circuits of the IC.

In general, the functionality of a programmable circuit is not established until the configuration data is loaded into the IC. A set of set bits can be used to program programmable circuits of ICs, such as FPGAs. Setting bits (one or more) are commonly referred to as "setting bitstreams". In general, a programmable circuit is not operational or non-functional without first loading the set bitstream into the IC. The set bitstream effectively implements a particular circuit design within the programmable circuit. The circuit design specifies, for example, the functional aspects of the programmable circuit block and the physical connectivity between the various programmable circuit blocks.

Circuits that are "hard-wired" or "hardened," that is, non-programmable, are manufactured as part of the IC. Unlike programmable circuits, hard-wired circuits or circuit blocks are not implemented after the IC is manufactured through the loading of the set bitstream. Hardwired circuits are generally considered to have dedicated circuit blocks and interconnects that can function, for example, without first loading a set bitstream into an IC, eg, PROC610.

In some cases, the hardwired circuit may have one or more modes of operation that can be set or selected according to register settings or values stored in one or more memory elements in the IC. The operating mode can be set, for example, through the loading of the set bitstream into the IC. Despite this capability, hardwired circuits are not considered programmable circuits because they are operational and have specific functionality when manufactured as part of an IC.

In the case of SOC, the set bitstream may specify the circuit to be implemented in the programmable circuit and the program code to be executed by the PROC610 or soft processor. In some cases, architecture 600 includes a dedicated configuration processor that loads the configuration bitstream into the appropriate configuration memory and / or processor memory. The dedicated configuration processor does not execute user-specified program code. In other cases, architecture 600 may utilize PROC610 to receive the configuration bitstream, load the configuration bitstream into the appropriate configuration memory, and / or extract the program code for execution.

FIG. 6 is intended to show an exemplary architecture that can be used to implement programmable circuits, eg, ICs that include programmable fabrics. For example, the number of logical blocks in a column, the relative width of the columns, the number and order of columns, the types of logical blocks contained in a column, the relative size of logical blocks, and the interconnects included at the top of FIG. The logical implementation form is only an example. In a real IC, for example, two or more adjacent columns of CLBs are generally included wherever CLBs appear to facilitate efficient implementation of user circuit design. However, the number of adjacent CLB columns can vary with the overall size of the IC. Moreover, the size and / or placement of blocks such as PROC610 within the IC is for illustration purposes only and is not intended as a limitation.

Architecture 600 can be used to implement hardware accelerators as described herein. In certain embodiments, one or more of endpoints, link circuits, and memory controllers may be implemented as hardwired circuit blocks. In certain embodiments, one or more of endpoints, link circuits, and memory controllers may be implemented using programmable circuits. In yet another embodiment, one or more of the mentioned circuit blocks may be implemented as hard-wired circuit blocks, the other being implemented using a programmable circuit.

The embodiments described herein include, for example, database acceleration, processing multiple video streams, real-time network traffic monitoring, machine learning, or any other application that may involve multiple hardware accelerators. , Can be used in any of various applications.

For illustration purposes, specific names are given to provide a complete understanding of the various invention concepts disclosed herein. However, the terminology used herein is merely for explaining certain aspects of the configuration of the present invention and is not limiting.

The singular forms "a", "an" and "the" as defined herein shall also include the plural, unless the context specifically dictates otherwise.

The term "about" as defined herein means inaccurate, but nearly correct or exact, close in value or quantity. For example, the term "about" can mean that the specified property, parameter, or value is within a predetermined quantity of exact property, parameter, or value.

The terms "at least one," "one or more," and "and / or" as defined herein are both conjunctive and selective in operation, unless otherwise stated. There is an open-ended expression. For example, "at least one of A, B, and C", "at least one of A, B, or C", "one or more of A, B, and C", "A. , B, or C, and each of the expressions "A, B, and / or C" are A only, B only, C only, A and B together, A and C. Means together, B and C together, or A, B and C together.

The term "automatically" as defined herein means without user intervention. The term "user" as defined herein means human.

The term "computer-readable storage medium" as defined herein refers to a storage medium that contains or stores program code for or with use by an instruction execution system, device, or device. means. The "computer-readable storage medium" as defined herein is not itself a transient propagating signal. The computer-readable storage medium can be, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination described above. The various forms of memory described herein are examples of computer-readable storage media. A non-exhaustive list of more specific examples of computer-readable storage media is portable computer disksets, hard disks, RAM, read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electronically erasable. It may include programmable read-only memory (EEPROM), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks and the like.

The term "if" as defined herein means "when" or "upon" or "in response to" or, depending on the context. It means "responsive to". Therefore, the phrase "if it is determined" or "if [stated condition or event] is detected" may be "determined" or "in response to the decision" or, depending on the context. "When [stated condition or event] is detected" or "in response to [stated condition or event] detected" or "in response to [stated condition or event] detected" Can be interpreted as meaning.

The term "in response" as defined herein and similar phrases as described above, such as "when", "when", or "to", are easy for an action or event. Means to respond to or react to. The response or response is automatic. Therefore, when the second action is performed "in response to" the first action, there is a causal relationship between the occurrence of the first action and the occurrence of the second action. The term "in response" indicates a causal relationship.

The terms "one embodied", "an embodied", "one or more embodiments", "specific embodiments", or similar terms as defined herein. Means that a particular feature, structure, or property described with respect to an embodiment is included in at least one embodiment described herein. Thus, throughout the disclosure, "in one embodied," "in an embodied," "in one or more embodiments," and "in a particular embodiment." The appearance of phrases and similar phrases can all, but not necessarily, refer to the same embodiment. The terms "embodiment" and "configuration" are used interchangeably within this disclosure.

The term "processor" as defined herein means at least one hardware circuit. Hardware circuits can be configured to perform the instructions contained in the program code. The hardware circuit can be an integrated circuit. Examples of processors include, but are not limited to, central processing units (CPUs), array processors, vector processors, digital signal processors (DSPs), FPGAs, programmable logic arrays (PLAs), ASICs, programmable logic circuits, and controllers.

The term "output" as defined herein refers to a physical memory element, such as storing in a device, writing to a display or other peripheral output device, sending or sending to another system, or exporting. It means things like that.

The term "real time" as defined herein allows a user or system to feel immediate enough for a particular process or decision to be made, or to allow a processor to keep up with some external process. Means the level of processing responsiveness.

The term "substantially" as defined herein means that the specified properties, parameters, or values do not need to be achieved exactly, but for example, tolerances known to those skilled in the art. It means that deviations or variations, including measurement errors, measurement accuracy limits, and other factors, can occur in quantities that do not interfere with the effect the property is intended to provide.

Terms such as first and second can be used herein to describe various elements. These terms are used only to distinguish one element from another, unless otherwise stated or the context specifically dictates otherwise. Should not be limited by.

Computer program products may include computer-readable storage media (s) on which computer-readable program instructions for causing a processor to perform aspects of the configuration of the invention described herein. Within this disclosure, the term "program code" is used interchangeably with the term "computer-readable program instructions." The computer-readable program instructions described herein are external computers or external storage from computer-readable storage media to their respective computing / processing devices or via networks such as the Internet, LAN, WAN and / or wireless networks. Can be downloaded to your device. The network may include edge devices including copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer-readable program instructions from the network and stores the computer-readable program instructions in a computer-readable storage medium in each computing / processing device. Forward to.

The computer-readable program instructions for performing the operations for the configuration of the invention described herein are assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, or It can be either source code or object code written in any combination of one or more programming languages, including an object-oriented programming language and / or a procedural programming language. Computer-readable program instructions may include state setting data. Computer-readable program instructions are entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on the remote computer, or entirely on the remote computer. Or it can be run on the server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including LAN or WAN, or the connection can be made to an external computer (eg, through the Internet using an Internet service provider). Can be done. In some cases, for example, an electronic circuit, including a programmable logic circuit, FPGA, or PLA, is used to personalize the electronic circuit in order to implement the aspects of the configuration of the invention described herein. Computer-readable program instructions can be executed by using the state information of computer-readable program instructions.

Some aspects of the configuration of the present invention have been described herein with reference to flow chart exemplary and / or block diagrams of methods, devices (systems), and computer program products. It will be appreciated that each block of the flowchart and / or block diagram, and the combination of blocks in the flowchart and / or block diagram, can be implemented by computer-readable program instructions, such as program code.

These computer-readable program instructions can be given to a general purpose computer, a dedicated computer, or the processor of another programmable data processor for manufacturing the machine, and as a result, through the processor of the computer or other programmable data processor. The command to be executed creates means for implementing the specified function / action in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored on a computer-readable storage medium that can instruct the computer, programmable data processor, and / or other device to function in a particular manner, so that the instructions are The stored computer-readable storage medium comprises a product containing instructions that implement the specified mode of operation in one or more blocks of the flowchart and / or block diagram.

Computer-readable program instructions also cause a computer, other programmable device, or other device to perform a series of operations on a computer, other programmable data processor, or other device to create a computer-implemented process. Instructions that can be loaded and thus executed on a computer, other programmable device, or other device implement the specified function / action in one or more blocks of flowcharts and / or block diagrams.

Flowcharts and block diagrams in the drawings show the architecture, functionality, and behavior of possible implementations of systems, methods, and computer program products in various aspects of the configuration of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of an instruction that comprises one or more executable instructions to implement the specified operation.

In some alternative implementations, the actions mentioned in the block may occur out of the order mentioned in the figure. For example, depending on the functionality involved, two blocks shown in succession can be executed substantially simultaneously, or the blocks can sometimes be executed in reverse order. In another example, the blocks can generally be performed in ascending order, and in yet another example, one or more blocks can be performed in variable order, and the results are stored and subsequently or immediately after. It is used in other blocks that do not come. In addition, each block of the block diagram and / or the flowchart example diagram, and the combination of the blocks in the block diagram and / or the flowchart example diagram perform the specified function or action, or the combination of the dedicated hardware and the computer instruction. Note that it can be implemented by a dedicated hardware-based system that does.

Corresponding structures, materials, acts, and equivalents of all means or steps and functional elements that may be found in the claims below integrate any structure, material, or act to perform a function. Is intended to be included.

In one or more embodiments, the system can communicate to the host processor through the communication bus with a host processor connected to the communication bus and a first hardware accelerator linked to the host processor so that it can communicate through the communication bus. It may include a second hardware accelerator linked to. The first hardware accelerator and the second hardware accelerator are directly connected through an accelerator link independent of the communication bus. The host processor is configured to initiate data transfer between the first hardware accelerator and the second hardware accelerator directly through the accelerator link.

In one aspect, the host processor is configured to communicate with the first hardware accelerator and the second hardware accelerator via the communication bus.

In another aspect, data transfer involves the first hardware accelerator accessing the memory of the second hardware accelerator through the accelerator link.

In another aspect, the host processor is configured to access the memory of the second hardware accelerator by sending data containing the target address to the first hardware accelerator, where the target address is the second hardware. Translated by the host processor to correspond to the accelerator, the first hardware accelerator initiates a transaction to access the memory of the second hardware accelerator over the accelerator link based on the target address.

In another aspect, the second hardware accelerator decrements the target address for data transfer by the upper limit of the address range for the second hardware accelerator in response to receiving a transaction over the accelerator link. It is set to divide and determine if the decremented target address is local.

In another aspect, the host processor receives data between the first hardware accelerator and the second hardware accelerator based on the status of the direct memory access circuit of the second hardware accelerator connected to the communication bus. Set to start the transfer.

In another aspect, the host processor is configured to automatically determine the sequence of the first hardware accelerator and the second hardware accelerator in the ring topology.

In another aspect, the host processor is configured to use the remote buffer flag to track the buffers corresponding to the first hardware accelerator and the second hardware accelerator.

In one or more embodiments, the hardware accelerator is an endpoint configured to communicate with the host processor over a communication bus, a memory controller connected to memory local to the hardware accelerator, and an endpoint. And a link circuit connected to the memory controller. The link circuit is configured to establish an accelerator link with a target hardware accelerator that is also connected to the communication bus. Accelerator links are direct connections between a hardware accelerator and a target hardware accelerator, independent of the communication bus.

The link circuit is configured to initiate data transfer with the target hardware accelerator over the accelerator link, which is in response to instructions from the host processor received by the hardware accelerator over the communication bus. Will be done.

In another aspect, the link circuit comprises a first memory map-stream mapper circuit and a second memory map-stream mapper circuit, each of which translates the data stream into a memory-mapped transaction and memory-mapped. It is set to convert the executed transaction to a data stream.

In another aspect, each memory map-stream mapper circuit is configured to decrement the target address in received transactions by the upper bound of the hardware accelerator's address range.

In another aspect, the link circuit is a first transceiver configured to send and receive stream data, and a first re-connected to the first transceiver and the first memory map-stream mapper circuit. Including transmission engine.

In another aspect, the link circuit is a second transceiver configured to send and receive stream data and a second re-connected to the second transceiver and the second memory map-stream mapper circuit. Further includes a transmission engine.

In one or more embodiments, the method is to receive an instruction sent from a host processor over a communication bus and a target address for data transfer within the first hardware accelerator, and the first. In response to the hardware accelerator comparing the target address with the upper bound of the address range corresponding to the first hardware accelerator and determining that the target address exceeds the address range based on the comparison. A first hardware accelerator performs a transaction with a second hardware accelerator in order to perform a data transfer using an accelerator link that directly connects the first hardware accelerator and the second hardware accelerator. It can include starting.

In another aspect, the accelerator link is independent of the communication bus.

In another aspect, initiating a transaction involves initiating a memory-mapped transaction and converting the memory-mapped transaction into a data stream that should be sent over an accelerator link.

In another aspect, the method subtracts the upper bound of the address range of the second hardware accelerator from the target address in response to receiving a transaction on the second hardware accelerator. This includes determining if the result of the subtraction is within the address range of the second hardware accelerator.

In another aspect, the second hardware accelerator receives the transaction as a data stream and converts the data stream into a memory-mapped transaction.

In another aspect, the method determines the status of the direct memory access circuit of the second hardware accelerator and initiates data transfer in response to the status of the direct memory access circuit of the second hardware accelerator. And include.

The description of the configuration of the present invention provided herein is for illustration purposes only and is not exhaustive or limited to the disclosed forms and examples. The terminology used herein is to describe the principles of construction of the invention, practical applications, or technical improvements to the techniques found on the market, and / or by others skilled in the art. Selected to allow understanding of the constitution of the invention disclosed in the document. Modifications and modifications may be apparent to those skilled in the art without departing from the scope and gist of the present invention described. Therefore, references should be made to the following claims, not to the above disclosure, as an indication of the scope of such features and implementations.

Claims (15)

With the host processor connected to the communication bus
A first hardware accelerator that is communicably linked to the host processor through the communication bus,
A system comprising a second hardware accelerator that is communicably linked to the host processor through the communication bus.
The first hardware accelerator and the second hardware accelerator are directly connected through an accelerator link independent of the communication bus.
A system in which the host processor is configured to initiate data transfer between the first hardware accelerator and the second hardware accelerator directly through the accelerator link. The system of claim 1, wherein the data transfer comprises the first hardware accelerator accessing the memory of the second hardware accelerator through the accelerator link. The host processor is set to access the memory of the second hardware accelerator by sending data including a target address to the first hardware accelerator, and the target address is the second hardware. Converted by the host processor to correspond to a hardware accelerator so that the first hardware accelerator accesses the memory of the second hardware accelerator via the accelerator link based on the target address. 2. The system of claim 2, which initiates a transaction of. The second hardware accelerator decrements the target address for the data transfer by the upper limit of the address range for the second hardware accelerator in response to receiving a transaction over the accelerator link. The system of claim 1, wherein the system is configured to do and determine if the deducted target address is local. The host processor is said to be between the first hardware accelerator and the second hardware accelerator based on the status of the direct memory access circuit of the second hardware accelerator connected to the communication bus. The system of claim 1, configured to initiate data transfer. The system of claim 1, wherein the host processor is configured to automatically determine the sequence of the first hardware accelerator and the second hardware accelerator in a ring topology. The system of claim 1, wherein the host processor is configured to use the remote buffer flag to track the buffers corresponding to the first hardware accelerator and the second hardware accelerator. It ’s an integrated circuit,
With endpoints configured to communicate with the host processor over the communication bus,
A memory controller connected to a memory local to the integrated circuit,
It comprises a link circuit connected to the endpoint and the memory controller, the link circuit is set to establish an accelerator link with a target hardware accelerator also connected to the communication bus, and the accelerator link. Is an integrated circuit that is independent of the communication bus and is a direct connection between the integrated circuit and the target hardware accelerator. The link circuit is configured to initiate data transfer to and from the target hardware accelerator over the accelerator link, and the data transfer is from the host processor received by the integrated circuit via the communication bus. 8. The integrated circuit according to claim 8, which is performed in response to the instruction of. The integrated circuit according to claim 8, wherein the target hardware accelerator is set to decrement the target address in a transaction received from the integrated circuit by an upper limit of the address range of the integrated circuit. In the first hardware accelerator, receiving the instruction sent from the host processor via the communication bus and the target address for data transfer, and
The first hardware accelerator compares the target address with the upper limit of the address range corresponding to the first hardware accelerator.
In response to the determination that the target address exceeds the address range based on the comparison, the first hardware accelerator causes the first hardware accelerator and the second hardware accelerator to move. A method comprising initiating a transaction with said second hardware accelerator to perform a data transfer using a directly connected accelerator link. 11. The method of claim 11, wherein the accelerator link is independent of the communication bus. In response to receiving the transaction in the second hardware accelerator, the second hardware accelerator subtracts the upper limit of the address range of the second hardware accelerator from the target address. 11. The method of claim 11, further comprising determining if the result of the subtraction is within said address range of the second hardware accelerator. Determining the status of the direct memory access circuit of the second hardware accelerator
11. The method of claim 11, further comprising initiating the data transfer in response to the status of the direct memory access circuit of the second hardware accelerator. The method of claim 11, wherein the data transfer comprises the first hardware accelerator accessing the memory of the second hardware accelerator through the accelerator link.

Images