JP2022139737A - Transmission device, receiving device, control method, and program - Google Patents

Abstract

To detect a communication failure and appropriately insert a recovery code when real-time communication of a code string obtained by encoding time-series information for each time-series processing unit is performed.SOLUTION: When a receiving side control signal indicating the normal state is received, normal state processing of transmitting a first code obtained by encoding time-series information of a current processing unit using information of the processing unit other than the current processing unit, and a transmission side control signal representing the normal state is executed, and when a receiving side control signal indicating the abnormal state is received, after transmitting a second code obtained by encoding the time-series information of the current processing unit without using the information of the processing unit other than the current processing unit, and a transmitting side control signal indicating that the second code has been transmitted, the abnormal state processing of transmitting the first code is further executed.SELECTED DRAWING: Figure 1

Description

The present invention relates to a technique for controlling transmission and reception of code strings obtained by encoding time-series information for each time-series processing unit.

In recent years, a dedicated frequency band (unmanned mobile image transmission system) has been established for video transmission from unmanned mobile objects such as robots and drones. Transmission devices are entering the market. In terms of the surrounding environment, imaging devices such as cameras can output ultra-high-definition video such as 4K resolution (3840 x 2160), and video codecs include H.264/AVC and H.264. By adopting a highly efficient compression method such as H.265/HEVC, wireless transmission of high-definition video, which requires a huge transmission capacity, has become possible. In addition, for the purpose of FPV (First Person View) of drones and remote control applications such as construction equipment and agricultural equipment, it is widely deployed in the market, such as those compatible with low delay and real-time transmission.

H. In video codecs such as H.264/265, compression/decompression processing is performed according to GOP (Group of Picture). In other words, an operation (LongGOP) is adopted in which the GOP size is increased by increasing the ratio of P pictures (Predictive Pictures) and B pictures (Bidirectionally Predictive Pictures). By the way, B pictures have a high compression rate and therefore contribute to high image quality at a low rate. As mentioned above, in systems where real-time performance is important, an operation method such as not using B-pictures (IPPP...) is adopted. A transmission method (P Only) in which a picture is configured as a picture (forward prediction picture) is also adopted.

H. 264/265 video codec, the transmitter (encoder) generates a fixed-length Mpeg-TS packet (188 bytes) and stores video data and data required for synchronization with the receiving side. Split and insert. The generated packet is transmitted to a receiver (decoder) using, for example, a radio system using OFDM modulated waves used in television broadcasting or an IP communication network, and the received data is decoded at the receiver, and video output/display is performed. It is something to do. In the case of IP transmission, the connection-oriented TCP/IP method is adopted when securing reliability is important, but in systems where real-time performance is emphasized, connectionless UDP-based systems are widely used. is deployed.

When transmitting a video signal, a transmission method is employed in which the three primary color components of light, R, G, and B, are not directly handled but converted into a luminance component Y and color difference components Cb and Cr. 2. Description of the Related Art In recent years, the number of video equipment such as cameras that support 4K UHD (Ultra High Definition), which is defined as ultra-high definition video, has increased. The 4K UHD video signal uses a YCbCr4:2:2 format in which the number of pixels for the color difference signal is reduced to 1/2 compared to the luminance signal, a video format with a color bit depth of 10 Bit and a frame frequency of 60 Hz (progressive progressive scanning method). is commonly used. For example, if an attempt is made to transmit (or store) this video without compression, a huge transmission rate is required. The actual required rate can be calculated by the following formula.
<3840 (pixels)×2160 (number of lines)×10 (bit depth)×2 (pixel ratio of color information)×60 (frame frequency)>.
Judging from this calculation result, it can be seen that approximately 10 Gbps is required.

However, when considering wireless transmission from mobile objects such as drone aerial photography, it is difficult to realize a transmission line that enables such high-speed communication. expected to become Therefore, a system in which data is compressed using a codec to transmit the video is generally used. Also, in recent video cameras, the H.264 standard is used. A codec such as H.264 or H.265 is built in, and video data can be stored while undergoing compression processing. The storage method is not limited to high-compression using ordinary IPB pictures, but there is also a method that saves all pictures as I (ALL Intra method) in consideration of superiority during material processing and editing. However, since I pictures have a larger amount of data than P and B pictures that contain only differential components, for example, if the bit rate is 100 Mbps when saving in the LongGOP format, the ALL Intra format will be 400 Mbps, so I pictures are used. It can be confirmed that a very large bit rate difference occurs by doing so. When the same image is represented by I-pictures, a higher bit rate is required compared to P-pictures. .

When building a real-time video transmission system, it is common to use a video codec that does not use LongGOP and does not use B-pictures (IPPP, etc.). There are issues related to Compression processing and decompression processing are performed according to the GOP, but since the P picture holds only difference information from the previous frame, it cannot be expressed as a single image with only the picture information. Information based on I-pictures (keyframes) or past I-pictures and P-pictures must be referenced. In other words, if some picture information is lost due to communication problems such as packet loss during video transmission, problems occur in the reference information, causing problems such as distortion of the displayed video after decoding and generation of block noise. Image failure is expected. In this case, the problem is not resolved until the next I-picture is inserted, and depending on the situation, there is a possibility that the image will continue to be corrupted for a long period of time. In order to solve such a problem, a process of inserting an I picture into a frame sequence when a failure occurs is called video refresh.

However, frequently inserting I-pictures with a large amount of data in order to solve such a problem puts pressure on the transmission line, and is not recommended for wireless transmission, which is particularly subject to severe bit rate restrictions. In situations where bitrate improvement cannot be expected, there remains a concern that a significant drop in image quality will occur. In fact, the difference in image quality between the I-picture and the P-pictures before and after it may appear remarkably, and such a state is generally expressed as a pulsing effect. Therefore, in order to avoid such a problem, it is desired to increase the GOP size and reduce the insertion of I pictures as much as possible.

On the other hand, Patent Literature 1 discloses a method of avoiding video interruption by sending a retransmission request to the transmitting side when a transmission error is detected on the receiving side and having the transmitting side preferentially transmit I-pictures. It is

Japanese Patent Application Laid-Open No. 2004-328022

However, the method of Patent Document 1 is based on the premise that a connection is established between the receiving side and the transmitting side, and a retransmission request from the receiving side is reliably transmitted to the transmitting side. It cannot be used for connectionless real-time communication where the signal is not guaranteed to be transmitted to the sender.

Such problems are not limited to connectionless real-time communication in which video time-series information is compressed and decompressed according to GOP. In the case of performing real-time communication of the obtained code string, a method of encoding time-series information of the current processing unit without using information of processing units other than the current processing unit, and a method of encoding time-series information of processing units other than the current processing unit This is common when a method of encoding is adopted in combination with a method of encoding time-series information of the current processing unit using information.

The present invention has been made in view of such a point, and when performing real-time communication of a code string obtained by encoding time-series information for each time-series processing unit, a processing unit other than the current processing unit A method that encodes the time-series information of the current processing unit without using the information of the current processing unit and a method that encodes the time-series information of the current processing unit using the information of the processing unit other than the current processing unit. It is an object of the present invention to provide a technique for detecting a communication failure and appropriately inserting a code for recovery when a method of performing encoding by means of data is adopted.

A transmitting device that transmits a code string obtained by encoding time-series information for each time-series processing unit, when receiving a receiving-side control signal indicating a normal state, encodes information of a processing unit other than the current processing unit. Execute normal processing for transmitting the first code obtained by encoding the time-series information of the current processing unit using and a transmitting-side control signal representing a normal state, and receive a receiving-side control signal representing an abnormal state In this case, the second code obtained by encoding the time-series information of the current processing unit without using the information of the processing units other than the current processing unit and the transmitting-side control signal indicating that the second code has been transmitted After the transmission, the abnormal processing for transmitting the first code is executed.

A receiving device that receives a code string, which is a string of codes for each time-series processing unit, determines whether or not received information has been received normally when receiving a transmitting-side control signal representing a normal state. If it is determined that the received information has been received normally, the first code of the current processing unit included in the received information is decoded using the information of the processing unit other than the current processing unit, and the normal state If it is determined that the received information has not been received normally, the receiving side control signal indicating an abnormal state is sent, and a processing unit other than the current processing unit is executed. The transmitting side determines whether or not a transmitting side control signal indicating that the second code of the current processing unit that can be decoded without using the information of has been transmitted has been received, and indicates that the second code has been transmitted When it is determined that the control signal has been received, the second code is decoded, and abnormal processing is executed to transmit a receiving-side control signal representing a normal state.

In the present invention, a transmitting side control signal representing a normal state, a receiving side control signal representing a normal state, a receiving side control signal representing an abnormal state, and a second code are transmitted between a transmitting apparatus and a receiving apparatus. Therefore, even if any of the control signals cannot be reliably transmitted, appropriate recovery processing can be performed according to the communication state. Therefore, when performing real-time communication of a code string obtained by encoding time-series information for each time-series processing unit, time-series information for the current processing unit is not used without using information for processing units other than the current processing unit. and a method of encoding the time-series information of the current processing unit using information of a processing unit other than the current processing unit. It is possible to detect a communication failure and appropriately insert a recovery code (second code).

FIG. 1 is a diagram illustrating the functional configuration of a wireless system according to an embodiment. FIG. 2 is a diagram exemplifying the functional configuration of the wireless transmitter of the embodiment. FIG. 3 is a diagram exemplifying the functional configuration of the wireless receiver of the embodiment. 4A to 4D are diagrams exemplifying the data structure of TS packets transmitted over a video line. FIG. 5A is a state transition diagram of the transmitter, and FIG. 5B is a state transition diagram of the receiver. FIG. 6 is a flow diagram to illustrate the processing of the wireless transmitter and wireless receiver. FIG. 7 is a flow diagram to illustrate the processing of the wireless transmitter and wireless receiver. FIG. 8 is a flow diagram to illustrate the processing of the wireless transmitter and wireless receiver. FIG. 9 is a flow diagram to illustrate the processing of the wireless transmitter and wireless receiver. FIG. 10 is a flow diagram to illustrate the processing of the wireless transmitter and wireless receiver. FIG. 11 is a flow diagram to illustrate the processing of the wireless transmitter and wireless receiver. FIG. 12 is a flow diagram to illustrate the processing of the wireless transmitter and wireless receiver. FIG. 13 is a flow diagram to illustrate the processing of the wireless transmitter and wireless receiver. FIG. 14 is a flow diagram to illustrate the processing of the wireless transmitter and wireless receiver.

Embodiments of the present invention will be described below.
[First embodiment]
<Configuration>
As illustrated in FIG. 1 , the radio system 1 of this embodiment has a radio transmitter 11 , a radio receiver 12 and an external control device 13 . The radio transmitter 11 has a transmitting device 111 , an imaging device 112 , a transmitting antenna 113 and a control line antenna 114 . The radio receiver 12 has a receiving device 121 , a display device 122 , a receiving antenna 123 and a control line antenna 124 . As illustrated in FIG. 2, the transmission device 111 of this embodiment has a video encoder 111a, a video line wireless transmission section 111b, a control section 111c, and a control line wireless reception section 111d. As illustrated in FIG. 3, the receiver 121 of this embodiment has a video decoder 121a, a video line wireless receiver 121b, a controller 121c, and a control line wireless transmitter 121d.

The transmitting device 111 and the receiving device 121 include, for example, a communication device, a processor (hardware processor) such as a CPU (central processing unit) that executes a predetermined program, and RAM (random-access memory)/ROM (read-only memory). It is configured by a general-purpose or dedicated computer equipped with a memory such as memory). The imaging device 112 is a device such as a camera that acquires video information. The display device 122 is a device such as a display, a projector, or VR goggles. The external control device 13 is configured by, for example, a general-purpose or dedicated computer including a processor for executing a predetermined program and memories such as RAM and ROM.

The wireless system 1 is a system that wirelessly transmits a video signal input from an imaging device 112 from a wireless transmitter 11 mounted on a moving object such as a drone to a wireless receiver 12 installed on the ground or the like. . A video line used for video transmission is assumed to be, for example, an unmanned mobile image transmission system. However, this does not limit the present invention, and a code string obtained by the wireless transmitter 11 encoding a time-series signal (for example, a time-series video signal) for each time-series processing unit (for example, a frame) , and the radio receiver 12 receives and decodes the code string transmitted from the radio transmitter 11 . Further, in this embodiment, an example in which a code string is transmitted from the radio transmitter 11 to the radio receiver 12 in real time will be described, but the present invention is not limited to this, and the code string is transmitted in real time. It does not have to be done in real time.

<Overview of processing>
Next, the outline of the processing of this embodiment is shown using FIGS. 1, 5A, and 5B.
The wireless transmitter 11 compresses and encodes the time-series video signal (transmission video data) input from the imaging device 112 for each frame (time-series processing unit) to generate a code string that is a string of codes for each frame. to generate In this embodiment, it is assumed that compression/decompression processing is performed based on GOP. In other words, the codes forming the code string of the present embodiment are assumed to be composed of I pictures (second code) or P (or B) pictures (first code). The radio transmitter 11 transmits (radio-transmits) a radio signal including this code string to the radio receiver 12 through the video line. The radio receiver 12 decompresses and decodes code string data included in the transmitted radio signal (received information), and outputs the resulting decoded time-series video signal to the display device 122 . The display device 122 displays the video represented by the decoded time-series video signal. The radio receiver 12 analyzes the radio signal (received information) when performing data decompression and decoding processing of a code string contained in the transmitted radio signal, and detects transmission errors (destruction of transmitted data) during radio transmission. , omission, etc.) has occurred. When a transmission error during wireless transmission is detected as a result of this error determination (when it is determined that the received information was not received normally), the wireless receiver 12 requests the wireless transmitter 11 to insert an I picture. to send out. In the present embodiment, it is assumed that all of these processes are performed fully automatically in consideration of convenience. However, this does not limit the invention, and some processing may be performed based on the input processing.

In such a framework, the radio transmitter 11 transmits a control signal (transmitting side control signal, hereinafter referred to as "original data Tx") for checking the state of the radio receiver 12 periodically or based on a predetermined trigger. The radio signal containing the image is transmitted to the radio receiver 12 through the video line. The radio receiver 12 transmits a control signal (receiving side control signal, hereinafter referred to as "original data Rx") based on the received original data Tx and the error determination result described above to the radio transmitter 11 through the control line.

《Example of original data Tx》
The original data Tx of this embodiment are illustrated. The original data Tx of this embodiment are the following three kinds of codes "00", "11", and "10".
(1) 00: Original data Tx representing normal operation or recovery from normal operation (sending-side control signal representing normal state)
(2) 11: Original data Tx representing that an I picture (key frame) has been transmitted (transmitting side control signal representing that a second code has been transmitted)
(3) 10: Original data Tx indicating waiting state after transmission of I-picture (transmitting-side control signal indicating waiting state)
However, these do not limit the present invention, and any codes may be assigned to the above (1) to (3).

《Example of original data Rx》
The original data Rx of this embodiment are illustrated. The original data Rx of this embodiment are the following two types of codes "00" and "01".
(1) 00: Original data Rx representing normal operation or recovery from normal operation (receiving side control signal representing normal state)
(2) 01: Original data Rx indicating that a transmission error has been detected (receiving side control signal indicating an abnormal state)
However, these do not limit the present invention, and any codes may be assigned to the above (1) and (2).

<<Example of State Transition of Radio Transmitter 11>>
The state transition of the wireless transmitter 11 is illustrated using FIG. 5A.
After starting the wireless transmitter 11, until receiving the original data Rx "01" (original data Rx indicating that the transmitted transmission error was detected: receiving side control signal indicating an abnormal state) from the wireless receiver 12, The wireless transmitter 11 always transmits the original data Tx "00" (original data Tx representing normal operation or recovery from normal operation: receiving side control signal representing normal state) to the wireless receiver 12 through the video line ( status “00”). At this time, for example, immediately after the radio transmitter 11 is activated, an I picture (second code) is transmitted together with the original data Tx "00", and then a P (or B) picture (first code) is transmitted with the original data Tx". 00”. Transmission with status "00" is performed periodically, for example (step S111).

When the radio transmitter 11 receives the original data Rx "01" (original data Rx indicating that a transmitted transmission error has been detected: a receiving-side control signal indicating an abnormal state) from the radio receiver 12, this triggers , the radio transmitter 11 transmits the original data Tx "11" (original data Tx indicating that the I picture has been transmitted: a transmitting-side control signal indicating that the second code has been transmitted) together with the I picture (second code) to the video image. It is transmitted to the wireless receiver 12 through the line (status "11"). This transmission of the I picture and the original data Tx “11” is performed only once each time the radio transmitter 11 receives the original data Rx “01” from the radio receiver 12, for example. However, this is not a limitation of the present invention, and the transmission of the I-picture and original data Tx "11" is performed only α times each time the radio transmitter 11 receives the original data Rx "01" from the radio receiver 12. may be done. However, α is an integer of 2 or more. The value of α may be determined in advance, or may be determined according to the communication status between the wireless transmitter 11 and the wireless receiver 12 . For example, the worse the communication situation between the wireless transmitter 11 and the wireless receiver 12 (for example, the lower the electric field strength of the received signal), the larger the value of α (step S112).

After transmitting the I picture and the original data Tx "11", the radio transmitter 11 sends the P (or B) picture (first code) together with the original data Tx "10" (which indicates a waiting state after transmitting the I picture). original data Tx: a transmission-side control signal indicating a waiting state) is transmitted to the wireless receiver 12 via the video line (status "10"). Transmission of this original data Tx "10" is transmitted together with, for example, a P (or B) picture. For transmission with status “10”, original data Rx “00” (original data Rx indicating normal operation or normal operation return: control signal on the receiving side indicating normal state) or original data Rx “01” (transmission error occurred). This is performed until the original data Rx representing the detection (receiving side control signal representing an abnormal state) is received. Transmission with status "10" is performed periodically, for example (step S113).

In status "10", if the original data Rx "01" (original data Rx indicating that a transmission error has been detected: a receiving side control signal indicating an abnormal state) is transmitted to the wireless transmitter 11 via the control line, the status Return to "11" (step S114). On the other hand, in status "10", the original data Rx "00" (original data Rx representing normal operation or normal operation recovery: receiving side control signal representing normal state) was transmitted to the wireless transmitter 11 through the control line. If the radio transmitter 11 receives a receiver control signal representing a normal state, it returns to status "00" and the radio transmitter 11, for example, together with the P (or B) picture (first code), returns the original Data Tx "00" (original data Tx representing normal operation or normal operation recovery: receiving side control signal representing normal state) is transmitted to the radio receiver 12 through the video line (step S115).

<<Example of state transition of the wireless receiver 12>>
FIG. 5B is used to illustrate state transitions of the wireless receiver 12 .
After the wireless receiver 12 is activated, the wireless receiver 12 becomes “Null”, which is a state of waiting for reception from the wireless transmitter 11 . When the wireless receiver 12 receives the original data Tx “00” from the wireless transmitter 11 (original data Tx representing normal operation or recovery from normal operation: transmitting-side control signal representing normal state), the wireless receiver 12 transmits wireless Determine whether or not a transmission error has occurred during transmission (execute determination processing for determining whether or not received information has been received normally). If no transmission error during wireless transmission is detected as a result of this error determination (if it is determined that the reception information has been received normally), the wireless receiver 12 outputs the original data Rx "00" (during normal operation or returning to normal operation). Original data Rx representing time (receiving side control signal representing normal state) is transmitted to the radio transmitter 11 through the control line (status "00"). This state is a normal state, and the display device 122 displays the video represented by the decoded time-series video signal. Also, the wireless receiver 12, for example, decodes the received P (or B) picture (first code) (step S121).

On the other hand, when a transmission error during wireless transmission is detected as the error determination result for "Null" (when it is determined that the reception information was not received normally), the wireless receiver 12 outputs the original data Rx "01" (transmission Original data Rx representing that an error has been detected (receiving side control signal representing an abnormal state) is transmitted to the radio transmitter 11 through the control line (status "01"). This state is an abnormal state, and is a state in which an I-picture insertion request is issued to the radio transmitter 11 (step S122).

When the wireless receiver 12 receives the original data Tx "00" from the wireless transmitter 11 in status "00", the wireless receiver 12 determines whether or not a transmission error has occurred during wireless transmission. If no transmission error during wireless transmission is detected as a result of this error determination, the wireless receiver 12 transmits the original data Rx "00" to the wireless transmitter 11 via the control line (status "00") (step S123). ).

On the other hand, when a transmission error during wireless transmission is detected as an error determination result with status "00", the wireless receiver 12 transmits the original data Rx "01" to the wireless transmitter 11 through the control line (status "00"). 01") (step S124).

In the status "01", the wireless receiver 12 transmits the original data Tx "11" (original data Tx indicating that the I-picture has been transmitted: the transmitting-side control signal indicating that the second code has been transmitted) from the wireless transmitter 11. Determine whether or not it has been received. Here, if the wireless receiver 12 has received the original data Tx "11" from the wireless transmitter 11, the wireless receiver 12 determines whether the I picture (second code) has been correctly received. Here, if the wireless receiver 12 could not correctly receive the I-picture, the wireless receiver 12 again receives the original data Rx "01" (original data Rx indicating that a transmission error was detected: abnormal state is transmitted to the wireless transmitter 11 through the control line (status "01"). On the other hand, if the wireless receiver 12 receives the original data Tx "10" before receiving the original data Tx "11" with the status "01", it is determined that the I-picture was not correctly received, and wireless reception is performed. The device 12 again transmits the original data Rx "01" to the wireless transmitter 11 through the control line (status "01"). As a result, the wireless transmitter 11 is again requested to insert an I picture (step S125).

In the status "01", if the wireless receiver 12 correctly received the original data Tx and the I picture from the wireless transmitter 11, the wireless receiver 12 transmits the original data Rx "00" to the wireless transmitter 11 through the control line. Send to In this case, the radio receiver 12 decodes the I picture (status "00") (step S126).

<Details of processing>
Next, the details of the processing of this embodiment will be illustrated with reference to FIGS. 1 to 14. FIG.
<<Details of Processing of Radio Transmitter 11>>
First, the details of the processing of the wireless transmitter 11 will be illustrated. First, a time-series video signal (time-series signal) captured by the imaging device 112 is input to the transmission device 111 (FIG. 2). Inside the transmission device 111, the time-series video signal is input to the video encoder 111a. The video encoder 111a packetizes the time-series video signal for each frame (time-series processing unit), performs encoding compression processing on each packet, obtains a code corresponding to each frame, and generates a string of these codes. output a code string that is When the video encoder 111a packetizes the time-series video signal, it inserts the original data Tx into the packet periodically or at a predetermined opportunity. The timing of inserting the original data Tx is as described above. The encoding compression method of the video encoder 111a is not limited. MPEG2-TS packet format such as H.264/265, P-frames that can be operated in consideration of real-time performance are frequently used, and LongGOP format is supported. The code string is input to the video line wireless transmission unit 111b. The video line wireless transmission unit 111b performs video line encoding and modulation processing on the code string, and outputs the resulting wireless signal from the transmission antenna 113 for wireless transmission. Although the encoding and modulation processing are not limited, for example, Reed-Solomon codes and OFDM modulated waves used in television broadcasting are assumed. The control unit 111c executes each control described above based on the original data Rx transmitted from the wireless receiver 12 through the control line. The control line is for transmitting and receiving control signals. The control line is connected to the control section 111c through the control line antenna 114 and the control line radio reception section 111d. The control unit 111c acquires data from the radio signal received by the control line radio reception unit 111d through the control line (for example, obtains original data Rx), generates data to be wirelessly transmitted to the radio receiver 12 through the control line, and performs video encoder 111a, video circuit wireless transmission unit 111b, and control line wireless reception unit 111d. Although there is no particular limitation on the control line, for example, a frequency band different from that of the video line can be used in an unmanned mobile image transmission system.

Details of the processing of the control unit 111c are illustrated using FIG. 5A.
After starting the wireless transmitter 11, the control unit 111c outputs the original data Tx "00" to the video encoder 111a until the original data Rx "01" is received through the control line, and transmits the original data Tx "00" to the video encoder 111a. It instructs the generation of a code of a signal including an I picture or a P (or B) picture, and generates and outputs a radio signal by video line encoding and modulation processing for a code string composed of the code to the video line wireless transmission unit 111b. to direct. That is, the control unit 111c causes the video encoder 111a and the video line wireless transmission unit 111b to generate a wireless signal corresponding to a code string composed of a signal containing original data Tx "00" and an I picture or P (or B) picture. and output (status "00") (step S111).

When the control unit 111c receives the original data Rx “01”, the control unit 111c outputs the original data Tx “11” to the video encoder 111a, and sends I An instruction is given to generate and output a radio signal corresponding to a code string consisting of codes including pictures and original data Tx "11" (status "11") (step S112).

After transmitting the I picture and the original data Tx "11", the control unit 111c outputs the original data Tx "10" to the video encoder 111a, and transmits the P (or B) picture and An instruction is given to generate and output a radio signal corresponding to a code string consisting of codes containing original data Tx "10" (status "10") (step S113).

When the control unit 111c receives the original data Rx "01" in status "10", the control unit 111c returns the process to status "11" (step S114). On the other hand, when the control unit 111c receives the original data Rx "00" in status "10", the control unit 111c returns the process to status "00" (step S115).

In the series of processes described above, for example, the control unit 111c holds the most recent original data Tx, and the control unit 111c generates (updates) the original data Tx only when new original data Rx is received. (that is, the original data Tx is generated when new original data Rx is received). For example, the original data Tx representing the insertion of an I picture is updated from "00" to "11" and then to "10" because the control unit 111c receives new original data Rx "01". only when Furthermore, the original data Tx is updated from "10" to "00" only when the control unit 111c receives new original data Rx "00".

FIG. 4A to FIG. 4D illustrate the encoding compression processing in the video encoder 111a and the packet configuration of each target frame. The data structure illustrated here is an MPEG2-TS packet. As illustrated in FIG. 4A, the TS packet 100 is, for example, a 188-byte packet and has a packet header 101 and an adaptation field 102 or payload 103 of, for example, 4 bytes. As illustrated in FIG. 4B, packet header 101 includes sync byte 101a, transport error indicator 101b, payload unit start indicator 101c, transport priority 101d, packet identifier 101e, payload scrambling control 101f, adaptation field control 101g, and It has a continuity counter 101f. As illustrated in FIG. 4C, the adaptation field 102 includes packet length specification 102a, discontinuity indicator 102b, random access indicator 102c, stream priority 102d, options field flags 102e, options field 102f (variable field length), and stuffing. It has 102g. As illustrated in FIG. 4D, option field 102f has PCR 102fa, OPCR 102fb, splice countdown 102fc, private data 102fd, and extension field 102fe.

As described above, in MPEG2-TS packets, time information such as PCR (Program Clock Reference) 102fa may be inserted in order to synchronize transmission and reception and appropriately reproduce video. This PCR102fa is stipulated to be inserted, for example, once every 100 ms. As described above, the TS packet 100 is configured such that, of the total length of 188 bytes, the first 4 bytes are the packet header 101a, and the remaining 184 bytes are the payload 103 and the adaptation field 102, or both. That is, the payload 103 may consist of a maximum of 184 bytes. By the way, the payload 103 is the contents of the transmission data, and in this system, it is the area into which the time-series video signal is inserted. On the other hand, the adaptation field 102 is an area into which the aforementioned PCR 102fa, private data 102fd, etc. are inserted. In order to enable maximum transmission as a payload, for example, the original data Tx shall be inserted in the adaptation field 102 (for example, in the private data 102fd), and the generation timing of the data is the timing when the PCR 102fa is inserted. Optimization is possible by matching. As described above, the area required for the original data Tx is 2 bits (less than 1 byte), and since the effect is limited to TS packets in which the adaptation field 102 exists, the effect on the line is slight.

<<Details of the processing of the wireless receiver 12>>
Next, the details of the processing of the radio receiver 12 will be illustrated. A radio signal transmitted from the radio transmitter 11 through the video line is received by the receiving antenna 123 and sent to the video line radio receiving section 121b. The video line radio receiver 121b performs demodulation processing, error correction decoding processing, and the like on the input radio signal, restores it to a packet code string, and sends it to the video decoder 121a. The video decoder 121 a performs data decompression and decoding processing on the packet code string to obtain packets, extracts a decoded time-series video signal from the packet, and outputs the decoded time-series video signal to the display device 122 . The display device 122 displays the video represented by the decoded time-series video signal. This image can be displayed in real time. Also, the original data Tx obtained from the packet is sent to the control unit 121c. As described above, the control line transmits and receives control signals. The control line is connected to the control section 121c through the control line antenna 124 and the control line radio transmission section 121d. The control unit 121c acquires data from the radio signal received through the control line, generates data to be wirelessly transmitted to the radio transmitter 11 through the control line, the video decoder 121a, the video circuit radio reception unit 121b, and the control line radio transmission unit 121d. can be controlled. The control unit 121c is also connected to the external control device 13, and based on control from the external control device 13, can control the receiving device 121 and control the transmitting device 111 by remote control.

Details of the processing of the control unit 121c are illustrated using FIG. 5B.
In "Null", when the control unit 121c receives the original data Tx "00", the control unit 121c determines whether or not a transmission error has occurred during wireless transmission. The details of this processing will be described later. If no transmission error during wireless transmission is detected as a result of this error determination (if the control unit 121c determines that the reception information has been received normally), the control unit 121c transmits the original data Rx “00” to the control circuit wirelessly. section 121d to cause the control circuit wireless transmission section 121d to transmit the original data Rx "00". In this case, the display device 122 normally displays the video represented by the decoded time-series video signal (status “00”) (step S121).

On the other hand, when a transmission error during wireless transmission is detected as the error determination result of "Null" (when the control unit 121c determines that the reception information was not received normally), the control unit 121c outputs the error determination result and the original data Rx "01" to the control circuit wireless transmission unit 121d to cause the control circuit wireless transmission unit 121d to transmit the original data Rx "01". In this case, the display device 122 is expected to generate image interruptions, block noise, and the like in the image represented by the decoded time-series image signal. Therefore, it is necessary to transmit an I-picture insertion request as described above (status "01") (step S122).

When the control unit 121c receives the original data Tx "00" in status "00", the control unit 121c determines whether or not a transmission error has occurred during wireless transmission. The details of this processing will be described later. If no transmission error during wireless transmission is detected as a result of this error determination (if the control unit 121c determines that the reception information has been received normally), the control unit 121c outputs this error determination result and the original data Rx"00". ” to the control line wireless transmission unit 121d, and causes the control line wireless transmission unit 121d to transmit the original data Rx “00” (status “00”) (step S123).

On the other hand, when a transmission error during wireless transmission is detected as an error determination result with status “00” (when the control unit 121c determines that the received information was not received normally), the control unit 121c makes this error determination. The result and the original data Rx "01" are output to the control line wireless transmission unit 121d, and the control line wireless transmission unit 121d is caused to transmit the original data Rx "01" (status "01") (step S124).

For status "01", the controller 121c determines whether or not the original data Tx "11" has been obtained. Here, if the control unit 121c has acquired the original data Tx "11", the control unit 121c determines whether the video line radio reception unit 121b has correctly received the I picture. The control unit 121c, for example, receives a status representing the state of each process from the video line radio reception unit 121b and makes this determination. Here, if the video line wireless reception unit 121b could not correctly receive the I-picture, the control unit 121c again outputs the original data Rx "01" to the control line wireless transmission unit 121d for control line wireless transmission. It causes the unit 121d to transmit the original data Rx "01" (status "01"). On the other hand, if the video line radio reception unit 121b receives the original data Tx "10" before receiving the original data Tx "11" with the status "01", the control unit 121c determines that the I picture could not be received correctly. The control unit 121c again outputs the original data Rx “01” to the control line radio transmission unit 121d, and causes the control line radio transmission unit 121d to transmit the original data Rx “01” (status “01”). ) (step S125).

If the original data Tx and the I-picture have been correctly received in the status "01", the control unit 121c outputs the original data Rx "00" to the control line wireless transmission unit 121d, and transmits the original data to the control line wireless transmission unit 121d. Data Rx "00" is transmitted (status "00") (step S126).

A method for determining whether or not a transmission error has occurred during wireless transmission will be exemplified.
The control unit 121c receives the status indicating the state of each process from the video line radio reception unit 121b and the video decoder 121a, and determines whether or not a transmission error has occurred during radio transmission, for example, as follows.
(a) The control unit 121c determines that a transmission error has occurred when, for example, the packet header 101 of the TS packet 100 is abnormal, lost, or has lost continuity. For example, the control unit 121c determines that a transmission error has occurred when loss of the synchronization byte 101a or lack of continuity in the counter value of the continuity counter 101f occurs.
(b) The control unit 121c determines that a transmission error has occurred when, for example, the adaptation field 102 is abnormal, lost, or lacks continuity. For example, the control unit 121c determines that a transmission error has occurred when an abnormality, lack of continuity, or the like occurs in the original data Tx. For example, the original data Tx transitions as shown in FIG. 5A, but if the original data Tx does not transition as shown in FIG. 5A, it is determined that a transmission error has occurred. That is, the control unit 121c controls the original data Tx "00" (transmitting side control signal indicating normal state), the original data Tx "11" (transmitting side control signal indicating that the second code has been transmitted), and the original data Whether or not a transmission error has occurred (whether or not the reception information has been received normally) may be determined based on the reception order of Tx "10" (transmitting side control signal indicating waiting state). . For example, as illustrated in FIG. 5A, the original data Tx never transitions from "00" to "10". If received, it may be determined that a transmission error has occurred. Similarly, when α=1, the original data Tx “11” is followed by the original data Tx “10”. If received, it may be determined that a transmission error has occurred.
(c) The control unit 121c generates a transmission error when an error is detected in error correction decoding processing (for example, error correction using the Viterbi algorithm, Reed-Solomon code, etc.) in the video line radio reception unit 121b. It is determined that

《Regarding transmission interval of original data》
When determining whether or not a transmission error has occurred based on the original data Tx transmitted from the wireless transmitter 11 to the wireless receiver 12, it is necessary to shorten the transmission interval of the original data Tx to some extent. This does not apply to original data Rx that is not used for any purpose. Also, in general, control lines often have a narrower transmission band than video lines. Therefore, when determining whether or not a transmission error has occurred based on the original data Tx, the transmission interval (average transmission interval) of the original data Tx should be shorter than the transmission interval (average transmission interval) of the original data Rx. may For example, the original data Tx may be transmitted every 100 ms on average, while the original data Rx may be transmitted every 1 s on average. This shall be adjusted according to the operating system. When a video failure occurs, it is desirable to send an I-picture as soon as possible to refresh the video, but in general, when a transmission error occurs, it may take some time until the video line is restored, so video refresh requests are frequently issued. This is not desirable because it places a strain on the video line.

In order to alleviate the tightness of the video line, the transmission interval of the original data Tx may be varied according to the importance of the original data Tx. For example, it is not necessarily the case that a transmission error has occurred when transmitting original data Tx "00" (transmitting side control signal representing a normal state), but original data Tx "11" (representing that the second code has been transmitted) Transmission-side control signal) or original data Tx "10" (transmission-side control signal indicating a waiting state) is transmitted, and a transmission error has occurred, so it is necessary to immediately transmit the I picture. Therefore, after transmitting original data Tx "11" (transmitting side control signal indicating that the second code has been transmitted), original data Tx "10" (transmitting side control signal indicating that it is in a waiting state) is transmitted. and/or the transmission interval of the original data Tx "10" (transmitting side control signal representing the waiting state) to that of the original data Tx "00" (transmitting side control signal representing the normal state) It may be shorter than the transmission interval.

《Isolation judgment of error mode》
In this embodiment, three types of codes "00", "11", and "10" are selectively used as the original data Tx, so the wireless receiver 12 can distinguish where the problem is occurring. Therefore, the radio receiver 12 may discriminate and determine the following error modes.
(1) When the radio receiver 12 receives the original data Tx "11" (transmitting side control signal indicating that the second code has been transmitted), but does not correctly receive the I picture (second code) error mode 1
In this case, it is suspected that there is a packet loss that occurred during transmission of the I-picture from the wireless transmitter 11, or that the video encoder 111a or the video decoder 121a is faulty.
(2) Before the radio receiver 12 receives the original data Tx "11" (transmitting-side control signal indicating that the second code has been transmitted), the original data Tx "10" (indicating a waiting state) Error mode 2 when receiving a control signal on the sending side
In this case, it is suspected that a packet loss occurred during transmission of the I-picture from the wireless transmitter 11 or that the video line is unstable.
(3) Original data Tx “00” (transmission-side control signal indicating normal state) even if the original data Rx “01” (receiving-side control signal indicating an abnormal state) is transmitted in the abnormal processing, and then waits for a certain period of time. Error mode 3 when receiving
In this case, it is suspected that the original data Rx "01" has not arrived in the abnormal processing and the control line is unstable.
(4) If the original data Rx “01” (receiving-side control signal indicating an abnormal state) is transmitted in the abnormal processing, and the I-picture (second code) cannot be received correctly even after waiting for a certain period of time (a transmission error occurs). error mode 4 when detection continues)
In this case, it is suspected that the video line is unstable, or that both the video line and the control line are unstable.
Note that the radio receiver 12 may distinguish between all of the error modes 1 to 4, or may distinguish only some of them from the other error modes.

<Specific example of processing>
A specific example of the processing of this embodiment is shown below.
<<Specific example 1>>
FIG. 6 illustrates processing when no transmission error occurs.
After starting the radio transmitter 11, the control unit 111c of the radio transmitter 11 outputs the original data Tx "00" (step S1), and the video encoder 111a and the video line radio transmission unit 111b convert the original data Tx "00" and I A radio signal corresponding to a code string composed of a signal including a picture or a P (or B) picture is transmitted to the radio receiver 12 through a video line (step S2).

The radio receiver 12 receives the radio signal and determines whether or not a transmission error has occurred during the radio transmission (step S3). As a result of this error determination, no transmission error during wireless transmission is detected, and the image is normally displayed. The control unit 121c sends the error determination result and the original data Rx "00" to the control circuit wireless transmission unit 121d, and causes the control circuit wireless transmission unit 121d to transmit the original data Rx "00" (step S4).

The control circuit radio transmission section 121d transmits the original data Rx "00" to the radio transmitter 11 through the control line. The wireless transmitter 11 determines that normal operation is restored as a result of the response (step S5).

The wireless transmitter 11 judges that the response result is normal operation, and as long as the normal operation continues after that, the processing of steps S1 to S5 is repeated.

<<Specific example 2>>
FIG. 7 exemplifies processing when video is refreshed immediately after the occurrence of a transmission error (a case in which restoration is completed immediately with a single refresh request).
After starting the radio transmitter 11, the control unit 111c of the radio transmitter 11 outputs the original data Tx "00" (step S1), and the video encoder 111a and the video line radio transmission unit 111b convert the original data Tx "00" and I A radio signal corresponding to a code string composed of a signal including a picture or a P (or B) picture is transmitted to the radio receiver 12 through a video line. However, it is assumed that a transmission error occurs during this transmission (step S2').

The radio receiver 12 receives the radio signal and determines whether or not a transmission error has occurred during the radio transmission (step S3). As a result of this error determination, a transmission error during wireless transmission is detected, and there is a possibility that an image failure will occur. The control unit 121c sends the error determination result and the original data Rx "01" to the control circuit wireless transmission unit 121d, and causes the control circuit wireless transmission unit 121d to transmit the original data Rx "01" (step S4').

The control circuit radio transmission unit 121d transmits the original data Rx "01" to the radio transmitter 11 through the control line (step S5').

When the radio transmitter 11 receives the original data Rx "01", the controller 111c outputs the original data Tx "11" to the video encoder 111a (step S6), assuming that there is an I-picture insertion request as a response. The video encoder 111a and the video wireless channel transmission unit 111b are made to generate and transmit a wireless signal corresponding to a code string composed of codes including the I picture and the original data Tx "11" (step S7).

The radio receiver 12 receives the radio signal and determines whether or not a transmission error has occurred during the radio transmission (step S8). As a result of this error determination, no transmission error during wireless transmission is detected, and the image is normally displayed. The control unit 121c sends the error determination result and the original data Rx "00" to the control circuit wireless transmission unit 121d, and causes the control circuit wireless transmission unit 121d to transmit the original data Rx "00" (step S9).

The control circuit radio transmission unit 121d transmits the original data Rx "00" to the radio transmitter 11 through the control line (step S10).

The wireless transmitter 11 judges that the response result is normal operation, and then executes the processing from step S1.

<<Specific example 3>>
FIGS. 8 to 10 illustrate processing when the video line is unstable and it takes time to insert/restore the I picture.
As illustrated in FIG. 8, steps S1, S2', S3, S4', S5', and S6 are executed after the wireless transmitter 11 is activated. The control unit 111c of the radio transmitter 11 causes the video encoder 111a and the video radio line transmission unit 111b to generate and transmit a radio signal corresponding to the code string including the I picture and the original data Tx "11". However, it is assumed that a transmission error occurs during this transmission (step S7').

The radio receiver 12 receives the radio signal and determines whether or not a transmission error has occurred during the radio transmission (step S8). Assume that a transmission error during wireless transmission is detected as a result of this error determination, and insertion of an I picture is not confirmed. After that, steps S4', S5', S6 and S7' are executed. It is also assumed here that a transmission error has occurred during the transmission of the radio signal in step S7'. After that, the same processing is repeated without the transmission error being resolved.

Thereafter, as illustrated in FIGS. 9 and 10, steps S6 to S10 are normally executed, and steps S1 to S5 are normally executed.

<<Specific example 4>>
FIGS. 11 to 14 illustrate processing when the control line is unstable and it is difficult for the radio transmitter 11 to confirm recovery even after the I-picture is inserted.
As illustrated in FIG. 11, after the radio transmitter 11 is activated, the processes of steps S1, S2, S3, and S4 are executed. The control circuit radio transmission section 121d of the radio receiver 12 transmits the original data Rx "00" to the radio transmitter 11 through the control line. However, it is assumed that a transmission error occurs during this transmission (step S5'').

After that, the processing of steps S1 to S5 is executed, and then the processing of step S1 is further executed as illustrated in FIG. Next, the video encoder 111a and the video line wireless transmission unit 111b of the wireless transmitter 11 transmit a wireless signal corresponding to a code string composed of a signal containing original data Tx "00" and an I picture or P (or B) picture. , to the radio receiver 12 through the video line. However, it is assumed that a transmission error occurs during this transmission (step S2').

After that, the processes of steps S3 and S4' are executed, and the control circuit radio transmission section 121d of the radio receiver 12 transmits the original data Rx "01" to the radio transmitter 11 through the control line. However, it is assumed that a transmission error occurs during this transmission (step S5''').

After that, the processing of steps S1, S2, S3, S4' and S5''' is repeated without the transmission error being resolved.

After that, the transmission error in S5''' is eliminated, and the processes of steps S5', S6 to S9 are executed as illustrated in FIG. , the original data Rx "00" is transmitted to the radio transmitter 11. However, it is assumed that a transmission error occurs during this transmission (step S10').

The wireless transmitter 11 determines that there is no response as a result of the response, and the controller 111c of the wireless transmitter 11 outputs the original data Tx "10" to the video encoder 111a (step S11). 111b to generate and transmit a radio signal corresponding to a code string consisting of codes containing P (or B) pictures and original data Tx "10" (step S12).

After that, the transmission error in step S10' is not resolved, and the processing of steps S8, S9, and S10' is repeated.

After that, as illustrated in FIG. 14, the transmission error in step S10' is resolved, the process of step S5 is executed, and the processes of steps S1 to S5 are executed.

<Characteristics of this embodiment>
In this embodiment, when a wireless signal is transmitted from the wireless transmitter 11 to the wireless receiver 12 and when the image is refreshed, the original data Tx corresponding to the situation is transmitted from the wireless transmitter 11 to the wireless receiver 12, and the situation The original data Rx according to is transmitted from the radio receiver 12 to the radio transmitter 11 . That is, when the radio transmitter 11 receives the original data Rx "00", it executes normal processing to transmit the P (or B) picture and the original data Tx "00", and transmits the original data Rx "01". is received, the I picture and the original data Tx "11" are transmitted, and then the P (or B) picture is transmitted. Further, when the wireless receiver 12 receives the original data Tx “00”, it executes determination processing for determining whether or not the reception information has been received normally. Then, the P (or B) picture is decoded and the original data Rx "00" is transmitted. If it is determined that the reception information was not received normally, the original data Tx "11" is received. If it is determined that the original data Tx "11" has been received, the I-picture is decoded and the abnormal processing is executed to transmit the original data Rx "00". By executing the processing based on the original data Tx and Rx in this way, even if any of the original data cannot be reliably transmitted, appropriate recovery processing can be performed according to the communication state. can. Therefore, when performing real-time communication of a code string obtained by encoding video time-series information for each frame based on GOP, it is possible to detect a communication failure and insert an I-picture for recovery (video refresh) appropriately and automatically. can be done systematically.

In order to ensure the reliability of data exchange between transmitters and receivers, a method of acknowledging receipt and requesting retransmission by ACK, such as the TCP/IP method, is usually adopted. However, in a situation where the line conditions change drastically due to wireless transmission and retransmission requests are generated many times, there remains a concern that the line will be overloaded and the delay time will increase significantly. On the other hand, the method of the present embodiment can be applied to a method such as the UDP method, in which the wireless transmitter 11 and the wireless receiver 12 each transmit necessary information in a streaming manner regardless of the status of the connection destination. Therefore, even if the communication situation becomes unstable in either or both of the wireless transmitter 11 and the wireless receiver 12, an appropriate response can be obtained without straining the line or significantly increasing the delay time. Video refresh can be automatically performed with timing.

Further, as illustrated in FIG. 5A, when the radio transmitter 11 receives the original data Rx “01” from the radio receiver 12, the radio transmitter 11 transmits the I picture and the original data Tx “11” α times (for example, once ) (status “11”), and then transmit P (or B) pictures and original data Tx “10” (status "Ten"). Also, as illustrated in FIG. 5B, when the wireless receiver 12 determines that it has received the original data Rx “01” before receiving the original data Tx “11”, it is possible to properly receive the I picture. Original data Rx “01” is transmitted assuming that there is no such data. As a result, the radio receiver 12 can determine that the I-picture was not properly received with only the original data Tx, and can request retransmission of the I-picture. Further, the wireless receiver 12 performs a video refresh process in which a P (or B) picture and original data Tx "10" are transmitted, and a normal process in which a P (or B) picture and original data Tx "00" are transmitted. state can be distinguished.

Further, as described above, the worse the communication situation between the wireless transmitter 11 and the wireless receiver 12, the larger the value of α. If the communication conditions between the wireless transmitter 11 and the wireless receiver 12 are bad, there is a high possibility that the I-picture will not be correctly received by the wireless receiver 12 . By increasing the value of α in such a case, the effect of reducing the number of redoing video refresh can be expected.

Also, the wireless receiver 12 may determine whether or not a transmission error has occurred based on the order of reception of the original data Tx "00", the original data Tx "11", and the original data Tx "10". . As a result, the original data itself can be used for transmission error detection, and even though each original data is 2 bits, it is possible to judge the presence or absence of a transmission error by viewing them as a series. As a result, transmission error detection accuracy can be improved without increasing transmission data.

Further, as described above, when determining whether or not a transmission error has occurred based on the original data Tx, the transmission interval (average transmission interval) of the original data Tx is replaced by the transmission interval (average transmission interval) of the original data Rx. ). As a result, tightness of the control line can be alleviated.

Further, as described above, the transmission interval from when the wireless transmitter 11 transmits the original data Tx “11” to when it transmits the original data Tx “10” and/or the transmission interval of the original data Tx “10” may be shorter than the transmission interval of the original data Tx "00". As a result, it is possible to reduce the congestion of the video line without significantly delaying the video refresh.

Further, as described above, the radio receiver 12 may discriminate between at least one of the error modes 1 to 4 and determine the transmission error. This allows the radio receiver 12 to distinguish where a problem is occurring.

[Second embodiment]
Although the first embodiment was directed to transmission of video time-series information, this can be generalized to transmission of time-series information. That is, this time-series information may be video time-series information as exemplified in the first embodiment, audio time-series information, biological time-series information, It may be seismic waveform time-series information, or other physical time-series information.

Although the first embodiment exemplifies a GOP-based system as an encoding system, the present invention is not limited to this. That is, any encoding method may be applied as long as it is a method for encoding time-series information for each time-series processing unit (for example, frame). That is, the P (or B) picture is the first code obtained by encoding the time-series information of the current processing unit using the information of the processing unit other than the current processing unit (for example, the processing unit other than the current processing unit). code based on the difference between the unit information and the time-series information of the current processing unit). The I-picture is a second code obtained by encoding time-series information of the current processing unit without using information of processing units other than the current processing unit (for example, time-series information (only) of the current processing unit). can be generalized to a code obtained by encoding alone).

[Other modifications, etc.]
It should be noted that the present invention is not limited to the above-described embodiments. For example, the various types of processing described above may not only be executed in chronological order according to the description, but may also be executed in parallel or individually according to the processing capacity of the device that executes the processing or as necessary. In addition, it goes without saying that appropriate modifications are possible without departing from the gist of the present invention.

As described above, the transmitting device 111 and the receiving device 121 are configured by, for example, a communication device, a general-purpose or dedicated computer including a processor for executing a predetermined program and memories such as RAM and ROM. This computer may have a single processor and memory, or may have multiple processors and memories. This program may be installed in the computer, or may be recorded in ROM or the like in advance. Also, some or all of the processing units are configured using an electronic circuit that realizes processing functions without using a program, rather than an electronic circuit that realizes a functional configuration by reading a program like a CPU. may An electronic circuit that constitutes one device may include a plurality of CPUs.

When the transmitting device 111 and the receiving device 121 described above are realized by a communication device and a computer, the processing contents of the functions that each device should have are described by a program. By executing this program on a computer, the above processing functions are realized on the computer. A program describing the contents of this processing can be recorded in a computer-readable recording medium. An example of a computer-readable recording medium is a non-transitory recording medium. Examples of such recording media are magnetic recording devices, optical discs, magneto-optical recording media, semiconductor memories, and the like.

The distribution of this program is carried out, for example, by selling, assigning, lending, etc. portable recording media such as DVDs and CD-ROMs on which the program is recorded. Further, the program may be distributed by storing the program in the storage device of the server computer and transferring the program from the server computer to other computers via the network.

A computer that executes such a program, for example, first stores the program recorded on a portable recording medium or the program transferred from the server computer once in its own storage device. When executing the process, this computer reads the program stored in its own storage device and executes the process according to the read program. As another form of execution of this program, the computer may directly read the program from a portable recording medium and execute processing according to the program. , may sequentially execute processing according to the received program. A configuration in which the above processing is executed by a so-called ASP (Application Service Provider) type service, which does not transfer the program from the server computer to this computer and realizes the processing function only by the execution instruction and result acquisition, is also possible. good.

At least a part of these processing functions may be realized by hardware instead of executing a predetermined program on a computer to realize the processing functions of the present apparatus.

In the present invention, for example, images from an image pickup device mounted on a moving object, such as distribution of aerial images by a drone, are output from a wireless device, received by a fixed receiving station deployed on the ground, etc., and real-time image transmission It can be used for a system that realizes As a result, even if the received image is interrupted due to an instantaneous interruption of radio waves or the like, or if the image is disturbed due to block noise or the like, the image can be quickly restored. However, the invention is not limited to this field of application.

Reference Signs List 1 wireless system 11 wireless transmitter 111 transmitter 111a video encoder 111b video line wireless transmitter 111c, 121c controller 111d control line wireless receiver 112 imaging device 113 transmission antenna 114, 124 control line Antenna 12 Radio receiver 121 Receiver 121a Video decoder 121b Video line radio receiver 121d Control line radio transmitter

Claims (11)

A transmission device that transmits a code string obtained by encoding time-series information for each time-series processing unit,
A first code obtained by encoding the time-series information of the current processing unit using information of a processing unit other than the current processing unit when a receiving-side control signal representing a normal state is received, and representing the normal state. Executes normal processing to transmit a transmission side control signal,
a second code obtained by encoding the time-series information of the current processing unit without using information of a processing unit other than the current processing unit when a receiving-side control signal representing an abnormal state is received; and a transmitting side control signal indicating that the code has been transmitted, and then executing an abnormal time process of further transmitting the first code. The transmitting device of claim 1,
In the abnormal processing, when receiving the receiving-side control signal indicating the abnormal state, after transmitting the second code and the transmitting-side control signal indicating that the second code has been transmitted α times, transmitting the first code and a transmitting control signal representing a waiting state until receiving the receiving control signal representing the normal state or the receiving control signal representing the abnormal state, wherein α is A transmitting device that is an integer of 1 or more. The transmitting device of claim 2,
The transmitting apparatus, wherein α is 1 or a larger value as the communication condition worsens. The transmitting device according to claim 2 or 3,
a transmission interval from transmission of a transmission-side control signal representing transmission of the second code to transmission of a transmission-side control signal representing the waiting state, and/or the waiting state; The transmission device, wherein a transmission interval of the transmission-side control signal representing the normal state is shorter than a transmission interval of the transmission-side control signal representing the normal state. A receiving device that receives a code string that is a string of codes for each time-series processing unit,
When receiving a transmission-side control signal indicating a normal state, executing determination processing for determining whether or not the received information has been received normally,
If it is determined that the received information has been received normally, the first code of the current processing unit included in the received information is decoded using the information of the processing unit other than the current processing unit, and the normal state is indicated. Execute normal processing to send the receiver control signal,
If it is determined that the received information has not been received normally, a receiving side control signal indicating an abnormal state is transmitted, and the current processing unit that can be decoded without using the information of the processing unit other than the current processing unit. determining whether or not a transmitting-side control signal indicating that the second code has been transmitted has been received, and if it is determined that a transmitting-side control signal indicating that the second code has been transmitted has been received, the second A receiving device that decodes a code and executes an abnormal time process of transmitting a receiving side control signal representing the normal state. The receiving device of claim 5,
The abnormal processing includes receiving a transmission-side control signal indicating a waiting state after transmission of the second code before receiving a transmission-side control signal indicating that the second code has been transmitted. The receiving device, including a process of transmitting a receiving-side control signal representing the abnormal state when determined. The receiving device of claim 6,
The determination process is performed based on the order of reception of the transmission-side control signal representing the normal state, the transmission-side control signal representing that the second code has been transmitted, and the transmission-side control signal representing the waiting state. , a receiving device including a process of determining whether or not the received information is normally received. The receiving device according to claim 6 or 7,
The determination process is
an error mode when a transmitter control signal indicating that the second code has been transmitted has been received but the second code has not been correctly received;
an error mode in which a transmitter control signal indicating the waiting state is received before receiving a transmitter control signal indicating that the second code has been transmitted;
an error mode in which the transmission-side control signal indicating the normal state is received even after waiting for a certain period of time after transmitting the receiving-side control signal indicating the abnormal state in the abnormal-time processing;
an error mode in which the second code cannot be correctly received even after waiting for a certain period of time after transmitting the receiving-side control signal representing the abnormal state in the abnormal state processing;
A receiving device including a process of distinguishing and determining at least one of A control method for a transmission device that transmits a code string obtained by encoding time-series information for each time-series processing unit,
A first code obtained by encoding the time-series information of the current processing unit using information of a processing unit other than the current processing unit when a receiving-side control signal representing a normal state is received, and representing the normal state. Executes normal processing to transmit a transmission side control signal,
a second code obtained by encoding the time-series information of the current processing unit without using information of a processing unit other than the current processing unit when a receiving-side control signal representing an abnormal state is received; A control method, further comprising transmitting a transmission-side control signal indicating that a code has been transmitted, and then executing an abnormality processing for further transmitting the first code. A control method for a receiving device that receives a code string that is a string of codes for each time-series processing unit,
When receiving a transmission-side control signal indicating a normal state, executing determination processing for determining whether or not the received information has been received normally,
If it is determined that the received information has been received normally, the first code of the current processing unit included in the received information is decoded using the information of the processing unit other than the current processing unit, and the normal state is indicated. Execute normal processing to send the receiver control signal,
If it is determined that the received information has not been received normally, a receiving side control signal indicating an abnormal state is transmitted, and the current processing unit that can be decoded without using the information of the processing unit other than the current processing unit. determining whether or not a transmitting-side control signal indicating that the second code has been transmitted has been received, and if it is determined that a transmitting-side control signal indicating that the second code has been transmitted has been received, the second A control device that decodes a code and executes an abnormality processing to transmit a receiving side control signal representing the normal state. A program for causing a computer to control the transmitting device according to any one of claims 1 to 4 or control the receiving device according to any one of claims 5 to 8.

Images