JPH0846792A - Image communication equipment and its system - Google Patents

Abstract

PURPOSE:To attain image communication optimum in image quality by coding the image while the image is inserted to a moving picture in the unit of a minimum image block whose size is smaller than an image size even when an image format in the case of coding is fixed. CONSTITUTION:An image CODEC section 11 compresses a band of raw image data of a large capacity through motion compensation, inter-frame prediction, inter-frame compensation, DC transformation and vector quantization conversion or the like and the capacity is made small so as to be sent. A multiplexer demultiplexer section 12 multiplexes a BAS from a voice data from a voice CODEC section 4 and a BAS from a system control section 14 in the unit of transmission frames. A valid/invalid designation section 18 designates whether or not a coded image block is valid or invalid, that is, a moving picture or not by a command from the control section 14 and a still picture management table 19 manages a still picture. Valid/invalid block information is managed by a valid/invalid block management table 15 and sent / received between video telephone sets prior to the transmission and reception.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image communication apparatus and system, for example, an image communication apparatus and system for transmitting and receiving image data and the like via a communication line.

[0002]

2. Description of the Related Art In the case of a conventional analog telephone line, for example, a telephone can transmit only voice, and even an image communication apparatus such as a facsimile machine can transmit image data only at a low speed.

However, with the recent advances in communication technology, semiconductor technology, and optical technology, digital lines have been established and high-speed and large-capacity data transmission has become possible.

In particular, a characteristic of digital transmission is that the same level of quality is maintained without deterioration of quality due to transmission.
Since it is possible to integrate media without requiring a transmission path according to the characteristics of the media of the transmission data, transmission between complex media terminals has become possible. Therefore, in addition to conventional telephones that only transmit audio, telephone terminals, so-called TV telephone devices, that can simultaneously transmit video have appeared.

Under these circumstances, international standardization by CCITT (International Telegraph and Telephone Consultative Committee) and the like is underway so that mutual communication is possible even between different composite terminals, and videophones and televisions using digital lines are being used. CCITT Recommendation (or Draft) H.264 provides service specifications, protocol specifications, and multimedia multiplexing frame configuration specifications for AV services as AV (Audio Visual) services such as conference systems. 320, H.M. 242, H.M. It has been announced as 221 mag.

H. In 221, from 64 Kbps to 192
Frame composition and terminal capability exchange in AV services up to 0 Kbps, and FAS (Fram Alignmen) in communication mode
t Signal) and BAS (Bit Allocation Signal) code allocation is defined. In addition, H. A protocol such as capability exchange and communication mode switching between AV terminals using BAS is defined in 242, and a system overview of AV services in general is defined in H320.

In the above recommendation (or draft recommendation),
After setting the end-to-end physical connection and establishing synchronization by FAS in the in-channel, B in the in-channel
A method for multimedia communication of images, voice, data, etc. between terminals is defined by procedures such as a sequence of terminal capability exchange using AS, a mode switching sequence by specifying a communication mode.

However, it is outside the stipulated range for each terminal to change its own terminal capability according to the situation and which communication mode is used within the range of the converted capability.

The information transfer rate of each medium in multimedia communication in which images, voices, and data are simultaneously transmitted is determined by designating a voice encoding method for voice information, and whether or not the data information is used. In this case, the transfer rate of the image information is determined by designating the transfer rate, and the remainder obtained by subtracting the transfer rate of the voice information and the transfer rate of the data information from the set information transfer rate of the entire communication path becomes the transfer rate of the image information.

[0010]

However, according to the above-mentioned conventional TV telephone apparatus, since the transmission image format for encoding is defined, the image format can be changed according to the use, for example. Can not.

For example, H.264 of ITU-TS (formerly CCITT). 261 Recommendation, CIF (Common Intermedi
ATE Format) and QCIF (Quarter CIF), there are only two standard formats, so you can only switch between two types of resolutions: normal image quality and higher-precision image quality. There is a problem that you cannot do it.

Further, when the input image format of the image to be transmitted does not match the specified format, the input image format is enlarged or reduced to match the input image format with the size of the specified format, or the received specified format and the output image are received. When the formats do not match, the specified image format must be enlarged or reduced in order to match the specified format with the output image format. Therefore, there is a problem that unnecessary scaling processing is required, or the image quality is degraded due to the scaling processing.

For example, when transmitting and receiving in a format in which a moving image is embedded in a still image and a screen is displayed, the transmitting side first transmits the still image in a prescribed format, and then transmits the moving image in a prescribed format. . That is, two frames are transmitted, and the receiving side performs processing of reducing and scaling the image of the received moving image frame and fitting it into the previously received still image to generate an image for one frame. .

Therefore, in order to solve the above-mentioned problems, the present invention provides an image communication device capable of flexibly dividing an image and transmitting and receiving a moving image of an arbitrary size more efficiently even if the image format is fixed. The purpose is to provide the system.

[0015]

In order to achieve the above-mentioned object, the present invention has the following constitution.

That is, block specifying means for specifying an effective block of image data constituting one screen in block units, encoding means for encoding the block specified by the block specifying means, and encoding by the encoding means. It is characterized by comprising an image transmitting means for transmitting the converted image data and a block information transmitting means for transmitting the block information designated by the block designating means.

Further, it is characterized in that it has a moving image input means for inputting a moving image, and the block designating means designates an area in which the moving image is fitted in one screen as an effective block.

For example, the block information transmitting means transmits block information designated as an effective block.

For example, the block information transmitting means transmits block information designated as a valid block and block information not designated as a valid block.

Further, the image receiving means for receiving the encoded image data, the decoding means for decoding the image data received by the image receiving means, the block information receiving means for receiving the block information, and the block information according to the block information. 1
It is characterized by having an image synthesizing means for synthesizing the images on the screen and a display means for displaying the image of one screen synthesized by the synthesizing means.

Further, it has a specific image data generating means for generating specific image data, the block designating means can further designate a specific image block, and the image synthesizing means designates the specific image block by the block designating means. If there is, the specific image data generated by the specific image data generating means is combined.

For example, the image synthesizing means synthesizes a moving image, a still image and a specific image.

For example, the image synthesizing means simultaneously synthesizes the received image received by the image receiving means and the transmitted image transmitted by the transmitting means as a moving image.

Further, a voice processing unit for performing voice processing, a voice encoding unit for encoding the voice data processed by the voice processing unit, and a voice for transmitting the voice data encoded by the voice encoding unit. And a transmission means.

In the image communication system including an image communication device capable of transmitting and receiving images, the first image communication device includes an image input unit for inputting an image and an effective block of image data forming one screen in block units. Block specifying means, encoding means for encoding the block specified by the block specifying means, image transmitting means for transmitting the image data encoded by the encoding means, and the block specifying means. The second image communication apparatus has block information transmitting means for transmitting the blocked block information, and the second image communication apparatus receives image data encoded by the image receiving means, and decodes the image data received by the image receiving means. Means, a block information receiving means for receiving block information, and an image for synthesizing one screen image according to the block information. And combining means, and having a display means for displaying an image of one screen synthesized by the synthesizing means.

In the image communication system including the image communication device capable of transmitting and receiving images, the first image communication device has an image input means for inputting an image, a block information receiving means for receiving block information, and the block. Encoding means for encoding the block designated by the designating means,
Image transmitting means for transmitting the image data encoded by the encoding means, and the second image communication device includes a block designating means for designating an effective block of image data forming one screen in block units; Block information transmitting means for transmitting the block information designated by the block designating means, image receiving means for receiving the image data encoded by the encoding means, and decoding for decoding the image data received by the image receiving means Means and an image synthesizing means for synthesizing an image of one screen according to the block information,
And a display unit for displaying the one-screen image combined by the combining unit.

[0027]

With the above-described structure, even if the transmission image format is fixed, it is possible to change to an optimal image format, code, and transmit the image. Further, any image format designated by the transmitting terminal or the receiving terminal can be transmitted. It is possible to efficiently encode the image of (1) and communicate, and to display only the image of the specified image format on the receiving side, which is a unique effect. With the above configuration, it is possible to more efficiently encode and transmit. In addition, it is possible to obtain a unique operation effect that a plurality of received images can be extracted and selectively displayed or simultaneously displayed.

[0028]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment according to the present invention will be described in detail below with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of the TV telephone apparatus of this embodiment. In FIG. 1, 1 is a microphone, 2 is a speaker, 3 is a gain adjusting function for volume level adjustment, and an echo is erased when the microphone 1 and the speaker 2 are used as audio input / output means according to an instruction from the system control unit 14. A voice processing unit 4 having an echo canceling function, a dial tone, a ringing tone, a busy tone, a ring tone, and other tone generating functions, and 64 kbps PCM (A-law), 64 kbps according to an instruction from the system control unit 14.
PCM (μ-law), 7 kHz audio (SB-A
DPCM), 32 kbps ADPCM, 16 kbps
(For example, APC-AB) Voice having a function of A / D converting and encoding a transmission voice signal according to a voice encoding / decoding algorithm such as 8 kbps, and a function of decoding a reception voice signal and performing D / A conversion The codec section, 5 is a camera that is equipped as a standard equipment to capture an image of a person, 6 is a monitor that displays the input image from the camera 5 and the received image from the partner device, and the image from the system control section 14, and 7 is various images An image processing unit that performs editing processing, a transmission image editing unit 8
Reference numeral 9 is a received image editing unit. The detailed configuration of the received image editing unit 9 is shown in FIG. 2 described later.

Further, in FIG. 1, reference numeral 10 is a specification for converting text data or graphic data to be inserted into an image into bitmap data based on a specific image management table in the system control section 14 and transferring it to the image processing section 7. The image generation unit 11 is an image codec unit having a function of encoding a transmission image and a function of decoding a reception image. The image codec unit 11 performs band compression on raw data of a large amount of image by various methods such as motion compensation, frame dropping, inter-frame prediction and inter-frame compensation, DCT conversion, vector quantization conversion, and the like to obtain image data. Has a smaller capacity so that it can be transmitted via a digital line. Currently IS
The basic interface of the DN line is 64 kbps, but as an image encoding method that can be transmitted at this transmission rate, H.264 of the ITU-TS recommendation is used. There is 261.

In FIG. 1, reference numeral 12 multiplexes the audio data from the audio codec unit 4, the image data from the image codec unit 11, and the BAS from the system control unit 14 for each transmission frame, and the reception frame is a structural unit. It is a demultiplexing unit that separates each medium and notifies each unit. There is 221.

A line interface 13 controls the line according to the ISDN user network interface.
14 includes a CPU, a ROM, a RAM, an auxiliary storage device, a character generator, an image signal generation circuit, and the like, monitors the state of each part of the device, controls the entire device, creates an operation / display screen according to the state, and creates an application program. A system control unit that performs execution, 15 is a valid / invalid block management table that manages the valid / invalid designating unit 18 designating valid / invalid blocks, and 16 is a specific encoding that manages specific encoded data. Data management table,
Reference numeral 17 denotes an operation unit including a keyboard, a touch panel, etc., which the operator uses to input control information to control the apparatus, and 18 denotes whether the image block to be encoded is valid or invalid according to an instruction from the system control unit 14. Is there, that is,
A valid / invalid designating section for designating whether or not it is a moving image, 19
Is a still image management table for managing still images.

Each valid / invalid block information in the transmitted / received image in this embodiment is managed in the valid / invalid block management table 15, and the information is transmitted to each TV telephone before transmitting / receiving the encoded image data. Sent and received between.

Hereinafter, referring to FIG. 2, the received image editing section 9
Will be described in detail. FIG. 2 is a block diagram showing a detailed configuration of the received image editing unit 9.

In FIG. 2, 20 is the camera 5 shown in FIG.
A transmission video RAM that stores the transmission images input from
Reference numeral 21 is a reception video RAM for accumulating the reception image decoded by the image codec unit 11, 22 is a color lookup table for converting the specific image information transferred from the specific image generation unit 10 into color information of image data, and 23 is a transmission. Video R
A video synchronization generator for generating vertical synchronization and horizontal synchronization, which are output timings of image data from the AM 20, the receiving video RAM 21, and the color lookup table 22.
Reference numeral 4 denotes a still image RAM that stores still images from the still image management table 19.

Further, the reference numeral 25 scales the image data output from the transmission video RAM 20 and the image data output from the reception video RAM 21 to a desired size in accordance with an instruction from the system control unit 14, and the image data from the transmission video RAM 20 is scaled. A synthesizing unit 26 for selecting image data, image data from the receiving video RAM 21, image data output from the color look-up table 22, or image data from the still image RAM 24 and outputting the selected data to the monitor 6, An effective area setting unit that calculates an effective image area from the effective / ineffective block management table 15 and specifies the effective area.

In the received image editing section 9, first, the specific image data from the color look-up table 22 is displayed on the entire screen, and then the transmitted video RAM 20 is displayed on the screen.
It is possible to fit and display the transmission image data from the reception image data from the reception video RAM 21.
That is, the transmitted image and the received image can be simultaneously displayed on the monitor 6. It is also possible to perform so-called superimpose display in which the specific image data from the color lookup table 22 is displayed on the transmission image data or the reception image data.

When the display areas of the transmission image data and the reception image data overlap each other, either one of the overlapping images can be selected and displayed.

The encoding process in the image codec section 11 shown in FIG. 1 will be described below with reference to FIG.
FIG. 3 is a block diagram showing a detailed configuration for performing an encoding process in the codec section 11.

In FIG. 3, reference numeral 60 is an encoding processing unit for performing an encoding process, and 61 is an instruction from the system control unit 14 for controlling each unit constituting the encoding processing unit 60 and for providing necessary information. A coding control unit that performs generation, 62 is a frame dropping processing unit that suppresses the generated code amount by thinning out image frames that are coding targets, and 63 is the same macroblock (hereinafter, MB) of the previous frame and the current frame. Calculates the mean square error between frames and the variance within a frame, and determines whether the encoding target is the difference value (INTER) between the images of the previous frame and the current frame or the original image value INTRA of the current frame. INTER / INTRA
A discriminating processing unit 64 is an MC discriminating unit for discriminating whether or not the target block of the current frame is searched within a certain range of the previous frame for pattern matching, that is, whether to perform motion compensation (MC). An MC control unit that performs MC control by instructing the presence / absence of motion compensation and a motion vector (direction, size) according to the determination result of the determination unit 64, and 66 indicates an INTER / INTRA process based on the determination result of the INTER / INTRA determination unit 63. Do the INTER
/ INTRA processing unit, 67 is a filter processing unit that performs filter processing after motion compensation, and 68 is a composite image due to accumulation of quantization error on the receiving side when INTER frames continue for the same MB, or transmission error. In case of disturbance of the
A refresh cycle counter for generating an INTRA frame in a certain period for refreshing, 69 controls a threshold value to improve the generation efficiency of data when quantized, and data less than a certain value is regarded as 0 value. , A quantization threshold control unit 70 for controlling the quantization step size according to the amount of data stored in the transmission buffer 81, and a quantization step size control unit 71 for detecting the storage amount in the transmission buffer 81. Transmission amount detection unit, 72
Is a frame header, GOB (Group of Block) header,
It is a header information generation unit that generates an MB header. Also,
Reference numeral 29 is a valid block registration unit that fetches valid block information from the valid / invalid designating unit 18 and registers the MB of the valid block.

73 is a frame memory, 74 and 75 are IN
A switch for selecting TER / INTRA processing, 76 is I
A subtracter that takes the difference between the previous frame and the current frame in the NTER processing, 77 is a DCT transform (lossy coding) unit that transforms from the spatial domain to the frequency domain by orthogonal transformation, 78 is a quantizer, and 79 is generated. V for variable length coding of data
LC (reversible encoding) section, 80 multiplexing section, 81 transmission buffer, 82 BCH section for generating error correction frame,
83 is an inverse quantizer, 84 is an inverse DCT converter, 85 is an adder that adds the previous frame and the current frame, 86 and 87 are switches that alternately switch between a read frame and a write frame, and 88 and 89 are frames. A memory 90 is an F for controlling the switches 86 and 87 and the frame memories 88 and 89.
M control unit, 91 is a motion compensation unit, 92, 9 for searching the previous frame to detect pattern matching with the current frame.
3 is a switch for selecting ON / OFF of the filter processing unit 94, 94 is a filter processing unit, and 95 is a FIFO memory 9
A switch for turning ON / OFF the input of data to 6, 96
Is a FIFO memory.

In FIG. 3, the solid line indicates the input / output of the image signal and the broken line indicates the input / output direction of the control signal.

Next, the structure of the image data of one screen (one frame) which is encoded by the above-mentioned structure shown in FIG. 3 will be described in detail with reference to FIG. FIG. 4A shows an example of the image format of one screen. Hereinafter, A pixel × B line is represented by A × B.

ITU-TS Recommendation H.264 In 261
Since there are a plurality of different standards such as NTSC, PAL, and digital television (HDTV) standards as video signals to be handled, a video signal format common to the world is adopted so that they can communicate with each other.

This is called CIF, and the number of samples is 3 for luminance Y.
52 × 288 and color differences Cr and Cb are defined by 176 × 144.

As for the sampling points (sampling points), as shown in FIG. 4B, the color difference (Cr, Cb) is equal to the four luminance points (Y1, Y2, Y3, Y4). It is set.

Further, a 1/4 format in which the CIF is halved both horizontally and vertically is called QCIF, and the number of samples is 176 × 144 for luminance Y and 88 × 72 for color differences Cr and Cb.
Is defined in.

As shown in FIG. 4A, the CIF,
Each QCIF format is composed of a plurality of GOB formats, and the GOB format is composed of 33 MB formats. Furthermore, the MB format is Y1,
Four 8 × 8 luminance blocks Y2, Y3, Y4, and 8
The color difference blocks Cr and Cb of × 8 are composed of a total of 6 blocks and have a hierarchical structure.

With the hierarchical structure of the image data shown in FIG. 4,
It becomes possible to perform encoding in MB units.

In GOB, the number of samples is 176 × luminance Y.
48, color difference Cr, Cb is defined as 88 × 24, and CIF
1/12 and 1/3 of QCIF. Also, CI
If F is composed of GOB1 to GOB12, QC
The IF is composed of GOB1, GOB3 and GOB5.

The coded image data is multiplexed in the multiplexing unit 80 shown in FIG. 3 to form a multiplexed frame structure. The frame structure of encoded image data is shown below in FIG. For convenience of description, the description will be given with the frame header (FH) added in FIG.

In FIG. 5, the upper part shows the frame structure by GOB. FH at the beginning of one frame of data
Is added, one block obtained by dividing the screen of one frame into 12 is set as GOB, and GOB1 to GOB12 are sequentially transmitted.

Each GOB forming a frame is divided as shown in the lower part of FIG. The lower part of FIG. 5 shows the detailed configuration of the FH and GOB shown in the upper part.

FH is the header information generator 7 shown in FIG.
PSC, TR, PTYP which will be generated in 2 and will be described later.
Composed of E. PSC has a frame start code,
20-bit "0000 0000 0000 000"
10000 ". TR is a frame number and uses a 5-bit value from" 1 "to" 30 ".
E is 6 bits of type information, and includes split screen instruction information, document camera instruction information, screen freeze release, and information source format instruction information (CIF, QCIF).

The GOB header is also included in the header information generation unit 7
GBSC, GN, GQU, which will be generated in step 2 and will be described later.
It is composed of ANT. GBSC is a GOB start code and is 16-bit "0000 0000 0000."
0001 ". GN is a GOB number, and 4-bit values from" 1 "to" 12 "are used. GN is" 0 ".
In this case, since it is used as the FSC PSC, the FH PSC and GOB GBSC + GN are both 2
It can be regarded as a continuous value of 0 bits. Also, G
QUANT is quantization characteristic information, which includes 5-bit quantization step size information Q determined by the quantization step size control unit 70 shown in FIG.

An MB header is also generated in the header information generation unit 72, and MBA, MTYPE, MQUAN described later are generated.
It is composed of T, MVD, and CBP. The MBA is a macroblock address indicating the position of the MB, only the first MB among the 33 MBs forming one GOB is an absolute value, and the subsequent MBs are variable length codes of the difference. MTYPE is the type information of MB, and INTRA
(Intra-frame encoding), INTER (Inter-frame differential encoding), MC (Motion compensation with inter-frame differential encoding),
The processing type applied to the data of the MB, such as FIL (filter), is shown. MQUANT is quantization characteristic information,
It has the same information as TGQUANT described above. MVD is motion vector information V. CBP is a significant block pattern in which the difference value is significant, and includes, as information, the number of the pixel block that is valid among the four Y blocks and Cr and Cb that compose the MB. In the CBP, among the blocks constituting the MB, the numbers 1 to 4 are assigned to Y, the numbers 5 to Cb, and the numbers 6 to Cr.

After the MB header, compression-coded image data TCOEFF follows. In the TCOEFF, as described above, the pixel block designated as the significant block in the CBP among the Y4 pieces and the Cr and Cb is compressed. At the end of the pixel block, EO
B is added.

The image data encoded as described above is added with an error correction frame bit in the BCH 82 and is actually transmitted on the line as an error correction frame. The structure of this error correction frame will be described with reference to FIG.

In FIG. 6, in one frame, the error correction frame bit Sn indicating the presence or absence of error correction is 1 bit, the fill identifier Fi indicating whether it is moving image data is 1 bit, and BCH encoding is performed. The image data is composed of 492 bits and the error correction parity is composed of 18 bits of 512 bits. Furthermore, 8 frames form one multi-frame.

The image compression method described above is based on ITU-
H. of TS. It has been recommended as H.261, and if it complies with this recommendation, mutual communication with other TV phones that comply with the recommendation becomes possible.

The encoding method in this embodiment will be briefly described below. In the present embodiment, efficient coding is performed by combining the following methods.

Taking advantage of the fact that the correlation between pixels is strong in the image of the natural world, that the frequency components are concentrated in the low frequency and the high frequency is small, the data in the frame is made into an 8 × 8 block, and the data is two-dimensional. Intra-frame coding method for DCT conversion.

An interframe coding method for obtaining a difference between frames when image blocks at the same position of the previous frame and the current frame are strongly correlated with each other, and performing a two-dimensional DCT conversion on an 8 × 8 block with respect to the difference value. .

When a similar image block relatively moves from the previous frame to the current frame, it is detected and only the information on the moving amount and moving direction of the image block is sent, and the image data itself is not sent. Motion compensation that reduces the amount of generated data by completing.

A zero run length coding method utilizing the fact that the coefficient value for each frequency after DCT transformation has a value in the low frequency region, but it hardly occurs in the high frequency region and continues to have a zero value.

A quantization method in which the data generation amount is adjusted by changing the data quantization step width in accordance with the data generation amount.

By assigning a short code value to a data pattern having a high frequency of occurrence and a long code value to a data pattern having a low frequency of occurrence, it is converted into a data amount smaller than the total generated data amount. Variable-length coding method.

A frame dropping method of skipping a frame and dropping the image data itself.

Image data is compressed by effectively using the plurality of encoding methods described above, and moving images can be communicated even in low rate communication.

Next, a method of designating valid / invalid blocks in this embodiment will be described with reference to FIG. Figure 7
(A) shows an example in which a valid / invalid block is designated for each MB in the CIF format. In FIG. 7A, a solid frame indicates an invalid block, a dotted frame indicates an invalid block in which specific image data exists, and a black-painted frame indicates a valid block. In addition, (b) of FIG.
An example in which a valid / invalid block is designated for each MB in the format is shown.

Next, FIG. 8 shows a specific example of the above-mentioned valid / invalid block on the display screen.

FIG. 8A shows a screen displayed on the monitor 6, and a range effective on the screen, that is, a range handled as a moving image is shown by a broken line frame. The inside of the broken line frame is an effective block on the screen, and the outside of the frame is an invalid block.

FIG. 8B shows an example in which a plurality of valid block ranges are designated as the first valid block and the second valid block. When a plurality of areas are to be encoded as a moving image in this way, encoding efficiency can be improved by designating and encoding a plurality of effective ranges.

FIG. 8C shows a case where the effective range is designated by a plurality of levels. In FIG. 8 (c), the first frame is a valid block, the range between the first frame and the second frame is a block in which specific image data of the invalid block is fitted, and the outermost frame. That is, the outside of the second frame is an invalid block.

In the present embodiment, for the invalid block, a mode in which image data is first transmitted and thereafter is not transmitted, and a mode in which image data is not transmitted from the beginning are prepared, and the operation unit 17 can arbitrarily You can switch.

Next, an example of the contents of the valid / invalid block management table 15 shown in FIG. 1 will be described with reference to FIGS. 9 and 10.

FIG. 9 shows an example of the contents of the valid / invalid block management table 15 applied to the CIF format image shown in FIG. 7A. In FIG. 9, the MB number is G
It is specified and managed by OB No., STARTM No. and ENDMB No.

GOB No. indicates the GOB sequential number (1 to 12) in the CIF1 screen shown in FIG. 4A. STARMB No. is the GO indicated by GOB No.
In B, MB number (1
˜33), and ENDMBNo. Is the MB number (1 to 33) indicating the end of the valid block. That is, ST
MB between ARTMBNo. And ENDMBNo.
Is a valid block. Therefore, (b) and (c) of FIG.
Even if a plurality of effective ranges and levels are specified as shown in, for example, a plurality of STARMB Nos. And ENs in the same GOB can be specified.
It becomes possible by specifying the DMB No. The valid / invalid block management table 15 shown in FIG. 9 also holds the presence / absence of specific image data and, if a specific image is to be displayed, its code information. Therefore, it is possible to simultaneously manage whether or not the specific image data is displayed in an overlapping manner, and also the code information of the specific image data at the time of displaying.

FIG. 10 shows an example of the contents of the valid / invalid block management table 15 applied to the image in the QCIF format shown in FIG. 7B.

In FIG. 10, GOB No. indicates the GOB sequential number (1, 3, 5) in the QCIF1 screen shown in (a) of FIG. 4 described above. STARMB No., END
The MB No. is the same as in FIG. 9 described above. Note that, in FIG. 10, an example in which the memory area used by the valid / invalid block management table 15 is reduced by treating a block that is an invalid block and does not display a specific image as “other block” Show.

An image communication system using the TV phone of this embodiment will be described below with reference to FIG. FIG.
FIG. 1 is a diagram schematically showing a state of communication of image data between a transmission terminal and a reception terminal (between TV phones) and its valid / invalid block information in the image communication system of the present embodiment.

FIG. 11A shows a valid / invalid block management table 1 in a transmitting terminal for transmitting image data.
An example of transferring the information of No. 5 to the receiving terminal is shown.

The transmitting terminal encodes only the image data of the valid block, first transmits the valid / invalid block information stored in the valid / invalid block management table 15, and then codes the encoded image data. To send.

The receiving terminal receives the valid / invalid block information transmitted from the transmitting terminal and the encoded image data, decodes it in the image codec section 11, and writes it in the receiving video RAM 21. Then, when only the moving image area is to be displayed on the monitor, moving image data of the block designated as valid is extracted by referring to the valid / invalid block information, and still image or specific image data is extracted except for this. Insert and display on the monitor. Of course, the designated area on the monitor when it is displayed on the monitor can be freely set on the receiving terminal side.

Also, although an example in which valid / invalid block information is transmitted from the transmitting terminal has been described, only valid block information may be transmitted, for example.

Next, FIG. 11B shows an example in which the receiving terminal creates valid / invalid block information and transfers it to the transmitting terminal, and only the valid block is transferred from the transmitting terminal. First, the receiving terminal sets a moving image (effective) block to be encoded, and writes specific image data or still image data in the other area. Then, only the valid block information is transmitted to the transmitting terminal side.

The transmitting terminal receives the valid block information sent from the receiving terminal, sets the valid block to be encoded in the transmitted image based on the valid block information, and sets the moving image data input from the camera in the valid block area. Inset. Then, only the moving image data is encoded and transmitted to the receiving terminal.

The receiving terminal receives the encoded image data, decodes it, and recognizes the moving image area from the effective block information created by itself. If only the moving image area is to be displayed on the monitor, moving image data is extracted from the effective block area and displayed on the monitor. Of course, the designated area on the monitor when it is displayed on the monitor can be freely set on the receiving terminal side.

The receiving terminal can simultaneously display the still image, the specific image, the received moving image, and the transmitted image transmitted by itself on one screen as shown in FIG. is there.

In this embodiment, the areas of the moving image, the still image and the specific image are distinguished by designating and managing the valid / invalid block range as described above. That is, by performing the encoding process as moving image data on the range designated as valid and not encoding the block designated as invalid as the still image, it is possible to optimize the format of any image size. Encoding enables efficient image communication.

In this embodiment, a TV phone capable of processing voice information and image information as shown in FIG. 1 has been described as an example, but the present invention is limited to such a TV phone. However, any device, method, and system for communicating moving images may be used.

The present invention may be applied to a system composed of a plurality of devices or an apparatus composed of one device. Further, it goes without saying that the present invention can be applied to the case where it is achieved by supplying a program to a system or an apparatus.

In the above-described embodiments of the present invention, the video conference or the video telephone for communicating the moving image data has been described as an example, but the present invention is not limited to the moving image, and the same applies to the case of transmitting a still image. Can be applied.

[0094]

As described above, according to the present invention, even if the image format at the time of encoding is fixed, the moving image is embedded in a size smaller than the image size in units of the smallest image block and encoded. By performing the encoding, the moving image can be encoded and transmitted in an arbitrary size. Therefore, it is possible to change the image size according to the application and the situation and execute the optimal encoding process, and obtain a unique effect that image communication can be performed with the optimal image quality.

Further, the coding process is not performed for the invalid block, and only the minimum necessary information such as the identification information of the image block is transmitted, so that the processing in the image codec section is also reduced and the amount is reduced. Can be assigned to the processing of effective blocks, and the circuit scale can be reduced. Therefore, it is possible to change the image size according to the application and the situation and execute the optimum encoding process, and there is an effect that the improvement of the image quality can be optimized.

Further, by embedding a previously encoded specific image in the invalid block area, it is possible to transmit an image in which a moving image is embedded in the specific image.
This has the effect of dramatically improving the convenience of the operator.

Further, when displaying the moving image transmitted from the transmitting side on the still image generated by the receiving side, the receiving side designates the position where the moving image is fitted, and the transmitting side makes a moving image at the designated position. By embedding the image and transmitting it, there is no need for the receiving side to perform the embedding process and the scaling process of the moving image, so that the image communication system can be economically constructed and the image quality can be obtained in good condition. is there.

[0098]

[Brief description of drawings]

FIG. 1 is a block configuration diagram of a TV phone according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a detailed configuration of an image processing unit in the present invention.

FIG. 3 is a block diagram showing a detailed configuration of an image codec unit in the present embodiment.

FIG. 4 is a diagram showing an image format in this embodiment.

FIG. 5 is a diagram showing a multiplexed frame structure of image data in the present embodiment.

FIG. 6 is a diagram showing error correction frame synchronization in the present embodiment.

FIG. 7 is a diagram showing an example of designation of valid / invalid blocks in block units according to the present embodiment.

FIG. 8 is a diagram showing an example of designation of valid / invalid blocks on a screen in the present embodiment.

FIG. 9 is a diagram showing an example of a CIF valid / invalid block management table in the present embodiment.

FIG. 10 is a diagram showing an example of a QCIF valid / invalid block management table in the present embodiment.

FIG. 11 is a diagram showing an example of a communication method in the present embodiment.

[Explanation of symbols]

 1 Microphone 2 Speaker 3 Audio processing unit 4 Audio codec unit 5 Camera 6 Monitor 7 Image processing unit 8 Transmitted image editing unit 9 Received image editing unit 10 Specific image generating unit 11 Image codec unit 12 Multiplexing / demultiplexing unit 13 Line interface unit 14 System control Part 15 Valid / invalid block management table 16 Specific encoded data management table 17 Operation part 18 Valid / invalid designating part 19 Still image management table

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Office reference number FI technical display location H04B 14/04 Z H04N 5/265 5/907 B 5/937 7/08 7/081 7/32 7/18 A H04N 7/08 Z 7/137 Z

Claims (11)

[Claims] 1. A block designating unit that designates, in block units, an effective block of image data that configures one screen, an encoding unit that encodes the block designated by the block designating unit, and a code by the encoding unit. An image communication apparatus comprising: an image transmitting unit for transmitting the converted image data; and a block information transmitting unit for transmitting the block information designated by the block designating unit. 2. A moving image input means for inputting a moving image is further provided, and the block designating means designates an area in which the moving image is fitted in one screen as an effective block. Image communication device. 3. The image communication apparatus according to claim 1, wherein the block information transmitting means transmits block information designated as an effective block. 4. The image communication apparatus according to claim 1, wherein the block information transmitting means transmits block information designated as a valid block and block information not designated as a valid block. 5. An image receiving unit that receives encoded image data, a decoding unit that decodes the image data received by the image receiving unit, a block information receiving unit that receives block information, and a block information receiving unit according to the block information. The image communication apparatus according to claim 1, further comprising: an image synthesizing unit that synthesizes an image of one screen, and a display unit that displays the image of one screen synthesized by the synthesizing unit. 6. The image forming apparatus further comprises specific image data generating means for generating specific image data, wherein the block designating means can further designate a specific image block, and the image synthesizing means specifies the specific image block by the block designating means. 6. The image communication apparatus according to claim 5, wherein if specified, the specific image data generated by the specific image data generating means is combined. 7. The image communication apparatus according to claim 5, wherein the image synthesizing unit synthesizes a moving image, a still image, and a specific image. 8. The image communication apparatus according to claim 5, wherein the image synthesizing unit simultaneously synthesizes a received image received by the image receiving unit and a transmitted image transmitted by the transmitting unit as a moving image. 9. A voice processing unit for performing voice processing, a voice encoding unit for encoding the voice data processed by the voice processing unit, and a voice for transmitting the voice data encoded by the voice encoding unit. 6. A transmission means is further included.
The image communication device described. 10. An image communication system comprising an image communication device capable of transmitting and receiving images, wherein the first image communication device comprises an image input means for inputting an image and an effective block of image data constituting one screen in block units. Block specifying means, an encoding means for encoding the block specified by the block specifying means, an image transmitting means for transmitting the image data encoded by the encoding means, and the block specifying means The second image communication apparatus includes a block information transmitting unit that transmits the block information, and an image receiving unit that receives the encoded image data, and a decoding unit that decodes the image data received by the image receiving unit. Means, block information receiving means for receiving block information, and combining one screen image according to the block information An image communication system comprising: an image synthesizing unit; and a display unit for displaying a one-screen image synthesized by the synthesizing unit. 11. An image communication system comprising an image communication device capable of transmitting and receiving images, wherein the first image communication device comprises an image input means for inputting an image, a block information receiving means for receiving block information, and the block. The second image communication apparatus has one screen, which has an encoding means for encoding the block designated by the designating means and an image transmitting means for transmitting the image data encoded by the encoding means. Block designation means for designating an effective block of image data in block units, block information transmission means for transmitting the block information designated by the block designation means, and image reception for receiving the image data encoded by the encoding means Means, decoding means for decoding the image data received by the image receiving means, and 1 according to the block information. An image communication system comprising: an image synthesizing means for synthesizing an image on a screen; and a display means for displaying an image on one screen synthesized by the synthesizing means.