JPH11252459A - Image compositing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To composite a supply image with a mixed image of main and sub images by outputting a supply image from a 1st supplying means at the detecting of a specified value in the sub image and outputting the mixed image from a generating means, when it is not detected. SOLUTION: When image data PXD of a sub video are prepared at the compositing of external videos, a part corresponding to a PDX 1 (2nd emphasis value) of the data PXD is switched from a main video to an output of the external videos, and the external videos are composited with the main and sub videos. That is, a PXD 1 circle is produced by using one 2nd emphasis value of four states of the data PXD, a character string is produced with a pixel value (either of them has a background value), the 2nd emphasis value is detected and an external video signal is switched and inserted. For a character string part having no 2nd emphasis value, the external video signal cannot be switched. For instance, if a pixel data PXD 2 circle part is a 1st emphasis value, it is not detected, and a triangular part of the main video becomes 6 star parts of the external video signal.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image synthesizing apparatus for synthesizing, for example, a broadcast video, a communication video, or an image composed of a main video and a sub video reproduced from an optical disk with an image captured by an image pickup device.

[0002]

2. Description of the Related Art Conventionally, as a method of synthesizing two or more video signals (images), a blue color of a video signal photographed with a subject placed on a blue background is detected, and the detected signal is used as a signal. A method using a chroma key, which is switched and inserted into another video signal and synthesized, or a frame signal of a previously generated square or round video is used.

[0003] On the other hand, there is a reproducing apparatus for compressing and recording a video signal on an optical disc, recording the video signal, and decoding and reproducing the video signal. Video-CD using MPEG-1 standard which is a video compression standard
And a DVD using the MPEG-2 system. The contents recorded therein include karaoke and educational programs for children. For example, suppose that a video signal of a gender duet is reproduced in karaoke content, and a male customer sings the karaoke song. The music performance and the female song are reproduced and the male customer's song is mixed, and the synthesized music and the song flow from the speaker.

Here, as a new function, when a video signal of a male customer is shot by a video camera, synthesized with a reproduced video and displayed on one screen, the following problem occurs.

[0005] It is difficult to set the background of the subject of the photographed video to a specific color such as blue, and the composite position on the screen differs depending on the position of the male image reproduced and / or the position of the female, and also on the contents individually. Each adjustment requires time and effort.

[0006] As another method of remote education, which is another application, when a textbook of a child's education program is used as content and the contents of a blackboard in a remote classroom are inserted and displayed as one screen, the same applies as described above. Problems occur.

As described above, in the conventional video synthesizing apparatus, one is that it is difficult to set the background of the subject of the captured video to a specific color, and the synthesizing position on the screen is different for each reproduced content. , Each situation is different, and it takes time to adjust each time.

[0008]

SUMMARY OF THE INVENTION According to the present invention, when a reproduced image to be reproduced from a recording medium is combined with an image or the like captured by an image pickup device to display the image on a single screen, the captured image or the like can be easily displayed. It is an object of the present invention to provide an image synthesizing apparatus that can select an insertion position and can synthesize images.

[0009]

An image synthesizing apparatus according to the present invention comprises a first supply means for supplying an image from an external device, and a second supply means for supplying a main picture and a sub-picture accompanying the main picture. Means, a generating means for generating a mixed image of the main video and the sub-picture supplied by the second supplying means, a detecting means for detecting a specific value in the sub-video, and a specification in the sub-video by the detecting means When a value is detected, a supply image from the first supply unit is output, and when a specific value in the sub-video is not detected by the detection unit, a mixed image from the generation unit is output. And an output unit for outputting a combined image in which the supply image is combined with the mixed image of the main video and the sub video.

An image synthesizing apparatus according to the present invention comprises: a first supply unit for supplying an image from an external device; a second supply unit for supplying a main image and a sub-image accompanying the main image; Determining means for determining a mixing ratio between a supply image from the first supply means and a main image from the second supply means based on a specific value of the sub-picture supplied by the supply means; Output means for outputting a composite image of the supply image from the first supply means and the main video from the second supply means at a certain mixing ratio.

An image synthesizing apparatus according to the present invention comprises: a first supply unit for supplying an image from an external device; a second supply unit for supplying a main image and a sub-image accompanying the main image; Depending on the specific value of the sub-picture supplied by the supply means, the mixing ratio of the supply image from the first supply means and the main picture from the second supply means, or the sub-picture from the second supply means Determining means for determining a mixing ratio with the main image from the second supply means; and a supply image from the first supply means and a second supply means at a mixing ratio determined by the determination means. Or a mixed image of the main image from the second supply unit and the sub-image from the second supply unit at a mixture ratio determined by the determination unit. The main image and the supplied image Outputs the image, or alternatively consists of output means for outputting a composite image of the main image and the sub-picture.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an image synthesizing apparatus including an optical disk reproducing apparatus and an image pickup apparatus according to an embodiment of the present invention will be described with reference to the drawings.

FIG. 1 shows an image synthesizing apparatus according to an embodiment of the present invention. This image display system
Optical disc reproducing apparatus 1 for reproducing data from an optical disc
And an operation unit 2 for instructing operation of the entire image display system, a camera unit 3 as an image pickup device, and a video reproduced by the optical disc reproducing device 1 and a video obtained by synthesizing a video from the camera unit 3 with this video. Display unit 4 for displaying
And an audio unit 5 for generating a sound reproduced by the optical disc reproducing apparatus 1.

In FIG. 1, an optical disk (DVD) 10 as a recording medium is placed on a turntable (not shown), clamped by a clamper, and
Is driven to rotate. If the playback mode is set, the information recorded on the optical disk 10 is picked up by the pickup unit 12. Pickup unit 12
Are controlled by the servo unit 13 for movement control, focus control, and tracking control in the disk radial direction. The servo unit 13 also sends a control signal to the disk motor drive unit 14 to rotate the motor 11 (that is, the optical disk 1).
0) is performed.

The output of the pickup section 12 is input to a demodulation / error correction section 15 and demodulated. The demodulated data demodulated here is input to the demultiplexer 17 via the buffer 16. The demodulated data is input to the DSI decoder 19 via the input buffer 18. The decoded DSI (data search information) is sent to the system control unit 30. The demodulated data is sent to the system control unit 30 via the system buffer 20. The data taken into the system control unit 30 through the system buffer 20 includes, for example, management information.

The demultiplexer 17 separates each pack.

The video pack (V) taken out of the demultiplexer 17 PCK) is input to the video decoder 22 via the buffer 21 and is decoded. Video signal (main video signal) output from video decoder 22
Is input to the video reproduction unit 23.

Also, the signal extracted from the demultiplexer 17 is
Sub picture pack (SP PCK) is buffer 2
4 to the sub-picture decoder 25 and
Coded. Output from the sub-picture decoder 25
The reproduced sub-picture (sub-video signal) is transmitted to the video playback unit 23.
Is input to

The PCI pack extracted from the demultiplexer 17 is input to a PCI decoder 29 via a buffer 28 and decoded. PCI decoder 2
9 is output from the system control unit 30 and the video reproduction unit 23.
Is input to

Thus, in the video reproducing section 23, a video signal in which the sub-picture from the sub-picture decoder 25 is superimposed on the main video signal from the video decoder 22 is obtained. Second emphasis value, first emphasis value or second emphasis value
Highlight value, coordinate value of highlight information from PCI decoder 29, contrast value of highlight information, color code value of highlight information, coordinate value of display command from sub-picture decoder 25, contrast value of display command,
At the position based on the color code value or the like of the display command, a video signal in which the video captured by the camera unit 3 is synthesized is obtained. The video signal from the video reproduction unit 23 is supplied to the display unit 4.

The audio pack (A) taken out of the demultiplexer 17 PCK) is input to an audio decoder 27 via a buffer 26 and decoded. The output of the audio decoder 27 is supplied to the audio unit 5 such as a speaker.

The demultiplexer 17 separates and derives main video information, sub-picture (caption and text) information, audio information, control information, and the like. That is, the optical disc 10 records sub-picture (caption and character) information, audio information, management information, control information, and the like in correspondence with video information.

In this case, various languages can be selected as subtitle information and subtitle information, and audio information, which are selected under the control of the system control unit 30. An operation input by a user is given to the system control unit 30 through the operation unit 2.

Therefore, the video decoder 22 for decoding the main video information performs a decoding process corresponding to the type of the display device. For example, the main video information is NTSC, PA
L, SECAM, wide screen, etc. are converted. The audio information of the stream specified by the user is input to the audio decoder 27 and decoded. As for the sub-picture, the sub-picture data of the stream specified by the user is input to the sub-picture decoder 25 and decoded.

Next, an optical disk reproducing apparatus and an optical disk format to which the present invention is applied will be described.

The recording data structure of the optical disk 10 according to the present invention will be described. This optical disk 10 is a double-sided bonded disk having a storage capacity of about 5 GB on one side, for example, and has a large number of recording tracks between a lead-in area on the inner circumference side of the optical disk 10 and a lead-out area on the outer circumference side of the optical disk 10. Is arranged. Each track is composed of a large number of logical sectors, and various information (appropriately compressed digital data) is stored in each sector.

FIG. 2 shows a volume space of the optical disk 10.

As shown in FIG. 2, the volume space includes a volume and file configuration zone, a DVD video zone, and other zones. The UDF bridge configuration is described in the volume and file configuration zone, and the data of the UDF bridge configuration can be read even by a computer of a predetermined standard. The DVD video zone is a video manager (V
MG) and a video title set (VTS). The video manager (VMG) and the video title set (VTS) are each composed of a plurality of files. The video manager (VMG) is information for controlling a video title set (VTS).

FIG. 3 shows a video manager (VMG).
And the structure of a video title set (VTS).

The video manager (VMG) includes a video manager information (VMGI) as control data for controlling a video title set and the like, and a video object set (VMGM) as data for displaying a menu. VOBS). Video Manager Information for Backup (V
MGI).

The video title set (VTS) includes a video title set information (VTSI) as control data and a video object set (VMGM) as data for menu display. VOBS)
And a video object set (VTSTT) for a title of a video title set, which is a video object set for displaying video. VOBS). It also has a video title set information (VTSI) for backup.

Further, a video object set for displaying images (VTSTT) VOBS) is composed of a plurality of cells. Each cell (Cel)
1) is assigned a cell ID number.

FIG. 4 hierarchically shows the relationship between the video object set (VOBS) and the cell (Cell) and the contents of the cell (Cell). When DVD playback processing is performed, for video breaks (scene change, angle change, story change, etc.) and special playback, a video object unit (VOBU) which is a cell (Cell) unit or a lower layer thereof
It is now handled in units.

Video Object Set (VOBS)
First, one or more video objects (VO
B IDN1 to VOB IDNi).
In addition, one video object has one or more cells (C IDN1 to C IDNj). Further, one cell is composed of one or a plurality of video object units (VOBU). One video object unit (VOBU) is composed of one navigation pack (NV
PCK), one or more audio packs (A
PCK), one or more video packs (V PC
K) One or more sub-picture packs (SP
PCK).

Navigation Pack (NV PCK)
Are mainly used as control data for controlling the reproduction and display of data in the video object unit to which it belongs, and control data for performing data search of the video object unit.

The video pack (V PCK) is main video information and is compressed according to a standard such as MPEG. Sub picture pack (SP PCK) is sub-picture information having auxiliary contents for the main video. For example, movie subtitles, scenarios, etc., using run-length compression technology. Audio Pack (A PCK)
Is audio information.

FIG. 5 shows a video object (VOB).
And the relationship with the cell. FIG. 5 (A)
Is a block arrangement in which one title (for example, a movie scene) is continuous, and cells in the block are continuously reproduced. On the other hand, FIG.
Shows an example of cell arrangement when a multi-scene is recorded. That is, a standard is defined for DVDs that allow simultaneous recording of events that can be recorded at different angles. For example, in the case of a baseball movie, an image of the entire stadium shot from behind the backnet and an image of the referee's face zoomed in are acquired at the same time, each image is divided into multiple units, and these are interleaved. And record it on the track. The example in FIG. 5B shows an example in which two scenes are divided into units and each unit is interleaved. When such a disc is reproduced, one of the units is obtained at intervals and reproduced. Which scene is to be selected is determined by the operation of the user, or when a priority is given and there is no user selection, the one with the higher priority is reproduced.

FIG. 6 shows a program chain (PGC).
Shows an example in which the reproduction order of the above-mentioned cells is controlled.

As a program chain (PGC),
Various program chains (PGC # 1, PG # 1) are set so that the reproduction order of the data cells can be variously set.
C # 2, PGC # 3...) Are prepared. Therefore, the order of cell reproduction is set by selecting a program chain.

An example is shown in which programs # 1 to #n described as program chain information (PGCI) are executed. The illustrated program has contents for sequentially designating cells after #s in the video object set (VOBS).

FIG. 7 shows a video title set (VT)
S) shows a video title set information (VTSI). Video title set program chain information table (VTS) in video title set information (VTSI) PGCIT) is described. Therefore, when a video object set (VOBS) in one video title set (VTS) is played,
This video title set program chain information table (VTS A program chain selected by the user from a plurality of program chains presented in (PGCIT) is used.

The following data is described in the VTSI.

VTSI MAT: a management table of video title set information, which describes what information exists in the video title set, and describes a start address and an end address of each information.

VTS PTT SRPT: Video title set Part of title search pointer table, in which title entry points and the like are described.

VTSM PGCI UT: Video title set menu program chain Information unit table, which is a chain for reproducing a video title set menu described in various languages. Therefore, it is possible to confirm from the menu what kind of video title set is described, and what kind of style of playback order is described.

VTS TMAPT: Video title set time map table, in which information on the recording position of the VOBU managed in the program chain is described.

VTSM C ADT: video title set menu cell This is an address table in which the start and end addresses of the cells constituting the video title set menu are described.

VTSM VOBU ADMAP: video title set menu video object unit address map, in which VTS describes the start address of the video object unit for menu C ADT: video title set cell address table in which the start and end addresses of the cells constituting the video title set body are described.

VTS VOBU ADMAP: Video title set video object unit address map, in which the start address of the video object unit of the title body is described.

In the reproducing apparatus, when a program chain is selected, the reproduction order of the cells is set by the program chain. For playback, the NV included in the video object unit is used. PCK is referenced. NV The PCK has information for controlling display contents and display timing, and information for data search. Therefore, this NV V based on the information in the PCK table The PCK is taken out and decoded.
Also, other packs are taken out and decoded,
In that case, A of the language specified by the user
PCK, SP The removal of the PCK is performed.

FIG. 8 shows a configuration example of one pack and one packet.

One pack is composed of a pack header and a packet. A pack start code, a system clock reference (SCR), and the like are described in the pack header. The pack start code is a code indicating the start of a pack, and includes a system clock reference (S
CR) is information indicating the location time in the reproduction elapsed time for the entire apparatus. The length of one pack is 2048 bytes, and is defined and recorded as one logical block on the optical disk.

One packet is composed of a packet header and video data, audio data, sub-picture data, or navigation data. The packet header of the packet may be provided with stuffing. Padding may be provided in the data portion of the packet.

FIG. 9 shows the NV The PCK is extracted and shown.

NV PCK basically controls display images.
Picture control information to control
(PCI) packet and the same video object
Data Search Information (DSI)
With packets. Each packet has a packet header and
Stream ID, followed by the data
Is described. Each packet header has a stream I
D is described and NV Indicates that this is a PCK,
The Ream ID identifies PCI and DSI.
You. Each packet header contains a packet start code.
, Stream ID, and packet length are described.
Data is described.

The PCI packet is navigation data for changing the display content in synchronization with the reproduction of the video data in the video object unit (VOBU) to which the packet belongs.

[0057] The PCI packet contains PC, which is general information.
I General Information (PCI GI),
Non-seamless angle information (NSML
ANGLI) and highlight information, that is, highlight-in.
Formation (HLI) and record which is record information
Describes the information (RECI)
I have.

FIG. 10 shows reproduction control general information (PCI G
I) is shown.

PCI The GI has a general description of this PCI.
It is information and the following information is described. This na
Logical block number that is the address of the navigation pack
Bar (NV PCK LBN), managed by this PCI
Indicates the attributes of the video object unit (VOBU)
Video object unit category (VOBUC
AT), a video object unit managed by this PCI.
Shows user's operation prohibition information during the knit display period
User operation control (VOBU UO
P CTL), opening the display of video object units
Indicates the start time (VOBU S PTM), Video Object
Indicates the end time of the display of the project unit (VOBU E
PTM). VOBU S Specified by PTM
The first video to be played is an I-picture in the MPEG standard.
It is. Furthermore, the video object unit
Video object unit that indicates the display time of the last video
Knit sequence end presentation tie
(VOBU SE E PTM) or the first video in a cell
Cell error indicating the relative display elapsed time from the video frame
Push Time (C ElTM) and the like are also described.

The NSM described in the PCI
L ANGLI indicates the address of the destination (destination) at the time of an angle change. That is, the video object also has images captured from different angles. Then, when the user specifies to display an image at an angle different from the currently displayed angle, the address of the VOBU to be shifted to the next reproduction is described.

The highlight information HLI is information for designating a specific area in the screen in a rectangular shape and changing the luminance of this area, the color of the sub-picture displayed here, and the like. The highlight information HLI includes a highlight general information (HL). GI), a button color information table (BTN) for allowing the user to make a button selection for color selection. CO
LIT) and a button information table (BTNIT) for the selection button are described.

The RECI is video, audio, and sub-picture information recorded in the video object unit, and describes what data is to be decoded. For example, there are a country code, a copyright holder code, a recording date, and the like.

A DSI packet is navigation data for executing a search for a video object unit.

The DSI packet contains DS which is general information.
I General Information (DSI GI)
And seamless playback information (S
ML PBI), seamless angle information
(SML AGLI), video object unit
Information (VOBU) SRI), synchronization information
Information (SYNCI).

Here, each aggregate of each pack will be described.

FIG. 11 shows the relationship between the video object unit (VOBU) and the video packs in this unit. Video data in a VOBU is composed of one or more GOPs. The encoded video data conforms to, for example, ISO / IEC13818-2. A VOBU GOP is composed of an I picture and a B picture, and this data sequence is divided into a video pack.

FIG. 12 shows the relationship between audio streams and audio packs. Audio streams include linear PCM, Dolby AC-3, MP
There is data such as EG.

FIG. 13 shows an example of the logical structure of a pack of encoded (run-length compressed) sub-pictures.

As shown in the upper part of FIG. 13, one pack (SP) of sub-pictures (sub-pictures) included in video data
PCK) is composed of, for example, 2048 bytes (2 kB). One pack of a sub-picture includes a packet header and sub-picture data after a head pack header. The pack header contains a reference time (SCR; System Clock) through reproduction of the entire file.
Reference) information, which has a predetermined relationship with the time of the system timer, and has an SCR of the same time information.
Each sub-picture packet to which is added is compiled and transferred to a decoder described later.

The first sub-picture packet includes, after the packet header, sub-picture data that has been run-length compressed together with a sub-picture unit header (SPUH) described later. Similarly, the second sub-picture packet includes run-length compressed sub-picture data after the packet header.

A sub-picture data unit 310 is obtained by collecting a plurality of such sub-picture data for one unit (one unit) of run-length compression. The sub-picture data unit 310 is provided with a sub-picture unit header 311. After the sub-picture unit header 311, pixel data 312 obtained by performing run-length compression of video data of one unit (for example, data of one horizontal line of a two-dimensional display screen) and display control sequence information of each sub-picture pack are included. A table 313 is included.

That is, the sub-picture data unit 31
0 is a sub-picture unit header (SPUH) 3 in which various parameters for displaying a sub-picture are recorded.
11, display data (compressed pixel data; PXD) 312 composed of run-length codes, and a display control sequence table (DCSQT) 313.

FIG. 14 shows an example of a part of the contents of the sub-picture unit header 311 in the run-length compressed data 310 for one unit shown in FIG.

The sub picture unit header (SPU
H) 311 includes a display size of the pixel data (PXD) 312 on the TV screen, that is, a display start position and a display range (width and height) (SPDSZ; 2 bytes), and a display control sequence in the sub-picture data packet. The recording start address (SP DCSQT SA;
2 bytes).

More specifically, parameters having the following contents are recorded in the sub-picture unit header (SPUH) 311 as shown in FIG.

(1) Information (SPDSZ) indicating the display start position and display range (width and height) of the display data on the monitor screen; (2) Recording start position information of the display control sequence table 33 in the packet (Sub picture display control sequence table start address SP
DCSQT SA).

FIG. 15 shows the data structure of the sub-picture unit again.

The sub-picture unit is composed of a plurality of sub-picture packets. That is, one pack of the sub-picture information included in the video data is composed of, for example, 2048 bytes (2 kB), and one pack of the sub-picture information includes, after the first pack header,
It contains one or more sub-picture packets. The pack header is provided with time (SCR; System Clock Reference) information serving as a reference throughout the reproduction of the entire file, and a packet in the sub-picture pack to which the SCR of the same time information is added is sent to a decoder described later. It is to be transferred.

In the packet header of the above-mentioned packet,
The time at which the playback system should start controlling the display of the sub-picture data unit is recorded as a presentation time stamp (PTS). However, as shown in FIG. 16, the PTS is recorded only in the header of the first sub-picture data packet in each sub-picture data unit (Y, W). In the PTS, in a plurality of sub-picture data units reproduced with reference to a predetermined reproduction time SCR, values along the reproduction order are described for each sub-picture data unit.

FIG. 17 shows a serial arrangement state (n, n + 1) of sub-picture units composed of one or more sub-picture packets, and a presentation time stamp (PTS) described in a packet header of one of the units (n + 1). ) And the progress of the display control of the unit (n + 1) corresponding to this (PTS). That is, the relationship between the processing time of the PTS, the display clear period of the sub-picture unit (n), and the display start time of the sub-picture unit (n + 1) to be displayed from now on is shown.

As shown in FIG. 18, the sub-picture unit header (SPUH) 311 contains the sub-picture unit size (2-byte SPUH). SZ) and the recording start address (2-byte SP) of the display control sequence table 33 in the packet. DCSQT SA) are recorded.

SPU The SZ describes the size of one unit by the number of bytes, and the maximum size is 53248 bytes. SP DCSQT SA is a display control sequence table (SP) based on the relative number of bytes from the first byte of the unit. DCSQT) is described.

As shown in FIG. 19, the display control sequence
Table (SP DCSQT) 313 includes one or more
Sub-picture display sequence (SP DCSQ0, S
P DCSQ1, ... SP DCSQn) are described in the order of execution.
Have been. Display control sequence table (SP DCS
QT) 313 is during the valid period of the sub-picture unit
To start / stop the display of sub-pictures and change the attributes
Display sequence information to be displayed.

FIG. 20 shows the sub-picture display control sequence (SP DCSQ). This SP The following contents are described as parameters of DCSQ.

When the execution of the video data display control is started
Sub picture display control start time (SP
DCSQ STM; Sub-Picture Display Control S
equence Start Time) and the next sub-picture display control
Sequence (SP DCSQ)
(SP NXT DCSQ SA; Addres of NextSP
DCSQ) and display control commands for sub-picture data
(SP COMMAND; Sub-Picture Display Control
l Command) (SP COMMAND1, SP COMM
AND2, SP COMMAND3, ...)
You.

Here, the packet header (FIGS. 15 and 16)
Presentation timestamp PT
S is, for example, the playback start time at the beginning of the file.
Reference time (SCR;
It is specified by relative time from System Clock Reference. As described above, the SCR is described in the pack header provided before the packet header.

Further, the sub-picture display control time (SP) which sets the display control sequence execution start time is set. D
CSQ (STM) is defined by a relative time (relative PTM) from the PTS described in the packet header.

Therefore, (SP DCSQ STM)
And the count value of the sub-timer are compared. If the count value of the sub-timer is greater than the display control sequence time, the display state of the output data decoded by the decoding means is controlled according to the sequence control data.

In practice, (SP DCSQ With respect to a video frame displayed first after the execution start time described as (STM) is described, control for display is started for a sub-picture represented in the video frame. Display control sequence time (SP
DCSQ "0000h" is described in STM). The value of the execution start time is equal to or longer than the PTS described in the sub-picture packet header, and is 0 or a positive integer. Based on the display control start time, one (SP) When the command in the DCSQ is executed, the next designated (SP DCS
The execution process is started when the command in Q) reaches the display control start time.

SP NXT DCSQ SA is indicated by a relative number of bytes from the first sub-picture unit,
Next SP It shows the DCSQ address. Next SP
If there is no DCSQ, this SP DCSQ
Is the relative number of bytes from the first byte of the relevant sub-picture unit, and the first SP The start address of DCSQ is described. SP DCCMDn describes one or more display control sequences.

FIG. 21 shows a display control command (SP) for performing display control. DCCMD).

The display control command (SP DCCMD) includes an instruction (FSTA) for setting a compulsory display start timing of pixel data. DSP), an instruction to set the display start timing of the pixel data (STA) DSP),
Command to set the display end timing of pixel data (S
TP DSP), instruction to set color code of pixel data (SET COLOR), an instruction to set the contrast between the pixel data and the main image (SET CONT
R), an instruction to set a display area for pixel data (SE)
T DAREA), an instruction to set the display start address of the pixel data (SET DSPXA), an instruction (C) for setting a change control of color and contrast of pixel data
HG COLCON), a command to end the display control (C
MD END). The respective codes and extension fields are as follows as shown in the figure.

That is, a forced display start timing command (FSTA) The code of the DSP is 00h and the extension field is 0 bytes. When this instruction is described, the forced display of the sub-picture unit having this code is executed regardless of the display state of the sub-picture.

Display start timing instruction (STA) DS
The code of P) is 00h, and the extension field is 0 bytes. This command is a display start command for the sub-picture unit. This command is ignored when the display of the sub-picture is turned off.

Display stop timing command (STP DS
The code of P) is 02h, and the extension field is 0 bytes. This command is a sub-picture unit display stop command. The sub-picture can be redisplayed by the previous display start command.

A color code setting instruction (SET COLO
The code of R) is 03h, and the extension field is 2 bytes. This command is a command for determining the color of each pixel of the pixel data, and is described in the extension field by a pallet code. The second pallet code for each pixel
Each palette code is described for an enhancement pixel (4 bits), a first enhancement pixel (4 bits), a pattern pixel (4 bits), and a background pixel (4 bits).

Here, this instruction (SET COLOR)
Is not present in the sub-picture unit,
The last used one before that is maintained and this instruction is used. This instruction is specified at the beginning of each line.

A contrast setting command (SET CONT
The code of R) is 04h, and the extension field is 2 bytes. This command is a command for setting a mixing ratio between the pixel data and the main image, and is described in the extension field by contrast designation data. The contrast designation data of the pixel includes a second emphasized pixel (4 bits), a first emphasized pixel (4 bits), a pattern pixel (4 bits),
Since there is a background pixel (4 bits), contrast designation data k for each pixel is described.

The contrast of the main image is (16-k) / 1
6, the contrast of the sub-picture is k / 16. 16 is a gradation. The value may be "0", in which case the sub-picture does not appear on the screen even though it exists. If the value is not “0”, k is treated as (value + 1).

Here, this instruction (SET CONTR)
Is not present in the sub-picture unit,
The last used one before that is maintained and this instruction is used. This instruction is specified at the beginning of each line.

Display area setting command (SET DARE
The code of A) is 05h, and the extension field is 6 bytes. This command is a command for setting a display area of square pixel data on the screen. In this instruction:
The start position (10 bits) and end position (10 bits) of the X-axis coordinates on the screen, and the start position (10 bits) and end position (10 bits) of the Y-axis coordinates are described. The remaining bits of the 6 bytes and the reservation are reserved. The value at the start position of the X-axis coordinate is subtracted from the value at the end position of the X-axis coordinate, and +1
Should be the same as the number of display pixels on one line. The origin of the Y-axis coordinate is line number 0. The origin of the X-axis coordinate is also 0. On the screen, it corresponds to the upper left corner. The Y-axis coordinate value is 2 to 479 (525 lines / 60 Hz
TV) or 2-574 (625 lines / 50H
z in the case of TV), whereby a sub-picture line is specified, and the X-axis coordinate value describes a value of 0 to 719, thereby specifying a pixel number.

Here, this instruction (SET DAREA)
Is not present in the sub-picture unit,
The instruction included in the last sub-picture unit sent earlier is used as it is.

Display start address setting instruction (SET DS
PXA) is 06h, and the extension field is 4 bytes. This command is a command indicating the first address of the image data to be displayed. The first addresses of the odd field (16 bits) and the even field (16 bits) are described by the relative number of bytes from the head of the sub-picture unit. The first of the position indicated by this address
Pixel data indicates a run-length compression code including the first pixel at the left end of the line.

Here, this instruction (SET DSPXA)
Is not present in the sub-picture unit,
The instruction included in the last sub-picture unit sent earlier is used as it is.

Color and contrast change control command (C
HG The code of (COLCON) is 07h, and the extension field is (pixel control data size + 2 bytes).

(CMD The code of END) is FFh, and the extension byte is 0 byte.

FIG. 22 shows the above (CHG COLCO
N) pixel control data (P
XCD (Pixel Control Data).

This PXCD is data for controlling the color and contrast of the pixel displayed as a sub-picture during the display period. The instruction described in the PXCD is a sub picture display control start time (SP DCSQ
It is performed for each video frame from the first video frame after the STM is described, and is performed until the next new PXCD is set. The old PXCD is canceled when the new PXCD is updated.

Line control information (LN) shown in FIG. CT
LI (Line Control Information) designates a line on which sub picture change control is performed. A plurality of lines on which the same conversion control is performed can be designated. The pixel control information (PX CTLI; Pixcel Control I
nformation) describes the designated position on the line where the change control is performed. One or more pixel control information (PX CT
LI) can designate a plurality of positions on a line where conversion control is performed.

As the end code of the pixel control data (PXCD), (0FFFFFFFh) is LN CTLI is described. P such that only this end code exists
When XCD arrives, (CHG COLON) means the end of the instruction itself.

With reference to FIG. 23, each of the above instructions will be further described.

LN The CTLI is composed of 4 bytes. The line number (10 bits) at which the change of the sub-picture starts, the number of changes (4 bits), and the end line number (1)
0 bit). The change start line number is the line number at which the change of the pixel control content starts, and is described by the line number of the sub-picture. The end line number is a line number at which the control state based on the pixel control content is stopped, and is also described by the line number of the sub-picture. The number of changes is
This is the number of change positions, and the pixel control information (PX
CTLI) number. The line numbers at this time are naturally 2 to 479 (when the television system is 525 lines / 60 Hz) or 2 to 574 (when the television system is 625 lines / 50 Hz).

Next, one pixel control information (PX) CTL
I) is composed of 6 bytes and describes a change start pixel number (10 bits) and control information for changing the color and contrast of each pixel following the pixel.

The palette codes for the pixels are for the second emphasized pixel (4 bits), for the first emphasized pixel (4 bits),
For pattern pixels (4 bits), for background pixels (4 bits)
Each pallet code is described. Also, the contrast designation data for the second emphasized pixel (4 bits), the first emphasized pixel (4 bits), the pattern pixel (4 bits), and the background pixel (4 bits) are described as the contrast designation data for the pixel. Have been.

The above-mentioned change start pixel numbers are described by the pixel numbers in the display order. When this is zero, SET CO
LOR and SET CONTR is ignored. A color pallet code is described as the color control information, and the contrast designation data is described as the contrast control information as described above.

When no change is requested in each of the above control information, the same code as the initial value is described.
The initial value is a color code and contrast control data specified from the beginning to be used for the sub-picture unit.

Further, the color and contrast change control command (CHG COLCON) may include information for designating a position where a video image taken by the camera unit 3 used in the present invention is synthesized.

Pixel control data (PXCD: Pixel Control Data) described in the (CHG COLCON) extension field of the display control command (SP DCCMD) in the display control sequence table of the sub-picture unit
Is output to the video reproduction unit 23, thereby indicating the synthesis area of the video captured by the camera unit 3.

Next, a method of compressing a sub-picture will be described.

FIG. 24 shows run-length compression rules 1 to 6 when pixel data (run-length data) of a sub-picture is created. According to this rule, the data length (variable length) of one unit of the unit is determined. Then, encoding (run-length compression) and decoding (run-length decompression) are performed with the determined data length.

FIG. 25 shows run length compression rule 1 used in the encoding method according to one embodiment when the preceding sub-picture pixel data (run length data) 312 is composed of 2-bit pixel data. ~ 6
It is to explain.

According to rule 1 shown in the first column of FIG. 24, when one to three identical pixels continue, one unit of encoded (run-length compressed) data is constituted by 4-bit data. In this case, the first two bits represent the number of continuous pixels, and the following two bits represent pixel data (such as pixel color information).

For example, the first compressed data unit CU01 of the pixel data PXD before compression shown in the upper part of FIG.
Are two 2-bit pixel data d0, d1 = (000
0) b (b is binary). In this example, the same 2-bit pixel data (00)
b is continuous (continuous).

In this case, as shown in the lower part of FIG. 25, d0, d1 = (1000) b obtained by connecting the 2-bit display (10) b of the continuation number "2" and the content (00) b of the pixel data.
Is the data unit CU01 * of the pixel data PXD after compression.
Becomes

In other words, according to rule 1, data unit C
(0000) b of U01 is (1) of data unit CU01 *.
000) b. In this example, no substantial bit length compression is obtained, but for example, the same pixel (00)
If b is three consecutive CU01 = (000000) b, CU01 * = (1100) b after compression, and 2
A bit compression effect is obtained.

According to rule 2 shown in the second column of FIG. 24, when 4 to 15 identical pixels continue, one unit of encoded data is constituted by 8-bit data. In this case, the first two bits are represented by an encoding header indicating that it is based on rule 2, the subsequent four bits represent the number of continuous pixels, and the subsequent two bits represent pixel data.

For example, the second compressed data unit CU02 of the pixel data PXD before compression shown in the upper part of FIG.
Are five 2-bit pixel data d2, d3, d4, d
5, d6 = (0101010101) b.
In this example, the same 2-bit pixel data (01) b is 5
It is continuous (continuous).

In this case, as shown in the lower part of FIG. 25, the encoded header (00) b, the 4-bit display (0101) b of the continuation number "5", and the content (01) b of the pixel data are connected. d2 to d6 = (00010101) b is the data unit CU02 * of the compressed pixel data PXD.

In other words, according to rule 2, the data unit C
(0101010101) b of U02 (10-bit length)
Is (00010101) b (8) of the data unit CU02 *.
(Bit length). In this example, the effective bit length compression amount is only 2 bits from 10 bits to 8 bits. However, when the number of continuations is, for example, 15 (a 30-bit length in which 15 01s of CU02 are continuous), this is 8 bits. It becomes compressed data (CU02 * = 00111101),
A 22-bit compression effect is obtained for the bits. That is, the bit compression effect based on Rule 2 is greater than that of Rule 1. However, rule 1 is also required to support run-length compression of a fine image with high resolution.

According to rule 3 shown in the third column of FIG. 24, when 16 to 63 identical pixels continue, one unit of encoded data is constituted by 12-bit data. In this case, the first 4 bits represent an encoded header indicating that the rule is based on rule 3,
The next 6 bits indicate the number of continuous pixels, and the subsequent 2 bits indicate pixel data.

For example, the third compressed data unit CU03 of the video data PXD before compression shown in the upper part of FIG.
Are 16 2-bit pixel data d7 to d22 = (10
1010... 1010) b. In this example, 16 pieces of the same 2-bit pixel data (10) b are continuous (continuous).

In this case, as shown in the lower part of FIG. 25, the encoded header (0000) b, the 6-bit display (010000) b of the continuation number “16”, and the content of the pixel data (10)
b and d7 to d22 = (0000000100001)
0) b is the data unit CU of the compressed pixel data PXD
03 *.

In other words, according to rule 3, the data unit C
(101010... 1010) b (32-bit length) of U03 is (000000001000) of the data unit CU03 *.
010) b (12-bit length). In this example, the substantial bit length compression amount is 20 bits from 32 bits to 12 bits, but the continuation number is, for example, 63 (CU0
In the case of 63 consecutive 10s of 3 and a length of 126 bits, this is 12-bit compressed data (CU03 * = 00).
0011111110), and a compression effect of 114 bits can be obtained for 126 bits. That is, the bit compression effect based on Rule 3 is greater than that of Rule 2.

According to rule 4 shown in the fourth column of FIG. 24, when 64 to 255 identical pixels continue, one unit of encoded data is constituted by 16-bit data. In this case, the first 6
A bit indicates an encoding header indicating that the rule is based on rule 4, the subsequent 8 bits indicate the number of continuous pixels, and the subsequent 2 bits indicate pixel data.

For example, the fourth compressed data unit CU04 of the uncompressed pixel data PXD shown in the upper part of FIG.
Are 69 2-bit pixel data d23 to d91 = (1
11111... 1111) b. In this example, 69 pieces of the same 2-bit pixel data (11) b are continuous (continuous).

In this case, as shown in the lower part of FIG. 25, the encoded header (000000) b and the continuation number “69” of 8
D23 to d91 that connect the bit display (00100101) b and the content (11) b of the pixel data = (000000)
0010010111) b is the pixel data PX after compression
D data unit CU04 *.

In other words, according to rule 4, data unit C
(111111... 1111) b (138-bit length) of U04 is (00000000) of data unit CU04 *.
10010111) converted to b (16-bit length).
In this example, the substantial bit length compression amount is 122 bits from 138 bits to 16 bits, but the continuation number is, for example, 255 (because 11 of CU01 continue 255, 51
In the case of (0-bit length), this becomes 16-bit compressed data (CU04 * = 000001111111111), and a 494-bit compression effect is obtained for 510 bits. That is, the bit compression effect based on Rule 4 is
Larger than that of Rule 3.

According to rule 5 shown in the fifth column of FIG. 24, when the same pixel continues from the switching point of the encoded data unit to the end of the line, one unit of the encoded data is constituted by 16-bit data. In this case, the first 14 bits represent an encoded header indicating that the data is based on rule 5, and the following 2 bits represent pixel data.

For example, the fifth compressed data unit CU05 of the uncompressed pixel data PXD shown in the upper part of FIG.
Is one or more 2-bit pixel data d92 to dn = (0
00000... 0000) b. In this example, the same two-bit pixel data (00) b is continuous (continuous) in a finite number. However, in rule 5, the number of continuous pixels may be one or more.

In this case, as shown in the lower part of FIG. 25, d92 to dn = connecting the encoded header (00000000000000) b and the content (00) b of the pixel data
(00000000000000000) b is the data unit CU05 * of the compressed pixel data PXD.

In other words, according to rule 5, the data unit C
(00000000 ... 0000) b (unspecified bit length) of U05 is (00000000) of data unit CU05 *.
00000000) b (16 bits long).
In rule 5, the same pixel continuation number up to the line end is 16
With a bit length or more, a compression effect can be obtained.

According to rule 6 shown in the sixth column of FIG. 24, when one pixel line in which data to be encoded is arranged ends, the length of one line of compressed data PXD is not an integral multiple of 8 bits (ie, When the data is not byte-aligned, 4-bit dummy data is added so that the compressed data PXD for one line is in units of bytes (that is, byte-aligned).

For example, data units CU01 * to CU05 of compressed pixel data PXD shown in the lower part of FIG.
The total bit length of * is always an integral multiple of 4 bits, but is not necessarily an integral multiple of 8 bits.

For example, data units CU01 * to CU05
If the total bit length of * is 1020 bits and 4 bits are insufficient for byte alignment, as shown at the bottom of FIG. 25, 4-bit dummy data CU06 * =
(0000) b is added to the end of the 1,020 bits to generate a byte-aligned 1024-bit data unit CU0.
1 * to CU06 * are output.

The two-bit pixel data arranged at the end of one unit does not necessarily indicate four types of pixel colors. Pixel data (00) b means a background pixel of the sub-picture, pixel data (01) b means a pattern pixel of the sub-picture, pixel data (10) b means a first emphasized pixel of the sub-picture, Pixel data (1
1) b may mean the second emphasized pixel of the sub-picture.

In this way, depending on the contents of the 2-bit pixel data, the run-length data can be selected from background pixels, sub-picture pattern pixels, sub-picture first emphasized pixels, and sub-picture second emphasized pixels. Can be determined.

If the number of constituent bits of the pixel data is larger, another type of sub-picture pixel can be specified. For example, pixel data is 3-bit (000) b to (11)
1) When composed of b, up to eight types of pixel colors + pixel types (enhancement effect) can be specified in sub-picture data to be run-length encoded / decoded.

The first of the sub-picture pixel data PXD
Information for designating a position for synthesizing an image or the like captured by the camera unit 3 used in the present invention may be described in the emphasized pixel and the second emphasized pixel of the sub-picture.

The content of the first emphasized pixel or the second emphasized pixel of the pixel data PXD is output from the sub-picture decoder 25 to the video reproducing unit 23. The content of the first emphasized pixel or the second emphasized pixel of the pixel data PXD is output to the video reproducing unit 23 to indicate a composite area of the video captured by the camera unit 3.

The highlight information (HLI) is information for applying highlight to one rectangular area in the display area of the sub-picture. The highlight information describes the color of the sub-picture in a specific rectangular area within the display area of the sub-picture and the mixing ratio (contrast) with the video. As shown in FIG. 26, the highlight information is valid for all sub-picture streams reproduced during the valid period. For example, when a video, a sub-picture, and highlight information are combined, a composite screen as shown in FIG. 27 is displayed on the monitor unit 6.

As shown in FIG. 28, the highlight information
A highlight general information (HL_GI) 113A, a button color information table (BTN_COLIT) 113B, and a button information table (BTNIT) 113C are described. As shown in FIG. 29, the button color information table (B
TN_COLIT) 113B includes button color information (BT
N_COLI) 113D, 113E, 113F are described, and the button information table (BTNIT) 113C includes
A maximum of 36 pieces of button information (BTNI) 113I,... Are described.

For example, as shown in FIG. 29, 36 pieces of button information (BTNI) 113I,...
A group mode, a two group mode composed of 18 pieces of button information, and a three group mode composed of 12 pieces of button information are described.

Highlight general information (HL_GI) 113
A is information of the entire highlight information. As shown in FIG. 30, the highlight general information (HL_GI) 113A includes the state of the highlight information (HLI_SS), the highlight start time (HLI_S_PTM), the highlight end time (HLI_E_PTM), and the button selection end time (BTN_SL_E_PTM). , Button mode (BT
N_MD), button start number (BTN_SN), number of valid buttons (BTN_Ns), number of buttons selectable by number (NSBTN_Ns), forced selection button number (FSL)
BTN_N), forced confirmation button number (FACBTN_
N) is described.

The state of the highlight information (HLI_SS) describes the state of the highlight information in the corresponding PCI. For example, in the case of "00", it is described that no valid highlight information exists, and in the case of "01", the previous V
It is described that highlight information different from the highlight information of the OBU exists. In the case of "10", it is described that the same highlight information as the highlight information of the previous VOBU exists, and in the case of "11". , It is described that the highlight information of the previous VOBU differs from the highlight information of only the button command.

Highlight information start PTM (HLI
_S_PTM) describes a highlight start time (start presentation time (SPTM)) at which the corresponding highlight information becomes valid. The highlight start time is equal to or longer than the display start time of the sub-video stream targeted by the highlight information. When the HLI_SS is described as “01”, the highlight start time of the highlight information is set during the VOBU corresponding to the PCI.
It has been updated. HLI_SS is "10"
Alternatively, when “11” is described, the highlight start time of the highlight information is the VOB corresponding to the PCI.
During U, it is used without interruption.

Highlight end time (HLI_E_PT)
M) describes a highlight end time at which the highlight information becomes invalid. The highlight end time is equal to or shorter than the display end time of the sub-video stream targeted by the highlight information. When HLI_SS is described as "01", the highlight end time of the highlight information is updated during the VOBU corresponding to the PCI. HLI_SS is “10” or “1”
When "1" is described, the highlight end time of the highlight information is used without interruption during the VOBU corresponding to the PCI. The highlight end time while the HLI is in the stationary state. (FFFFFFFFh) is described as (HLI_E_PTM).

The button selection end time (BTN_SL_E_
PTM) describes the end time of button selection. The button selection end time is equal to or shorter than the display end time of the sub-video stream targeted by the highlight information.
When HLI_SS is described as “01”, the button selection end time of the highlight information is updated during the VOBU corresponding to the PCI. HLI_
When the SS is described as “10” or “11”, the button selection end time of the highlight information is determined by the PCI
Are used without interruption during the corresponding VOBU. While the HLI is in the stationary state, the button selection end time (BTN_SL_E_PTM) is set to (FFFFFF
FFh) is described.

In the button mode (BTN_MD), grouping of buttons and a display type of a sub-picture corresponding to each group are described. For example, the number of button groups (BTNGR_Ns), the display type of sub-picture corresponding to button group 1 (BTNGR1_DSPTY), and the display type of sub-picture corresponding to button group 2 (BTNG)
R2_DSPTY) and the display type (BTNGR3_DSPTY) of the sub-picture corresponding to the button group 3 are described. The number of button groups (BTNGR_Ns) is
In the case of "01", there are one group, in the case of "10", there are two groups, and in the case of "11", there are three groups. When the display type is “01”, the display type is wide (9/1
6) In the case of "10", letter box, and in the case of "11", pan-scan.

The button start number (BTN_SN) describes the offset number of the first button in the button group. The offset number can be described in the range of 1 to 255. Button start number (BTN_
SN) is commonly applied to each button group.

The number of valid buttons (BTN_Ns) describes the number of valid buttons in the button group. If the button group is 1, the number of buttons is from 1 to 36,
When the button group is 2, the description can be made in the range from 1 to 18, and when the button group is 3, the description can be made in the range from 1 to 12. The number of valid buttons (BTN_Ns) is commonly applied to each button group.

The number of buttons that can be selected by number (NSBTN_
In Ns), the number of buttons that can be selected by a button number in the button group is described. If the number of buttons is 1, the number of buttons is from 1 to 36, and the number of buttons is 2.
In the case of 1 to 18 and the button group is 3,
It can be described in the range of 1 to 12. The number of buttons that can be selected by number (NSBTN_Ns) is commonly applied to each button group.

Forced selection button number (FSLBTN_N)
Describes a button number that is forcibly selected in the highlight start time (HLI_S_PTM). Thus, even if the presentation starts during the highlight valid period, the button number set in the highlight information is selected. When the button group is 1, the button number is in the range from 1 to 36 and 63. When the button group is 2, the button number is in the range from 1 to 18 and 63.
If the button group is 3, range from 1 to 12 and 6
3 can be described. Forced selection button number (FSLBT
N_N) is commonly applied to each button group.

Forced confirmation button number (FACBTN_N)
Contains the button selection end time (BTN_SL_E_PT
In M), a button number for forcibly confirming the state is described. When the button group is 1, the button number ranges from 1 to 36 and 63, and when the button group is 2,
When the button group is 3 and the range from 1 to 18 is 63, the range can be described as 1 to 12 and 63. The forced confirmation button number (FACBTN_N) is commonly applied to each button group.

Button color information table (BTN_COLI)
T) 113B includes three pieces of button color information (BTN_COLI) 113D, 113E, and 113, as shown in FIG.
F is described. Button color information (BTN_COLI) 1
13D,... In the order of description, button color numbers (BTN_COL)
N) is assigned from 1. Button color information (BTN_
As shown in FIG. 31, the selected color information (SL_COLI) 113G and the determined color information (AC_COLI) 113H are described in each of the CCOLs 113D,. The selection color information (SL_COLI) 113G describes the color and contrast to be changed when the button is in the selected state. The confirmed color information (AC_COLI) 113H describes the color and contrast to be changed when the button is in the confirmed state. The selected state of the button is a state in which the selected color is displayed. In this state, the user can change the highlighted button to another button. The confirmed state of the button is a state in which the confirmed color is displayed and the button command is executed. In this state, the user is prohibited from changing the highlighted button to another button.

As shown in FIG. 32, the selected color information (SL_COLI) 113G contains the selected color code of the emphasized pixel 2,
The selection color code of the enhancement pixel 1, the selection color code of the pattern pixel, the selection color code of the background pixel, the selection contrast of the enhancement pixel 2, the selection contrast of the enhancement pixel 1, the selection contrast of the pattern pixel, and the selection contrast of the background pixel are described. ing.

As shown in FIG. 33, the final color information (AC_COLI) 113H contains the final color code of the emphasized pixel 2,
The fixed color code of the emphasized pixel 1, the fixed color code of the pattern pixel, the fixed color code of the background pixel, the fixed contrast of the emphasized pixel 2, the fixed contrast of the emphasized pixel 1, the fixed contrast of the pattern pixel, and the fixed contrast of the background pixel are described. ing.

Button Information Table (BTNIT) 113
As shown in FIG. 34, 36 pieces of button information (B
TNI) 113I,... Are described. In accordance with the description contents of the number of button groups (BTNGR_Ns), one group mode in which all 36 pieces of button information (BTNI) 113I,. ,
Can be used as three modes, that is, a two-group mode grouped by..., A three-group mode grouped by twelve button information (BTNI) 113I,. Since the description area of the button information (BTNI) 113I in each group mode is fixed, zero is described in all areas where no effective button information (BTNI) 113I exists. Button numbers (BTNI) in the order of description of the button information (BTNI) 113I in each button group
NN) is assigned from 1.

In the button group, the buttons for which the user number can be specified are from BTN_ # 1 to the number of the value described in NSBTN_Ns.

As shown in FIG. 34, the button information (BTNI) 113I contains the button position information (BTN_POS).
I) 113J, adjacent button position information (AJBTN_P
I) 113K and button command (BTN_CMD) 1
13L is described.

Button position information (BTN_POSI) 11
As shown in FIG. 35, the color numbers (1 to 3) used by the buttons and the display rectangular area on the video display screen are described in 3J. Button position information (BTN_POSI) 11
3J includes a button color number (BTN_COL) of the button.
N), the start X coordinate of the rectangular area where the button is displayed (St
art X-coordinate, End X-coordinate of the rectangular area in which the button is displayed (End X-coordinate)
nate), the start Y coordinate of the rectangular area where the button is displayed (Start Y-coordinate), and the end Y coordinate of the rectangular area where the button is displayed (End Y-coo d).
rdinate) and auto operation mode (Auto)
action mode) is described. The auto operation mode describes whether the selected state is not maintained, or the selected state or the fixed state is maintained.

Adjacent button position information (AJBYN_POS)
I) 113K describes the button numbers located in four directions, up, down, left, and right, and whether the target button has a selected state. The button having no selection state is a button that immediately transitions to the final state without moving to the selected state when the target button is moved. For example, an upper button number, a lower button number, a left button number, and a right button number are described. It corresponds to the instruction of the select key 5m.

Button Command (BTN_CMD) 113
L describes a command to be executed when the button is determined. According to this command, for example, a program for shifting to another selection screen or a program chain for reproducing a title is designated.

In the highlight information HLI, information for designating a position for synthesizing an image shot by the camera unit 3 used in the present invention may be described.

The contents of the HLI in the PCI are output from the PCI decoder 29 to the video reproducing unit 23. The content of the HLI, for example, an emphasized pixel, is output to the video reproduction unit 23 to indicate a synthesis area of the video captured by the camera unit 3.

Next, the normal reproducing operation of the above-mentioned reproducing apparatus will be described.

FIG. 36 is a flowchart when the reproducing operation is started. When the power is turned on, the system control unit 30 starts up a ROM program provided in advance, drives the disk motor 14, and starts reading data (step S1). First ISO
The data of the volume and the file structure part is read in accordance with -9660 or the like. The read data is temporarily stored in the memory of the system control unit 30. Thus, the system control unit 30 grasps the type of data on the optical disc, the recording position, and the like.

As a result, the system control unit 30
The video manager (VM)
G) and its manager information (VMG
Obtain I). VMGI has a video manager
Screw table (VMGI MAT)
Since various management information related to
What information is recorded on the disc based on the information
Can be displayed in a menu format
(Steps S2 and S3). And the finger from the user
Waiting for confirmation (step S4). This specification is an example
For example, a video title set is specified.

When a designation is made by an operation input from the user, reproduction of the designated video title set is started (step S5). If there is no designation from the user even after the lapse of the predetermined time, a predetermined video title set is reproduced (step S6). When the reproduction is completed, the process proceeds to an end step (steps S7 and S7).
8).

FIG. 37 is a flowchart further showing the operation when a video title set is designated.

When a video title set is designated, control data (video title set information VTSI) of the title set is read (step S11). As described above, this includes information on the program chain (PGC) and a menu for selecting the program chain. Therefore, the system control unit 30 can recognize the control information of the video title set (step S12). The user
Referring to the menu screen, a program chain is selected (step S13). In this case, the program chain may be automatically determined without the menu screen. When the program chain is determined by the selection, the cell playback order is determined according to the selected program chain,
Reproduction is performed (step S14). When the program chain is automatically determined, or when the selection information of the program chain is not input within a predetermined time, the reproduction is performed in a predetermined cell reproduction order (step S15).

Referring to FIG. 38, the operation of processing the sub-picture in sub-picture decoder 25 will be described.

When a sub-picture unit is constructed in the memory 62, decoding processing based on the PTS is managed. That is, the presentation time stamp (PT
S) is compared with the count value of the main system timer 70. If the count value of the main system timer 70 is larger than the presentation time stamp (PTS), the data unit is decoded and output. The decoded output data is supplied to the buffer 64 and the output control unit 6 under the control of the sequence control unit 66.
5 and displayed.

Here, the subsystem timer 71 sets the control unit 6 every time the sub-picture data unit is switched.
It is reset by the reset pulse from 1 and counts the system clock for the timer counter. That is, the subsystem timer 71 indicates the processing elapsed time of a single sub-picture data unit by a count value. The information indicating the elapsed time of the subsystem timer 71 is referred to by the sequence control unit 66.

In FIG. 38, a sub-picture pack (SP PCK) is taken into the memory 62 via the buffer 26. The pack is identified by the stream ID described in the packet header. The designated stream ID (sub stream ID) is input to the sub picture decoder control unit 61 via the system control unit responding to the user operation, and is stored in the register.

Among the packets captured in the buffer 26, the designated stream ID and the input substream ID
The packet in which is matched is to be captured. Then, the count value of the reference time of the main system timer 54 is given to the sub-picture decoder control unit 61 and compared with the system clock reference (SCR) of the packet captured in the buffer 26. The count value of the main system timer 54 is compared with the value of the SCR and the same S
The packet having the CR is stored in the memory 62 for unit construction. By the above processing, the memory 62
One or more sub-picture units (FIG. 22)
Reference) will be accumulated. When the sub-picture unit is constructed in the memory 62, decoding processing based on the PTS is managed.

The sub-picture unit header (SPUH) included in this sub-picture unit is referred to by the sub-picture decoder control section 61, and the size and address are recognized. As a result, the run-length-compressed data (PXD) is sent to the run-length decoder 63, and the display control sequence table (SP DCSQT)
Is sent to the sequence control unit 66.

At this point, the presentation time stamp (PTS) is compared with the count value of the main system timer 54. If the count value of the main system timer 54 is larger than the presentation time stamp (PTS), the run of the data unit is executed. Length decoding and output processing are performed. The decoded output data is stored in the buffer 6 under the control of the sequence controller 66.
4. Output via the output control unit 65 and displayed.

That is, the run length data (PXD)
Are decoded by the run-length decoder 63.
This decoding process is performed according to the rules described above. The decoded pixel data is stored in the buffer memory 64.
And wait for the output timing.

On the other hand, included in the sub-picture unit
Display control sequence table (SP SCQT)
The data is input to the can control unit 66 and analyzed. sequence
The control unit 66 includes a plurality of registers for holding various control commands.
It has a resistor 217. The sequence control unit 66
In response to the register command, the next pixel output
Output what color and / or contrast
To decide. This determination signal is given to the output control unit 65.
Can be The sequence control unit 66 includes a buffer memory
Readout timing of pixel data held in 64
Signals and addresses are also provided.

In the output control unit 65, the buffer memory 64
, And a color code and / or contrast data are added to the pixel data from the sequence control unit 66 in accordance with a command from the sequence control unit 66 and output. This output sub-picture is superimposed on the main video.

As described above, regarding the display of the sub-picture, the presentation time stamp (PTS) included in the sub-picture unit and the system timer 7 are included.
The timing signal is obtained from the comparator 69 when the output of 0 has a predetermined relationship. The decode processing of the run-length data is started based on this timing signal.

The display control of the sub-picture is executed or in a standby state according to the timing signal from the comparator 72. Sub time stamp extraction unit 64
Is the SP stored in the sequence control unit 66 DC
This is a register for holding SQ start time data. The comparator 72 outputs the output data from the subsystem timer 71 and each SP. The DCSQ start time data is successively compared, and when they match, a timing signal is output.

When the display control is performed, the use of the commands shown in FIGS. 21 and 22 will be further described.

In display control, the command SET D
AREA sets the display position and display area of the sub-picture, and the command SET The display color of the sub-picture is set by COLOR, and the command SET CO
The contrast of the sub-picture with respect to the main video is basically set by the NTR. These are basic commands.

Then, the display start timing command STA
After executing the DSP, another display control sequence DCSQ
To end display instruction STP Until the DSP is executed, the color and contrast switching command CHG is displayed during the display. While performing display control based on COLCON,
The decoding of the run-length compressed pixel data PXD is performed.

The sequence controller 66 sets the command SET
The display position and display area of the sub-picture are set by DAREA, and the command SET The display color of the sub-picture is set by COLOR, and the command SET CO
The contrast of the sub-picture with respect to the main picture is basically set by the NTR. Then, the display start timing instruction STA After executing the DSP, the display end timing instruction STP is executed in another display control sequence DCSQ. DSP
Is executed, the color and contrast switching command CHG is displayed during the display. Performs display control based on COLCON.

The outline of the reproducing process will be described with reference to FIGS.

FIG. 39A shows a decoded image of a main video signal compressed and recorded on the optical disk 10 and shows an image on a TV monitor screen.

FIG. 39B shows a decoded image of a PXD (pixel data) signal of the sub-picture signals compressed and recorded on the optical disk 10 in the same manner. The pixel data PXD is 2-bit information and can control four states of a background value / pixel value / first emphasis value / second emphasis value. The background value is a value in a case where only the main video signal is displayed with a normal contrast value of 0 and a mixing ratio of the main video signal and the sub video signal of 1: 0. Pixel values are usually used to display characters. The first or second emphasis value is used for, for example, a contour portion of a character displayed in a pexel value.

FIG. 39C shows the positions of buttons on the screen for highlight information. This is the decoded pixel data PXD
This is used for the user to select a menu by, for example, brightening a part of. The number of highlighted buttons is a maximum of 36 on one screen, and the user can select one of them. The figure shows an example in which there are two buttons. Each button has coordinate data, and color code data and contrast data can also be selected.

FIG. 39D shows a color and contrast switching command CHG in the sub-picture unit. Indicates the position of COLCON on the screen. This can be used for control such as automatically changing the color code data / contrast data of each pixel data PXD in the plurality of areas, changing the color of the karaoke playback lyrics and the like according to the song. Color and contrast switching command CHG The arbitrary number n of COLCON coordinates can be secured in the vertical direction and a maximum of 8 in the horizontal direction, and the contrast ratio / color code of each of a total of n * 8 rectangles can be independently specified.

FIG. 39E shows an area of the display position information in the sub-picture signal. The maximum size is almost the same as the TV screen. The display data includes display coordinate data, color code data of the display area, and contrast data. By comparing the horizontal coordinate value and the vertical coordinate value as the display coordinate data with the values of the horizontal counter and the vertical counter synchronized with the main video signal, the position on the TV screen can be determined. It also becomes time information, and can specify the read time of the compressed pixel data PXD.

Although not shown, the size of the reproduced pixel data PXD can be recorded beyond the size of the display area, and a read address is separately set at the time of reproduction. Data PX
Since D can be read, if the address value is changed, the pixel data PXD on the screen scrolls in the vertical direction.

In the optical disk 10, the optical disk 10 shown in FIG.
The time for controlling the pixel data PXD with the highlight information
The color and contrast switching command C of FIG.
HG When the time to control with COLCON is the same time
Stipulates that control with highlight information takes precedence.
You.

FIG. 39F shows an output video signal synthesized with the main video signal. The "5 stars" and "triangle" of the main video signal of FIG. 39 (A) and the "7" of the sub video signal of FIG. 39 (B).
This is an example in which the characters “star” and “Aieokakikuke” are mixed and displayed. The sub-picture signal “ABC” in FIG.
DEFGH "is not displayed because the color and contrast switching command CH of the sub-picture signal shown in FIG.
G This is an example in which the background value is controlled by COLCON. In the optical disc 10, a sub-picture signal (the above-described pixel data PXD, color and contrast switching command CHG
COLCON, display position information, etc.) and highlight information are different streams (another pack in MPEG).

Next, the configuration of the video reproducing unit 23 according to the first embodiment will be described with reference to FIG. When the video playback unit 23 having this configuration is used, the captured video as the external video signal from the camera unit 3, the pixel data of the main video and the sub video, and the composite video are shown in FIGS. It will be described using FIG.

[0207] As shown in FIG.
Horizontal / vertical counter 81, window logic unit 82, coordinate register 83, color code register 84, contrast register 8
5, a color palette register 86, a detection unit 87, a video signal mixing unit 90, and a video switch 91.

The horizontal / vertical counter 81 counts based on a horizontal / vertical synchronization signal synchronized with the main video signal from the video decoder 22, and this count value is output to the window logic unit 82.

The window logic section 82 has a horizontal / vertical counter 81
Is compared with the coordinate data from the coordinate register 83 and converted into time information. A time signal DSP relating to display coordinates as this time information
Is the time to decode the pixel data PXD, which is output to the sub-picture decoder control unit 61 in the sub-picture decoder 25. Highlight information HLI (or command data color and contrast switching command C)
HG Time information SEL on the coordinates of (COLCON)
Are the color code register 84 and the contrast register 85
(The display register or the highlight information HLI (or the command data color and contrast switching command CHG). COLCON).

The coordinate register 83 is provided for the PCI decoder 29.
The display area data in the highlight information, the display area data in the pixel data from the sub-picture decoder 25, and the display area data in the instruction data from the sub-picture decoder 25 are stored as coordinate data. The coordinate data is output to the window logic unit 82.

The color code register 84 stores the color conversion code data included in the pixel data from the sub-picture decoder 25, the color conversion code data included in the highlight information from the PCI decoder 29, and the sub-picture decoder 2
5 stores the color conversion code data included in the instruction data from step 5, and outputs the color conversion code data to the color palette register 86.

[0212] The contrast register 85 stores the contrast data included in the pixel data from the sub-picture decoder 25, the contrast data included in the highlight information from the PCI decoder 29, and the contrast data included in the command data from the sub-picture decoder 25. This contrast data is output to the video signal mixing section 90 as a mixing ratio.

The color pallet register 86 converts the color conversion code data from the color code register 84 into a color pallet signal. This color pallet signal is output to the video signal mixing section 90 as a sub-video signal.

[0214] The detecting section 87 is a sub-picture decoder 2
5 to detect the second emphasis value of the pixel data PXD from the pixel switch PXD.
1 is output.

[0215] The video signal mixing section 90 includes the video decoder 2
2 is mixed with the sub-video signal from the color palette register 86 at a mixing ratio from the contrast register 85, and this mixed video signal is output to the video switch 91.

[0216] The video switch 91 is
The mixed video signal from the camera unit 3 and the external video signal from the camera unit 3 are switched based on the switching signal from the detection unit 87 and output to the display unit 4 as a composite video output. While the switching signal is not supplied from the detection unit 87,
The mixed video signal is output from the video signal mixing unit 90, and while the switching signal is supplied from the detection unit 87, the camera unit 3
To output an external video signal.

The 2-bit 4-values of the pixel data PXD decoded by the sub-picture decoder 25 are as follows:
A contrast register 85 and a color code register 84 are selected. The selected 4 bits of the color code selects one of 16 types of the color palette register 84 and is output to the video signal mixing section 90. The 2-bit 4-value of the pixel data PXD also selects the contrast register 85. The main video signal and the color pallet signal are mixed by the video signal mixing unit 90 at one of the 16 selected 4-bit contrast values. When the mixed output signal level is expressed by an equation, the mixed video output = (1−α) * main video signal + α * color pallet signal. Here, α is (a normalized value of) the contrast value.

Further, the detection unit 8 detects one of four values of the pixel data PXD from the sub-picture decoder 25.
The mixed video output is switched to an external video signal and inserted by switching the video switch 91 while the detection signal is being supplied.

For example, a video insertion area of the sub-picture pixel data PXD described on the optical disc 10 (DVD video standard) is created with a second round emphasis value, and the sub-picture pixel data PXD of the sub-picture pixel data PXD described on the optical disc 10 is created. When the character string part is created with the pixel value (there is a background value in both), the second
The external video signal is not switched to the character string portion without the emphasis value.

That is, when synthesizing the external video shown in FIG. 41A with the main video shown in FIG.
When the sub-picture pixel data PXD as shown in (C) is prepared, as shown in FIG. 41 (D), a portion corresponding to PXD1 (second emphasis value) of the pixel data PXD is shifted from the main picture to the outside. Switching to video output, the external video is combined with the main video and sub-video.

That is, 1 of 4 states of the pixel data PXD
PXD1 circle is created using the two second emphasis values, and a character string is created using the pel value (there is a background value in both).
Upon detecting the emphasis value, the external video signal is switched and inserted.
The external video signal cannot be switched to the character string portion without the second emphasis value.

For example, the round part of the pixel data PXD2 is the first
If the value is an enhancement value, no detection is performed, and the triangular portion of the main image becomes the 6-star portion of the external video signal.

Further, two detection values, a first emphasis value and a second emphasis value, are prepared as the detection values of the detection section 87, and the system control section 30
If it is possible to change the detection value, the first emphasis value and the second
By changing the detection value between the two regions created at different positions with the emphasis value, the insertion position of the external video signal is switched. As long as the character string is created by the pexel value, it does not affect the switching of the external video signal.

In this case, when the detection by the detection section 87 is set to the first emphasis value, the position of the pixel data PXD2 circle is switched. As long as the character string is created by the pixel value, it does not affect the switching of the external video signal.

Next, the configuration of the video reproducing unit 23 according to the second embodiment will be described with reference to FIG. 43 (A) to 43 (D) of FIG. 43 show the captured video, the main video, the sub-video pixel data, and the composite video as the external video signal from the camera unit 3 when using the video playback unit 23 having this configuration. It will be described using FIG.
The same parts as those of the video reproducing unit 23 shown in FIG. 40 are denoted by the same reference numerals, and description thereof will be omitted.

That is, the detection unit 87 is removed, and the output of the color palette register 86, the external video signal from the camera unit 3, and the switching signal from the system control unit 30 are supplied to the video switch 92. The output, the main video signal from the video decoder 22 and the mixing ratio from the contrast register 85 are supplied to the video signal mixing section 93.

The video switch 92 switches between a color pallet signal from the color pallet register 86 and an external video signal from the camera section 3 based on a switching signal from the system control section 30 and outputs the same. Are output to the video signal mixing unit 93.

The video signal mixing section 93 is provided with the video decoder 2
2 and a sub-video signal or an external video signal from the video switch 92,
The mixed video signal is output to the display unit 4 as a composite video output.

That is, when synthesizing the external video shown in FIG. 43A with the main video shown in FIG.
When the pixel data PXD of the sub-picture as shown in FIG. 43C is prepared, as shown in FIG. 43D, the external picture is mixed in place of the sub-picture in the portion corresponding to the pixel data PXD, The external video is combined with the video.

That is, one of the four states of the pixel data PXD
Two second emphasis values are created by circles (there is a background value), and four states of the pixel data PXD are made to correspond to four types of contrasts (mixing ratios), and then the external video signals are mixed and inserted with the contrast signals. . The 6-star portion of the external video signal is mixed with the triangular portion of the main video.

In this case, the pixel value other than the background value, the first
The emphasis value and the second emphasis value are detected and switched to the external video signal, and the mixing ratio between the external video signal and the first video signal is controlled by these four values.

In this example, pixel data P such as another character string
Although coexistence with XD is not possible, there is a restriction on the production of the optical disk 10 (DVD video), but there is an advantage that it can be easily achieved.

Next, the configuration of the video reproduction unit 23 according to the third embodiment will be described with reference to FIG. When the video playback unit 23 having this configuration is used, the captured video as the external video signal from the camera unit 3, the highlight information of the main video and the sub video, and the composite video are shown in FIGS. This will be described with reference to FIG. The same parts as those of the video reproducing unit 23 shown in FIG. 40 are denoted by the same reference numerals, and description thereof will be omitted.

That is, the detection section 87 is removed, and the time information SEL relating to the coordinates of the highlight information HLI from the window logic section 82 is output to the video switch 91 as a switching signal.

Accordingly, video switch 91 switches between the mixed video signal from video signal mixing section 90 and the external video signal from camera section 3 based on the switching signal (time information SEL) from window logic section 82. And outputs it to the display unit 4 as a composite video output.

That is, when synthesizing the external video shown in FIG. 45A with the main video shown in FIG.
When the sub-picture highlight information HLI as shown in (C) is prepared, as shown in FIG. 45 (D), the portion corresponding to HL-2 of the highlight information HLI is changed from the main picture to the external picture of the external picture. The mode is switched to the output, and the external video is combined with the main video and the sub video.

This is an example in which the coordinate data of the highlight information HLI is used to compare with the coordinates of the main image to switch and insert an external image. The 6-star portion of the external video signal is inserted into the triangular portion of the main video. FIGS. 45A to 45D show examples in which the button 1 is inserted into the coordinate area selected by the button 2. When the user selects the button 1, the insertion position is the button 1
Becomes In this case, the external video signal to be switched is a rectangular area.

In the optical disk 10 (DVD video standard), the highlight information HLI can have a maximum of 36 buttons, which can be selected by the user, and the positions of the buttons on the screen are different. Therefore, there is a merit that it can be inserted into the coordinate area of the button selected by the user.

In this case, an external video is inserted corresponding to the coordinate area of the highlight information HLI.

Next, the configuration of the video reproducing unit 23 according to the fourth embodiment will be described with reference to FIG. When the video playback unit 23 having this configuration is used, the captured video, the main video, the highlight information of the sub video, and the composite video as the external video signal from the camera unit 3 are shown in FIGS. This will be described with reference to FIG. The same parts as those of the video reproducing unit 23 shown in FIG. 40 are denoted by the same reference numerals, and description thereof will be omitted.

That is, instead of the detector 87 for detecting the second enhancement value of the pixel data PXD, a detector for detecting a contrast value of a highlight area corresponding to one of the four second enhancement values of the pixel data PXD. A contrast value from the contrast register 85 is output to the detection unit 94, and a detection output of the detection unit 94 is output to the video switch 91 as a switching signal. In this case, the pixel data PX
The content is created by changing the contrast value of the display area and the contrast value of the highlight area corresponding to one of the second emphasis values in the four states D and writing them on the optical disk (DVD video) 10.

Thus, the video switch 91 switches between the mixed video signal from the video signal mixing section 90 and the external video signal from the camera section 3 based on the switching signal from the detection section 94, and displays it as a composite video output. Output to section 4.

That is, when the external video shown in FIG. 47A is synthesized with the main video shown in FIG.
When the sub-picture highlight information HLI as shown in FIG. 47C is prepared, as shown in FIG. 47D, a portion corresponding to HL2 of the highlight information HLI and the second emphasis value of the pixel data PXD. Is switched from the main video to the output of the external video, and the external video is combined with the main video and the sub video.

Thus, the highlight area and the graphic area created by the second emphasis value of the pixel data PXD are switched to the external video signal.

That is, 1 of 4 states of the pixel data PXD
Two second emphasis values are created by circles, a character string is created by a pexel value (there is a background value in both), a second emphasis value is detected, an area 1 signal is obtained, and coordinate data of highlight information is obtained. Area 2 by comparing with the coordinates of the main image using area 1 and area 2
Is switched to the external video signal and inserted. The 6-star portion of the external video signal is inserted into the triangular portion of the main video.

FIGS. 47A to 47D show examples in which the button 2 is inserted into the coordinate area selected by the button 2. When the user selects the button 1, the insertion position is the button 1. The character string can use four values.

In the optical disk 10 (DVD video standard), the button of the highlight information HLI can be selected by the user, and by changing the position of each button, the position of the pixel data PXD can be switched to a different position. In the duet, the female side can insert a heart-shaped insertion and the male side can insert a spade-type external video signal.

In this case, when a predetermined contrast in the pixel data PXD area is detected in the coordinate area of the highlight information HLI, an external image is inserted.

Next, the configuration of the video reproducing unit 23 according to the fifth embodiment will be described with reference to FIG. When the video playback unit 23 having this configuration is used, the captured video as the external video signal from the camera unit 3, the highlight information of the main video and the sub video, and the composite video are shown in FIGS. This will be described with reference to FIG. The same parts as those of the video reproducing unit 23 shown in FIG. 40 are denoted by the same reference numerals, and description thereof will be omitted.

That is, instead of the video switch 91,
A video signal synthesizing unit 95 is provided.
The mixed video signal from the video signal mixing section 90, the external video signal from the camera section 3, the mixing ratio from the contrast register 85, and the time information SEL (switching signal) relating to the coordinates of the highlight information HLI from the window logic section 82. Supplied.

As a result, the video signal synthesizing section 95 outputs the mixed video output composed of the main video and the sub video and the external video only in the rectangular area of the highlight coordinates of the selected button using the contrast value of the pixel data PXD. Are mixed and output to the display unit 4 as a video output.

The video signal mixing section 95 uses the contrast value from the contrast register 85 as the mixing ratio based on the time information SEL from the window logic section 82 with respect to the highlight coordinate area, and sets the mixing ratio to 0 outside the area. . That is, the mixture of the highlight areas becomes not the main video signal and the sub video signal but the main video signal and the external video signal.

That is, when synthesizing the external video shown in FIG. 49A with the main video shown in FIG.
When the highlight information HLI of the sub-picture as shown in (C) is prepared, as shown in FIG. 49 (D), it corresponds to the button HL2 of the highlight information HLI and the second emphasis value of the pixel data PXD. An external image is mixed in place of the sub-image instead of the sub-image, and the external image is combined with the main image and the sub-image.

That is, a circle and a character string are created in the four states of the pixel data PXD, and the contrast and coordinate values of the highlight information HLI are changed so as to change the contrast between the circle and the character string. Is detected, the round area is switched to the external video signal and inserted. For example, a circle is formed with the second emphasis value, the contrast value of the button HL1 is set to 15, and the contrast value of the button HL2 is set to 14, and 15 is detected, and the triangular portion of the main image is a 6-star portion of the external video signal. Is inserted.

When the contrast value is selected, the button HL2 is mixed with the main video signal. If a contrast value other than 14 or 15 is used for the character string portion, no external video signal is mixed in the character string portion, and the color of the lyrics can be changed.

In this case, an external image is mixed in the coordinate area of the highlight information HLI.

Next, the configuration of the video reproducing unit 23 according to the sixth embodiment will be described with reference to FIG. When the video playback unit 23 having this configuration is used, the captured video, the main video, the highlight information of the sub video, and the composite video as the external video signal from the camera unit 3 are shown in FIGS. This will be described with reference to FIG. The same parts as those of the video reproducing unit 23 shown in FIG. 40 are denoted by the same reference numerals, and description thereof will be omitted.

That is, instead of the detecting section 87 for detecting the second emphasis value of the pixel data PXD, the color code (predetermined value) of the color palette corresponding to one of the four emphasis values of the pixel data PXD is detected. And a color code from the color code register 84 is output to the detection unit 96.
The detection output of the detection unit 96 is used as the switching signal as the video switch 9.
1 is output. In this case, the content is created by describing the color code of the highlight area corresponding to one of the four second states of the pixel data PXD as a predetermined value on the optical disk (DVD video) 10. in this case,
The contrast is no longer restricted, and content creation similar to that of the related art is possible.

Thus, the video switch 91 switches the mixed video signal from the video signal mixing section 90 and the external video signal from the camera section 3 on the basis of the switching signal from the detection section 96 and displays the resultant as a composite video output. Output to section 4.

That is, when synthesizing the external video shown in FIG. 51A with the main video shown in FIG.
When the sub-picture highlight information HLI as shown in (C) is prepared, as shown in FIG. 51 (D), it corresponds to the button HL2 of the highlight information HLI and the second emphasis value of the pixel data PXD. The part is switched from the main video to the output of the external video, and the external video is combined with the main video and the sub video.

That is, a circle and a character string are created in the four states of the pixel data PXD, and the area other than the coordinate area of the button 2 of the highlight information HLI and the background value of the pixel data PXD is detected, and the AND area between the area and the area is detected. Mix with video signal. The 6-star portion of the external video signal is mixed with the triangular portion of the main video.
No external video signal is mixed in the character string portion.

In FIGS. 51A to 51D, the button HL2
This is an example in which the coordinate area selected in step (1) is mixed, and when the user selects the button HL1, the mixed position becomes the button HL1.

In this case, when a predetermined color code value in the area of the pixel data PXD is detected in the coordinate area of the highlight information HLI, an external image is inserted.

Next, the configuration of the video reproducing unit 23 according to the seventh embodiment will be described with reference to FIG. The same parts as those of the video reproducing unit 23 shown in FIG. 40 are denoted by the same reference numerals, and description thereof will be omitted.

In other words, the inverter circuit 97 for removing the detection unit 87 and inverting the time information DSP relating to the display coordinates from the window logic unit 82, the output of the inverter circuit 97, and the coordinates of the highlight information HLI from the window logic unit 82. And an AND circuit 98 for ANDing the time information SEL with respect to the time information SEL on the coordinates of the highlight information HLI from the window logic unit 82 from the AND circuit 98 and the time information on the display coordinates from the window logic unit 82 The logical product signal of the inverted signal of the DSP is used as a switching signal as the video switch 9.
1 is output.

Thus, the video switch 91 switches between the mixed video signal from the video signal mixing section 90 and the external video signal from the camera section 3 based on the switching signal from the AND circuit 98, and displays it as a composite video output. Output to section 4.

In this case, since the highlight area is not included in the display area, this can prevent the restriction on the production of the sub-picture data of the conventional content. Thus, the production of the character content using the pixel data PXD is the same as in the related art.

In this case, an external image is inserted outside the display area in the coordinate area of the highlight information HLI.

Next, the configuration of the video reproducing unit 23 according to the eighth embodiment will be described with reference to FIG. Since the configuration is almost the same as that of FIG. 44 shown in the third embodiment, the description is omitted. However, instead of the highlight information from the PCI decoder 29, the instruction data from the sub-picture decoder 25 is
The data is supplied to a coordinate register 83, a color code register 84, and a contrast register 85.

The color and color in the instruction data of the sub-picture unit
And contrast switching command CHG Area of COLCON
Is an arbitrary number in the vertical direction and the maximum in the horizontal direction.
Eight rectangular areas can be set simultaneously (on the same screen)
You. These coordinate values, contrast values, and color code values are
At the time of reproduction when producing contents of the optical disk (DVD) 10
It is possible to change over time.

In this embodiment, a plurality of (separated) color and contrast switching commands CH are displayed in the display area.
G All areas of the COLCON area are switched to external video signals.

In this case, the color and contrast switching command CHG An external image is inserted when the coordinate area of COLCON is detected.

Next, the configuration of the video reproducing unit 23 according to the ninth embodiment will be described with reference to FIG. Since the configuration is almost the same as that of FIG. 46 shown in the fourth embodiment, the description is omitted. However, instead of the highlight information from the PCI decoder 29, the instruction data from the sub-picture decoder 25 is
The data is supplied to a coordinate register 83, a color code register 84, and a contrast register 85.

That is, a character string portion and a figure of an insertion area are created by the four values of the pixel data PXD, and a color and contrast switching command CHG is generated separately from the contrast and color code of the display coordinate area. The contrast and the coordinate value of the information of COLCON are changed, and it is detected that it is a graphic part of the synthesis area, and the graphic area of the pixel data PXD is switched to an external video signal and inserted.

With respect to the four output values of the pixel data PXD, the contrast value of the display area is as follows: pixel data PXD = background value → contrast value = one of 16 bits of 4 bits (example = 0) pixel data PXD = pexel value → contrast Value = 1 value among 16 bits of 4 bits (example = 13) Pixel data PXD = first enhancement value → contrast value = 1 value of 16 bits of 4 bits (example = 8) Pixel data PXD = second enhancement value → contrast Value = 1 value out of 4 bits and 16 values (example = 7)

[0276] For example, a circle figure is created at the insertion portion with the second emphasis value, and the color and contrast switching command CHG is generated.
If the contrast value is set to 15 and the detection level is set to 15 in the COLCON information, the color and contrast switching command CHG The circular area of the rectangular area specified by COLCON has a contrast of 15,
A detection output is obtained by the detection unit 94 and the video switch 91
Is switched to output an external video signal.

Another color and contrast switching command CHG If the content is produced with the contrast value of the COLCON area being other than 15, the area can be controlled as before. For example, in the string part,
If the second emphasis value is not used, the character string can be changed temporally, and the color of the lyrics can be changed.

In this case, the color and contrast switching command CHG Pixel data P in the coordinate area of COLCON
When a predetermined contrast in the XD area is detected, an external image is inserted.

Next, the configuration of the video reproducing unit 23 according to the tenth embodiment will be described with reference to FIG. Since the configuration is almost the same as that of FIG. 50 shown in the sixth embodiment, the description is omitted.
However, instruction data from the sub-picture decoder 25 is supplied to the coordinate register 83, the color code register 84, and the contrast register 85 instead of the highlight information from the PCI decoder 29.

In this case, the detection of the contrast value is changed to the detection of the color code in the ninth embodiment, and the detection is performed similarly.

In this case, the color and contrast switching command CHG Pixel data P in the coordinate area of COLCON
An external image is inserted when a predetermined color code value in the XD area is detected.

Next, the configuration of the video reproducing unit 23 according to the eleventh embodiment will be described with reference to FIG. The same parts as those of the video reproducing unit 23 of the sixth embodiment shown in FIG. 50 are denoted by the same reference numerals, and description thereof will be omitted. However, the PCI decoder 29
The instruction data from the sub-picture decoder 25 is supplied to the coordinate register 83, the color code register 84, and the contrast register 85 in place of the highlight information from.

That is, instead of the video switch 91,
A video signal synthesizing unit 99 is provided.
The mixed video signal from the video signal mixing section 90, the external video signal from the camera section 3, the mixing ratio from the contrast register 85, and the switching signal from the detection section 96 are supplied.

As a result, the video signal synthesizing section 99 outputs the color and contrast switching command CHG. COLCON
The external video is mixed with the mixed video output composed of the main video and the sub video based on the switching signal from the detection unit 96 at the contrast value of, and is output to the display unit 4 as a video output.

That is, when the switching signal is not supplied from the detecting section 96, the video signal synthesizing section 99 outputs a mixed video composed of the main video and the sub-video. When the switching signal is supplied from the detection unit 96, the video signal synthesizing unit 99 outputs a mixed video obtained by mixing the external video with the mixed video including the main video and the sub-video at the mixing ratio supplied.

In this case, the color and contrast switching command CHG Pixel data P in the coordinate area of COLCON
When a predetermined color code value in the XD area is detected, the external image is mixed.

Next, the configuration of the video reproducing unit 23 according to the twelfth embodiment will be described with reference to FIG. The same parts as those of the video reproducing unit 23 of the seventh embodiment shown in FIG. 52 are denoted by the same reference numerals, and description thereof will be omitted. However, the PCI decoder 29
The instruction data from the sub-picture decoder 25 is supplied to the coordinate register 83, the color code register 84, and the contrast register 85 in place of the highlight information from.

In this case, while the image is switched to the display area for the highlight information in the seventh embodiment, the color and contrast switching command CHG of the instruction code of the sub-picture unit is changed. The image is switched to the COLCON area, and is similarly executed.

In this case, the color and contrast switching command CHG An external image is inserted outside the display area in the coordinate area of COLCON.

Next, the configuration of the video reproduction unit 23 according to the thirteenth embodiment will be described with reference to FIG. Since the configuration is almost the same as that of FIG. 54 shown in the ninth embodiment, the description is omitted.
However, in the case of the ninth embodiment, the time information DSP from the window logic unit 82 is output as the decoding time of the sub-picture decoder 25. In this embodiment, however, the OR circuit 100 is provided, The time information DSP or the time information SEL from the logic unit 82 may be output as the decoding time of the sub-picture decoder 25.

In this case, the color and contrast switching command CHG An external image is inserted outside the display area in the coordinate area of COLCON.

Next, the configuration of the video reproducing unit 23 according to the fourteenth embodiment will be described with reference to FIG. Since the configuration is almost the same as that of FIG. 56 shown in the eleventh embodiment, the description is omitted. However, in the case of the eleventh embodiment, the window logic unit 8
2 from the sub-picture decoder 25
However, in the case of this embodiment, an OR circuit 101 is provided so that the time information DSP or the time information SEL from the window logic unit 82 is output as the decoding time of the sub-picture decoder 25. May be.

In this case, the color and contrast switching command CHG An external image is mixed outside the display area in the COLCON coordinate area.

The effects of the respective logics that can take coordinates on the various embodiments described above will be described with reference to FIG.

1) The shape of the composite portion is a square shape or an arbitrary shape.

2) The synthesized part is composed of the first video (main video) and the second video.
Switching of video (external video) or the composite part is first
Mixing of video (main video) and second video (external video).

3) The user can select or not select one of the positions prepared by the user for the position of the combining part.

4) The position of the synthesized part can be automatically changed in a predetermined time, or cannot be changed.

5) Compatibility with this functionless set
Or not.

The above 1) has an arbitrary shape if the pixel data PXD is used. 2) can be mixed if there is pixel data PXD. 3) can be selected by using the highlight information HLI. 4) is a color and contrast switching command CHG If you use COLCON, you can control the position, color and display / non-display at a specified time. 5) is possible with and without restrictions on content production.

As described above, it is possible to insert an external video signal from a camera or the like into a reproduced video signal from a DVD simply by adding a small circuit to the reproduction process without changing the current DVD video standard. , Can be mixed. Further, by changing the standard of the sub-picture signal of the DVD slightly, compatibility is completely maintained, and the same function can be realized.

[0302] In the above embodiment, insertion and mixing of an external video signal with a sub-video signal attached to a main video signal for DVD reproduction has been described as an example. However, the same processing can be performed in broadcasting and communication. Not to mention. Further, although the external video signal is a video signal of a camera-captured video, a still image captured electronically may be used, and the external video signal to be inserted and synthesized is not specified.

[0303]

As described above, according to the present invention,
When an attempt is made to combine a reproduced image reproduced from a recording medium with an image or the like captured by an image pickup device and display it on one screen,
It is possible to provide an image synthesizing apparatus capable of easily selecting an insertion position of a captured image and the like and synthesizing.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an image synthesizing apparatus according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram of a volume space which is a logical format recorded on an optical disc.

FIG. 3 Video manager (V) in volume space
FIG. 2 is an explanatory diagram showing the structure of a video title set (MG) and a video title set (VTS).

FIG. 4 is an explanatory diagram showing a relationship between a video object set (VOBS) and a cell, and the contents of the cell.

FIG. 5 is an explanatory diagram showing the relationship between video objects and cells.

FIG. 6 shows a cell (C) by a program chain (PGC).
FIG. 11 is an explanatory diagram showing an example in which the reproduction order is controlled for the “.ells”).

FIG. 7 is an explanatory diagram of a video title set information (VTSI) in a video title set (VTS).

FIG. 8 is a diagram showing a configuration example of one pack and a packet.

FIG. 9 shows a navigation pack (NV). FIG.

FIG. 10 is an explanatory diagram of general information of picture control information (PCI).

FIG. 11 is an explanatory diagram of a video object unit.

FIG. 12 is an explanatory diagram of an audio stream.

FIG. 13 is an explanatory diagram of a sub-picture unit.

FIG. 14 is an explanatory diagram of a sub-picture unit.

FIG. 15 is an explanatory diagram of a sub-picture unit.

FIG. 16 is an explanatory diagram showing a continuous configuration of sub-picture units.

FIG. 17 is an explanatory diagram showing display timing of a sub-picture unit.

FIG. 18 is an explanatory diagram showing a header configuration of a sub-picture unit.

FIG. 19 is an explanatory diagram of a sub-picture display control sequence table.

FIG. 20 is an explanatory diagram of a sub-picture display control sequence table.

FIG. 21 is an explanatory diagram of a sub-picture display control command.

FIG. 22 is an explanatory diagram of a sub-picture display control command.

FIG. 23 is an explanatory diagram of the contents of a sub-picture display control command.

FIG. 24 is an explanatory diagram of a run-length compression rule.

FIG. 25 is an explanatory diagram showing an example of run-length compressed data.

FIG. 26 is a diagram showing a validity period of highlight information for each sub-video stream in a reproduction period of one sub-video unit.

FIG. 27 is an exemplary view for explaining a video, a sub-picture, highlight information, and a synthesized image obtained by synthesizing them;

FIG. 28 is a view showing parameters and contents of highlight information (HLI) in reproduction control information (PCI).

FIG. 29 is a view for explaining the contents of highlight information (HLI).

FIG. 30 is a view showing parameters and contents of highlight generation information (HL_GI) in highlight information (HLI).

FIG. 31 is a view showing the configuration of a button color information table (BTN_COLIT) in highlight information (HLI).

FIG. 32 is a diagram showing the details of the description of the selection color information (SL_COLI).

FIG. 33 is a view showing details of the description content of fixed color information (AC_COLI).

FIG. 34 is a view showing the configuration of a button information table (BTNI) in highlight information (HLI).

FIG. 35 is a diagram showing details of the description content of button position information (BTN_POSI) in the button information (BTNI).

FIG. 36 is a flowchart showing a menu playback operation of the playback device.

FIG. 37 is a flowchart showing a title playback operation of the playback device.

FIG. 38 is a diagram showing a configuration of a sub-picture decoder of the playback device.

FIG. 39 is a view for explaining an outline of a reproducing process.

FIG. 40 is a block diagram illustrating a configuration of a video reproduction unit according to the first embodiment.

FIG. 41 is a diagram for describing pixel data of a captured video, a main video, and a sub video and a composite video according to the first embodiment.

FIG. 42 is a block diagram illustrating a configuration of a video playback unit according to the second embodiment.

FIG. 43 is a diagram for explaining pixel data of a captured video, a main video, and a sub video and a composite video according to the second embodiment;

FIG. 44 is a block diagram illustrating a configuration of a video playback unit according to the third embodiment.

FIG. 45 is a view for explaining highlight information of a captured video, a main video, and a sub video and a composite video according to the third embodiment;

FIG. 46 is a block diagram illustrating a configuration of a video playback unit according to the fourth embodiment.

FIG. 47 is a view for explaining highlight information of a captured video, a main video, and a sub video and a composite video according to the fourth embodiment;

FIG. 48 is a block diagram illustrating a configuration of a video playback unit according to the fifth embodiment.

FIG. 49 is an exemplary view for explaining highlight information of a captured video, a main video, and a sub video and a composite video according to the fifth embodiment;

FIG. 50 is a block diagram showing a configuration of a video playback unit according to the sixth embodiment.

FIG. 51 is a diagram illustrating highlight information of a captured video, a main video, and a sub video and a composite video according to the sixth embodiment;

FIG. 52 is a block diagram illustrating a configuration of a video playback unit according to the seventh embodiment.

FIG. 53 is a block diagram showing a configuration of a video reproduction unit according to the eighth embodiment.

FIG. 54 is a block diagram showing a configuration of a video reproduction unit according to the ninth embodiment.

FIG. 55 is a block diagram showing a configuration of a video playback unit according to the tenth embodiment.

FIG. 56 is a block diagram showing a configuration of a video reproduction unit according to the eleventh embodiment.

FIG. 57 is a block diagram showing a configuration of a video reproducing unit according to the twelfth embodiment.

FIG. 58 is a block diagram showing a configuration of a video reproducing unit according to the thirteenth embodiment.

FIG. 59 is a block diagram showing the configuration of a video playback unit according to the fourteenth embodiment.

FIG. 60 is a view for explaining effects of various logics that can take coordinates with respect to various embodiments.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Optical disk reproduction apparatus 2 ... Operation part 3 ... Camera part 4 ... Display part 5 ... Audio part 10 ... Optical disk 22 ... Video decoder 23 ... Video reproduction part 25 ... Sub picture decoder 27 ... Audio decoder 29 ... PCI decoder 30 ... System control Unit 81 horizontal / vertical counter 82 window logic unit 83 coordinate register 84 color code register 85 contrast register 86 color pallet register 87 detection unit 90 video signal mixing unit 91 video switch

Claims (20)

[Claims] A first supply unit for supplying an image from an external device; and a second supply unit for supplying a main image and a sub-image attached to the main image.
Supply means, a generation means for generating a mixed image of the main image and the sub-image supplied by the second supply means, a detection means for detecting a specific value in the sub-image, When a specific value in the video is detected,
When the supply image from the first supply unit is output and the specific value in the sub-image is not detected by the detection unit, the mixed image is output from the generation unit, so that the main image and the sub-image are output. Output means for outputting a composite image in which the supply image is combined with the mixed image of (i) and (ii). 2. The image according to claim 1, wherein the second supply unit is configured to supply a main image transmitted by broadcasting or communication and a sub-image attached to the main image. Synthesizer. 3. The apparatus according to claim 2, wherein said second supply means supplies a main video reproduced from a recording medium and a sub-video accompanying the main video and reproduced together with the main video. 2. The image synthesizing device according to 1. 4. The image synthesizing apparatus according to claim 1, wherein the specific value in the sub-picture detected by the detecting means is a specific value of pixel data of the sub-picture. 5. The image synthesizing apparatus according to claim 1, wherein the specific value in the sub-picture detected by the detection means is a second enhancement value of pixel data of the sub-picture. 6. The image according to claim 1, wherein the specific value in the sub-picture detected by the detection means is a first emphasis value or a second emphasis value of the pixel data of the sub-picture. Synthesizer. 7. The image synthesizing apparatus according to claim 1, wherein the specific value in the sub-picture detected by the detecting means is a coordinate value of highlight information of the sub-picture. 8. The image synthesizing apparatus according to claim 1, wherein the specific value in the sub-picture detected by the detection means is a contrast value of highlight information of the sub-picture. 9. The image synthesizing apparatus according to claim 1, wherein the specific value in the sub-picture detected by the detection means is a color code value of highlight information of the sub-picture. 10. A specific value in a sub-picture detected by the detecting means is a coordinate value of highlight information added to pixel data of the sub-picture, and is a mixed value determined by a contrast value of the highlight information. 2. The image synthesizing apparatus according to claim 1, wherein a composite image of a supply image from the first supply unit and a main image from the second supply unit is output in a ratio. 11. The image synthesizing apparatus according to claim 1, wherein the specific value in the sub-picture detected by the detecting means is a coordinate value of a display command added to pixel data of the sub-picture. . 12. The image synthesizing apparatus according to claim 1, wherein the specific value in the sub-picture detected by the detecting means is a contrast value of a display command added to pixel data of the sub-picture. . 13. The image composition according to claim 1, wherein the specific value in the sub-picture detected by the detecting means is a color code value of a display command added to pixel data of the sub-picture. apparatus. 14. A specific value in a sub-picture detected by the detection means is a color code value of a display command given to pixel data of the sub-picture, and a mixing ratio determined by a contrast value of the display command. 2. The image synthesizing apparatus according to claim 1, wherein the image synthesizing unit outputs a composite image of a supply image from the first supply unit and a main video from the second supply unit. 15. A first supply unit for supplying an image from an external device, a second supply unit for supplying a main image and a sub-image attached to the main image, and a supply unit for supplying an image. Determining means for determining a mixing ratio between a supply image from the first supply means and a main image from the second supply means according to a specific value of the sub-picture to be obtained; An image synthesizing apparatus, comprising: an output unit that outputs a composite image of a supply image from the first supply unit and a main video from the second supply unit. 16. The apparatus according to claim 15, wherein said second supply means supplies a main image transmitted by broadcasting or communication and a sub-image attached to said main image.
An image synthesizing apparatus according to claim 1. 17. The apparatus according to claim 17, wherein said second supply means supplies a main video reproduced from a recording medium and a sub-video accompanying the main video and reproduced together with the main video. 16. The image synthesizing device according to 15. 18. A first supply unit for supplying an image from an external device, and a second supply unit for supplying a main image and a sub-image attached to the main image.
And a mixing ratio of a supply image from the first supply unit and a main image from the second supply unit, or a mixing ratio of the main image from the second supply unit, based on a specific value of the sub-image supplied by the second supply unit. Determining means for determining a mixing ratio between the sub-image from the second supplying means and the main image from the second supplying means; and supplying from the first supplying means at a mixing ratio determined by the determining means. Outputting a mixed image of the image and the main image from the second supply unit, or outputting the mixed image of the main image from the second supply unit and the second supply unit at a mixture ratio determined by the determination unit Output means for outputting a composite image of the main video and the supply image by outputting a mixed image of the main video and the sub-video, or outputting a composite image of the main video and the sub-video. Synthesizer. 19. The apparatus according to claim 18, wherein said second supply means supplies a main image transmitted by broadcasting or communication and a sub-image attached to said main image.
An image synthesizing apparatus according to claim 1. 20. The apparatus according to claim 20, wherein said second supply means supplies a main video reproduced from a recording medium and a sub-video accompanying the main video and reproduced together with the main video. 19. The image synthesizing device according to 18.