JP2002300602A - Window-type image pickup/display device and two-way communication method using the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a window-type image pickup/display device which makes a viewer feel to see objects and persons which are located at a distant place through a window. SOLUTION: A window-type image pickup/display device is equipped with an image pickup/display panel which has a large number of image pickup elements and image display elements which are arranged in a uniform distribution through the surface of a plane board, where the image pickup elements detect at least the brightness and direction of incident light, and the image display elements project light whose brightness is proportional to that of the incident light in the direction in which it seems that the incident light travels rectilinearly from the display elements; an image interface circuit which generates image signals on the basis of image information received through the large number of image pickup elements arranged on the image pickup/display panel, and image information fed to a large number of display elements on the basis of image signals received through other image pickup/display panels elements; and a signal transmitting-receiving means which transmits signals sent from the image interface circuit to a transmission circuit, and sends image signals received from the transmission circuit to the image interface.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a window-shaped imaging and display device, and more particularly, to a window-shaped device having both an imaging function and a display function. This device can be installed in both two remote spaces and communicate with each other to create the effect of making the two spaces feel as if they were connected to both sides of the device. Can be used.

[0002]

2. Description of the Related Art Conventionally, attempts have been made to hang a thin display panel on a wall of a room so as to make it appear that an object actually exists behind the display panel. The reality was poor.

[0003] In order to increase the sense of reality, it is conceivable to project a stereoscopic image. However, since these images are all images displaying images taken from a certain direction by a camera, a person who looks at the display panel can view the images. It is not possible to see different images depending on the position and direction of the camera. You only see what was shot with the camera.

[0004] There are also stereoscopic image display devices such as a stereoscopic vision system and a stereoscopic television as shown in FIG. 21. These devices are designed to make an image of a subject stand out from a wall and to be seen. There is a space in the back of the display panel, and the effect that the space where the viewer is located through the display panel and the space behind the display panel seem to be connected is poor. In addition, this type of device has a limited range of positions of a viewer who can obtain stereoscopic vision. This kind of image is unnatural, so if you watch it for a long time, you will be tired mentally.

In a conventional case where a videophone is constructed using a camera as an image pickup device and a flat display panel as a display device, as shown in FIG. 22, a camera and a display device are provided for each of two terminals for talking. Then, an image taken by the camera is sent to the other party, and the image sent from the other party is displayed on his / her own display device.

[0006] Incidentally, since the talker talks while looking at the other party's face, he speaks while keeping his eyes on the display screen. For this reason, since the display device is located in front of the speaker and there is no space for placing the camera, the camera is installed above the display screen or the like. As a result, the image of the caller displayed on the screen becomes an image projected from above, that is, an image in a face-down state, and the eyes of the callers do not match.

[0007] In addition, there is a TV telephone conference system in which a conversation is made between two places where a plurality of participants are present, as shown in FIG. 23, instead of a one-to-one call as described above. The above-mentioned video telephone is extended so that it can be used by a plurality of persons, and an image taken by a camera on a transmitting side is transmitted to a receiving side and is displayed on a display panel on the receiving side. Therefore, the image viewed on the receiving side is an image captured by the camera on the transmitting side, and different images are not seen depending on the angle of the viewer on the receiving side. If different,
It is about the difference in the outline of the screen seen on the receiving side.

[0008] In this TV telephone conference system, the conference participant is looking at the display screen, and the camera is installed at an appropriate place outside the display screen. The line of sight with the person is unnatural.

As a video telephone system improved from this, a system as shown in FIG. 24 has been developed. In this system, a half mirror is placed in front of the display device, and when the speaker looks at the listener's face reflected on the display device, the light coming from the speaker's face is separated by the half mirror and guided to the camera to shoot It is.

[0010] By doing so, the callers can talk with each other while keeping their eyes on each other. However, since a half mirror must be provided, there is a disadvantage that the apparatus becomes complicated and large. Further, since the direction of the half mirror is limited as shown in the figure, it functions well when viewed from the center of the display screen, but does not exhibit the expected function when viewed from the oblique side of the screen.

As a device replacing the above-mentioned display device using a half mirror, an image pickup plane provided with a large number of image pickup devices having an image pickup function and an image display plane provided with a large number of display elements having a display function are planarly overlapped. A combined and integrated imaging and display device has been developed and is described in Japanese Patent Application Laid-Open No. 1-280978.

[0012] The above-mentioned image pickup and display device comprises an image pickup panel in which image pickup elements are arranged in a matrix as shown in FIG. 25 and an image display panel in which display elements are similarly arranged in a matrix.
The position in the horizontal direction is adjusted so that the image pickup element and the display element do not overlap in the optical path direction, and are superimposed to have a structure as shown in a sectional view in FIG. In this example, the imaging plane body is formed on the image display plane body, and light incident from the outside is converged by the converging lens body and projected on the imaging element. Light transmitted through a display element such as a liquid crystal, which is formed by an optical switch, passes through between image pickup elements and is emitted.

In this imaging display device, since the image display surface and the imaging surface are uniformly and integrally arranged in a plane,
No line of sight mismatch occurs. In addition, by providing one image sensor corresponding to one display element, the screen resolutions of the image capturing side and the display side can be matched.

[0014]

As described above, the conventional display device transmits a video signal in the field of view of a camera photographed by a camera placed at a photographing site to a display device placed at another place for display. Since the information transmission system only allows the viewer to view the image, there is no change in the displayed image even if the position or angle of the viewer changes, and the viewer feels as if he were looking at a painting.

[0015] In view of the above situation, a first object of the present invention is to connect two window-shaped imaging and display devices respectively installed in distant spaces by a communication line so that the distant spaces can pass through one window. An object of the present invention is to provide a device that produces an effect that makes the user feel as if they are connected.

A second object of the present invention is to display an image in the field of view of a camera on a display screen, as in the case of displaying an image captured by the camera on a display device. Instead of a one-way action
A person viewing the display panel (display screen) of the window-shaped imaging display device can select and view an image to be viewed from images transmitted by changing a position or an angle. And to enable more realistic video signal transmission. Another object of the present invention is to provide a device that can be applied to a very wide range according to the size, shape, and location of the display panel.

In order to more clearly understand this, a window-shaped imaging display device to be realized by the present invention will be described with reference to FIG. As shown in the figure, when the window-shaped imaging and display apparatus of the present invention is installed at two separate places and connected via a communication line to perform mutual communication, the image viewed by A and the image viewed by B appear as shown in the figure. The images C and C see are all different, just like looking through the window glass at the other side of the window.

In the above example, the case where three persons (A, B, C) at the place W1 look at the objects D, E, and F at the place W2 has been described. The present invention can be applied to a case where an exhibit such as the above is stored in a remote room and can be viewed at the exhibition hall using the window-shaped imaging display device. In this case, since the viewer is only at the place W1, one-way communication is performed. Therefore, the device installed at the place W2 may be a panel device dedicated to imaging, and the device installed at the place W1 may be a panel device dedicated to display. it can.

In order for a person to be present at the place W2 as well as at the place W1 and to be able to see the place W1, it is necessary to install a device having an imaging and display function on both sides. In such a case, there is a TV conference call system as described below with reference to FIG.

As described above with reference to FIGS. 22 and 23, in a videophone or videoconferencing system in which a camera is installed, a user who has a conversation through a television is required because of the location of the camera. There was a problem that the line of sight shifted. Further, as described above, an imaging display device having both functions of imaging and display has been proposed as a device for overcoming the drawback that the talker's line of sight is deviated. Cannot be identified, the image captured by the image sensor group is an image determined by the light entering the image display device, and the visible range does not differ depending on the position and angle of the viewer.

A third problem of the present invention is that, as shown in FIG. 27, in a TV telephone conference system, attendees A, B, and C at a location W1 are connected to conference attendees D, E, and
When looking at F, each of the attendees D, E, and F at the location W2 can have their own unique view of the seat that they are sitting at, and conversely, the attendees D, E, and F at the location W1 also have their own unique views. An object of the present invention is to provide a window-shaped imaging display device having a field of view.

In the above-mentioned TV telephone conference system, a large-scale window-like image display device is required in order to allow all of the conference attendees to overlook all of them. Is not allowed, the window-shaped imaging and display device of the present invention is very effective.

This will be described with reference to FIG.
Even if there is only one person at the location W1, the naturalness is greatly increased if the direction and gaze of the person's face are known. For example, as shown, when A is looking in the direction of B, A
Both B can see the other person in front of them. At this point, E can see A facing B. From this state, when A looks in the direction of E, both A and E can see the opponent's figure in front of themselves, but this time B sees A in the direction of E.

The fact that A is facing B when talking to B means that A and B at a distance are looking at each other through the window image display device of the present invention. To give a natural feeling.

A fourth object of the present invention is to save the three-dimensional information of the location by taking advantage of the fact that the window-shaped imaging display device can be taken in only by installing it as a communication terminal device. Another object of the present invention is to make it possible to easily create reproducible contents.

Another object of the present invention is to provide voice, temperature and humidity,
By transmitting information on the medium used by humans to perceive space, such as pressure, heat distribution, and smell on the display, at the same time as the video, we will obtain a device that enables more realistic connection of space. is there.

[0027]

In order to solve the above-mentioned problems, the present invention provides a window-shaped imaging display device having the following means:
And a two-way communication method using the same. That is, a large number of image pickup elements and display elements are formed so as to be uniformly distributed over the entire surface of the flat substrate, and the image pickup element detects at least the luminance and the direction of the incident light, and the incident light is emitted from the display element. An imaging display panel configured to emit outgoing light having a luminance proportional to the incident light in a direction in which it appears to go straight,
A video signal is generated based on the video information received by the multiple imaging elements on the imaging display panel, and video information to be supplied to the multiple display elements is generated based on the video signals received from the other imaging display panels. Provided is a window-shaped imaging and display device comprising: a video interface circuit; and a signal transmitting / receiving means for transmitting a signal transmitted from the video interface circuit onto a transmission line and transmitting a video signal received from the transmission line to the video interface circuit. I do.

In the above window-shaped image pickup display device, each image pickup element of the image pickup display panel is constituted by an image pickup element assembly composed of a predetermined number of image pickup elements formed on a hemispherical projection surface. The direction of the incident light can be determined by the position of the image sensor on the surface, and the display elements are constituted by the same number of display elements as the image sensor assembly formed on the hemispherical projecting surface. There is also provided a window-shaped imaging and display device comprising a display element assembly, wherein the direction of emitted light can be specified by the position of the display element on the hemispherical projecting surface.

In the above window-shaped imaging and display apparatus, the apparatus further comprises a three-dimensional audio input / output means, an audio interface means for performing signal processing of the audio input / output, and a composite video by mixing the video signal and the audio signal. There is also provided a window-shaped imaging and display device provided with signal processing means for generating a signal.

Further, in the above window-shaped imaging display device, the device further comprises environment information input / output means and environment information interface means for performing signal processing of a signal input to or output from the environment input / output means. A window-shaped imaging and display device provided with the same is also provided.

According to another aspect of the present invention, a large number of image pickup elements and display elements are formed so as to be uniformly distributed over the entire surface of the flat substrate, and the image pickup elements form at least the luminance and the direction of incident light. An imaging panel configured to detect
A video interface circuit that generates a video signal based on video information received by a number of image sensors on the imaging panel; and a signal transmission unit that transmits a signal transmitted from the video interface circuit to a transmission line. Window-shaped imaging panel device, and a large number of display elements are formed so as to be uniformly distributed over the entire surface of the flat substrate, and are proportional to the incident light in a direction in which the incident light appears to go straight from the display element. A display panel that emits outgoing light having the same brightness, a video signal that is received from the imaging panel via a transmission line, and is supplied to a number of display elements based on the received video signal. A window-like display panel device comprising: a video interface circuit for generating information; and
The present invention provides a window-shaped imaging display device provided with:

According to the present invention, moreover, a large number of image pickup elements and display elements are formed so as to be uniformly distributed over the entire surface of the flat substrate, and the image pickup element detects at least the luminance and the direction of the incident light. An imaging display panel configured to emit outgoing light having a luminance proportional to the incident light in a direction in which the incident light appears to go straight from the display element; and an image display panel based on image information received by a number of imaging elements on the imaging display panel. A video interface circuit that generates a video signal and generates video information to be supplied to a number of display elements based on a video signal received from another imaging display panel; and a transmission circuit that transmits a signal sent from the video interface circuit. Two or more remote window-shaped imaging and display devices having signal transmitting and receiving means for transmitting video signals transmitted on a line and received from a transmission line to the video interface circuit. A method of performing two-way video data communication through a transmission path, which is installed at each of the locations, detects the position and direction of light coming from a subject using an image sensor of a window-shaped imaging display device on the transmission side, and sends it to the reception side. On the receiving side, there is also provided a two-way communication method for viewing an image in a field of view determined by the relationship between the position of the viewer and the position of the window-shaped imaging display device in the video signal sent from the transmitting side.

[0033]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the window-shaped imaging display device of the present invention will be described below with reference to the drawings. FIG. 1 shows the appearance and usage of a window-shaped imaging display device of the present invention. In the example shown in the figure, in order to make the description easy to understand, a case is shown where the present apparatus is installed so as to be back-to-back at the same position on the wall surface of two adjacent rooms separated by a wall. Objects D, E, and F to be photographed are placed in one room, and the object is photographed by the apparatus.

Also, there are a plurality of viewers (observers) (A, B, C) in the adjacent room, so that the apparatus can be viewed from a plurality of positions. As will be described later, the two window-shaped imaging and display devices arranged back to back form a kind of window and can shoot and display the light so as to appear to be transmitted through this window. , C can see the objects F, E, D, respectively, in the direction in which their eyes are directed.

As shown in FIG. 2, the condition for obtaining such an effect is that two window-shaped imaging and display devices installed at distant places W1 and W2 communicate with each other so that they can communicate with each other. The object of the present invention is to obtain information on the other party and obtain information in a range selected by the viewer from information sent from the other party depending on the position of the viewer.

In the figure, a viewer A at a location W1
Stands at the position where the user looks at the window-shaped imaging display device installed as the communication terminal device T1 from the left side of the device, and when the window-shaped imaging device is viewed from this position, the right scene of the window-shaped imaging display device can be seen. In this way, viewer A will feel as though he is looking through the window behind the window.

Similarly, for both the viewer B and the viewer C, when viewing the window-shaped imaging and display device T1 from the position where they stand in the place W1, the scene of the place W2 just passes the window through the window glass. If the image is displayed on the window-shaped imaging display device as if looking at the scene in the above, the viewers B and C will feel as if they are looking at the window-shaped imaging display device and looking at the scene beyond the window.

This will be described in more detail below. Generally, when looking at an object on the opposite side through a window glass, FIG.
As shown in the figure, the angle (θ) is added to the position (x, y) of the window glass surface.
The light incident at (゜, φ) is emitted from the same position on the back surface of the panel at an angle (180 ° -θ ゜, 180 ° -φ ゜).

Therefore, if an attempt is made to make the same window-like panel as this window glass, light coming from various directions (ie, for various values of the above θ and φ) for all points on the panel surface on the imaging side. At least information about the luminance (color for more naturalness) is taken in, and the information is transmitted to the display side, and the light that is 180 ° out of phase with the incident angle from the corresponding position on the display side panel surface May be formed so as to emit light at a corresponding luminance (and color).

FIG. 4 illustrates the basic configuration of a window-shaped imaging and display device according to the present invention that adopts the above conditions. The incident light is incident on the panel surface at an angle (θ ゜, φ ゜). When the angle changes, the image pickup device to which the light is applied in the hemispherical projection surface of the image pickup device group is made different so that The direction of the incident light can be detected. On the display-side panel surface, light of the corresponding brightness (and color) may be emitted from the corresponding display element on the hemispherical protruding surface, which is a display element group at the corresponding position.

The configuration and operation of the above-described apparatus for giving a feeling of seeing an object through a window will be described below. here,
The description will be made two-dimensionally for easy understanding. As shown in FIG. 5, if two window-shaped imaging display devices are placed one on top of the other back-to-back and light incident on one surface is made to go straight out of the other surface, then glass The feeling of seeing things through is obtained. In other words, the angle of incidence of the incident light is made equal to the angle of emission of the emitted light. When light is transmitted through the window glass, the incident position and the outgoing position of the light are almost equal. However, in the present apparatus, only the relative position is important, and the absolute position does not necessarily have to match. However, it is necessary to adjust the positions of the incident light and the output light so that they visually match.

The two window-shaped imaging and display devices are connected by a communication line, and information of light incident on the surface of one device by signal transmission is transmitted to the display surface of the other device. Since light does not transmit through these two devices, the two window-shaped imaging and display devices do not need to be installed back to back on the wall as described above, but are installed on the wall of a completely separate room. It may be. Alternatively, one may be installed on a wall in a room and the other may be installed on an outer wall of a house.

An imaging display panel of the window-shaped imaging display device of the present invention which can obtain the phenomenon shown in FIG. 5 will be described below with reference to the drawings. FIG. 6 exemplifies an arrangement of the image sensor and the display device on the image display panel.
(A) shows an arrangement in which imaging elements and display elements are alternately arranged for each column. (B) shows an image pickup element and a display element arranged in a line and arranged in a staggered manner. (C)
The arrangement is the same as that of (b), except that the shape of the imaging element and the display element is hexagonal.

As described above, by uniformly arranging the image pickup device and the display device on the panel, this panel has not only the function as the image pickup device but also the function as the display device, and this panel is provided with an optical device. When viewed from the above, it has a function of receiving incident light and emitting outgoing light.

FIG. 7 shows an arrangement in which two imaging display panels are arranged back to back. The upper panel shows a portion where an imaging element is formed, and the lower panel has a display element formed. Is shown. Both the imaging element and the display element are formed on the surface of a hemispherical projecting surface of the same size.

Now, when the incident light enters from the upper right to the lower left as indicated by the arrow in the figure, it is incident on the corresponding rightward imaging element on the hemispherical projection surface of the imaging display panel. The position of the incident light can be detected by attaching a number (address) to the hemispherical projection surface, and the incident angle can be detected by the position of the image sensor on the projection surface.

In the figure, the hemispherical protruding surfaces constituting the image pickup element group arranged in a matrix of k rows and j columns on the upper side have the (a), (b-1), (b), and (b + 1) columns of the a rows. Projection surfaces Ca1 to Cay are drawn. On the lower side, (j-b + 2), (j-b + 1), and (j-) of the a-th row of the hemispherical protruding surface forming the display element group arranged in a matrix of k rows and j columns.
b) The protruding surfaces La1 to Ray of the row are drawn. The arrangement order is such that when the upper side is an address increasing from left to right, the lower side is an address increasing from right to left. It is preferable to shift the correspondence between the incident light and the output light by a time delay.

The above-mentioned window-shaped image pickup display panel device is formed by forming an image pickup element assembly having a plurality of image pickup elements formed on a hemispherical projecting surface and forming a plurality of display elements on the hemispherical projecting surface. The display element assemblies are arranged alternately so that adjacent image pickup device assemblies and display element assemblies form a pair (this is called a two-element type). The window-shaped imaging display panel device to be described is an imaging display panel device in which a device in which both an imaging element and a display element are formed on one hemispherical protruding surface (imaging display element assembly) is arranged in a matrix. It can be configured similarly to the two-element type.

In the figure, it is assumed that the matrix is composed of k rows and j columns, and 1 <a <k and 1 <b <j. When a certain stage of the CCD formed in m stages on one hemispherical protruding surface is represented by c, 1 <c <m. Assuming that the position in the circumferential direction at step _c is d _c , 1
<D _c <n _c . Here, _nx is an even number, n
₁ = 1 (first _{stage), n c = 4c-2} (c -th stage), n _m =
n (m-th stage).

FIG. 9 shows in detail one of the hemispherical protruding surfaces formed on the panel of the above-mentioned two-element type window-shaped imaging display device. As shown in FIG. 1A, a plurality of image pickup devices are formed by CCDs or the like on the hemispherical protruding surface, and these CCD image pickup devices are arranged hierarchically between the top and bottom of the hemisphere. The top is formed by one CCD, and the bottom is formed by a number n of CCDs determined by the resolution of how finely the direction of incident light is detected. The number of intermediate stages between the top and the bottom is a number that increases stepwise from 1 to n.

FIG. 5B shows a cross section of the hemispherical protruding surface. A CCD is formed on the surface of the hemispherical projecting surface, and the outside thereof is covered with a layer on which a light guide forming a lens is formed. Light that has entered the surface of the CCD almost perpendicularly
However, the light incident from an oblique direction hardly reaches the CCD. This allows any CCD
The direction from which the light came depends on whether the light is received.

FIG. 3C is a plan view of the CCD array shown in FIG. The operation of this CCD array will be described in detail later. However, in brief, the contents of the m-stage CCD are sequentially shifted to the next stage, and the contents of the last m-stage CCD are shifted by the shift register. And output serially from the shift register to the outside. At this time, since the contents of the m-1 stage CCD have been shifted to m stages, the contents of the m-1 stage CCD are then transferred to the shift register through m stages and output therefrom in series.

FIG. 10 shows another one of the hemispherical protruding surfaces formed on the panel of the two-element window image pickup display device in detail. As shown in FIG. 1A, an LCD (liquid crystal) display element is formed on the present hemispherical protruding surface. The hemispherical protruding surface is formed by forming the same number of LCDs as the CCDs on corresponding positions on the hemispherical protruding surface having the same shape and the same dimensions as the arrangement of the CCD described with reference to FIG.

The light passing through the LCD switch is shown in FIG.
, The direction of travel of the light is specified by the light emitted from which LCD element. FIG. 1C shows a drive circuit for a display element (LCD). The position data of the display element to be driven is set in the register RG1, and the scan data is set in the register RG2.

FIG. 11 shows an assembly of an image sensor and a display device having both the image sensor and the display device mounted on one hemispherical protruding surface. This image pickup device / display device assembly has a display device formed by an LCD or the like on a hemispherical protruding surface, and the upper surface is covered with a light-transmitting layer, and a CCD or the like is placed thereon so as not to overlap the lower display device. An imaging element is formed, and an adjacent imaging element and display element are treated as a pair of elements.

Here, a method of reading out electric charges from the image pickup device assembly will be described with reference to FIG. The reading order is such that the lowermost image sensor is read first, the contents (charges) of the image sensor in the upper stage are sequentially read, and finally the charges of the image sensor at the top of the hemisphere are read. Therefore, at time T1, the contents of the n elements in the m-th stage are read in parallel to the shift register, and the contents of the upper-stage elements are sequentially shifted in parallel to the lower stage.

At time T2, charges shifted from the nk image sensors of the (m-1) th stage (where k is the number of elements reduced compared to the (m) th stage) to the m stage are stored in the shift register in parallel. read out. In this way, at time Tm-1, the charge shifted from the second stage element to the mth stage is read out to the shift register, and at time Tm, the charge from one element at the first stage is read out to the shift register. It is.

When the electric charge is read out from a set of image pickup elements formed on one hemispherical projecting surface (this is referred to as an image pickup element assembly), the electric charge is subsequently read out from the next image pickup element assembly. In this manner, at time t1, charge signals are read from the x rows of imager assemblies in the first row, and at the next time t2, charge signals are read from x row of imager assemblies in the second row, and so on. At time ty, a charge signal is read from the x number of image sensor assemblies in the y-th row. The charge signal thus read out is processed as a signal of one frame, and transmitted to a device on the receiving side (that is, a display side).

Next, a method of supplying a display signal to a display element assembly (a set of display elements formed on a hemispherical projection surface) will be described below with reference to FIG. The display element assembly is arranged in a matrix of x columns and y rows, and signals to be transmitted are from 1 row and 1 column to 1 row x column, 2 rows and 1 column to 2 rows and x column,. Since the signals are sent in the order of y rows and x columns, a gate array that opens in this order is provided so that the image signals to be displayed on each display element assembly are oriented.

The signals orientated to one display element assembly by the above-described method include m, (m−1),. By inputting as a signal having, and specifying a stage to be displayed in synchronization with the signal, it is possible to display the same image as at the time of imaging.

In the above, the structure and operation of the image sensor assembly, the display device assembly, and the image sensor / display device assembly comprising the image sensor group formed on the hemispherical projecting surface have been described. The arrangement of the assembly on the window-shaped imaging display is shown in FIG. The imaging element assembly and the display element assembly arranged on the imaging display panel are of two element type (type in which the imaging element assembly and the display element assembly are separately formed) and one element type (the imaging element and the display element There is a type formed as one assembly), but two display panels comprising the same type of element assembly arrangement are installed at two separate places W1 and W2, and two-way communication is performed via a data communication device. To do.

The reading of information from the image pickup device assembly arranged as described above and the display by the display device assembly will be described below with reference to FIG. First, the m-th row of n is arranged in one row and one column of the display element assembly array of the imaging display panel.
Input and display. Next, the (m-1) -th stage n
, But the number of elements that actually contribute to the display is less than n. In this manner, the display elements are sequentially displayed, and finally, when the display of the first row of n elements (actually one element to be displayed) is completed, a display element assembly of one row and two columns is displayed. Similarly, display is performed up to one row and x column.

When the display of the first line is completed, the display of the second line is performed. When the display of the second line is completed, the display of the third line is performed. In this way, ultimately x
Reading is performed for all the imaging element assemblies in the column y row. The configuration on the display side has the same configuration, and sequentially displays all the display element assemblies arranged in x columns and y rows on the imaging display panel.

As a method of extracting a video signal from the image pickup device, a signal is sequentially extracted from the image pickup devices in each image pickup device assembly by charge transfer, and when reading of one image pickup device assembly is completed, the next image pickup device assembly is used. A method of extracting a video signal and reading out the video signal from all the imaging elements by this method is considered. In this method, signals can be read out in series, so that the circuit configuration is simplified. However, high-speed signal processing is required to read out signals from an enormous number of image sensors.

In view of this, a method of transferring signals read from the respective image pickup device assemblies in parallel is also conceivable. In this method, a transfer of a number equal to the maximum number of pixels of the image pickup device assembly (the number n of pixels in the m-th stage) is performed. It requires a path and is complicated in circuit configuration.
In addition, the amount of signal is significantly different between the transfer path for transferring only the first stage and the transfer path for transferring the signal for only the m-th stage, resulting in imbalance in the signal transmission amount and poor efficiency. However, there is an advantage that the time required to scan one field or frame is reduced. (4) Since the amount of image information to be captured is large in this window-shaped imaging and display device, the amount of imaging elements and display elements is large, and the amount of data to be handled is also large. However, it is highly expected that the signals to be handled include many signals having a large correlation. Therefore, the video signal taken into the signal processing circuit can be compressed by applying the image compression technique.

FIG. 16 shows an example of a pixel array when handling information having a large correlation. Of the light incident on the imaging display panel, light coming from adjacent places often has the same luminance (and color) or similar luminance (or color). This means that the light entering a certain range of image sensors arranged on the panel often has the same or similar luminance (color).
The transmission signal can be reduced by applying signal compression to such a signal, but when a signal of the same luminance (color) always comes, it can be replaced with one pixel, that is, one image sensor. it can.

From this viewpoint, the arrangement of the image pickup device and the display device can be set as shown in FIG. That is, it is possible to provide an imaging device assembly that only distinguishes light that is incident almost vertically on the top of the hemispherical shape and light that is incident on the top, bottom, left and right. Corresponding to this, the display element assembly can also be a display element assembly that emits light while distinguishing the top, top, bottom, left and right.

The image pickup device assembly and the display device assembly are formed by forming an image pickup device or a display device on a hemispherical protruding surface. As shown in FIG. 17, light is condensed or diverged by a lens. By performing the above, it is also possible to form the imaging element and the display element in a plane.

Next, the configuration of a bidirectional information communication system using the window-shaped imaging and display device of the present invention will be described below with reference to FIG. As shown in the figure, a window-shaped imaging and display device shown as terminal devices T1 and T2 is installed at one place W1 and another place W2, and these terminal devices T1 and T2 can be connected to each other by a transmission line to perform information communication with each other. To do.

The terminal device T1 captures video, audio, and other environmental information, such as temperature, as environmental information of the place W1, processes these signals, and transmits these signals from the transmitting device.

Now, a case where information is transmitted from the terminal device T1 to the terminal device T2 will be described. In the terminal device T1, a voice to be transmitted to the terminal device T2 is input from the microphone 2A, and is processed through the voice interface 3A. Send to device 10A. Similarly, a person or an object to be an image to be transmitted is imaged by the image sensor 4A, and the image
To the signal processing device 10A via the. If there is information to be sent in addition to video and audio, the information is detected by a sensor 7A that can detect the information, and the environment information interface 9
A is sent to the signal processing device 10A via A.

The audio signal, the video signal, and the environment signal are comprehensively processed in the signal processing device 10A. That is, amplification of each signal, detection of correlation between signals, information compression, and the like are performed here. The signal thus processed is transmitted and received by the transmitting / receiving device 1
1A, where signal processing for transmission such as modulation is performed, and then transmitted over a transmission line.

The terminal device T2 installed at the place W2 receives the signal transmitted via the transmission line network 12 by the device on the receiving side of the transmission / reception device 11B, performs communication signal processing such as demodulation, and then performs signal processing. The signal is sent to the device 10B, where signal processing such as amplification and signal decompression is performed. The output of the signal processing device 10B is separated into an audio signal, a video signal, and an environmental information signal.
Output from B. The video signal is transmitted to the video interface 6
B and display by the display element 5B. The environment information is sent to the environment interface 9B, and is sent from the conditioner 8B as appropriate.

When transmitting a signal from the terminal device T2 to the terminal device T1, the signal is transmitted through the reverse route. That is, the audio input collected by the microphone 2B of the terminal device T2, the video input input to the image sensor 4B, and the environment information input sensed by the sensor 7B are transmitted via the audio interface 3B, the video interface 6B, and the environment information interface 9B, respectively. Taken into the signal processing device 10B,
After performing necessary signal processing, the signal is transmitted through the transmission / reception device 11B.

In the terminal device T1, the signal received by the receiving circuit of the transmitting / receiving device 11A is sent to the signal processing device 10A.
Then, necessary signal processing is performed, and the signal is sent to the audio interface 3A, the video interface 6A, and the environment information interface 9A, and output from the speaker 1A, the display element 5A, and the conditioner 9A.

The above description has been made with reference to the terminal device T1 and the terminal device T.
This is an explanation of signal transmission between the two, but the two-way information communication system is not limited to such one-to-one communication,
It is also possible to perform multicast (one-to-many) communication with many other terminal devices via the transmission line network 12. In addition, by providing the storage device 14 and storing and reproducing the content, it is also possible to use the previously acquired environment information.

The window-shaped imaging display device of the present invention has a wide application range other than the above. For example, if it is formed on a curved surface as shown in FIG. 19 or formed on a bay window, a wide-angle scene can be seen. . In addition, if a room covered with the present apparatus is created, an image in all directions can be displayed, and there is an effect of giving a sense that the user himself is present in a remote place.

As shown in FIG. 20, when the present imaging display device is installed on the entire surface of the wall, it is difficult to convey a drama or L.D.
Realism such as IVE can be conveyed. In addition, when the display panel is placed on the door or fusuma wall of the house, the effect of connecting remote rooms, desks, tabletops, etc. can be obtained.
Cooperative life beyond space becomes possible.

[0079]

According to the present invention, the window-shaped image pickup and display apparatus is provided with two remote points.
By installing one in each of the two spaces and connecting them with a two-way communication line, an effect is created that makes those spaces seem to be connected to each other through a window.

The window-shaped imaging display device of the present invention has a realistic feeling that spaces are actually connected because the position of a person (observer) watching the display panel and the image displayed according to the movement change. to be born.

When humans talk and communicate with each other through the window-shaped imaging display device of the present invention, compared to a conventional videophone or videoconference system using a pictorial image taken by a specific camera, there is a greater variety. The means of expressing emotions and the means of communicating intentions and effects are created, and deeper communication becomes possible.

The window-shaped imaging and display apparatus of the present invention uses an imaging display panel having both imaging and display functions, instead of performing only imaging and display as in a conventional camera or display. Can obtain information and transmit it.

A "window" can be made by using two window-shaped imaging and display devices of the present invention and installing one outdoors and the other on an empty wall indoors. Also, by sending the environment information stored in the storage device to the window-shaped imaging display device, various scenery can be viewed.

If data transmission using the window-shaped imaging display device of the present invention is applied, not only use between one-to-one terminals but also
By distributing the environmental information detected by one terminal to many terminals, it is possible to use the environment information in a content distribution business.

The communication method using the window-shaped imaging display device of the present invention is based on simultaneous transmission of information such as temperature, humidity, smell, pressure on the display, heat, etc., which are used for human being to recognize the environment. , It is possible to create a more realistic “connecting space” effect.

[Brief description of the drawings]

FIG. 1 is a schematic diagram illustrating the function of a window-shaped imaging display device according to the present invention.

FIG. 2 is a schematic diagram illustrating the principle of a window-shaped imaging and display device according to the present invention.

FIG. 3 is a schematic diagram illustrating the principle of the window-shaped imaging display device according to the present invention.

FIG. 4 is a schematic diagram illustrating the principle of a window-shaped imaging display device according to the present invention.

FIG. 5 is a diagram showing a relationship between incident light and outgoing light of the window-shaped imaging display device according to the present invention.

FIG. 6 is a diagram showing an arrangement of an image pickup element and a display element on a panel surface of the window-shaped image pickup display device according to the present invention.

FIG. 7 is a diagram showing an arrangement of image pickup elements and display elements on a panel surface of the window-shaped image pickup display device according to the present invention.

FIG. 8 is a diagram showing an arrangement of image pickup elements and display elements on a panel surface of the window-shaped image pickup display device according to the present invention.

FIG. 9 is a schematic diagram showing a structure and a readout circuit of an imaging element assembly of the window-shaped imaging display device according to the present invention.

FIG. 10 is a schematic diagram showing a structure of a display element assembly and a display drive circuit of the window-shaped imaging display device according to the present invention.

FIG. 11 is a schematic diagram showing a structure of an image pickup device / display element assembly and a display drive circuit of the window image pickup display device according to the present invention.

FIG. 12 is an operation time chart for reading video information from an image pickup device assembly array.

FIG. 13 is a time chart of a display circuit drive signal for displaying video information on a display element assembly array.

FIG. 14 is a schematic diagram showing an element arrangement on a panel of the window-shaped imaging display device according to the present invention.

FIG. 15 is an operation time chart of a video signal readout from the image pickup device assembly and a display drive circuit of the display device assembly.

FIG. 16 is a schematic diagram illustrating a configuration example of an imaging element assembly and a display element assembly.

FIG. 17 is a schematic diagram showing another configuration example of the imaging element assembly and the display element assembly.

FIG. 18 is a block diagram of a bidirectional communication system using the window-shaped imaging display device according to the present invention.

FIG. 19 is a schematic view showing an application example of the window-shaped imaging display device according to the present invention.

FIG. 20 is a schematic view showing an application example of the window-shaped imaging display device according to the present invention.

FIG. 21 is a conceptual explanatory diagram of a conventional stereoscopic vision system.

FIG. 22 is a conceptual explanatory diagram of a videophone device.

FIG. 23 is a conceptual explanatory diagram of a video conference telephone device.

FIG. 24 is a conceptual explanatory diagram of another example of the videophone device.

FIG. 25 is a schematic view showing a matrix arrangement of an image sensor.

FIG. 26 is a schematic diagram showing a cross-sectional structure of a conventional imaging display device.

FIG. 27 is a conceptual explanatory diagram of a video conference telephone device using the window-shaped imaging display device according to the present invention.

FIG. 28 is a conceptual explanatory diagram showing a state of conversation between distant points using the window-shaped imaging display device according to the present invention.

[Explanation of symbols]

W1, W2 ... two separate places, A, B, C ...
Viewer (viewer), D, E, F ... Object placed at location W2, T1 ... Window-shaped imaging display device placed at location W1

Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat II (Reference) H04N 7/15 630 H04N 7/15 630Z F term (Reference) 5C022 AA12 AB68 AC42 CA02 5C024 AX01 BX07 CY15 CY16 CY18 CY45 CY50 DX06 DX07 GX02 GX21 GY01 GZ04 5C058 AB02 BA01 BA31 BA35 BB01 5C061 AA06 AA16 AA21 AB03 AB06 AB14 AB17 5C064 AA02 AC02 AC06 AC08 AC12 AC16 AD01 AD14

Claims (6)

[Claims] An image sensor and a display element are formed so as to be uniformly distributed over the entire surface of a flat substrate, and at least the luminance and direction of incident light are detected by the image sensor, and the image sensor and the display element are detected from the display element. An imaging display panel configured to emit outgoing light having a luminance proportional to the incident light in a direction in which the incident light appears to travel straight; and a video signal based on video information received by a number of imaging devices on the imaging display panel. And a video interface circuit for generating video information to be supplied to a number of display elements based on video signals received from other imaging display panels, and transmitting a signal sent from the video interface circuit onto a transmission line. A window imaging and display device comprising: a signal transmission / reception unit that transmits a video signal transmitted and received from a transmission line to the video interface. 2. The window-shaped imaging and display device according to claim 1, wherein each of the imaging elements of the imaging and display panel is an imaging element assembly composed of a predetermined number of imaging elements formed on a hemispherical projecting surface. The direction of the incident light can be determined based on the position of the image sensor on the hemispherical projecting surface, and the display elements have the same number as the number of the image sensor assemblies formed on the hemispherical projecting surface. A window-shaped imaging and display device comprising a display element assembly composed of display elements, wherein the direction of emitted light can be specified by the position of the display element on the hemispherical projecting surface. 3. The window-shaped imaging and display device according to claim 1, wherein said device further comprises a three-dimensional audio input / output means and an audio interface means for performing signal processing of the audio input / output, and the video signal and the audio signal. And a signal processing means for generating a composite video signal by mixing the two. 4. The window-shaped imaging and display device according to claim 1, wherein the device further performs environment information input / output means and environmental information for performing signal processing of a signal input to or output from the environment input / output means. A window-shaped imaging display device provided with an interface means. 5. An imaging panel in which a large number of imaging elements and display elements are formed so as to be uniformly distributed over the entire surface of a flat substrate, and the imaging elements detect at least luminance and direction of incident light. A video interface circuit that generates a video signal based on video information received by a number of image sensors on the imaging panel; and a signal transmission unit that transmits a signal transmitted from the video interface circuit to a transmission line. A window-shaped imaging panel device comprising: a plurality of display elements formed so as to be uniformly distributed over the entire surface of the flat substrate, and incident light from the display elements in a direction in which the incident light appears to go straight. A display panel configured to emit outgoing light having a luminance proportional to the image signal received from the image pickup panel via a transmission line; A video interface circuit for generating image information supplied to the plurality of display elements based on the item,
And a window-shaped display panel device having the same. 6. A large number of image pickup elements and display elements are formed so as to be uniformly distributed over the entire surface of the flat substrate, and at least the luminance and direction of incident light are detected by the image pickup elements. An imaging display panel configured to emit outgoing light having a luminance proportional to the incident light in a direction in which the incident light appears to travel straight; and a video signal based on video information received by a number of imaging devices on the imaging display panel. And a video interface circuit for generating video information to be supplied to a number of display elements based on video signals received from other imaging display panels, and transmitting a signal sent from the video interface circuit onto a transmission line. Two or more remote window-shaped imaging and display devices having signal transmitting and receiving means for transmitting and transmitting a video signal received from a transmission line to the video interface circuit. This is a method of performing two-way video data communication through transmission paths, which is installed at each of the locations, and detects the position and direction of light coming from the subject with the image sensor of the window-shaped imaging display device on the transmitting side and sends it to the receiving side. , On the receiving side,
A two-way communication method for viewing an image in a visual field determined by a relationship between a position of a viewer and a position of a window-shaped imaging display device from a video signal transmitted from a transmission side.