JP2005293291A - Image input/output device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image input/output device for generating three-dimensional information of an object such as an object to be measured, and for executing predetermined measurement on the basis of the three-dimensional information, and for projecting operation instruction information corresponding to the measurement result to the object. <P>SOLUTION: Measurement is performed on the basis of the three-dimensional information acquired for the object, and an image corresponding to the measurement result is projected on the object or an image projection enabled face on the projecting direction. For example, information for operating the approximate volume division of the object on the basis of the three-dimensional information of the object can be projected on the object. On the other hand, the three-dimensional information of the object and the image pickup data of designated circumstances for the object are acquired, and measurement of the object is performed on the basis of the image pickup data and the three-dimensional information, and an image corresponding to the measurement result can be projected on the object or the image projection enabled face on the projecting direction. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image input / output device capable of imaging and projecting, and in particular, can project a comparison result obtained by comparing imaging data of a subject obtained by imaging and reference information about the subject onto the subject. The present invention relates to a possible image input / output device.

2. Description of the Related Art Conventionally, a three-dimensional measurement apparatus that projects a light pattern onto a subject, images the light pattern projected onto the subject, and performs predetermined computation on the image signal to obtain three-dimensional information about the subject. Are known. For example, in Patent Document 1, a coded multi-slit light pattern is projected onto a measurement object, and the coded multi-slit light pattern imaging device projected onto the measurement object is imaged, and an image signal obtained by the imaging is obtained. A three-dimensional measuring device that calculates the three-dimensional position of the contour of the object to be measured is described.
Japanese Patent Publication No. 8-20232

  However, a general image input / output device including a three-dimensional measuring device as described in Patent Document 1 and a projection device and an imaging device only provides information on the appearance of the object to be measured. There was a problem that its use was limited.

  The present invention has been made to solve the above-described problems, and generates three-dimensional information of a subject such as an object to be measured, performs predetermined measurement based on the three-dimensional information, and performs the measurement. An object of the present invention is to provide an image input / output device capable of projecting work instruction information according to a result onto a subject.

  In order to achieve this object, the image input / output device according to claim 1 includes: a light source unit that emits light; and a spatial modulator unit that performs spatial modulation on the light emitted from the light source unit and outputs image signal light. A projection unit that projects the image signal light output by the spatial modulation unit in a projection direction; an imaging unit that captures an image of a subject that is at least partially in the projection direction; When the image signal light having the pattern shape is projected on the subject or on a surface on which the image can be projected in the projection direction by the projection unit, the subject based on the imaging data captured by the imaging unit Or three-dimensional information generating means for generating three-dimensional information of a surface capable of projecting an image in the projection direction, the three-dimensional information generated by the three-dimensional information generating means, and the subject An image signal corresponding to the measurement information obtained by the measurement by the measurement means, the measurement means for measuring the subject based on the designation information generated from the imaging data captured by the imaging means The light is output by the spatial modulation unit, and the output image signal light is projected by the projection unit onto the subject or a surface on which the image can be projected in the projection direction.

  According to the image input / output device of claim 1, when the light emitted from the light source means is spatially modulated by the spatial modulation means and output as image signal light, the image signal light is Projection is performed in the projection direction by the projection means. On the other hand, a subject having at least a part in the projection direction is imaged by the imaging means, and the imaging data is acquired. Here, when the image signal light having a predetermined pattern shape is projected by the projection unit onto the subject or a surface on which the image can be projected in the projection direction, the image signal light captured by the imaging unit is captured. Based on the data, the three-dimensional information generating means generates three-dimensional information of the subject or the surface on which the image can be projected in the projection direction. The measurement unit measures the subject based on the three-dimensional information thus generated and the designation information generated from the imaging data obtained by imaging the designation state for the subject by the imaging unit. When measurement is performed by the measurement unit, image signal light corresponding to the measurement information obtained by the measurement is output from the spatial modulation unit, and the output image signal light is output from the subject or the projection by the projection unit. The image is projected onto a plane capable of projecting an image. As a result, an image corresponding to the measurement information corresponding to the designated situation is projected onto the subject or the surface on which the image can be projected in the projection direction.

  The image input / output device according to claim 2, further comprising a designated point input unit for inputting a designated point on the subject, wherein the measuring unit is input by the designated point input unit. The subject is measured for a point, line segment, or region formed by at least one designated point.

  The image input / output device according to claim 3 is the image input / output device according to claim 2, wherein the designated point input means captures the pointing member indicating the designated point on the subject by shifting the time by the imaging means. Then, the designated point is input based on the imaging data.

  The image input / output device according to claim 4 is the image input / output device according to any one of claims 1 to 3, wherein the measurement unit is based on the three-dimensional information generated by the three-dimensional information generation unit. The volume information of the subject is measured, and image signal light projected onto the subject or a surface on which the image can be projected in the projection direction by the projection unit is instruction information corresponding to the volume information.

  The image input / output device according to claim 5 is the image input / output device according to any one of claims 1 to 4, further comprising density information acquisition means for acquiring density information of the subject, wherein the measurement means includes the tertiary Based on the three-dimensional information generated by the original information generation means and the density information acquired by the density information acquisition means, the weight distribution information of the subject is measured, and the subject or the projection is measured by the projection means. The image signal light projected on the image projectable surface in the direction is instruction information corresponding to the weight distribution information.

  The image input / output device according to claim 6 is a light source means for emitting light, a spatial modulation means for spatially modulating light emitted from the light source means to output image signal light, and an output from the spatial modulation means. An image having a predetermined pattern shape, a projection unit that projects the image signal light to be projected in a projection direction, an imaging unit that captures an image of a subject that is at least partially in the projection direction, and acquires the imaging data When the signal light is projected onto the subject or the image-projectable surface in the projection direction by the projection unit, the image projection on the subject or the projection direction is performed based on the imaging data captured by the imaging unit. Three-dimensional information generating means for generating three-dimensional information of possible surfaces, and instruction information for the subject based on the three-dimensional information generated by the three-dimensional information generating means Display information generation means, image signal light based on the instruction information generated by the instruction information generation means is output by the spatial modulation means, and the output image signal light is output by the projection means to the subject or the object The image is projected onto a surface capable of projecting an image in the projection direction.

  According to the image input / output device of claim 6, when the light emitted from the light source means is spatially modulated by the spatial modulation means and output as image signal light, the image signal light is Projection is performed in the projection direction by the projection means. On the other hand, a subject having at least a part in the projection direction is imaged by the imaging means, and the imaging data is acquired. Here, when the image signal light having a predetermined pattern shape is projected by the projection unit onto the subject or a surface on which the image can be projected in the projection direction, the image signal light captured by the imaging unit is captured. Based on the data, the three-dimensional information generating means generates three-dimensional information of the subject or the surface on which the image can be projected in the projection direction. Based on the three-dimensional information thus generated, instruction information for the subject is generated by the instruction information generating means. When the instruction information is generated by the instruction information generation unit, the image signal light corresponding to the instruction information is output from the spatial modulation unit, and the output image signal light is output from the subject or the projection direction by the projection unit. The image is projected onto the upper image projectable surface. Thereby, an image corresponding to the instruction information is projected onto the subject or the photographing direction.

  An image input / output device according to a seventh aspect is the image input / output device according to any one of the first to sixth aspects, wherein the projection unit has no effect on a surface on which the subject or the image can be projected in the projection direction. Spatial light modulation control means is provided for controlling the spatial modulation means based on the three-dimensional information generated by the three-dimensional information generation means so that distorted image signal light is projected.

  The image input / output device according to claim 8 is the image input / output device according to any one of claims 1 to 7, wherein the image is projected onto the subject or an image projectable surface in the projection direction by the projection unit. Signal light is character information relating to the measurement information or the instruction information.

  The image input / output device according to claim 9 is the image input / output device according to any one of claims 1 to 8, wherein the projection unit projects an image onto a surface on which the subject or the image can be projected in the projection direction. Signal light is graphic information based on the measurement information or the instruction information.

  The image input / output device according to claim 10 is the image input / output device according to any one of claims 1 to 9, wherein the image is projected onto the subject or an image projectable surface in the projection direction by the projection unit. The signal light is a work instruction line based on the measurement information or the instruction information.

  The image input / output device according to claim 11 is the image input / output device according to any one of claims 1 to 10, wherein the image is projected onto the subject or an image-projectable surface in the projection direction by the projection unit. Work instruction information for classifying the subject into substantially equal volumes or substantially equal weights according to the volume information or weight distribution information of the subject measured by the measuring means, or indication information by the instruction information generating means The work instruction information for classifying the subject into a substantially equal volume or a substantially equal weight.

  According to a twelfth aspect of the present invention, in the image input / output device according to the eleventh aspect, the work instruction information is a cutting instruction line.

  An image input / output device according to a thirteenth aspect is the image input / output device according to the twelfth aspect, further comprising cutting means for cutting the subject in accordance with the cutting instruction line.

  The image input / output device according to claim 14 is the image input / output device according to any one of claims 1 to 13, wherein the projection unit includes the light source unit, the spatial modulation unit, and the projection unit, and the imaging unit. An imaging unit provided integrally with the projection unit, and a movable unit that can move relative positions of the subject.

  15. The image input / output device according to claim 15, wherein the projection unit performs the measurement after turning off the image signal having the predetermined pattern shape. The image signal light based on the information or the instruction information is projected.

  The image input / output device according to claim 16 is the image input / output device according to any one of claims 1 to 15, wherein the image pickup means receives image signal light based on the measurement information or the instruction information by the projection means. An image of the projected subject or a surface on which the image in the projection direction can be projected is captured, and the captured image data is acquired.

  According to the image input / output device according to claim 1, while the three-dimensional information for the subject is acquired, the imaging data of the specified situation for the subject is acquired, and based on the imaging data and the three-dimensional information, Measurement is performed on the subject, and an image corresponding to the measurement information, which is the measurement result, is projected onto the subject or an image-projectable surface in the projection direction. Thus, for example, when the specified situation specifies a predetermined area or a line segment in the subject specified by the user, and the subject is measured for the specified area or line segment. Measurement information according to the area or line segment desired by the user is projected on the subject or in the vicinity of the subject, providing convenience for the user and making the measurement information easy for the user. Has the effect of being able to be recognized.

  According to the image input / output device described in claim 2, in addition to the effect produced by the image input / output device according to claim 1, the image input / output device is formed by inputting at least one or more specified points. A point, a line segment, or a region is measured for the subject, and an image corresponding to the measurement information that is the measurement result is projected onto the subject or a plane on which an image can be projected in the projection direction. Therefore, since the point, line segment, and area where measurement information is required are only specified and input, there is an effect that desired measurement information can be easily acquired.

  According to the image input / output device according to claim 3, in addition to the effect of the image input / output device according to claim 2, the input of the designated point is an imaging in which the indication member pointing to the designated point is imaged at different times. Since it is performed based on the data, for example, the designated point can be input by imaging the trajectory of the user's fingertip over time. Therefore, it is easy to input a designated point for which measurement information is desired, and there is an effect of providing convenience to the user.

  According to the image input / output device of the fourth aspect, in addition to the effect produced by the image input / output device according to any one of the first to third aspects, the volume information of the subject is based on the three-dimensional information of the subject. Measurement is performed, and instruction information corresponding to the volume information is projected onto the subject. Therefore, for example, when the subject is divided into substantially equal volumes, a line that divides the subject into substantially equal volumes is projected as instruction information, or when the density information of the subject is referred to, the subject The instruction information corresponding to the weight of the predetermined volume is projected, so that the work can be made more efficient and the work labor can be reduced.

  According to the image input / output device according to claim 5, in addition to the effect produced by the image input / output device according to any one of claims 1 to 4, based on the three-dimensional information of the subject and the density information of the subject, Weight distribution information of the subject is measured, and instruction information corresponding to the weight distribution information is projected onto the subject. Therefore, for example, when the subject is divided into substantially equal weights, a line that divides the subject into substantially equal weights is projected as instruction information, or an instruction according to weight distribution information of a predetermined area in the subject By projecting information, it is possible to increase the efficiency of the work and reduce the work effort.

  According to the image input / output device according to claim 6, the instruction information is generated based on the three-dimensional information acquired for the subject, and the image corresponding to the instruction information can be projected on the subject or the projection direction. Is projected onto a flat surface. Therefore, for example, cutting information for dividing the subject into substantially equal volumes based on the three-dimensional information of the subject is projected onto the subject as instruction information, so that it provides convenience to the user and There is an effect that the user can easily recognize the instruction information.

  According to the image input / output device described in claim 7, in addition to the effect produced by the image input / output device according to any one of claims 1 to 6, the image corresponding to the obtained measurement information includes the subject and the projection direction. Based on the three-dimensional shape of the image-projectable surface, the image is projected as a distortion-free image on the subject or the image-projectable surface in the projection direction. Therefore, the projected image is measured by distortion. There is an effect that an image corresponding to information or instruction information can be accurately projected. For example, a line that divides the subject into substantially equal volumes is projected at an accurate position, so that the subject can be accurately divided into substantially equal volumes.

  According to the image input / output device of the eighth aspect, in addition to the effect produced by the image input / output device according to any one of the first to seventh aspects, the character information related to the measurement information or the instruction information is displayed on the subject or the projection direction. The image is projected onto a plane where the image can be projected. Therefore, for example, when measuring the length of a predetermined line segment, the measurement value is projected as character information, so that the information can be easily recognized.

  According to the image input / output device of the ninth aspect, in addition to the effect produced by the image input / output device according to any one of the first to eighth aspects, the graphic information related to the measurement information or the instruction information is in the subject or the projection direction. The image is projected onto a plane where the image can be projected. Therefore, for example, when a three-dimensional figure having a desired shape is dug out from a subject, the graphic information relating to the three-dimensional figure is projected, thereby improving the efficiency of the work and reducing the work effort.

  According to the image input / output device of the tenth aspect, in addition to the effect produced by the image input / output device according to any one of the first to ninth aspects, a work instruction line based on measurement information or instruction information is provided on the subject. Therefore, it is possible to increase the efficiency of the work and reduce the work effort.

  According to the image input / output device described in claim 11, in addition to the effect produced by the image input / output device according to any one of claims 1 to 10, the subject is substantially omitted according to the volume information or weight distribution information of the subject. The work instruction information for dividing the subject into equal volumes or substantially equal weights, or the work instruction information generated as instruction information for dividing the subject into substantially equal volumes or substantially equal weights is projected. When an operation for dividing one object into approximately equal volumes or approximately equal weights is required, there is an effect that the efficiency of the operation can be improved and the work labor can be reduced.

  According to the image input / output device according to claim 12, in addition to the effect achieved by the image input / output device according to claim 11, as the work instruction information, a cutting instruction line for dividing the subject into a substantially equal volume or a substantially equal weight. Is projected onto the subject, so that when it is necessary to cut and divide one object that is the subject into units of substantially equal volume or substantially equal weight, the work is made more efficient, There is an effect that work labor can be reduced.

  According to the image input / output device of the thirteenth aspect, in addition to the effect produced by the image input / output device according to the twelfth aspect, the cutting according to the cutting instruction line projected by the cutting means is executed, so that the work efficiency is improved. In addition, there is an effect that work labor can be reduced.

  According to the image input / output device according to claim 14, in addition to the effect produced by the image input / output device according to any one of claims 1 to 13, a projection unit including a light source unit, a spatial modulation unit, and a projection unit, The imaging unit having the imaging unit and provided integrally with the projection unit is configured to be movable relative to the subject. In the case of performing the above, there is an effect that the imaging direction and the projection direction can be reduced from being obstructed by the work, thereby improving the work efficiency.

  According to the image input / output device described in claim 15, in addition to the effect produced by the image input / output device according to any one of claims 1 to 14, an image having a predetermined pattern shape is projected, and the image is turned off. After that, an image based on measurement information or instruction information is projected. That is, three-dimensional information is obtained based on the image having the predetermined pattern shape, and then measurement information or instruction information based on the three-dimensional information is obtained, and the measurement information or instruction information is the predetermined pattern shape. Is projected after the image having the light is turned off. Therefore, for example, the acquisition and measurement of the three-dimensional shape of the subject can be executed in conjunction with each other, and there is an effect that the work efficiency can be improved.

  According to the image input / output device of the sixteenth aspect, in addition to the effect produced by the image input / output device according to any one of the first to fifteenth aspects, the image is projected onto the subject or a surface capable of projecting an image in the projection direction. Since the measurement information or instruction information is imaged and the image data is acquired, there is an effect that the measurement information or instruction information can be recorded.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. FIG. 1 is an external perspective view of an image input / output device 1 of the present invention.

  The image input / output device 1 includes a digital camera mode that functions as a digital camera, a webcam mode that functions as a web camera, a stereoscopic image mode for detecting a three-dimensional shape and acquiring a stereoscopic image, and a planarized flat surface of a curved document. Planarized image mode for acquiring a digitized image, volume / weight measurement mode for measuring the volume and weight, projecting and displaying the result, measuring based on the specified specified point, and projecting the result It is an apparatus having various modes such as a designated point measurement mode to be displayed.

  In FIG. 1, in order to detect the three-dimensional shape of the document P as a subject, particularly in the stereoscopic image mode and the planarized image mode, striped pattern light obtained by alternately arranging light and dark from the image projection unit 13 described later is used. A state of projection is illustrated.

  The image input / output device 1 is connected to an imaging head 2 formed in a substantially box shape, a pipe-shaped arm member 3 having one end connected to the imaging head 2, and the other end of the arm member 3. And a base 4 formed in a substantially L shape.

  The imaging head 2 is a case in which an image projection unit 13 and an image imaging unit 14 described later are included. On the front surface of the imaging head 2, a cylindrical barrel 5 is disposed at the center thereof, a finder 6 is disposed obliquely above the barrel 5, and a flash 7 is disposed on the opposite side of the finder 6. In addition, a part of the lens of the imaging optical system 21 which is a part of the image capturing unit 14 described later is exposed on the outer surface between the finder 6 and the flash 7, and an image of the subject is input from the exposed part. The

  The lens barrel 5 is a cover that protrudes from the front of the imaging head 2 and encloses the projection optical system 20 that is a part of the image projection unit 13 therein. The lens barrel 5 holds the projection optical system 20 so that the projection optical system 20 can be moved as a whole for focus adjustment and is prevented from being damaged. Further, from the end surface of the lens barrel 5, a part of the lens of the projection optical system 20 that is a part of the image projection unit 13 is exposed on the outer surface, and image signal light is projected from the exposed part toward the projection surface. The

  The viewfinder 6 is composed of an optical lens that is disposed from the back of the imaging head 2 through the front. When the user looks in from the back of the image pickup apparatus 1, a range that substantially matches the range in which the image pickup optical system 21 forms an image on the CCD 22 can be seen.

  The flash 7 is a light source for supplementing a necessary light amount in, for example, a digital camera mode or a designated point measurement mode, and is composed of a discharge tube filled with xenon gas. Therefore, it can be repeatedly used by discharging from a capacitor (not shown) built in the imaging head 2.

  On the upper surface of the imaging head 2, a release button 8 is disposed on the front side, a mode switch 9 is disposed behind the release button 8, and a monitor LCD 10 is disposed on the opposite side of the mode switch 9.

  The release button 8 is composed of a two-stage push button switch that can be set in two types of states, “half-pressed state” and “full-pressed state”. The state of the release button 8 is managed by a processor 15 which will be described later, and in the “half-pressed state”, the well-known autofocus (AF) and automatic exposure (AE) functions are activated, and the focus, aperture, and shutter speed are adjusted. Imaging or the like is performed in the “fully pressed state”.

  The mode switch 9 is a switch that can be set in various modes such as a digital camera mode, a webcam mode, a stereoscopic image mode, a planarized image mode, a volume / weight measurement mode, a designated point measurement mode, and an off mode. The state of the mode changeover switch 9 is managed by the processor 15, and the processing of each mode is executed when the state of the mode changeover switch 9 is detected by the processor 15.

  The monitor LCD 10 is composed of a liquid crystal display, and receives an image signal from the processor 15 and displays an image to the user. For example, the monitor LCD 10 displays a captured image in a digital camera mode, a webcam mode, a volume / weight measurement mode, a three-dimensional shape detection result image in a stereoscopic image mode or a specified point measurement mode, a planarized image in a planarized image mode, and the like. Is done.

  Furthermore, an antenna 11 as an RF (wireless) interface and a connecting member 12 that connects the imaging head 2 and the arm member 3 are disposed on the side surface of the imaging head 2.

  The antenna 11 transmits the captured image data acquired in the digital camera mode or the stereoscopic image data acquired in the stereoscopic image mode via the RF driver 24 described later to the external interface by wireless communication, and is used in the volume / weight measurement mode. Work content information to be received is received from an external interface by wireless communication, and is output to an RF driver 24 described later.

  The connecting member 12 is formed in a ring shape in which an internal thread is formed on the inner peripheral surface, and is fixed to the side surface of the imaging head 2 so as to be rotatable. A male screw is formed on one end side of the arm member 3. By fitting the female screw and the male screw, the imaging head 2 and the arm member 3 can be detachably connected, and the imaging head 2 can be fixed at an arbitrary angle. Therefore, the imaging head 2 can be removed and used as a normal digital camera (digital camera).

  The arm member 3 is for holding the imaging head 2 in a predetermined imaging position so as to be changeable, and is composed of a bellows-like pipe that can be bent into an arbitrary shape. Therefore, the imaging head 2 can be directed to an arbitrary position.

  The base 4 is mounted on a mounting table such as a desk and supports the imaging head 2 and the arm member 3. Since it is formed in a substantially L shape in plan view, the imaging head 2 and the like can be stably supported. In addition, since the base 4 and the arm member 3 are detachably connected, it is convenient to carry and can be stored in a small space.

  FIG. 2 is a diagram schematically illustrating the internal configuration of the imaging head 2. The imaging head 2 mainly includes an image projection unit 13, an image imaging unit 14, and a processor 15.

  The image projection unit 13 is a unit for projecting an arbitrary projection image on a projection plane, and along the projection direction, a substrate 16 and a plurality of LEDs 17 (hereinafter collectively referred to as “LED array 17A”), A light source lens 18, a projection LCD 19, and a projection optical system 20 are provided. The image projection unit 13 will be described in detail with reference to FIG.

  The image capturing unit 14 is a unit for capturing an image of a document P as a subject, and includes an image capturing optical system 21 and a CCD 22 along the light input direction.

  The imaging optical system 21 is composed of a plurality of lenses and has a known autofocus function, and automatically adjusts the focal length and aperture to form an image of light from the outside on the CCD 22.

  The CCD 22 is configured by arranging photoelectric conversion elements such as CCD (Charge Coupled Device) elements in a matrix, and a signal corresponding to the color and intensity of light of an image formed on the surface via the imaging optical system 21. Is converted into digital data and output to the processor 15.

  The processor 15 includes a flash 7, a release button 8, a mode switch 9, a monitor LCD driver 23 through a monitor LCD 10, an RF driver 24 through an antenna 11, a power supply interface 25 through a battery 26, an external memory 27, a cache The LED array 17A via the memory 28, the light source driver 29, the projection LCD 19 via the projection LCD driver 30, and the CCD 22 via the CCD interface 31 are electrically connected and managed by the processor 15.

  The external memory 27 is a detachable flash ROM, and stores captured images and three-dimensional information captured in the digital camera mode, the webcam mode, the stereoscopic image mode, and the designated point measurement mode. Specifically, an SD card, a compact flash (registered trademark) card, or the like can be used.

  The cache memory 28 is a high-speed storage device. For example, the captured image captured in the digital camera mode or the designated point measurement mode is transferred to the cache memory 28 at a high speed, and is processed in the processor 15 before being stored in the external memory 27. Specifically, SDRAM, DDRRAM, or the like can be used.

  The power interface 25 includes a battery 26, the light source driver 29 includes an LED array 17A, the projection LCD driver 30 includes a projection LED 19, and the CCD interface 31 includes various ICs (integrated circuits) that control the CCD 22. ing.

  3A is an enlarged view of the image projection unit 13, FIG. 3B is a plan view of the light source lens 18, and FIG. 3C is a diagram showing an arrangement relationship between the projection LCD 19 and the CCD 22.

  As described above, the image projection unit 13 includes the substrate 16, the LED array 17A, the light source lens 18, the projection LCD 19, and the projection optical system 20 along the projection direction.

  The substrate 16 is used to mount the LED array 17A and to perform electrical wiring with the LED array 17A. Specifically, it is possible to use a substrate having a single layer or a multilayer structure in which an insulating resin is applied to an aluminum substrate and then a pattern is formed by electroless plating, or a core of a color epoxy substrate.

  The LED array 17A is a light source that emits radial light toward the projection LCD 19, and a plurality of LEDs 17 (light emitting diodes) are arranged in a staggered pattern on the substrate 16 and bonded via a silver paste. Further, they are electrically connected via bonding wires.

  By using a plurality of LEDs 17 as a light source in this way, the efficiency of converting electricity into light (electro-optical conversion efficiency) is improved compared with the case where an incandescent bulb, a halogen lamp, or the like is used as a light source. Generation of ultraviolet rays can be suppressed. Therefore, it is possible to drive with power saving, and to save power and extend the life. Moreover, the temperature rise of an apparatus can be reduced.

  As described above, the LED 17 generates extremely low heat rays as compared with a halogen lamp or the like. Therefore, a resin lens can be used as the light source lens 18 and the projection optical system 20 described later. Therefore, compared with the case where a glass lens is employed, each of the lenses 18 and 20 can be configured at low cost and light weight.

  Each LED 17 constituting the LED array 17A emits the same emission color, and is configured to emit amber color using four elements of Al, In, Ga, and P as materials. Therefore, it is not necessary to consider correction of chromatic aberration that occurs when emitting a plurality of emission colors, and it is not necessary to employ an achromatic lens as the projection optical system 20 in order to correct chromatic aberration. The degree of freedom in design can be improved.

  In addition, by using an amber LED of a four-element material having an electro-optical conversion rate as high as about 80 lumen / W compared to other luminescent colors, it is possible to achieve higher brightness, power saving, and longer life. The effect related to arranging the LEDs 17 in a staggered manner will be described with reference to FIG.

  Specifically, the LED array 17A is composed of 59 LEDs 17, and each LED 17 is driven at 50 mW (20 mA, 2.5 V). Eventually, all 59 LEDs 17 are driven with power consumption of approximately 3 W. Further, the brightness as the luminous flux value when the light emitted from each LED 17 passes through the light source lens 18 and the projection LCD 19 and is irradiated from the projection optical system 20 is about 25 ANSI lumens even in the case of the entire surface irradiation. Is set to

  By adopting this brightness, for example, in the stereoscopic image mode, when detecting the three-dimensional shape of a subject such as the face of a person or animal, the person or animal does not give glare to the person or animal. It is possible to detect a three-dimensional shape that is not collapsed.

  The light source lens 18 is a lens that collects light emitted radially from the LED array 17A, and the material thereof is made of an optical resin typified by acrylic.

  Specifically, the light source lens 18 includes a convex lens portion 18a that protrudes toward the projection LED 19 at a position facing each LED 17 of the LED array 17A, and a base portion 18b that supports the lens portion 18a. An epoxy encapsulant 18c for sealing the LED 17 and filling the substrate 16 and the light source lens 18 in an internal space of the base portion 18b and filling an opening containing the LED array 17A, and the base portion 18b And a positioning pin 18d for connecting the light source lens 18 and the substrate 16 to each other.

  The light source lens 18 is fixed on the substrate 16 by inserting positioning pins 18 d into the long holes 16 formed in the substrate 16 while enclosing the LED array 17 </ b> A inside the opening.

  Therefore, the light source lens 18 can be disposed in a space-saving manner. In addition to the function of mounting the LED array 17A on the substrate 16, the function of supporting the light source lens 18 is also used, so that a part for supporting the light source lens 18 is not required and the number of parts is reduced. be able to.

  Each lens portion 18a is disposed at a position facing each LED 17 of the LED array 17A in a one-to-one relationship.

  Therefore, the radial light emitted from each LED 17 is efficiently condensed by each lens unit 18 facing each LED 17 and irradiated to the projection LED 19 as radiation having high directivity as shown in the figure. The reason why the directivity is improved in this way is that the uneven transmittance in the plane can be suppressed by making light incident on the projection LCD 19 substantially perpendicularly. At the same time, since the projection optical system 20 has telecentric characteristics and its incident NA is about 0.1, it is regulated so that only light within ± 5 ° vertical can pass through the internal diaphragm. . Therefore, it is important to improve the image quality to align the light emitted from the LED 17 with the vertical emission angle and to enter a light flux of about ± 5 °.

  The projection LCD 19 is a spatial modulation element that spatially modulates the light collected through the light source lens 18 and outputs image signal light to the projection optical system 20. It is composed of plate-like liquid crystal displays having different ratios.

  Further, as shown in (C), each pixel constituting the projection LCD 19 includes one pixel column arranged in a straight line along the longitudinal direction of the liquid crystal display, and the one pixel column is a liquid crystal display. Other pixel columns that are shifted by a predetermined distance in the longitudinal direction are alternately arranged in parallel.

  Note that (C) is a state in which the front surface of the imaging head 2 is directed to the front side of the paper, light is irradiated toward the projection LCD 19 from the back side of the paper, and a subject image is formed on the CCD 22 from the side of the paper.

  In this way, by arranging the pixels constituting the projection LCD 19 in a staggered pattern in the longitudinal direction, the light subjected to spatial modulation by the projection LCD 19 in the direction orthogonal to the longitudinal direction (short direction) is ½ pitch. Can be controlled. Therefore, the projection pattern can be controlled with a fine pitch, and the three-dimensional shape can be detected with high accuracy by increasing the resolution.

  In particular, in a stereoscopic image mode, a planarized image mode, a volume / weight measurement mode, and a specified point measurement mode, which will be described later, a striped pattern light in which light and dark are alternately arranged toward the subject in order to detect the three-dimensional shape of the subject. By aligning the fringe direction with the short direction of the projection LCD 19, it is possible to control the light / dark boundary with a 1/2 pitch, and similarly detect a three-dimensional shape with high accuracy. can do.

  In the imaging head 2, the projection LCD 19 and the CCD 22 are arranged in a relationship as shown in (C). Specifically, since the wide surface of the projection LCD 19 and the wide surface of the CCD 22 are arranged in substantially the same direction, when an image projected on the projection surface from the projection LCD 19 is formed on the CCD 22, The projected image can be formed as it is without bending the projected image with a half mirror or the like.

  The CCD 22 is disposed on the longitudinal direction side (the direction side in which the pixel column extends) of the projection LCD 19. Therefore, in particular, in the case of detecting the three-dimensional shape of the subject using the principle of triangulation in the stereoscopic image mode, the planarized image mode, the volume / weight measurement mode, or the designated point measurement mode, the CCD 22 and the subject Since the formed inclination can be controlled at 1/2 pitch, a three-dimensional shape can be detected with high accuracy as well.

  The projection optical system 20 is a plurality of lenses that project the image signal light that has passed through the projection LED 19 toward the projection surface, and is composed of a telecentric lens made of a combination of glass and resin. Telecentric means a configuration in which the principal ray passing through the projection optical system 20 is parallel to the optical axis in the incident side space and the position of the exit pupil is infinite. By using telecentricity in this way, only light passing through the projection LCD 19 at a vertical angle of ± 5 ° can be projected as described above, so that the image quality can be improved.

  FIG. 4 is a diagram for explaining the arrangement of the LED array 17A. (A) is a figure which shows the illumination intensity distribution of the light which passed the light source lens 18, (b) is a top view which shows the arrangement state of LED array 17A, (c) shows the synthetic | combination illumination intensity distribution in projection LCD19 surface. FIG.

  As shown in (a), the light that has passed through the light source lens 18 reaches the surface of the projection LCD 19 as light having an illuminance distribution as shown on the left side at a half-value spread half angle θ (= approximately 5 °). Designed to be.

  Further, as shown in (b), the plurality of LEDs 17 are arranged in a staggered pattern on the substrate 16. Specifically, an LED row in which a plurality of LEDs 17 are arranged in series at a d pitch is arranged in parallel at a √3 / 2d pitch, and the LED rows are moved 1 / 2d in the same direction every other row. It is arranged to be in a state.

  In other words, the distance between one LED 17 and the LCD 17 around the one LED 17 is set to be d (triangular lattice arrangement).

  The length d is determined to be equal to or less than the full width at half maximum (FWHM (Full Width Half Maximun)) of the illuminance distribution formed in the projection LCD 19 by the light emitted from one of the LEDs 17.

  Accordingly, the combined illuminance distribution of the light passing through the light source lens 18 and reaching the surface of the projection LCD 19 is substantially linear including small ripples as shown in FIG. Can be irradiated. Therefore, the illuminance unevenness in the projection LCD 19 can be suppressed, and as a result, a high-quality image can be projected.

  FIG. 5 is an electrical block diagram of the image input / output device 1. Note that the description of the above-described configuration is omitted. The processor 15 includes a CPU 35, a ROM 36, and a RAM 37.

  The CPU 35 uses the RAM 37 in accordance with the processing by the program stored in the ROM 36 to detect the pressing operation of the release button 8, fetch the image data from the CCD 22, transfer and store the image data, and the mode switch 9. Various processes, such as detection of the state of, are performed.

  The ROM 36 includes a camera control program 36a, a pattern light photographing program 36b, a luminance image generation program 36c, a code image generation program 36d, a code boundary extraction program 36e, a lens aberration correction program 36f, and a triangulation calculation program 36g. A document orientation calculation program 36h and a plane conversion program 36i are stored.

  The camera control program 36a is a program related to the control of the entire imaging apparatus 1 including the main process shown in FIG.

  The pattern light imaging program 36b is a program for imaging a state in which pattern light is projected onto a subject and a state in which the original is not projected in order to detect the three-dimensional shape of the document P.

  The luminance image generation program 36c takes the difference between the pattern light presence image obtained by imaging the pattern light projected by the pattern light imaging program 36b and the pattern light no image obtained by imaging the state where the pattern light is not projected. This is a program for generating a luminance image of the patterned light.

  In addition, a plurality of types of pattern light are projected in time series and imaged for each pattern light, and the difference between each of the plurality of captured pattern light images and the pattern light no image is obtained. A luminance image is generated.

  The code image generation program 36d is a program that generates a code image in which a plurality of luminance images generated by the luminance image generation program 36c are superimposed and a predetermined code is assigned to each pixel.

  The code boundary extraction program 36e obtains code boundary coordinates with sub-pixel accuracy using the code image generated by the code image generation program 36d and the luminance image generated by the luminance image generation program 36c. It is a program.

  The lens aberration correction program 36f is a program that corrects the aberration of the imaging optical system 20 with respect to the code boundary coordinates obtained with subpixel accuracy by the code boundary extraction program 36e.

  The triangulation calculation program 36g is a program for calculating the three-dimensional coordinates of the real space related to the boundary coordinates from the boundary coordinates of the code subjected to the aberration correction by the lens aberration correction program 36f.

  The document orientation calculation program 36h is a program for estimating the three-dimensional shape of the document P from the three-dimensional coordinates calculated by the triangulation calculation program 36g.

  The plane conversion program 36i is a program that generates a planarized image that is captured from the front of the document P, based on the three-dimensional shape of the document P calculated by the document orientation calculation program 36h.

  The RAM 37 includes a pattern light existence image storage unit 37a, a pattern light no image storage unit 37b, a luminance image storage unit 37c, a code image storage unit 37d, a code boundary coordinate storage unit 37e, an ID storage unit 37f, Aberration correction coordinate storage unit 37g, three-dimensional coordinate storage unit 37h, document orientation calculation result storage unit 37i, plane conversion result storage unit 37j, projection image storage unit 37k, work content information storage unit 37l, three-dimensional An information storage unit 37m, a pointing coordinate input unit 37n, and a working area 37o are allocated as storage areas.

  The pattern light present image storage unit 37a stores a pattern light present image obtained by imaging the state in which the pattern light is projected onto the document P by the pattern light photographing program 36b. The pattern light no image storage unit 37b stores a pattern light no image obtained by imaging the state in which the pattern light is not projected onto the document P by the pattern light photographing program 36b.

  The luminance image storage unit 37c stores a luminance image generated by the luminance image generation program 36c. The code image storage unit 37d stores a code image generated by the code image generation program 36d. The code boundary coordinate storage unit 37e stores the boundary coordinates of each code obtained by the code boundary extraction program 36e with subpixel accuracy extracted. The ID storage unit 37f stores an ID assigned to a luminance image having a change in brightness at a pixel position having a boundary. The aberration correction coordinate storage unit 37g stores the boundary coordinates of the code subjected to the aberration correction by the lens aberration correction program 36f. The three-dimensional shape coordinate storage unit 37h stores the three-dimensional coordinates of the real space calculated by the triangulation calculation program 36g.

  The document orientation calculation result storage unit 37i stores parameters related to the three-dimensional shape of the document P calculated by the document orientation calculation program 36h. The plane conversion result storage unit 37j stores the plane conversion result generated by the plane conversion program 36i. The projection image storage unit 37k stores image information to be projected from the image projection unit 13. The working area 37o stores data temporarily used for calculation by the CPU 15.

  The work content information storage unit 37l stores work content information that is information indicating the work content in the volume / weight measurement mode. The three-dimensional information storage unit 37m stores the three-dimensional information acquired in the volume / weight measurement mode or the designated point measurement mode. The finger point coordinate storage unit 37n stores a designated point designated by a finger point by the user in the designated point measurement mode. The working area 37o stores data temporarily used for calculation by the CPU 15.

FIG. 6 is a flowchart of the main process. Digital camera processing (S605), webcam processing (S607), and stereoscopic image processing (S609) in this main processing.
), Planarized image processing (S611), volume / weight measurement processing (S613), and designated point measurement processing (S615) will be described in detail later.

  In the main process, first, when the power supply is activated (S601), the processor 15 and other interfaces are initialized (S602).

  Then, a key scan for determining the state of the mode switch 9 is performed (S603), it is determined whether or not the setting of the mode switch 9 is the digital camera mode (S604), and if it is the digital camera mode (S604: Yes), The process proceeds to a digital camera process to be described later (S605).

  On the other hand, if it is not the digital camera mode (S604: No), it is determined whether or not the setting of the mode switch 9 is the webcam mode (S606). If it is the webcam mode (S606: Yes), the process proceeds to the webcam process described later. (S607).

  On the other hand, if it is not the webcam mode (S605: No), it is determined whether the setting of the mode switch 9 is the stereoscopic image mode (S608). If it is the stereoscopic image mode (S608: Yes), the stereoscopic image processing described later is performed. (S609).

  On the other hand, if it is not the stereoscopic image mode (S608: No), it is determined whether or not the setting of the mode switch 9 is the planar image mode (S610). If it is the planar image mode (S610: Yes), it will be described later. The process proceeds to planarized image processing (S611).

  On the other hand, if it is not the planar image mode (S610: No), it is determined whether the setting of the mode switch 9 is the volume / weight measurement mode (S612), and if it is the volume / weight measurement mode (S612: Yes). Then, the process proceeds to a volume / weight measurement process described later (S613).

  On the other hand, if it is not the volume / weight measurement mode (S612: No), it is determined whether the setting of the mode switch 9 is the designated point measurement mode (S614), and if it is the designated point measurement mode (S614: Yes), The process proceeds to a designated point measurement process to be described later (S615).

  On the other hand, if it is not the designated point measurement mode (S614: No), it is determined whether the setting of the mode switch 9 is in the off mode (S612). If it is not the off mode (S612: No), the processing from S603 is performed. If it is repeatedly in the off mode (S612: Yes), the process ends.

  FIG. 7 is a flowchart of the digital camera process (S605 in FIG. 6). The digital camera process is a process for acquiring an image captured by the image capturing unit 14.

  In this process, first, a high resolution setting signal is transmitted to the CCD 22 (S701). Thereby, a high quality captured image can be provided to the user.

  Next, a finder image (an image in a range visible through the finder 6) is displayed on the monitor LCD 10 (S702). Therefore, the user can confirm the captured image (imaging range) before actual imaging with the image displayed on the monitor LCD 10 without looking into the finder 6.

  Next, the release button 8 is scanned (S703a), and it is determined whether the release button 8 is half-pressed (S703b). If half-pressed (S703b: Yes), the auto focus (AF) and automatic exposure (AE) functions are activated to adjust the focus, aperture, and shutter speed (S703c). If it is not half-pressed (S703b: No), the processing from S703a is repeated.

  Next, the release button 8 is scanned again (S703d), and it is determined whether or not the release button 8 has been fully pressed (S703e). If it is fully pressed (S703e: Yes), it is determined whether or not the flash mode is set (S704).

  As a result, if the flash mode is selected (S704: Yes), the flash 7 is projected (S705) and photographed (S706). If the flash mode is not selected (S704: No), the flash 7 is not projected. A picture is taken (S706). If it is determined in S703e that the button has not been fully pressed (S703e: No), the processing from S703a is repeated.

  Next, the captured image is transferred from the CCD 22 to the cache memory 28 (S707), and the captured image stored in the cache memory 28 is displayed on the monitor LCD 10 (S708). In this way, by transferring the captured image to the cache memory 28, the captured image can be displayed on the monitor LCD 10 at a higher speed than when transferring to the main memory. Then, the captured image is stored in the external memory 27 (S709).

  Finally, it is determined whether or not the mode change switch 9 has changed (S710). If there is no change (S710: Yes), the processing from S702 is repeated, and if there is a change (S710: No), The process ends.

  FIG. 8 is a flowchart of the webcam process (S607 in FIG. 6). The webcam process is a process of transmitting a captured image (including a still image and a moving image) captured by the image capturing unit 14 to an external network. In this embodiment, it is assumed that a moving image is transmitted as an imaged image to an external network.

  In this process, a low resolution setting signal is first transmitted to the CCD 22 (S801), the well-known autofocus (AF) and automatic exposure (AE) functions are activated, and the focus, aperture, and shutter speed are adjusted (S802). ), Shooting is started (S803).

  Then, the captured image is displayed on the monitor LCD 10 (S804), the finder image is stored in the projection image storage unit 37k (S805), the projection process described later is performed (S806), and stored in the projection image storage unit 37k. Project the projected image onto the projection plane.

  Further, the captured image is transferred from the CCD 22 to the cache memory 28 (S807), and the captured image transferred to the cache memory 28 is transmitted to the external network via the RF driver 24 and the antenna 11 which are RF interfaces (S808).

  Finally, it is determined whether or not the mode change switch 9 has changed (S809). If there is no change (S809: Yes), the processing from S802 is repeated. If there is a change (S809: No), The process ends.

  FIG. 9 is a flowchart of the projection process (S806 in FIG. 8). This process is a process of projecting the image stored in the shadow image storage unit 37k from the projected image projection unit 13 onto the projection plane. In this process, first, it is confirmed whether or not an image is stored in the projection image storage unit 37k (S901). If stored (S901: Yes), the image stored in the projection image storage unit 37k is transferred to the projection LCD driver 30 (S902), and the projection LCD driver 30 outputs an image signal corresponding to the image to the projection LCD 19. The image is displayed on the projection LCD 19 (S903).

  Next, the light source driver 29 is driven (S904), the LED array 17A is turned on by an electrical signal from the light source driver 29 (S905), and the process is terminated.

  Thus, when the LED array 17A is turned on, the light emitted from the LED array 17A reaches the projection LCD 19 via the light source lens 18, and the projection LCD 19 performs spatial modulation according to the image signal transmitted from the projection LCD driver 30. And output as image signal light. The image signal light output from the projection LCD 19 is projected as a projection image on the projection surface via the projection optical system 20.

  FIG. 10 is a flowchart of the stereoscopic image processing (S609 in FIG. 6). The stereoscopic image processing is processing for detecting a three-dimensional shape of a subject and acquiring, displaying, and projecting a three-dimensional shape detection result image as the stereoscopic image.

  In this process, first, a high resolution setting signal is transmitted to the CCD 22 (S1001), and a finder image is displayed on the monitor LCD 10 (S1002).

  Next, the release button 8 is scanned (S1003a), and it is determined whether the release button 8 is half-pressed (S1003b). If half-pressed (S1003b: Yes), the auto focus (AF) and automatic exposure (AE) functions are activated to adjust the focus, aperture, and shutter speed (S1003c). If not half-pressed (S1003b: No), the processing from S1003a is repeated.

  Next, the release button 8 is scanned again (S1003d), and it is determined whether or not the release button 8 is fully pressed (S1003e). If it is fully pressed (S1003e: Yes), it is determined whether or not the flash mode is set (S1003f).

  As a result, if the flash mode is selected (S1003f: Yes), the flash 7 is projected (S1003g), photographed (S1003h), and if the flash mode is not selected (S1003f: No), the flash 7 is not projected. A picture is taken (S1003h). If it is determined in S1003e that the button has not been fully pressed (S1003e: No), the processing from S1003a is repeated.

  Next, a three-dimensional shape detection process described later is performed to detect the three-dimensional shape of the subject (S1006).

  Next, the three-dimensional shape detection result in the three-dimensional shape detection process (S1006) is stored in the external memory 27 (S1007), and the three-dimensional shape detection result is displayed on the monitor LCD 10 (S1008). The three-dimensional shape detection result is displayed as an aggregate of three-dimensional coordinates (X, Y, Z) in the real space of each measurement vertex.

  Next, a three-dimensional shape detection result image as a three-dimensional image (3D CG image) displaying the surface by connecting the measurement vertices as a three-dimensional shape detection result with a polygon is stored in the projection image storage unit 37k (S1009). A projection process similar to the projection process of S806 in FIG. 8 is performed (S1010). In this case, the coordinates on the projection LCD 19 with respect to the obtained three-dimensional coordinates are obtained by using the inverse function of the formula for converting the coordinates on the projection LCD 19 described in FIG. 18 into the three-dimensional space coordinates. The three-dimensional shape result coordinates can be projected on the projection plane.

  Then, it is determined whether or not the mode change switch 9 has changed (S1011). If there is no change (S1011: Yes), the processing from S702 is repeated. If there is a change (S1011: No), the processing is performed. finish.

  FIG. 11A is a diagram for explaining the principle of the spatial code method used to detect a three-dimensional shape in the above-described three-dimensional shape detection process (S1006 in FIG. 10). It is a figure which shows the pattern light different from (a). Either (a) or (b) may be used for the pattern light, and further, a gray level code which is a multi-tone code may be used.

  Details of this spatial coding method are described by Kosuke Sato and one other, “Distance Image Input by Spatial Coding”, IEICE Transactions, 85/3 Vol. J 68-D No3 p369-375.

  The spatial code method is one type of method for detecting the three-dimensional shape of a subject based on triangulation between projected light and an observed image. As shown in (a), the distance between the projection light source L and the observation device O is determined. It is characterized in that it is set apart by D, and the space is divided into long and narrow fan-shaped regions and coded.

  When the three mask patterns A, B, and C in the figure are projected in order from the MSB, each fan-shaped area is coded into bright “1” and dark “0” by the mask. For example, since the area including the point P is not exposed to light in the masks A and B and bright in the mask C, it is encoded as 001 (A = 0, B = 0, C = 1).

  Each fan-shaped region is assigned a code corresponding to the direction φ, and each can be regarded as one slit beam. Therefore, the scene is photographed for each mask with a camera as an observation device, and the bit plane of the memory is constructed by binarizing the light / dark pattern.

  Thus, the horizontal position (address) of the obtained multiple bit-plane image corresponds to the observation direction θ, and the contents of the memory at this address give the projected light code, that is, φ. The coordinates of the point of interest are determined from θ and φ.

  In addition, as a mask pattern used in this method, a case where a pure binary code such as mask patterns A, B, and C is used is illustrated in FIG. There is a risk that a large error will occur.

  For example, the point Q in (a) indicates the boundary between the region 3 (011) and the region 4 (100), but if 1 of the mask A is shifted, the code in the region 7 (111) may be generated. In other words, a large error may occur when the Hamming distance is 2 or more between adjacent regions.

  Therefore, as the mask pattern used in this method, as shown in (b), by using a code whose hamming distance is always 1 between adjacent regions, the coding error as described above can be avoided. It is said that.

  FIG. 12A is a flowchart of the three-dimensional shape detection process (S1006 in FIG. 10). In this process, first, an imaging process is performed (S1210). This imaging process uses a plurality of pure binary code mask patterns shown in FIG. 11 (a) to emit striped pattern light (see FIG. 1) in which light and dark are alternately arranged from the image projection unit 13. This is a process of acquiring a pattern light existence image obtained by sequentially projecting on a subject and imaging a state in which each pattern light is projected, and a pattern light no image obtained by imaging a state in which the pattern light is not projected.

  When the imaging process is completed (S1210), a three-dimensional measurement process is performed (S1220). The three-dimensional measurement process is a process of actually measuring the three-dimensional shape of the subject using the pattern light existence image and the pattern light no image acquired by the imaging process. Thus, when the three-dimensional measurement process ends (S1220), the process ends.

  FIG. 12B is a flowchart of the imaging process (S1210 in FIG. 12A). This process is executed based on the pattern light imaging program 36a. First, the image imaging unit 14 images a subject without projecting pattern light from the image projection unit 13, thereby acquiring a pattern light no-image (S1211). ). The acquired pattern light no-image is stored in the pattern light no-image storage unit 37b.

  Next, the counter i is initialized (S1212), and it is determined whether or not the value of the counter i is the maximum value imax (S1213). The maximum value imax is determined by the number of mask patterns to be used. For example, when 8 types of mask patterns are used, the maximum is imax (= 8).

  If it is determined that the value of the counter i is smaller than the maximum value imax (S1213: Yes), the i-th mask pattern among the mask patterns to be used is displayed on the projection LCD 19, and the i-th mask pattern is displayed. The i-th pattern light projected by the above is projected onto the projection surface (S1214), and the state where the pattern light is projected is photographed by the image capturing unit 14 (S1215).

  In this way, a pattern light existence image obtained by imaging the state in which the i-th pattern light is projected onto the subject is acquired. The acquired pattern light existence image is stored in the pattern light existence image storage unit 37a.

  When the photographing is finished, the projection of the i-th pattern light is finished (S1216), "1" is added to the counter i to project the next pattern light (S1217), and the processing from S1213 is repeated.

  If it is determined that the value of the counter i is greater than the maximum value imax (S1213: No), the process ends. That is, in this imaging process, one pattern light no image and the maximum value imax pattern light existence images are acquired.

  FIG. 12C is a flowchart of the three-dimensional measurement process (S1220 in FIG. 12A). This process is executed based on the luminance image generation program 36c, and first generates a luminance image (S1221). Here, the luminance is a Y value in the YCbCr space, and is a value calculated from Y = 0.22989 · R + 0.5866 · G + 0.1145 · B from the RGB value of each pixel. By obtaining the Y value for each pixel, a luminance image relating to the image with and without pattern light is generated. The generated luminance image is stored in the luminance image storage unit 37c. A number corresponding to the pattern light number is assigned to each luminance image.

  Next, the code image generation program 36d generates a code image coded for each pixel by combining the generated luminance images using the spatial code method described above (S1222).

  The code image can be generated by comparing each pixel of the luminance image related to the image with pattern light stored in the luminance image storage unit 37c with a predetermined luminance threshold and combining the results. The generated code image is stored in the code image storage unit 37d.

  Next, a code boundary coordinate detection process described later is performed by the code boundary extraction program 36e (S1223), and the boundary coordinates of the code assigned to each pixel are detected with subpixel accuracy.

  Next, lens aberration correction processing is performed by the lens aberration correction program 36f (S1224). By this processing, it is possible to correct the error in the code boundary coordinates detected in S1223 that includes an error due to the influence of distortion or the like of the imaging optical system 21.

  Next, real space conversion processing based on the triangulation principle is performed by the triangulation calculation program 36g (S1225). The code boundary coordinates on the CCD space after the aberration correction is performed by this processing are converted into the three-dimensional coordinates in the real space, and the three-dimensional coordinates as the three-dimensional shape detection result are obtained.

  FIG. 13 is a diagram for explaining the outline of the code boundary coordinate detection process (S1223 in FIG. 12). In the upper diagram, the bright and dark boundaries of the actual pattern light in the CCD space are indicated by the boundary line K. The pattern light is encoded by the spatial code method described above, and the boundary between one code and another code is indicated by a bold line in the figure. It is the figure shown by.

  That is, since the coding in the spatial coding method described above is performed on a pixel-by-pixel basis, an error in sub-pixel accuracy occurs between the boundary line K of the actual pattern light and the coded boundary (thick line in the figure). Therefore, the purpose of this code boundary coordinate detection process is to detect code boundary coordinates with sub-pixel accuracy.

  In this process, first, at a certain detection position (hereinafter referred to as “curCDX”), a first pixel G that changes from a certain target code (hereinafter referred to as “curCode”) to another code is detected (first pixel detection step). .

  For example, in curCDXX, when each pixel is detected in order from the top, it is a pixel having curCode up to the boundary (thick line), but curCode changes in the pixel next to the boundary, that is, the first pixel G. This is detected as the first pixel G.

  Next, at the pixel position of the first pixel G, all of the luminance images having a change in brightness are extracted from the luminance images stored in the luminance image storage unit 37c in S1221 of FIG. 12 (luminance image extraction step). ).

  Next, in order to specify a pixel area to be used for approximation, the detection position is moved to the left by “2”, and the code image is referred to at the position of the detection position curCDXX-2 to change from the target code (curCode). Pixel (boundary pixel (pixel H at the detection position of cur CCDX-2)) is searched for, and a predetermined range centered on that pixel (in the present embodiment, -3 pixels and +2 pixels in the Y-axis direction) Range) is specified (part of the pixel area specifying means).

  Next, within the predetermined range, as shown in the lower left graph in the figure, an approximate expression (shown by a solid line) in the Y direction is obtained, and the approximate expression The Y coordinate Y1 at the intersection with the luminance threshold value bTh is obtained (part of the boundary coordinate detection step).

  Note that the luminance threshold value bTh may be calculated from a predetermined range (for example, one half of the average luminance of each pixel) or may be a fixed value given in advance. As a result, the boundary between light and dark can be detected with sub-pixel accuracy.

  Next, the detection position is moved to the right side of “1” from curCCDX-2, and the same processing as described above is performed in curCCDX-1 to obtain a representative value in curCCDX-1 (part of the boundary coordinate detection step).

  In this manner, each detection is performed in a pixel area (see the lower right hatched portion in the figure) composed of a predetermined range in the Y-axis direction centering on the boundary pixel and a range of curCCDX-2 to curCCDX + 2 in the X-axis direction. The representative value at the position is obtained.

  The above processing is performed on all luminance images having pixels that change from curCode to another code, and a weighted average value of representative values for each luminance image is finally adopted as boundary coordinates in curCode (boundary coordinate detection step) Part of).

  As a result, the boundary coordinates of the code can be detected with high accuracy with sub-pixel precision, and by performing the real space conversion process (S1225 in FIG. 12) based on the above-described triangulation principle using the boundary coordinates, The three-dimensional shape of the subject can be detected with high accuracy.

  In addition, since the boundary coordinates can be detected with sub-pixel accuracy by using the approximate expression calculated based on the luminance image in this way, the number of images to be captured is not increased as in the prior art, and pure binary. Pattern light brightened and darkened by a code may be used, and it is not necessary to use a gray code which is a special pattern light.

  In this embodiment, each detection position is composed of a range from “−3” to “+2” in the Y-axis direction around the boundary pixel, and a range from curCCDX-2 to curCCDX + 2 as the detection position in the X-axis direction. Although the region to be processed has been described as a pixel region for obtaining approximation, the ranges of the Y axis and the X axis of the pixel region are not limited to these. For example, only a predetermined range in the Y-axis direction centering on the boundary pixel at the curCDX detection position may be set as the pixel region.

  FIG. 14 is a flowchart of the code boundary coordinate detection process (S1223 in FIG. 12). This process is executed based on the code boundary extraction program 36e. First, each element of the code boundary coordinate sequence in the CCD space is initialized (S1401), and curCCDX is set as the start coordinate (S1402).

  Next, it is determined whether curCCDX is equal to or less than the end coordinate (S1403). If it is equal to or less than the end coordinate (S1403: Yes), curCode is set to “0” (S1404). That is, curCode is initially set to a minimum value.

  Next, it is determined whether curCode is smaller than the maximum code (S1405). If the curCode is smaller than the maximum code (S1405: Yes), the curCCDX refers to the code image to search for the curCode pixel (S1406), and determines whether the curCode pixel exists (S1407).

  As a result, if a curCode pixel exists (S1407: Yes), the curCDXX searches for a code pixel larger than the curCode by referring to the code image (S1408), and a curCode pixel larger than the curCode is found. It is determined whether or not it exists (S1409).

  As a result, if there is a pixel with a code larger than curCode (S1409: Yes), a process for obtaining a later-described boundary with subpixel accuracy is performed (S1410). Then, in order to obtain boundary coordinates for the next curCode, “1” is added to curCode (S1411), and the processing from S1405 is repeated.

  That is, since the boundary exists at the pixel position of the pixel having the curCode or the pixel position of the pixel of the Code that is larger than the curCode, in the present embodiment, the boundary is temporarily the pixel position of the pixel of the CurCode that is larger than the curCode. It is assumed that the process is in progress.

  In addition, when curCode does not exist (S1407: No), or when a pixel with a code larger than curCode does not exist (S1409: No), in order to obtain boundary coordinates for the next curCode, “ 1 "is added (S1411), and the processing from S1405 is repeated.

  Thus, for curCode from 0 to the maximum code, the processing from S1405 to S1411 is repeated, and when curCode becomes larger than the maximum code (S1405: No), “dCCDX” is added to curCCDX to change the detection position (S1412). ) The process from S1403 is repeated at the new detection position in the same manner as described above.

  Then, curCDXX is changed, and finally when curCDX becomes larger than the end coordinate (S1403), that is, when the detection from the start coordinate to the end coordinate is completed, the process ends.

  FIG. 15 is a flowchart of processing (S1410 in FIG. 14) for obtaining code boundary coordinates with subpixel accuracy.

  In this process, first, a change in brightness at a pixel position of a pixel having a code larger than the curCode detected in S1409 of FIG. 14 from the luminance image stored in the luminance image storage unit 37c in S1221 of FIG. All the luminance images having are extracted (S1501).

  Then, the mask pattern number of the extracted luminance image is stored in the array PatID [], and the number of extracted luminance images is stored in noPatID (S1502). The arrays PatID [] and noPatID are stored in the ID storage unit 37f.

  Next, the counter i is initialized (S1503), and it is determined whether or not the value of the counter i is smaller than noPatID (S1504). As a result, if it is determined to be small (S1504: Yes), the CCDY value at the boundary is obtained for the luminance image having the mask pattern number of PatID [i] corresponding to the counter i, and the value is stored in fCDY [i]. (S1505).

  When the processing of S1505 is completed, “1” is added to the counter i (S1506), and the processing from S1504 is repeated. If it is determined in S1504 that the value of the counter i is greater than noPatID (S1504: No), that is, when the processing of S1505 is completed for all the luminance images extracted in S1501, the fCCDY obtained in the processing of S1505 The weighted average value of [i] is calculated, and the result is set as a boundary value (S1507).

  In place of the weighted average value, the median value of fCCDY [i] obtained in the processing of S1505 can be calculated, and the result can be used as a boundary value, or the boundary value can be calculated by statistical calculation.

  In other words, the boundary coordinates are expressed by the curCDXX coordinates and the weighted average value obtained in S1507. The boundary coordinates are stored in the code boundary coordinate storage unit 37e, and the process ends.

  FIG. 16 is a flowchart of the process (S1505 in FIG. 15) for obtaining the CCDY value of the boundary for the luminance image having the mask pattern number of PatID [i].

  In this process, first, a process represented by “ccdx = MAX (curCCDX−dx, 0)” for setting a larger value of “curCDX−dx” and “0” as ccdx is performed, and a counter j is set. Initialization is performed (S1601).

  Specifically, “0” in S1601 means the minimum value of the CCDX value. For example, the curCDX value as a detection position is “1” and the preset dx value is “2”. Assuming that “curCDXX-dx” is “−1”, which is smaller than “0” which is the minimum value of the CCDX value, the subsequent processing in “−1” is set as “ccdx = 0”. I do.

  That is, for a position smaller than the minimum value of the CCDX value, processing for excluding the subsequent processing is performed.

  The value of “dx” can be set in advance to an appropriate integer including “0”. In the example described with reference to FIG. 13, this “dx” is set to “2”. According to the example, the ccdx is set to “curCDX-2”.

  Next, it is determined whether or not ccdx <= MIN (curCCDX + dx, ccdW−1) (S1602). That is, “MIN (curCCDX + dx, ccdW−1)” on the left side is a smaller value among “curCDXX + dx” and “ccdW−1” obtained by subtracting “1” from the maximum CCDX value “ccdW”. Therefore, the value is compared with the “ccdx” value.

  That is, for a position larger than the maximum value of the CCDX value, processing for excluding the subsequent processing is performed.

  If the result of determination is that ccdx is smaller than MIN (curCCDX + dx, ccdW−1) (S1602: Yes), the code image and the luminance image to which PatID [i] is assigned are referred to and the pixel having the boundary exists. The eCDY value of the pixel position is obtained (S1603).

  For example, if the detection position is curCCDX-1 shown in FIG. 13, the pixel I is detected as a pixel candidate having a boundary, and the eCDY value is obtained at the position of the pixel I.

  Next, from the luminance image having the mask pattern number of PatID [i], an approximation regarding the luminance in the ccdy direction within the range of MAX (eCDY-dy, 0) <= ccdy <= MIN (eCCY + dy-1, ccdH-1). A polynomial Bt = fb (ccdy) is obtained (S1604).

  Next, the ccdy value at which the approximate polynomial Bt and the luminance threshold value bTh intersect is obtained, and the value is stored in efCCDY [j] (S1605). By these S1604 and S1605, the boundary coordinates with subpixel accuracy can be detected.

  Next, “1” is added to each of ccdx and counter j (S1605), and the processing from S1602 is repeated. That is, the boundary of subpixel accuracy is detected at each detection position within a predetermined range on the left and right with the cur CCDX as the center.

  If it is determined in S1602 that “ccdx” is larger than “MIN (curCCDX + dx, ccdW−1)” (S1602: No), ccCDY [j] calculated in the range of curCCDX−dx to curCCDX + dx is ccdy. = Approximate polynomial of fy (ccdx) is obtained (S1606). Since each value detected in S1605 by this process is used, the detection accuracy of the boundary coordinates can be improved as compared with the case of detecting the boundary coordinates at one detection position.

  The intersection of the approximate polynomial obtained in this way and curCCDX is set as the CCDY value of the boundary for the luminance image having the mask pattern number of PatID [i] (S1607), and the process ends. As shown in the flowchart of FIG. 15, the processing up to this point is executed for each of the extracted luminance images, and a weighted average value is calculated for the obtained boundary coordinates. Since the boundary coordinates are used (S1507), the detection accuracy of the boundary coordinates can be further improved.

  FIG. 17 is a diagram for explaining the lens aberration correction process (S1224 in FIG. 12). In the lens aberration correction processing, as shown in FIG. 17A, the incident light beam is deviated from the position to be imaged by the ideal lens due to the aberration of the imaging optical system 21. This is a process of correcting the position to the position where the image should be originally formed.

  For example, as shown in FIG. 17B, this aberration correction is data obtained by calculating the aberration of the optical system in the imaging range of the imaging optical system 21 using the half field angle hfa that is the angle of incident light as a parameter. Correct based on.

  This aberration correction processing is executed based on the lens aberration correction program 36f and performed on the code boundary coordinates stored in the code boundary coordinate storage unit 37e, and the data subjected to the aberration correction processing is stored in the aberration correction coordinate storage unit 37g. Stored.

  More specifically, the following camera calibration (approximate expression) (1) to (3) is used to convert arbitrary point coordinates (ccdx, ccdy) in the real image to coordinates (ccdcx, ccdcy) in the ideal camera image. To correct.

  In this embodiment, the aberration amount dist (%) is described as dist = f (hfa) using a half angle of view hfa (deg). In addition, the focal length of the imaging optical system 21 is focal length (mm), the ccd pixel length pixel length (mm), and the center coordinates of the lens in the CCD 22 are (Centx, Centy).

(1) ccdcx = (ccdx−Centx) / (1 + dist / 100) + Centx
(2) ccdcy = (ccdy−Centy) / (1 + dist / 100) + Centy
(3) hfa = arctan [(((ccdx-Centx) ² + (ccdy-Centy) ² ) ^0.5 ) × pixellength / focallength]
FIG. 18 is a diagram for explaining a method of calculating the three-dimensional coordinates in the three-dimensional space from the coordinates in the CCD space in the real space conversion process (S1225 in FIG. 12) based on the triangulation principle.

  In the real space conversion process based on the triangulation principle, the triangulation calculation program 36g calculates the three-dimensional coordinates in the three-dimensional space for the code boundary coordinates subjected to the aberration correction stored in the aberration correction coordinate storage unit 37g. The three-dimensional coordinates calculated in this way are stored in the three-dimensional coordinate storage unit 37h.

  In this embodiment, as the coordinate system of the image input / output apparatus 1 for the document P that is curved in the lateral direction to be imaged, the optical axis direction of the imaging optical system 21 is the Z axis, and the imaging optical system 21 is along the Z axis. A point that is VPZ away from the principal point position is an origin, a horizontal direction with respect to the image input / output device 1 is an X axis, and a vertical direction is a Y axis.

  Further, the projection angle θp from the image projection unit 13 to the three-dimensional space (X, Y, Z), D is the distance between the optical axis of the imaging lens optical system 20 and the optical axis of the image projection unit 13, and the imaging optical system 21. The field of view in the Y direction is Yftop to Yfbottom, the field of view in the X direction is from Xfstart to Xfend, the length (height) in the Y-axis direction of the CCD 22 is Hc, and the length (width) in the X-axis direction is Wc. The projection angle θp is given based on a code assigned to each pixel.

In this case, the three-dimensional space position (X, Y, Z) corresponding to the arbitrary coordinates (ccdx, ccdy) of the CCD 22 is on the point on the image plane of the CCD 22, the projection point of the pattern light, and the XY plane. It can be obtained by solving five equations for the triangle formed by the intersecting points.
(1) Y = − (tan θp) Z + PPZ + tan θp−D + cmp (Xtarget)
(2) Y = − (Ytarget / VPZ) Z + Ytarget
(3) X = − (Xtarget / VP) Z + Xtarget
(4) Ytarget = Yftop− (ccdcy / Hc) × (Yftop−Yfbottom)
(5) Xtarget = Xfstart + (ccdcx / Wc) × (Xfend−Xfstart)
Note that cmp (Xtarget) in (1) is a function for correcting the deviation between the imaging optical system 20 and the image projection unit 13, and in an ideal case where there is no deviation, it can be considered that cmp (Xtarget) = 0. it can.

  On the other hand, as described above, the relationship between the arbitrary coordinates (lcdcx, lcdccy) on the projection LCD 19 included in the image projection unit 13 and the three-dimensional coordinates (X, Y, Z) in the three-dimensional space is as follows: It can be expressed by the formulas (1) to (4).

In this embodiment, the principal point position (0, 0, PPZ) of the image projection unit 13, the Y-direction field of the image projection unit 13 from Ypftop to Ypfbottom, the X-direction field of view from Xpfstart to Xpfend, and the Y of the projection LED 19 The length (height) in the axial direction is Hp, and the length (width) Wp in the X-axis direction.
(1) Y = − (Yptarget / PPZ) Z + Yptarget
(2) X = − (Xptarget / PPZ) Z + Xptarget
(3) Yptarget = Ypftop− (lcdcy / Hp) × (Xpftop−Xpfbottom)
(4) Xptarget = Xpfstart + (lcdcx / Wp) × (Xpfend−Xpfstart)
By using this relational expression, the LCD space coordinates (lcdcx, lcdcy) can be calculated by giving the three-dimensional space coordinates (X, Y, Z) to the above expressions (1) to (4). it can. Therefore, for example, an LCD element pattern for projecting an arbitrary shape and character in a three-dimensional space can be calculated.

  FIG. 19 is a flowchart of planarized image processing (S611 in FIG. 6). Planarized image processing is, for example, when a document P in a curved state as shown in FIG. 1 is imaged or when a rectangular document is imaged from an oblique direction (the captured image has a trapezoidal shape). This is a process of acquiring and displaying a flattened image corrected to a state in which the document is not curved or a state in which an image is taken from a direction perpendicular to the surface.

  In this process, first, a high resolution setting signal is transmitted to the CCD 22 (S1901), and a finder image is displayed on the monitor LCD 10 (S1902).

  Next, the release button 8 is scanned (S1903a), and it is determined whether the release button 8 is half-pressed (S1903b). If half-pressed (S1903b: Yes), the auto focus (AF) and automatic exposure (AE) functions are activated to adjust the focus, aperture, and shutter speed (S1903c). If not half-pressed (S1903b: No), the processing from S1903a is repeated.

  Next, the release button 8 is scanned again (S1903d), and it is determined whether the release button 8 has been fully pressed (S1903e). If it is fully pressed (S1903e: Yes), it is determined whether or not the flash mode is set (S1903f).

  As a result, if the flash mode is selected (S1903f: Yes), the flash 7 is projected (S1903g), photographed (S1903h), and if the flash mode is not selected (S1903f: No), the flash 7 is not projected. A picture is taken (S1903h). If it is determined in S1903e that the button has not been fully pressed (S1903e: No), the processing from S1903a is repeated.

  Next, a three-dimensional shape detection process that is the same as the above-described three-dimensional shape detection process (S1006 in FIG. 10) is performed to detect the three-dimensional shape of the subject (S1906).

  Next, based on the three-dimensional shape detection result obtained by the three-dimensional shape detection process (S1906), a document attitude calculation process for calculating the attitude of the document P is performed (S1907). With this process, the position L, the angle θ, and the curvature φ (x) of the document P with respect to the image input apparatus 1 are calculated as the posture parameters of the document P.

  Next, based on the calculation result, a plane conversion process to be described later is performed (S1908), and a flattened image flattened in a state where the original P is not curved even if it is curved is generated.

  Next, the planarized image obtained by the planar change process (S1908) is stored in the external memory 27 (S1909), and the planarized image is displayed on the monitor LCD 10 (S1910).

  Then, it is determined whether or not the mode change switch 9 has changed (S1911). If there is no change (S1911: Yes), the processing from S702 is repeated again, and if there is a change (S1911: No). ), The process ends.

  FIG. 20 is a diagram for explaining the document orientation calculation process (S1907 in FIG. 19). Note that, as an assumption condition for a document such as a book, it is assumed that the curvature of the document P is uniform in the y direction. In this document orientation calculation process, first, as shown in FIG. 20A, regression line approximation is performed on points arranged in two columns at a three-dimensional space position from the coordinate data related to the code boundary stored in the three-dimensional coordinate storage unit 37h. Obtain the two curves.

  For example, it can be obtained from position information (a boundary relating to the boundary between the code 63 and the code 64 and the boundary between the code 191 and the code 192) of the upper and lower quarters of the projected range of the pattern light.

  Assuming a straight line connecting points where the positions of the two curves in the X-axis direction are “0”, the point where the straight line intersects the Z-axis, that is, the point where the optical axis intersects the original P, is 3 A dimension space position (0, 0, L) is defined, and an angle formed by the straight line with the XY plane is defined as an inclination θ around the X axis of the document P.

  Next, as shown in FIG. 20B, the document P is rotationally converted in the reverse direction by the inclination θ around the X axis obtained earlier, that is, the document P is parallel to the XY plane. Assuming that

  Then, as shown in FIG. 20C, the displacement in the Z-axis direction can be expressed as a curve φ (X) as a function of X for the cross section of the document P in the XZ plane. Thus, the position L, the angle θ, and the curvature φ (x) of the document P are calculated as the document orientation parameters, and the process ends.

  FIG. 21 is a flowchart of the plane conversion process (S1908 in FIG. 19). In this process, first, the processing area of the process is assigned to the working area 37o of the RAM 37, and the variable of the counter b used for the process is set to an initial value (b = 0) (S2101).

  Next, based on the position L of the original P, the inclination θ, and the curvature φ (x) based on the calculation result in the original attitude calculation program 36h, the pattern light no-image 4 stored in the pattern light no-image storage unit 37b. Each corner point is obtained by inverse transformation of the curve (a process equivalent to a “curving process” described later) that moves −L in the Z direction, rotates −θ in the X-axis direction, and further changes to φ (x). A rectangular area formed by dots (that is, a rectangular area that is an image in which the surface of the original P on which characters or the like are written is observed from a substantially orthogonal direction) is set, and pixels included in the rectangular area The number a is obtained (S2102).

  Next, coordinates on the non-patterned light image corresponding to each pixel constituting the set rectangular area are obtained, and pixel information of each pixel of the planarized image is set from pixel information around this coordinate.

  That is, first, it is determined whether or not the counter b has reached the pixel number a (S2103). If the counter b has not reached the pixel number a (S2103: No), a curve calculation process is performed in which one pixel constituting the rectangular area is rotated and moved by a curve φ (x) around the Y axis (S2104). A tilt θ is rotated about the X axis (S2105), and is shifted by a distance L in the Z axis direction (S2106).

  Next, the coordinates (ccdcx, ccdcy) on the CCD image obtained by imaging the ideal camera with the inverse function of the previous triangulation are obtained from the obtained three-dimensional space position (S2107), and the imaging optical system used According to the aberration characteristics of 20, the coordinates (ccdx, ccdy) on the CCD image captured by the actual camera are obtained by the inverse function of the previous camera calibration (S2108), and the pixel of the pattern light no image corresponding to this position Is stored in the working area 37o of the RAM 37 (S2109).

  Then, “1” is added to the counter b in order to execute the above-described processing from S2103 to S2109 for the next pixel (S2110).

  In this way, when the processing from S2104 to S2110 is repeated until the counter b reaches the number of pixels a (S2103: Yes), in S2101, the processing area assigned to the working area 37o is released to execute the processing (S2111). ), The process ends.

  FIG. 22A is a diagram for explaining the outline of the bending process (S2104 in FIG. 21), and FIG. 22B shows the document P flattened by the plane conversion process (S1908 in FIG. 19). Yes. The details of the curving process are disclosed in detail in the IEICE Transactions DIIVol. J86-D2 No. 3 p409 “Curved Document Shooting with an Eye Scanner”.

  The curve Z = φ (x) is obtained by converting a three-dimensional shape constituted by the obtained code boundary coordinate sequence (real space), a cross-sectional shape parallel to the XZ plane at an arbitrary Y value, and a least square method. Is expressed by an equation approximated by a polynomial.

  When flattening a curved curved surface, as shown in (a), the flattened points corresponding to the points on Z = φ (x) are from Z = φ (0) to Z = φ (x). Are associated with each other according to the length of the curve.

  For example, even when the document P in a curved state as shown in FIG. 1 is imaged by such plane conversion processing including the bending processing, as shown in FIG. 22B, a planarized planar image is obtained. Since the accuracy of the OCR process can be improved by using the flattened image in this way, it is possible to clearly recognize characters, figures, and the like described in the original by the image. .

  Next, a volume / weight measurement mode, which is one of the operation modes in the image input / output device 1, will be described with reference to FIGS. The volume / weight measurement mode is a mode for measuring the volume / weight of the subject according to the work content information and projecting the measurement information obtained as a result.

  FIG. 23 is a flowchart of the volume / weight measurement process (S613 in FIG. 6) executed in the volume / weight measurement mode. This volume / weight measurement process is a process of performing measurement according to various work contents such as volume measurement and weight measurement using the three-dimensional shape detection result of the subject and projecting an image based on the measurement result.

  In this process, first, a high resolution setting signal is transmitted to the CCD 22 (S2301). Thereby, a high quality captured image can be provided to the user.

  Next, work content information is received via the antenna 11 (S2302), and stored in the received work content information storage unit 37l (S2303). The “work content information” is information indicating the work content specified by the user, such as a work for dividing a meat chunk into equal volumes or equal weights, or a work for digging a predetermined shape from a piece of wood. When the designated work content is a work that requires acquisition of weight information or weight distribution information of the subject, the work content information includes the density data of the subject.

  Next, a finder image (an image that can be seen through the finder 6) is displayed on the monitor LCD 10 (S2304). Therefore, the user can confirm the captured image (imaging range) before actual imaging with the image displayed on the monitor LCD 10 without looking into the finder 6.

  Next, the release button 8 is scanned (S2305a), and it is determined whether the release button 8 is half-pressed (S2305b). If half-pressed (S2305b: Yes), the auto focus (AF) and automatic exposure (AE) functions are activated to adjust the focus, aperture, and shutter speed (S2305c). If not half-pressed (S2305b: No), the processing from S2305a is repeated.

  Next, the release button 8 is scanned again (S2305d), and it is determined whether or not the release button 8 is fully pressed (S2305e). If it is fully pressed (S2305e: Yes), it is determined whether or not the flash mode is set (S2309).

  If the result of determination in S2309 is flash mode (S2309: Yes), the flash 7 is projected (S2310) and photographed (S2311). If it is determined in S2305e that the button has not been fully pressed (S2305e: No), the processing from S2305a is repeated.

  On the other hand, if the flash mode is not set (S2309: No), the flash 7 is shot without projecting (S2311). If it is determined in S2308 that the button has not been fully pressed (S2308: No), the processing from S2305 is repeated.

  Next, a three-dimensional shape detection process similar to the three-dimensional shape detection process of S1006 in FIG. 10 is performed to detect the three-dimensional shape of the subject (S2312).

  Next, as a three-dimensional shape detection result in the three-dimensional shape detection process (S2312), the three-dimensional coordinates (X, Y, Z) in the real space of each measurement vertex are stored in the three-dimensional information storage unit 37m (S2313).

  Next, based on the three-dimensional shape detection result, measurement according to the work content information stored in the work content information storage unit 371 is performed, and measurement information which is the measurement result is generated, and projection based on the measurement information is performed. Information is generated (S2314). Note that “projection information based on measurement information” is information that is projected in an object or projection direction by a distortion-free image projection process (S2316) described later in accordance with the measurement information. The “projection information based on measurement information” includes, for example, character information or graphic information corresponding to the measurement information, work instruction information such as cutting lines, cutting lines, and cutting depth information according to the work content information. Is exemplified.

  In the process of S2314, when the work content information is work information that does not require measurement information, the work content information corresponding to the work content information stored in the work content information storage unit 37l is based on the three-dimensional shape detection result. Instruction information may be generated, and projection information based on the instruction information may be generated.

  Next, the measurement result image corresponding to the projection information based on the measurement information is stored in the projection image storage unit 37k (S2315), and the measurement result image stored in the projection image storage unit 37k is used for distortion of the subject surface or image projection. The distortion-free image conversion process for correcting the measurement result image stored in the projection image storage unit 37k so that the image is projected in the object or the projection direction as an undistorted image without depending on the projection direction of the unit 13. (S2316) is executed. The distortion-free projection image conversion process (S2316) will be described later.

  Next, a projection process similar to the projection process of S806 in FIG. 8 is performed (S2317). Accordingly, the measurement result image is projected as an undistorted image in the object or the projection direction. When an undistorted measurement result image is projected in the object or the projection direction by the projection processing (S2317), the user performs an operation according to the measurement result image. For example, when the measurement result image is a cutting line that divides the subject into substantially equal volumes, cutting is performed according to the cutting line.

  Next, it is determined whether or not the end of the work for the subject is instructed (S2318). Note that the completion of the work is recognized by operating a work end button (not shown) provided in the image input / output apparatus 1 or detecting a work end signal received from the external network via the antenna 11. The

  If the end of the work is not instructed as a result of the determination in the process of S2316 (S2316: No), the process of S2316 is repeated. On the other hand, when the end of the work is instructed (S2316: Yes), the light source driver 29 is turned off (S2317), and the LED array 17A is turned off by an electric signal from the light source driver 29 (S2318). Therefore, the image projected in the imaging area 100 is not projected.

  Next, it is determined whether or not the mode change switch 9 has changed (S2319). If the mode change switch 9 has not changed (S2319: Yes), the processing from S2302 is repeated. For example, when the work content information is a work for cutting a single meat chunk into a substantially equal volume on a meat processing line, the processes of S2302 to S2319 are repeatedly executed for each meat chunk.

  On the other hand, as a result of the determination in the process of S2319, if there is a change in the mode switch 9 (S2319: No), the process is terminated.

  FIG. 24 is a flowchart of the above-described image conversion process for distortion-free projection (S2316 in FIG. 23). This distortion-free projection image conversion process (S2316) projects an image displayed on the projection LCD 19 in a distortion-free state in the subject or in the projection direction in accordance with image information such as measurement result information stored in the projection image storage unit 37k. It is a process of converting into a possible image.

  In this process, first, a processing area of the process is assigned to the working area 37o of the RAM 37, and a variable of the counter q used for the process is set to an initial value (q = 0) (S2401).

  Next, a space of LCD space coordinates (lcdcx, lcdcy) is set as a rectangular area that becomes an image after being transformed into an image for distortion-free projection (that is, an image that is undistorted on a curved subject) The number of pixels Qa included in this rectangular area is obtained (S2402).

  Next, image information such as measurement result information stored in the projection image storage unit 37k is arranged at coordinates (ccdcx, cdccy) on the ideal camera image (S2403).

  Next, for each pixel on the LCD space coordinates (lcdcx, lcdcy) constituting the set rectangular area, it is a point on the surface of the subject stored in the three-dimensional shape coordinate storage unit 37h by the processing of S2311 described above. By using the three-dimensional coordinates (X, Y, Z), pixel information of each pixel of the image for distortion-free projection is set.

  That is, first, it is determined whether or not the counter q has reached the number of pixels Qa (S2404). If the counter q has not reached the number of pixels Qa (S2404: No), the LCD space coordinates (lcdcx, lcdcy) of the pixel corresponding to the value of the counter q are obtained as coordinates on the subject stored in the external memory 27 (X , Y, Z) (S2405).

  Next, the coordinates (X, Y, Z) on the subject obtained by the conversion in S2405 are converted into coordinates (ccdcx, cdccy) on the ideal camera image (S2406).

  Next, the pixel information arranged at the coordinates (ccdcx, ccdcy) obtained by the conversion in S2406 is acquired, and the pixel information is converted into the LCD space coordinates (lcdcx, lcdcy) corresponding to the value of the counter q. (S2407).

  Then, “1” is added to the counter q in order to execute the above-described processing from S2404 to S2407 for the next pixel (S2408).

  Thus, when the processing from S2404 to S2408 is repeated until the counter q reaches the number of pixels Qa (S2404: Yes), the pixel information associated with the LCD space coordinates (lcdcx, lcdcy) constituting the set rectangular area Are stored in the projection image storage unit 37k (S2409).

  Finally, in S2401, the processing area allocated to the working area 37o for executing the process is released (S2410), and the process is terminated.

  In the projection process (S2317) following the distortion-free projection image conversion process (S2316), the pixel information on the LCD space coordinates (lcdcx, lcdcy) is stored in the projection image storage unit 37k by the process of S2409. On the projection LCD 19, a projection image projected without distortion on the distorted curved surface is displayed. Therefore, an undistorted image is projected on the subject or the projection direction.

  Therefore, by executing the distortion-free image conversion process (S2316), the object has a three-dimensional shape, and therefore the surface is curved or the projection direction is oblique. A simple measurement result image can be projected. As a result, for example, in the operation of cutting the subject into substantially equal volumes, the cutting instruction line can be projected at an accurate position on the subject.

  FIG. 25 is a diagram for explaining a first example in the volume / weight measurement processing mode. In the first example in the volume / weight measurement processing mode, an operation of cutting the meat chunk as the workpiece W1 conveyed by the conveyor 200 into a substantially equal volume is performed in the meat processing line.

  In FIG. 25, in the meat processing line, in order to cut the meat chunk as the work W1 conveyed by the conveyor 200 into a substantially equal volume, the stripes are formed by alternately arranging the light and dark for detecting the three-dimensional shape. The pattern light is projected from the image projection unit 13 of the image input / output device 1 onto the workpiece W1 located in the imaging region 100. FIG. 25 corresponds to part of the processing executed in the above-described three-dimensional shape detection processing (S2310).

  The imaging region 100 is also a surface on which an image can be projected in the projection direction by the image projection unit 14 in the image input / output device 1, that is, an image projection surface. Here, a frame indicating the boundary of the imaging region 100 may be projected from the projection LCD 19 as a projected image. By indicating the boundary of the imaging region 100, the user can clearly grasp the region that can be imaged.

  In the first example of the volume / weight measurement processing mode shown in FIG. 25, one of the bases 4 of the image input / output device 1 is configured to be short so as not to disturb the conveyance of the conveyor 200.

  FIG. 26 is a diagram for explaining a state where a measurement result image is projected as a result of the processing of S2315 in the volume / weight measurement processing described above. When the image input / output device 1 receives the work content information related to the work of dividing the work W1 into four substantially equal volumes, the measurement information is obtained by the process of S2312 based on the three-dimensional shape result generated by the process of S2310. As the projection information based on the volume information, cutting line information for dividing the work W1 into substantially equal volumes is generated. As a result, as shown in FIG. 26, the generated cutting line information is projected onto the workpiece W1 as three linear measurement result images R1. When the user cuts the workpiece W1 based on the measurement result image R1, the workpiece W1 can be divided into four substantially equal volumes.

  When cutting the workpiece W1 based on the measurement result image R1, the image input / output device 1 may be provided with a cutting means such as a cutter so that the workpiece W1 is automatically cut. In this case, the workpiece W1 may be automatically cut by the cutting unit based on the cutting line information generated by the processing of S2310 without projecting the measurement result image R1. As a result, the work cutting operation can be made more efficient.

  In FIG. 26, the work of dividing the work W1 into four parts with substantially equal volumes has been described. However, the image input / output device 1 receives work content information related to the work of dividing the work W1 into a plurality of substantially equal weight blocks. In this case, a measurement result image R1 that divides the work W1 into substantially equal weights is projected onto the work W1. In the case of such work for dividing the work W1 into a plurality of equal weight blocks, the work content information includes density information or density distribution information of the work W1. In this case, in the process of S2312, when the volume information is acquired based on the three-dimensional shape result generated by the process of S2310, the density information or the density distribution information is integrated into the volume to obtain the weight information or the weight distribution information. Obtained as measurement information. Then, as projection information based on the weight information or the weight distribution information, cutting line information for dividing the workpiece W1 into a plurality of masses having substantially equal weights is generated.

  When the image input / output device 1 receives as a work content information that a predetermined weight or a part of a predetermined volume in the workpiece W1 is obtained, in the processing of S2312, the three-dimensional shape result generated by the processing of S2310 is displayed. Since the measurement information corresponding to the work is acquired based on the measurement result, the measurement result image R1 based on the measurement information can be projected onto the work W1.

  As described above, measurement information such as volume information and weight information for the workpiece W1 is obtained based on the three-dimensional shape information of the workpiece W1 according to the work content, and a cutting line or the like is obtained as a measurement result image based on the measurement information. Since the work instruction information is directly projected onto the work W1, the user can efficiently perform the work.

  FIG. 27 is a diagram illustrating a second example in the volume / weight measurement mode described above. In the second example, an operation of cutting a piece of wood as a work W2 arranged in the imaging region 100 into a predetermined shape is performed.

  FIG. 27 shows a state where cutting depth information images R2a to R2b and a cutting line image R3 are projected as measurement result images as a result of the processing of S2315 in the volume / weight measurement processing described above. When the image input / output device 1 receives the work content information related to the work of cutting the workpiece W2 into a predetermined shape, based on the three-dimensional shape result generated by the process of S2310, the measurement information is obtained as the measurement information by the process of S2312. Information on a cutting part necessary for obtaining a shape to be obtained is acquired. Then, as projection information based on the acquired measurement information, cutting depth information for instructing a cutting depth at the time of cutting on the workpiece W2, and information on a cutting line for instructing a predetermined cutting process (for example, rounding a corner). Is generated. As a result, as shown in FIG. 27, the generated cutting depth information and cutting line information are projected on the workpiece W2 as cutting depth information images R2a to R2c and a cutting line image R3, respectively.

  In addition, cutting depth information image R2a-R2c represents the cutting depth by the magnitude | size of the cross mark. For example, in FIG. 27, an instruction is given to cut the portion projected with the cutting depth information image R2a, which is the smallest cross mark, at the shallowest depth. The cross mark is larger in the cutting depth information image R2b and larger in the cutting depth information image R2c, and instructs to cut at a deeper cutting depth in this order.

  Further, the cutting information image R3 is an image corresponding to information on the cutting line generated as a result of the processing of S2312, and is an image for instructing predetermined cutting (for example, rounding a corner). .

  When the user performs cutting according to the cutting depth information images R2a to R2c and the cutting instruction line R3, the workpiece W2 can be formed into a target shape. Since the cutting depth information images R2a to R2c and the cutting instruction line R3 are directly projected onto the workpiece W2, efficient work can be performed.

  As described above, according to the volume / weight measurement mode, various measurements based on the three-dimensional information are executed according to the work content information, and an image corresponding to the measurement result is a work as the subject or the subject. Therefore, measurement information desired by the user can be easily viewed.

  Further, as measurement information based on the three-dimensional information, volume information, weight distribution information, and weight information of a work that is a subject can be obtained, and the work is abbreviated based on the volume information, weight distribution information, and weight information. Work instruction lines for dividing into equal volumes, approximately equal weights, etc. can be projected on the work, so work can be done efficiently based on the work instruction lines projected on the work. It can be carried out.

  Next, a designated point measurement mode that is one of the operation modes in the image input / output device 1 will be described with reference to FIGS. This designated point measurement mode is a mode in which measurement can be performed for a point, a line segment, or a region formed by at least one point designated by a finger or a pointer on a stick.

  FIG. 28 is a diagram for explaining the designated point measurement mode. In FIG. 28, as an example of the designated point measurement mode, measurement of the distance between two points pointed by the user as “two-point measurement” for a part as the workpiece W3 arranged in the imaging region 100 is performed. Is shown.

  In this designated point measurement mode, as shown in FIG. 28, measurement instruction buttons B1 to B3 and an input end button B4 are projected on a predetermined position in the imaging region 100 (on the lower side of the imaging region 100 in FIG. 28). Yes. The measurement instruction buttons B1 to B3 are buttons for instructing execution of the measurement associated with each button. The measurement instruction buttons B1 to B3 are input by being pointed by the user and corresponding measurement is executed.

  The measurement instruction button B1 is used to instruct “3D input”. When this measurement instruction button B1 is input, a three-dimensional shape is measured by a specified point measurement process described later, and a workpiece W3 is obtained as the measurement result image. Are projected.

  The measurement instruction button B2 is for instructing “two-point measurement”, and when this measurement instruction button B2 is input, the distance between the two points designated by the finger pointing with the fingertip or the like is used as the three-dimensional shape detection result. The measurement result image is projected on the workpiece W3 or the imaging region 100. FIG. 28 illustrates a case where “two-point measurement” corresponding to the selection button image B2 is selected. As shown in FIG. 28, the distance between two points between the point on the workpiece W3 pointed by the right hand 300a and the point on the workpiece W3 pointed by the right hand 300b moved from the position of the right hand 300a is measured. Then, a state in which the measurement result image R4 is projected onto the workpiece W3 is illustrated.

  The measurement result image R4 shown in FIG. 28 is a numerical value indicating the distance between two points. By projecting the measurement image as character information in this way, the user can easily recognize the information.

  The measurement instruction button B3 is for instructing “height measurement”. When the measurement instruction button B3 is input, the height between the position designated by the finger pointing by the fingertip or the like and the surface on which the workpiece W3 is placed is set. Is measured, and as the measurement result image, character information indicating the height by a numerical value is projected on the work W3 or the imaging region 100.

  The input end instruction button B4 is used to end the input of the designated point by the fingertip when executing the above “two-point measurement” or “height measurement”. Is input by.

  FIG. 29 is a flowchart of the designated point measurement process (S615 in FIG. 6) executed in this designated point measurement mode. This designated point measurement process is a process for executing the measurement selected by the above-described selection button images B1 to B4 using the 3D shape detection result of the subject and projecting an image based on the measurement result.

  In this process, first, a low resolution moving image setting signal is transmitted to the CCD 22 (S2901). Therefore, since low-resolution moving image data is used, the amount of calculation is small and processing can be performed at high speed.

  Next, the designated point measurement mode initial screen image is stored in the projection image storage unit 37k (S2902). The image data of the designated point measurement mode initial screen image is stored in advance in a region not shown in the ROM 37.

  Next, a finder image is displayed on the monitor LCD 10 (S2902), and then a projection process similar to the projection process of S806 of FIG. 8 is performed (S2904). As a result of the processing of S2904, when the designated point measurement mode initial screen image is projected, the selection button images B1 to B4 are projected at predetermined positions in the imaging region 100 (see FIG. 28).

  Next, after the well-known autofocus (AF) and automatic exposure function (AE) are activated and the focus, aperture, and shutter speed are adjusted (S2905), the inside of the imaging area 100 is photographed at a predetermined interval, and two consecutive images are taken. The generated image is inspected by taking the difference of each pixel of the captured image data to be obtained, and finger pointing coordinates, which are the coordinates pointed by the fingertip, are acquired (S2906). For example, a method described in Japanese Patent Laid-Open No. 7-168949 can be used to determine the finger difference coordinate by the fingertip or the like.

  Next, it is determined whether or not the finger point coordinates acquired in S2906 point to the measurement instruction buttons B1 to B3 (S2907), and any of the measurement instruction buttons B1 to B3 must be pointed. If (S2907: No), the processing of S2906 to S2907 is repeated. On the other hand, if any of the measurement instruction buttons B1 to B3 is pointed (S2907: Yes), the video shooting is stopped (S2908), and the measurement information generation process (S2909) for obtaining the measurement information is executed. . The measurement information generation process (S2909) will be described later.

  After the measurement information generation process (S2909) is executed, the release button 8 is scanned again (S2910), and it is determined whether the release button 8 is fully pressed (S2911). If the release button 8 is not fully pressed (S2911: No). , S2911 is repeated. On the other hand, if it is fully pressed (S2911: Yes), it is determined whether or not it is in the flash mode (S2912). If it is in the flash mode (S2912: Yes), the flash 7 is applied (S2913). Is photographed (S2914). On the other hand, if the flash mode is not set (S2912: No), the image is shot without hitting the flash 7 (S2914). In the process of S2911, before it is determined that the release button 8 is fully pressed, the half-pressed state of the release button is detected, and the auto focus (AF) and automatic exposure (AE) functions are activated, It is assumed that the focus, aperture, and shutter speed are adjusted.

  Although details will be described later, an image based on the measurement information generated according to the instruction measurement instruction buttons B1 to B3 selected by the user is projected into the imaging region 100 after the execution of the measurement information generation process (S2909). Therefore, an image based on the measurement information is captured by the process of S2914. Accordingly, an image based on the projected measurement information can be recorded as captured image data.

  Next, the captured image captured by the processing of S2914 is transferred from the CCD 22 to the cache memory 28 (S2915), and the captured image transferred to the cache memory 28 is stored in the external memory 27 (S2916). Therefore, various measurement information obtained as a result of measurement according to the type of measurement instruction button can be stored as an image.

  Next, the light source driver 29 is turned off (S2917), and the LED array 17A is turned off by an electric signal from the light source driver 29 (S2918). Therefore, the image projected in the imaging area 100 is not projected.

  Next, it is determined whether or not the mode change switch 9 has changed (S2919). If there is no change in the mode change switch 9 (S2919: Yes), the processing returns to S2901, and the processing of 2901 to S2919 is repeated. On the other hand, if the result of the determination in S2919 is that there is a change in the mode switch 9 (S2919: No), the process ends.

  FIG. 30 is a flowchart of the above-described measurement information generation process (S2909 in FIG. 29). The measurement information generation process (S2909) is a process of generating measurement information according to the instruction measurement instruction buttons B1 to B3 selected by the user.

  In this process, first, a high resolution setting signal is transmitted to the CCD 22 (S3001).

  Next, the release button 8 is scanned (S3002a), and it is determined whether or not the release button 8 is half-pressed (S3002b). If half-pressed (S3002b: Yes), the auto focus (AF) and automatic exposure (AE) functions are activated to adjust the focus, aperture, and shutter speed (S3002c). If not half-pressed (S3002b: No), the processing from S3002a is repeated.

  Next, the release button 8 is scanned again (S3002d), and it is determined whether or not the release button 8 has been fully pressed (S3002e). If it is fully pressed (S3002e: Yes), it is determined whether or not the flash mode is set (S3006).

  If the result of determination in S3006 is flash mode (S3006: Yes), the flash 7 is projected (S3007) and photographed (S3008). If it is determined in S3002e that the button has not been fully pressed (S3002e: No), the processing from S3002a is repeated.

  On the other hand, if the flash mode is not set (S3006: No), the flash 7 is shot without projecting (S3008). If it is determined in S3005 that the button has not been fully pressed (S3005: No), the processing from S3002 is repeated.

  Next, a three-dimensional shape detection process similar to the three-dimensional shape detection process of S1006 in FIG. 10 is performed to detect the three-dimensional shape of the subject (S3009).

  Next, as a three-dimensional shape detection result in the three-dimensional shape detection process (S3009), the three-dimensional coordinates (X, Y, Z) in the real space of each measurement vertex are stored in the three-dimensional information storage unit 37m (S3010).

  Next, it is determined what the measurement instruction buttons B1 to B3 whose inputs are confirmed by the process of S2907 in FIG. 29 are (S3011), and thereafter, the process according to the type of the measurement instruction buttons B1 to B3 is performed.

  If the input of the measurement instruction button B2 or B3 is confirmed as a result of the determination in S3011, (S3011: Two-point measurement, height measurement), a pointing instruction range measurement process (S3012) described later is executed. Thus, measurement according to the type of the measurement instruction button and projection of the measurement result are performed. After the execution of this pointing instruction range measurement process (S3012), the process ends.

  On the other hand, when the input of the measurement instruction button B1 is confirmed as a result of the determination in the process of S3011, (S3011: 3D input), the 3D shape stored in the 3D information storage unit 37m as a result of the process of S3010 The detection result is stored in the external memory 27 (S3013), and the three-dimensional shape detection result is displayed on the monitor LCD 10 (S3014). The three-dimensional shape detection result is displayed as an aggregate of three-dimensional coordinates (X, Y, Z) in the real space of each measurement vertex.

  Next, a 3D shape detection result image as a stereoscopic image (3D CG image) displaying the surface by connecting the measurement vertices as a 3D shape detection result with a polygon is stored in the projection image storage unit 37k (S3015). A projection process similar to the projection process of S806 in FIG. 8 is performed (S3016). In this case, as described above with reference to FIG. 18, the inverse function of the expression for converting the coordinates on the projection LCD 19 into the three-dimensional space coordinates is used, and the calculated three-dimensional coordinates on the projection LCD 19 are displayed. By obtaining the coordinates, the three-dimensional shape result coordinates can be projected onto the projection plane. Then, after the processing of S3016, the processing ends.

  FIG. 31 is a flowchart of the above-described finger pointing instruction range measurement process (S3012 in FIG. 30). In this finger point indication range measurement process (S3012), a point, a line segment, or a region formed by at least one point designated by a finger or a pointer on a stick is measured, and the measured image is taken as a subject or a projection direction. Is a process of projecting to

  In this process, first, a low-resolution moving image setting signal is transmitted to the CCD 22 (S3101). Therefore, since low-resolution moving image data is used, the amount of calculation is small and processing can be performed at high speed.

  Next, after the well-known autofocus (AF) and automatic exposure (AE) functions are activated and the focus, aperture, and shutter speed are adjusted (S3102), a predetermined process is performed in the same manner as in the process of S2906 (FIG. 29) described above. The inside of the imaging area 100 is imaged at intervals, and the generated image is inspected by taking the difference between each pixel of two consecutive captured image data, and finger pointing coordinates, which are the coordinates pointed by the fingertip, are acquired. (S3103).

  Next, it is determined whether or not the finger point coordinates acquired in S3103 point to the input end instruction button B4 (S3104). If the input end instruction button B4 is not pointed (S3104: No), the finger point coordinates are stored in the finger point coordinate storage unit 37n (S3105), and the process returns to S3104. On the other hand, if the input end instruction button B4 is pointed (S3104: Yes), the moving image shooting is stopped (S3106).

  By performing the above-described processing of S3013 to S3015, the locus pointed to by the pointing member such as the user's finger is imaged over time by shifting the time, whereby the locus pointed to by the pointing member is acquired.

  Next, measurement information corresponding to the type of the measurement instruction button and the finger point coordinates stored in the finger point coordinate storage unit 37n is generated (S3107). In the process of S3107, for example, in the case of “two-point measurement” corresponding to the measurement instruction button B2, the two-point finger coordinate stored in the finger coordinate storage unit 37n and the three-dimensional shape coordinate storage unit 37m The distance between the two points is measured based on the stored three-dimensional shape detection result.

  Next, the measurement result image based on the measurement information is stored in the projection image storage unit 37k (S3108), and the distortion-free image conversion processing similar to S2316 in FIG. 23 described above is executed (S3109). The projection process similar to the projection process of S806 of S8 is performed (S3110). As a result, the measurement result image stored in the projection image storage unit 37k is projected on the object or the projection direction as an undistorted image without depending on the distortion of the subject surface or the projection direction of the image projection unit 13. Then, after the process of S3110, the process ends.

  In the above-described processing of S3017, in the case of “height measurement” corresponding to the measurement instruction button B3, one point finger coordinate stored in the finger coordinate storage unit 37n and the three-dimensional shape coordinate storage unit Based on the three-dimensional shape detection result stored in 37m, the height of the surface of the imaging region 100 and the position pointed to (that is, one point finger coordinate stored in the finger point coordinate storage unit 37n) Is measured.

  In this embodiment, as the next range measurement process (S3012) in the designated point measurement mode, “two-point measurement” for measuring the distance between the two points to be pointed and the point pointed to the surface of the imaging region 100 Although only “height measurement” that measures the height of the finger is described, the area, volume, weight, etc. of the area surrounded by the finger trajectory pointed by the user's finger are obtained as measurement information It may be configured.

  As described above, according to the designated point measurement processing mode, the measurement based on the three-dimensional information is performed on points, line segments, and regions designated by the locus of the pointing member such as the user's finger, pen, or pointing rod. Information can be obtained, and various images based on the measurement information are projected on the work that is the subject and the vicinity of the work, so that the user can easily view the desired measurement information.

  In addition, since the point, line segment, and area for which measurement information is desired are specified by imaging the trajectory of the pointing member such as the user's finger, pen, or pointing rod over time, Measurement information to be obtained can be easily acquired, and the designated point can be easily input.

  The arm member 3 of the image input / output device 1 can be bent into an arbitrary shape as described above, and the imaging head 2 having the image projection unit 13 and the image imaging unit 14 can be directed to an arbitrary position. Therefore, in the above-described volume / weight measurement mode and specified point measurement mode, the imaging direction and the projection direction can be reduced so that the imaging direction and the projection direction are not disturbed by the work by reducing the direction to the work. As a result, work efficiency can be improved. For example, when the user's dominant arm is the right arm, working from the left front is convenient because the work is hardly disturbed.

  FIG. 32 is a view for explaining the light source lens 60 of the second embodiment related to the light source lens 18 of the first embodiment described above, and (a) is a side view showing the light source lens 60 of the second embodiment. (B) is a top view which shows the light source lens 60 of 2nd Example. In addition, the same code | symbol is attached | subjected to the same member as mentioned above, and the description is abbreviate | omitted.

  The light source lens 18 in the first embodiment is configured by integrally arranging the lens portions 18a on the base 18b from a convex aspherical shape corresponding to each LED 17, whereas in the second embodiment The light source lens 50 is configured by separately forming a resin lens formed in a bullet shape including each of the LEDs 17.

  In this way, by configuring the light source lenses 50 including the respective LEDs 17 separately, the positions of the respective LEDs 17 and the corresponding optical lenses 50 are determined on a one-to-one basis. The accuracy can be improved and the light emission direction is aligned.

  On the other hand, when the lens array is collectively aligned on the substrate 16, the light emission direction varies due to the positioning error when each LED 17 is die-bonded and the difference between the linear expansion coefficients of the lens array and the substrate. There is a risk of becoming.

  Therefore, the surface of the projection LCD 19 is irradiated with light whose light incident directions from the LEDs 17 are aligned perpendicularly to the surface of the projection LCD 19 so that it can pass through the aperture of the projection optical system 20 uniformly. Irradiance unevenness can be suppressed, and as a result, a high-quality image can be projected. The LED 17 included in the light source lens 50 is mounted on the substrate 16 via an electrode 51 composed of a lead and a reflector.

  Further, on the outer peripheral surface of the group of light source lenses 50 in the second embodiment, a fixing member 52 having a frame-like elasticity that binds the light source lenses 50 and restricts them in a predetermined direction is disposed. The fixing member 52 is made of a resin material such as rubber or plastic.

  Since the light source lens 50 of the second embodiment is configured separately from each LED 17, the angle of the optical axis formed by the convex tip of each light source lens 50 is correctly aligned and faces the projection LCD 19. It is difficult to install.

  Therefore, the group of light source lenses 50 is surrounded by the fixing member 52, the outer peripheral surfaces of the light source lenses 50 are brought into contact with each other, and the light source lenses 50 are arranged so that the optical axis of each light source lens 50 faces the projection LCD 19 at a correct angle. By restricting the position, the light can be irradiated from each light source lens 50 toward the projection LCD 19 substantially vertically. Therefore, the light aligned in the direction perpendicular to the surface of the projection LCD 19 is irradiated so that it can pass through the aperture of the projection lens uniformly, so that unevenness in the illuminance of the projection image can be suppressed. Therefore, a higher quality image can be projected.

  The fixing member 52 may have a rigidity defined in advance to a predetermined size. The fixing member 52 is made of a material having an elastic force, and the position of each light source lens 50 is set to a predetermined position by the elastic force. You may make it regulate to.

  FIG. 33 is a view for explaining a second embodiment relating to the fixing member 52 for restricting the light source lens 50 described in FIG. 32 to a predetermined position, and (a) is a perspective view showing a state in which the light source lens 50 is fixed. (B) is a partial sectional view thereof. In addition, the same code | symbol is attached | subjected to the same member as mentioned above, and the description is abbreviate | omitted.

  The fixing member 60 of the second embodiment is formed in a plate shape in which a through-hole 60a having a conical cross section having a cross section along the outer peripheral surface of each light source lens 50 is formed. Each light source lens 50 is inserted and fixed in each through hole 60a.

  Further, an urging plate 61 having elasticity is interposed between the fixing member 60 and the substrate 16, and an electrode 51 is provided between the urging plate 61 and the lower surface of each light source lens 50. An annular O-ring 62 having elasticity is arranged so as to surround the outer periphery.

  The LED 17 included in the light source lens 50 is mounted on the substrate 16 via an urging plate 61 and an electrode 51 that penetrates a through hole formed in the substrate 16.

  According to the fixing member 60 described above, each light source lens 50 is fixed by being passed through each through hole 60a having a cross section along the outer peripheral surface of the light source lens. The optical axis of the light source lens 50 can be fixed so as to face the projection LCD 19 at a correct angle.

  Further, at the time of assembly, the LED 17 can be urged and fixed to the correct position by the urging force of the O-ring 62.

  Further, an impact force that may occur when the apparatus 1 is transported can be absorbed by the elastic force of the O-ring 62, and the position of the light source lens 50 is shifted due to the impact, and the light source lens. The inconvenience that light cannot be irradiated from 50 to the projection LCD 19 vertically can be prevented.

  Note that the three-dimensional information generation unit according to claim 1 corresponds to the process of S3009 in the measurement information generation process of FIG. 30, and the measurement unit corresponds to the process of S3107 in the pointing instruction information generation process of FIG. To do.

  The designated point input means according to claim 2 corresponds to the processing of S3103 to S3105 in the pointing instruction information generation processing of FIG.

  Further, the density information acquisition means according to claim 5 corresponds to the processing of S2302 in the volume / weight measurement processing of FIG.

  Further, the three-dimensional information generation means according to claim 6 corresponds to the processing of S2312 in the volume / weight measurement processing of FIG. 23, and the instruction information generation means includes the processing of S2314 in the volume / weight measurement processing of FIG. Is applicable.

  Further, the spatial modulation control means described in claim 7 corresponds to the distortion-free image conversion processing of FIG.

  Further, the cutting means according to claim 13 corresponds to “a cutting means such as a cutter for cutting a work as a subject based on a measurement result image”.

  Although the present invention has been described based on the embodiments, the present invention is not limited to the above-described embodiments, and various modifications can be easily made without departing from the gist of the present invention. It can be done.

  For example, in the above-described embodiment, the process of acquiring and displaying a planarized image has been described as the planarized image mode. However, a known OCR function is installed, and the planarized planar image is read by this OCR function. You may comprise as follows. In such a case, it is possible to read the text written on the document with higher accuracy than when reading a curved document by the OCR function.

  Further, in S1501 of FIG. 15 in the above-described embodiment, a case has been described in which all luminance images having light and dark changes are extracted and provisional CCDY values are obtained for all of them. It is not necessary that the number of sheets is limited to one or more. The boundary coordinates can be obtained at high speed by reducing the number of extractions.

  Further, in S1507 of FIG. 15 in the above embodiment, fCCDY [i] is weighted and averaged, and in S1607 of FIG. 16 efCCDY [j] is an approximate polynomial, each value is averaged. The averaging method is not limited to these. For example, a method of taking a simple average value of each value, a method of employing a median value of each value, calculating an approximate expression of each value, and calculating the approximate expression For example, a method of using the detection position in the boundary coordinates as a boundary coordinate, a method of obtaining by statistical calculation, or the like may be used.

  Further, for example, in the three-dimensional shape detection process in the planar image mode in the above embodiment, in order to detect the three-dimensional shape of the document P, a striped pattern light in which a plurality of types of light and dark are alternately arranged is projected. Although the case has been described, the light for detecting the three-dimensional shape is not limited to such pattern light.

  For example, as shown in FIG. 34, when the three-dimensional shape of a curved document is simply detected, two strip-shaped slit lights 70 and 71 may be projected from the image projection unit 13. In this case, a three-dimensional shape can be detected at high speed from only two captured images, as compared with the case of projecting eight pattern lights.

  Further, in the volume / weight measurement process (FIG. 23) in the above embodiment, the measurement information obtained by the process of S2314 may be stored in the external memory 27. In addition, after the projection processing in S2317, an image projected on a work that is a subject may be captured.

  Further, the measurement information obtained by the process of S3107 in the finger pointing range measurement process (FIG. 31) in the above embodiment may be stored in the external memory 27. Thus, the measurement information obtained from the measurement information can be transmitted from the external memory 27 to the external interface by wireless communication and used in the external device.

1 is an external perspective view of an image input / output device. It is a figure which shows the internal structure of an imaging head. (A) is an enlarged view of an image projection part, (b) is a top view of a light source lens, (c) is a front view of projection LCD19. It is a figure for demonstrating regarding the arrangement | sequence of a LED array. It is an electrical block diagram of an image input / output device. It is a flowchart of a main process. It is a flowchart of a digital camera process. It is a flowchart of a webcam process. It is a flowchart of a projection process. It is a flowchart of a three-dimensional image process. (A) is a figure for demonstrating the principle of the space code method, (b) is a figure which shows the mask pattern (gray code) different from (a). (A) is a flowchart of a three-dimensional shape detection process. (B) is a flowchart of an imaging process. (C) is a flowchart of a three-dimensional measurement process. It is a figure for demonstrating the outline of a code boundary coordinate detection process. It is a flowchart of a code boundary coordinate detection process. It is a flowchart of the process which calculates | requires a code boundary coordinate with subpixel precision. It is a flowchart of the process which calculates | requires the boundary CCDY value about the brightness | luminance image with the mask pattern number of PatID [i]. It is a figure for demonstrating a lens aberration correction process. It is a figure for demonstrating the method of calculating the three-dimensional coordinate in a three-dimensional space from the coordinate in CCD space. It is a flowchart of planarization image processing. FIG. 6 is a diagram for explaining document orientation calculation processing. It is a flowchart of a plane conversion process. (A) is a figure for demonstrating the outline about a curvature calculation process, (b) is a figure which shows the planarization image planarized by the plane conversion process. It is a flowchart of a volume and weight measurement process. It is a flowchart of the image conversion process for distortion-free projection. It is a figure for demonstrating the 1st example in the volume and weight measurement process mode. It is a figure explaining the state by which the measurement result image was projected as a result of a volume and weight measurement process. It is a figure explaining the 2nd example in volume and weight measurement mode. It is a figure explaining designated point measurement mode. It is a flowchart of a designated point measurement process. It is a flowchart of a measurement information generation process. It is a flowchart of a finger pointing instruction range measurement process. (A) is a side view which shows the light source lens 60 of 2nd Example, (b) is a top view which shows the light source lens 60 of 2nd Example. (A) is a perspective view which shows the state which fixed the light source lens 50, (b) is the fragmentary sectional drawing. It is a figure which shows the other example as pattern light projected on a to-be-photographed object.

Explanation of symbols

3 Arm member (movable means)
13 Image projection unit (projection unit)
14 Image capturing unit (imaging unit)
17 LED (light source means)
19 Projection LCD (spatial modulation means)
20 Projection optical system (projection means)
21. Imaging optical system (part of imaging means)
22 CCD (part of imaging means)

Claims (16)

Light source means for emitting light;
Spatial modulation means for spatially modulating light emitted from the light source means to output image signal light; and
Projection means for projecting the image signal light output by the spatial modulation means in the projection direction;
Imaging means for imaging a subject having at least a part in the projection direction and acquiring the imaging data;
When image signal light having a predetermined pattern shape is projected on the subject or on a surface capable of projecting an image in the projection direction by the projection unit, based on imaging data captured by the imaging unit, 3D information generating means for generating 3D information of a subject or a surface capable of projecting an image in the projection direction;
Measuring means for measuring the subject based on the three-dimensional information generated by the three-dimensional information generating means and the designation information generated from the imaging data obtained by imaging the designation status for the subject by the imaging means; Prepared,
Image signal light corresponding to measurement information obtained by measurement by the measurement means is output by the spatial modulation means, and the output image signal light is projected onto the subject or the projection direction by the projection means. An image input / output device that projects onto a possible surface. Comprising designated point input means for inputting a designated point on the subject;
2. The measurement unit according to claim 1, wherein the object is measured with respect to a point, a line segment, or a region formed by at least one or more specified points input by the specified point input unit. The image input / output device described.   The designated point input unit is configured to capture an indication member indicating a designated point on the subject by shifting the time by the imaging unit, and to input the designated point based on the imaging data. The image input / output device according to claim 2. The measuring means measures volume information of the subject based on the three-dimensional information generated by the three-dimensional information generating means;
4. The image signal light projected onto the subject or a surface on which the image can be projected in the projection direction by the projection unit is instruction information according to the volume information. The image input / output device described. Density information acquisition means for acquiring density information of the subject;
The measuring means measures the weight distribution information of the subject based on the three-dimensional information generated by the three-dimensional information generating means and the density information acquired by the density information acquiring means,
5. The image signal light projected onto the subject or the image-projectable surface in the projection direction by the projection unit is instruction information according to the weight distribution information. The image input / output device described in 1. Light source means for emitting light;
Spatial modulation means for spatially modulating light emitted from the light source means to output image signal light; and
Projection means for projecting the image signal light output by the spatial modulation means in the projection direction;
Imaging means for imaging a subject having at least a part in the projection direction and acquiring the imaging data;
When image signal light having a predetermined pattern shape is projected onto the subject or a surface capable of projecting an image in the projection direction by the projection unit, the subject based on the image data captured by the imaging unit Or three-dimensional information generating means for generating three-dimensional information of a surface capable of projecting an image in the projection direction;
Instruction information generating means for generating instruction information for the subject based on the three-dimensional information generated by the three-dimensional information generating means,
Image signal light based on the instruction information generated by the instruction information generation means is output by the spatial modulation means, and the output image signal light can be projected on the subject or the projection direction by the projection means. An image input / output device that projects onto a surface.   The spatial modulation means is generated by the three-dimensional information generation means so that an undistorted image signal light is projected by the projection means on the subject or a surface capable of projecting an image in the projection direction. 7. The image input / output device according to claim 1, further comprising a spatial light modulation control unit that controls the three-dimensional information.   8. The image signal light projected onto the subject or the image projectable surface in the projection direction by the projection unit is character information related to the measurement information or the instruction information. An image input / output device according to claim 1.   9. The image signal light projected onto the subject or a surface capable of projecting an image in the projection direction by the projection unit is graphic information based on the measurement information or the instruction information. The image input / output device according to any one of the above.   10. The image signal light projected onto the subject or the image projectable surface in the projection direction by the projection means is a work instruction line based on the measurement information or the instruction information. The image input / output device according to any one of the above.   The image signal light projected onto the subject or a surface on which the image can be projected in the projection direction by the projection means is approximately equal to the subject according to the volume information or weight distribution information of the subject measured by the measurement means. Work instruction information for classifying into a volume or a substantially equal weight, or work instruction information for classifying the subject into a substantially equal volume or a substantially equal weight, which is generated as instruction information by the instruction information generating means. The image input / output device according to claim 1.   12. The image input / output apparatus according to claim 11, wherein the work instruction information is a cutting instruction line.   13. The image input / output device according to claim 12, further comprising cutting means for cutting the subject in accordance with the cutting instruction line.   The projection unit having the light source unit, the spatial modulation unit, and the projection unit, the imaging unit having the imaging unit, and the relative position of the subject are moved relative to each other. The image input / output device according to claim 1, further comprising movable means that enables the image input / output device.   The projection unit projects an image signal light based on the measurement information or the instruction information after the image signal having the predetermined pattern shape is turned off. Image input / output device.   The imaging means captures the subject on which the image signal light based on the measurement information or the instruction information is projected by the projection means or a surface on which the image can be projected in the projection direction, and acquires the imaging data. The image input / output device according to claim 1, wherein the image input / output device is provided.

Images