JP2009089171A - Network camera system, its control method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enhance operationality of a network camera by improving a response to a control instruction of an image transmitted from a remote camera. <P>SOLUTION: In a network camera system including a viewer 120 receiving the image from the network, and a camera 100 transmitting the image to the network, an operation detection part 122 for detecting the operation on the camera 100 determines the amount of the first movement, based on the detected operation. An image reception part 122 receives the amount of the second movement being an actual amount of movement of the camera 100 controlled by a pan-tilt control part 106 of the camera 100. A pan-tilt difference detection part 124 finds the difference between the amount of the first movement and the amount of the second movement. An image clipping part 125 processes the image received from the camera 100, based on the difference. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a network camera system or the like that controls a camera remote from a remote place through a network and browses or stores an image taken by the camera at a remote place.

  Conventionally, moving images shot by a camera installed at a remote location are transmitted over a network, and the moving image is received and displayed by a personal computer (PC). Also, zoom and pan / tilt operations of a camera installed at a remote location are performed. Techniques for performing are generally known. Such a technique is used and commercialized in a video conference system, a surveillance camera, and the like.

  In order to control a camera installed at a remote location through a network, a control command is transmitted to the camera via the network, so that the control command is delayed until it is reflected in the transferred image (turnaround). Occurs. As a technique for avoiding this delay in a pseudo manner, for example, in Patent Document 1, electronic zoom is performed during the delay time, and when an image in which a control command is reflected arrives, an image obtained by electronic zoom is sent from the camera. A technique for replacing is disclosed.

  Patent Document 2 discloses a technique for transmitting a moving image captured by a wide-angle camera as a panoramic image to a client via a network separately from the stream 1 in addition to the main video (stream 1) actually being remotely shot. Has been.

Japanese Patent Laid-Open No. 11-205653 JP 2004-187068 A

  However, when control such as pan / tilt or zoom of the camera is performed from a client such as a PC placed at a remote place away from the camera as described above, the control command (command ) Is sent. Then, the camera interprets the command and performs mechanical control. Therefore, it takes a long time (turnaround time) until the command is reflected in the camera, and the operability of the camera from a remote place is not good.

  Accordingly, an object of the present invention is to provide a network camera system with improved operability by improving the response to an image control command sent from a remote camera while having the turnaround time. And

The network camera system of the present invention is a network camera system including a display device capable of receiving an image from a network and an imaging device capable of transmitting an image to the network, wherein the display device detects an operation on the imaging device. And an operation instruction to the imaging apparatus based on the operation detected by the operation detection means, the movement amount determination means for determining the first movement amount based on the operation detected by the operation detection means, and the operation detected by the operation detection means. An instruction unit; and an image processing unit that processes an image received from the imaging device. The imaging device includes: an imaging unit that captures an image wider than a display image displayed by the display device; and the display device. Receiving means for receiving an operation instruction; operation control means for operating the imaging apparatus based on the operation instruction received by the receiving means; and the operation control Notification means for notifying the display device of the movement amount of the imaging device operated by the stage as a second movement amount, and the image processing means includes the first movement amount and the second movement amount. And processing an image received from the imaging apparatus.
The display device of the present invention is a display device that can receive an image from an imaging device capable of transmitting an image to the network through the network, and an operation detection unit that detects an operation on the imaging device, and the operation detection unit includes: Based on the detected operation, a movement amount determining means for determining a first movement amount, an instruction means for giving an operation instruction to the imaging apparatus based on the operation detected by the operation detecting means, and an image received from the imaging apparatus Image processing means for processing the image processing means, and receiving means for receiving, as a second movement amount, a movement amount of the imaging device operated based on an operation instruction received by the imaging device from the instruction means, and the image processing means Is configured to process an image received from the imaging device based on the first movement amount and the second movement amount.
The network camera system control method of the present invention is a network camera system control method including a display device capable of receiving an image from a network and an imaging device capable of transmitting an image to the network. , Based on the operation detection step for detecting the operation on the imaging device, the movement amount determination step for determining the first movement amount based on the operation detected in the operation detection step, and the operation detected in the operation detection step, An instruction step for giving an operation instruction to the imaging device and an image processing step for processing an image received from the imaging device, wherein the imaging device captures a wider image than the display image displayed by the display device. A photographing step, a receiving step for receiving an operation instruction from the display device, and a receiving step in the receiving step. An operation control step of operating the imaging device based on the operated instruction, and a notification step of notifying the display device of a movement amount of the imaging device operated in the operation control step as a second movement amount. In the image processing step, an image received from the imaging device is processed based on the first movement amount and the second movement amount.
The program of the present invention is a program for controlling a display device capable of receiving an image through a network from an imaging device capable of transmitting an image to the network, and an operation detection step of detecting an operation on the imaging device; A movement amount determining step for determining a first movement amount based on the operation detected in the operation detection step; an instruction step for instructing the imaging apparatus to perform an operation based on the operation detected in the operation detection step; Causing the computer to execute an image processing step of processing an image received from the device, and a receiving step of receiving, as a second movement amount, the movement amount of the imaging device operated based on the operation instruction received by the imaging device; In the image processing step, an image received from the imaging device based on the first movement amount and the second movement amount. Is a program to process.

  According to the present invention, a pan / tilt operation, for example, on a remote imaging device performed on a display device is immediately reflected in an image displayed on the display device, so that a comfortable operational feeling can be realized. The operability of the network camera system can be improved.

(First embodiment)
Hereinafter, a first embodiment of a network camera system according to the present invention will be described with reference to FIGS.

  FIG. 1 is a diagram showing a configuration of a network camera system according to the present embodiment. The network camera system includes a camera 100 that is an imaging device and a viewer 120 that is a display device, which are connected via a network. By connecting to the network, the camera 100 can transmit an image to the network, and the viewer 120 can receive an image from the camera 100 through the network.

  The camera 100 includes a lens 101, a sensor 102, an image processing unit 103, an image transmission unit 104, a control command reception unit 105, a pan / tilt control unit 106, and a motor 107 (a pan head motor).

  In the camera 100 as described above, the sensor 102 is a sensor such as a CCD or a CMOS, and converts the light passing through the lens 101 into an electrical signal. The image processing unit 103 develops the electrical signal from the sensor 102 and converts it to an appropriate resolution or performs other image processing. The image transmission unit 104 transmits the image processed by the image processing unit 103 to the network. The control command receiving unit 105 receives control commands such as pan / tilt / zoom through the network. The pan / tilt control unit 106 controls the motor 107 built in the camera platform on which the camera 100 is mounted, based on the control command received by the control command receiving unit 105. The lens 101 and the sensor 102 correspond to the photographing means in the present invention, the control command receiving unit 105 corresponds to the receiving means in the present invention, and the pan / tilt control unit 106 corresponds to the operation control means in the present invention. To do.

  Next, the viewer 120 includes an image reception unit 121, an operation detection unit 122, a camera pan / tilt position reception unit 123, a pan / tilt difference detection unit 124, an image cutout unit 125, a pan / tilt instruction unit 126, an image recording unit 127, And an image display unit 128.

  In the viewer 120 as described above, the image receiving unit 121 receives an image from the outside, that is, the camera 100. The operation detection unit 122 detects an operation performed by a user using a digital user interface, a keyboard, or the like, such as pan / tilt / zoom. The camera pan / tilt position receiving unit 123 receives information such as the direction (movement amount) of the camera 100 from the pan / tilt control unit 106 of the camera 100. The pan / tilt difference detection unit 124 detects the difference between the pan state of the image currently being displayed and recorded and the actual pan state of the camera 100. The image cutout unit 125 cuts out an image for display and recording from the image sent from the camera 100 using the difference detected by the pan / tilt difference detection unit 124. The pan / tilt instruction unit 126 instructs pan / tilt operation of the camera 100. The image recording unit 127 records an image, and the image display unit 128 displays the image. The operation detection unit 122 corresponds to the operation detection unit and the movement amount determination unit in the present invention, the pan / tilt instruction unit 126 corresponds to the instruction unit in the present invention, and the image cutout unit 125 detects the pan / tilt difference detection. In combination with the unit 124, it corresponds to the image processing means in the present invention.

  In the present embodiment, the camera 100 transmits an image larger than the image size displayed and recorded by the viewer 120, and the viewer 120 extracts and displays and records a part of the image received from the camera 100. When the pan / tilt operation is not performed, the viewer 120 cuts out a predetermined area located at the center of the image transmitted from the camera 100 as a display image.

  In this embodiment, the pan / tilt amount of the camera 100 and the pan / tilt amount by the operation of the viewer 120 are calculated in units of pixels to be displayed. The processing is light.

  In the present embodiment, the operation detection unit 122 includes a digitizer that detects coordinates when a hand or pen touches the surface of the image display unit 128. Thereby, the operation detection unit 122 detects the transition of coordinates by the digitizer when the user interface displayed on the image display unit 128 is operated with a hand or a pen, and detects the operation state.

  FIG. 2 is a diagram illustrating an example of a screen displayed on the viewer 120. 2 includes an image display area 201, a pan operation scroll bar 202, a tilt operation scroll bar 203, and a zoom operation scroll bar 204.

  In the image display area 201, a display image cut out from the image from the camera 100 is displayed. The pan operation scroll bar 202 is used for pan operation, and includes a scroll thumb 202a and scroll arrows 202b and 202c. The tilt operation scroll bar 203 is used when a tilt operation is performed, and includes a scroll thumb 203a and scroll arrows 203b and 203c. The zoom operation scroll bar 204 is used when performing a zoom operation.

  When the scroll thumbs 202a and 203a are operated (dragged) using, for example, a pen, the direction of the camera 100 moves in the dragged direction. Further, when any one of the scroll arrows 202b and 202c and the scroll arrows 203b and 203c is operated (pen down), the direction of the camera 100 moves at a constant speed in any one of the left, right, and upper directions. In this case, the screen from the camera 100 appears to scroll.

  Next, an image displayed in the image display area 201 when the pan operation of the camera 100 is performed by the viewer 120 will be described with reference to FIG.

  FIG. 3A shows the shooting area of the camera 100 and the display recording area of the viewer 120 when panning is performed in the positive direction of the X axis (rightward on the paper) as time progresses (downward on the paper). FIG.

  When the pan operation is not performed, the viewer 120 cuts out the area indicated by the display recording image 301b from the center of the image area from the camera 100 indicated by the camera image 301a as referred to by the frame 1 (301). , Display and record.

  Here, in the viewer 120, when the pan operation is continuously performed in the X direction shown in FIG. 3A by continuously pressing the scroll arrow 202c, the viewer 120 instructs the camera 100 to perform a pan operation command (control command). Is sent over the network. The image sent from the camera 100 at this time is an image that is referred to a frame 2 (302) that is equal to the frame 1 (301) and does not take a pan operation into consideration. Here, the viewer 120 cuts out an area shifted in the X direction by the number of pixels corresponding to the movement amount (pan operation amount) by the pan operation as the display recording image 302b from the camera image 302a referred to by the frame 2 (302). Display and record.

  FIG. 3B is a diagram showing details of the frame 2 (302). In FIG. 3B, the number of pixels corresponding to the pan operation amount is “d” referred to in the figure. In frame 2 (302), the display record image 302b is cut out from a position shifted by “d” from the cut out position immediately after the start of the pan operation.

  As described above, after the pan operation is started, a pan operation command is transmitted to the camera 100 and until the pan of the camera 100 is started, the position is similarly shifted by the pan operation amount from the position where the pan operation is started. The image is cut out from the image and displayed and recorded repeatedly. Frame 3 (303) shows a state where the pan operation is further performed from the state of frame 2 (302).

  Next, an image cut out by the viewer 120 when the camera 100 itself receives a pan operation command and starts a pan operation will be described. When the shooting area of the camera 100 is moved by “p4” pixels in the X direction as in the frame 4 (304), and the pan operation amount at this time is moved by “d4” with respect to the frame 1 (301), The viewer 120 sets “d4 ′” referred to in FIG. This cut-out position “d4 ′” is a position shifted by “d4-p4” from the center of the frame 4 (304).

  As described above, when the pan operation of the camera 100 itself starts, the viewer 120 sets a position obtained by subtracting the movement amount of the camera 100 from the pan operation amount as a cutout position. In the above description, the movement amount by the pan operation in the viewer 120 corresponds to the first movement amount referred to in the present invention, and the actual movement amount of the camera 100 corresponds to the second movement amount referred to in the present invention.

  Next, a case where the shooting area of the camera 100 is moved to a position where it should be displayed by pan operation will be described. In this case, for example, as referred to by the frame 6 (306), since the shooting display area of the camera 100 matches the position corresponding to the area to be originally displayed due to the pan of the camera 100 itself, the viewer 120 is cut out. Stop adjusting. While the pan operation of the camera 100 follows the pan operation, the display record image 306b is cut out from the center of the image transmitted from the camera 100, and displayed and recorded as it is.

  Next, FIG. 4 is a flowchart for explaining the flow of processing in the network camera system according to the present embodiment, and specifically relates to the contents described with reference to FIG. 4A is a flowchart for explaining the processing of the viewer 120, and FIG. 4B is a flowchart for explaining the processing of the camera 100.

  First, in step S401 of the processing on the viewer 120 side, the viewer 120 determines whether an operation (operation update) such as pan / tilt has been performed. If the operation has been performed, in step S402, the viewer 120 determines an ideal shooting direction of the camera. In step S <b> 403, the viewer 120 compares the ideal camera shooting direction determined in step S <b> 402 with the current camera 100 shooting direction to determine the ideal camera shooting direction and the current shooting direction. Extract the difference. This difference corresponds to the movement amount “d” or “d4 ′” by the pan operation in FIG.

  Next, in step S <b> 404, the viewer 120 transmits the difference obtained in step S <b> 403, that is, “d” or “d4 ′” to the camera 100. Next, if it is determined in step S405 that there is a difference extracted in step S403, the position is shifted from the center of the image received from the camera 100 in step S406, or shifted by “d” or “d4 ′” in the example of FIG. The image is cut out from the electronic pan / tilt. In step S407, the viewer 120 displays and accumulates (records) the clipped image.

  In step S403, it is desirable that the shooting direction of the camera 100 can be received in synchronization with the frequency of arrival of images from the camera 100. Therefore, for example, the configuration may be such that the shooting direction of the camera 100 is sent in the header information of the image of the camera 100. Here, the process of transmitting the shooting direction of the camera 100 corresponds to one process example of the notification means in the present invention.

  On the other hand, in the processing on the camera 100 side, first, in step S411, the camera 100 determines whether or not an operation command has been received from the viewer 120 (operation update). If received, in step S412, the camera 100 controls movement of the camera direction, and transmits the shooting direction of the camera 100 to the viewer 120 in step S413. Further, simultaneously with moving the direction of the camera 100 (step S412), an image is periodically generated (step S414: frame capture) and transmitted to the viewer 120.

  Next, an example of a temporal relationship between the user's operation and the operations of the viewer 120 and the camera 100 will be described with reference to FIG. In this figure, the time axis is shown on the right side of the page, and the user's operation, the operation of the viewer 120, and the camera 100 are shown in the down direction on the page, corresponding to the time axis.

  From time t 0, the camera 100 captures an image, generates an image stream, and transmits the image stream to the viewer 120. The viewer 120 displays an image cut out from the center of the received image from time t0 to t1 described later.

  From time t1 to t6, the user performs a pan operation by pressing the scroll arrow 202c. In this case, the viewer 120 notifies the camera 100 of a pan operation command at time t1, and the pan operation command reaches the camera 100 at time t2. Until the image reflecting the panning operation reaches the viewer 120 at the time t3, the viewer 120 electronically pans the pixel corresponding to the screen movement by the panning operation. This electronic pan corresponds to step S406 in FIG.

  When the pan operation starts to be reflected in the image sent from the camera 100 at time t3, the viewer 120 generates an electronic pan image by taking into account (considering) the amount of movement of the camera 100, and displays and records it. During this period, the camera 120 periodically notifies the camera 100 of the amount of movement by the ideal pan, and the camera 100 also transmits the amount of movement by the actual pan to the viewer 120 in synchronization with the image data.

  When the pan operation of the camera 100 follows the ideal pan movement amount of the viewer 120 at time t5, the viewer 120 cuts out the image display record image from the center of the image sent again and displays and records it. Do.

  When the user releases the pen from the scroll arrow 202c with the viewer 120 to stop the pan operation at time t6, the pan movement notification from the viewer 120 stops, and the pan operation with the camera 100 also stops at time t7. Thereafter, in the viewer 120, a display record image is cut out from the center portion of the image sent from the camera 100 and displayed and recorded.

  The pan operation (lateral camera movement) has been described above, but the same processing is performed for the tilt operation (vertical camera movement).

  Note that panning and tilting operations increase the distortion at the periphery of the image due to increased lens aberrations.However, when the image is cut out, distortion correction is performed to generate a pan / tilt screen that does not feel strange. May be.

  As described above, the problem that occurred when pan / tilt operation was performed with a conventional camera, that is, the turnaround time required for the operation to be reflected in the image being viewed took a poor response to the operation. The problem is solved. That is, according to the present invention, the control command can be reflected on the display image with very good response, and the camera position can be adjusted by remote control without stress, thereby improving the operability.

(Second embodiment)
Hereinafter, a second embodiment of the network camera system according to the present invention will be described with reference to FIGS. In the present embodiment, an image (main stream) to be displayed on the viewer when pan / tilt operation is not performed and an image (substream) obtained by photographing a region larger than the main stream are transmitted to the pan / tilt operation. A method for performing a response at high speed will be described.

  FIG. 6 is a diagram showing a configuration of the network camera system according to the present embodiment. The network camera system according to the present embodiment includes a camera 100 and a viewer 620, which are connected via a network. The configuration of the network camera system according to the present embodiment is basically the same as that of the network camera system according to the first embodiment, but the viewer 620 performs image synthesis instead of the image clipping unit 125 shown in FIG. There is a difference in having a portion 625. The image composition unit 625 performs processing for extracting and compositing images from the main stream and the substream described above. The image composition unit 625, in combination with the pan / tilt difference detection unit 124, corresponds to the image processing means in the present invention.

  In this embodiment, an image displayed in the image display area 201 (similar to the first embodiment) when the pan operation of the camera 100 is performed by the viewer 620 will be described with reference to FIG.

  FIG. 7A shows a shooting area of the camera 100 and a display recording area in the viewer 620 when panning is performed in the positive direction of the X axis (rightward on the paper) as time progresses (downward on the paper). FIG.

  When the pan operation is not performed, the viewer 620 transmits, from the camera 100, two moving image streams, an area represented by the main stream 701b and an area represented by the substream 701a, as referred to by the frame 1 (701). Is done. In this case, the viewer 620 normally displays and records the main stream 701b having the same size as the area to be displayed. The substream 701a captures a wider area (wide range) than the main stream 701b, but the resolution is lower than that of the main stream 701b. Ideally, the main stream 701b is located in a predetermined area at the center of the shooting area of the substream 701a.

  Here, when the pan operation is continuously performed in the X direction in FIG. 7A in the viewer 620, the viewer 620 transmits a pan operation command to the camera 100 through the network. The main stream at this time is an area indicated by 702b that is referred to by a frame 2 (702) that is equal to the frame 1 (701) and does not take a pan operation into consideration. Here, the viewer 620 generates a display recording image 702c corresponding to the pan operation from the main stream 702b and the substream 702a referred to by the frame 2 (702).

  FIG. 7B is a diagram showing details of the frame 2 (702). In FIG. 7B, an area 1 (722) and an area 2 (721) are image areas sent by the main stream 702b. Here, the viewer 620 generates an image shifted in the X direction by an amount corresponding to the region 3 (723) in consideration of the pan operation. That is, the image of “d1” corresponding to the region 1 (722) is cut out from the image of the main stream 702b and the region 2 (721) is excluded, and “d2” corresponding to the region 3 is extracted from the substream 702a. Cut out the image. Since the images of region 1 (722) and region 3 (723) cut out here have different resolutions, the number of pixels also differs. Therefore, the viewer 620 interpolates the region 3 (723) so that the region 3 (723) is matched with the resolution of the region 1 (722), that is, the region 3 (723) is set to a high resolution. Then, the resolutions of the region 1 (722) and the region 3 (723) are combined, and both are combined to be displayed and recorded as one display recording image 702c.

  As described above, when a pan operation is performed, a predetermined area is cut out from the main stream, a predetermined area is cut out from the substream, and the two are combined, and the pan / tilt position from the camera 100 is changed. Repeat until a notification is received. Frame 3 (703) shows a state where the pan operation is further performed from the state of frame 2 (702).

  Next, an image synthesized by the viewer 620 when the camera 100 itself receives a pan operation command and starts a pan operation will be described. Here, like the frame 4 (704), the shooting areas of the main stream 704a and the substream 704b move by “p4” pixels in the X direction, and the pan operation amount is only “d4” with respect to the frame 1 (701). A display recording image 704c to be combined when moved is taken as an example.

  In this case, the area cut out from the main stream 704a and the substream 704b takes into account the movement amount “p4” pixels of the shooting area of the camera 100 and the pan operation amount “d4”. 7C, the width cut out from the main stream 704a is obtained by subtracting “d4-p4” from the predetermined width in the X direction of the main stream 704b, that is, “d4 ′” in the figure. . That is, the region 1 ′ (742) referred to in the drawing is cut out. The width cut out from the substream 704a is “d4-p4”, that is, “d4 ′” in the figure. That is, the region 3 ′ (743) referred to in the drawing is cut out.

  As described above, when the pan operation of the camera 100 itself starts, the viewer 620 extracts and combines an area in which the pan operation amount and the movement amount of the camera 100 are added.

  Next, a case where the shooting area of the camera 100 is moved to a position where it should be displayed by a pan operation will be described. In this case, for example, as referred to by the frame 6 (706), the display area of the main stream 706b matches the area to be originally displayed due to the pan of the camera 100 itself, and thus the image synthesis is stopped. Then, the display is switched to the display of only the main stream, and the main stream image is displayed as it is while the pan operation of the camera follows the pan operation.

  FIG. 8 is a flowchart for explaining the flow of processing in the network camera system according to the present embodiment, and specifically relates to the contents described with reference to FIG. 8A is a flowchart for explaining the processing of the viewer 620, and FIG. 8B is a flowchart for explaining the processing of the camera 100.

  In FIG. 8A, step S406 in FIG. 4 is replaced with step S806. In step S806, using the difference between the actual shooting direction of the camera 100 and the ideal shooting direction of the camera 100 in step S403, the area 1, area 3, or area 1 ′ in FIG. The region 3 ′ is extracted and synthesized. The other processes are the same as those described with reference to FIG.

  Next, an example of a temporal relationship between the user operation, the viewer 620, and the operation of the camera 100 will be described with reference to FIG. In this figure, the time axis is shown on the right side of the drawing, and the user's operation, the operation of the viewer 620, and the camera 100 are shown in the down direction on the drawing so as to correspond to the time axis.

  From time t0, the camera 100 generates a main stream and a substream and transmits them to the viewer 620. In addition, the viewer 620 displays the main stream from time t0 to later-described t1.

  From time t1 to t4, the user performs a pan operation by pressing the scroll bar 202c. In this case, the viewer 620 notifies the camera 100 of a pan operation command at time t1, and the pan operation command reaches the camera 100 at time t2. Then, until the image reflecting the panning operation at time t3 reaches the viewer 620, the viewer 620 combines the images as if performing the panning operation from the main stream and the substream. This image synthesis corresponds to step S806 in FIG.

  When the pan operation starts to be reflected in the image sent from the camera 100 at time t3, the viewer 620 generates a composite image in consideration of the movement amount of the camera 100, and displays and records it. During this time, the viewer 620 periodically notifies the camera 100 of the pan movement amount.

  When the user releases the pen from the scroll arrow 202c with the viewer 620 to stop the pan operation at time t4, the pan movement notification from the viewer 620 stops, and the pan operation with the camera 100 also stops at time t5. Thereafter, the viewer 620 only displays and records the main stream.

  The pan operation (lateral camera movement) has been described above, but the same processing is performed for the tilt operation (vertical camera movement).

  Note that panning and tilting operations increase the distortion at the periphery of the image due to increased lens aberrations.However, when combining images, distortion correction is performed to generate a pan / tilt screen that does not feel uncomfortable. May be.

  As described above, the problem that occurred when pan / tilt operation of a conventional camera was performed, that is, the turnaround time until the operation was reflected in the image being viewed took, and the response to the operation was poor. The problem is solved. That is, according to the present invention, the control command can be reflected on the display screen with very good response, the camera position can be remotely controlled without stress, and the operability is improved. In the above description of the present embodiment, the main stream corresponds to the first image referred to in the present invention, and the substream corresponds to the second image referred to in the present invention.

(Third embodiment)
Hereinafter, a third embodiment of the network camera system according to the present invention will be described with reference to FIG.

  FIG. 10 is a diagram showing a configuration of the network camera system according to the present embodiment. The network camera system according to the present embodiment includes a camera 100 and a viewer 1020, which are connected via a network. The configuration of the network camera system according to the present embodiment is the same as that of the second embodiment in that the viewer 1020 includes a turnaround time detection unit 1029, an image size determination unit 1030, and an image size instruction unit 1031. There is a difference. In the third embodiment, with such a configuration, in the camera 100, feedback is applied to the image processing unit 103 by the control command receiving unit 105. Details will be described below.

  The turnaround time detection unit 1029 indicates a response time (control turnaround) until the time when the control command transmitted from the viewer 1020 to the control command receiving unit 105 via the network returns to the viewer 1020 via the camera 100 again. Measure regularly.

  The image size determination unit 1030 determines the resolution and image size of the substream described in the second embodiment using the control turnaround.

  For example, when the control turnaround is short, it is determined that the network is fast and the bandwidth is sufficient, and the sub-stream shooting range (zoom) is smaller than the main stream and smaller, and the resolution is high. Decide to do. On the other hand, when the control turnaround is long, it is determined that the network is low speed and there is no bandwidth, and the sub-stream shooting range is widened and the resolution is lowered.

  The image size instruction unit 1031 transmits the resolution and image size of the substream determined by the image size determination unit 1030 to the control command reception unit 105 of the camera 100 via the network and reflects them in the image processing unit 103.

  As described above, by periodically monitoring the network load status, it is possible to increase / decrease the bandwidth occupied by the substream network and realize communication that ensures the image quality of the main stream.

  In addition, by changing the shooting range of the substream according to the turnaround time, if the control turnaround is long, the difference amount so as to cover the slow response of the camera to the user's pan / tilt operation (S803) It becomes possible to take large.

  As mentioned above, although this invention was demonstrated by the above-mentioned embodiment, based on the technical idea of this invention, a various change is possible. For example, in the above-described embodiment, the pan / tilt operation of the camera has been described. However, an operation in an oblique direction combining pan / tilt can be handled. It can also be applied to zoom operations.

  In order to realize the present invention, a storage medium in which a program code (computer program) of software that realizes the functions of the above-described embodiments may be used. In this case, the object of the present invention is achieved by supplying the storage medium to the system or apparatus, and the computer (or CPU or MPU) of the system or apparatus reads and executes the program code stored in the storage medium.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention.

  As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  Needless to say, the OS (basic system or operating system) running on the computer performs part or all of the actual processing based on the instruction of the program code.

  Furthermore, the program code read from the storage medium may be written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. In this case, based on the instruction of the written program code, the CPU or the like provided in the function expansion board or function expansion unit may perform part or all of the actual processing.

It is a figure which shows the structure of the network camera system which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the screen displayed on the viewer which concerns on embodiment of this invention. FIG. 6 is a diagram illustrating an image displayed when a pan operation of a camera is performed with a viewer in the first embodiment of the present invention. It is a flowchart explaining the flow of a process in the network camera system which concerns on the 1st Embodiment of this invention. It is a figure explaining the temporal relationship of a user's operation and operation of a viewer and a camera in the network camera system concerning a 1st embodiment of the present invention. It is a figure which shows the structure of the network camera system which concerns on the 2nd Embodiment of this invention. FIG. 10 is a diagram illustrating an image displayed when a pan operation of a camera is performed with a viewer in the second embodiment of the present invention. It is a flowchart explaining the flow of a process in the network camera system which concerns on the 2nd Embodiment of this invention. It is a figure explaining the temporal relationship of a user's operation and the operation | movement of a viewer and a camera in the network camera system which concerns on the 2nd Embodiment of this invention. It is a figure which shows the structure of the network camera system which concerns on the 3rd Embodiment of this invention.

Explanation of symbols

100 Camera 120, 620, 1020 Viewer 101 Lens 102 Sensor 103 Image processor 104 Image transmitter 105 Control command receiver 106 Pan / tilt controller 107 Motor 120 Viewer 121 Image receiver 122 Operation detector 123 Camera pan / tilt position reception Unit 124 pan / tilt difference detection unit 125 image cutout unit 126 pan / tilt instruction unit 127 image recording unit 128 image display unit 625 image composition unit 1029 turnaround time detection unit 1030 image size determination unit 1031 image size instruction unit

Claims (12)

A network camera system including a display device capable of receiving an image from a network and an imaging device capable of transmitting an image to the network,
The display device
Operation detecting means for detecting an operation on the imaging device;
A movement amount determining means for determining a first movement amount based on the operation detected by the operation detecting means;
Based on the operation detected by the operation detection means, an instruction means for giving an operation instruction to the imaging device;
Image processing means for processing an image received from the imaging device;
The imaging device
Photographing means for photographing an image wider than the display image displayed by the display device;
Receiving means for receiving an operation instruction from the display device;
Operation control means for operating the imaging apparatus based on the operation instruction received by the receiving means;
Notification means for notifying the display device of a movement amount of the imaging device operated by the operation control means as a second movement amount;
The network camera system, wherein the image processing unit processes an image received from the imaging device based on the first movement amount and the second movement amount.   The image processing means cuts out the display image from an image received from the imaging device based on a difference between the first movement amount and the second movement amount. Network camera system.   3. The network camera system according to claim 2, wherein the image processing unit cuts out the display image based on the first movement amount until the second movement amount is received. 4. The imaging device captures a first image in the same range as a display image displayed by the display device and a second image obtained by capturing a wider range than the display image,
The image processing means generates the display image from the first image and the second image, based on a difference between the first movement amount and the second movement amount. The network camera system according to 1.   The image processing means excludes an area corresponding to the difference from the first image, cuts out an area corresponding to the difference from the second image, and generates the display image. Item 5. The network camera system according to Item 4.   The image processing means excludes an area corresponding to the first movement amount from the first image until the second movement amount is received, and then removes the first image from the second image. 6. The network camera system according to claim 5, wherein an area corresponding to the movement amount is cut out.   The network camera system according to claim 4, wherein the first image is an image having a higher resolution than the second image.   The network camera system according to claim 7, wherein the image processing unit includes a unit that adjusts a resolution between the first image and the second image.   The network camera system according to claim 1, wherein the operation detection unit detects a pan / tilt operation on the imaging apparatus. A display device capable of receiving an image from an imaging device capable of transmitting an image to the network through the network,
Operation detecting means for detecting an operation on the imaging device;
A movement amount determining means for determining a first movement amount based on the operation detected by the operation detecting means;
Based on the operation detected by the operation detection means, an instruction means for giving an operation instruction to the imaging device;
Image processing means for processing an image received from the imaging device;
Receiving means for receiving, as a second movement amount, the amount of movement of the imaging device operated based on the operation instruction received from the instruction means;
The display device, wherein the image processing means processes an image received from the imaging device based on the first movement amount and the second movement amount. A control method for a network camera system, comprising: a display device capable of receiving an image from a network; and an imaging device capable of transmitting an image to the network,
In the display device,
An operation detection step of detecting an operation on the imaging device;
A movement amount determination step for determining a first movement amount based on the operation detected in the operation detection step;
An instruction step for giving an operation instruction to the imaging device based on the operation detected in the operation detection step;
An image processing step for processing an image received from the imaging device;
In the imaging apparatus,
A photographing step of photographing a wider range of images than the display image displayed by the display device;
A receiving step of receiving an operation instruction from the display device;
An operation control step of operating the imaging device based on the operation instruction received in the reception step;
A notification step of notifying the display device of a movement amount of the imaging device operated in the operation control step as a second movement amount;
In the image processing step, an image received from the imaging device is processed based on the first movement amount and the second movement amount. A method for controlling a network camera system, comprising: A program for controlling a display device capable of receiving an image via a network from an imaging device capable of transmitting the image to the network,
An operation detection step of detecting an operation on the imaging device;
A movement amount determination step for determining a first movement amount based on the operation detected in the operation detection step;
An instruction step for giving an operation instruction to the imaging device based on the operation detected in the operation detection step;
An image processing step for processing an image received from the imaging device;
Causing the computer to execute a reception step of receiving, as a second movement amount, the movement amount of the imaging device operated based on the operation instruction received by the imaging device;
In the image processing step, a program for processing an image received from the imaging device based on the first movement amount and the second movement amount.

Images