JP6098963B1 - MONITORING SYSTEM, VIDEO DISPLAY METHOD, AND COMPUTER PROGRAM - Google Patents

Abstract

To divide and display a plurality of different types of captured images on a monitor and to easily set the execution or non-execution of mask processing for each captured image for each captured image to improve user convenience. . An information processing apparatus includes setting information on execution or non-execution of mask processing for changing at least a part of a moving body appearing in a captured image of an imaging area captured by an individual camera into a mask image for each camera. Hold on. The information processing apparatus captures an individual edited video obtained by performing mask processing on individual captured images captured by a plurality of cameras set to perform mask processing or a single camera configured to perform mask processing. The edited video obtained by masking the individual captured video and the individual captured video captured by at least one camera set not to execute the mask processing are divided and displayed on the monitor. [Selection] Figure 15

Description

The present invention relates to an edited video after mask processing for changing an image area of a moving body (for example, a person) detected in a captured image of a monitoring area to a mask image for making it impossible to specify who the moving body is. The present invention relates to a monitoring system , a video display method, and a computer program .

  In stores such as convenience stores, cameras that shoot inside the store are installed, and there are a wide variety of monitoring systems that monitor people in the store using video (hereinafter referred to as “captured video”) obtained by imaging with the camera. It is popular. In the operation of such a monitoring system, when using a photographed image for purposes other than monitoring (for example, the purpose of grasping the flow of people in a store and promoting marketing analysis and service improvement), the person shown by the camera ( For example, customers' privacy needs to be protected.

  Here, for example, a monitoring system disclosed in Patent Document 1 is known as a prior art for grasping a tendency of a person's action for each attribute of a person in a store. The monitoring system of Patent Literature 1 includes a monitoring device that generates and outputs a mask video in which a person's image area detected from a moving image captured in a monitoring area is changed to a mask image. The monitoring device acquires area information for each person detected from the captured image, determines an attribute for each person detected from the captured video, and sets a mask image for each attribute. The monitoring device generates a contour line for each person based on the region information and attributes for each person, and generates and outputs a mask video in which the inside of the contour line is changed to a mask image designated for each attribute.

Japanese Patent No. 5834193

  According to Patent Document 1, since a mask image is generated for each position information and person attribute (for example, gender, age) in a moving image of a person detected in a monitoring area, privacy is ensured by the mask image. The behavior of the person in the store can be grasped above.

  However, in Patent Document 1, it is a camera that changes the image area of a person detected in a moving image to a mask image, and the mask is set by a receiving device (for example, a client PC used by a user) that receives the moving image from the camera. It is not taken into account that processing is performed.

  For this reason, while the client PC receives different captured images captured by a plurality of cameras and separately displays them on the monitor, the setting of whether or not to perform mask processing is individually set for each captured image. It was not possible to do this, and the user-friendliness was not good. For example, when the camera is set in advance to perform mask processing, while the video distributed from the camera is displayed on the monitor of the client PC, the camera imaging process is temporarily stopped and the camera Unless the mask processing is changed to the setting that does not execute, the client PC cannot be changed to the setting that disables the mask processing in the camera during the display on the monitor.

The present invention has been devised in view of the above-described conventional situation, and when a plurality of different types of captured images are divided and displayed on a monitor, the setting of execution or non-execution of mask processing for each captured image is captured. It is an object of the present invention to provide a monitoring system , a video display method, and a computer program that are easily set for each user and improve user convenience.

The present invention performs execution or non-execution of a mask process for changing a plurality of cameras that capture an imaging area and at least a part of a moving body that appears in the captured image of the imaging area captured by each of the cameras to a mask image. the setting information of the execution and a processing device for holding for each of the camera, the information processing apparatus, the information processing apparatus of the individual captured image captured by multiple cameras execution of the masking process is set wherein in each edited video obtained by masking, or as the mask processing edited video imaging picture obtained by mask processing by the information processing apparatus each execution is imaged by the at least one camera is set in A monitoring system that divides and displays, on a monitor, individual captured images captured by at least one camera for which non-execution of mask processing is set Subjected to.

Further, the present invention is a video display method for displaying an image using an information processing apparatus for displaying a captured image of the imaging area which is more imaging a plurality of cameras for imaging the imaging area to the monitor, the information The processing apparatus holds, for each camera, setting information for performing or not executing mask processing for changing at least a part of the moving body appearing in the captured image of the imaging area captured by each camera to a mask image. The information processing device is configured to perform individual editing video obtained by performing mask processing on each captured image captured by a plurality of cameras set to execute the mask processing by the information processing device or the mask processing. execution is set at least one camera by non-execution of the mask processing and editing images obtained by masking individual captured image captured by the information processing apparatus And individual captured image captured by the constant has been at least one camera, the dividing displayed on the monitor, to provide an image display method.

  According to the present invention, when a plurality of different types of captured images are divided and displayed on the monitor, the execution or non-execution of mask processing for each captured image is simply set for each captured image, and the user-friendliness is improved. Can be improved.

The figure which shows an example of the system configuration | structure of the surveillance camera system of each embodiment Block diagram showing an example of the internal configuration of an omnidirectional camera or fixed camera in detail Block diagram showing in detail an example of the internal configuration of a PTZ camera Block diagram showing an example of the internal configuration of a client PC in detail Block diagram showing an example of the internal configuration of the recorder in detail The figure which shows an example of a camera setting screen Figure showing an example of the area setting screen Explanatory drawing which shows the 1st example of the time-series transition of the screen accompanying the user operation regarding the setting of a mask process (MOR process). Explanatory drawing which shows the 2nd example of the time-series transition of the screen accompanying user operation regarding the setting of a mask process (MOR process). The flowchart explaining in detail an example of the operation procedure of the client PC accompanying the user operation regarding the setting of the mask process (MOR process) in the first embodiment. The flowchart explaining in detail an example of the operation procedure of the mask process (MOR process) common to each embodiment FIG. 11 is a flowchart for explaining in detail an example of an operation procedure of background image update processing in steps S15 and S16 in FIG. Explanatory diagram schematically showing the background image generation procedure by initial learning Explanatory drawing which shows the concept of a mask process (MOR process) typically (A) And (B) The figure which shows the example of a display which imaged the imaging | video of several cameras into 3 parts, (C) The figure which shows the example of a display which divided | segmented the imaging | video images of several cameras into 4 parts The flowchart explaining in detail an example of the operation procedure performed in the client PC when displaying the captured images of a plurality of cameras in the second embodiment. The figure which shows the example of a display which divided the picked-up image of a camera, and the several different picked-up image of the past of the same camera into 4 parts The flowchart explaining in detail an example of the operation procedure performed in the client PC when displaying the captured video of the camera and the recorded video from the recorder in the second embodiment. FIG. 10 is a sequence diagram for explaining in detail an example of an operation procedure for setting a mask process (MOR process) of a client PC for a mask camera and a normal camera and displaying a captured image in the third embodiment. The figure which shows an example of the camera setting screen in 3rd, 4th each embodiment The figure which shows the 4-part dividing display example of the edit image | video with which the picked-up image displayed on the monitor of client PC was masked. Diagram showing an example of the recorder setting screen (A) Sequence diagram for explaining in detail a first example of an operation procedure for setting mask processing (MOR processing) in the fourth embodiment. (B) Mask processing (MOR processing) in the fourth embodiment. Sequence diagram for explaining in detail the second example of setting operation procedure The sequence diagram explaining in detail the 1st example of the operation | movement procedure regarding acquisition and the display of the captured image in the client PC after operation start in 4th Embodiment The sequence diagram explaining in detail the 2nd example of the operation | movement procedure regarding acquisition and a display of the captured image in the client PC after operation start in 4th Embodiment FIG. 24 and FIG. 25 are flowcharts for explaining in detail an example of the display procedure of the client PC in step S69.

(Background to the contents of the first embodiment)
According to Patent Document 1 described above, a mask image is generated for each position information and person attribute (for example, gender, age) in a moving image of a person detected in a monitoring area. Once secured, the behavior of the person in the store can be grasped.

  However, Patent Document 1 does not consider the idea of changing a person existing in an arbitrary area in a monitoring area directly designated by a user (for example, a monitor) to a captured image of a camera to a mask image. . Although it is important to protect the privacy of a person existing in the captured video, depending on the installation environment of the camera, when one or more persons existing in all areas in the captured video all become mask images. The visibility of the captured image deteriorates. For this reason, it is desired to achieve both the protection of the privacy of a person and the suppression of deterioration of the visibility of the captured image.

  Therefore, in the following first embodiment, the privacy of a person shown in the captured image of the camera is protected and the deterioration of the visibility of the captured image of the camera is suppressed so as to suit the environment of the place where the camera is installed. An example of a monitoring system and a mask processing setting method that improve the usability of the user will be described.

  Hereinafter, each embodiment specifically disclosing the monitoring system according to the present invention will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of already well-known matters and repeated descriptions for substantially the same configuration may be omitted. This is to avoid the following description from becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

  Hereinafter, a monitoring camera system according to the present invention, a monitoring camera system in which at least one surveillance camera is installed in a monitoring area (for example, a store, a factory, a sales office, a public facility such as a library or an event venue, an office, or a bank). An example of a manned surveillance camera system or an unmanned surveillance camera system is possible.

  The present invention can also be expressed as a mask processing setting method in the monitoring system. Further, the present invention executes a program for causing a client PC (Personal Computer), which is a computer constituting the monitoring system, to execute an operation specified by the mask processing setting method, or an operation specified by the mask processing setting method. It can also be expressed as a recording medium on which a program for causing the program to be recorded is recorded.

  Hereinafter, a user of a monitoring camera system as an example of a monitoring system (for example, a monitoring person browsing a captured image of a monitoring area) is simply referred to as a “user”. In the following description, “omnidirectional camera”, “PTZ camera”, and “fixed camera” are simply referred to as “camera”. Therefore, in the following description, when simply described as “camera”, the camera indicates at least one of an omnidirectional camera, a PTZ camera, and a fixed camera.

(First embodiment)
FIG. 1 is a diagram illustrating an example of a system configuration of a monitoring camera system 100 according to each embodiment. In the surveillance camera system 100, recorders REC1 and REC4 and routers RUT2, RUT3 and RUT4 are connected via a network NW. The surveillance camera system 100 includes a client PC 10 used by a user, a plurality of cameras (refer to an omnidirectional camera, a PTZ (Pan Tilt Zoom) camera, and a fixed camera described later), and a plurality of recorders (refer to a later description). It is a configuration.

  The recorder REC1 is connected to the omnidirectional camera CAM11, the PTZ cameras CAM12 and CAM13, and the fixed camera CAM14 via a router RUT1 having a known routing function. The captured images captured by these omnidirectional cameras CAM11, PTZ cameras CAM12, CAM13, and fixed camera CAM14 are stored as recorded images in the recorder REC1 via the router RUT1 in association with the corresponding camera identification information and imaging date / time. The The omnidirectional camera CAM11, the PTZ cameras CAM12, CAM13, and the fixed camera CAM14 may each capture the same imaging area or different imaging areas. Thus, in each embodiment including this embodiment, a plurality of cameras (that is, the omnidirectional camera CAM11, the PTZ cameras CAM12, CAM13, and the fixed camera CAM14) are associated with the recorder REC1.

  The recorder REC4 has a function as an online storage, and stores data such as a captured video requested to be recorded from a connection destination device (for example, another recorder, the client PC 10) of the network NW.

  The recorder REC2 is connected to the network NW, the PTZ camera CAM21, and the fixed camera CAM22 via a router RUT2 having a known routing function. The captured images captured by the PTZ camera CAM21 and the fixed camera CAM22 are stored as recorded images in the recorder REC2 via the router RUT2 in association with the corresponding camera identification information and imaging date / time. The PTZ camera CAM21 and the fixed camera CAM22 may each capture the same imaging area or different imaging areas. Thus, in each embodiment including the present embodiment, a plurality of cameras (that is, the PTZ camera CAM21 and the fixed camera CAM22) are associated with the recorder REC2.

  The recorder REC3 is connected to the network NW, the omnidirectional camera CAM31, the PTZ cameras CAM32, CAM33, and the fixed camera CAM34 via a router RUT3 having a known routing function. The captured images respectively captured by these omnidirectional cameras CAM31, PTZ cameras CAM32, CAM33, and fixed camera CAM34 are stored as recorded images in the recorder REC3 via the router RUT3 in association with the corresponding camera identification information and imaging date / time. The The omnidirectional camera CAM31, the PTZ cameras CAM32, CAM33, and the fixed camera CAM34 may each capture the same imaging area or different imaging areas. Thus, in each embodiment including this embodiment, a plurality of cameras (that is, the omnidirectional camera CAM31, the PTZ cameras CAM32, CAM33, and the fixed camera CAM34) are associated with the recorder REC3.

  The client PC 10 as an example of the information processing apparatus is imaged by the above-described cameras (specifically, omnidirectional cameras CAM11 and CAM31, PTZ cameras CAM12, CAM13, CAM21, CAM32, CAM33, fixed cameras CAM14, CAM22, and CAM34). Each of the captured images has a function of performing a mask process for changing at least a part or the whole of a moving body (for example, a person, the same applies hereinafter) appearing in the captured image to a mask image. The client PC 10 displays the captured video transmitted (distributed) from various cameras or recorders on the monitor DP, or monitors the edited video after masking the captured video transmitted (distributed) from various cameras or recorders. Or display on DP. In the following description, the moving body is not limited to a person, and may be an automobile or a robot.

  Here, the mask image is subjected to image processing (that is, mask processing) for making it difficult to specifically identify who the person is in order to protect the privacy of the person appearing in the captured video. It is an image. The mask image is, for example, an image in which the inside of a contour line of a person appearing in a captured video is processed to be translucent. In the following description, the mask process may be referred to as a MOR (Moving Object Remover) process. The MOR process, which will be described in detail later with reference to FIGS. 11 and 12, shows a process of replacing a moving object region where a moving object appears in a captured image with a mask image (for example, a translucent image). As a result, it is difficult to specifically identify who the person appears in the captured video, and the privacy of the person can be protected.

  In the following description, an image captured by the camera is referred to as “captured image”, and a mask image after the captured image is masked by, for example, the client PC 10 or a mask camera having a mask function (see later) is captured image. The video superimposed on is called “edited video”.

  The client PC 10 is connected to the network NW and the system controller 5 via a router RUT 4 having a known routing function. The system controller 5 is connected to a joystick J1 provided to support user operations. For example, the system controller 5 performs various settings related to the operation of the devices constituting the surveillance camera system 100.

  The network NW may be a wired network (for example, an intranet, the Internet, a wired LAN (Local Area Network), or a wireless network (for example, a wireless LAN), for example, the recorders REC1, REC2, REC3, and REC4 are connected via the network NW. The monitor DP may be directly connected to the client PC 10. In addition, the monitor DP is illustrated as being provided separately from the client PC in Fig. 1. However, the monitor DP is shown in Fig. 4. The system controller 5, the joystick J1, and the client PC 10 may be installed in, for example, a monitoring room where a user resides during monitoring.

  The network NW may be a wide area network (WAN), and in this case, for example, a recorder is provided for each of a plurality of bases or a plurality of stores in a company, The picked-up image picked up for each store is stored in each recorder.

  2 is a block diagram showing in detail an example of the internal configuration of the omnidirectional camera CAM11,... Or the fixed camera CAM14,. The internal configurations of the omnidirectional cameras CAM11 and CAM31 and the fixed cameras CAM14, CAM22, and CAM34 are the same. Here, for the sake of simplicity of description, for example, the internal configuration of the omnidirectional camera CAM11 will be described. In the description of FIG. 2, the omnidirectional camera CAM11 may be read as another omnidirectional camera (for example, the omnidirectional camera CAM31) or another fixed camera (for example, the fixed camera CAM14). However, in the description of FIG. 2, the contents of different configurations of the omnidirectional camera CAM11 and the fixed camera will be described. The omnidirectional camera CAM11 shown in FIG. 2 includes a processor C11, a communication unit C12, a lens C13, an image sensor C14, a memory C15, and a power management unit C16.

  The processor C11 performs signal processing for overall control of operations of each unit of the omnidirectional camera CAM11, data input / output processing with other units, data calculation processing, and data storage processing. The processor C11 is configured using, for example, a central processing unit (CPU), a micro processing unit (MPU), a digital signal processor (DSP), or a field-programmable gate array (FPGA).

  For example, the processor C11 acquires omnidirectional image data obtained by imaging in the image sensor C14 in accordance with a user operation using the system controller 5 or the client PC 10, and further, a specific range of the omnidirectional image data. Two-dimensional panoramic image data (that is, image data obtained by two-dimensional panorama conversion) obtained by cutting out an image of (direction) is generated and stored in the memory C15. Further, the processor C11 acquires captured image data having a fixed angle of view obtained by imaging by the image sensor C14 in accordance with a user operation using the system controller 5 or the client PC 10, and stores it in the memory C15.

  The communication unit C12 is a network interface (I / F) that controls data communication with the router RUT1, and transmits omnidirectional image data, two-dimensional panoramic image data, or captured image data with a fixed angle of view via the router RUT1. To the recorder REC1. Further, the communication unit C12 transmits the omnidirectional image data, the two-dimensional panoramic image data, or the captured image data with a fixed angle of view to the client PC 10 in association with the camera identification information and the imaging date and time via the router RUT1 and the recorder REC1. It doesn't matter.

  For example, in the omnidirectional camera CAM11, the lens C13 is configured by using a lens having a wide angle of view or a fisheye lens. An optical image of the subject light is formed on the light receiving surface (imaging surface) of the sensor C14. On the other hand, the lens C13 is configured by using, for example, a lens having a fixed angle of view in the fixed camera CAM14. The lens C13 receives and collects subject light from the angle of view of the imaging area to be imaged by the fixed camera CAM14, and collects it. An optical image of the subject light is formed on the light receiving surface (imaging surface) of C14.

  The image sensor C14 is configured by using, for example, a complementary metal oxide semiconductor (CMOS) sensor or a charge coupled device (CCD) sensor, and receives an optical image of subject light from the imaging area collected by the lens C13 as a light receiving surface (imaging surface). ), The omnidirectional image data of the imaging area or the captured image data of a fixed angle of view is acquired. The omnidirectional image data or the captured image data having a fixed angle of view is acquired by the processor C11.

  The memory C15 includes a ROM 15z in which programs for setting the operation of the omnidirectional camera CAM11 or the fixed camera CAM14 and data of setting values are stored, captured image data having a fixed angle of view, omnidirectional image data, or a partial range thereof. A RAM 15y that stores cut-out image data and work data obtained by cutting out, and a memory card 15x that is detachably connected to the omnidirectional camera CAM11 or the fixed camera CAM14 and stores various data.

  The power management unit C16 supplies DC power to each unit of the omnidirectional camera CAM11 or the fixed camera CAM14. The power management unit C16 may supply DC power to other devices (for example, PTZ cameras CAM12 and CAM13) connected to the router RUT1.

  FIG. 3 is a block diagram showing in detail an example of the internal configuration of the PTZ cameras CAM12, CAM13,. The internal configurations of the PTZ cameras CAM12, CAM13, CAM21, CAM32, and CAM33 are the same. Here, for the sake of simplicity of description, for example, the internal configuration of the PTZ camera CAM12 will be described. In the description of FIG. 3, the PTZ camera CAM12 may be replaced with another PTZ camera (for example, the PTZ camera CAM13). Moreover, about each part similar to the omnidirectional camera CAM11 and the fixed camera CAM14, the code | symbol corresponding to each part of FIG. 2 is attached | subjected, and the description is abbreviate | omitted. The PTZ camera CAM 12 is a camera that can adjust the optical axis direction (imaging direction), for example, according to a view angle change instruction from the system controller 5 or the client PC 10 used by the user.

  The PTZ camera CAM12 includes, for example, a processor C21, a communication unit C22, a lens C23, an image sensor C24, and a memory C25 as well as the omnidirectional camera CAM11, and also includes an imaging direction control unit C27 and a lens drive motor C28. For example, when there is a view angle change instruction from the system controller 5 or the client PC 10, the processor C21 notifies the imaging direction control unit C27 of the view angle change instruction.

  The imaging direction control unit C27 controls the imaging direction of the PTZ camera CAM12 in at least one of the pan direction and the tilt direction according to the view angle change instruction notified from the processor C21, and further changes the zoom magnification as necessary. A control signal for this is output to the lens drive motor C28. The lens drive motor C28 drives the lens C23 in accordance with this control signal, changes the imaging direction (that is, the optical axis direction), and adjusts the focal length of the lens C23 to change the zoom magnification.

  The lens C23 is configured using one or two or more lenses. In the lens C23, the optical axis directions of pan rotation and tilt rotation are changed by driving the lens driving motor C28 according to the control signal from the imaging direction control unit C27.

  FIG. 4 is a block diagram showing an example of the internal configuration of the client PC 10 in detail. The client PC 10 illustrated in FIG. 4 includes a processor 11, a communication unit 12, an operation unit 13, a storage 15, a memory 16, and a power management unit 17.

  The processor 11 performs signal processing for overall control of operations of each unit of the client PC 10, data input / output processing with other units, data calculation processing, and data storage processing. The processor 11 is configured using, for example, a CPU, MPU, DSP, or FPGA. The processor 11 includes an image input unit 111, a background image processing unit 112, a mask processing setting unit 113, a mask processing unit 114, and a video output control unit 115.

  The image input unit 111 inputs captured images transmitted from various cameras and recorders and received by the communication unit 12.

  The background image processing unit 112 will be described in detail with reference to FIG. 11 and FIG. 12. The background image processing unit 112 uses the captured video acquired by the image input unit 111 to perform background image processing performed in mask processing (MOR processing) in the client PC 10. Perform various generation and update processes. Details of generation and update of the background image will be described later.

  The mask process setting unit 113 performs settings related to a mask process (MOR process) that can be executed by the client PC 10 in response to a user operation (described later).

  The mask processing setting unit 113, for example, in response to a user operation that specifies whether or not to perform mask processing on the captured video acquired by the image input unit 111, the distribution source (transmission source) of the captured video Is generated in association with a camera (that is, an omnidirectional camera, a PTZ camera, or a fixed camera), and mask setting information indicating execution or non-execution of mask processing is generated and stored in the storage 15 or the memory 16.

  Further, the mask processing setting unit 113, for example, position information (for example, coordinate information) of the designated area in response to a user operation that designates a non-target area for mask processing in the captured video acquired by the image input unit 111, for example. Is generated and stored in the storage 15 or the memory 16. Although details of generation of the mask setting information will be described later, the mask setting information holds at least the position information of the mask processing execution / non-execution area and the mask processing non-target area for each camera.

  Based on the mask setting information generated by the mask processing setting unit 113, the mask processing unit 114 selects a region not to be masked specified by the mask setting information from the captured video acquired by the image input unit 111. The excluded area is selected as a mask process target area. The mask processing unit 114 generates a mask image in which the outline of the moving body (for example, a person) is changed to a mask image (for example, a translucent image) in the selected mask processing target region. Details of the mask processing will be described later.

  The video output control unit 115 displays the captured video from the camera set not to execute the mask process or the edited video already masked (see later) by the camera as it is on the monitor DP. In addition, the video output control unit 115 uses the captured video from the camera set to execute the mask process in the client PC 10 and the mask image generated by the mask processing unit 114 for the captured video. An edited video in which a mask image is superimposed on the position of the moving body (for example, a person) inside is generated. The video output control unit 115 displays the edited video on the monitor DP.

  The communication unit 12 receives captured video or edited video (described later) transmitted from various cameras and recorders via the router RUT 4 and outputs the received video to the processor 11.

  The operation unit 13 is a user interface (UI) for notifying the processor 11 of the contents of a user input operation, and is configured by a pointing device such as a mouse or a keyboard. The operation unit 13 may be configured using, for example, a touch panel or a touch pad that is arranged corresponding to the display screen of the monitor DP and can be directly input by a user's finger or stylus pen.

  The monitor DP as an example of a display unit is configured using, for example, an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence), and is set by a mask processing setting unit 113 (see FIG. 6) or an area setting screen. (See FIG. 7) or a captured image or an edited video output by the video output control unit 115 is displayed.

  The storage 15 as an example of a storage unit stores, for example, mask setting information generated by the mask processing setting unit 113, and is further generated by a captured image distributed from (transmitted to) various cameras and recorders or the processor 11. Stores edited video data.

  The memory 16 includes a ROM 161 that stores a program for specifying the operation of the client PC 10 and data of setting values, and a RAM 162 that stores various data generated by the processor 11 and work data being processed. Although not shown in FIG. 4, the memory 16 may further include a memory card (not shown) that is connected to the client PC 10 so as to be freely inserted and removed and that stores various data.

  The power management unit 17 supplies DC power to each unit of the client PC 10.

  FIG. 5 is a block diagram showing in detail an example of the internal configuration of the recorders REC1,. The internal configuration of the recorders REC1, REC2, REC3, and REC4 is the same. For the sake of simplicity, the recorder REC1, REC2, REC3, and REC4 will be described as the internal configuration of the recorder REC1, for example. In the description of FIG. 5, the recorder REC1 may be replaced with another recorder (for example, the recorder REC2). The recorder REC1 shown in FIG. 5 includes a processor R11, a communication unit R12, a memory R13, a storage R14, and a power management unit R16.

  The processor R11 performs signal processing for overall control of operations of each unit of the recorder REC1, data input / output processing with other units, data calculation processing, and data storage processing. The processor R11 is configured using, for example, a CPU, MPU, DSP, or FPGA.

  For example, the processor R11 captures image data captured by various cameras (for example, the omnidirectional camera CAM11, the PTZ cameras CAM12, CAM13, and the fixed camera CAM14) in accordance with a user operation using the system controller 5 or the client PC 10. The captured video data is stored as a recorded video in the storage R14 in association with the imaging date and camera identification information.

  The communication unit R12 captures captured image data captured by various cameras (for example, the omnidirectional camera CAM11, the PTZ cameras CAM12, CAM13, and the fixed camera CAM14) or an edited image obtained by masking the captured image (see below). ) Is received via the router RUT1 and output to the processor R11. Further, the communication unit R12 transmits the recorded video data read from the storage R14 by the processor R11 to the client PC 10 in response to an instruction from the client PC 10.

  The memory R13 includes a ROM 131 that stores a program for defining the operation of the recorder REC1 and data of set values, and a RAM 132 that stores various data generated by the processor R11 and work data being processed. Although not shown in FIG. 5, the memory R13 may further include a memory card (not shown) that is detachably connected to the recorder REC1 and stores various data.

  The storage R14 stores captured video data distributed (transmitted) from various cameras, for example. The storage R14 may store the edited video data when the edited video data generated by the client PC 10 is transmitted from the client PC 10 to the recorder REC1.

  The power management unit R15 supplies DC power to each unit of the recorder REC1.

  Next, the operation of the monitoring camera system 100 of the present embodiment will be described in detail with reference to FIGS.

  FIG. 6 is a diagram illustrating an example of the camera setting screen WD1. FIG. 7 is a diagram illustrating an example of the area setting screen WD2. FIG. 8 is an explanatory diagram showing a first example of time-series transition of a screen accompanying a user operation related to setting of mask processing (MOR processing). FIG. 9 is an explanatory diagram illustrating a first example of time-series transition of a screen accompanying a user operation regarding setting of mask processing (MOR processing). FIG. 10 is a flowchart for explaining in detail an example of an operation procedure of the client PC 10 in accordance with a user operation regarding the setting of the mask process (MOR process) in the first embodiment.

  First, the camera setting screen WD1 shown in FIG. 6 will be described.

  In FIG. 6, a part of the camera setting screen WD1 is extracted and shown. The mask processing setting unit 113 displays the camera setting screen WD1 on the monitor DP, for example, by a user operation. In this camera setting screen WD1, at least IP (Internet) is provided to individual cameras of, for example, a maximum of 64 cameras (that is, composed of an omnidirectional camera, a PTZ camera, and a fixed camera) by the user's operation using the operation unit 13. Protocol) address, mask processing (MOR processing), setting of mask processing (MOR processing), initial learning period, and normal learning period can be associated and set. In addition, by switching the four tabs (camera 1-16, camera 17-32, camera 33-48, camera 49-64) provided on the camera setting screen WD1, about the corresponding camera in units of 16 cameras, Information on at least an IP (Internet Protocol) address, mask processing (MOR processing), setting of mask processing (MOR processing), initial learning period, and normal learning period can be set.

  In the IP address column CLM1, the IP address of the corresponding camera is input.

  In the column CLM2 of mask processing (MOR processing), either of execution (ON) or non-execution (OFF) of mask processing (MOR processing) in the client PC 10 is selected (see frame WK2).

  In the column CLM3 for setting the mask process (MOR process), a setting button (see frame WK1) for displaying an area setting screen WD2 for setting a non-target area for the mask process (MOR process) is displayed. However, the setting button of the column CLM3 is displayed so that it can be pressed for a camera for which ON is selected in the column CLM2, but the setting button of the column CLM3 is not displayed so that it can be pressed for a camera for which OFF is selected in the column CLM2. (For example, refer to the dotted lines on the fifth, eighth, tenth, and fifteenth stages from the top of FIG. 6 in the row CLM3).

  In an initial learning period column CLM4, an initial learning period in a mask process (MOR process) described later is input (see frame WK3). Similarly, the initial learning period of the column CLM4 can be input for a camera whose ON is selected in the column CLM2, but the initial learning period of the column CLM4 cannot be input for a camera whose OFF is selected in the column CLM2. For example, see the fifth, eighth, tenth, and fifteenth dotted lines from the top of the page of FIG.

In the normal learning period column CLM5, a normal learning period in a mask process (MOR process) described later is input (see frame WK3). Similarly, the normal learning period of the column CLM5 can be input for a camera whose ON is selected in the column CLM2, but the normal learning period of the column CLM5 cannot be input for a camera whose OFF is selected in the column CLM2. (For example, refer to the dotted lines on the fifth, eighth, tenth, and fifteenth tiers from the top of FIG. 6 of the row CLM5).

  Next, the area setting screen WD2 shown in FIG. 7 will be described.

  In FIG. 7, the entire area setting screen WD2 is shown. When the setting button of the column CLM3 is pressed for a camera whose ON is selected in the column CLM2 of the camera setting screen WD1, the mask processing setting unit 113 switches the region setting screen WD2 to the monitor DP and displays it. The mask processing setting unit 113 may display the camera setting screen WD1 and the region setting screen WD2 separately on the monitor DP. In this area setting screen WD2, a non-target area for mask processing (MOR processing) can be set in association with the camera while the captured video MV1 captured by the camera is displayed by the user's operation using the operation unit 13. .

  On the area setting screen WD2, items of device name TM1, visual area TM2, non-visual area TM3, deletion TM4, and image call TM5, a display area for video, and a set button STBT for setting completion are shown. . The display area for video is a display area of the captured video MV1 (that is, live video) currently captured by the camera (for example, PTZ camera CAM13) specified by the device name TM1. However, the video displayed in the video display area of the area setting screen WD2 is not limited to the captured video MV1 (that is, live video) currently captured by the camera (for example, the PTZ camera CAM13), and is already in the recorder REC1, for example. It may be a recorded video recorded in the past. In this case, the client PC 10 receives the past recorded video distributed (transmitted) from the recorder REC1, and displays it in the video display area of the area setting screen WD2.

  In the device name TM1, for example, a PTZ camera CAM13 is shown.

  In the visual area TM2, area selection buttons VA1, VA2, and VA3 for designating areas not to be masked are displayed. In other words, if a region that is not subject to mask processing is designated, even if there is a person who moves within that region, the mask processing is not performed and the image is displayed in the video. The area selection button VA1 is used to designate a non-target area for mask processing with a rectangle (for example, a rectangular shape) (see FIG. 8). The area selection button VA2 is used for designating a non-maskable area as a polygon (for example, a pentagon) (see FIG. 9). The area selection button VA3 is used to select a mask processing non-target area designated by operating the area selection button VA1 or the area selection button VA2.

  In the non-visual area TM3, area selection buttons IVA1, IVA2, and IVA3 for designating an area (hidden area) to be hidden from the captured image are displayed. In other words, when a hidden area is designated, the hidden area is filled with, for example, black, and is not displayed in the video. The area selection button IVA1 is used to designate a hidden area with a quadrangle (for example, a rectangular shape). The area selection button IVA2 is used to designate a hidden area with a polygon (for example, a pentagon). The area selection button IVA3 is used to select a hidden area designated by the operation of the area selection button IVA1 or the area selection button IVA2.

  The deletion TM4 includes a selection button DL1 for deleting one selected mask processing non-target area or one selected hidden area, and all selected mask processing non-target areas or all selected mask processing. A selection button DL2 for deleting the hidden area is displayed. Note that, when only one of the non-masking target areas or the hidden areas is selected, the corresponding area portion is indicated by a dotted line in the captured video MV1.

  The set button STBT is pressed by a user operation when a non-masking target area or a hidden area is designated and the designated mask processing non-target area or hidden area is determined. Accordingly, the mask processing setting unit 113 displays the position information (more specifically, the position information in the captured video MV1) of the specified non-target mask processing area and the hidden area by pressing the set button STBT. The data is stored in the storage 15 or the memory 16 in association with the camera specified by the above.

  Here, the setting of the non-target area for the mask process will be described with reference to FIGS. 8 and 10, for example. By a user operation for displaying the camera setting screen WD1 (see FIG. 6), the mask processing setting unit 113 displays the camera setting screen WD1 on the monitor DP (S1).

  When a user operation selects the column CLM2 of the record of the PTZ camera CAM13 on the camera setting screen WD1 (see FIG. 6) and the setting button (see the frame WK1) of the column CLM3 of the record is pressed (S2, YES), the mask processing setting unit 113 displays an area setting screen WD2 corresponding to the PTZ camera CAM13 on the monitor DP (S3). Unless the setting button of the column CLM3 is pressed (S2, NO), the processing of the processor 11 of the client PC 10 does not proceed to step S3.

  In the area setting screen WD2 of FIG. 8, the area selection button VA1 (for example, a rectangular area selection button VA1) of the visual area TM2 is pressed by a user operation (S4), and the persons JB1, JB2, and JB3 are shown. It is assumed that the television apparatus in the captured video MV1 is designated to be surrounded by a drag operation or the like (S5). The mask processing setting unit 113 draws a rectangular frame JGA1 around the television device designated by the region selection button VA1 on the region setting screen WD3. Further, the mask processing setting unit 113 uses the position information (for example, coordinate information) of the frame JGA1 as a region not to be masked when the set button STBT is pressed by a user operation on the region setting screen WD3 (S6). Then, camera setting information associated with the PTZ camera CAM 13 is generated and stored in the storage 15 or the memory 16 (S7). The mask processing unit 114 refers to the camera setting information generated by the mask processing setting unit 113 and masks moving bodies (for example, persons JB1, JB2, JB3) other than the inside of the frame JGA1 in the captured video MV1 as mask images MK1, MK2. , MK3 to perform mask processing. The video output control unit 115 generates an edited video using the mask images MK1, MK2, and MK3 generated by the mask processing unit 114 and the captured video MV1, and displays the edited video on the video display screen WD4 of the monitor DP.

  Thereby, since the inside of the frame JGA1 is a non-masking target area, the masking process is not performed. For example, even if a person moving to the television set in the frame NMK1 of the video display screen WD4 is displayed, It is possible to suppress the deterioration of the visibility of the edited video displayed on the video display screen WD4 without being changed to the mask image (for example, the mask image MK1).

  Note that the video output control unit 115 executes mask processing (more specifically, normal learning) when performing normal learning (see below) executed during mask processing in the mask processing unit 114. “MOR Running” indicator ID1 indicating that the image is displayed on the upper left of the video display screen WD4. Accordingly, the user can visually and easily grasp the state in which the mask process is being executed in the client PC 10 by browsing the video display screen WD4 of the monitor DP.

  Here, the setting of the non-target area for mask processing will be described with reference to FIGS. 9 and 10, for example. By a user operation for displaying the camera setting screen WD1 (see FIG. 6), the mask processing setting unit 113 displays the camera setting screen WD1 on the monitor DP (S1).

  When a user operation selects the column CLM2 of the record of the PTZ camera CAM13 on the camera setting screen WD1 (see FIG. 6) and the setting button (see the frame WK1) of the column CLM3 of the record is pressed (S2, YES), the mask processing setting unit 113 displays an area setting screen WD2 corresponding to the PTZ camera CAM13 on the monitor DP (S3). Unless the setting button of the column CLM3 is pressed (S2, NO), the processing of the processor 11 of the client PC 10 does not proceed to step S3.

  In the area setting screen WD2 of FIG. 9, the area selection button VA2 (for example, the pentagonal area selection button VA2) of the visual area TM2 is pressed by the user operation (S4), and the persons JB1, JB2, and JB3 are shown. It is assumed that the pentagonal vertices PT1, PT2, PT3 are selected and the sides between the vertices are designated by a drag operation or the like so that the head periphery of the person JB1 in the captured video MV1 is surrounded (S5). The mask processing setting unit 113 draws a pentagonal frame JGA2 around the head of the person JB1 designated by the region selection button VA2 on the region setting screen WD3. Furthermore, the mask processing setting unit 113 uses the position information (for example, coordinate information) of the frame JGA2 as an area not to be masked when the set button STBT is pressed by a user operation on the area setting screen WD3 (S6). Then, camera setting information associated with the PTZ camera CAM 13 is generated and stored in the storage 15 or the memory 16 (S7). The mask processing unit 114 refers to the camera setting information generated by the mask processing setting unit 113 and masks moving bodies (for example, persons JB2, JB3) other than the inside of the frame JGA2 in the captured video MV1 as mask images MK2, MK3 (FIG. 8)) is performed. The video output control unit 115 generates an edited video using the mask images MK2 and MK3 generated by the mask processing unit 114 and the captured video MV1 and displays it on a video display screen (not shown) of the monitor DP.

  Thereby, since the inside of the frame JGA2 is a non-masking target area, the masking process is not performed. For example, even if a person moving around the head of the person JB1 in the video display screen (not shown) is projected. The person is not changed to the mask image (for example, the mask image MK1), and the degradation of the visibility of the edited video displayed on the video display screen can be suppressed.

  Next, details of the operation contents of the mask process (MOR process) in the processor 11 of the client PC 10 will be described with reference to FIGS. 11 and 12. FIG. 11 is a flowchart illustrating in detail an example of an operation procedure of mask processing (MOR processing) common to the embodiments. FIG. 12 is a flowchart illustrating in detail an example of the operation procedure of the background image update process in steps S15 and S16 of FIG.

  In FIG. 11, the background image processing unit 112 of the processor 11 analyzes a captured video (for example, a captured video for a certain period of time after the start of imaging by the camera) acquired by the image input unit 111 and moves in the captured video. A background image made up of a region without an image (hereinafter referred to as “still region”) is generated (S11). Hereinafter, the process of step S11 is defined as “initial learning” in the mask process (MOR process).

  FIG. 13 is an explanatory diagram schematically showing a procedure for generating a background image by initial learning. The background image processing unit 112 acquires captured video for a certain period of time, and in a plurality of captured images constituting the captured video (see the left side of the paper in FIG. 13), by known image processing (for example, calculation of difference between frames), A still area and a moving area (hereinafter referred to as “moving body area”) are detected between frames of the captured image. In FIG. 13, the person JB4 is shown as a moving object. The background image processing unit 112 generates a background image BG1 including a region without motion (that is, a still region) by the above-described image processing.

  Note that during the initial learning process of step S11, the background image processing unit 112 cannot separate the still area and the moving object area in the captured video. For this reason, the captured image during the initial learning period is not displayed on the monitor DP. In other words, the video output control unit 115 does not display the captured video distributed (transmitted) from the camera on the monitor DP during the initial learning period.

  FIG. 14 is an explanatory diagram schematically showing the concept of mask processing (MOR processing). After the completion of the initial learning in step S11, the mask processing unit 114 of the processor 11 uses the pixel value (for example, luminance value or RGB (Red Green Blue) value, etc.) of the background image BG1 generated in step S11 and the camera. The pixel value of the current captured video AA1 distributed (transmitted) is determined in units of pixels constituting a background image or captured video or a unit of a predetermined pixel area composed of a plurality of pixels (for example, a unit of 4 pixels × 4 pixels). Compare.

  As a result of the comparison described above, the mask processing unit 114 determines that the region in the captured image having a pixel value larger than a certain threshold (known) is the moving object region OB1 (S12). The mask processing unit 114 changes the image of the corresponding part in the captured video determined to be the moving object region to the mask image MSK1 (for example, a translucent image), and the edited video BB1 in which the mask image MSK1 is superimposed on the moving object region in the captured video AA1. Is generated and displayed on the monitor DP (S13). The process of displaying the edited video BB1 on the monitor DP may be performed by the video output control unit 115. Further, the mask processing unit 114 may store the generated edited video BB1 in the storage 15 or the memory 16 in association with the camera identification number (for example, camera ID) or transmit it to the recorder (for example, the recorder REC1). You may memorize it.

  On the other hand, as a result of the comparison described above, the mask processing unit 114 determines an area in the captured image in which the pixel value is smaller than a certain threshold (known) as a still area (S14). The mask processing unit 114 displays the image of the corresponding part in the captured image determined to be a still area as it is on the monitor DP as the current captured image or background image (S14). Note that the video output control unit 115 may perform the process of displaying the image of the corresponding part in the captured video determined to be a still region on the monitor DP. Further, the mask processing unit 114 may store the image of the corresponding part in the captured video determined to be a still area in the storage 15 or the memory 16 in association with the camera identification number (for example, camera ID) or the recorder ( For example, it may be transmitted and stored in the recorder REC1). Steps S12, S13, and S14 are not performed in the order of time series, but are performed substantially simultaneously or simultaneously in parallel. The same applies to steps S15 and S16.

  After step S14, the background image processing unit 112 updates the background image of the video area determined as the still area in step S14 using the video area and the background image generated in step S11 (S15). For example, when the background image is updated once per second (see FIG. 12) and 1% of the information of the new captured video is used and 99% of the information of the current background image is used, about 100 seconds later. The background image is completely updated. Details of the processing in step S15 will be described later with reference to FIG.

  After step S14, the background image processing unit 112 updates the background image of the video area determined as the moving object area in step S12 using the video area and the background image generated in step S11 (S16). Hereinafter, “normal learning” in the mask process (MOR process) is defined as the process in steps S15 and S16. For example, when updating the background image once a second (see FIG. 12) and using 0.1% of new captured video information and 99.9% of current background image information, The background image is completely updated after about 1000 seconds. In this case, the normal learning period is the longer period (ie, 1000 seconds = about 17 minutes) of the 100 seconds exemplified in step S15 and the 1000 seconds exemplified in step S16. Details of the processing in step S16 will be described later with reference to FIG.

  If the mask process (MOR process) ends after steps S15 and S16 (S17, YES), the process of the processor 11 shown in FIG. 11 ends. On the other hand, if the mask process (MOR process) does not end after steps S15 and S16 (S17, YES), the process of the processor 11 returns to step S12.

  Each process of step S15 and S16 is performed in the order of the process shown in FIG. The difference between the process of step S15 and the process of step S16 is a value of a background image update ratio X described later. In the description of FIG. 12, for example, the background image update ratio X = 0.01 (= 1%) for the process of step S15, and the background image update ratio X = 0.001 (= 0.1) for the process of step S16. %).

  More specifically, the background image update ratio X = 0.01 (= 1%) means that 1% of new captured video information is used and 99% of current background image information is used in the background image update. It means to do. Similarly, the background image update ratio X = 0.001 (= 0.1%) means that 0.1% of the information of the new captured video is used in the update of the background image, and the current background image is 99.9%. Means to use. In any case, updating the background image means taking in information of a new captured video in accordance with the value of the background image update ratio X.

  In the description of FIG. 12, the pixel value is an RGB value (that is, a value indicating the intensity of red, green, and blue in the pixel in the image). When the color table is 24-bit RGB, (R, G, B) = (255, 255, 255) indicates white, and (R, G, B) = (0, 0, 0) indicates black. , (R, G, B) = (255, 0, 0) indicates red, (R, G, B) = (0, 255, 0) indicates green, and (R, G, B) = ( 0, 0, 255) indicates blue.

  In FIG. 12, the background image processing unit 112 sets the coordinates (x, y) of the pixels constituting the background image to be updated to (0, 0) (S21). The background image processing unit 112 acquires the pixel value (R0, G0, B0) at the pixel at coordinates (x, y) from the current captured video (S22). The background image processing unit 112 acquires the pixel values (R1, G1, B1) at the pixel at coordinates (x, y) from the current background image (for example, the background image generated in the initial learning in step S11) (S23). .

  The background image processing unit 112 uses the pixel values (R0, G0, B0) acquired in step S22 and the pixel values (R1, G1, B1) acquired in step S23, and formulas (1) to (3). Accordingly, the pixel values (R2, G2, B2) at the coordinates (x, y) of the pixels constituting the updated background image are calculated (S24).

R2 = R0 × X + R1 × (1.0−X) (1)
G2 = G0 × X + G1 × (1.0−X) (2)
B2 = B0 × X + B1 × (1.0−X) (3)

  For example, in the case of R0 = 100, R1 = 50, and X = 0.1 (= 10%), R2 = 100 × 0.1 + 50 × 0.9 = 55 is calculated by Equation (1). Thereby, the background image processing unit 112 can update the R (element) value of the corresponding pixel from 50 (= R1) to 55 (= R2).

  After step S24, when the background image processing unit 112 updates the pixel values by calculation using the equations (1) to (3) for the coordinates (x, y) of all the pixels of the background image to be updated. (S25, YES), the processing of the background image processing unit 112 shown in FIG. 12 ends.

  On the other hand, after step S24, the pixel value is updated by calculation using Equations (1) to (3) for the coordinates (x, y) of all the pixels of the background image to be updated by the background image processing unit 112. If not (S25, NO), the background image processing unit 112 sets the coordinates (x, y) of the pixel to the next coordinates (S26). After step S26, the background image processing unit 112 returns to step S22.

  Accordingly, the processor 11 performs the mask process (MOR) shown in FIGS. 11 and 12 to change a moving area (for example, a person) in the captured image into a mask image, so that An edited video that makes it difficult to identify who the person is can be generated and displayed on the monitor DP, and the privacy of the person can be protected. Further, the processor 11 can update the background image to follow the environmental change even when an environmental change occurs, for example, when a new fixed object is installed in the imaging area, or the light is turned off or turned on. it can. For example, the processor 11 can update a newly-installed fixed object in the background image, and can also generate an appropriate edited video corresponding to the environmental change and display it on the monitor DP.

  In the process of step S16, after a sufficient time for the background image processing unit 112 to determine that the new fixed object is a static region has elapsed, it is captured as a background image. It is preferable that the background image is captured after the background follow-up time has elapsed. The background follow-up time corresponds to a normal learning period. For example, in an environment (for example, a place) where the possibility of stopping is low, such as a person on a road, it is preferable to set a normal learning period (for example, 5 minutes = 300 seconds) with a short background following time. On the other hand, in an environment (for example, a place) where there is a high possibility of stopping such as a person in a library, office, factory, etc., the background follow-up time is preferably set to a longer normal learning period (for example, 20 minutes = 1200 seconds). .

  As described above, the surveillance camera system 100 according to the first embodiment includes at least one camera (for example, the omnidirectional camera CAM11,...) That images the imaging area, and the captured image AA1 or the captured image of the imaging area captured by the camera. A client PC 10 that displays on the monitor DP an edited video BB1 obtained by performing a mask process for changing at least part or all of the moving object region OB1 of the person appearing in AA1 into a mask image. In addition, the client PC 10 displays an area setting screen WD2 for setting a non-masking target area including the display area of the captured video MV1 on the monitor DP (see FIG. 7). In response to a user operation, the client PC 10 performs camera setting in which an area specified in the display area of the captured video MV1 in the area setting screen WD2 (see, for example, the frame JGA1 illustrated in FIG. 8) is not subjected to mask processing. The information is stored in the storage 15 or the memory 16 in association with the camera corresponding to the captured video MV1 (for example, the camera that distributed the captured video).

  As a result, the client PC 10 individually sets the non-target area of the mask process (MOR process) performed in the client PC 10 for each camera by a simple operation of the user so as to suit the environment of the place where the various cameras are installed. Can be set to Therefore, in order to efficiently investigate the cause of the trouble when there is any trouble, the surveillance camera system 100 captures the captured video MV1 or recorder (for example, the recorder REC1) distributed from the camera (for example, the PTZ camera CAM13). ) Can be changed to a mask image so that the person's privacy can be protected, and the visibility of the camera's captured image displayed in an area not subject to mask processing (MOR processing) can be changed. Can be prevented. In other words, the surveillance camera system 100 achieves both the protection of the privacy of the person shown in the captured image of the camera and the suppression of the deterioration of the visibility of the captured image of the camera so as to suit the environment of the place where the camera is installed. User convenience can be improved.

  Further, the client PC 10 uses the camera setting information generated for each camera to set the mask processing target area excluding the non-mask processing target area (see, for example, the frame JGA1 shown in FIG. 8) from the entire area of the captured video MV1. The edited video obtained by performing the masking process is displayed on the monitor DP. As a result, the client PC 10 can output video after ensuring the visibility of the video area portion that the user is paying attention to in the captured video, for example, and the person appearing in the video of the mask processing target area is used as the mask image. Since the display can be replaced, it is difficult to specify who is in the mask processing target area, and privacy can be ensured.

  Further, the client PC 10 displays a camera setting screen WD1 for setting at least execution or non-execution of mask processing for each camera on the monitor DP, and any of the cameras in the camera setting screen WD1 according to a user operation. When the execution of the mask processing corresponding to is designated, the camera setting screen WD1 is switched to the region setting screen WD2 corresponding to the designated camera and displayed on the monitor DP. Thereby, the user can easily set execution or non-execution of the mask process (MOR process) on the captured image captured by any camera on the camera setting screen WD1. Furthermore, when the user sets execution of mask processing (MOR processing) on the camera setting screen WD1 and presses the setting button, the region setting screen WD2 is displayed on the monitor DP, so execution of mask processing (MOR processing) is performed. And the setting of the non-target region can be performed continuously.

  Further, the client PC 10 sets a rectangular area designated in the display area of the captured video MV1 in the area setting screen WD2 as a non-maskable area by a user operation. As a result, the user can select a portion of which the visibility is to be ensured by making it a non-target area of the mask process while viewing the captured video MV1 displayed on the area setting screen WD2.

  Further, the client PC 10 sets a polygonal area (for example, a pentagonal area) designated in the display area of the captured video MV1 in the area setting screen WD2 as a non-maskable area by a user operation. As a result, the user can select a portion whose visibility is to be ensured by making it a non-target area of the mask process while viewing the captured image MV1 displayed on the area setting screen WD2 in a polygonal shape (for example, a pentagonal area). Can do.

  In addition, the client PC 10 displays a camera setting screen WD1 for setting at least execution or non-execution of the mask processing for each camera on the monitor DP, and performs imaging in the mask processing in accordance with a user operation on the camera setting screen WD1. The initial learning period for generating the background image of the video MV1 and the normal learning period for updating the background image in the mask process are stored in the storage 15 or the memory 16 in association with each camera.

  As a result, the user can perform initial learning and normal learning of the background image of the mask processing (MOR processing) performed in the client PC 10 by a simple operation of the user so as to suit the environment of the place where the various cameras are installed. The period can be individually set for each camera, and user convenience can be improved. Therefore, the client PC 10 determines the behavioral characteristics of a person existing in the vicinity (for example, a place where the movement of the person is large, a place where the movement of the person is small, depending on the environment (for example, the place) where the camera (for example, the PTZ camera CAM 13) is set The background image can be generated and updated so as to conform to (2), and the edited video after the mask processing suitable for the environment can be generated and displayed.

(Background to the contents of the second embodiment)
According to Patent Document 1, since a mask image is generated for each position information and person attribute (for example, gender, age) in a moving image of a person detected in a monitoring area, privacy is ensured by the mask image. The behavior of the person in the store can be grasped above.

  However, in Patent Document 1, it is a camera that changes the image area of a person detected in a moving image to a mask image, and the mask is set by a receiving device (for example, a client PC used by a user) that receives the moving image from the camera. It is not taken into account that processing is performed.

  For this reason, while the client PC receives different captured images captured by a plurality of cameras and separately displays them on the monitor, the setting of whether or not to perform mask processing is individually set for each captured image. It was not possible to do this, and the user-friendliness was not good. For example, when the camera is set in advance to perform mask processing, while the video distributed from the camera is displayed on the monitor of the client PC, the camera imaging process is temporarily stopped and the camera Unless the mask processing is changed to the setting that does not execute, the client PC cannot be changed to the setting that disables the mask processing in the camera during the display on the monitor.

  Therefore, in the second embodiment, when a plurality of different types of captured images are divided and displayed on a monitor, the execution or non-execution of mask processing for each captured image is simply set for each captured image, An example of a monitoring system and a video display method for improving user convenience will be described.

(Second Embodiment)
As a monitoring system according to the second embodiment, the surveillance camera system 100 described in the first embodiment will be described as an example. Since the internal configuration of the surveillance camera system 100 of the second embodiment is the same as that of the surveillance camera system 100 of the first embodiment, description of the same contents is simplified or omitted, and different contents are described.

  FIG. 15A and FIG. 15B are diagrams illustrating a display example in which captured images of a plurality of cameras are divided into three. FIG. 15C is a diagram illustrating a display example in which captured images of a plurality of cameras are divided into four.

  Assume that the client PC 10 receives different captured images distributed (transmitted) from a plurality of cameras (for example, the omnidirectional camera CAM11, the PTZ cameras CAM12, and CAM13). When it is set on the camera setting screen WD1 (see FIG. 6) that the mask processing (MOR processing) is performed in each camera, the processor 11 performs processing shown in FIGS. 11 and 12 in the mask processing unit 114. Then, the edited video MV2, MV3, and MV4 after mask processing in which the person in each captured video is changed (replaced) to a mask image is generated. In the video output control unit 115, the processor 11 can display the video display screen WD5a or the video display screen 5b in which the edited videos MV2, MV3, and MV4 generated by the mask processing unit 114 are divided into three and displayed on the monitor DP. Yes (see FIG. 15A or FIG. 15B).

  In the video display screen WD5a of FIG. 15A, the display area of the edited video MV2 out of the entire display area 3DV1 of the edited video is larger than the display areas of the other edited videos MV3 and MV4. In the video display screen WD5b of FIG. 15B, the display area of the edited video MV4 out of the entire display area 3DV2 of the edited video is larger than the display areas of the other edited videos MV2 and MV3. The processor 11 is designated with the three-divided display form shown in FIG. 15A in response to a user operation on the display switching box CGB1 shown in FIG. 15A, FIG. 15B, or FIG. The display mode (for example, the 3-part display mode shown in FIG. 15B or the 4-part display mode shown in FIG. 15C) can be switched. However, in order to avoid complication of the drawings of FIGS. 15A and 15B, the display switching box CGB1 is not shown in FIGS. 15A and 15B. In addition, the processor 11 can set by user operation which edit video display area can be maximized in the three-divided display mode shown in FIGS. 15A and 15B.

  Assume that the client PC 10 receives different captured images distributed (transmitted) from a plurality of cameras (for example, the omnidirectional camera CAM11, the PTZ cameras CAM12, CAM13, and the fixed camera CAM14). When it is set on the camera setting screen WD1 (see FIG. 6) that the mask processing (MOR processing) is performed in each camera, the processor 11 performs processing shown in FIGS. 11 and 12 in the mask processing unit 114. Then, the edited video MV2, MV3, MV4, and MV5 after mask processing in which the person in each captured video is changed (replaced) to a mask image are generated. In the video output control unit 115, the processor 11 can display on the monitor DP a video display screen WD6 in which the edited videos MV2, MV3, MV4, and MV5 generated by the mask processing unit 114 are divided into four (FIG. 5). 15 (C)).

  In the video display screen WD6 in FIG. 15C, the display areas of the four edited videos MV2, MV3, MV4, and MV5 are equal in the entire edited video display area 4DV1.

  In FIG. 15A, FIG. 15B, and FIG. 15C, all the images displayed on the image display screens WD5a, WD5b, and WD6 are the edited images after the processor 11 performs the masking process. It is shown. However, at least one video may be a captured video that has not been subjected to mask processing. This is caused by setting information on whether or not to perform mask processing (MOR processing) for each camera on the camera setting screen WD1. In other words, when the client PC 10 displays a plurality of videos in a divided manner, the client PC 10 executes an edited video or a mask process in which a mask process corresponding to each camera is executed according to the camera setting information set by the user on the camera setting screen WD1. It is only necessary to divide and display the captured video as it is received.

  For example, in the case of a four-divided display mode, one captured image distributed from a camera installed near the entrance of the user's company is not subjected to mask processing by the client PC 10 and is directly displayed as a video display screen (for example, a video display screen WD6). Is displayed. Furthermore, the three captured images distributed from other (three) cameras installed in the company building are each subjected to mask processing, and the plurality of edited videos obtained by the mask processing are the same image display screen. May be displayed in the remaining display area. Accordingly, a user (for example, a supervisor) can appropriately determine and set the specific necessity of the person appearing at each location according to the circumstances of the installation location of each camera. For example, the user can grasp the face of the person in the captured image of the place where the user wants to grasp the face. On the other hand, the person's face becomes a mask image in a place where the face does not need to be grasped or where the face should not be grasped. Therefore, the privacy protection of the person can be observed.

  Next, a first operation procedure of split display of the client PC 10 of this embodiment will be described with reference to FIG. FIG. 16 is a flowchart illustrating in detail an example of an operation procedure performed in the client PC 10 when displaying captured images of a plurality of cameras according to the second embodiment. The process illustrated in FIG. 16 is performed in the client PC 10 for each camera that distributes the captured video to the client PC 10.

  In FIG. 16, the mask processing setting unit 113 of the processor 11 determines whether or not the client PC 10 has acquired the captured video of the camera to be divided and displayed (S31). If the mask processing setting unit 113 determines that the captured video of the camera that is the target of split display has not been acquired (NO in S31), the mask processing setting unit 113 waits until the captured video of the camera that is the target of split display is acquired. (S32). In this case, the video output control unit 115 does not display the captured video from the camera that is the target of the split display, and therefore displays nothing on the monitor DP. That is, the display area in the video display screen of the monitor DP is blank. Alternatively, if there is a captured video that the client PC 10 has already received, the video output control unit 115 may display the captured video in a display area in the video display screen of the monitor DP.

  When the mask processing setting unit 113 determines that the captured video of the camera to be divided is acquired (S31, YES), the storage 15 or the memory 16 is based on the camera identification information included in the captured video. Is read with reference to the record of the corresponding camera in the camera setting information stored in (S33). The mask process setting unit 113 determines whether or not the setting to execute the mask process (MOR process) is performed on the captured video in the read record (S34).

  When it is determined that the mask record (MOR process) is set to be executed in the read record (S34, YES), the mask process setting unit 113 notifies the mask process unit 114 to that effect. In response to the notification from the mask processing setting unit 113, the mask processing unit 114 performs the mask processing (MOR processing) shown in FIGS. 11 and 12 on the captured image from the corresponding camera, and performs the mask processing after the mask processing. An edited video is generated (S35). The video output control unit 115 displays the edited video generated in step S35 in a corresponding display area (for example, the display area of the edited video MV2 in FIG. 15A) in the video display screen of the monitor DP (S35).

  On the other hand, when the mask processing setting unit 113 determines that the mask record (MOR process) is not set in the read record (S34, NO), the original captured image (that is, the mask process is performed). The image output control unit 115 is notified that an imaged image obtained by non-imaging imaging (the same applies hereinafter) is displayed (S36). In response to the notification from the mask processing setting unit 113, the video output control unit 115 converts the original captured video received from the camera by the client PC 10 from the corresponding display area (for example, FIG. 15A) in the video display screen of the monitor DP. It is displayed on the display area of the edited video MV2 (S36).

  If the divided display of the client PC 10 is finished after step S35 or step S36 (S37, YES), the processing of the processor 11 shown in FIG. 16 is finished. On the other hand, if the divided display of the client PC 10 does not end after steps S35 and S36 (S37, NO), the processing of the processor 11 returns to step S31.

  FIG. 17 is a diagram illustrating a display example in which a captured image of a camera and a plurality of different captured images of the same camera are divided into four. The client PC 10 receives the current captured video (live video) distributed (transmitted) from one camera (for example, the fixed camera CAM 14), and further, a plurality of different recordings distributed from one recorder (for example, the recorder REC1). Assume that video is received. The plurality of different recorded videos are recorded videos captured in the past by a camera (for example, the fixed camera CAM 14) that distributes the current captured video (live video). When it is set on the camera setting screen WD1 (see FIG. 6) that the mask process (MOR process) is performed in this camera, the processor 11 causes the mask processing unit 114 to follow the processes shown in FIGS. Edited video MV6, edited video MV7, MV8, and MV9 after mask processing in which a person in each captured video and recorded video is changed (replaced) to a mask image are generated. In the video output control unit 115, the processor 11 can display on the monitor DP a video display screen WD7 that is obtained by dividing the edited video MV6, edited video MV7, MV8, and MV9 generated by the mask processing unit 114 into four. Yes (see FIG. 17).

  In the video display screen WD7 of FIG. 17, the display area of four videos (specifically, the edited video MV6, the edited video MV7, MV8, and MV9) out of the entire display area 4DV2 of the edited video and the edited video. It is. The edited recording videos MV7, MV8, and MV9 are, for example, those having the oldest or newest shooting date in the order of the edited recording video MV7, the edited recording video MV8, and the edited recording video MV9. As a result, the client PC 10 can divide and display on the monitor DP the edited video and the edited recorded video obtained by masking (MOR processing) the current captured video captured by the same camera and the recorded video captured in the past. Therefore, it is possible to effectively support the tracking of the user's person and the grasping of the action.

  Next, a second operation procedure of split display of the client PC 10 of this embodiment will be described with reference to FIG. FIG. 18 is a flowchart illustrating in detail an example of an operation procedure performed in the client PC 10 when displaying a captured image of a camera and a recorded image from a recorder in the second embodiment. The process illustrated in FIG. 18 is performed in the client PC 10 for each camera that distributes the current captured video (live video) to the client PC 10 or for each recorder that distributes past recorded video captured by the camera.

  Further, when the process shown in FIG. 18 and the process shown in FIG. 16 are the same, the same step number as the step number of the process shown in FIG. explain.

  In FIG. 18, the mask processing setting unit 113 of the processor 11 determines whether the client PC 10 has acquired the captured video from the camera that is the target of split display or the recorded video from the recorder (S31A). When it is determined that the captured video from the camera or the recorded video from the recorder is not acquired (S31A, NO), the mask processing setting unit 113 acquires the captured video from the camera or the recorded video from the recorder. Wait (S32). The client PC 10 can receive and acquire the corresponding recorded video distributed from the recorder when requesting the corresponding recorded video from the recorder by the user's designation of the imaging date and time. The same applies to each embodiment. It is.

  When the mask processing setting unit 113 determines that the captured video from the camera that is the target of split display or the recorded video from the recorder is acquired (S31A, YES), identification of the camera included in the captured video or the recorded video Based on the information, the camera setting information stored in the storage 15 or the memory 16 is read with reference to the corresponding camera record (S33A). The mask process setting unit 113 determines whether or not the setting to execute the mask process (MOR process) is performed on the captured video or the recorded video in the read record (S34A).

  When it is determined that the mask record (MOR process) is set to be executed in the read record (S34A, YES), the mask process setting unit 113 notifies the mask process unit 114 to that effect. In response to the notification from the mask processing setting unit 113, the mask processing unit 114 performs the mask processing (MOR processing) illustrated in FIGS. 11 and 12 on the captured video from the corresponding camera or the recorded video from the recorder. Then, the edited video or edited video recorded after the mask process is generated (S35A). The video output control unit 115 displays the edited video or the edited recorded video generated in step S35 in a corresponding display area in the video display screen of the monitor DP (for example, the display area of the edited video MV6 or the edited recorded video MV7 in FIG. 17). (S35A).

  On the other hand, when it is determined that the mask record (MOR process) is not set to be executed in the read record (S34A, NO), the mask process setting unit 113 displays the original captured video or recorded video. Is notified to the video output control unit 115 (S36A). In response to the notification from the mask processing setting unit 113, the video output control unit 115 displays the original captured video or recorded video received by the client PC 10 from the camera in a corresponding display area (for example, FIG. 17) in the video display screen of the monitor DP. It is displayed in the display area of the edited video MV6 or edited video MV7 (S36A).

  As described above, the surveillance camera system 100 according to the second embodiment includes a plurality of cameras (for example, the omnidirectional cameras CAM11,...) That capture an imaging area and a moving body that appears in the captured images of the imaging areas captured by the individual cameras. A client PC 10 that holds setting information for execution or non-execution of mask processing for changing at least a part of (for example, a person) into a mask image. In addition, the client PC 10 uses an individual edited video obtained by performing mask processing on individual captured images captured by a plurality of cameras set to execute mask processing, or a single camera set to execute mask processing. The edited video obtained by masking the individual captured video and the individual captured video captured by at least one camera set not to execute the mask processing are divided and displayed on the monitor DP.

  Similarly to the first embodiment, the client PC 10 according to the second embodiment can also perform mask processing that replaces at least a part or all of the moving person appearing in the captured image with a mask image. As a result, when the client PC 10 divides and displays a plurality of different captured images on the monitor DP, the masking process is executed or not performed for each camera as a distribution source (transmission source) of each captured image received by the client PC 10. Since execution settings can be easily set, user convenience can be improved. Accordingly, the client PC 10 can change the setting for performing mask processing for each captured image to be divided and displayed so as to adapt to the circumstances of the camera installation location. It can contribute to business and high-precision marketing analysis.

  The surveillance camera system 100 according to the second embodiment further includes a recorder (for example, REC1) that stores a captured image of an imaging area captured by each camera in association with an imaging date and time as a recorded video. The client PC 10 captures an edited image obtained by masking the captured image captured by the camera set to execute the mask process and an image captured at different past imaging dates and times by the camera set to execute the mask process. Individual edited and recorded videos obtained by masking a plurality of recorded videos are divided and displayed on the monitor DP.

  As a result, the client PC 10 can divide and display on the monitor DP the edited video and the edited recorded video obtained by masking (MOR processing) the current captured video captured by the same camera and the recorded video captured in the past. Therefore, it is possible to effectively support, for example, tracking of a person appearing in the vicinity of the installation location of the camera or grasping an action in monitoring work or marketing analysis.

(Background to the contents of the third embodiment)
According to Patent Document 1, since a mask image is generated for each position information and person attribute (for example, gender, age) in a moving image of a person detected in a monitoring area, privacy is ensured by the mask image. The behavior of the person in the store can be grasped above.

  However, in Patent Document 1, it is a camera that changes the image area of a person detected in a moving image to a mask image, and the mask is set by a receiving device (for example, a client PC used by a user) that receives the moving image from the camera. It is not taken into account that processing is performed.

  When the mask processing cannot be performed in the client PC, the mask processing cannot be performed on a captured image captured by a camera without a function for executing the mask processing (hereinafter referred to as “mask function”). It is difficult to add a mask function to an existing camera without a mask function. For this reason, it was difficult to improve monitoring efficiency and marketing analysis accuracy.

  Therefore, in the third embodiment, mask processing is appropriately performed on a captured image captured by a camera that does not have a mask function without adding a mask function separately to a camera that does not have a mask function. An example of a monitoring system and a video display method for improving monitoring efficiency and marketing analysis accuracy will be described.

(Third embodiment)
The monitoring camera system 100 described in the first embodiment will be described as an example of a monitoring system according to the third embodiment. Since the internal configuration of the surveillance camera system 100 of the third embodiment is the same as that of the surveillance camera system 100 of the first embodiment, description of the same contents is simplified or omitted, and different contents are described.

  In the third and fourth embodiments, in the surveillance camera system 100 shown in FIG. 1, at least the fixed camera CAM 14 has the mask function described above. That is, the fixed camera CAM 14 can execute the mask process (MOR process) shown in FIGS. 11 and 12 on the captured image captured by itself. In other words, in each of the third and fourth embodiments, the processor C11 of the fixed camera CAM 14 includes the image input unit 111, the background image processing unit 112, and the mask processing setting unit 113 in the processor 11 of the client PC 10 illustrated in FIG. In addition, it is assumed that the same components as the mask processing unit 114 are included.

  Note that the fixed camera CAM 14 performs a mask process equivalent to the mask process in the client PC 10 (that is, the mask process performed in the region excluding the non-target region for the mask process based on the user operation described in the first embodiment). Even if it cannot be performed, for example, the mask processing described in Patent Document 1 may be performed. Further, in each of the third and fourth embodiments, the mask camera is described as one unit. However, the present invention is not limited to only one unit, and a plurality of cameras may operate as a mask camera.

  In each of the third and fourth embodiments, a camera having a mask function like the fixed camera CAM14 is referred to as a “mask camera” for convenience, and a mask function like other cameras excluding the fixed camera CAM14. For convenience, a camera that does not have a camera is called a “normal camera”.

  Next, an operation procedure related to setting of mask processing and display of captured video of the client PC 10 according to the present embodiment will be described with reference to FIGS. FIG. 19 is a sequence diagram illustrating in detail an example of an operation procedure for setting a mask process (MOR process) of the client PC 10 for the mask camera and the normal camera and displaying a captured image in the third embodiment. FIG. 20 is a diagram illustrating an example of a camera setting screen in the third and fourth embodiments. FIG. 21 is a diagram illustrating an example of four-division display of an edited video that is displayed on the monitor DP of the client PC 10 and in which the captured video is masked.

  In FIG. 19, when there is an operation for starting operation of the client PC 10 by a user operation (S41), the client PC 10 executes mask processing in the mask camera by a user operation on the camera setting screen WD1A (see FIG. 20), for example ( When ON or non-execution (OFF) is designated, the camera setting information in which the mask process is designated to be executed or non-execution in the mask camera and an instruction to set the camera setting information are transmitted to the mask camera ( S42).

  Here, the camera setting screen WD1A shown in FIG. 20 will be described. Descriptions of items overlapping or similar to the camera setting screen WD1 shown in FIG. 6 are simplified or omitted, and different contents will be described.

  In FIG. 20, a part of the camera setting screen WD1A is extracted and shown. The mask processing setting unit 113 displays the camera setting screen WD1A on the monitor DP, for example, by a user operation. In this camera setting screen WD1A, at least an IP address, for each individual camera of, for example, a maximum of 64 cameras (that is, an omnidirectional camera, a PTZ camera, and a fixed camera) is operated by the user using the operation unit 13. The mask process (MOR process), the setting of the mask process (MOR process), the initial learning period, and the normal learning period can be set in association with each other. Further, by switching the four tabs (camera 1-16, camera 17-32, camera 33-48, camera 49-64) provided on the camera setting screen WD1A, for the corresponding camera in units of 16 cameras, Information on at least an IP address, mask processing (MOR processing), setting of mask processing (MOR processing), initial learning period, and normal learning period can be set.

  In the mask process (MOR process) column CLM2A, the mask process is not executed (OFF), the mask process (CAM-MOR) is executed by the mask camera, or the mask process (PC-MOR) is executed by the client PC 10 (ON). ) Is selected (see frame WK2A). In other words, execution (ON) of the mask process on the camera setting screen WD1 in FIG. 6 is executed on the camera setting screen WD1 in FIG. 20 by executing the mask process (CAM-MOR) on the mask camera or on the client PC 10. This corresponds to execution (ON) of processing (PC-MOR).

  In the column CLM3 for setting the mask process (MOR process), when the mask process (CAM-MOR) in the mask camera or the mask process (PC-MOR) in the client PC 10 is selected in the column CLM2A, the mask process is performed. A setting button (see frame WK1) for displaying an area setting screen WD2 for setting the non-target area is displayed. However, the setting button of the column CLM3 is not displayed so as to be able to be pressed for a camera for which non-execution of mask processing (OFF) is selected in the column CLM2A (for example, the fifth column, the eighth column from the page of FIG. 20 in the column CLM3, (See the dotted lines on the 10th and 15th stages).

  In FIG. 19, the mask camera follows the instruction transmitted from the client PC 10 in step S42, and executes (ON) or disables mask processing specified by the camera setting information based on the camera setting information transmitted together with the instruction. Execution (OFF) is set, and a response to the effect is returned to the client PC 10 (S43).

  The client PC 10 instructs the normal camera and the mask camera to start imaging and distribute the captured video (S44). The normal camera starts imaging in accordance with an instruction from the client PC 10 (S45), and thereafter continues to distribute the captured video obtained by imaging (S47). The client PC 10 receives a captured image distributed from a normal camera.

  The mask camera starts imaging in accordance with an instruction from the client PC 10 (S46), and further determines whether the mask process set in step S43 is executed (ON) or not executed (OFF). That is, when the mask camera is set to execute (ON) in step S43, the mask camera displays the edited video obtained by performing mask processing (see FIGS. 11 and 12) on the captured video obtained by imaging. Distribution is performed toward the PC 10 and the distribution is continued (S48). On the other hand, if the mask process is set to non-execution (OFF) in step S43, the mask camera distributes the captured image obtained by the imaging as it is to the client PC 10, and continues the distribution (S48).

  The client PC 10 receives the received image when it is set in the camera setting information corresponding to the normal camera or the mask camera that the mask camera executes the mask process or the mask process itself is not executed. The video is displayed on the monitor DP as it is (S49, see FIG. 21).

  On the other hand, if the client PC 10 is set to execute the mask process in the camera setting information corresponding to the normal camera or the mask camera, the client PC 10 performs the mask process (see FIG. 11 and FIG. 12), and the edited video obtained by the mask process is displayed on the monitor DP (S49, see FIG. 21).

  In the video display screen WD7A of FIG. 21, the display areas of four videos (specifically, the edited videos MV6A, MV7A, MV8A, and MV9A) are equal in the entire display area of the edited video (see FIG. 17). The edited videos MV6A and MV9A are, for example, the edited videos obtained as a result of the masked image captured by the mask camera being displayed on the client PC 10 as they are. The edited video MV7A is, for example, displayed on the edited video obtained as a result of the captured video captured by the mask camera being masked by the client PC 10 without being masked by the mask camera. The edited video MV8A is displayed, for example, as an edited video obtained as a result of masking the captured video captured by the normal camera in the client PC 10. As a result, the client PC 10 can select the mask camera or the client PC 10 itself according to the execution (ON) or non-execution (OFF) setting of the mask process corresponding to the normal camera having no mask function or the mask camera having the mask function. Since the edited video that has undergone the mask processing can be divided and displayed on the monitor DP, it is possible to contribute to the efficiency of monitoring work and marketing analysis while ensuring the privacy of the person in the captured video captured by each camera.

  As described above, the surveillance camera system 100 according to the third embodiment captures an imaging area, and executes a mask process for changing at least a part of a moving body (for example, a person) appearing in a captured image of the imaging area to a mask image. A normal camera (for example, an omnidirectional camera CAM11) that cannot be performed, and a client PC 10 that holds setting information on whether or not to perform mask processing on a captured image. When the client PC 10 is set to execute mask processing on the captured video, the client PC 10 performs mask processing on the captured video and displays the edited video obtained by the mask processing on the monitor DP. On the other hand, the client PC 10 displays the captured image as it is on the monitor DP when the non-execution of the mask processing in the client PC 10 for the captured image is set.

  Similarly to the first embodiment, the client PC 10 of the third embodiment can also perform mask processing that replaces at least a part or all of the moving person appearing in the captured image with a mask image. As a result, the client PC 10 appropriately performs mask processing on the captured image captured by the normal camera not having the mask function without separately adding a mask function to the normal camera not having the mask function. , Monitoring efficiency and marketing analysis accuracy can be improved.

  Further, the surveillance camera system 100 according to the third embodiment can execute a mask process for capturing an imaging area and changing at least a part of a moving body (for example, a person) appearing in a captured image of the imaging area into a mask image. A mask camera (for example, a fixed camera CAM14). The client PC 10 holds setting information indicating whether or not to execute mask processing in the mask camera, and when the execution of mask processing in the mask camera is set, the edited video based on the mask processing in the mask camera is directly displayed on the monitor DP. indicate.

  As a result, when the mask processing is performed by setting the execution of the mask processing for the mask camera that already has the mask function, the client PC 10 receives the edited video after the mask processing generated by the mask camera and displays it as it is. As a result, the execution of mask processing in the client PC 10 can be omitted, the processing load can be reduced, and load distribution can be achieved.

  Further, the surveillance camera system 100 according to the third embodiment can execute a mask process for capturing an imaging area and changing at least a part of a moving body (for example, a person) appearing in a captured image of the imaging area into a mask image. A mask camera (for example, a fixed camera CAM14). The client PC 10 holds setting information on whether or not to execute mask processing in the mask camera, and when the mask processing in the mask camera is set not to be executed, the captured image is masked and obtained by this mask processing. The edited video is displayed on the monitor DP.

  Thereby, the client PC 10 has a mask function superior to the mask processing in the mask camera, for example (for example, excluding the non-maskable area for the mask processing based on the user operation described in the first embodiment) When the mask processing performed in the area can be performed), the edited video after the mask processing according to the user's preference can be obtained rather than the edited video obtained by the mask processing performed by the mask camera, and user convenience can be improved. .

(Background to the contents of the fourth embodiment)
According to Patent Document 1, since a mask image is generated for each position information and person attribute (for example, gender, age) in a moving image of a person detected in a monitoring area, privacy is ensured by the mask image. The behavior of the person in the store can be grasped above.

  However, in Patent Document 1, it is a camera that changes the image area of a person detected in a moving image to a mask image, and the mask is set by a receiving device (for example, a client PC used by a user) that receives the moving image from the camera. It is not taken into account that processing is performed.

  That is, in Patent Document 1, since the mask process can be performed only by the camera, the live image (captured image) currently being captured can be masked by the camera, but the past captured image (which is already stored in the recorder) Hereinafter, the mask process could not be performed on “recorded video”.

  For this reason, when a recorded video of an event is verified afterwards, for example, an authorized person (manager, etc.) and an unauthorized person (general user) access to protect the privacy of the subject person. It was necessary to prepare two types of recorded video. For example, an authorized person can browse a recorded video in which a person in the recorded video is not masked, and a general user browses a recorded video in which a person in the recorded video is masked. Under such an operation, the recording capacity of the recorder is inevitably increased in order to record the captured image of the same scene. Particularly in recent years, the image quality of captured images has been improved (for example, 4K), and the amount of captured image data stored in the recorder is enormous.

  In other words, in order to prepare the two types of recorded video described above, one camera is not sufficient, for example, two cameras (specifically, a function for executing mask processing (hereinafter referred to as “mask function”). ) And a camera that does not have a mask function. For this reason, since the number of parts increases in system construction, the cost increase cannot be avoided, and the system configuration is complicated. Further, if the above-described client PC cannot perform mask processing, the present invention is not limited to the case where two types of recorded video are required only by the presence / absence of authority (whether or not the authority is the same as that of the above-mentioned manager and general user). Depending on the content and type of the recorded video, two types of recorded video, that is, a recorded video that has been masked and a recorded video that has not been masked, may be required.

  Therefore, in the fourth embodiment, when browsing a recorded video of an event that has been captured by the camera and is already stored in the recorder in the past, a mask process is performed or masked using one data of the recorded video. An example of a monitoring system and an image display method for easily selecting whether to perform processing and improving the monitoring efficiency and marketing analysis accuracy will be described.

(Fourth embodiment)
The monitoring camera system 100 described in the third embodiment will be described as an example of a monitoring system according to the fourth embodiment. Since the internal configuration of the monitoring camera system 100 of the fourth embodiment is the same as that of the monitoring camera system 100 of the third embodiment, the description of the same contents will be simplified or omitted, and different contents will be described.

  In the fourth embodiment, in the surveillance camera system 100 shown in FIG. 1, each recorder to which at least one camera is connected (that is, the recorders REC1, REC2, and REC3) has at least one connected to each recorder. Paired (in other words, tied) with two cameras. More specifically, an omnidirectional camera CAM11, PTZ cameras CAM12, CAM13, and a fixed camera CAM14 are paired (linked) to the recorder REC1. A PTZ camera CAM21 and a fixed camera CAM22 are paired (linked) to the recorder REC2. The recorder REC3 is paired (linked) with an omnidirectional camera CAM31, a PTZ camera CAM32, a CAM33, and a fixed camera CAM34. The pairing relationship between the at least one camera and the recorder is not limited to the fourth embodiment, and may be similarly applied to the first, second, and third embodiments described above.

  FIG. 22 is a diagram showing an example of the recorder setting screen WD8. The processor 11 of the client PC 10 displays the recorder setting screen WD8 on the monitor DP in response to a user operation for displaying the recorder setting screen WD8.

  On the recorder setting screen WD8, the recorder is set by a user operation. For example, the IP address of the recorder, the HTTP (Hypertext Transfer Protocol) port number, the name, the FTP (File Transfer Protocol) port number, the time zone, and the setup of at least one paired camera are performed. FIG. 22 exemplarily shows respective devices, models, and IP addresses for four recorders, for example.

  In the recorder setting screen WD8 displayed on the monitor DP, a record (frame WK6) for specifying one of the recorders “Device mng.” (Device manager, see frame WK4), “Recorder” (see frame WK5), by user operation. When the selection is made by pressing in the order of “reference”, the processor 11 switches the display so that the Edit button (see the frame WK7) that cannot be pressed before the selection can be pressed.

  When the Edit button (see frame WK7) is pressed by a user operation, the processor 11 sets a camera setting screen (see FIG. 20) related to at least one camera paired with the recorder of the selected record (see frame WK6). ) Is displayed on the monitor DP. Items that can be set on the camera setting screen (see FIG. 20) are the same as those described in the third embodiment (see FIG. 20), and thus detailed description thereof is omitted.

  Next, an operation procedure related to the mask processing setting of the client PC 10 of the present embodiment will be described with reference to FIGS. 23 (A) and 23 (B). FIG. 23A is a sequence diagram illustrating in detail a first example of an operation procedure for setting mask processing (MOR processing) in the fourth embodiment. FIG. 23B is a sequence diagram illustrating in detail a second example of the operation procedure for setting mask processing (MOR processing) in the fourth embodiment.

  In FIG. 23A, when there is an operation for starting device setting of the client PC 10 by a user operation (S51), the client PC 10 is based on, for example, pressing the Edit button (see frame WK7) on the recorder setting screen WD8. The displayed camera setting screen (see FIG. 20) is displayed on the monitor DP. The client PC 10 executes (ON) or does not perform mask processing on the mask camera (that is, the fixed camera CAM 14 capable of executing the mask function) paired with the recorder (for example, the recorder REC1) by a user operation on the camera setting screen. The recorder is instructed to execute (OFF) (S52).

  The recorder (for example, the recorder REC1) executes (ON) or does not execute the mask processing for the mask camera (that is, the fixed camera CAM14) paired with itself (for example, the recorder REC1) in accordance with the instruction from the client PC 10 in step S52. The mask camera is instructed to set (OFF) (S53). The mask camera sets execution (ON) or non-execution (OFF) of the mask process according to the instruction from the recorder in step S53 (S54), and returns a response to the effect that the setting is completed to the recorder. When the recorder receives a response indicating the completion of setting from the mask camera, the recorder similarly returns a response indicating the completion of setting to the client PC 10.

  Thereby, for example, when the client PC 10 and the mask camera do not exist in the same base or store, the client PC 10 cannot set the mask processing directly for the mask camera, but the client PC 10 and the mask camera By interposing a recorder that mediates the data transmission / reception, the mask processing from the client PC 10 to the mask camera can be set smoothly.

  In FIG. 23B, when there is an operation for starting device setting of the client PC 10 by a user operation (S51), the client PC 10 is based on, for example, pressing the Edit button (see frame WK7) on the recorder setting screen WD8. The displayed camera setting screen (see FIG. 20) is displayed on the monitor DP. The client PC 10 executes (ON) or does not perform mask processing on the mask camera (that is, the fixed camera CAM 14 capable of executing the mask function) paired with the recorder (for example, the recorder REC1) by a user operation on the camera setting screen. The execution (OFF) setting is instructed directly to the mask camera (S52A).

  The mask camera sets execution (ON) or non-execution (OFF) of mask processing in accordance with the instruction from the client PC 10 in step S52A (S54A), and directly returns a response indicating that the setting has been completed to the client PC 10.

  Thereby, for example, when the client PC 10 and the mask camera exist in the same base or store, the client PC 10 smoothly and easily sets the mask processing directly on the mask camera without going through the recorder. be able to.

  Next, an operation procedure related to display of a captured image after setting of mask processing of the client PC 10 of the present embodiment will be described with reference to FIGS. FIG. 24 is a sequence diagram illustrating in detail a first example of an operation procedure regarding acquisition and display of a captured image in a client PC after the start of operation according to the fourth embodiment. FIG. 25 is a sequence diagram illustrating in detail a second example of an operation procedure regarding acquisition and display of a captured image in a client PC after the start of operation according to the fourth embodiment. FIG. 26 is a flowchart for explaining in detail an example of the operation procedure of the display of the client PC in step S69 of FIGS.

  In the description of FIG. 25, when the process shown in FIG. 25 and the process shown in FIG. 24 are the same, the same step number is assigned and the description is simplified or omitted, and different contents will be described.

  In FIG. 24, when there is an operation for starting the operation of the client PC 10 by a user operation (S61), the client PC 10 instructs the recorder to start distributing video (that is, to record (S62)).

  In accordance with the instruction of the client PC 10 in step S62, the recorder starts distributing the captured video or edited video, or instructs the mask camera and the normal camera to start distributing the captured video or edited video (S63). The mask camera starts imaging in accordance with the instruction of the recorder in step S63 (S64). Similarly, the normal camera starts imaging according to the instruction of the recorder in step S63 (S65).

  Since the normal camera cannot perform the masking process, after starting imaging, distribution of the captured video obtained by imaging to the recorder is continued (S66). When the recorder receives the captured video (live video) distributed from the normal camera in step S66, the recorder stores it in the storage R14 (S68).

  After the start of imaging, the mask camera masks the captured image obtained by the start of imaging in step S64 when the execution (ON) of the mask process is set in FIG. 23 (A) or FIG. 23 (B). Distribution of the edited video obtained by the processing (for example, see FIGS. 11 and 12) to the recorder is continued (S67). On the other hand, the mask camera, after starting imaging, when the non-execution (OFF) of the mask processing is set in FIG. 23A or FIG. 23B, imaging obtained by starting imaging in step S64. Distribution of the video to the recorder is continued (S67). Note that when the recorder receives the captured video (live video) or the edited video distributed from the mask camera in step S67, the recorder stores it in the storage R14 (S68).

  In step S62, for example, when a user operation for distributing a recorded video stored in the recorder in the past is performed, an instruction to start video distribution is given by the user operation (in other words, an imaging date and time desired by the user). And recording date and time) are included. In this case, the recorder reads from the storage R14 the recorded video imaged at the imaging date and time transmitted from the client PC 10 and distributes it to the client PC 10 (S68).

  In step S62, for example, when a user operation for distributing a captured image captured by the camera is performed, the captured image (live image) currently captured by the camera is distributed in accordance with an instruction to start distributing the image. The In this case, the recorder distributes the captured video distributed from the normal camera in step S66 and the captured video or edited video distributed from the mask camera in step S67 to the client PC 10, and continues the distribution (S68).

  The client PC 10 executes the mask processing set for each type of camera paired (linked) with the captured video (live video), recorded video, or edited video distributed from the recorder in step S68 ( The video is displayed on the monitor DP according to the setting of ON or non-execution (OFF) (S69). Details of step S69 will be described later with reference to FIG.

  In FIG. 25, since the normal camera cannot perform the masking process, after starting the imaging, the distribution of the captured video obtained by the imaging to the recorder and the client PC 10 is continued (S66A). When the recorder receives the captured video (live video) distributed from the normal camera in step S66A, the recorder stores it in the storage R14 (S68A).

  After the start of imaging, the mask camera masks the captured image obtained by the start of imaging in step S64 when the execution (ON) of the mask process is set in FIG. 23 (A) or FIG. 23 (B). The distribution of the edited video obtained by the processing (for example, see FIGS. 11 and 12) to the recorder and the client PC 10 is continued (S67A). On the other hand, the mask camera, after starting imaging, when the non-execution (OFF) of the mask processing is set in FIG. 23A or FIG. 23B, imaging obtained by starting imaging in step S64. The distribution of the video to the recorder and the client PC 10 is continued (S67A). When the recorder receives the captured video (live video) or edited video distributed from the mask camera in step S67A, the recorder stores it in the storage R14 (S68A).

  In step S62, for example, when a user operation for distributing a recorded video stored in the recorder in the past is performed, an instruction to start video distribution is given by the user operation (in other words, an imaging date and time desired by the user). And recording date and time) are included. In this case, the recorder reads out the recorded video imaged at the imaging date and time transmitted from the client PC 10 from the storage R14 and distributes it to the client PC 10 (S68A).

  The client PC 10 executes the mask processing set for each type of camera paired (linked) with the captured video (live video), recorded video, or edited video distributed from the recorder in step S68 ( The video is displayed on the monitor DP according to the setting of ON or non-execution (OFF) (S69). Details of step S69 will be described later with reference to FIG.

  In FIG. 26, the client PC 10 receives and acquires the video distributed from any of the recorder, the mask camera, and the normal camera in step S68 or step S68A (S71).

  The client PC 10 reads the camera setting information (see FIGS. 22 and 20) stored in the storage 15 or the memory 16, and the camera paired with the recorder of the video distribution source acquired in step S71 or the video of the video. Reference is made to the setting of execution or non-execution of mask processing for the distribution source camera (S72).

  If the client PC 10 determines that the mask processing non-execution (OFF) is set in the camera setting information, the video (specifically, the captured video or the recorded video) acquired in step S71 is directly monitored DP (S73).

  If the client PC 10 determines that the execution (ON) of the mask process in the corresponding camera is set in the camera setting information, the image acquired in step S71 (specifically, the mask process is performed by the mask camera). The edited video that has been applied is displayed on the monitor DP as it is (S73).

  If the client PC 10 determines that execution of mask processing in the client PC 10 is set for the corresponding camera in the camera setting information, the video (specifically, captured video or edited video) acquired in step S71. Is masked (FIGS. 11 and 12), and the edited video obtained by the masking is displayed on the monitor DP (S74). However, when the client PC 10 receives the edited video distributed from the recorder in step S71, if the edited video data includes information indicating that the mask processing has been performed by the mask camera, the client PC 10 May be preferentially displayed on the monitor DP without masking the received edited video without taking into account the contents of the current camera setting information. As a result, the client PC 10 can omit the execution of further mask processing on the edited video that has already been masked in another device (that is, the mask camera), and can reduce the processing load on the client PC 10. Degradation of the visibility of the edited video can be suppressed.

  As described above, the surveillance camera system 100 according to the fourth embodiment is associated with at least one camera (for example, the omnidirectional camera CAM11, the fixed camera CAM14,...) That images the imaging area, and captures the imaging area. Execution of mask processing for changing at least a part of a moving object (for example, a person) appearing in the captured image in the imaging area to a mask image, or a recorder (for example, the recorder REC1) that stores the video as a recorded video in association with the imaging date And a client PC 10 that holds non-executable setting information for each camera associated with the recorder. For example, when execution of mask processing is set in response to designation of an imaging date and time by a user operation, the client PC 10 displays an edited video image obtained by masking a recorded video imaged at the designated imaging date and time. Display on monitor DP. On the other hand, when the non-execution of the mask process is set, the client PC 10 displays the recorded video captured at the designated imaging date and time as it is on the monitor DP.

  Similarly to the first embodiment, the client PC 10 of the fourth embodiment can also perform mask processing that replaces at least a part or all of the moving person appearing in the captured image with a mask image. As a result, when the client PC 10 browses a recorded video of an event that has been captured by the camera in the past and is already stored in the recorder, the client PC 10 performs a mask process in the client PC 10 using one piece of the recorded video. It is possible to easily select whether or not to perform mask processing, and it is possible to improve monitoring efficiency and marketing analysis accuracy.

  In the surveillance camera system 100 of the fourth embodiment, at least one camera includes a normal camera (for example, an omnidirectional camera CAM11) that cannot perform mask processing. The client PC 10 displays the edited video obtained by masking the captured video on the monitor DP when the execution (ON) of the mask processing for the captured video captured by the normal camera is set. On the other hand, the client PC 10 displays the captured image as it is on the monitor DP when the non-execution (OFF) of the mask processing for the captured image captured by the normal camera is set.

  Thereby, when the client PC 10 receives the current captured video (live video) captured by the normal camera, depending on the setting of execution or non-execution of the mask processing on the captured image by the normal camera, Captured video or edited video that matches the user's preference can be displayed on the monitor DP, which can contribute to improving the accuracy of user monitoring operations and marketing analysis.

  In the surveillance camera system 100 of the fourth embodiment, at least one camera includes a mask camera (for example, a fixed camera CAM 14) capable of performing mask processing. When execution of mask processing in the mask camera is set for the captured image captured by the mask camera, the client PC 10 directly displays the edited image in which the captured image is masked by the mask camera on the monitor DP. On the other hand, when execution of mask processing in the client PC 10 is set for the captured image captured by the mask camera, the client PC 10 masks the captured image and displays the resulting edited image on the monitor DP. .

  As a result, when the mask process is performed by setting the execution of the mask process for the mask camera that already has the mask function, the client PC 10 receives the edited video after the mask process generated by the mask camera and displays it as it is. The execution of the mask process in the client PC 10 can be omitted, the processing load can be reduced, and the load can be distributed. Further, when the mask processing is performed by setting the execution of the mask processing in the client PC 10, the client PC 10 has a mask function superior to, for example, the mask processing in the mask camera (for example, the user described in the first embodiment, The edited video obtained by performing mask processing in a region excluding the non-target region for mask processing based on the operation) can be obtained, and the video suitable for the user's needs can be confirmed.

  Further, the client PC 10 displays the captured image as it is on the monitor DP when the non-execution of the mask process for the current captured image (live image) captured by the mask camera is set. This allows the user to check the current live video when it is necessary to view the current live video as it is due to, for example, event conditions or restrictions, and to check the video that meets the user's needs. Can do.

  While various embodiments have been described above with reference to the drawings, it goes without saying that the present invention is not limited to such examples. It is obvious for those skilled in the art that various changes or modifications can be conceived within the scope of the claims, and it is obvious that the present invention also applies to the combination of the above-described embodiments. It is understood that it belongs to the technical scope of

In the present invention, when a plurality of different types of captured images are divided and displayed on a monitor, the setting of execution or non-execution of mask processing for each captured image is easily set for each captured image to improve user convenience. It is useful as a monitoring system , a video display method, and a computer program .

10 Client PC
11 Processor 12 Communication Unit 13 Operation Unit 14 Monitor 15 Storage 16 Memory 17 Power Management Unit 100 Surveillance Camera System (Monitoring System)
111 Image Input Unit 112 Background Image Processing Unit 113 Mask Processing Setting Unit 114 Mask Processing Unit 115 Video Output Control Unit 161 ROM
162 RAM
CAM11, CAM31 Omnidirectional camera CAM12, CAM13, CAM21, CAM32, CAM33 PTZ camera CAM14, CAM22, CAM34 Fixed camera NW Network REC1, REC2, REC3, REC4 Recorder

Claims (4)

A plurality of cameras for imaging the imaging area;
Information processing apparatus for holding setting information on execution or non-execution of mask processing for changing at least a part of a moving body appearing in a captured image of the imaging area captured by each individual camera into a mask image for each camera And comprising
The information processing apparatus is an individual edited video obtained by performing mask processing on the individual captured video captured by a plurality of cameras set to execute the masking process on the information processing apparatus , or
At least one camera to the non-execution of the masking process and obtained editing video the mask processing by the mask at least one camera information processing apparatus individual captured image captured by the execution is set in the process is set The individual captured images captured by the
Monitoring system. The monitoring system according to claim 1,
A recorder that stores the captured video of the imaging area captured by each of the cameras in association with the imaging date and time as a recorded video;
The information processing apparatus of the past by the camera in which the image pickup image captured by the mask processing camera execution is set in the information processing apparatus executing the edited video obtained by masking the mask processing with is set Dividing and displaying on the monitor the individual edited and recorded videos obtained by masking the plurality of recorded videos captured at different imaging dates and times and stored in the recorder with the information processing device ;
Monitoring system. A video display method of displaying an image using an information processing apparatus for displaying a captured image of the imaging area which is more imaging a plurality of cameras for imaging the imaging area to the monitor,
The information processing apparatus sets execution information or non-execution setting information of a mask process for changing at least a part of a moving body appearing in a captured image of the imaging area captured by each camera to a mask image for each camera. Hold on
The information processing apparatus includes:
An individual edited video obtained by performing mask processing on the individual imaged images captured by a plurality of cameras set to execute the mask process by the information processing apparatus , or at least one set to execute the mask process An edited video obtained by masking each captured video imaged by a camera with the information processing apparatus and an individual captured video imaged by at least one camera in which non-execution of the mask processing is set, Split display on monitor,
Video display method. In an information processing apparatus, which is a computer, that displays on a monitor captured images of the imaging area captured by a plurality of cameras that capture the imaging area.
Holding setting information on execution or non-execution of mask processing for changing at least a part of a moving body appearing in a captured image of the imaging area captured by each of the cameras into a mask image;
Individual edited videos obtained by masking individual captured videos captured by a plurality of cameras set to execute the masking process by the information processing apparatus, or
An edited video obtained by performing mask processing on the individual captured video captured by the at least one camera set to execute the mask processing by the information processing apparatus and at least one camera set to not execute the mask processing The step of splitting and displaying individual captured images captured by the above-mentioned on the monitor,
Computer program.