JP2019027908A - Exterior surface material investigation system - Google Patents

Abstract

To investigate abnormality of the exterior surface material of a structure in a short time.SOLUTION: An exterior surface material investigation system processes an image imaged by an aircraft 10 so as to be displayed on a terminal device 30 and investigates a state of the exterior surface material of a structure. The aircraft 10 includes: a visible light image imaging means for imaging a visible light image; and a non-visible light image imaging means for imaging a non-visible light image by an electromagnetic wave or a sound wave other than the visible light. The terminal device 30 or the aircraft 10 includes: image processing means for executing the predetermined image processing for the visible light image and/or the non-visible light image; aircraft control means for controlling a flying state of the aircraft based on the image processing result of the image processing means; and abnormality detection means for detecting the abnormality of the exterior surface material based on the image processing result by the image processing means.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an outer surface material investigation system.

  Conventionally, as a method for inspecting abnormalities such as peeling of an outer surface material (for example, tile, siding, brick, surface concrete, etc.) of a structure, there is a technique disclosed in Patent Document 1, for example.

  In the technique disclosed in Patent Document 1, an infrared heater that irradiates infrared rays on the outer wall of a building and an infrared camera that visualizes a temperature difference between a normal portion and a peeled portion of the outer wall that is irradiated with infrared rays and heated are arranged. Then, the wall surface temperature pattern is created by computer simulation from the image obtained by the infrared camera, and the normal portion and the peeled portion are color-coded to identify the peeled portion.

JP 05-31745 A

  However, in the technique disclosed in Patent Document 1, since an infrared heater is used, the inspection target is limited to a range where infrared rays from the infrared heater reach. Therefore, in order to inspect a wide range, the inspection is performed. There is a problem that it takes time.

  An object of the present invention is to provide an outer surface material investigation system capable of inspecting abnormality of an outer surface material of a structure in a short time.

In order to solve the above-described problems, the present invention provides an external material inspection system that processes an image captured by a flying object and displays the processed image on a terminal device to investigate the state of the external surface material of the structure. A visible light image capturing unit that captures a light image; and a non-visible light image capturing unit that captures a non-visible light image by an electromagnetic wave or a sound wave in a region other than visible light; and the terminal device or the flying object includes: Image processing means for executing predetermined image processing on the visible light image and / or the non-visible light image, and flying object control for controlling a flight state of the flying object based on an image processing result by the image processing means And an abnormality detecting means for detecting an abnormality of the outer surface material based on an image processing result by the image processing means.
According to such a structure, it becomes possible to test | inspect the abnormality of the outer surface material of a structure for a short time.

According to the present invention, the image processing means executes a process of detecting an image component corresponding to an element constituting the outer surface material, and the flying object control means applies the element detected by the image processing means. The distance from the structure of the flying object and / or the flight speed is adjusted based on the corresponding image component.
According to such a configuration, the distance and the flight speed can be set appropriately.

According to the present invention, the image processing means executes a process of detecting an image component corresponding to an element constituting the outer surface material, and the flying object control means applies the element detected by the image processing means. The elevation angle or depression angle of the visible light image capturing unit and the non-visible light image capturing unit is adjusted based on the corresponding image component.
According to such a configuration, the elevation angle or depression angle can be set appropriately.

Further, according to the present invention, the invisible light image capturing unit detects infrared rays emitted from a subject, captures a thermal image indicating the temperature of each part of the subject, and the image processing unit captures the visible light image. A spatial frequency component corresponding to an element constituting the outer surface material is specified based on the visible light image captured by the means, and the thermal image in the thermal image is based on a relative relationship between the visible light image and the thermal image. A process of specifying a spatial frequency component of an element and extracting a band component corresponding to the specified spatial frequency is performed.
According to such a configuration, it is possible to reliably extract image components corresponding to the elements constituting the outer surface material.

In the present invention, the abnormality detection unit refers to the band component extracted by the image processing unit, and the temperature of the part corresponding to the element constituting the outer surface material is different from the adjacent part. It is determined to be abnormal.
According to such a configuration, it is possible to reliably detect an abnormal location.

Further, in the present invention, when an abnormality is detected by the abnormality detection unit, the flying object control unit sets the flying object to a hovering state, and the abnormality detection unit includes a plurality of the images captured during hovering. It is characterized by determining the presence or absence of abnormality based on an invisible light image.
According to such a configuration, occurrence of erroneous detection can be reduced.

Further, the present invention is characterized in that the flying object control means receives designation of the structure to be investigated on a map and sets a flight path based on the designated structure.
According to such a configuration, the flight path can be easily set.

Further, the present invention is characterized in that the flying object control means performs control so as to image a predetermined outer surface of the structure by alternately repeating vertical flight and horizontal flight.
According to such a configuration, it is possible to simplify the control of the flying object and to detect the abnormal part without omission.

Moreover, the present invention is characterized in that the flying object control means controls the outer surface of the structure to be photographed in the order of east, south, west, and north.
According to such a configuration, the outer surface can be inspected in the order in which the temperature rise / temperature drop is likely to occur.

Further, according to the present invention, the terminal device includes a display unit that displays an image, and the display unit displays the visible light image and the non-visible light image side by side, or the visible light image. The invisible light image is superimposed and displayed on the screen.
According to such a configuration, the operator can detect an abnormal location while visually confirming it.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to provide the outer surface material investigation system which can test | inspect the abnormality of the outer surface material of a structure for a short time.

It is a figure which shows the structural example of the outer surface material investigation system which concerns on embodiment of this invention. It is a figure which shows the detailed structural example of the flying body shown in FIG. It is a figure which shows the detailed structural example of the transmitter shown in FIG. It is a figure which shows the detailed structural example of the terminal device shown in FIG. It is a figure which shows the detailed structural example of the server shown in FIG. It is a figure which shows the relationship between the abnormality of an outer surface material, and temperature. It is a figure which shows an example of the flight path | route of a flying body. It is an example of the structure displayed on the map. This is an example in which a visible light image and a thermal image are displayed side by side. This is an example in which a visible light image and a thermal image are superimposed and displayed. This is an example in which a visible light image and a thermal image are displayed side by side. This is an example in which a visible light image and a thermal image are superimposed and displayed. It is a flowchart explaining an example of the process performed when inspecting an outer surface material with a flying body. It is a flowchart explaining an example of the initial adjustment process shown in FIG. It is a flowchart explaining an example of the target component extraction process shown in FIG. It is a flowchart explaining an example of the imaging process shown in FIG. It is a flowchart explaining an example of the adjustment process in flight shown in FIG. It is a flowchart explaining an example of the abnormality detection process shown in FIG. 17 is a flowchart for explaining an example of image display / storage processing shown in FIG.

  Next, an embodiment of the present invention will be described.

(A) Description of Configuration of Embodiment of the Present Invention FIG. 1 is a diagram showing a configuration example of an outer surface material investigation system according to an embodiment of the present invention. As shown in FIG. 1, the outer surface material investigation system 1 includes an air vehicle 10, a transmitter 20, a terminal device 30, a base station 40, a network 50, and a server 60. To do.

  The flying body 10 is an unmanned aircraft UAV (Unmanned Air Vehicle), and is configured by, for example, a drone that can be hovered (stopped in the air) by four or more propellers. In addition, you may make it comprise with a small helicopter, a balloon, etc.

  The transmitter 20 is operated by a pilot, transmits a control signal to the flying object 10, receives an image captured by the flying object 10, and supplies the image to the terminal device 30.

  The terminal device 30 is configured by a smartphone, a tablet terminal, or the like, connected to the transmitter 20 by a connection cable or the like, and exchanges information with the transmitter 20. The terminal device 30 has an LCD (Liquid Crystal Display) touch panel, and controls the flying object 10 in response to the operation when the operator operates the LCD touch panel. In addition, the terminal device 30 displays an image transmitted from the flying object 10.

  The base station 40 transmits / receives information to / from the terminal device 30 by radio waves, and transmits / receives information to / from the server 60 via the network 50.

  The network 50 is configured by the Internet, for example, and exchanges information between the base station 40 and the server 60 based on a predetermined protocol.

  The server 60 stores information such as images transmitted from the terminal device 30 via the base station 40 and the network 50.

  FIG. 2 shows a configuration example of the flying object 10. In the example of FIG. 2, the flying object 10 includes an infrared light imaging device 11, a visible light imaging device 12, a gimbal 13, an image processing unit 14, a transmission / reception unit 15, an antenna 15 a, a control unit 16, and a GPS (Global Positioning System) 17. , Sensor 18 and motors 19-1 to 19-4.

  Here, the infrared image pickup device 11 detects the temperature of each part of the subject from the infrared rays radiated from the subject using the fact that the radiance of the black body is proportional to the fourth power of the absolute temperature of the black body, Output as an image. Each pixel constituting the thermal image has information indicating the temperature of each part of the subject.

  The visible light imaging element 12 separates the visible light emitted from the subject into three primary colors, photoelectrically converts the light intensity of each color into a corresponding charge, and outputs it as a visible light image. Since the infrared light image sensor 11 and the visible light image sensor 12 have the same angle of view, a visible light image of the same region of the subject and a thermal image corresponding to the visible light image can be obtained.

  The gimbal 13 controls the inclination of the fixing member to which both the infrared light imaging element 11 and the visible light imaging element 12 are fixed by electronic control, so that the infrared light imaging element can be used even when the flying object 10 is swung. 11 and the visible light image sensor 12 are controlled so as not to change. The gimbal 13 is configured to be able to control the vertical tilt of the infrared light image sensor 11 and the visible light image sensor 12. Thereby, the infrared light image sensor 11 and the visible light image sensor 12 can stably image the same region of the subject.

  The image processing unit 14 performs, for example, compression processing on the images output from the infrared light imaging device 11 and the visible light imaging device 12 and supplies the images to the transmission / reception unit 15.

  The transmission / reception unit 15 transmits the image supplied from the image processing unit 14 to the transmitter 20 via the antenna 15a, and receives the control signal transmitted from the transmitter 20 via the antenna 15a. This is supplied to the control unit 16. The antenna 15a transmits and receives information to and from the transmitter 20 by radio waves.

  The control unit 16 includes a gimbal 13 and a motor 19 based on a control signal supplied from the transmission / reception unit 15, position information supplied from the GPS 17, information indicating a flight state of the flying object 10 supplied from the sensor 18, and the like. -1 to 19-4 are controlled.

  The GPS 17 receives signals having time information transmitted from a plurality of artificial satellites, detects information (latitude information and longitude information) related to its own position from these time differences, and supplies the information to the control unit 16.

  The sensor 18 includes, for example, a gyro sensor, a magnetic sensor, an atmospheric pressure sensor, an ultrasonic sensor, and a vision positioning sensor, and determines the attitude, altitude, distance from the obstacle, the shape of the obstacle, and the like of the flying object 10. Detect and notify the control unit 16.

  The motors 19-1 to 19-4 are configured by, for example, brushless motors, and propellers are attached to the motors 19-1 to 19-4. The control unit 16 can move in the target direction or change the direction of the flying object 10 by individually controlling the rotation speeds of the motors 19-1 to 19-4.

  FIG. 3 is a diagram illustrating a detailed configuration example of the transmitter 20 illustrated in FIG. 1. In the example of FIG. 3, the transmitter 20 includes a transmission / reception unit 21, an antenna 21 a, a control unit 22, joysticks 23 and 24, an operation unit 25, and an I / F (Interface) 26.

  Here, the transmitting / receiving unit 21 transmits / receives information to / from the flying object 10 via the antenna 21a by radio waves.

  The control unit 22 generates a control signal for controlling the flying object 10 based on information supplied from the joysticks 23 and 24, the I / F 26, and the operation unit 25, and the flying object via the transmission / reception unit 21. 10 to send. The control unit 22 receives an image captured by the flying object 10 via the transmission / reception unit 21 and supplies the image to the terminal device 30 via the I / F 26.

  The joysticks 23 and 24 are operated in the front-rear direction or the left-right direction by the operator's thumb or the like, generate a signal corresponding to the operation amount, and supply the signal to the controller 22.

  The operation unit 25 includes operation buttons, a jog dial, an LCD touch panel, and the like. The operation unit 25 generates a signal corresponding to an operation by the operator and supplies the signal to the control unit 22.

  The I / F 26 is configured by, for example, a USB (Universal Serial Bus) or the like, and exchanges information such as control information and image information between the control unit 22 and the terminal device 30.

  FIG. 4 shows a detailed configuration example of the terminal device 30 shown in FIG. As shown in FIG. 4, the terminal device 30 includes a CPU (Central Processing Unit) 31, a ROM (Read Only Memory) 32, a RAM (Random Access Memory) 33, a GPS 34, a GPU (Graphics Processing Unit) 35, an I / F 36, A bus 37 and an LCD touch panel 38 are provided.

  Here, the CPU 31 is a central processing unit that controls each unit in accordance with a program stored in the ROM 32.

  The ROM 32 is configured by a nonvolatile semiconductor storage device, and stores a basic program executed by the CPU 31 and the like.

  The RAM 33 is composed of a volatile semiconductor storage device, and temporarily stores programs to be executed by the CPU 31 and data being calculated.

  The GPS 34 receives signals from a plurality of artificial satellites, specifies its own position based on these time differences, and supplies it to the CPU 31.

  The GPU 35 executes a drawing process based on the drawing command supplied from the CPU 31 and supplies the obtained image data to the LCD touch panel 38 for display.

  The I / F 36 is configured by, for example, a USB or the like, and is connected to the I / F 26 illustrated in FIG. 3 by a connection cable, and exchanges information with the I / F 26.

  The bus 37 is a signal line group for mutually connecting the CPU 31, ROM 32, RAM 33, GPS 34, GPU 35, and I / F 36, and enabling exchange of information among them.

  The LCD touch panel 38 is configured by, for example, a touch sensor superimposed on a display unit of a liquid crystal panel. The LCD touch panel 38 displays an image or the like drawn by the GPU 35 on the LCD, and when the touch sensor is operated by the operator, transmits information indicating an operation location to the CPU 31 via the I / F 36.

  FIG. 5 is a diagram illustrating a configuration example of the server 60 illustrated in FIG. 1. 5 includes a CPU 61, a ROM 62, a RAM 63, an HDD 64, an I / F 65, and a bus 66.

  Here, the CPU 61 is a central processing unit that controls each unit in accordance with programs stored in the ROM 62 and the HDD 64.

  The ROM 62 is configured by a nonvolatile semiconductor memory device, and stores a basic program executed by the CPU 61 and the like.

  The RAM 63 is composed of a volatile semiconductor storage device, and temporarily stores programs to be executed by the CPU 61 and data being calculated.

  The HDD 64 is configured by, for example, a magnetic storage device and stores programs and data executed by the CPU 61.

  The I / F 65 performs, for example, protocol conversion processing when information is exchanged with the base station 40 via the network 50.

  The bus 66 is a signal line group for mutually connecting the CPU 61, the ROM 62, the RAM 63, the HDD 64, and the I / F 65 so that information can be exchanged therebetween.

(B) Description of Operation of the Embodiment of the Present Invention Next, the operation of the embodiment of the present invention will be described. The exterior material inspection system according to the present embodiment includes an exterior material (for example, tile, siding, brick, surface concrete, etc.) of a structure (for example, an apartment house, an office building, a detached house, a bridge, a pier, a tunnel, or a dam). ) To detect abnormalities such as peeling or breakage.

  More specifically, for example, when a plurality of tiles are arranged and bonded to the surface of a concrete body or the like, as shown in FIG. Think of it peeling off. In this case, when the temperature of the tile rises due to sunlight, the heat of the tile is conducted to the housing on the back surface. However, when the tiles are peeled off from the enclosure, an air layer exists between the tiles and the enclosure, and heat is not easily transferred to the enclosure. It will be in a higher state.

  FIG. 6B shows the temperature of the tile on the alternate long and short dash line shown in FIG. 6A when the temperature rises. As shown in FIG. 6B, the temperature of the tile that is peeled when the temperature rises is higher than that of the tile that is not peeled. FIG. 6C shows the temperature of the tile on the alternate long and short dash line shown in FIG. As shown in FIG. 6C, when the temperature drops (for example, when the sun goes down), the heat of the tile and the casing is radiated into the atmosphere through the tile. At this time, since the heat from the housing is less likely to be transferred due to the presence of the air layer, the peeled portion is at a temperature higher than that of the non-peeled tile, as shown in FIG. Is also low. In addition, although the temperature difference of the peeling tile and the surrounding tile is based also on conditions, it is often less than 1-2 degreeC, for example.

  In this embodiment, by detecting a tile having a temperature higher or lower than that of the surrounding area from a thermal image captured by the infrared light image sensor 11, it is possible to find a tile in which peeling (an abnormality has occurred). .

  Next, the operation of this embodiment will be described.

  For example, the case of investigating the outer surface material of a structure B as shown in FIG. 7 (in the example of FIG. 7, an apartment house) B will be described as an example. In this case, the flying object 10 is placed near the structure B, and the transmitter 20, the terminal device 30, and the flying object 10 are powered on. When the terminal device 30 is turned on and application software (hereinafter simply referred to as “application”) stored in the ROM 32 and the RAM 33 and the like is started, the CPU 31 The current position is detected by the GPS 34 according to the control, and a map centered on the current position is acquired from a map server (not shown) via the network 50 and displayed on the LCD touch panel 38. As a result, a map centering on the current position (position where the terminal device 30 exists) as shown in FIG. 8 is displayed on the LCD touch panel 38, for example. With reference to such a map, for example, when an operation of designating the structure B corresponding to FIG. 7 as an inspection target (for example, an operation of tapping a corresponding position on the LCD touch panel 38), as shown in FIG. The structure B is selected as an inspection target and is surrounded by a broken line, for example.

  The CPU 31 of the terminal device 30 acquires the contour of the designated structure from the map information, and acquires position information (latitude information and longitude information) of the vertices of each part of the contour. For example, in the example of FIG. 8, the position information of x1, y1 to x8, y8 is acquired. In addition, when the vertical height of the structure is known, information indicating the height may be input.

  When the shape of the structure is specified, the CPU 31 determines the flight path of the flying object 10. Specifically, for each surface of the structure B, a route that alternately repeats the vertical flight and the horizontal flight is set. For example, in the example of FIG. 7, a route indicated by a solid arrow in the drawing is set. In addition, although the structure B has four surfaces of east, west, south, and north, as the priority of these surfaces, it is desirable to set in order of east, south, west, and north. Because they appear in this order.

  When the flight path is determined, the CPU 31 of the terminal device 30 supplies information that instructs the transmitter 20 to move the flying object 10 to the start position of the flight path and to start hovering. As a result, the flying object 10 moves to the start position of the flight path shown in FIG. 7 and starts hovering at a predetermined height (for example, a height at which the tile of the structure is within the angle of view).

  When the flying object 10 starts hovering, the terminal device 30 executes an initial adjustment process. That is, the CPU 31 of the terminal device 30 adjusts so that the image component corresponding to the elements constituting the outer surface material of the structure B is maximized. More specifically, the CPU 31 obtains the vertical and horizontal pixel sizes of the elements (tiles in the example of FIG. 7) constituting the outer surface material of the structure B from the visible light image obtained by the visible light imaging element 12 of the flying object 10. To do. For example, 50 pixels in the horizontal direction and 10 pixels in the vertical direction are acquired as the tile image size. Next, the number of pixels of elements (tiles in the example of FIG. 7) in the thermal image obtained by the infrared light image sensor 11 is obtained from the ratio of the effective pixel numbers of the visible light image sensor 12 and the infrared light image sensor 11. For example, when the ratio of the number of effective pixels is 2: 1 (for example, visible light is 1920 × 1080 and thermal image is 960 × 540), 25 pixels horizontally and 5 pixels vertically are obtained.

  Next, CPU31 calculates | requires the horizontal frequency fh and the vertical frequency fv of the element which comprises the outer surface material in a thermal image. More specifically, as shown in FIG. 6A, the outer surface material is repeatedly arranged at a constant period in the horizontal direction and the vertical direction, and thus the spatial frequency of these elements is obtained. In the present example, the reciprocal of 25 horizontal pixels and 5 vertical pixels is substantially the spatial frequency.

  When the spatial frequency is obtained, the CPU 31 extracts frequency components of the horizontal frequency fh and the vertical frequency fv from the thermal image. As a result, components relating to the components of the outer surface material (components corresponding to the tiles in FIG. 6A, hereinafter referred to as “target components”) are extracted. As the target component, not only the fundamental frequency component corresponding to the tile but also the harmonic component (for example, odd order / even order harmonic) is extracted, and the tile is abnormal vertically or horizontally. In order to deal with the case, the frequency component of 1 / n (n is a natural number) of the fundamental frequency component may be extracted.

  Next, the CPU 31 adjusts the tilt angle (elevation angle or depression angle) of the gimbal 13 of the flying object 10 and sets the spectrum of the target component to be maximum, for example. Moreover, CPU31 adjusts the distance from the structure B of the flying body 10, for example, sets it so that the spectrum of a target component may become the maximum. Such an operation is set so that the elements constituting the outer surface material are detected most clearly in the thermal image.

  When the initial adjustment process is completed, the CPU 31 causes the flying object 10 to execute an imaging process. In the imaging process, the flying object 10 is caused to fly along a route as shown in FIG. Further, the tilt is adjusted by the gimbal 13 and the distance from the structure B and the flight speed are adjusted so that a clear thermal image with little noise can be taken. As a method for adjusting the tilt, distance, and flight speed, for example, adjustment can be made so that the spectrum of the target component is maximized.

  Further, when an abnormal part is detected during flight, for example, when a tile having a temperature difference from the surroundings is detected by peeling, an abnormality detection process is executed. In the abnormality detection process, in order to prevent erroneous detection, for example, the flying object 10 is hovered at a place where the abnormality is detected, a plurality of thermal images are captured, and, for example, noise is obtained by obtaining a moving average of these images. The components are reduced, and the obtained thermal image is used to distinguish whether the temperature abnormality is due to peeling or noise. As a result, when it is determined to be due to peeling, marking is performed on the peeling tile on a thermal image or a visible light image.

  The CPU 31 supplies the image data obtained as a result of the above processing to the GPU 35 and causes the LCD touch panel 38 to display the image data. 9 and 10 show an example of an image displayed on the LCD touch panel 38. FIG. Here, FIG. 9 is a display example in which the visible light image and the thermal image are displayed side by side, and FIG. 10 is a display example in which the visible light image and the thermal image are superimposed and displayed. In FIG. 9, the visible light image is displayed on the left side and the corresponding thermal image is displayed on the right side. In the thermal image, the abnormal portion is marked with a solid rectangle. In FIG. 10, a thermal image is superimposed on a visible light image, and abnormal portions are marked with a solid rectangle. Since these images are taken in the same range of the structure B from the same position, the correspondence relationship can be easily understood when they are juxtaposed, and can be easily superimposed as a single image.

  11 and 12 show an example of an actually captured image. FIG. 11 shows a display example in which the visible light image and the thermal image are displayed side by side, and FIG. 12 shows a display example in which the visible light image and the thermal image are superimposed and displayed.

  The image captured as described above is transferred to the server 60 via the network 50 and stored in a predetermined folder or directory.

  When the inspection of one outer surface of the structure B is completed, imaging of the other outer surface is started. More specifically, external inspections are performed in the order of east, south, west, and north. When inspecting a new outer surface, as described above, after performing the initial adjustment process by hovering, the imaging process is performed.

  As described above, according to the embodiment of the present invention, the structure of the structure B is detected, the flight path is automatically set, and the visible light image and the thermal image are captured by the flying object 10. Therefore, the time required for the inspection can be greatly shortened.

  In addition, the initial adjustment process adjusts the depression angle or elevation angle of the gimbal 13 and adjusts the distance to the structure B to be optimal, so that there is little noise and an optimal thermal image for analysis can be obtained. it can.

  Further, during the flight of the flying object 10, the depression angle or elevation angle of the gimbal 13 is adjusted by the in-flight adjustment process, and the distance to the structure B and the flight speed are adjusted to be optimal, so that there is little noise, and The thermal image optimal for analysis can always be obtained.

  In addition, the spatial frequency component corresponding to the size of the elements (for example, tiles) constituting the outer surface material is extracted, and the position and the flight speed are adjusted based on the extracted target component. The flight state can be controlled so that the constituent elements are imaged optimally. Further, since the depression angle or elevation angle of the gimbal 13 is controlled based on the extracted target component, for example, the influence of the sky reflection or the like is eliminated, and the angle of the elements constituting the outer surface material is optimally imaged. Can be adjusted.

  In addition, since the spatial frequency component corresponding to the size of the elements (for example, tiles) constituting the outer surface material is extracted and the abnormality is detected based on the extracted components, the influence other than the elements constituting the outer surface material is affected. And abnormalities such as peeling can be reliably detected. More specifically, the captured thermal image includes noise as an effect of sky reflection, wall contamination, shade, reflection, effects of trees, indoor heating, and the like. By extracting the spatial frequency component corresponding to the size of the outer surface material, it is possible to reduce these noise components having a spatial frequency different from that of the outer surface material, and to perform more reliable determination.

  In addition, when an abnormality is detected, the flying object 10 is put in a hovering state, for example, a plurality of thermal images are taken, and a thermal image obtained by, for example, performing a moving average on the plurality of thermal images. Based on the above, since the presence / absence of abnormality is re-determined, erroneous determination can be reduced.

  Next, a flowchart for realizing the above operation will be described with reference to FIGS. First, FIG. 13 is a flowchart illustrating an example of processing executed in the terminal device 30 when imaging of the structure B is started. When the flowchart of FIG. 13 is started, the following steps are executed.

  In step S10, the CPU 31 receives a target structure on the map. More specifically, the CPU 31 acquires information (latitude information and longitude information) related to the current position from the GPS 34, acquires a map corresponding to the current position from a map server (not shown) via the network 50, and the LCD touch panel 38. To display. With reference to the map displayed in this manner, the operator can specify the target structure.

  In step S11, the CPU 31 determines a flight path. More specifically, the CPU 31 extracts the contour of the structure B specified in step S10 by contour extraction processing or the like, and specifies the latitude and longitude of the vertex of the contour. Then, the outer surface of the structure B is identified from the position of the apex, and the identified outer surface is selected in the order of east, south, west, and north, and the flight path in which the vertical direction and the horizontal direction are alternately repeated as shown in FIG. Set. Note that the distance between the flying object 10 and the structure B may change due to an initial adjustment process or the like to be described later. In this case, the flight path may be changed, so the set path can be flexibly changed. It is desirable to make it possible.

  In step S <b> 12, the CPU 31 transmits a control signal so as to hover with respect to the flying object 10. More specifically, the CPU 31 transmits a control signal to the flying object 10 via the transmitter 20 to cause the flying object 10 to perform hovering. The hovering position is the start position of the flight path, and the hovering altitude is a position where the lower end of the tile on the outer surface of the structure B can be imaged.

  In step S <b> 13, the CPU 31 executes an initial adjustment process that is executed before imaging of a predetermined outer surface is started. Details of this processing will be described later with reference to FIG. In addition, by performing the initial adjustment process, the tilt of the gimbal is optimally adjusted, and the distance between the flying object 10 and the structure B is optimally adjusted.

  In step S <b> 14, the CPU 31 executes an imaging process for imaging while flying a predetermined outer surface of the structure B automatically. Details of this processing will be described later with reference to FIG. In addition, the predetermined outer surface of the structure B is imaged by performing this imaging process.

  In step S15, the CPU 31 determines whether or not to capture other external surfaces. If it is determined that other external surfaces are to be imaged (step S15: Y), the process proceeds to step S16, and otherwise (step S15: In N), the process ends (end). For example, when imaging of the outer surface on the east side of the structure B is completed, since the outer surface on the south side is imaged, it is determined as Y and the process proceeds to step S16.

  In step S <b> 16, the CPU 31 controls the flying object 10 and moves it in front of another outer surface. More specifically, when imaging of the outer surface on the east side of the structure B is completed, it is moved to the front of the outer surface on the south side.

  Next, the details of the initial adjustment process shown in FIG. 13 will be described with reference to FIG. When the process shown in FIG. 14 is started, the following steps are executed.

  In step S <b> 30, the CPU 31 of the terminal device 30 executes a process of extracting a target component that is an image component related to an element constituting the outer surface material. Details of this processing will be described later with reference to FIG.

  In step S31, the CPU 31 adjusts the tilt (elevation angle or depression angle) of the gimbal 13. Thereby, the tilt of the gimbal 13 is adjusted.

  In step S32, the CPU 31 determines whether or not the spectrum of the target component extracted in step S30 has reached the maximum angle. If the spectrum has reached the maximum (step S32: Y), the process proceeds to step S33. Otherwise (step S32: N), the process returns to step S30, and the same processing as described above is repeated. In addition, the elevation angle or depression angle of the gimbal 13 is adjusted by the processes in steps S30 to S32 so that the components of the outer surface material are accurately imaged.

  In step S <b> 33, the CPU 31 executes a process of extracting a target component that is an image component related to an element constituting the outer surface material. Details of this processing will be described later with reference to FIG.

  In step S34, the CPU 31 adjusts the positional relationship between the flying object 10 and the structure B in the front-rear direction. For example, in the example of FIG. 7, the distance between the flying object 10 and the structure B in the Y direction is adjusted.

  In step S35, the CPU 31 determines whether or not the spectrum of the target component extracted in step S33 has reached the maximum position, and when it has reached the maximum (step S35: Y), the process returns to the original processing (step S35: Y). In other cases (step S35: N), the process returns to step S33, and the same processing as described above is repeated. In addition, the distance between the flying body 10 and the structure B is adjusted by the process of step S33 to step S35 so as to accurately capture the components of the outer surface material.

  Next, the details of the target component extraction processing described in FIG. 14 will be described with reference to FIG. When the flowchart shown in FIG. 15 is started, the following steps are executed.

  In step S <b> 50, the CPU 31 acquires a visible light image and a thermal image from the flying object 10. More specifically, the CPU 31 acquires a visible light image and a thermal image captured by the infrared light imaging device 11 and the visible light imaging device 12 of the flying object 10 via the transmitter 20.

  In step S51, the CPU 31 acquires the number of vertical and horizontal pixels of the tile from the visible light image acquired in step S50. More specifically, for example, an edge of a visible light image is detected by a Laplacian filter, and the number of vertical and horizontal pixels in the horizontal direction and the vertical direction is specified from the detected edge interval.

  In step S52, the CPU 31 specifies the horizontal frequency fh of the elements (tiles in the example of FIG. 6) constituting the outer surface material in the thermal image. More specifically, the horizontal frequency fh is obtained from the number of pixels in the horizontal direction obtained in step S51 based on the ratio of the number of pixels of the visible light image and the thermal image (for example, 2: 1).

  In step S53, the CPU 31 specifies the vertical frequency fv of the elements (tiles in the example of FIG. 6) constituting the outer surface material in the thermal image. More specifically, the vertical frequency fv is obtained from the number of pixels in the vertical direction obtained in step S51 based on the ratio of the number of pixels of the visible light image and the thermal image (for example, 2: 1).

  In step S54, the CPU 31 executes a process of extracting band components of fh and fv from the thermal image using, for example, a DOG (Difference of Gaussian) filter. As a result, it is possible to attenuate components due to sky reflection, dirt on the wall surface, shade, reflection, effects of trees, indoor heating, and the like, which are considered to have a lower spatial frequency than tiles. In addition, in the thermal image, the temperature of the joints around the tile appears higher than the temperature of the tile due to the difference in the amount of moisture contained, but such a component has a higher spatial frequency than the tile itself. Components can also be attenuated. When the process is completed, the process returns to the original process.

  Next, details of the imaging process shown in FIG. 13 will be described with reference to FIG. When the flowchart shown in FIG. 16 is started, the following steps are executed.

  In step S <b> 70, the CPU 31 controls the flying object 10 to start ascending. More specifically, the CPU 31 sends the control signal via the transmitter 20 to raise the flying object 10 in the vertical direction. The flying object 10 is controlled based on the output from the GPS 17 and the output from the sensor 18 so that the flying object 10 is lifted without being displaced in the vertical direction and without being tilted. The sensor 18 also includes an ultrasonic sensor and an image sensor for detecting an obstacle. Therefore, when the obstacle is detected, the sensor 18 avoids the obstacle and returns to the original flight path after the avoidance. May be.

  In step S71, the CPU 31 executes an in-flight adjustment process for adjusting the tilt of the gimbal 13, the distance between the structure B and the flying object 10, and the flight speed during the flight. Details of this processing will be described later with reference to FIG.

  In step S <b> 72, the CPU 31 executes an abnormality detection process that is a process of detecting an abnormality in the outer surface material of the structure B. Details of this process will be described later with reference to FIG.

  In step S <b> 73, the CPU 31 executes a process of displaying the visible light image and the thermal image captured by the flying object 10 on the LCD touch panel 38 and storing them in the server 60. Details of this processing will be described later with reference to FIG.

  In step S74, the CPU 31 determines whether or not the upper edge of the structure B (for example, the boundary between the roof of the structure B and the sky) is detected based on the visible light image captured by the flying object 10. If it is determined that the upper edge has been detected (step S74: Y), the process proceeds to step S75. Otherwise (step S74: N), the process returns to step S71 and the same processing as described above is performed. repeat. Note that flight and imaging corresponding to the arrow from the bottom to the top in the Z-axis direction shown in FIG. 7 are executed by the processing in steps S70 to S74.

  In step S75, the CPU 31 moves the flying object 10 left or right by a certain distance. For example, in the example of FIG. 7, the flying object 10 moves from the right to the left in the X-axis direction near the roof of the structure B. Note that the moving distance is desirably set so that a part of the captured images overlaps in order to prevent an imaging omission. For example, when moving to the left as shown in FIG. 7, the left end area of the image captured before the movement overlaps with the right end area of the image captured after the movement (for example, 10 to 20% of the image width). It is desirable to set it to overlap.

  In step S76, the CPU 31 controls the flying object 10 to start descending. More specifically, the CPU 31 sends the control signal via the transmitter 20 to lower the flying object 10 in the vertical direction.

  In step S77, the CPU 31 executes an in-flight adjustment process for adjusting the tilt of the gimbal 13, the distance between the structure B and the flying object 10, and the flight speed during the flight. Details of this processing will be described later with reference to FIG.

  In step S <b> 78, the CPU 31 executes an abnormality detection process that is a process of detecting an abnormality in the outer surface material of the structure B. Details of this process will be described later with reference to FIG.

  In step S <b> 79, the CPU 31 executes a process of displaying the visible light image and the thermal image captured by the flying object 10 on the LCD touch panel 38 and storing them in the server 60. Details of this processing will be described later with reference to FIG.

  In step S80, the CPU 31 has lowered to a predetermined altitude (for example, an altitude at which the lower end of the tile on the outer surface of the structure B can be imaged) based on information from the altitude sensor of the sensor 18 of the flying object 10 or the GPS 17. If it is determined that the vehicle has descended to a predetermined altitude (step S80: Y), the process proceeds to step S81. Otherwise (step S80: N), the process returns to step S77 and the same process is repeated. . Note that the flight and imaging corresponding to the arrow from the top to the bottom in the Z-axis direction shown in FIG. 7 are executed by the processing in steps S76 to S80.

  In step S81, the CPU 31 moves the flying object 10 to the left or right by a predetermined distance. For example, in the example of FIG. 7, in the vicinity of the first floor of the structure B, the flying object 10 moves from right to left in the X-axis direction. Note that the moving distance is desirably set so that a part of the captured images overlaps in order to prevent an imaging omission. For example, when moving to the left as shown in FIG. 7, the left end area of the image captured before the movement overlaps the right end area of the image captured after the movement (for example, 10 to 20% of the image width). It is desirable to set it to overlap.

  In step S82, the CPU 31 determines whether or not imaging of a predetermined outer surface has been completed. If it is determined that the imaging has been completed (step S82: Y), the CPU 31 returns (returns) to the original processing, and otherwise. In (Step S83: N), the processing returns to Step S70 and the same processing as described above is repeated.

  Next, the details of the in-flight adjustment process shown in FIG. 16 will be described with reference to FIG. When the flowchart shown in FIG. 17 is started, the following steps are executed.

  In step S <b> 90, the CPU 31 of the terminal device 30 executes a process (process shown in FIG. 15) for extracting a target component that is an image component related to an element constituting the outer surface material.

  In step S91, the CPU 31 adjusts the tilt of the gimbal 13. Thereby, the elevation angle or depression angle of the gimbal 13 is adjusted.

  In step S92, the CPU 31 determines whether or not the spectrum of the target component extracted in step S90 has reached the maximum angle. If the spectrum has reached the maximum (step S92: Y), the process proceeds to step S93. Otherwise (step S92: N), the process returns to step S90, and the same processing as described above is repeated. Note that the elevation angle or depression angle of the gimbal 13 is adjusted so as to accurately capture the components of the outer surface material during the flight of the flying object 10 by the processing of Step S90 to Step S92.

  In step S <b> 93, the CPU 31 executes a process (process shown in FIG. 15) for extracting a target component that is an image component related to an element constituting the outer surface material.

  In step S94, the CPU 31 adjusts the positional relationship between the flying object 10 and the structure B in the front-rear direction. For example, in the example of FIG. 7, the distance between the flying object 10 and the structure B in the Y direction is adjusted.

  In step S95, the CPU 31 determines whether or not the spectrum of the target component extracted in step S93 has reached the maximum position, and if it has reached the maximum (step S95: Y), the process proceeds to step S96. Otherwise (step S95: N), the process returns to step S93, and the same process as described above is repeated. It should be noted that the distance between the flying object 10 and the structure B is adjusted so that the components of the outer surface material can be accurately imaged during the flight of the flying object 10 by the processing of step S93 to step S95.

  In step S <b> 96, the CPU 31 executes a process (process shown in FIG. 15) for extracting a target component that is an image component related to an element constituting the outer surface material.

  In step S97, the CPU 31 adjusts the flying speed with the flying object 10. For example, in the example of FIG. 7, the ascending speed or the descending speed of the flying object 10 is adjusted.

  In step S98, the CPU 31 determines whether or not the spectrum of the target component extracted in step S96 has reached the maximum position. If the spectrum has reached the maximum (step S98: Y), the process returns to the original process (step S98: Y). In other cases (step S95: N), the process returns to step S95 and the same process as described above is repeated. It should be noted that the flight speed of the flying object 10 is adjusted so that the components of the outer surface material can be accurately imaged during the flight of the flying object 10 by the processing of step S96 to step S98.

  Next, the details of the abnormality detection process shown in FIG. 16 will be described with reference to FIG. When the flowchart shown in FIG. 18 is started, the following steps are executed.

  In step S <b> 110, the CPU 31 executes a process (process shown in FIG. 15) for extracting a target component that is an image component related to an element constituting the outer surface material.

  In step S111, the CPU 31 refers to the target component extracted by the process of step S110, determines whether there is a tile having a temperature different from that of the surrounding area, and determines that it exists (step S111: Y). The process proceeds to step S112, and otherwise returns (returns) to the original process (step S111: N). More specifically, since information indicating the temperature difference of the tile in the thermal image is extracted as the target component, if there is a portion where this temperature difference is different from the surroundings, it is determined that the step is Y. Proceed to S112.

  In step S112, the CPU 31 controls the flying object 10, interrupts ascending flight or descending flight, and starts hovering. As a result, the flying object 10 is in the hovering state at the place where the abnormality is detected during the ascending flight or the descending flight.

  In step S113, the CPU 31 executes noise reduction processing using a plurality of images. More specifically, the CPU 31 takes a plurality of thermal images of places where an abnormality has been detected. For example, about 2 to 10 thermal images are taken. And noise can be reduced by calculating | requiring the addition average etc. of the several imaged thermal image. Note that noise may be reduced by a method other than averaging.

  In step S <b> 114, the CPU 31 performs a process (process shown in FIG. 15) for extracting a target component, which is an image component related to an element constituting the outer surface material, from the thermal image with reduced noise.

  In step S115, the CPU 31 refers to the target component obtained by the process in step S114, determines whether there is a tile having a temperature different from that of the surrounding area, and determines that it exists (step S115: Y). The process proceeds to step S116, and in other cases (step S115: N), it is determined that there is a false detection, and the process returns (returns) to the original process. In addition, it is possible to reduce false detection by detecting an abnormal part based on the thermal image in which noise is reduced by the processes in steps S112 to S114. It should be noted that Y may be determined not only if there are tiles having different temperatures from the surroundings, but also if, for example, noise reduction processing does not change the shape of regions having different temperatures.

  In the processing from step S112 to step S114, the hovering is performed on the flight path. However, the structure B may be approached in order to reliably detect the abnormal part.

  In step S116, the CPU 31 executes a process of marking a tile having a temperature different from that of the periphery on the image. As a result, for example, as shown in FIGS. 9 and 10, the tile at the abnormal location is surrounded by a solid rectangle.

  In step S117, the CPU 31 executes a process for issuing a warning. For example, the CPU 31 can emit a warning sound indicating that an abnormality has been detected from a speaker (not shown).

  When an abnormality is detected in step S115, not only a warning is given, but also the flying object 10 is brought close to the corresponding tile, and a small amount of paint is sprayed and marked. You may make it return to a path | route.

  Next, the details of the image display / storage processing shown in FIG. 16 will be described with reference to FIG. When the flowchart shown in FIG. 19 is started, the following steps are executed.

  In step S <b> 130, the CPU 31 acquires a thermal image from the flying object 10 via the transmitter 20, for example.

  In step S131, the CPU 31 performs a process (process shown in FIG. 15) for extracting a target component, which is an image component related to the elements constituting the outer surface material, on the thermal image acquired in step S130.

  In step S132, the CPU 31 converts the target component (information indicating temperature) obtained by the processing in step S131 into information indicating hue. As a result, it is converted into information whose hue changes from black, blue, green, red, and white as the temperature difference increases.

  In step S133, the CPU 31 acquires a visible light image from the flying object 10 via the transmitter 20, for example.

  In step S134, for example, when the image juxtaposition mode is selected by performing a predetermined operation on the LCD touch panel 38 (step S134: Y), the CPU 31 proceeds to step S135, and otherwise. In (Step S134: N), the process proceeds to Step S136. For example, when the icon for selecting the image display mode displayed on the LCD touch panel 38 is operated and the image juxtaposition mode is selected as the display mode, the process proceeds to step S135, and the image superposition mode is selected as the display mode. If so, the process proceeds to step S136.

  In step S135, the CPU 31 displays the thermal image and the visible light image side by side on the LCD touch panel 38. As a result, for example, the thermal image and the visible light image are displayed side by side in the manner shown in FIG. 9 or FIG.

  In step S136, the CPU 31 proceeds to step S137 if the image superimposing mode is selected by performing a predetermined operation on the LCD touch panel 38 (step S136: Y), and otherwise In (Step S136: N), the process proceeds to Step S141. For example, when the icon for selecting the image display mode displayed on the LCD touch panel 38 is operated and the image superposition mode is selected as the display mode, the process proceeds to step S137, and otherwise, the process proceeds to step S141.

  In step S137, the CPU 31 multiplies each pixel constituting the visible light image by α (α <1).

  In step S138, the CPU 31 multiplies each pixel constituting the thermal image by (1−α).

  In step S139, the CPU 31 combines the visible light image and the thermal image obtained in steps S137 and S138. By the above processing, the thermal image is superimposed and synthesized on the visible light image by the alpha blending process.

  In step S140, the CPU 31 causes the LCD touch panel 38 to display the image obtained by the combining process in step S139. As a result, for example, the thermal image and the visible light image are superimposed and displayed in the manner shown in FIG. 10 or FIG.

  In step S <b> 141, the CPU 31 transmits the thermal image and the visible light image via the network 50 and stores them in the server 60. Note that the target component obtained by the process of step S131 may be stored instead of the thermal image.

  As described above, according to the flowcharts shown in FIGS. 13 to 19, the above-described operation can be realized.

(C) Description of Modified Embodiment Each of the above embodiments is an example, and it is needless to say that the present invention is not limited to the case described above. For example, in the above example, an infrared thermal image is used as the invisible light image, but an image other than this may be used. Specifically, for example, ultraviolet rays may be used, electromagnetic waves in the MHz or GHz band may be used, or ultrasonic waves may be used. In the case where ultraviolet rays are used, a two-dimensional ultraviolet imaging element may be used. In addition, when using electromagnetic waves in the MHz or GHz band, a plurality of receiving antennas are arranged in the horizontal direction, and the electromagnetic waves transmitted from the transmitting antennas are reflected by the structure B, and from the time difference when reaching these receiving antennas By detecting the angle in the horizontal direction, the reflectance of the object can be specified by the intensity of the received signal. Similarly, in the case of ultrasonic waves, a plurality of receiving elements are arranged in the horizontal direction, the angle in the horizontal direction is detected from the time difference when reaching the plurality of receiving elements, and the reflection of the object is determined by the intensity of the received signal. The rate etc. can be specified.

  Moreover, in the above embodiment, although the process shown in FIGS. 13-19 was performed in the terminal device 30, the flowchart which shows the flowchart shown in FIGS. You may make it arrange | position among the transmitter 20, the terminal device 30, and the server 60, or distribute | distributing them. For example, the process shown in FIGS. 13 to 17 may be executed in the flying object 10 and the process shown in FIG. 19 may be executed in the terminal device 30 or the server 60. Of course, all the processes shown in FIGS. 13 to 17 may be executed in the terminal device 30 or the server 60.

  Further, in the above embodiment, the structure has been described by taking a collective housing as an example, but in addition to this, for exterior buildings for office buildings, detached houses, bridges, piers, tunnels, dams, etc. An abnormality may be detected. Moreover, although the tile was taken as an example as an element constituting the outer surface material, in addition to this, it is also possible to inspect siding, brick, and surface concrete.

  In the embodiment shown in FIG. 7, the structure B is imaged when flying in the vertical direction, but the structure B may be imaged when flying in the horizontal direction. In the example of FIG. 7, the rooftop is excluded from the imaging target, but the rooftop may also be imaged. In that case, in the example of FIG. 7, the entire rooftop can be captured by repeating the operation of capturing images while flying in the X direction after capturing images during the flight in the X direction and moving in the Y direction. Of course, it is also possible to repeat the operation of capturing an image while flying in the Y direction, moving in the X direction, and then flying while flying in the Y direction. In addition, about rooftop, you may make it detect peeling of the waterproof sheet etc. of a rooftop part.

  In the above embodiment, the target component is extracted from the thermal image, and the imaging state is determined based on the size of the target component. However, the imaging state may be determined by other methods. For example, when the component having a lower spatial frequency (mainly a noise component such as sky reflection) and / or a higher component (mainly a noise component due to a joint portion) is less than the target component, the imaging state is determined to be good. It may be. Alternatively, the visible light image may be converted into a frequency domain, a component related to the tile may be extracted mainly from the spatial frequency included, and a component corresponding to the component may be extracted from the thermal image. According to such a configuration, it is possible to identify and extract an appropriate spatial frequency component based on a visible light image that is not easily affected by sky reflection or the like.

  Moreover, in the above embodiment, it is not specified whether the infrared light image sensor 11 and the visible light image sensor 12 are moving images or still images, but any of these is used. be able to. That is, in the case of a still image, the image can be captured so that there is a range that overlaps with the still image captured immediately before in the vertical direction. For example, when the flying object 10 is rising, at the timing when the upper part of the still image captured immediately before and a part of the lower part of the still image to be captured (for example, 10 to 20% of the still image) overlap. What is necessary is just to image. Of course, since an image for controlling the flight of the flying object 10 is required, still images taken at a predetermined interval (for example, several images per second) are used for flight control, and the above-described overlapping images are used. Anomaly detection can be performed by extracting still images that satisfy the conditions. Also, in the case of a moving image, processing can be performed in the same manner as in the case of a still image by extracting an I (Intra) picture that constitutes a GOP (Group of Pictures).

  Further, in the above embodiment, when an abnormality is detected, the flying object 10 is hovered and imaged. However, instead of hovering, imaging is performed a plurality of times while flying at a location where the abnormality is detected. However, when the shape of the part where the temperature is different from that of the periphery does not change regardless of the imaging position or the like, it may be determined as abnormal. Further, in the above embodiment, the imaging of the structure B by the flying object 10 is performed only once. However, an abnormal part having the same shape is captured more than a predetermined number of times obtained by capturing images multiple times. You may make it determine with it being abnormal. For example, after flying the route shown in FIG. 7 in the direction of the arrow, the image is taken while flying again in the opposite direction of the arrow, and when an abnormality of the same shape is detected at the same location, it is determined that the abnormality has occurred. Good. This is because the true abnormal part is likely to be imaged as an abnormal part having the same shape regardless of the imaging state (for example, the flight path, the tilt of the gimbal 13, the distance from the structure B, etc.).

  Further, in the above embodiment, as processing for detecting an abnormality, tiles having a temperature different from that of surrounding tiles (higher or lower) are detected by image processing. However, for example, CNN (Convolutional Neural Network) or the like is used. An abnormal part may be detected by a learning process using the above neural network. Specifically, an abnormality detection CNN is supplied to a CNN having a Convolution layer, a Pooling layer, a fully connected layer, and the like by supplying an image obtained by imaging a peeled tile as a learning image and performing a learning process. Can be configured.

  Note that the flowcharts shown in FIGS. 13 to 19 are examples, and the present invention is not limited to the flowcharts shown in FIGS. 13 to 19.

1: External surface material inspection system 10: Aircraft 11: Infrared light imaging device (invisible light imaging means)
12: Visible light imaging device (visible light imaging means)
13: Gimbal 14: Image processing unit 15: Transmission / reception unit 15a: Antenna 16: Control unit (aircraft control means)
17: GPS
18: Sensor 19: Motor 20: Transmitter 21: Transmitter / receiver 21a: Antenna 22: Control unit 23: Joystick 24: Joystick 25: Operation unit 26: I / F
30: Terminal device 31: CPU (image processing means, flying object control means, abnormality detection means)
32: ROM
33: RAM
34: GPS
36: I / F
37: Bus 38: LCD touch panel (display means)
40: Base station 50: Network 60: Server 61: CPU
62: ROM
63: RAM
64: HDD
65: I / F
66: Bus

In order to solve the above-described problems, the present invention provides an external material inspection system that processes an image captured by a flying object and displays the processed image on a terminal device to investigate the state of the external surface material of the structure. A visible light image capturing unit that captures a light image; and an invisible light image capturing unit that detects infrared rays emitted from the subject and captures a thermal image indicating the temperature of each part of the subject as a non-visible light image. Then, the terminal device or the flying object specifies a spatial frequency component corresponding to an element constituting the outer surface material based on the visible light image captured by the visible light image capturing unit, and the visible light image Image processing means for specifying a spatial frequency component of the element in the thermal image based on a relative relationship of the thermal image and executing a process of extracting a band component corresponding to the specified spatial frequency; A flying object control means for controlling the flight conditions of the aircraft based on the image processing result by the image processing means, and abnormality detecting means for detecting an abnormality of said outer members on the basis of a result of image processing by the image processing means, It is characterized by having.
According to such a structure, it becomes possible to test | inspect the abnormality of the outer surface material of a structure for a short time.

Further, the present invention, the aircraft control means, based on the band component corresponding to the spatial frequencies detected by the image processing means, for adjusting the distance and / or flying speed of the said structure of the aircraft It is characterized by that.
According to such a configuration, the distance and the flight speed can be set appropriately.

Further, the present invention, the aircraft control means, based on the band component corresponding to the spatial frequencies detected by the image processing means, elevation or depression angle of the visible light image pickup means and said non-visible light image pickup means It is characterized by adjusting.
According to such a configuration, the elevation angle or depression angle can be set appropriately.

Claims (10)

In the exterior material investigation system that processes the image captured by the flying object and displays it on the terminal device, and investigates the state of the exterior material of the structure,
The aircraft is
A visible light image capturing means for capturing a visible light image;
A non-visible light image capturing means for capturing a non-visible light image by electromagnetic waves or sound waves in a region other than visible light,
The terminal device or the flying body is
Image processing means for executing predetermined image processing on the visible light image and / or the invisible light image;
A flying object control means for controlling a flight state of the flying object based on an image processing result by the image processing means;
An abnormality detection means for detecting an abnormality of the outer surface material based on an image processing result by the image processing means,
External material survey system characterized by that. The image processing means executes a process of detecting an image component corresponding to an element constituting the outer surface material,
The aircraft control means adjusts the distance from the structure and / or the flight speed of the aircraft based on the image component corresponding to the element detected by the image processing means. The outer surface material investigation system according to claim 1. The image processing means executes a process of detecting an image component corresponding to an element constituting the outer surface material,
The flying object control means adjusts the elevation angle or depression angle of the visible light image imaging means and the invisible light image imaging means based on the image component corresponding to the element detected by the image processing means. The exterior material inspection system according to claim 1 or 2, characterized in that The invisible light image capturing means detects infrared radiation emitted from the subject, captures a thermal image indicating the temperature of each part of the subject,
The image processing unit specifies a spatial frequency component corresponding to an element constituting the outer surface material based on the visible light image captured by the visible light image capturing unit, and relative to the visible light image and the thermal image. Specifying a spatial frequency component of the element in the thermal image based on a general relationship, and performing a process of extracting a band component corresponding to the specified spatial frequency,
The exterior material inspection system according to any one of claims 1 to 3, wherein:   The abnormality detection means refers to the band component extracted by the image processing means, and determines that an abnormality is detected when the temperature of the portion corresponding to the element constituting the outer surface material is different from the adjacent portion. The exterior material inspection system according to claim 4, wherein When an abnormality is detected by the abnormality detection means, the flying object control means sets the flying object in a hovering state,
The abnormality detection means determines the presence or absence of abnormality based on the plurality of invisible light images captured during hovering;
The exterior material inspection system according to claim 5, wherein   The said flying object control means receives designation | designated of the said structure used as the object of investigation on a map, and sets a flight path | route based on the designated said structure. Item 1. The exterior material inspection system according to item 1.   The said flying object control means controls to image a predetermined outer surface of the structure by alternately repeating vertical flight and horizontal flight. The exterior material inspection system according to claim 1.   The outer surface material according to any one of claims 1 to 8, wherein the flying object control means controls the outer surface of the structure to be photographed in order of east, south, west, and north. Survey system. The terminal device has display means for displaying an image,
The display means displays the visible light image and the non-visible light image juxtaposed or displays the non-visible light image superimposed on the visible light image.
The exterior material inspection system according to any one of claims 1 to 9, wherein

Images