WO2018212023A1

WO2018212023A1 - Building method for building object identification apparatus, storage device, setting device and object identification method

Info

Publication number: WO2018212023A1
Application number: PCT/JP2018/017819
Authority: WO
Inventors: 剛成竹
Original assignee: ダイキン工業株式会社
Priority date: 2017-05-15
Filing date: 2018-05-08
Publication date: 2018-11-22
Also published as: JP2018194950A

Abstract

The purpose of this invention is to identify, with a high degree of accuracy and a high degree of certainty, image information of a specific object, from a group of image data having a huge amount of data. Provided is a building method in which setting conditions of a machine learning apparatus (1) which performs machine learning are adjusted, and an object identification device (10) is built, said object identification device (10) identifying an object having predetermined characteristics from a group of image data with position information, obtained by associating position information with individual image information. This building method comprises an extraction step, a setting step and an adjustment step. In the extraction step, an image is extracted on the basis of a rule base related to an object having predetermined characteristics, from part of a group of image data with position information. The setting step sets a label to the extracted image, if an object having predetermined characteristics is shown. The adjustment step inputs an image to which a label has been set to a machine learning apparatus, and adjusts setting conditions in accordance with the label, on the basis of output results with respect to the input.

Description

Construction method, storage device, setting device, and object identification method for constructing object identification device

Conventionally, it has been studied to obtain image information of an object installed outdoors and provide useful information based on the image information. For example, Patent Document 1 (Japanese Patent No. 5735071) discloses an information search apparatus that provides information on a facility where a mat is installed to a user based on image information of the mat provided in a store.

By the way, image information of objects installed outside the room can also be obtained using an online map service. In the online map service, a group of image data with position information in which position information is associated with individual image information is provided.

However, many objects other than the target object are projected in the image data group provided by the online map service. For this reason, there is a case where an object cannot be extracted with high accuracy and high accuracy from such an image data group.

An object of the present invention is to identify image information of a specific object with high accuracy and high accuracy from an image data group having a huge amount of data.

The construction method according to the first aspect of the present invention adjusts the setting conditions of a machine learning device that executes machine learning, and obtains predetermined characteristics from an image data group with position information in which position information is associated with individual image information. An object identification device for identifying a predetermined object is constructed. Specifically, this construction method includes an extraction step, a setting step, and an adjustment step. In the extraction step, an image is extracted from a part of the image data group with position information based on a rule base related to an object having a predetermined feature. In the setting step, when an object having a predetermined feature is projected on the image extracted in the extraction step, a label indicating that the object has a predetermined feature is set. In the adjustment step, the image in which the label is set in the setting step is input to the machine learning device, and the setting condition is adjusted corresponding to the label based on the output result for the input.

In the construction method according to the first aspect, the above-described steps are executed to adjust the setting conditions of the machine learning device. Therefore, from a group of image data with position information having a huge amount of data, an object having a predetermined feature is detected. An object identification device that can be identified by an image can be constructed. In other words, by using the object identification device constructed by this construction method, it is possible to identify the image information of a specific object with high accuracy and high accuracy from a large amount of image data group.

The construction method according to the second aspect of the present invention is the construction method according to the first aspect, wherein the setting conditions are readjusted by inputting image data with position information to the machine learning device in which the setting conditions are adjusted by the adjustment step. The readjustment step is further provided.

In the method according to the second aspect, since the setting condition is readjusted using the machine learning device in which the setting condition is adjusted, the setting condition of the machine learning device can be further optimized. As a result, the image information of the object can be extracted with higher accuracy and accuracy.

The construction method according to the third aspect of the present invention is an apparatus that constitutes at least a part of a refrigeration apparatus in which the object circulates the refrigerant in the refrigerant circuit in the construction method of the first aspect or the second aspect. Therefore, according to this construction method, it is possible to construct an object identification device that identifies equipment constituting at least a part of the refrigeration apparatus.

In the present invention, the “refrigeration apparatus” means an arbitrary heat source machine that moves heat on the principle of a heat pump. For example, an air conditioner etc. are mentioned as a freezing apparatus.

The construction method according to the fourth aspect of the present invention is the construction method according to the third aspect, wherein the feature of the object indicates information on the type of refrigerant. Therefore, according to this construction method, it is possible to construct an object identification device that identifies equipment having information on the type of refrigerant.

The construction method according to the fifth aspect of the present invention is a construction method according to the first aspect to the fourth aspect, in which the feature of the object indicates information on any one of the manufacturer, the model, the year, or any combination. is there. Therefore, according to this construction method, it is possible to construct an object identification device that identifies an object having characteristics relating to any one of the manufacturer, the model, the year, or any combination. In other words, according to the object identification device constructed by this construction method, any one or any combination of the manufacturer, model, and year of the air conditioner can be identified.

The construction method according to the sixth aspect of the present invention is the construction method according to the first aspect to the fifth aspect, wherein the feature of the object relates to deterioration. Therefore, according to this construction method, it is possible to construct an object identification device that identifies an air conditioner having characteristics relating to deterioration. In other words, the feature relating to the deterioration of the air conditioner can be identified by the object identification device constructed by this construction method.

It should be noted that the “feature relating to deterioration” is, for example, a feature indicating the degree of deterioration of the object and / or the type of deterioration of the object. “Deterioration types” include wear, cracks, damage, disconnection, loosening (such as screws), dropout (such as screws), deformation, scratches, poor contact, missing parts, rust, corrosion, dirt, etc. It is done.

The construction method according to the seventh aspect of the present invention is the construction method according to the first aspect to the sixth aspect, wherein the image data group with position information is an image in which a landscape is photographed from the lateral direction and / or a landscape from above. It is composed of captured images. Therefore, according to this construction method, the object identification device can be constructed based on the image data group with position information composed of street view type photographs, ground photographs, aerial photographs, satellite photographs and the like.

The construction method according to an eighth aspect of the present invention is the construction method according to the first aspect to the seventh aspect, in which the adjustment step is performed when each part of the image data group with position information is input to the machine learning device. The portion where the object is projected is specified using the coordinate information set in the image data with information.

In the construction method according to the eighth aspect, since the portion where the object is copied is specified using the coordinate information, it is necessary to cut out and save the portion where the object is copied from the image data with position information in advance. There is no. In other words, in this construction method, learning by the machine learning device can be executed using the image data group with position information that is prohibited from storing data after processing.

The storage device according to the ninth aspect of the present invention is used in the construction method according to the eighth aspect, and stores the coordinate information set in each piece of image data with position information in association with the image data with position information. .

Since the storage device according to the ninth aspect stores the coordinate information, it is possible to construct the object identification device from the machine learning device without saving the image data group with position information after processing.

A setting device according to a tenth aspect of the present invention adjusts a setting condition of a machine learning device that executes machine learning, and obtains predetermined characteristics from an image data group with position information in which position information is associated with individual image information. It is used for constructing an object identification device for identifying an object having the object. Specifically, the setting device includes an extraction unit, a setting unit, and an adjustment unit. The extraction unit extracts an image from a part of the image data group with position information based on a rule base related to an object having a predetermined feature. In the setting unit, when an object having a predetermined feature is projected on the image extracted by the extraction unit, a label indicating that the object has the predetermined feature is set. The adjustment unit inputs the image with the label set by the setting unit to the machine learning device, and adjusts the setting condition corresponding to the label based on the output result with respect to the input.

In the setting device according to the tenth aspect, the object identification device can be constructed by adjusting the setting conditions of the machine learning device. By using such an object identification device, image information of an object having a specific feature can be extracted with high accuracy and high accuracy from a group of image data with position information having an enormous amount of data.

The object identification method according to the eleventh aspect of the present invention is to construct the first object identification device from the image data group with the first position information using any one of the construction methods from the first viewpoint to the eighth viewpoint, Using the construction method according to any one of the first aspect to the eighth aspect, the second object identification device is constructed from the image data group with the second position information different from the image data group with the first position information. . And this target object identification method identifies a target object using a 1st target object identification apparatus and a 2nd target object identification apparatus.

In the object identification method according to the eleventh aspect, since the object is recognized based on the respective results of the object identification apparatus constructed based on the image data groups with different position information, the accuracy and accuracy of identifying the object Can be increased.

An object identification method according to a twelfth aspect of the present invention is the object identification method according to the eleventh aspect, wherein one of the image data group with the first position information or the image data group with the second position information is the position information. Is an image data group in which a landscape is photographed from the lateral direction, and the other is an image data group in which a landscape is photographed from above, in which position information is individually associated. Therefore, according to such an object identification method, an object is identified based on a combination of image data groups with position information composed of street view type photographs, ground photographs, aerial photographs, satellite photographs, and the like. It is possible to improve accuracy and accuracy of identifying an object in a predetermined place.

In the construction method according to the first aspect, it is possible to construct an object identification device capable of identifying an image of an object having a predetermined feature from a group of image data with position information having a huge amount of data from a machine learning device.

In the construction method according to the second aspect, the setting conditions of the machine learning device can be further optimized.

In the construction method according to the third aspect, it is possible to construct an object identification device that identifies equipment constituting at least a part of the refrigeration apparatus.

In the construction method according to the fourth aspect, it is possible to construct an object identification device that identifies equipment having information on the type of refrigerant.

In the construction method according to the fifth aspect, it is possible to construct an object identification device that identifies an object having characteristics relating to any combination of manufacturer, model, year, or any combination.

In the construction method according to the sixth aspect, it is possible to construct an object identification device that identifies an air conditioner having characteristics relating to deterioration.

In the construction method according to the seventh aspect, the object identification device can be constructed based on the image data group with position information generated from street view type photographs, ground photographs, aerial photographs, satellite photographs and the like.

In the construction method according to the eighth aspect, learning by the machine learning device can be executed using an image data group with position information that is prohibited from storing data after processing.

By using the storage device according to the ninth aspect, it is possible to construct the object identification device from the machine learning device without saving the image data group with position information after processing.

By using the setting device according to the tenth aspect, it is possible to construct an object identification device that can identify an image of an object having a predetermined feature from an image data group with position information having a huge amount of data from a machine learning device. .

In the object identification method according to the eleventh aspect, the accuracy and accuracy of identifying the object can be increased.

In the object identification method according to the twelfth aspect, the accuracy and accuracy of identifying an object at a predetermined place can be improved.

It is a figure for demonstrating the relationship between the machine learning apparatus 1 and the target object identification apparatus 10 which concern on one Embodiment of this invention. It is a schematic diagram which shows the concept of the machine learning apparatus 1 which concerns on the same embodiment. It is a figure for demonstrating the concept of the machine learning apparatus 1 which concerns on the same embodiment. It is a figure for demonstrating the concept of the machine learning apparatus 1 which concerns on the same embodiment. It is a figure which shows an example of the output result of the target object identification apparatus 10 which concerns on the embodiment. It is a schematic diagram which shows the structure of the setting apparatus 20 which concerns on the embodiment. It is a figure for demonstrating the teacher data which concern on the embodiment. It is a figure for demonstrating the construction method which builds the target object identification apparatus 10 which concerns on the embodiment. It is a figure for demonstrating the modification of the embodiment. It is a schematic diagram which shows the concept of the target object identification system which concerns on the application example of the embodiment.

(1) Overview of Object Identification Device An object identification device 10 according to an embodiment of the present invention is constructed from a machine learning device 1 as shown in FIG. First, adjustment of the setting conditions of the machine learning device 1 will be described.

(1-1) Machine Learning Device FIG. 2 is a schematic diagram showing the concept of the machine learning device 1 according to an embodiment of the present invention.

The machine learning device 1 executes machine learning by a neural network having a multilayer structure, and functions as a so-called deep learning machine. The neural network having a multilayer structure is formed of an input layer L1, a hidden layer L2, and an output layer L3. Each layer includes a plurality of units U11, U12,... Umn (m and n are natural numbers), and a parameter apq (p and q are natural numbers) is multiplied as a weight w by an output value x of a unit belonging to the previous layer. It is input as an input value z of a unit belonging to the later layer after the bias bp is applied. For example, in the example shown in FIG. 3, the following relational expressions (Formula 1 and Formula 2) are established between the previous layer La and the subsequent layer Lb.

In the later layer Lb, the input value z is converted by the activation function f. The activation function f is appropriately selected according to the purpose.

Then, in the machine learning device 1, as shown in FIG. 4, the error E is calculated from the output data obtained by inputting the input data and the “teacher data” indicating the correct answer to the input data using a loss function. The weight w described above is adjusted based on the calculated error E. Such adjustment of the weight w is also referred to as “learning” in the following description.

Note that the number of units in each layer and the number of hidden layers described above are merely examples, and any number can be adopted. Although a fully connected neural network is shown here, the machine learning device 1 according to the present embodiment is not limited to this, and may be a convolutional neural network or the like.

(1-2) Object Identification Device The object identification device 10 displays an image in which an object having a predetermined feature is copied from a “image data group with position information” in which position information is associated with individual image information. To identify. Here, the object identification device 10 identifies “air conditioner” as an example of the object. Specifically, the target object identification device 10 cuts out a large number of candidate areas in each image, and calculates the existence probability of the target object in each candidate area, thereby identifying the image projected by the air conditioner. Such an object identification device 10 is constructed by inputting and learning image data on which various air conditioners are copied to the machine learning device 1 described above.

It should be noted that the image data group with position information includes image data in which a landscape is photographed from the horizontal direction, image data in which a landscape is photographed from above, or the like. Here, the image data in which the landscape is photographed from the horizontal direction includes not only an image in which the landscape is photographed from the horizontal direction but also an image in which the landscape is photographed from a viewpoint toward obliquely upward and obliquely downward. Further, the image data obtained by photographing the landscape from above includes not only an image obtained by photographing the landscape directly below from above but also an image obtained by photographing the landscape obliquely downward from a high place. For example, the image data group with position information includes a ground photograph, an aerial photograph, a satellite photograph, and the like. More specifically, the image data group with position information includes a street view type image data group including position information for each region and a landscape image corresponding to the position information.

Also, the object identification device 10 is constructed so as to identify the air conditioner according to the feature of any one of manufacturer, model, year, or any combination. Specifically, such an object identification device 10 uses image data on which an air conditioner capable of distinguishing appearance characteristics (the overall design, the design of parts, a logo, the shape and color of a fan, etc.) is displayed. It is constructed by inputting into the learning device 1 and learning.

Also, the object identification device 10 is constructed so as to identify an air conditioner having information on the type of refrigerant. Specifically, such an object identification device 10 machine-learns image data obtained by copying an air conditioner with a seal indicating the type of refrigerant (a seal printed with characters such as R410A, R407C, and R32). It is constructed by inputting to the device 1 and learning.

Also, the object identification device 10 is constructed so as to identify an air conditioner having characteristics relating to deterioration. Specifically, such an object identification device 10 is constructed by inputting image data on which an air conditioner capable of distinguishing a pattern related to deterioration is projected and learning the machine learning device 1. It should be noted that the characteristics relating to deterioration include characteristics indicating the degree of deterioration and / or the type of deterioration. “Degradation types” are subdivided into wear, cracks, breakage, disconnection, loosening (such as screws), dropout (such as screws), deformation, scratches, poor contact, missing parts, rust, corrosion, and dirt. Is done. Further, the deterioration types may be subdivided into those caused by aging deterioration (gradual change over a long period of time) of the object and those caused by abnormality (change occurring in a short period of time) occurring in the object. For example, abnormalities that occur in an object include ignition, overheating, physical impact due to an accident, and the like.

Then, when the image data copied from the air conditioner is input, the object identification device 10 classifies the image data to identify whether the air conditioner has a predetermined feature.

Furthermore, the object identification device 10 can also reflect the position information associated with the image data obtained by projecting the air conditioner having a predetermined characteristic in the map information. For example, as shown in FIG. 5, the object identification device 10 can indicate the position information of the air conditioner for each “manufacturer” on a map. This makes it possible to provide useful information (such as the presence of a recall machine and a sales proposal for air-conditioning related equipment) to an air conditioner maintenance company.

Note that information for causing the machine learning device 1 to learn is stored in the storage device 5. The storage device 5 stores the coordinate information of the portion where the object is projected in the image data with position information in association with the image data with position information.

(2) Setting Device and Construction Method for Constructing Object Identification Device In this embodiment, the setting conditions of machine learning device 1 are adjusted using setting device 20 in order to construct object identification device 10 described above. To do.

(2-1) Setting Device FIG. 6 is a schematic diagram showing the configuration of the setting device 20 according to this embodiment.

The setting device 20 sets the setting conditions of the machine learning device 1 using the “image data group with position information”, and constructs the target object identification device 10 that identifies a target having a predetermined feature. Here, the setting device 20 includes an input unit 21, an output unit 22, an acquisition unit 23, a storage unit 24, and a processing unit 25.

The input unit 21 inputs information to the setting device 20. For example, the input unit 21 includes a keyboard, a mouse, and / or a touch screen. Various commands are input to the setting device 20 via the input unit 21, and processing according to the commands is executed in the processing unit 25.

The output unit 22 outputs various information from the setting device 20. For example, the output unit 22 includes a display and a speaker.

The acquisition unit 23 acquires an image data group with position information. For example, the acquisition unit 23 acquires the image data group with position information from the external server device 2 via the network. However, the acquisition unit 23 may acquire the image data group with position information stored in the storage medium by reading the storage medium or the like instead of via the network.

The storage unit 24 stores information input to the setting device 20, information calculated by the setting device 20, and the like. For example, the storage unit 24 includes a memory and a hard disk device. The storage unit 24 stores a program for realizing each function of the processing unit 25 described later.

In addition, the storage unit 24 stores “rule information” for classifying the characteristics of the object on a rule basis. For example, when the object is an air conditioner, the rule information is related to a predetermined feature of the air conditioner, and “the overall shape is“ square ”and some shapes have“ circle ”. Yes, it is information indicating the content such as “the whole color is“ white ”and“ the fan part has a predetermined pattern ””. Then, by using this rule information, it is possible to extract an image in which an object matching these contents is copied from a large number of image data groups.

The processing unit 25 executes information processing in the setting device 20. Specifically, the processing unit 25 includes a CPU, a GPU, a cache memory, and the like. The processing unit 25 functions as an extraction unit 251, a setting unit 252, and an adjustment unit 253 by executing a program incorporated in the storage unit 24.

The extraction unit 251 extracts, as “learning data”, an image that matches the rule information related to the object from a part of the image data group with position information acquired by the acquisition unit 23. For example, the extraction unit 251 projects an object that matches rule information such as ““ square ”,“ circle ”,“ white ”,“ a predetermined pattern on the fan ”, etc., from a large amount of image data group. Extract the image that is.

The setting unit 252 classifies whether or not an object including the feature of the target object is projected in the “learning data” image. Then, when the setting unit 252 classifies that an object including the feature of the object is copied in the image extracted by the extraction unit 251, the setting unit 252 indicates that the image has the classified feature. Set “Teacher Label”. Note that the setting unit 252 does not copy the object including the feature of the object to the image when the object including the feature of the target is not copied in the image extracted by the extraction unit 251. Set.

Specifically, the setting unit 252 applies ““ square ”,“ circle ”,“ white ”,“ there is a predetermined pattern on the fan part ”, etc. A teacher label indicating that “the air conditioner has the characteristics of“ V-type model ”of“ 2010 model year ”whose manufacturer is“ Company D ”” is set. The correspondence relationship between the teacher label and the rule information is stored in the storage unit 24.

The adjustment unit 253 inputs the image of the object for which the teacher label has been set by the setting unit 252 to the machine learning device 1 as “teacher data”, and sets the setting condition corresponding to the teacher label based on the output result for the input. adjust. Specifically, the setting condition is adjusted by the adjustment unit 253 based on the error back propagation method or the like.

Note that the adjustment unit 253 specifies the portion where the object is projected in the image data with position information as the teacher data, using the coordinate information. For example, in the example shown in FIG. 7, in the image data with position information G, the part A of the air conditioner includes coordinate information P1 (x1, y1), P2 (x2, y1), P3 (x2, y2), P4 (x1 , Y2). These coordinate information P1 to P4 are stored in the storage device 5 together with the corresponding image data G with position information.

(2-2) Construction Method FIG. 8 is a diagram for explaining a construction method for constructing the object identification device 10 according to the present embodiment.

The object identification device 10 identifies an object having a predetermined characteristic. Here, as an example, it is assumed that an object having a predetermined feature is “an air conditioner having a feature of“ V model ”of“ year model of 2010 ”where“ manufacturer is company D ””. Hereinafter, a construction method for constructing the object identification device 10 capable of identifying such an object will be described.

First, the object to be identified and its characteristics are determined. For example, it is determined that the object to be identified is “air conditioner”, and its feature is “2010 model” “V model” with “Manufacturer is company D”. It is determined.

Accordingly, the operation of the extraction unit 251 of the setting device 20 extracts an image on a rule basis from a part of the image data group with position information based on the rule information related to the object having the above-described characteristics. (Step S1). For example, an object that matches the rule information such as ““ square ”,“ circle ”,“ white ”“ predetermined pattern on the fan ”, etc., from the large amount of image data group with position information by the extraction unit 251 An image in which is projected is extracted.

Next, it is classified by the operator 3 whether or not the object having the above-described features is projected on the image extracted by the extraction unit 251 via the setting unit 252. Then, a “teacher label” indicating that the object has the above-described feature is set via the setting unit 252 for the image classified as the object having the above-described feature is copied (step S2). . That is, for an image in which an object that matches the rule information of ““ square ”,“ circle ”,“ white ”,“ there is a predetermined pattern on the fan ”, A teacher label is set to indicate that the air conditioner has the feature of “V model” of a certain “2010 model”.

Subsequently, the image of the object for which the teacher label is set by the setting unit 252 is input to the machine learning device 1 as “teacher data”. Here, the teacher data includes image data with position information in which the object is copied, and coordinate information of an area in which the object in the image data with position information is copied. In short, when the machine learning device 1 learns, a portion where the object is projected is cut out from the image data with position information based on the coordinate information each time. As a result, the machine learning device 1 can be adjusted without processing and storing the image data with position information.

Then, the adjusting unit 253 adjusts the setting condition corresponding to the teacher label based on the input and output to the machine learning device 1. Specifically, the weight w of each layer in the machine learning device 1 is adjusted (step S3).

Thereby, an object identification device 10 that can identify an object having the above-described features is constructed from the machine learning device 1. In other words, image data on which an air conditioner capable of distinguishing external features (““ square ”,“ circle ”,“ white ”“ predetermined pattern on the fan part ”)” is projected to the machine learning device 1 By inputting and learning, the object identification device 10 capable of identifying “an air conditioner having characteristics of a manufacturer (Company D), a model (V type), and a model year (2010)” is constructed.

Here, for the sake of convenience, an example of identifying an air conditioner having one type of feature has been described. However, by adjusting the setting conditions of the machine learning device 1 in the same manner, a plurality of objects having different features can be obtained. It is possible to construct an object identification device 10 that can be identified.

(3) Features (3-1)
As described above, the construction method for constructing the object identification device according to the present embodiment adjusts the setting conditions of the machine learning device 1 that executes machine learning, and the position in which position information is associated with individual image information. An object identification device 10 for identifying an object having a predetermined feature is constructed from the image data group with information. Specifically, this construction method includes an extraction step, a setting step, and an adjustment step. In the extraction step, an image is extracted from a part of the image data group with position information based on a rule base related to an object having a predetermined feature. In the setting step, when an object having a predetermined feature is projected in the image extracted in the extraction step, a teacher label indicating that the object has a predetermined feature is set. In the adjustment step, the image in which the teacher label is set in the setting step is input to the machine learning device 1, and the setting condition is adjusted in accordance with the teacher label based on the output result for the input.

Therefore, in this construction method, the above-described steps are executed to adjust the setting conditions of the machine learning device 1, so that an image of an object having a predetermined feature is obtained from a group of image data with position information having an enormous amount of data. An object identification device that can extract information with high accuracy and high accuracy can be constructed.

In particular, the object identification device 10 according to the present embodiment can detect an “air conditioner” as an object. Here, there are millions of air conditioners installed outdoors, which is a huge number. Therefore, if the object is an air conditioner, a huge number of objects can still be detected even if the object detection rate is low (less than 50%). Therefore, when the object is an air conditioner, it is possible to generate useful information based on images of an enormous number of objects by using such an object identification device 10. Note that useful information having a high utility value even if an object is not detected includes location information of a product sold via an agent.

In addition, regarding the above-described effects, the same argument holds even when the object to be identified is not an air conditioner but a vending machine.

(3-2)
In addition, the object identification device that can identify an image in which an air conditioner having a feature related to any one of manufacturer, model, year, or any combination is imaged from image data with position information by the construction method according to the present embodiment. 10 can be built. In other words, according to the object identification device 10 constructed by the construction method according to the present embodiment, any one or any combination of the manufacturer, model, and year of the air conditioner can be identified.

Similarly, according to the object identification device 10 constructed by the construction method according to the present embodiment, an air conditioner having information on the type of refrigerant can be identified.

Similarly, according to the target object identification device 10 constructed by the construction method according to the present embodiment, it is possible to identify characteristics relating to deterioration of the air conditioner. Note that the characteristics related to deterioration are characteristics indicating the degree of deterioration and / or the type of deterioration. The “degradation type” includes those caused by aging degradation of an object and those caused by an abnormality occurring in the object. Specifically, types of deterioration include wear, cracks, breakage, disconnection, loosening (such as screws), dropout (such as screws), deformation, scratches, poor contact, missing parts, rust, corrosion, dirt, etc. Is mentioned.

In the construction method according to the present embodiment, it is possible to construct a target object identification device 10 that handles a plurality of features as a set and classifies each feature for each set.

(3-3)
In the construction method according to the present embodiment, the image data group with position information is generated from an image of a landscape photographed from the lateral direction and / or an image of a landscape photographed from above. Therefore, according to this method, the object identification apparatus 10 is based on a group of image data with position information that is composed of commonly used street view type photographs, ground photographs, aerial photographs, satellite photographs, and the like. Can be built.

(3-4)
Further, in the construction method according to the present embodiment, when learning the machine learning device 1, the portion where the object is projected is specified based on the coordinate information set in the image data with position information. Therefore, the object identification device 10 can be constructed using an image data group that is prohibited from processing and storing data. As a supplement, image data groups with location information generated from commonly used street view type photographs, ground photographs, aerial photographs, satellite photographs, etc. are prohibited from being saved after processing the data. There may be. Even in such a case, in the construction method according to the present embodiment, the portion where the object is copied is specified using the coordinate information, so the portion where the object is copied from the image data with position information in advance There is no need to cut out and save. In other words, in the construction method according to the present embodiment, the learning of the machine learning device 1 can be performed using the image data group with position information that is prohibited from storing data after processing.

The coordinate information and the image data with position information that is learning data are stored in the storage device 5. However, the information is not limited to this, and the information may be stored in the setting device 20 or the machine learning device 1.

(3-5)
In addition, the setting device 20 according to the present embodiment adjusts the setting conditions of the machine learning device 1 that executes machine learning, and has predetermined characteristics from an image data group with position information in which the position information is associated with individual image information. The object identification device 10 for identifying the object having Specifically, the setting device 20 includes an extraction unit 251, a setting unit 252, and an adjustment unit 253. The extraction unit 251 extracts an image from a part of the image data group with position information based on a rule base corresponding to a predetermined feature. The setting unit 252 sets a label indicating that it has a predetermined feature when an object having the predetermined feature is projected in the image extracted by the extraction unit 251. In the adjustment unit 253, the image in which the teacher label is set by the setting unit 252 is input to the machine learning device 1, and the setting condition is adjusted corresponding to the teacher label based on the output result with respect to the input.

Therefore, since the setting device 20 according to the present embodiment has the above configuration, the object identification device 10 can be constructed by adjusting the setting conditions of the machine learning device 1. By using such an object identification device 10, image information of an object having a specific feature can be extracted with high accuracy and high accuracy from a group of image data with position information having an enormous amount of data.

(4) Modification (4-1)
In the setting device 20 according to the present embodiment, the adjustment unit 253 may be capable of readjusting the setting conditions by inputting the image data with position information to the machine learning device 1 in which the setting conditions are adjusted.

Specifically, as shown in FIG. 9, the operator 3 selects image data detected from the object identification device 10 (T1) and adds it as new teacher data (T2). Then, the machine learning device 1 learns anew including these new teacher data, and adjusts the weight w between the units of each layer (T3).

Thereby, since the condition is readjusted using the machine learning device 1 in which the setting condition is adjusted, the setting condition of the machine learning device 1 can be further optimized. As a result, it is possible to construct the object identification device 10 that can extract the image information of the object with higher accuracy and accuracy.

(4-2)
In the above embodiment, the “air conditioner” is taken as an example of the object, but the present invention is not limited to this. That is, the object identification device constructed by the construction method according to the present invention identifies an arbitrary object acquired from the image data group with position information. For example, an arbitrary device constituting a part of a refrigeration apparatus that circulates a refrigerant in a refrigerant circuit can be identified as an object. Specifically, in addition to an air conditioner for cooling and heating, an air conditioner dedicated to cooling or heating, a floor heater, a hot water supply device, a dehumidifier, and the like can be given. Furthermore, the object is not limited to the refrigeration apparatus, and an apparatus such as a vending machine installed on the road, a satellite antenna installed on the roof of a building, a veranda, or the like can be identified.

(5) Application example By using the construction method of the object identification device according to the present embodiment, it is possible to apply to a further object identification method. Specifically, an object identification system including a plurality of object identification devices is constructed using the construction method according to the present embodiment, and an object identification method for identifying an object using the object identification system is provided. Can be provided.

FIG. 10 is a schematic diagram showing a concept of an object identification system according to an application example of the present embodiment.

The first object identification device 10A is constructed by causing the machine learning device 1 to learn the image data group with the first position information using the construction method described above. Here, the “first image data group with position information” is an image data group in which a landscape is photographed from a lateral direction in which position information is individually associated. Then, the first object identification device 10A acquires an arbitrary first image data group with position information, and identifies image data with position information in which an image of the object is projected. The identification result by the first object identification device 10A is sent to the determination device 15 described later.

The second object identification device 10B is constructed by causing the machine learning device 1 to learn the image data group with the second position information using the construction method described above. Here, the “second image data group with position information” is an image data group in which a landscape is photographed from above in which position information is individually associated. Then, the second object identification device 10B acquires an arbitrary image data group with position information, and identifies image data with position information in which an image of the object is copied. The identification result by the second object identification device 10B is sent to the determination device 15 described later.

The determination device 15 collates the identification result received from the first object identification device 10A and the identification result received from the second object identification device 10B, and the position information of the image data with position information included therein matches. If it is determined that the target object exists in the matched position information. That is, the determination device 15 identifies an object based on the logical product of the identification result of the first object identification device 10A and the identification result of the second object identification device 10B. Further, the determination device 15 reflects the matched position information on the map information, and displays the place where the object exists on the map (see FIG. 5).

If it is the target object identification method using the target object identification system mentioned above, each identification result of 10A of 1st target object identification apparatuses and the 2nd target object identification apparatus 10B constructed | assembled based on the image data group with a different positional information Therefore, the accuracy and accuracy of identifying the object can be improved. Furthermore, here, both the image data group in which the landscape is imaged from the lateral direction as the image data group with the first position information, and the image data group in which the landscape is imaged from above as the image data group with the second position information are included. Since the object is identified by using it, the accuracy and accuracy of identifying the object can be increased.

Note that the determination device 15 may identify an object based on a logical sum instead of a logical product of the identification result of the first object identification device 10A and the identification result of the second object identification device 10B. That is, the determination device 15 collates the identification result received from the first object identification device 10A and the identification result received from the second object identification device 10B, and the position information corresponding to any of the identification results. May be reflected on the map information, and the place where the object exists may be displayed on the map. In this case, the completeness of the detection position of the object can be improved.

<Appendix>
The present invention is not limited to the above embodiments as they are. The present invention can be embodied by modifying the components without departing from the scope of the invention in the implementation stage. Further, the present invention can form various inventions by appropriately combining a plurality of constituent elements disclosed in the above embodiments. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements may be appropriately combined in different embodiments.

DESCRIPTION OF SYMBOLS 1 Machine learning apparatus 2 Server apparatus 3 Worker 5 Storage apparatus 10 Object identification apparatus 10A 1st object identification apparatus 10B 2nd object identification apparatus 15 Determination apparatus 20 Setting apparatus 21 Input part 22 Output part 23 Acquisition part 24 Storage part 25 processing unit 251 extraction unit 252 setting unit 253 adjustment unit

Japanese Patent No. 5735071

Claims

Object identification for identifying an object having a predetermined characteristic from a group of image data with position information in which position information is associated with individual image information by adjusting setting conditions of the machine learning device (1) that performs machine learning A construction method for constructing a device (10), comprising:
An extraction step of extracting an image from a part of the image data group with the position information based on a rule base related to the object having the predetermined feature;
A setting step of setting a label indicating that the object having the predetermined characteristic is copied in the image extracted by the extraction step when the object having the predetermined characteristic is copied;
An adjustment step of inputting an image in which a label is set in the setting step to the machine learning device, and adjusting the setting condition corresponding to the label based on an output result with respect to the input;
A construction method comprising:
A readjustment step for readjusting the set conditions by inputting image data with position information to the machine learning device in which the set conditions are adjusted by the adjusting step;
The construction method according to claim 1, further comprising:
The object is a device constituting at least a part of a refrigeration apparatus for circulating a refrigerant in a refrigerant circuit.
The construction method according to claim 1 or 2.
The characteristics indicate information on the type of refrigerant.
The construction method according to claim 3.
The feature indicates information of manufacturer, model, year, or any combination,
The construction method according to any one of claims 1 to 4.
The feature is related to degradation,
The construction method according to any one of claims 1 to 5.
The image data group with position information is composed of an image of a landscape taken from the lateral direction and / or an image of a landscape taken from above.
The construction method according to any one of claims 1 to 6.
In the adjustment step, when a part of the image data group with position information is input to the machine learning device, a part in which the object is projected using the coordinate information set in each image data with position information Identify
The construction method according to any one of claims 1 to 7.
A storage device (5) used in the construction method according to claim 8,
A storage device that stores coordinate information set in each piece of image data with position information in association with the image data with position information.
Object identification for identifying an object having a predetermined characteristic from a group of image data with position information in which position information is associated with individual image information by adjusting setting conditions of the machine learning device (1) that performs machine learning A setting device (20) used to construct the device (10),
An extraction unit (251) for extracting an image from a part of the image data group with position information based on a rule base related to the object having the predetermined feature;
A setting unit (252) for setting a label indicating that the target object has the predetermined feature when the target object having the predetermined feature is copied in the image extracted by the extraction unit;
An adjustment unit (253) that inputs an image in which a label is set by the setting unit to the machine learning device, and adjusts the setting condition corresponding to the label based on an output result with respect to the input;
A setting device.
Using the construction method according to any one of claims 1 to 8, the first object identification device (10A) is constructed from the image data group with the first position information,
Using the construction method according to any one of claims 1 to 8, a second object identification device (10B) is generated from an image data group with second position information different from the image data group with first position information. Build and
An object identification method for identifying the object using the first object identification device and the second object identification device.
One of the image data group with the first position information or the image data group with the second position information is an image data group in which a landscape is photographed from the lateral direction in which the position information is individually associated, and the other is , Is a group of image data in which the landscape is photographed from above where the position information is individually associated.
The method for identifying an object according to claim 11.