CN112949423A - Object recognition method, object recognition device, and robot - Google Patents
Object recognition method, object recognition device, and robot Download PDFInfo
- Publication number
- CN112949423A CN112949423A CN202110168772.6A CN202110168772A CN112949423A CN 112949423 A CN112949423 A CN 112949423A CN 202110168772 A CN202110168772 A CN 202110168772A CN 112949423 A CN112949423 A CN 112949423A
- Authority
- CN
- China
- Prior art keywords
- image
- object recognition
- target image
- processed
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 57
- 238000007781 pre-processing Methods 0.000 claims abstract description 29
- 238000001514 detection method Methods 0.000 claims description 83
- 238000012549 training Methods 0.000 claims description 25
- 238000004590 computer program Methods 0.000 claims description 23
- 230000009471 action Effects 0.000 claims description 9
- 230000000717 retained effect Effects 0.000 claims description 9
- 238000005286 illumination Methods 0.000 claims description 8
- 238000009877 rendering Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 description 8
- 238000012545 processing Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000002203 pretreatment Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20132—Image cropping
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Geometry (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Manipulator (AREA)
Abstract
The application is applicable to the technical field of robots, and provides an object identification method, an object identification device and a robot, which comprises the following steps: preprocessing an image to be processed to obtain a target image, wherein the preprocessing comprises at least one of the following steps: detecting definition, chroma and brightness; if the target image meets the preset condition, carrying out object recognition on the target image by adopting a local object recognition algorithm; and outputting a corresponding object recognition result. By the method, the object can be identified off line.
Description
Technical Field
The present application relates to the field of robotics, and in particular, to an object recognition method, an object recognition apparatus, a robot, and a computer-readable storage medium.
Background
The existing object recognition algorithm generally directly recognizes an image and is deployed in the cloud. Because the image is directly identified, the conclusion of error identification is easily obtained; moreover, since the object recognition algorithm is deployed in the cloud, the object recognition algorithm may be limited by network conditions, for example, when the network is not good, the function experience of object recognition is not good, and when the network is not available, the object recognition function may not be used.
Therefore, it is necessary to provide a new method to solve the above technical problems.
Disclosure of Invention
The embodiment of the application provides an object identification method, so that a robot can identify an object off line.
In a first aspect, an embodiment of the present application provides an object identification method, applied to a robot, including:
preprocessing an image to be processed to obtain a target image, wherein the preprocessing comprises at least one of the following steps: detecting definition, chroma and brightness;
if the target image meets the preset condition, carrying out object recognition on the target image by adopting a local object recognition algorithm;
and outputting a corresponding object recognition result.
In a second aspect, an embodiment of the present application provides an object recognition apparatus, which is applied to a robot, and includes:
the target image determining unit is used for preprocessing an image to be processed to obtain a target image, and the preprocessing comprises at least one of the following steps: detecting definition, chroma and brightness;
the object recognition unit is used for recognizing the object of the target image by adopting a local object recognition algorithm if the target image meets the preset condition;
and the object recognition result output unit is used for outputting a corresponding object recognition result.
In a third aspect, embodiments of the present application provide a robot, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the method according to the first aspect when executing the computer program.
In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the computer program implements the method according to the first aspect.
In a fifth aspect, the present application provides a computer program product, which when run on a terminal device, causes the terminal device to execute the method of the first aspect.
Compared with the prior art, the embodiment of the application has the advantages that:
in the embodiment of the application, an image to be processed is preprocessed to obtain a target image, if the target image meets a preset condition, a local object recognition algorithm is adopted to recognize an object of the target image, and a corresponding object recognition result is output. Wherein the pre-treatment comprises at least one of: the image recognition method comprises the steps of definition detection, chrominance detection and luminance detection, namely, in the embodiment of the application, only the to-be-processed image detected by at least one of the definition detection, the chrominance detection and the luminance detection is subjected to object recognition, and the to-be-processed image meeting the preset condition has higher image quality, so that the to-be-processed image subjected to the detection is subjected to object recognition, and the accuracy of the obtained object recognition result can be improved. In addition, since the object recognition algorithm is a local algorithm, recognition of the object can be achieved even off-line.
It is understood that the beneficial effects of the second aspect to the fifth aspect can be referred to the related description of the first aspect, and are not described herein again.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the embodiments or the description of the prior art will be briefly described below.
Fig. 1 is a schematic flowchart of a first object identification method according to an embodiment of the present application;
fig. 2 is a schematic flow chart of a second object identification method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a target image having 2 subjects according to an embodiment of the present application;
FIG. 4 is a schematic flow chart illustrating a third object recognition method according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of an object recognition apparatus according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a robot according to an embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise.
In an existing object recognition algorithm, object recognition is usually performed on an image directly, and the image is deployed in a cloud. Since the robot is usually in a motion state, an image captured by the robot in the motion state is likely to be a blurred image, and if the image captured by the robot is directly recognized, a wrong recognition result is likely to be obtained. In addition, the robot in the motion state may also be in an environment with poor network conditions, and at this time, the robot is difficult to upload images obtained by shooting the robot to the cloud in time, or the cloud is difficult to issue the obtained object recognition results to the robot in time. In either case, it is difficult for the robot to obtain the object recognition result of the image captured by the robot in time. In order to solve the above technical problem, an embodiment of the present application provides an object recognition method, in which an object recognition algorithm is deployed on a robot, the robot performs a shooting action to obtain a shot image, and then performs preprocessing such as sharpness detection, chrominance detection, and/or luminance detection on the image, and only when the preprocessed image meets a preset condition, a local object recognition algorithm is used to perform corresponding object recognition.
The following describes an object identification method provided in an embodiment of the present application with reference to the drawings.
Fig. 1 shows a flowchart of a first object identification method provided in an embodiment of the present application, which is applied to a robot and detailed as follows:
step S11, preprocessing the image to be processed to obtain a target image, where the preprocessing includes at least one of: definition detection, chrominance detection and luminance detection.
The image to be processed is an image obtained after the robot performs a shooting action (an action of taking a picture or an action of taking a video stream) (the image obtained after the robot performs the shooting action is subsequently referred to as an original image), or the image to be processed is an image obtained after the original image is subjected to certain processing.
In some embodiments, the image to be processed is an image frame in a video stream captured by the robot.
Specifically, (1) if the image to be processed is one image frame in a video stream shot by a robot, considering that adjacent image frames are usually similar and dissimilar, a shaking phenomenon exists, and a shaken image is usually an unclear image, at this time, the definition of the image to be processed is detected by the following method: and acquiring the last image frame of the image to be processed in the video stream, namely the image to be compared. Comparing the image to be processed with the image to be compared to obtain the difference of the pixel value of the image to be processed and the pixel value of the image to be compared on each pixel point, calculating the mean value of the difference of the pixel value of each pixel point, if the mean value of the difference of the pixel value of each pixel point is larger than a preset similarity threshold value, judging that the definition of the image to be processed meets a preset definition condition, and otherwise, judging that the definition of the image to be processed does not meet the preset definition condition. (2) If the image to be processed is an image obtained by the robot taking a picture, considering that the probability that the robot takes a pure-color image is very small, and when the image has a blur in a certain area, the pixel values of the area are usually very close, at this time, the sharpness of the image to be processed is detected by the following method: determining a region to be processed in an image to be processed through a preset window, respectively calculating the difference between the pixel value of the pixel and the pixel values of other pixels in the region to be processed for each pixel in the region to be processed, moving the preset window according to a preset step length and a preset sequence (for example, moving from the left side of the image to be processed to the right side of the image to be processed and then moving from the upper side of the image to be processed to the lower side of the image to be processed) after obtaining a calculation result, respectively calculating the difference between the pixel value of the pixel and the pixel values of other pixels in the new region to be processed for each pixel in the new region to be processed, obtaining another calculation result, returning to the step of moving the preset window according to the preset step length and subsequent steps until all regions of the image to be processed are covered by the movement of the preset window, and obtaining each calculation result, counting the average value of the sum of each calculation result, if the average value of the sum of each calculation result is greater than a preset ambiguity threshold value, judging that the definition of the image to be processed does not meet a preset definition condition, otherwise, judging that the definition of the image to be processed meets the preset definition condition.
Specifically, the chromaticity detection is performed on the image to be processed in the following manner: the method comprises the steps of obtaining the chromatic value of each pixel point in an image to be processed, judging whether the chromatic value of each pixel point is within a preset chromatic value range, if so, judging that the chromaticity of the image to be processed meets a preset chromaticity condition, and otherwise, judging that the chromaticity of the image to be processed does not meet the preset chromaticity condition.
Specifically, the luminance detection is performed on the image to be processed in the following manner: the method comprises the steps of obtaining the brightness value of each pixel point in an image to be processed, judging that the brightness of the image to be processed meets a preset brightness condition if the maximum brightness value is smaller than a preset maximum brightness threshold value and the minimum brightness value is smaller than a preset minimum brightness threshold value, and otherwise, judging that the brightness of the image to be processed does not meet the preset brightness condition.
Before the above step S11, the method includes: and cutting out the central area of the original image according to a preset proportion, taking the cut image as an image to be processed, and taking the original image as an image obtained after the robot executes shooting action.
The preset ratio is a ratio of the central region to the original image, for example, assuming that the preset ratio is 1/3, the length of the central region is 1/3, and the width of the central region is 1/3. It should be noted that the center point of the central area is the center point of the original image, that is, the central point of the original image is taken as the center and extends in the left, right, up and down directions of the original image, the length value of the central area obtained after extending in the left and right directions is taken as the width of the central area, and the length value of the central area obtained after extending in the up and down directions is taken as the length of the central area.
In the embodiment, the robot only concerns the object in the center of the visual field, so that the central area of the original image is cut out to be used as the image to be processed, and an irrelevant background can be removed, thereby being beneficial to improving the accuracy of the subsequently obtained object recognition result.
In some embodiments, the cutting out the central area of the original image according to a preset scale, and the cutting out the obtained image as the image to be processed specifically includes: if the color mode of the original image is not the target mode, converting the color mode of the original image into the target mode, cutting out the central area of the original image converted into the target mode according to a preset proportion, and taking the cut image as an image to be processed.
In some embodiments, the cutting out the central area of the original image according to a preset scale, and the cutting out the obtained image as the image to be processed specifically includes: cutting out the central area of the original image according to a preset proportion, if the color mode of the cut image is not the target mode, converting the color mode of the cut image into the target mode, and converting the color mode of the cut image into the cut image of the target mode to be used as the image to be processed.
And step S12, if the target image meets the preset conditions, carrying out object recognition on the target image by using a local object recognition algorithm.
The local object recognition algorithm is an object recognition algorithm based on deep learning and running on the robot, that is, the object recognition algorithm of the embodiment is an offline object recognition algorithm, and even if the robot is disconnected from the cloud, the robot can also recognize objects in the image.
In step S13, the corresponding object recognition result is output.
Specifically, an object recognition result is output in the form of voice or text, and the object recognition result is used for indicating the category to which the object belongs.
In the embodiment of the application, the image to be processed is preprocessed to obtain the target image, if the target image meets the preset condition, the local object recognition algorithm is adopted to recognize the object of the target image, and the corresponding object recognition result is output. Wherein the pre-treatment comprises at least one of: the image recognition method comprises the steps of definition detection, chrominance detection and luminance detection, namely, in the embodiment of the application, only the to-be-processed image detected by at least one of the definition detection, the chrominance detection and the luminance detection is subjected to object recognition, and the to-be-processed image meeting the preset condition has higher image quality, so that the to-be-processed image subjected to the detection is subjected to object recognition, and the accuracy of the obtained object recognition result can be improved. In addition, since the object recognition algorithm is a local algorithm, the recognition of the object can be realized even if the robot is in an off-line state.
Fig. 2 shows a schematic flowchart of a second object identification method provided in an embodiment of the present application, in this embodiment, step S22 and step S23 are further described for step S12, and step S21 and step S24 are respectively the same as step S11 and step S13, and are not repeated here.
Step S21, preprocessing the image to be processed to obtain a target image, where the preprocessing includes at least one of: definition detection, chrominance detection and luminance detection.
And step S22, if the target image meets the preset conditions, performing subject detection on the target image by using a local object recognition algorithm.
It should be noted that the above object recognition algorithm includes 2 parts, one part is object detection, and the other part is object classification of the obtained subject.
Specifically, an object detection algorithm in a local object recognition algorithm is adopted to perform object detection on a target image, so as to obtain a detection result of an object included in the target image, wherein the detection result includes an object frame, a category corresponding to the object frame, and a confidence coefficient of the category corresponding to the object frame. And judging whether the confidence of the category corresponding to the object frame of the object is greater than a preset confidence threshold, if so, retaining the detection result of the object, namely, the object in the object frame in the detection result of the object is the main body of the target image. If the confidence coefficient is not greater than the preset confidence coefficient threshold value, which is equivalent to that the target image has no protruding object, the fact that the main body does not exist in the image is indicated, and at the moment, information of the undetected object can be output.
In step S23, if the subject is present in the target image, the subject is recognized.
In this embodiment, when it is determined that the target image has the subject, the trained multi-class classification network is used to perform object recognition on the subject.
In some embodiments, since the target image may have a plurality of subjects, at this time, step S23 includes:
and A1, if at least 2 subjects exist in the target image, keeping the subject closest to the center of the target image.
And A2, performing object recognition on the reserved main body.
In the above-described a1 and a2, it is considered that the robot focuses only on the object at the center of its field of view, and therefore, in order to quickly realize the recognition of the object, only the subject closest to the center of the target image is reserved as the subject to be recognized. As shown in fig. 3, there are 2 subjects whose target images are detected: the subject 1 and the subject 2, and the subject 1 is closer to the center point of the target image, so the subject 1 is retained, and the subject 1 will be subjected to object recognition later.
In some embodiments, step a2 specifically includes:
and adjusting the image corresponding to the reserved main body to a preset size, and performing object recognition on the image corresponding to the reserved main body after size adjustment, wherein the preset size is equal to the size of a training sample adopted by an object recognition algorithm.
Specifically, if the object recognition algorithm includes 2 parts, one part is object detection, and the other part is object classification of the obtained subject, that is, the object recognition model corresponding to the object recognition algorithm includes 2 parts, one part is a network for object detection, and the other part is a classification network for object classification of the obtained subject. The training sample size herein refers to the size of a training sample used to train a classification network for object classification. For example, if the training sample size used to train the classification network is 224 × 224, the preset size is also 224 × 224 here. That is, if the image size of the image corresponding to the retained subject is different from the training sample size (e.g., larger than the training sample size or smaller than the training sample size), the image corresponding to the retained subject is resized.
In this embodiment, the size of the image corresponding to the reserved main body is adjusted to be the preset size, and the preset size is the same as the size of the training sample, so that the classification network is favorable for more accurate object identification on the object.
In some embodiments, considering that the object that the robot wishes to recognize is not necessarily right at the center of the target image but not far from the center, the main body that needs to be retained may be determined by combining the area of the main body and the distance from the center point, specifically, step S23 includes:
a 1', if there are at least 2 subjects in the target image, determining two subjects closest to the center of the target image, assuming that the determined subjects are a first subject and a second subject, the distance between the first subject and the center of the target image is a first distance, the distance between the second subject and the center of the target image is a second distance, the area of the object frame of the first subject is a first area, the area of the object frame of the second subject is a second area, determining the ratio of the first area to the first distance, and determining the ratio of the second area to the second distance, leaving the subject corresponding to the larger ratio.
Specifically, the larger the area of the object frame is, the closer the distance to the center is, the larger the obtained ratio is, and conversely, the smaller the obtained ratio is.
A 2', performing object recognition on the retained subject.
In the above-mentioned a1 'and a 2', the areas of the object frames of different subjects are determined, the distances between the different subjects and the center point are determined, the ratio of the area to the distance of the same subject is determined, and finally the subject with the largest ratio is retained. When the ratio is the maximum, the area of the object frame representing the main body is larger and is closer to the central point, which means that the main body is more likely to be the main body to be recognized by the robot, and therefore, the accuracy of the obtained object recognition result can be improved only by recognizing the main body of the type.
In step S24, the corresponding object recognition result is output.
In the embodiment of the application, the subject detection is performed on the target image, the subject is identified only after the subject is detected, and the subject is not directly identified, so that the category to which the background information of the target image belongs can be shielded, and the robot only focuses on the category to which the subject belongs when performing the subject identification, so that the subject is identified after the subject detection is performed on the target image, and the obtained subject identification result can be ensured to be more accurate, and the robot more meets the requirements of the robot.
Fig. 4 is a schematic flowchart illustrating a third object identification method provided in this embodiment of the application, in this embodiment, step S42 is a further description of step S12, and step S41 and step S43 are the same as step S11 and step S13, respectively, and are not repeated here.
Step S41, preprocessing the image to be processed to obtain a target image, where the preprocessing includes at least one of: definition detection, chrominance detection and luminance detection.
And step S42, if the target image meets the preset condition, adjusting the target image to a preset size, and performing object recognition on the size-adjusted target image by using a local object recognition algorithm, wherein the preset size is equal to the size of the training sample used by the object recognition algorithm.
In this embodiment, if the object recognition algorithm only includes a classification network for classifying the object of the image, the preset size is a size of a training sample for training the classification network. If the object recognition model corresponding to the object recognition algorithm includes 2 parts, one part is a network for object detection, and the other part is a classification network for classifying the obtained subject, the preset size is the size of a training sample for training the network for object detection. Of course, if the training samples for training the network for object detection and the network for object classification are the same, the size of the training sample herein also refers to the size of the training sample for training the classification network for object classification.
It should be noted that the process of performing object recognition on the resized target image by using the local object recognition algorithm is the same as the process of steps S22 and S23, that is, the local object recognition algorithm performs object recognition on the resized target image after subject recognition.
In step S43, the corresponding object recognition result is output.
In the embodiment of the application, the size of the target image is adjusted to be the preset size, and the preset size is equal to the size of the training sample adopted by the object recognition algorithm, so that a more accurate object recognition result is obtained.
In some embodiments, if the preprocessing includes sharpness detection, chroma detection and luminance detection, and the preset conditions include sharpness conditions, chroma conditions and luminance conditions, the step S11 (or step S21 or step S41) includes:
and B1, performing definition detection, chrominance detection and luminance detection on the image to be processed.
B2, if the definition and the chromaticity of the image to be processed respectively accord with the definition condition and the chromaticity condition, but the brightness of the image to be processed does not accord with the brightness condition, judging whether the brightness of the image to be processed is in the adjustable range, if the brightness of the image to be processed is in the adjustable range, performing high dynamic illumination rendering on the image to be processed, and enabling the image to be processed after the high dynamic illumination rendering to accord with the definition condition, the chromaticity condition and the brightness condition.
Specifically, if the brightness of the image to be processed is within the adjustable Range, it indicates that the maximum brightness value corresponding to the brightness of the image to be processed is smaller than the preset maximum brightness threshold after the brightness of the image to be processed is subjected to High-Dynamic Range (HDR), and the minimum brightness value is smaller than the preset minimum brightness threshold.
In some embodiments, if the definition of the image to be processed does not meet the definition condition, or the chromaticity of the image to be processed does not meet the chromaticity condition, or the luminance of the image to be processed is not within the adjustable range, the image to be processed is not identified, that is, the robot refuses to identify the image to be processed.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.
Fig. 5 shows a block diagram of an object recognition device provided in an embodiment of the present application, which corresponds to the object recognition method in the above embodiment, and only shows a part related to the embodiment of the present application for convenience of description when the object recognition device is applied to a robot.
Referring to fig. 5, the object recognition device 5 includes: a target image determination unit 51, an object recognition unit 52, and an object recognition result output unit 53. Wherein:
the target image determining unit 51 is configured to perform preprocessing on the image to be processed to obtain a target image, where the preprocessing includes at least one of: definition detection, chrominance detection and luminance detection.
The definition detection, the chromaticity detection, the brightness detection and the like are described in detail in the above object identification method, and are not described herein again.
And the object recognition unit 52 is configured to perform object recognition on the target image by using a local object recognition algorithm if the target image meets the preset condition.
And an object recognition result output unit 53 for outputting a corresponding object recognition result.
In the embodiment of the application, only the to-be-processed image detected by at least one of definition detection, chromaticity detection and brightness detection is subjected to object recognition, and the image quality of the to-be-processed image meeting the preset condition is high, so that the to-be-processed image subjected to detection is subjected to object recognition, and the accuracy of the obtained object recognition result can be improved. In addition, since the object recognition algorithm is a local algorithm, recognition of the object can be achieved even off-line.
In some embodiments, the object identification unit 52 comprises a subject detection module and a subject presence processing module, wherein:
and the main body detection module is used for carrying out main body detection on the target image by adopting a local object recognition algorithm if the target image meets the preset condition.
And the main body existence processing module is used for carrying out object recognition on the main body if the target image has the main body.
In some embodiments, the subject presence processing module comprises:
and the central main body reserving module is used for reserving a main body closest to the center of the target image if at least 2 main bodies exist in the target image.
And the first reserved main body identification module is used for carrying out object identification on the reserved main body.
In some embodiments, the first retention body identification module is specifically configured to:
and adjusting the image corresponding to the reserved main body to a preset size, and performing object recognition on the image corresponding to the reserved main body after size adjustment, wherein the preset size is equal to the size of a training sample adopted by an object recognition algorithm.
In some embodiments, the subject presence processing module comprises a ratio determination module and a second retained subject identification module, wherein:
the ratio determining module is used for determining two subjects closest to the center of the target image if the target image has at least 2 subjects, assuming that the determined subjects are a first subject and a second subject, the distance between the first subject and the center of the target image is a first distance, the distance between the second subject and the center of the target image is a second distance, the area of an object frame of the first subject is a first area, the area of an object frame of the second subject is a second area, determining the ratio of the first area to the first distance, determining the ratio of the second area to the second distance, and reserving the subject corresponding to the larger ratio.
And the second reserved main body identification module is used for carrying out object identification on the reserved main body.
In some embodiments, the object identification unit 52 is specifically configured to:
and if the target image meets the preset condition, adjusting the target image to a preset size, and performing object recognition on the size-adjusted target image by adopting a local object recognition algorithm, wherein the preset size is equal to the size of a training sample adopted by the object recognition algorithm.
In some embodiments, the object recognition device 5 includes:
and the cutting unit is used for cutting out the central area of the original image according to a preset proportion, the cut image is used as an image to be processed, and the original image is an image obtained after the robot performs shooting action.
The preset proportion refers to the proportion of the central area to the original image.
In some embodiments, the clipping unit is specifically configured to: if the color mode of the original image is not the target mode, converting the color mode of the original image into the target mode, cutting out the central area of the original image converted into the target mode according to a preset proportion, and taking the cut image as an image to be processed.
In some embodiments, the clipping unit is specifically configured to: cutting out the central area of the original image according to a preset proportion, if the color mode of the cut image is not the target mode, converting the color mode of the cut image into the target mode, and converting the color mode of the cut image into the cut image of the target mode to be used as the image to be processed.
In some embodiments, if the preprocessing includes sharpness detection, chroma detection and luminance detection, and the preset conditions include sharpness conditions, chroma conditions and luminance conditions, the target image determining unit 51 includes:
and the definition detection module is used for performing definition detection, chrominance detection and brightness detection on the image to be processed.
And the brightness processing module is used for judging whether the brightness of the image to be processed is within an adjustable range if the definition and the chromaticity of the image to be processed respectively accord with the definition condition and the chromaticity condition but the brightness of the image to be processed does not accord with the brightness condition, performing high-dynamic illumination rendering on the image to be processed if the brightness of the image to be processed is within the adjustable range, and enabling the image to be processed after the high-dynamic illumination rendering to accord with the definition condition, the chromaticity condition and the brightness condition.
It should be noted that, for the information interaction, execution process, and other contents between the above-mentioned devices/units, the specific functions and technical effects thereof are based on the same concept as those of the embodiment of the method of the present application, and specific reference may be made to the part of the embodiment of the method, which is not described herein again.
Fig. 6 is a schematic structural diagram of a robot according to an embodiment of the present application. As shown in fig. 6, the robot 6 of this embodiment includes: at least one processor 60 (only one processor is shown in fig. 6), a memory 61, and a computer program 62 stored in the memory 61 and executable on the at least one processor 60, the steps in any of the various method embodiments described above being implemented when the computer program 62 is executed by the processor 60:
preprocessing an image to be processed to obtain a target image, wherein the preprocessing comprises at least one of the following steps: detecting definition, chroma and brightness;
if the target image meets the preset condition, performing object recognition on the target image by adopting a local object recognition algorithm;
and outputting a corresponding object recognition result.
Optionally, if the target image meets the preset condition, performing object recognition on the target image by using a local object recognition algorithm, including:
if the target image meets the preset conditions, performing main body detection on the target image by adopting a local object recognition algorithm;
and if the target image has the main body, performing object recognition on the main body.
Optionally, if the target image has a subject, performing object recognition on the subject, including:
if the target image has at least 2 subjects, retaining the subject closest to the center of the target image;
and carrying out object identification on the reserved main body.
Optionally, performing object recognition on the retained subject, including:
and adjusting the image corresponding to the reserved main body to a preset size, and performing object recognition on the image corresponding to the reserved main body after size adjustment, wherein the preset size is equal to the size of a training sample adopted by an object recognition algorithm.
Optionally, if the target image meets the preset condition, performing object recognition on the target image by using a local object recognition algorithm, including:
and if the target image meets the preset condition, adjusting the target image to a preset size, and performing object recognition on the size-adjusted target image by adopting a local object recognition algorithm, wherein the preset size is equal to the size of a training sample adopted by the object recognition algorithm.
Optionally, before preprocessing the image to be processed, the method includes:
and cutting out the central area of the original image according to a preset proportion, taking the cut image as an image to be processed, and taking the original image as an image obtained after the robot executes shooting action.
Optionally, if the preprocessing includes sharpness detection, chrominance detection, and luminance detection, and the preset condition includes a sharpness condition, a chrominance condition, and a luminance condition, the preprocessing the image to be processed includes:
performing definition detection, chrominance detection and luminance detection on an image to be processed;
if the definition and the chromaticity of the image to be processed respectively accord with the definition condition and the chromaticity condition, but the brightness of the image to be processed does not accord with the brightness condition, judging whether the brightness of the image to be processed is in the adjustable range, if the brightness of the image to be processed is in the adjustable range, performing high dynamic illumination rendering on the image to be processed, and enabling the image to be processed after the high dynamic illumination rendering to accord with the definition condition, the chromaticity condition and the brightness condition.
The robot 6 may be a humanoid robot or a robot of other shape. The robot may include, but is not limited to, a processor 60, a memory 61. Those skilled in the art will appreciate that fig. 6 is merely an example of the robot 6, and does not constitute a limitation on the robot 6, and may include more or less components than those shown, or combine some of the components, or different components, such as input and output devices, network access devices, etc.
The Processor 60 may be a Central Processing Unit (CPU), and the Processor 60 may be other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 61 may in some embodiments be an internal storage unit of the robot 6, such as a hard disk or a memory of the robot 6. The memory 61 may also be an external storage device of the robot 6 in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the robot 6. Further, the memory 61 may also include both an internal storage unit and an external storage device of the robot 6. The memory 61 is used for storing an operating system, an application program, a BootLoader (BootLoader), data, and other programs, such as program codes of the computer program. The memory 61 may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
An embodiment of the present application further provides a network device, where the network device includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, the processor implementing the steps of any of the various method embodiments described above when executing the computer program.
The embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the computer program implements the steps in the above-mentioned method embodiments.
The embodiments of the present application provide a computer program product, which when running on a mobile terminal, enables the mobile terminal to implement the steps in the above method embodiments when executed.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium and can implement the steps of the embodiments of the methods described above when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code to a photographing device/robot, a recording medium, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), an electrical carrier wave signal, a telecommunications signal, and a software distribution medium. Such as a usb-disk, a removable hard disk, a magnetic or optical disk, etc. In certain jurisdictions, computer-readable media may not be an electrical carrier signal or a telecommunications signal in accordance with legislative and patent practice.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/network device and method may be implemented in other ways. For example, the above-described apparatus/network device embodiments are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implementing, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.
Claims (10)
1. An object recognition method, applied to a robot, includes:
preprocessing an image to be processed to obtain a target image, wherein the preprocessing comprises at least one of the following steps: detecting definition, chroma and brightness;
if the target image meets the preset condition, carrying out object recognition on the target image by adopting a local object recognition algorithm;
and outputting a corresponding object recognition result.
2. The object recognition method according to claim 1, wherein if the target image meets a preset condition, performing object recognition on the target image by using a local object recognition algorithm includes:
if the target image meets the preset condition, performing main body detection on the target image by adopting a local object recognition algorithm;
and if the target image has a main body, carrying out object recognition on the main body.
3. The object recognition method according to claim 2, wherein the performing object recognition on the subject if the subject exists in the target image comprises:
if the target image has at least 2 subjects, retaining the subject closest to the center of the target image;
and carrying out object identification on the reserved main body.
4. The object recognition method of claim 3, wherein the performing object recognition on the retained subject comprises:
adjusting the image corresponding to the reserved main body to a preset size, and performing object recognition on the image corresponding to the reserved main body after size adjustment, wherein the preset size is equal to the size of a training sample adopted by the object recognition algorithm.
5. The object recognition method according to claim 1, wherein if the target image meets a preset condition, performing object recognition on the target image by using a local object recognition algorithm includes:
and if the target image meets the preset condition, adjusting the target image to a preset size, and performing object recognition on the target image after size adjustment by adopting a local object recognition algorithm, wherein the preset size is equal to the size of a training sample adopted by the object recognition algorithm.
6. The object recognition method according to any one of claims 1 to 5, wherein before the preprocessing the image to be processed, comprising:
and cutting out a central area of an original image according to a preset proportion, wherein the cut image is used as the image to be processed, and the original image is obtained after the robot executes shooting action.
7. The object recognition method according to any one of claims 1 to 5, wherein if the preprocessing includes sharpness detection, chroma detection, and luminance detection, and the preset conditions include sharpness conditions, chroma conditions, and luminance conditions, the preprocessing the image to be processed includes:
performing definition detection, chrominance detection and luminance detection on an image to be processed;
if the definition and the chromaticity of the image to be processed respectively accord with the definition condition and the chromaticity condition, but the brightness of the image to be processed does not accord with the brightness condition, judging whether the brightness of the image to be processed is within an adjustable range, if so, performing high dynamic illumination rendering on the image to be processed, and enabling the image to be processed after the high dynamic illumination rendering to accord with the definition condition, the chromaticity condition and the brightness condition.
8. An object recognition device, applied to a robot, comprising:
the target image determining unit is used for preprocessing an image to be processed to obtain a target image, and the preprocessing comprises at least one of the following steps: detecting definition, chroma and brightness;
the object recognition unit is used for recognizing the object of the target image by adopting a local object recognition algorithm if the target image meets the preset condition;
and the object recognition result output unit is used for outputting a corresponding object recognition result.
9. A robot comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110168772.6A CN112949423B (en) | 2021-02-07 | 2021-02-07 | Object recognition method, object recognition device and robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110168772.6A CN112949423B (en) | 2021-02-07 | 2021-02-07 | Object recognition method, object recognition device and robot |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112949423A true CN112949423A (en) | 2021-06-11 |
| CN112949423B CN112949423B (en) | 2024-05-24 |
Family
ID=76243277
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110168772.6A Active CN112949423B (en) | 2021-02-07 | 2021-02-07 | Object recognition method, object recognition device and robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112949423B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113505700A (en) * | 2021-07-12 | 2021-10-15 | 北京字跳网络技术有限公司 | Image processing method, device, equipment and storage medium |
| CN114240870A (en) * | 2021-12-06 | 2022-03-25 | 杭州玖众科技有限公司 | Picture processing method |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110175980A (en) * | 2019-04-11 | 2019-08-27 | 平安科技（深圳）有限公司 | Image definition recognition methods, image definition identification device and terminal device |
| CN110335216A (en) * | 2019-07-09 | 2019-10-15 | Oppo广东移动通信有限公司 | Image processing method, image processing apparatus, terminal device and readable storage medium storing program for executing |
| CN111222419A (en) * | 2019-12-24 | 2020-06-02 | 深圳市优必选科技股份有限公司 | Object identification method, robot and computer readable storage medium |
| CN111797835A (en) * | 2020-06-01 | 2020-10-20 | 深圳市识农智能科技有限公司 | Disease identification method, disease identification device and terminal equipment |
| CN111861965A (en) * | 2019-04-08 | 2020-10-30 | Tcl集团股份有限公司 | Image backlight detection method, image backlight detection device and terminal equipment |
| WO2020252917A1 (en) * | 2019-06-20 | 2020-12-24 | 平安科技（深圳）有限公司 | Fuzzy face image recognition method and apparatus, terminal device, and medium |
| CN112183212A (en) * | 2020-09-01 | 2021-01-05 | 深圳市识农智能科技有限公司 | Weed identification method and device, terminal equipment and readable storage medium |
-
2021
- 2021-02-07 CN CN202110168772.6A patent/CN112949423B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111861965A (en) * | 2019-04-08 | 2020-10-30 | Tcl集团股份有限公司 | Image backlight detection method, image backlight detection device and terminal equipment |
| CN110175980A (en) * | 2019-04-11 | 2019-08-27 | 平安科技（深圳）有限公司 | Image definition recognition methods, image definition identification device and terminal device |
| WO2020252917A1 (en) * | 2019-06-20 | 2020-12-24 | 平安科技（深圳）有限公司 | Fuzzy face image recognition method and apparatus, terminal device, and medium |
| CN110335216A (en) * | 2019-07-09 | 2019-10-15 | Oppo广东移动通信有限公司 | Image processing method, image processing apparatus, terminal device and readable storage medium storing program for executing |
| CN111222419A (en) * | 2019-12-24 | 2020-06-02 | 深圳市优必选科技股份有限公司 | Object identification method, robot and computer readable storage medium |
| CN111797835A (en) * | 2020-06-01 | 2020-10-20 | 深圳市识农智能科技有限公司 | Disease identification method, disease identification device and terminal equipment |
| CN112183212A (en) * | 2020-09-01 | 2021-01-05 | 深圳市识农智能科技有限公司 | Weed identification method and device, terminal equipment and readable storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113505700A (en) * | 2021-07-12 | 2021-10-15 | 北京字跳网络技术有限公司 | Image processing method, device, equipment and storage medium |
| CN114240870A (en) * | 2021-12-06 | 2022-03-25 | 杭州玖众科技有限公司 | Picture processing method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112949423B (en) | 2024-05-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3327668B1 (en) | Image-processing device and method for foreground mask correction for object segmentation | |
| WO2020140610A1 (en) | Image processing method and device, and computer-readable storage medium | |
| US10949984B2 (en) | Object tracking system and method thereof | |
| CN112348778B (en) | Object identification method, device, terminal equipment and storage medium | |
| CN109711407B (en) | License plate recognition method and related device | |
| CN111079613B (en) | Gesture recognition method and device, electronic equipment and storage medium | |
| CN109214996B (en) | Image processing method and device | |
| CN109005367B (en) | High dynamic range image generation method, mobile terminal and storage medium | |
| CN108805838B (en) | Image processing method, mobile terminal and computer readable storage medium | |
| WO2020140611A1 (en) | Vin code identification method based on image processing, device and medium | |
| CN110599516A (en) | Moving target detection method and device, storage medium and terminal equipment | |
| CN112949423A (en) | Object recognition method, object recognition device, and robot | |
| US20230127009A1 (en) | Joint objects image signal processing in temporal domain | |
| CN115115611B (en) | Vehicle damage identification method and device, electronic equipment and storage medium | |
| CN111080665B (en) | Image frame recognition method, device, equipment and computer storage medium | |
| US20160180545A1 (en) | Method and electronic device for object tracking in a light-field capture | |
| CN114359789B (en) | Target detection method, device, equipment and medium for video image | |
| CN111882565A (en) | Image binarization method, device, equipment and storage medium | |
| CN111027564A (en) | Low-illumination imaging license plate recognition method and device based on deep learning integration | |
| CN112036342B (en) | Document snapshot method, device and computer storage medium | |
| CN112052726A (en) | Image processing method and device | |
| CN116403200A (en) | License plate real-time identification system based on hardware acceleration | |
| CN114037741B (en) | Self-adaptive target detection method and device based on event camera | |
| CN111476056A (en) | Target object identification method and device, terminal equipment and computer storage medium | |
| Gurrala et al. | Enhancing Safety and Security: Face Tracking and Detection in Dehazed Video Frames Using KLT and Viola-Jones Algorithms. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |