CN115019112A - Target object detection method and device based on image and electronic equipment - Google Patents
Target object detection method and device based on image and electronic equipment Download PDFInfo
- Publication number
- CN115019112A CN115019112A CN202210946780.3A CN202210946780A CN115019112A CN 115019112 A CN115019112 A CN 115019112A CN 202210946780 A CN202210946780 A CN 202210946780A CN 115019112 A CN115019112 A CN 115019112A
- Authority
- CN
- China
- Prior art keywords
- target object
- image
- recognized
- detection model
- size
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 111
- 238000000034 method Methods 0.000 claims abstract description 41
- 238000012545 processing Methods 0.000 claims abstract description 21
- 238000010586 diagram Methods 0.000 claims abstract description 11
- 238000004590 computer program Methods 0.000 claims description 12
- 230000009467 reduction Effects 0.000 claims description 8
- 230000004069 differentiation Effects 0.000 claims description 4
- 206010039203 Road traffic accident Diseases 0.000 abstract description 3
- 238000013461 design Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000014509 gene expression Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 235000002566 Capsicum Nutrition 0.000 description 1
- 239000006002 Pepper Substances 0.000 description 1
- 235000016761 Piper aduncum Nutrition 0.000 description 1
- 235000017804 Piper guineense Nutrition 0.000 description 1
- 244000203593 Piper nigrum Species 0.000 description 1
- 235000008184 Piper nigrum Nutrition 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 150000003839 salts Chemical class 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a target object detection method and device based on an image and electronic equipment, and relates to the technical field of image processing, wherein the method comprises the following steps: acquiring an image to be recognized, wherein the image to be recognized is an image acquired by an image sensing device of a vehicle; inputting the image to be recognized into a target object detection model to obtain a feature map of the image to be recognized; the target object detection model comprises the YOLOv3 algorithm; inputting the characteristic diagram into a class detection model of the target object to obtain a class detection result of the target object; and outputting prompt information according to the class detection result of the target object. According to the scheme of the invention, the recognition rate of the target object is improved, the prompt information can be output according to the class detection result of the target object, the condition that the driver pays attention to the interior of the vehicle is prompted, and the occurrence of traffic accidents can be reduced.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a target object detection method and apparatus based on an image, and an electronic device.
Background
With the development of computer vision technology and the popularization of automatic driving and in-vehicle intelligent application scenes, the demand that a vehicle uses a camera to identify a target object is more and more, such as: in-vehicle occupant recognition, driver eye tracking, expression recognition, and the like. In the related method for detecting a target object, the recognition rate of the target object still needs to be improved.
Disclosure of Invention
The embodiment of the invention provides a target object detection method and device based on an image and electronic equipment, which can determine a target object through an image acquired by an image acquisition device of a vehicle so as to prompt a driver to pay attention to the condition in the vehicle.
In order to achieve the purpose, the invention adopts the following technical scheme:
in a first aspect, a method for image-based target object detection is provided, the method comprising: acquiring an image to be identified, wherein the image to be identified is an image acquired by an image sensing device of a vehicle; inputting the image to be recognized into a target object detection model to obtain a feature map of the image to be recognized; the target object detection model comprises the YOLOv3 algorithm; inputting the characteristic diagram into a class detection model of the target object to obtain a class detection result of the target object; and outputting prompt information according to the class detection result of the target object.
According to the method of the first aspect, the image to be recognized is obtained, and the target object in the image to be recognized is recognized through the target object detection model, so that the feature map of the image to be recognized is obtained; and finally, outputting prompt information based on a class detection result of the target object obtained by the class detection model of the target object, so that the identification rate of the target object is improved, and a driver can pay attention to the internal condition of the vehicle according to the prompt information.
With reference to the first aspect, in one possible design, the inputting the image to be recognized into a target object detection model to obtain a feature map of the image to be recognized includes: acquiring the size of an image required to be input by the target object detection model; judging whether the size of the image to be recognized is the same as the size of the image required to be input by the target object detection model; if the difference is different, the size of the image to be recognized is adjusted to the size of the image required to be input by the target object detection model by using a high-low dimension difference differentiation algorithm.
According to the possible design scheme, the target object detection model has a requirement on the size of an input image, and if the size of the image to be recognized is different from the size of the image required to be input by the target object detection model, the target object cannot be recognized and subsequent steps cannot be performed, so that when the size of the image to be recognized is different from the size of the image required to be input by the target object detection model, the size of the image to be recognized is adjusted to be the same as the size of the target object detection model by using a high-low dimension difference differentiation algorithm, and therefore errors in subsequent classification of the target image are avoided.
With reference to the first aspect, in a possible design, if the difference is not the same, adjusting the size of the image to be recognized to the size of the image required to be input by the target object detection model by using a difference in height dimension difference algorithm includes: determining a gray scale image of the image to be recognized and a binary image corresponding to the image to be recognized according to the image to be recognized; determining the depth value of each pixel point in the image to be recognized according to the gray-scale image and the binary image to obtain a depth image of the image to be recognized; and fusing the image to be recognized and the depth map to obtain an input image, wherein the size of the input image accords with the size of the image required to be input by the target object detection model.
According to a possible design scheme, when the size of the image to be recognized and the size of the input image of the target object detection model are not used, the gray scale image and the binary image of the image to be recognized are obtained through the image to be recognized, then the pixel values of the pixel points in the gray scale image and the binary image of the image to be recognized are respectively determined, then the depth value of each pixel point is determined, the depth image of the image to be recognized can be obtained, then the depth image of the image to be recognized and the image to be recognized are fused, so that the input image with the same size as the size of the input image required by the target object detection model is obtained, the size of the image to be recognized can be rapidly adjusted, and the efficiency is improved.
With reference to the first aspect, in a possible design, before the inputting the image to be recognized into the target object detection model and obtaining the feature map of the image to be recognized, the method includes: carrying out noise reduction processing on the image to be identified to obtain an image with higher definition; and replacing the image with higher definition with an image to be identified.
According to a possible design scheme, the noise reduction processing can be performed on the image to be recognized, so that the target object detection is performed on the image to be recognized more accurately.
With reference to the first aspect, in one possible design, the category of the target object includes a first target object and a second target object; outputting prompt information according to the type detection result of the target object comprises: if the target object comprises a first target object, acquiring position information of a reference target object, wherein the reference target object is a target object which is contained in the vehicle and is except for the first target object; taking a reference target object with a difference value between the corresponding position information and the position information of the first target object smaller than a first difference threshold and larger than a second difference threshold as a second target object, wherein the first difference threshold is larger than the second difference threshold; acquiring position information of the second target object; judging whether the minimum distance between the first target object and the second target object is smaller than a second difference threshold value or not; and if so, outputting the prompt information, wherein the prompt information is used for prompting the driver to pay attention to the condition in the vehicle.
According to a possible design scheme, the embodiment can output prompt information to prompt the driver to pay attention to the internal situation of the vehicle when the distance difference between the first target object and the second target object is smaller than the second difference threshold value through the position relationship between the first target object and the second target object.
With reference to the first aspect, in a possible design, after the if, outputting the prompt message, the method includes: and when the first target object is detected to be separated from the second target object, associating the first target object with the second target object, and recording the information of the second target object.
According to a possible embodiment, the position relationship between the first target object and the second target object is determined in a subsequent step by associating the first target object with the second target object.
With reference to the first aspect, in a possible design, the associating the first target object with the second target object after detecting that the first target object is separated from the second target object includes: determining first interest information of the first target object and second interest information of the second target object; generating virtual association relation identification information for the first concern information and the second concern information, and performing association storage on the virtual association relation identification information.
According to a possible design scheme, the embodiment can associate the first attention information of the first target object with the attention information of the second target object and generate the virtual association relationship identification information, so that when the first target object and the second target object exist in the image to be recognized at the same time next time, the image to be recognized can be recognized quickly, the relationship between the first target object and the second target object can be determined quickly according to the virtual association relationship identification information, the prompt information is output, and accidents caused by re-recognition can be avoided.
In a second aspect, there is provided an image-based target object detection apparatus, the apparatus comprising: the device comprises an image to be identified acquisition module, a recognition module and a recognition module, wherein the image to be identified acquisition module is used for acquiring an image to be identified, and the image to be identified is an image acquired by an image sensing device of a vehicle; the first detection module is used for inputting the image to be recognized into a target object detection model to obtain a feature map of the image to be recognized; the target object detection model comprises the YOLOv3 algorithm; the second detection module is used for inputting the characteristic diagram into a class detection model of the target object to obtain a class detection result of the target object; and the prompting module is used for outputting prompting information according to the class detection result of the target object.
In addition, for technical effects of the apparatus for detecting a target object based on an image according to the second aspect, reference may be made to technical effects of the method for detecting a target object based on an image according to the first aspect, which are not described herein again.
In a third aspect, an embodiment of the present invention provides an electronic device. The electronic device includes: a processor, a camera and a memory; the memory is configured to store a computer program, which, when executed by the processor, causes the electronic device to perform the method according to any of the implementations of the first aspect.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, including: computer programs or instructions; the computer program or instructions, when executed on a computer, cause the computer to perform the method of any one of the possible implementations of the first aspect.
Drawings
FIG. 1 is a flowchart of a method for detecting a target object based on an image according to an embodiment of the present invention;
fig. 2 is a flowchart illustrating a specific step of step S130 according to an embodiment of the present invention;
fig. 3 is a flowchart illustrating a specific step of step S1330 according to an embodiment of the present invention;
fig. 4 is a flowchart illustrating a specific step of step S140 according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an image-based target object detection apparatus according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a readable storage medium according to an embodiment of the present invention.
Detailed Description
The technical solution of the present invention is described below with reference to the accompanying drawings.
In the embodiments of the present invention, words such as "exemplary", "for example", etc. are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, the term using examples is intended to present concepts in a concrete fashion. In addition, in the embodiments of the present invention, the expression "and/or" may mean both, or may be either of both.
In the embodiments of the present invention, the "image" and the "picture" may be mixed, and it should be noted that the intended meaning is consistent when the difference is not emphasized. "of", "corresponding", and "corresponding" may sometimes be used in combination, it being noted that the intended meaning is consistent when no distinction is made.
With the development of computer vision technology and the popularization of automatic driving and in-vehicle intelligent application scenes, the demand that a vehicle uses a camera to identify a target object is more and more, such as: in-vehicle occupant recognition, driver eye tracking, expression recognition, and the like.
In contrast, the inventors found in the research on the related target object detection method that the recognition rate of the target object is still to be improved when the target object is recognized by the related target object detection method.
Therefore, in order to overcome the above-mentioned defects, embodiments of the present invention provide a method, an apparatus, an electronic device, and a readable storage medium for image-based target object detection, where the method includes: acquiring an image to be identified, wherein the image to be identified is an image acquired by an image sensing device of a vehicle; inputting the image to be recognized into a target object detection model to obtain a feature map of the image to be recognized; the target object detection model comprises the YOLOv3 algorithm; inputting the characteristic diagram into a class detection model of the target object to obtain a class detection result of the target object; and outputting prompt information according to the class detection result of the target object. The pre-constructed detection model of the target object can accurately identify the target object in the vehicle, the identification rate of the target object is improved, in addition, when a dangerous target object is detected, an alarm is triggered to prompt a driver to pay attention to the condition in the vehicle, and the occurrence of traffic accidents caused by the distraction of the driver can be reduced.
Referring to fig. 1, fig. 1 is a flowchart of a method for detecting a target object based on an image according to an embodiment of the present invention, where the method may be applied to a situation where a driver needs to check the inside of a vehicle during driving the vehicle (for example, monitoring whether a child or an old person is dangerous in a rear seat). Specifically, the method includes steps S110 to S140.
Step S110: and acquiring an image to be identified, wherein the image to be identified is an image acquired by an image acquisition device of the vehicle.
In some embodiments, the image capture device of the vehicle may be an in-vehicle monitor, a tachograph, an in-vehicle camera, and/or the like. The image to be recognized may be an image of each frame of a surveillance video of the interior of the vehicle collected by a vehicle event recorder of the vehicle.
Step S120: inputting the image to be recognized into a target object detection model to obtain a feature map of the image to be recognized; the target object detection model includes the YOLOv3 algorithm.
Optionally, as a mode, the target object detection model may be in a server, or may be in an on-board system, where the on-board system may be a system installed inside the vehicle and having a data processing function, and further, when an image inside the vehicle is acquired in real time by an on-board camera installed inside the vehicle, the acquired image may be sent to the server or the on-board system where the target object detection model is located.
Alternatively, the image capturing device may start capturing the image of the interior of the vehicle when it is detected that the vehicle starts to start. Specifically, when the engine of the vehicle is detected to be started, the image acquisition device starts to acquire the image of the interior of the vehicle in real time, and stores the image of the interior of the vehicle acquired in real time into a storage area of the vehicle, and when the image acquired by the image acquisition device needs to be acquired, the image of the interior of the vehicle acquired by the image acquisition device can be read from the storage area.
Step S130: and inputting the characteristic diagram into a class detection model of the target object to obtain a class detection result of the target object.
The target object may be understood as an object to be identified, and optionally, the target object may be a person or other objects. By way of example, the target object may be identified by a target object detection model from feature information included in the extracted input image. Specifically, the feature information may be an outline or a feature point or a color of the target object, and is not particularly limited herein.
Specifically, as shown in fig. 2, step S130 includes:
step S1310, obtaining the size of the image required to be input by the neural network.
Step S1320, determining whether the size of the image to be recognized is the same as the size of the image required to be input by the target object detection model.
Step S1330, if not, adjusting the size of the image to be recognized to the size of the image required to be input by the target object detection model by using a difference in height and dimension difference algorithm.
For example, if the target object detection model requires that the size of the input image is X × Y, but the size of the image to be recognized is X × X, in this case, the size X × X of the image to be recognized needs to be adjusted to X × Y, where Y may be the number of channels corresponding to the image. For example, the size of the input image is required to be 30 × 3, and in this case, the size of the image to be recognized is required to be 30 × 3.
Specifically, as shown in fig. 3, step S1330 includes:
step S1331, determining the gray level image of the image to be identified and the binary image corresponding to the image to be identified according to the image to be identified.
And step S1332, determining the depth value of each pixel point in the image to be recognized according to the gray-scale image and the binary image, and obtaining the depth image of the image to be recognized.
And S1333, fusing the image to be recognized and the depth map to obtain an input image, wherein the size of the input image accords with the size of the image required to be input by the target object detection model.
Illustratively, the average pixel value of each pixel point in the image to be recognized is calculated by utilizing the pixel value of each pixel point in the corresponding gray-scale image and the pixel value of the binary image, the obtained average pixel value is determined as the depth value of each pixel point, then the depth map of the image to be recognized is determined according to the depth value, and then the depth map and the image to be recognized are fused, so that the size of the obtained image can meet the size required to be input by the target object detection model.
Optionally, if the size of the image to be recognized is X × Y, but the size of the image to be recognized is a × B, the bilinear interpolation algorithm may be used to adjust the size of the image to be recognized to the size of the image to be recognized.
Optionally, the size of the image to be recognized may also be adjusted by adding a deconvolution layer to the target object detection model.
Please continue to refer to fig. 1, step S140: and outputting prompt information according to the class detection result of the target object.
Specifically, the category of the target object includes a first target object and a second target object, as shown in fig. 4, step S140 includes:
step S1401: and if the target object comprises a first target object, acquiring the position information of a reference target object, wherein the reference target object is a target object which is included in the vehicle and is other than the first target object.
In particular, the first target object may be a dangerous object in the vehicle, such as a knife, a pen, an object with a sharp tip, a medicine, a lighter, etc. The second target object may be a child, a baby, an elderly person, or the like, which needs to be cared for.
Step S1402: and taking the reference target object with the difference value between the corresponding position information and the position information of the first target object smaller than a first difference threshold and larger than a second difference threshold as a second target object, wherein the first difference threshold is larger than the second difference threshold.
Specifically, when the difference between the position information of the reference target object and the position information of the first target object is smaller than or equal to the first difference threshold and larger than the second difference threshold, it indicates that a danger exists currently.
Step S1403: and acquiring the position information of the second target object.
Step S1404: and judging whether the minimum distance between the first target object and the second target object is smaller than a second difference threshold value.
Step S1405: and if so, outputting the prompt information, wherein the prompt information is used for prompting the driver to pay attention to the condition in the vehicle.
Specifically, when the minimum distance between the first target object and the second target object is smaller than the second difference threshold, it indicates that a danger is about to occur, and thus, a prompt message is sent to remind the driver of paying attention to the situation inside the vehicle, so that the danger can be avoided.
Optionally, after step S1405, the method further includes: and when the first target object is detected to be separated from the second target object, associating the first target object with the second target object.
Specifically, first attention information of the first target object and second attention information of the second target object are determined; generating virtual incidence relation identification information for the first concern information and the second concern information, and performing incidence storage on the virtual incidence relation identification information.
The first attention information may be information that the shape, contour, etc. of the first target object can indicate the outer shape of the first target object, and may also be a label on the first target object, for example, a prompt label that usually gives attention to children on some dangerous goods. The second attention information may be facial feature information of the second attention object.
The virtual association identification information is identification information that does not exist in the display, and the virtual association identification information may be an influence that a first target object corresponding to the first attention information may have on a second target object corresponding to the second attention information. For example, the first object is a tool, the second object is a child, the tool may cause scratches or even stabs of the child to different degrees, the virtual association flag information may be a red exclamation mark, and when the first attention information and the second attention information exist in another image to be recognized at the same time, the virtual association relation identification information may be directly acquired, so that the driver can be quickly reminded of paying attention to the situation inside the vehicle.
Optionally, before step S110, the method further includes: carrying out noise reduction processing on the image to be identified to obtain an image with higher definition; and replacing the image with higher definition with an image to be identified.
Specifically, the image captured by the image capturing device of the vehicle cannot be directly used, and because there are problems such as exposure and noise, it is necessary to perform noise reduction processing on the image captured by the image capturing device. The denoising processing on the image may include removing gaussian noise, salt and pepper noise, etc. from the image, performing white balance processing on the image, and performing clipping processing on the image. And then after the image collected by the image collecting device is subjected to noise reduction processing, an image which has less noise, more proper white balance proportion and more pertinence compared with an original image can be obtained. Further, after the white balance processing is performed on the original image, the obtained image can have higher definition than the original image.
In the scheme of the invention, the target object in the image to be recognized is recognized through the target object detection model by acquiring the image to be recognized, so as to obtain the characteristic diagram of the image to be recognized; and finally, outputting prompt information based on a class detection result of the target object obtained by the class detection model of the target object, so that the identification rate of the target object is improved, and a driver can pay attention to the internal situation of the vehicle according to the prompt information, so that the occurrence of traffic accidents caused by the fact that the driver pays attention to the internal situation of the vehicle at any time can be avoided.
Fig. 5 is a schematic structural diagram of an image-based target object detection apparatus according to an embodiment of the present invention. As shown in fig. 5, the photographing parameter acquiring apparatus 500 includes: an image to be recognized acquisition module 510, a first detection module 520, a second detection module 530, and a processing module 540.
Specifically, the module 510 for acquiring an image to be recognized is configured to acquire an image to be recognized, where the image to be recognized is an image acquired by an image sensing device of a vehicle; the first detection module 520 is configured to input the image to be recognized into a target object detection model, so as to obtain a feature map of the image to be recognized; the target object detection model comprises the YOLOv3 algorithm; the second detection module 530, inputting the feature map into a class detection model of the target object, to obtain a class detection result of the target object; and the prompt module 540 is configured to output prompt information according to the class detection result of the target object.
Optionally, the first detecting module 520 includes: the second acquisition module is used for acquiring the size of the image required to be input by the target object detection model; the judging module is used for judging whether the size of the image to be recognized is the same as the size of the image required to be input by the target object detection model; and the adjusting module is used for adjusting the size of the image to be identified to the size of the image required to be input by the target object detection model by using a high-low dimension difference differentiation algorithm if the sizes are different.
Optionally, the adjusting module includes: the first determining unit is used for determining a gray-scale image of the image to be recognized and a binary image corresponding to the image to be recognized according to the image to be recognized in English; the depth map determining unit is used for determining the depth value of each pixel point in the image to be recognized according to the gray map and the binary image to obtain the depth map of the image to be recognized; and the fusion unit is used for fusing the image to be recognized and the depth map to obtain an input image, wherein the size of the input image accords with the size of the image required to be input by the target object detection model.
Optionally, the image-based target object detecting apparatus 500 further includes: the noise reduction module is used for carrying out noise reduction processing on the image to be identified to obtain an image with higher definition; and the replacing module is used for replacing the image with higher definition with the image to be identified.
Optionally, the prompting module 540 includes: the position information of the reference target object is acquired but always used for acquiring the position information of the reference target object if the target object contains a first target object, wherein the reference target object is a target object which is included in the vehicle and is except the first target object; a second target object determination unit, configured to use, as a second target object, a reference target object whose difference between the corresponding position information and the position information of the first target object is smaller than a first difference threshold and larger than a second difference threshold, where the first difference threshold is larger than the second difference threshold; a position information acquisition unit of a second target object for acquiring position information of the second target object; a determination unit configured to determine whether a minimum distance between the first target object and the second target object is smaller than a second difference threshold; and the prompting unit is used for outputting the prompting information if the vehicle is in the normal state, and the prompting information is used for prompting the driver to pay attention to the condition in the vehicle.
Optionally, the image-based target object detecting apparatus 500 further includes: and the association module is used for associating the first target object with the second target object after the first target object is detected to be separated from the second target object.
Optionally, the association module includes: a second determination unit configured to determine first attention information of the first target object and second attention information of the second target object; and the association unit is used for generating virtual association relation identification information for the first concern information and the second concern information and performing association storage on the virtual association relation identification information.
For convenience of explanation, fig. 5 shows only main components of the image-based target object detecting apparatus 500.
In addition, the technical effects of the image-based target object detection apparatus 500 may refer to the technical effects of any one of the aforementioned image-based target object detection methods, and will not be described herein again.
Optionally, an embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium includes a computer program or an instruction, and when the computer program or the instruction runs on a computer, the image-based target object detection method or the training method provided in any embodiment of the present invention is executed.
Optionally, an embodiment of the present invention further provides an electronic device, which is configured to execute the method and the apparatus for detecting a target object based on an image according to any embodiment of the present invention, or is configured to execute the method and the apparatus for training according to any embodiment of the present invention.
As shown in fig. 6, the electronic device 2000 may include a processor 2001.
Optionally, the electronic device 2000 may also include a memory 2002 and/or a transceiver 2003.
The processor 2001 is coupled to the memory 2002 and the transceiver 2003, such as may be connected via a communication bus.
The following describes each component of the electronic device 2000 in detail with reference to fig. 6:
the processor 2001 is a control center of the electronic device 2000, and may be a single processor or a collective term for a plurality of processing elements. For example, the processor 2001 is one or more Central Processing Units (CPUs), or may be an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention, such as: one or more microprocessors (digital signal processors, DSPs), or one or more Field Programmable Gate Arrays (FPGAs).
Alternatively, the processor 2001 may perform various functions of the electronic device 2000 by executing or executing software programs stored in the memory 2002 and invoking data stored in the memory 2002.
In particular implementations, processor 2001 may include one or more CPUs such as CPU0 and CPU1 shown in fig. 6 as one embodiment.
In particular implementations, the electronic device 2000 may also include multiple processors, such as the processor 2001 and the processor 2004 shown in fig. 6, as an example. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (e.g., computer program instructions).
The memory 2002 is used for storing a software program for executing the scheme of the present invention, and is controlled by the processor 2001 to execute the software program.
Alternatively, the memory 2002 may be a read-only memory (ROM) or other type of static storage device that may store static information and instructions, a Random Access Memory (RAM) or other type of dynamic storage device that may store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact disc, laser disc, optical disc, digital versatile disc, blu-ray disc, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to such. The memory 2002 may be integrated with the processor 2001 or may be separate and coupled to the processor 2001 through an interface circuit (not shown in fig. 6) of the electronic device 2000, which is not particularly limited in the embodiments of the present invention.
A transceiver 2003 for communication with other electronic devices. For example, where the electronic device 2000 is a smartphone, the transceiver 2003 may be used to communicate with a network device or with another terminal device. As another example, where the electronic device 2000 is a network device, the transceiver 2003 may be used to communicate with a terminal device or with another network device.
Optionally, the transceiver 2003 may include a receiver and a transmitter (not separately shown in fig. 6). Wherein the receiver is configured to implement a receive function and the transmitter is configured to implement a transmit function.
Alternatively, the transceiver 2003 may be integrated with the processor 2001, or may exist separately and be coupled to the processor 2001 through an interface circuit (not shown in fig. 6) of the electronic device 2000, which is not particularly limited in the embodiment of the present invention.
It should be noted that the structure of the electronic device 2000 shown in fig. 6 does not constitute a limitation of the electronic device, and an actual electronic device may include more or less components than those shown, or combine some components, or arrange different components.
In addition, for technical effects of the electronic device 2000, reference may be made to the technical effects of the shooting parameter obtaining method described in the foregoing method embodiment, and details are not described here again.
It should be understood that the processor in the embodiments of the present invention may be a Central Processing Unit (CPU), and the processor may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
It will also be appreciated that the memory in embodiments of the invention may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The non-volatile memory may be a read-only memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an electrically Erasable EPROM (EEPROM), or a flash memory. Volatile memory can be Random Access Memory (RAM), which acts as external cache memory. By way of example, but not limitation, many forms of Random Access Memory (RAM) are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct bus RAM (DR RAM).
The above embodiments may be implemented in whole or in part by software, hardware (e.g., circuitry), firmware, or any combination thereof. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. The procedures or functions according to the embodiments of the present invention are wholly or partially generated when the computer instructions or the computer program are loaded or executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more collections of available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium may be a solid state disk.
It should be understood that the term "and/or" herein is merely one type of association relationship that describes an associated object, meaning that three relationships may exist, e.g., a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone, wherein A and B can be singular or plural. In addition, the "/" in this document generally indicates that the former and latter associated objects are in an "or" relationship, but may also indicate an "and/or" relationship, which may be understood with particular reference to the former and latter text.
In the present invention, "at least one" means one or more, "a plurality" means two or more. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of the singular or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c may be single or multiple.
It should be understood that, in the various embodiments of the present invention, the sequence numbers of the above-mentioned processes do not imply an order of execution, and the order of execution of the processes should be determined by their functions and internal logics, and should not limit the implementation processes of the embodiments of the present invention in any way.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the embodiments provided in the present invention, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (10)
1. An image-based target object detection method, the method comprising:
acquiring an image to be identified, wherein the image to be identified is an image acquired by an image sensing device of a vehicle;
inputting the image to be recognized into a target object detection model to obtain a feature map of the image to be recognized; the target object detection model comprises the YOLOv3 algorithm;
inputting the characteristic diagram into a class detection model of the target object to obtain a class detection result of the target object; and
and outputting prompt information according to the class detection result of the target object.
2. The method for detecting the target object according to claim 1, wherein the inputting the image to be recognized into a target object detection model to obtain the feature map of the image to be recognized comprises:
acquiring the size of an image required to be input by the target object detection model;
judging whether the size of the image to be recognized is the same as the size of the image required to be input by the target object detection model;
if the difference is different, the size of the image to be recognized is adjusted to the size of the image required to be input by the target object detection model by using a high-low dimension difference differentiation algorithm.
3. The method for detecting a target object according to claim 2, wherein if the difference is not the same, adjusting the size of the image to be recognized to the size of the image required to be input by the target object detection model by using a difference in height dimension algorithm comprises:
determining a gray scale image of the image to be recognized and a binary image corresponding to the image to be recognized according to the image to be recognized;
determining the depth value of each pixel point in the image to be recognized according to the gray-scale image and the binary image to obtain a depth image of the image to be recognized;
and fusing the image to be recognized and the depth map to obtain an input image, wherein the size of the input image accords with the size of the image required to be input by the target object detection model.
4. The method for detecting the target object according to claim 1, wherein before the image to be recognized is input into a target object detection model and a feature map of the image to be recognized is obtained, the method further comprises:
carrying out noise reduction processing on the image to be identified to obtain an image with higher definition; and
and replacing the image with higher definition with an image to be identified.
5. The method of claim 1, wherein the category of target objects comprises a first target object and a second target object; outputting prompt information according to the type detection result of the target object comprises:
if the target object comprises a first target object, acquiring position information of a reference target object, wherein the reference target object is a target object which is included in the vehicle and is other than the first target object;
taking a reference target object with a difference value between the corresponding position information and the position information of the first target object smaller than a first difference threshold and larger than a second difference threshold as a second target object, wherein the first difference threshold is larger than the second difference threshold;
acquiring position information of the second target object;
determining whether the minimum distance between the first target object and the second target object is less than the second difference threshold;
and if so, outputting the prompt information, wherein the prompt information is used for prompting the driver to pay attention to the condition in the vehicle.
6. The method according to claim 5, wherein if yes, after outputting the prompt message, the method further comprises:
and when the first target object is detected to be separated from the second target object, associating the first target object with the second target object.
7. The method according to claim 6, wherein associating the first target object with the second target object after detecting that the first target object is separated from the second target object comprises:
determining first interest information of the first target object and second interest information of the second target object;
generating virtual incidence relation identification information for the first concern information and the second concern information, and performing incidence storage on the virtual incidence relation identification information.
8. An image-based target object detection apparatus, comprising:
the device comprises an image to be identified acquisition module, a recognition module and a recognition module, wherein the image to be identified acquisition module is used for acquiring an image to be identified, and the image to be identified is an image acquired by an image sensing device of a vehicle;
the first detection module is used for inputting the image to be recognized into a target object detection model to obtain a feature map of the image to be recognized; the target object detection model comprises the YOLOv3 algorithm;
the second detection module is used for inputting the characteristic diagram into a class detection model of the target object to obtain a class detection result of the target object;
and the prompting module is used for outputting prompting information according to the class detection result of the target object.
9. An electronic device, comprising: a processor coupled to the memory and the camera, respectively;
the processor configured to execute the computer program stored in the memory to cause the electronic device to perform the target object detection method of any one of claims 1-7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium comprises a computer program or instructions which, when run on a computer, causes the target object detection method according to any one of claims 1-7 to be performed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210946780.3A CN115019112A (en) | 2022-08-09 | 2022-08-09 | Target object detection method and device based on image and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210946780.3A CN115019112A (en) | 2022-08-09 | 2022-08-09 | Target object detection method and device based on image and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115019112A true CN115019112A (en) | 2022-09-06 |
Family
ID=83065358
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210946780.3A Pending CN115019112A (en) | 2022-08-09 | 2022-08-09 | Target object detection method and device based on image and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115019112A (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101188018A (en) * | 2007-12-06 | 2008-05-28 | 北大方正集团有限公司 | A method and device for automatically retreating land during the typesetting process |
| US20180348780A1 (en) * | 2017-06-06 | 2018-12-06 | PlusAI Corp | Method and system for object centric stereo in autonomous driving vehicles |
| CN109460705A (en) * | 2018-09-26 | 2019-03-12 | 北京工业大学 | Oil pipeline monitoring method based on machine vision |
| CN110503159A (en) * | 2019-08-28 | 2019-11-26 | 北京达佳互联信息技术有限公司 | Character recognition method, device, equipment and medium |
| CN111291696A (en) * | 2020-02-19 | 2020-06-16 | 南京大学 | A Recognition Method of Handwritten Dongba Character Based on Convolutional Neural Network |
| CN112084919A (en) * | 2020-08-31 | 2020-12-15 | 广州小鹏汽车科技有限公司 | Target detection method, target detection device, vehicle and storage medium |
| CN112270745A (en) * | 2020-11-04 | 2021-01-26 | 北京百度网讯科技有限公司 | Image generation method, device, equipment and storage medium |
| WO2021203618A1 (en) * | 2020-04-08 | 2021-10-14 | 浙江啄云智能科技有限公司 | Image sample generating method and system, and target detection method |
| WO2021227645A1 (en) * | 2020-05-14 | 2021-11-18 | 华为技术有限公司 | Target detection method and device |
-
2022
- 2022-08-09 CN CN202210946780.3A patent/CN115019112A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101188018A (en) * | 2007-12-06 | 2008-05-28 | 北大方正集团有限公司 | A method and device for automatically retreating land during the typesetting process |
| US20180348780A1 (en) * | 2017-06-06 | 2018-12-06 | PlusAI Corp | Method and system for object centric stereo in autonomous driving vehicles |
| CN109460705A (en) * | 2018-09-26 | 2019-03-12 | 北京工业大学 | Oil pipeline monitoring method based on machine vision |
| CN110503159A (en) * | 2019-08-28 | 2019-11-26 | 北京达佳互联信息技术有限公司 | Character recognition method, device, equipment and medium |
| CN111291696A (en) * | 2020-02-19 | 2020-06-16 | 南京大学 | A Recognition Method of Handwritten Dongba Character Based on Convolutional Neural Network |
| WO2021203618A1 (en) * | 2020-04-08 | 2021-10-14 | 浙江啄云智能科技有限公司 | Image sample generating method and system, and target detection method |
| WO2021227645A1 (en) * | 2020-05-14 | 2021-11-18 | 华为技术有限公司 | Target detection method and device |
| CN112084919A (en) * | 2020-08-31 | 2020-12-15 | 广州小鹏汽车科技有限公司 | Target detection method, target detection device, vehicle and storage medium |
| CN112270745A (en) * | 2020-11-04 | 2021-01-26 | 北京百度网讯科技有限公司 | Image generation method, device, equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 叶华等: "单幅图像的深度标签流形学习", 《红外与激光工程》 * |
| 吴明清: "基于机器视觉红枣体积测量及分级方法研究", 《中国博士学位论文》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102418446B1 (en) | Picture-based vehicle damage assessment method and apparatus, and electronic device | |
| CN110532976B (en) | Fatigue driving detection method and system based on machine learning and multi-feature fusion | |
| US10373024B2 (en) | Image processing device, object detection device, image processing method | |
| US9697430B2 (en) | Method and apparatus for identifying road signs | |
| CN110532985B (en) | Target detection method, device and system | |
| US8077204B2 (en) | Vehicle periphery monitoring device, vehicle, vehicle periphery monitoring program, and vehicle periphery monitoring method | |
| US10235768B2 (en) | Image processing device, in-vehicle display system, display device, image processing method, and computer readable medium | |
| CN111368612B (en) | Overguard detection system, personnel detection method and electronic equipment | |
| CN106709443B (en) | Method and terminal for detecting wearing state of safety belt | |
| CN107909007B (en) | lane line detection method and device | |
| US11713046B2 (en) | Driving assistance apparatus and data collection system | |
| US10671861B2 (en) | Image recording system, image recording method and image recording program | |
| US20140334672A1 (en) | Method for detecting pedestrians based on far infrared ray camera at night | |
| JP2008130086A (en) | Method and apparatus for determining classification boundary | |
| CN107133587B (en) | Method and device for generating obstacle distance determination image | |
| CN109383519B (en) | Information processing method, information processing system and recording medium | |
| WO2018042976A1 (en) | Image generation device, image generation method, recording medium, and image display system | |
| CN115019112A (en) | Target object detection method and device based on image and electronic equipment | |
| US10688929B2 (en) | Driving assistance system and method of enhancing a driver's vision | |
| EP4304191A2 (en) | Camera system, method for controlling the same, and computer program | |
| CN114514565A (en) | Object recognition device and object recognition program | |
| CN112241660A (en) | A vision-based anti-theft monitoring method and device | |
| JP2017058950A (en) | Recognition device, image pickup system, and image pickup device, and recognition method and program for recognition | |
| CN112183413B (en) | Parking space detection method and device, storage medium and vehicle | |
| CN118575206A (en) | Method and device for detecting wearing of safety belt, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220906 |
|
| RJ01 | Rejection of invention patent application after publication |