CN115797934B

CN115797934B - Meter registration method, apparatus, electronic device and storage medium

Info

Publication number: CN115797934B
Application number: CN202211545398.8A
Authority: CN
Inventors: 姜楠; 聂磊
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2022-12-01
Filing date: 2022-12-01
Publication date: 2023-12-01
Anticipated expiration: 2042-12-01
Also published as: CN115797934A

Abstract

The disclosure provides an instrument registration method, an instrument registration device, electronic equipment and a storage medium, and relates to the technical fields of computer vision, deep learning and industrial quality inspection. The specific implementation scheme is as follows: according to target detection information obtained by detecting the instrument image of the pointer instrument, determining a dial image from the instrument image; performing key point detection on the dial image to obtain scale detection information, wherein the scale detection information comprises position information of at least one key point, the key point represents a point of a target scale, the target scale is provided with at least one key point corresponding to the target scale, and the target scale is a scale with an indication; obtaining pointer identification information according to image segmentation information obtained by carrying out image segmentation on the disc image; and obtaining instrument indication identification information according to the scale detection information, the pointer identification information and the scale indication information corresponding to the scale detection information.

Description

Meter registration method, apparatus, electronic device and storage medium

Technical Field

The present disclosure relates to the field of artificial intelligence, and more particularly to the field of computer vision, deep learning, and industrial quality inspection. And in particular to a meter reading method, a meter reading device, an electronic device and a storage medium.

Background

The pointer instrument has the characteristics of electromagnetic interference resistance, shock resistance, oil stain resistance, freezing resistance, water resistance and the like, and is widely applied to the industrial fields of electric power, energy sources, aviation and the like.

Along with the development of image processing technology and industrial manufacturing technology, an inspection robot can be used for carrying a vision sensor to collect instrument images of pointer instruments deployed on an industrial site, and then the instrument readings are determined by using the image processing technology.

Disclosure of Invention

The disclosure provides a meter reading method, a meter reading device, electronic equipment and a storage medium.

According to an aspect of the present disclosure, there is provided an instrument indication recognition method, including: according to target detection information obtained by detecting an instrument image of a pointer instrument, determining a dial image from the instrument image; detecting key points of the dial plate image to obtain scale detection information, wherein the scale detection information comprises position information of at least one key point, the key point represents a point of a target scale, the target scale is provided with at least one key point corresponding to the target scale, and the target scale is a scale with an indication; and obtaining instrument indication identification information according to the scale detection information, the pointer identification information and scale indication information corresponding to the scale detection information.

According to another aspect of the present disclosure, there is provided an instrument registration recognition device including: the first determining module is used for determining a dial image from the instrument image according to target detection information obtained by detecting the instrument image of the pointer instrument; the first obtaining module is used for carrying out key point detection on the dial image to obtain scale detection information, wherein the scale detection information comprises position information of at least one key point, the key point represents a point of a target scale, the target scale is provided with at least one key point corresponding to the target scale, and the target scale is a scale with an indication; the second obtaining module is used for obtaining pointer identification information according to image segmentation information obtained by carrying out image segmentation on the dial image; and a third obtaining module, configured to obtain meter indication identifying information according to the scale detecting information, the pointer identifying information, and scale indication information corresponding to the scale detecting information.

According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described above.

According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method as described above.

According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements a method as described above.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 schematically illustrates an exemplary system architecture to which meter reading identification methods and apparatus may be applied, according to embodiments of the present disclosure;

FIG. 2 schematically illustrates a flow chart of a meter reading identification method according to an embodiment of the disclosure;

FIG. 3 schematically illustrates an example schematic diagram of a pointer meter according to an embodiment of the disclosure;

FIG. 4 schematically illustrates an example schematic diagram of performing M stages of feature extraction on a surface disk image, resulting in at least one keypoint feature map corresponding to an Mth stage, according to an embodiment of the disclosure;

FIG. 5 schematically illustrates an example schematic diagram of determining a transformation matrix from a target ellipse and a circumscribed circle according to an embodiment of the disclosure;

FIG. 6A schematically illustrates a schematic diagram of a meter reading identification method according to an embodiment of the disclosure;

FIG. 6B schematically illustrates an example schematic diagram of meter reading identification information according to an embodiment of the disclosure;

FIG. 7 schematically illustrates a block diagram of a meter reading identification device according to an embodiment of the disclosure; and

fig. 8 schematically illustrates a block diagram of an electronic device suitable for implementing a meter reading identification method according to an embodiment of the disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Fig. 1 schematically illustrates an exemplary system architecture to which meter reading identification methods and apparatus may be applied, according to embodiments of the present disclosure.

It should be noted that fig. 1 is only an example of a system architecture to which embodiments of the present disclosure may be applied to assist those skilled in the art in understanding the technical content of the present disclosure, but does not mean that embodiments of the present disclosure may not be used in other devices, systems, environments, or scenarios.

As shown in fig. 1, a system architecture 100 according to this embodiment may include a patrol robot 101, an electronic device 102, and a network 103. The network 103 may be a medium used to provide a communication link between the inspection robot 101 and the electronic device 102. The network 103 may include various connection types. Such as at least one of a wired and wireless communication link, etc.

The industrial equipment of the industrial site may be provided with pointer instruments. The inspection robot 101 may be provided with a vision sensor. The vision sensor may be used to capture a meter image of the pointer meter. In addition, the inspection robot 101 may include a cradle head, a chassis, an industrial personal computer, and the like. The vision sensor located on the inspection robot may be referred to as a mobile vision sensor. In addition, the industrial site may also be provided with a stationary visual sensor. The fixed vision sensor may also be used to capture meter images of the pointer meter.

The electronic device 102 may include at least one of: a terminal device 1021 and a server 1022. The terminal device 1021 may be a variety of electronic devices that have a display screen and support web browsing. For example, the terminal device 102_1 may include at least one of a smart phone, a tablet computer, a laptop portable computer, a desktop computer, and the like.

The server 102_2 may be a server providing various services. For example, the server 102_2 may be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of large management difficulty and weak service expansibility in the traditional physical hosts and VPS services (Virtual Private Server, virtual private servers).

The inspection robot 101 may be used to inspect at an industrial site and send meter images acquired during inspection to the electronic device 102. The inspection robot 101 may store the acquired meter image locally.

The electronic device 102 can determine the dial image from the meter image based on the target detection information obtained by detecting the meter image of the pointer meter. And detecting key points of the dial plate image to obtain scale detection information. And obtaining pointer identification information according to the image segmentation information obtained by carrying out image segmentation on the disc image. And obtaining instrument indication identification information according to the scale detection information, the pointer identification information and the scale indication information corresponding to the scale detection information.

It should be noted that the meter reading identification method provided in the embodiment of the present invention may be executed by the electronic device 102. Accordingly, the meter reading recognition device provided by the embodiment of the invention can also be arranged in the electronic device 102.

It should be understood that the number of inspection robots, electronic devices, and networks in fig. 1 is merely illustrative. There may be any number of inspection robots, electronic devices, and networks, as desired for implementation.

It should be noted that the sequence numbers of the respective operations in the following methods are merely representative of the operations for the purpose of description, and should not be construed as representing the order of execution of the respective operations. The method need not be performed in the exact order shown unless explicitly stated.

Fig. 2 schematically illustrates a flow chart of a meter reading identification method according to an embodiment of the disclosure.

As shown in fig. 2, the method 200 includes operations S210 to S240.

In operation S210, a dial image is determined from the meter image according to target detection information obtained by detecting the meter image of the pointer meter.

In operation S220, the key point detection is performed on the disc image to obtain scale detection information.

In operation S230, pointer identification information is obtained from image division information obtained by image division of a disk image.

In operation S240, meter indication identification information is obtained from the scale detection information, pointer identification information, and scale indication information corresponding to the scale detection information.

According to an embodiment of the present disclosure, the scale detection information may include location information of at least one key point. The keypoints may characterize the points of the target scale. The target scale may have at least one key point corresponding to the target scale. The target scale may be a scale with an indication.

According to embodiments of the present disclosure, pointer meters may be used to display measured parameters of an industrial site. The measurement parameters may include at least one of: environmental parameters, production index parameters, and equipment operating parameters, etc. The production index parameter may refer to an index parameter used to evaluate the resource production information. For example, the environmental parameters may include at least one of: temperature, humidity, pressure, etc. The device operating parameters may include at least one of: current and voltage, etc. The production index parameter may include at least one of: oil, gas and water, etc.

According to embodiments of the present disclosure, the meter type of the pointer meter may include at least one of the following depending on the measured parameter: a pointer instrument corresponding to the environmental parameter, a pointer instrument corresponding to the production index parameter and a pointer instrument corresponding to the equipment operation parameter. For example, the pointer meter corresponding to the environmental parameter may include at least one of: temperature pointer instrument, humidity pointer instrument, pressure pointer instrument, etc. The pointer meter corresponding to the production index may include at least one of: oil pointer instrument, gas pointer instrument, water pointer instrument, etc. The pointer meter corresponding to the device operating parameter may include at least one of: voltage pointer meters, current pointer meters, and the like.

According to embodiments of the present disclosure, the pointer meter may include a dial plate. The dial may include a scale, a scale reading, and a pointer. Depending on the scale's distribution information, the pointer instrument type may include at least one of: pointer instruments with uniformly distributed scales and pointer instruments with non-uniformly distributed scales. Further, the pointer instrument has a scale range corresponding to the pointer instrument. For example, the scale range may be greater than or equal to 180 ° and less than or equal to 360 °. The scale range may be greater than 0 ° and less than or equal to 60 °.

According to embodiments of the present disclosure, the meter image may include a dial area image and a non-dial area image. The dial area image may refer to an image corresponding to the dial area. The non-dial area image may refer to an image corresponding to the non-dial image. The dial area may include a text area and a numeric area. The text region may refer to a region for describing meter information. The meter information may include at least one of: producer, meter type, meter range, serial number, etc. The indication area may refer to an area where an indication can be acquired. The indication area may include a pointer area, a scale area, and a scale indication area. The graduated flask may include at least one graduation. The scale has a length. The scale may comprise a first type of scale and a second type of scale. The length of the first type of scale is greater than the length of the second type of scale. Furthermore, the scale may also comprise a third type of scale. The third type of scale has a length that is greater than the length of the second type of scale and less than the length of the first type of scale. The scale indicia area may include at least one indicia. The at least one scale may include a scale with an indication and a scale without an indication. For example, the first type of scale may be a scale with an indication. The third type of scale may be a scale with an indication. The second type of scale may be a scale without an indication.

Fig. 3 schematically illustrates an example schematic diagram of a pointer meter according to an embodiment of the disclosure.

As shown in fig. 3, in 300, pointer meter 301 may include a scale 3011, a scale 3012, a scale 3013, a scale 3014, a scale 3015, a scale 3016, a scale 3017, a scale 3018, and a scale 3019. Furthermore, the pointer meter 301 includes a scale 30110. The scale 3015 may include a point 30151 and a point 30152.

Since the scales 3011 to 3019 are scales having indication numbers, the scales 3011 to 3019 may be target scales. Since the scale 30110 is a scale having no indication, the scale 30110 is not a target scale. The scales 3011 to 3019 may be of a first type. The scale 30110 may be a second type of scale. The points 30151 and 30152 on the scale 3015 may be key points.

According to embodiments of the present disclosure, the target detection information may be used to determine location information of the object. The object may include a dial area and a non-dial area. The target detection information may be obtained by performing target detection on the meter image. The scale detection information may be used to determine location information of the keypoints. The scale detection information may include location information of at least one key point. The key points may be points on the target scale. The target scale may be a scale with an indication. For example, the target scale may comprise at least one of a first type of scale and a third type of scale. The target scale may have at least one key point corresponding to the target scale. For example, a target scale may have two keypoints corresponding to the target scale. The two key points may be points located at both ends of the target scale. The scale indication information corresponding to the scale detection information may include an indication corresponding to the target scale. The indicia may have a correspondence to the target scale. Since the key points are points on the target scale, the indication and the key points may have a correspondence. The pointer identification information may include at least one pointer.

According to the embodiment of the present disclosure, the determination of the target scale and the number of target scales may be configured according to actual service requirements, which is not limited herein. For example, the number of target scales may be determined according to the number of target scales of the pointer meter of various predetermined meter types, and it is sufficient to be able to satisfy the pointer indication recognition requirements of the pointer meter of various predetermined meter types. For example, the number of target graduations may be determined from at least one target number. The target number may refer to the number of target graduations included by the pointer meter of the predetermined meter type. For example, the number of target scales may be determined based on a sum of at least one target number. Since the number of target scales can be determined according to the number of target scales of various predetermined types of pointer meters, versatility of the scale detection method can be ensured.

According to an embodiment of the present disclosure, since the number of target scales is determined according to the number of target scales with at least one predetermined type of pointer meter, for example, the number of target scales may be the sum of the number of target scales with at least one predetermined type of pointer meter, the actual number of target scales of the pointer meter is less than or equal to the number of target scales. In the case where the actual number of target scales of the pointer instrument is smaller than the number of target scales, the target scales larger than the actual number may be characterized by a predetermined identification. The predetermined identification characterizes the target scale as having an invisible property. The predetermined identifier may be configured according to actual service requirements, and is not limited herein, and for example, the predetermined identifier may be 0.

According to the embodiment of the disclosure, since the target scale is a scale with an indication, the indication can increase the characteristic information of the key points, and thus the accuracy of scale detection information and the universality of a scale detection method can be improved. Since the target scale is a scale having an indication, it is possible to solve the problem that it is difficult to obtain accurate scale detection information with a small number of scales in the case of a pointer instrument in which scales are unevenly distributed, thereby improving the versatility of the scale detection method.

According to embodiments of the present disclosure, a raw meter image of a pointer meter may be acquired. The raw meter image may be acquired by a fixed vision sensor. Alternatively, the raw meter image may be acquired by a mobile vision sensor. The mobile vision sensor may refer to a vision sensor located on the inspection robot. The raw meter image may be stored in at least one of a local database and a cloud database.

According to an embodiment of the present disclosure, acquiring an original meter image of a pointer meter may include: in response to detecting the indication identifying instruction, a raw meter image of the pointer meter is acquired from the data source. The data source may include at least one of: local databases, cloud databases, and network resources. Acquiring the raw meter image of the pointer meter from the data source may include: a data interface is invoked. And acquiring an original instrument image of the pointer instrument from a data source by utilizing a data interface.

According to the embodiment of the disclosure, the instrument image can be preprocessed to obtain the instrument image. The pre-treatment may comprise at least one of: image denoising, image enhancement, and the like. Alternatively, the raw meter image may be determined as the meter image.

According to the embodiment of the disclosure, after the meter image is obtained, the target detection can be performed on the meter image to obtain target detection information. For example, the meter image may be processed based on a conventional target detection method to obtain target detection information. Alternatively, the meter image may be processed based on a target detection method of deep learning to obtain target detection information. The conventional target detection method may include at least one of: a target detection method based on macroscopic features and a target detection method based on feature points. The macro-features may include at least one of: shape, color, size, etc. For example, the macro-feature based target detection method may include a Hough circle transform based target detection method. The feature point-based object detection method may include at least one of: a SIFT (Scale-invariant feature transform) based target detection method and a SURF (Speeded Up Robust Features, accelerated robust feature) based target detection method, etc. The deep learning-based target detection method may include at least one of: a single-stage target detection method and a two-stage target detection method. For example, a two-stage target detection method may include at least one of: R-CNN (i.e., region Convolutional Neural Network), fast R-CNN, etc. The single-stage target detection method may comprise at least one of: SSD (i.e., single Shot MultiBox Detector) and YOLO (i.e., you Only Look Once), etc.

According to an embodiment of the present disclosure, performing target detection on an instrument image to obtain target detection information may include: and extracting features of the instrument image to obtain an instrument feature map of at least one scale. And fusing the instrument feature graphs of at least one scale to obtain fusion features. And obtaining a target detection information pair according to the fusion characteristics. Alternatively, feature extraction is performed on the meter image to obtain meter feature information. And obtaining at least one candidate frame according to the instrument characteristic information. And obtaining target detection information according to at least one candidate frame.

According to the embodiments of the present disclosure, after the target detection information is determined, the position information of the dial area may be determined according to the target detection information. The dial image is determined from the meter image based on the position information of the dial region. The meter image may include an image of at least one pointer meter. After the dial image is determined, key point detection can be performed on the dial image to obtain scale detection information. For example, performing key point detection on the disc image to obtain scale detection information may include: and processing dial images based on the traditional key point detection method to obtain scale detection information. Alternatively, the dial image may be processed based on a key point detection method of deep learning to obtain scale detection information. The conventional keypoint detection method may comprise at least one of the following: a key point detection method based on ASM (namely Active Shape Mode), a key point detection method based on AAM (namely Active Appearnce Model), a key point detection method based on cascade shape regression, and the like. The key point detection method based on deep learning processes the dial image to obtain scale detection information, and may include: and extracting features of the dial image to obtain a key point feature map of at least one scale. And obtaining scale detection information according to the key point feature map of at least one scale. According to the key point feature map of at least one scale, obtaining scale detection information can include: and obtaining a thermodynamic diagram corresponding to at least one target scale according to the key point characteristic diagram of at least one scale. And determining scale detection information according to the thermodynamic diagram corresponding to the at least one target scale. Alternatively, processing is performed on the key point feature map of at least one scale based on the regression location method to obtain scale detection information.

According to the embodiment of the disclosure, the disc image can be subjected to image segmentation to obtain image segmentation information. And obtaining pointer identification information according to the image segmentation information. The image segmentation may include one of: semantic segmentation, instance segmentation, and scene segmentation. For example, image segmentation may be performed on the disk image to obtain image segmentation information, which may include: and extracting features of the dial image to obtain a first backbone feature map. And according to the first backbone feature map, semantic segmentation information and pixel feature characterization are obtained. And obtaining image segmentation information according to the semantic segmentation information and the pixel characteristic characterization. Alternatively, feature extraction is performed on the disk image, and a second backbone feature map is obtained. And obtaining a first intermediate feature map of at least one scale according to the second backbone feature map. And obtaining a second intermediate feature map according to the second backbone feature map and the intermediate feature map of at least one scale. And obtaining image segmentation information according to the second intermediate feature map. Pointer identification information can be obtained from the image division information.

According to the embodiment of the disclosure, after the scale detection information and the pointer identification information are obtained, the meter indication identification information may be obtained from the pointer identification information, the scale detection information, and the scale indication information corresponding to the scale detection information. For example, the scale to which the pointer points may be determined according to the pointer point corresponding to at least one pointer included in the pointer identification information. When the scale is a target scale, the instrument indication identification information can be obtained according to the indication corresponding to the target scale in the scale indication information. Under the condition that the scale is a non-target scale, the distance between the scale and two adjacent target scales can be determined, and instrument indication identification information is determined according to the distance and the respective indications of the two adjacent target scales. Further, the scale indication information may include indications of the respective scales. The position information of each scale may also be stored in advance.

According to an embodiment of the present disclosure, at least one pointer included according to pointer identification information

According to the embodiment of the disclosure, since the meter indication identifying information is obtained according to the pointer identifying information and the scale detecting information, the scale detecting information is obtained according to the key point detection of the dial image, the scale detecting information comprises the position information of the target scale with indication, and the pointer identifying information is obtained according to the image dividing information obtained by dividing the dial image, the meter indication identifying method can be suitable for indication identification of various types of pointer meters in various environments, and the universality of the meter indication identifying method is improved.

According to an embodiment of the present disclosure, operation S210 may include the following operations.

And extracting the characteristics of the instrument image of the pointer instrument to obtain an instrument characteristic diagram of at least one scale. And fusing the instrument feature graphs of at least one scale to obtain a fused feature graph. And obtaining target detection information according to the fusion feature map. The dial area is determined from the at least one area based on the confidence and category information. The dial image is determined from the meter image based on the position information of the dial region.

According to embodiments of the present disclosure, the target detection information may include location information, category information, and confidence of at least one region. The confidence level may characterize the confidence level of the category information. The category information may include dial areas and non-dial areas.

According to the embodiment of the disclosure, the feature extraction can be performed on the meter image, so as to obtain the meter feature map with at least one scale. For example, the meter image may be processed using a cross-stage local extraction method to obtain a meter signature of at least one dimension. Fusing the instrument feature graphs of at least one scale to obtain a fused feature graph, which may include: and processing the instrument characteristic map of at least one scale by using a path aggregation method to obtain a fusion characteristic map. Path aggregation methods may refer to aggregation methods from small scale to large scale. The path aggregation method makes it easier for lower layer information to be transferred to higher layers.

According to embodiments of the present disclosure, at least one target category information may be determined from at least one category information. In the case where it is determined that the confidence level corresponding to the target category information is greater than or equal to the predetermined confidence level threshold, the area corresponding to the target category information is determined as the dial area. The target category information may characterize category information in which the area is a dial area. The dial image is determined from the meter image based on the position information of the dial region.

According to an embodiment of the present disclosure, operation S220 may include the following operations.

And extracting features of the dial image to obtain a key point feature map of at least one scale. And obtaining scale detection information according to the key point feature map of at least one scale.

According to embodiments of the present disclosure, scale may refer to image resolution. Each scale may have at least one keypoint feature map corresponding to the scale.

According to the embodiment of the disclosure, the dial image can be processed based on a single-stage serial method, so that a key point characteristic diagram of at least one scale is obtained. Alternatively, the dial image is processed based on a multi-stage series method to obtain a key point feature map of at least one scale. Alternatively, the dial image is processed based on a multi-stage parallel method, and a key point feature map of at least one scale is obtained.

According to an embodiment of the present disclosure, obtaining scale detection information according to a key point feature map of at least one scale may include: and obtaining a thermodynamic diagram corresponding to at least one target scale according to the key point characteristic diagram of at least one scale. Scale detection information is determined from a thermodynamic diagram corresponding to at least one target scale. Alternatively, the key point feature map of at least one scale is processed based on a regression location method to obtain scale detection information.

According to the embodiment of the disclosure, since the key point feature map of at least one scale can provide richer information, the scale detection information is obtained by using the key point feature map of at least one scale, and the accuracy of the scale detection information is improved.

According to an embodiment of the present disclosure, feature extraction is performed on a surface disc image to obtain a key point feature map of at least one scale, which may include the following operations.

And carrying out feature extraction of M stages on the dial image to obtain at least one key point feature map corresponding to the M-th stage. And obtaining the key point feature map of at least one scale according to the at least one key point feature map corresponding to the M stage.

According to an embodiment of the present disclosure, the mth stage may have T _m And a plurality of parallel hierarchies. The image resolution of the keypoint feature map of the same parallel hierarchy is the same. The image resolution of the keypoint feature map is different for different parallel levels.

According to embodiments of the present disclosure, M may be an integer greater than or equal to 1. M may be an integer greater than or equal to 1 and less than or equal to M. T (T) _m May be an integer greater than or equal to 1.

According to embodiments of the present disclosure, the M-phase may include an input phase, an intermediate phase, and an output phase. The input phase may refer to phase 1. The output phase may refer to the mth phase. The intermediate stages may refer to stages 2 through M-1. The number of parallel stages of each stage may be the same or different. In stages 1 to M-1, the current stage may have at least one more parallel hierarchy than the previous stage. The mth stage may be the same number of parallel stages as the M-1 stage. M may be configured according to actual service requirements, which is not limited herein. For example, m=4. In stages 1 to 3, the current stage may be at least one more parallel hierarchy than the previous stage. Stage 1 has t1=2 parallel levels. Stage 2 has T ₂ =3 parallel levels. Stage 3 has T ₃ =4 parallel levels. Stage 4 has T ₄ =4 parallel levels.

According to embodiments of the present disclosure, the image resolution of the keypoint feature map of the same parallel hierarchy is the same. The image resolutions of the keypoint feature maps of different parallel levels are different, for example, the image resolution of the keypoint feature map of the current parallel level is smaller than the image resolution of the keypoint feature map of the upper parallel level. The image resolution of the keypoint feature map of the current parallel hierarchy of the current stage may be determined from the image resolution of the keypoint feature map of the upper parallel hierarchy of the previous stage. For example, the image resolution of the current-stage keypoint feature map of the current stage may be obtained by downsampling the image resolution of the keypoint feature map of the upper parallel hierarchy of the previous stage.

According to an embodiment of the present disclosure, in a case where M > 1, performing feature extraction on a disk image in M stages to obtain at least one key point feature map corresponding to the M stages may include: and responding to m=1, and carrying out feature extraction on the surface disc image to obtain an intermediate key point feature map of at least one scale corresponding to the 1 st stage. And obtaining the key point feature map of at least one scale corresponding to the stage 1 according to the intermediate key point feature map of at least one scale corresponding to the stage 1. And responding to M which is more than 1 and less than or equal to M, carrying out feature extraction on the key point feature map of at least one scale corresponding to the M-1 stage to obtain the intermediate key point feature map of at least one scale corresponding to the M-1 stage. And obtaining the key point feature map of at least one scale corresponding to the mth stage according to the intermediate key point feature map of at least one scale corresponding to the mth stage.

According to an embodiment of the present disclosure, in the case where m=1, performing feature extraction on a table image in M stages, to obtain at least one key point feature map corresponding to the M stages may include: and carrying out feature extraction on the dial image to obtain an intermediate key point feature map of at least one scale corresponding to the 1 st stage. And obtaining the key point feature map of at least one scale corresponding to the stage 1 according to the intermediate key point feature map of at least one scale corresponding to the stage 1.

According to an embodiment of the present disclosure, obtaining a keypoint feature map of at least one scale according to at least one keypoint feature map corresponding to the mth stage may include: at least one keypoint feature map corresponding to the mth stage may be determined as a keypoint feature map of at least one scale.

According to the embodiment of the disclosure, since the image resolutions of the key point feature maps of the same parallel hierarchy are the same and the image resolutions of the key point feature maps of different parallel hierarchies are different, the high-resolution feature characterization can be maintained in the whole feature extraction process, and the parallel hierarchies from high resolution to low resolution can be gradually increased. Deep semantic information is directly extracted on the high-resolution feature representation, but not used as the supplement of low-level feature information of the image, so that the image has enough classification capability and avoids the loss of effective spatial resolution. At least one parallel hierarchy can capture the context information and acquire rich global and local information. In addition, the information is repeatedly exchanged on the parallel hierarchy to realize multi-scale fusion of the features, so that more accurate position information of key points can be obtained, and the accuracy of scale detection information is improved.

According to an embodiment of the present disclosure, in a case where M is an integer greater than 1, performing feature extraction on a panel image in M stages, to obtain at least one key point feature map corresponding to an mth stage may include the following operations.

And (3) carrying out convolution processing on at least one key point feature map corresponding to the m-1 stage to obtain at least one intermediate key point feature map corresponding to the m stage. And carrying out feature fusion on at least one middle key point feature map corresponding to the mth stage to obtain at least one key point feature map corresponding to the mth stage.

According to embodiments of the present disclosure, M may be an integer greater than 1 and less than or equal to M.

According to the embodiment of the disclosure, for the m-1 stage, for a key point feature map in at least one key point feature map, convolution processing may be performed on the key point feature map to obtain an intermediate key point feature map of the m stage, so that at least one intermediate key point feature map of the m stage may be obtained.

According to an embodiment of the present disclosure, feature fusion is performed on at least one intermediate keypoint feature map corresponding to the mth stage, to obtain at least one keypoint feature map corresponding to the mth stage, which may include: and fusing the intermediate key point feature map of the m-th stage and the intermediate key point feature maps of other parallel stages except for the parallel stage where the intermediate key point feature map is located with respect to the intermediate key point feature map of at least one intermediate key point feature map corresponding to the m-th stage, so as to obtain the key point feature map corresponding to the intermediate key point feature map of the m-th stage. Other parallel levels may refer to at least some parallel levels of the mth stage other than the parallel level at which the intermediate keypoint feature map is located.

According to an embodiment of the present disclosure, feature fusion is performed on at least one intermediate keypoint feature map corresponding to the mth stage, to obtain at least one keypoint feature map corresponding to the mth stage, which may include the following operations.

For T _m And obtaining the key point feature map corresponding to the ith parallel hierarchy according to the other intermediate key point feature maps corresponding to the ith parallel hierarchy and the intermediate key point feature map corresponding to the ith parallel hierarchy in the ith parallel hierarchy.

According to an embodiment of the present disclosure, the other intermediate keypoint feature map corresponding to the ith parallel hierarchy is the one corresponding to T _m And at least part of parallel hierarchies except the ith parallel hierarchy corresponds to the middle key point feature map. i may be greater than or equal to 1 and less than or equal to T _m Is an integer of (a).

According to an embodiment of the disclosure, in the case that I < 1, up-sampling is performed on at least one first other intermediate keypoint feature map, so as to obtain an up-sampled keypoint feature map corresponding to the at least one first other intermediate keypoint feature map. And downsampling the at least one second other intermediate key point feature map to obtain a downsampled key point feature map corresponding to the at least one second other intermediate key point feature map. The first other intermediate keypoint feature map may be referred to as T _m Other intermediate keypoint feature maps in the parallel hierarchy that are greater than the ith parallel hierarchy. The second other intermediate keypoint feature map may be referred to as T _m Other intermediate keypoint feature maps in each parallel hierarchy that are smaller than the ith parallel hierarchy. The image resolution of the upsampled keypoint feature map is the same as the resolution of the intermediate keypoint feature map of the ith parallel hierarchy. The resolution of the downsampled keypoint feature map is the same as the resolution of the intermediate keypoint feature map of the i parallel hierarchies.

According to an embodiment of the present disclosure, in case i=1, for at least one second otherAnd up-sampling the intermediate key point feature map to obtain a down-sampling key point feature map corresponding to at least one first other intermediate key point feature map. The first other intermediate keypoint feature map may be referred to as T _m Other intermediate keypoint feature maps in more than the 1 st parallel level in the parallel levels. The image resolution of the upsampled keypoint feature map is the same as the resolution of the intermediate keypoint feature map of the 1 st parallel hierarchy.

According to an embodiment of the present disclosure, in the case of i=i, at least one second other intermediate keypoint feature map is downsampled, resulting in a downsampled keypoint feature map corresponding to the at least one second other intermediate keypoint feature map. The second other intermediate keypoint feature map may be referred to as T _m Other intermediate keypoint feature maps in each parallel hierarchy that are smaller than the ith parallel hierarchy. The resolution of the downsampled keypoint feature map is the same as the resolution of the intermediate keypoint feature map of the i parallel hierarchies.

According to the embodiment of the disclosure, a keypoint feature map corresponding to an ith parallel hierarchy is obtained from an up-sampled keypoint feature map corresponding to at least one first other intermediate keypoint feature map, a down-sampled keypoint feature map corresponding to at least one second other intermediate keypoint feature map, and an intermediate keypoint feature map of the ith parallel hierarchy. For example, the upsampled keypoint feature map corresponding to the at least one first other intermediate keypoint feature map, the downsampled keypoint feature map corresponding to the at least one second other intermediate keypoint feature map, and the intermediate keypoint feature map of the ith parallel hierarchy may be fused to obtain the keypoint feature map corresponding to the ith parallel hierarchy. The fusion may include at least one of: splicing and adding.

Fig. 4 schematically illustrates an example schematic diagram of performing feature extraction on a table disc image in M stages, resulting in at least one keypoint feature map corresponding to an mth stage, according to an embodiment of the disclosure.

As shown in fig. 4, in 400, m=4, for example, a 1 st stage 401, a 2 nd stage 402, a 3 rd stage 403, and a 4 th stage 404. Stage 1 has two parallel levels, for example, a 1 st parallel level 405 and a 2 nd parallel level 406. Stage 2 402 has three parallel levels, e.g., level 1 parallel level 405, level 2 parallel level 406, and level 3 parallel level 407. Stage 3 403 is specifically four parallel levels, e.g., parallel level 1 405, parallel level 2 406, parallel level 3 407, and parallel level 4 408.

The at least one keypoint feature map corresponding to stage 4 may include a keypoint feature map 409, a keypoint feature map 410, a keypoint feature map 411, and a keypoint feature map 412. Furthermore, the "up-right arrow" between the last two columns of each stage in fig. 4 characterizes "up-sampling". The "lower left arrow" characterizes "downsampling".

And carrying out feature extraction of N cascade levels on the dial image to obtain a key point feature map of at least one scale.

According to embodiments of the present disclosure, N may be an integer greater than 1. N may be configured according to actual service requirements, which is not limited herein. For example, n=4.

According to the embodiment of the disclosure, feature extraction of N cascade levels can be performed on the table image, and at least one key point feature map corresponding to the N cascade levels is obtained. And obtaining the key point feature map of at least one scale according to the at least one key point feature map corresponding to the N cascade levels. For example, for an nth cascade level of the N cascade levels, a key point feature map of a scale corresponding to the nth cascade level is obtained from the key point feature maps of other cascade levels and the key point feature map corresponding to the nth cascade level. Other cascade levels may refer to at least some of the N cascade levels except for the nth cascade level.

According to the embodiment of the disclosure, since the key point feature map of at least one scale can provide richer information, the accuracy of the scale detection information can be improved by determining the scale detection information according to the key point feature map of at least one scale.

According to an embodiment of the present disclosure, obtaining scale detection information according to a key point feature map of at least one scale may include the following operations.

And obtaining a thermodynamic diagram corresponding to at least one key point according to the key point characteristic diagram of at least one scale. Scale detection information is determined from a thermodynamic diagram corresponding to at least one key point.

According to embodiments of the present disclosure, a keypoint may have a thermodynamic diagram corresponding to the keypoint. The pixel values of the pixels in the thermodynamic diagram may characterize the probability values that the pixels are the keypoints.

According to an embodiment of the present disclosure, obtaining a thermodynamic diagram corresponding to at least one keypoint according to a keypoint feature map of at least one scale may include: the first target keypoint feature map may be determined from the at least one scale keypoint feature map. And obtaining a thermodynamic diagram corresponding to at least one key point according to the first target key point characteristic diagram. The first target keypoint feature map may be a maximum image resolution keypoint feature map of the at least one scale of keypoint feature maps. Alternatively, the keypoint feature map of at least one scale may be fused to obtain a second target keypoint feature map. And obtaining a thermodynamic diagram corresponding to at least one key point according to the second target key point characteristic diagram.

According to embodiments of the present disclosure, for a keypoint of the at least one keypoint, a pixel having a probability value greater than or equal to a predetermined probability threshold may be determined from a thermodynamic diagram corresponding to the keypoint. And obtaining the position information corresponding to the key point according to the pixels with probability values larger than or equal to a preset probability threshold value in the thermodynamic diagram corresponding to the key point. The predetermined probability threshold may be configured according to actual traffic demands, and is not limited herein. For example, the predetermined probability threshold may be 0.85.

And processing the key point feature map of at least one scale based on a regression position method to obtain scale detection information.

According to an embodiment of the present disclosure, the regression location method may refer to a method of performing location regression processing directly from a key point feature map. And processing the key points of at least one scale by using a regression position method to obtain scale detection information.

According to an embodiment of the present disclosure, processing a key point feature map of at least one scale based on a regression location method to obtain scale detection information may include: a third target keypoint feature map may be determined from the at least one scale keypoint feature map. And processing the third target key point feature map based on the regression position method to obtain scale detection information. The third target keypoint feature map may be a maximum image resolution keypoint feature map of the at least one scale of keypoint feature maps. Alternatively, the keypoint feature map of at least one scale may be fused to obtain a fourth target keypoint feature map. And processing the fourth target key point feature map based on the regression position method to obtain scale detection information.

According to an embodiment of the present disclosure, operation S230 may include the following operations.

And obtaining first intermediate pointer identification information according to image segmentation information obtained by carrying out image segmentation on the disc image. And performing skeleton extraction on the first intermediate pointer identification information to obtain second intermediate pointer identification information. And performing straight line fitting on the second intermediate pointer information to obtain pointer identification information.

According to an embodiment of the present disclosure, the first intermediate pointer identification information may include at least one pointer region. The second intermediate pointer identification information may include a pointer corresponding to at least one pointer region.

According to the embodiment of the disclosure, the disc image can be segmented, and image segmentation information is obtained. And obtaining first intermediate pointer identification information according to the image segmentation information. The first intermediate pointer identification information may include at least one pointer region. After the at least one pointer region, the at least one pointer region may be processed using a morphological method to obtain second pointer identification information. And performing straight line fitting on the second pointer identification information to obtain the pointer identification information.

According to an embodiment of the present disclosure, processing at least one pointer region using a morphological method to obtain second pointer identification information may include: at least one pointer region may be processed using a skeleton extraction method to obtain second pointer identification information. Skeleton extraction (i.e., binary image refinement) may refer to refining a connected region into a width of one pixel for feature extraction and target topology characterization. The skeleton extraction method may include at least one of: a skeleton extraction algorithm based on hit-miss transformation and a skeleton extraction algorithm based on medial axis transformation. For example, the K3M algorithm. Setting starts burning from the boundary of the object in the binary image, the object is gradually thinned, but it is necessary to ensure that pixels satisfying a predetermined condition are preserved or "burned out" during the burning process. And under the condition that the combustion is determined to be finished, the last remaining binary image is the skeleton of the binary image.

According to an embodiment of the present disclosure, the processing at least one pointer region by using a skeleton extraction method to obtain second pointer identification information may include: and obtaining a communication area corresponding to the at least one pointer area according to the at least one pointer area. And processing the connected region corresponding to the at least one pointer region by using a skeleton extraction method to obtain second pointer identification information.

According to the embodiment of the disclosure, after the second pointer identification information is obtained, a pointer corresponding to at least one pointer region included in the second pointer identification information may be subjected to a straight line fitting based on a straight line fitting method to obtain the pointer identification information.

According to an embodiment of the present disclosure, the meter reading identification method may further include the following operations.

And extracting features of the dial image to obtain a backbone feature map. And according to the backbone feature map, semantic segmentation information and pixel feature characterization are obtained. And obtaining image segmentation information according to the semantic segmentation information and the pixel characteristic characterization.

According to embodiments of the present disclosure, semantic segmentation information may characterize coarse semantic segmentation information.

According to the embodiment of the disclosure, the backbone feature map can be processed to obtain semantic segmentation information and pixel feature characterization, and then image segmentation information is obtained according to the semantic segmentation information and the pixel feature characterization.

According to an embodiment of the present disclosure, processing the backbone feature map to obtain semantic segmentation information and pixel feature characterization may include: and up-sampling the backbone characteristic diagram to obtain the pixel characteristic characterization. The resolution of the pixel feature characterization is the same as the image resolution of the backbone feature map. And (5) up-sampling and convolution are carried out on the backbone characteristic diagram to obtain semantic segmentation information.

According to an embodiment of the present disclosure, obtaining image segmentation information from semantic segmentation information and pixel feature characterization may include the following operations.

And obtaining the characteristic representation of the object region according to the semantic segmentation information and the characteristic representation of the pixel. A relationship matrix between the semantic segmentation information and the object region feature characterization is determined. And obtaining the object context characteristic representation according to the relation matrix and the object region characteristic representation. And fusing the pixel characteristic representation and the object context characteristic representation to obtain the object enhanced context characteristic representation. And obtaining image segmentation information according to the object enhancement context characteristic representation.

According to an embodiment of the present disclosure, obtaining an object context feature representation from a relationship matrix and an object region feature representation may include: and carrying out weighted summation on the characteristics of the object region according to the values of the pixel and the object region characteristic in the relation matrix to obtain the object context characteristic.

According to an embodiment of the present disclosure, fusing the pixel feature representation and the object context feature representation to obtain the object enhancement context feature representation may include: the pixel characteristic representation and the object context characteristic representation can be spliced, and then the object enhancement context characteristic representation is obtained after the splicing information is convolved.

According to the embodiment of the disclosure, since the object enhancement context feature representation is obtained by fusing the pixel feature representation and the object context feature representation, the object context feature representation is determined according to the relation matrix and the object region feature representation, the object region feature representation is obtained according to the semantic segmentation information and the pixel feature representation, and the relation matrix is a matrix between the semantic segmentation information and the object region feature representation, the object region feature representation is propagated to the pixels according to the object region feature representation and the pixel feature representation, and therefore the accuracy of the image segmentation information is improved.

According to embodiments of the present disclosure, obtaining image segmentation information from object enhancement context feature characterization may include the following operations.

And carrying out cavity convolution processing on the dial image to obtain cavity characteristic characterization. And obtaining image segmentation information according to the object enhancement context characteristic representation and the cavity characteristic representation.

According to the embodiment of the disclosure, the object enhancement context feature characterization and the cavity feature characterization can be fused to obtain a fusion feature characterization. And obtaining image segmentation information according to the fusion characteristic representation. For example, the object enhancement contextual feature representation and the hole feature representation may be stitched to obtain a fusion feature representation. And reducing the number of channels represented by the fusion characteristics to the expected number of channels to obtain image segmentation information. The number of expected channels may be configured according to actual service requirements, and is not limited herein.

According to the embodiment of the disclosure, the cavity feature characterization is obtained by carrying out cavity convolution processing on the dial image, and then the image segmentation information is obtained according to the object enhanced context feature characterization and the cavity feature characterization, so that the context information of the dial image is captured in a plurality of proportions, and the cavity convolution can be used for relieving the contradiction between large calculated amount and lost image resolution caused by a large receptive field, thereby improving the accuracy of the image segmentation information.

And extracting features of the dial image to obtain a second backbone feature map. And obtaining a first intermediate feature map of at least one scale according to the second backbone feature map. And obtaining a second intermediate feature map according to the second backbone feature map and the intermediate feature map of at least one scale. And obtaining image segmentation information according to the second intermediate feature map.

According to an embodiment of the present disclosure, obtaining a first intermediate feature map of at least one scale from a second backbone feature map may include: and carrying out pooling treatment on the second backbone characteristic map to obtain a third intermediate characteristic map with at least one scale. And carrying out convolution processing on the third intermediate feature map of at least one scale to obtain a fourth intermediate feature map of at least one scale. And up-sampling the fourth intermediate feature map of at least one scale to obtain a fifth intermediate feature map of at least one scale. And obtaining image segmentation information according to the second backbone characteristic diagram and the fifth intermediate characteristic diagram of at least one scale. For example, the second backbone graph and the fifth intermediate feature graph of at least one scale may be fused to obtain a sixth intermediate feature graph. And obtaining image segmentation information according to the sixth intermediate feature map.

In the case where it is determined that the pointer instrument is a predetermined type of pointer instrument, the dial image is corrected according to the scale detection information, resulting in a corrected dial image.

And obtaining pointer identification information according to the image segmentation information obtained by carrying out image segmentation on the corrected dial image.

According to an embodiment of the present disclosure, in an ideal case, the photographing axis of the vision sensor coincides with the rotation center of the pointer, and the photographing plane is parallel to the dial plane. In the process of acquiring the instrument image, because the mobile vision sensor and the fixed vision sensor are based on the cloud deck shooting pointer instrument, a shooting inclination angle exists, so that inclination distortion of the instrument image exists, and accuracy of instrument indication identification information is affected.

According to an embodiment of the present disclosure, the predetermined class of pointer meters may be pointer meters having a scale range greater than or equal to the predetermined scale range. The predetermined scale range may be configured according to actual service requirements, and is not limited herein. For example, the predetermined scale range may be 180 °.

According to an embodiment of the present disclosure, it may be determined whether the pointer meter is a predetermined type of pointer meter. In the case where it is determined that the pointer instrument is a predetermined type of pointer instrument, the dial image may be corrected according to the scale detection information, resulting in a corrected dial image. For example, it is determined whether the scale range of the pointer instrument is greater than or equal to a predetermined scale range. And correcting the dial image according to the scale detection information under the condition that the scale range of the pointer instrument is larger than the preset scale range, so as to obtain a corrected dial image, and determining pointer identification information for the dial image by using the corrected dial image.

According to the embodiment of the disclosure, the accuracy of the dial image is improved, and the accuracy of meter indication recognition is further improved by correcting the dial image according to the scale detection information under the condition that the pointer meter is determined to be a pointer meter of a preset type.

According to the embodiment of the disclosure, the dial image is corrected according to the scale detection information, so that the corrected dial image can be obtained, and the following operation can be included.

And performing ellipse fitting on the position information of at least one key point to obtain a target ellipse. And determining an circumscribed circle corresponding to the target ellipse. And determining a transformation matrix according to the target ellipse and the circumscribed circle. And correcting the dial image according to the transformation matrix to obtain a corrected dial image.

According to embodiments of the present disclosure, the center of the circumscribed circle may coincide with the center of the target ellipse. The diameter of the circumscribed circle may be coincident with the major axis of the target ellipse.

According to the embodiment of the disclosure, the target ellipse can be obtained according to the position information of at least one key point based on an ellipse fitting method. For example, the ellipse fitting method may include at least one of: an ellipse fitting method based on a least square method, an ellipse fitting method based on a direct calculation method, an ellipse fitting method based on a standard equation and the like.

According to embodiments of the present disclosure, after determining the target ellipse, a minimum circumscribed circle corresponding to the target ellipse may be determined. The center of the minimum circumscribing circle is consistent with the center of the target ellipse. The diameter of the smallest circumscribed circle may be coincident with the major axis of the target ellipse. The diameter of the smallest circumscribed circle may coincide with the major axis of the target ellipse. The diameter of the smallest circumscribing circle may also be parallel to the minor axis of the target ellipse.

According to embodiments of the present disclosure, after determining the target ellipse and the minimum circumscribing circle, at least four intersection points may be determined according to the target ellipse and the minimum circumscribing circle. A transformation matrix is determined based on the at least four intersection points. The transformation matrix may be a perspective transformation matrix. And correcting the dial image according to the perspective transformation matrix to obtain a corrected dial image.

According to embodiments of the present disclosure, determining a transformation matrix from the target ellipse and the circumscribed circle may include the following operations.

And respectively determining the intersection points of the major axis and the minor axis of the target ellipse and the circumscribing circle to obtain a plurality of intersection points. A transformation matrix is determined based on the first position information and the second position information corresponding to the plurality of intersections.

According to an embodiment of the present disclosure, the first position information may be position information where the intersection point is located in the target ellipse. The second position information may be position information where the intersection point is located in the circumscribed circle.

According to an embodiment of the present disclosure, the transmission transformation matrix may include 8 parameters to be solved. And determining an intersection point of the long axis of the target ellipse and the minimum circumscribing circle to obtain a first intersection point, a second intersection point, a third intersection point and a fourth intersection point. And determining the intersection point of the short axis of the target ellipse and the minimum circumscribing circle to obtain a fifth intersection point, a sixth intersection point, a seventh intersection point and an eighth intersection point. The first intersection point and the second intersection point are an intersection point pair. The third intersection point and the fourth intersection point are an intersection point pair. The fifth intersection point and the sixth intersection point are an intersection point pair. The seventh intersection point and the eighth intersection point are an intersection point pair. A transmission transformation matrix is determined based on the first position information of the first intersection, the second position information of the second intersection, the first position information of the third intersection, the second position information of the fourth intersection, the first position information of the fifth intersection, the second position information of the sixth intersection, the first position information of the seventh intersection, and the second position information of the eighth intersection.

Fig. 5 schematically illustrates an example schematic diagram of determining a transformation matrix from a target ellipse and a circumscribed circle according to an embodiment of the disclosure.

As shown in fig. 5, at 500, a minimum circumscribed circle 502 corresponding to a target ellipse 501 is determined. An intersection point of the major axis of the target ellipse 501 and the minimum circumscribed circle 502 is determined, and a first intersection point 5011, a second intersection point 5021, a third intersection point 5012, and a fourth intersection point 5022 are obtained. An intersection of the minor axis of the target ellipse 501 and the minimum circumscribed circle 502 is determined, resulting in a fifth intersection 5013, a sixth intersection 5023, a seventh intersection 5014, and an eighth intersection 5024. The transformation matrix is determined from the first, second, third, fourth, fifth, sixth, seventh and eighth intersections 5011, 5021, 5022, 5013, 5023, 5024.

According to an embodiment of the present disclosure, the pointer identification information may include at least one pointer.

And determining the center of the pointer according to the center of the circumscribed circle. And translating the pointer straight line to pass through the center of the pointer to obtain the corrected pointer straight line. And obtaining corrected pointer identification information according to the corrected pointer straight line.

According to an embodiment of the present disclosure, operation S240 may include the following operations.

And obtaining instrument indication identification information according to the scale detection information and the corrected pointer identification information.

According to embodiments of the present disclosure, the pointer straight line may be a straight line in which the pointer is located.

According to the embodiment of the disclosure, in the case that the meter image has oblique distortion, the dial plane and the pointer are not in the same plane, and thus, in the case that the pointer is projected to the dial plane, there is caused a deviation between the pointer direction and the actual pointer direction. The actual pointer direction of the pointer is through the center of the pointer.

According to embodiments of the present disclosure, the center of the circumscribed circle may be determined as the pointer center. And translating the pointer straight line to pass through the center of the pointer to obtain the corrected pointer straight line. The corrected pointer straight line is the actual pointer point.

According to the embodiment of the disclosure, the meter indication identification information can be obtained according to the corrected scale detection information and the corrected pointer identification information. The corrected scale detection information may be obtained from the corrected dial image.

According to the embodiment of the disclosure, the pointer center is determined according to the circle center of the circumscribed circle, the pointer straight line is translated to the pointer center, the corrected pointer straight line is obtained, the accuracy of pointer identification information is improved, and the accuracy of instrument indication identification is further improved.

Fig. 6A schematically illustrates a schematic diagram of a meter reading identification method according to an embodiment of the present disclosure.

As shown in fig. 6A, in 600A, a dial image 603 is determined from a meter image 601 based on target detection information 602 obtained by detecting the meter image 601 of the pointer meter. The dial image 603 is subjected to key point detection, and scale detection information 604 is obtained. Pointer identification information 605 is obtained from image division information obtained by image division of the disk image 603. The meter indication identification information 607 is obtained from the scale detection information 604, the pointer identification information 605, and the scale indication information 606 corresponding to the scale detection information 604.

According to the embodiment of the disclosure, the instrument indication recognition method does not need to be calibrated on site, so that the method has higher efficient site deployment capability. In addition, the method is based on the data-driven instrument indication recognition method, so that the method has better iterative evolutionary capability.

Fig. 6B schematically illustrates an example schematic diagram of meter reading identification information according to an embodiment of the disclosure.

As shown in fig. 6B, in 600B, the meter image 608 may be a meter image of a pressure pointer meter. The meter image 609 may be a meter image of a temperature pointer meter. The meter image 610 may be a meter image of a wetness pointer meter.

The meter image 608, the meter image 609, and the meter image 610 may be processed using the meter reading identification method according to the embodiment of the present disclosure to obtain meter reading identification information of the pressure pointer meter, meter reading identification information of the temperature pointer meter, and meter reading identification information of the humidity pointer meter. The gauge indication identifying information of the pressure pointer gauge is 0.3MP. The meter reading identification information of the temperature pointer meter is 20 ℃. The meter reading identification information of the wetness indicator meter is 80% RH. "RH" characterizes "relative humidity (i.e., relative Humidity)".

The above is only an exemplary embodiment, but not limited thereto, and other meter indication recognition methods known in the art may be included as long as it can be adapted to indication recognition of various types of pointer meters in various environments, and the versatility of indication recognition schemes is improved.

In the technical scheme of the disclosure, the related processes of collecting, storing, using, processing, transmitting, providing, disclosing and the like of the personal information of the user accord with the regulations of related laws and regulations, and the public order colloquial is not violated.

Fig. 7 schematically illustrates a block diagram of a meter reading identification device according to an embodiment of the disclosure.

As shown in fig. 7, the meter reading identification device 700 may include a first determination module 710, a first obtaining module 720, a second obtaining module 730, and a third obtaining module 740.

The first determining module 710 is configured to determine a dial image from the meter image according to target detection information obtained by detecting the meter image of the pointer meter.

The first obtaining module 720 is configured to perform key point detection on the dial image to obtain scale detection information. The scale detection information includes location information of at least one key point. The key points characterize the points of the target scale. The target scale has at least one key point corresponding to the target scale, the target scale being a scale having an indication.

The second obtaining module 730 is configured to obtain pointer identification information according to image segmentation information obtained by performing image segmentation on the disc image.

The third obtaining module 740 is configured to obtain meter indication identifying information according to the scale detecting information, the pointer identifying information, and the scale indication information corresponding to the scale detecting information.

According to an embodiment of the present disclosure, the first obtaining module 720 may include a first obtaining sub-module and a second obtaining sub-module.

The first obtaining submodule is used for extracting features of the dial image to obtain a key point feature map of at least one scale.

And the second obtaining submodule is used for obtaining scale detection information according to the key point feature map of at least one scale.

According to an embodiment of the present disclosure, the first obtaining sub-module may include a first obtaining unit and a second obtaining unit.

The first obtaining unit is used for extracting the features of M stages of dial images to obtain at least one key point feature map corresponding to the M th stage.

The second obtaining unit is used for obtaining at least one key point characteristic diagram according to at least one key point characteristic diagram corresponding to the M-th stage;

according to an embodiment of the present disclosure, the mth stage has T _m And a plurality of parallel hierarchies. The image resolution of the keypoint feature map of the same parallel hierarchy is the same. The image resolution of the keypoint feature map is different for different parallel levels.

According to an embodiment of the present disclosure, M is an integer greater than or equal to 1. M is an integer greater than or equal to 1 and less than or equal to M. T (T) _m Is an integer greater than or equal to 1.

According to an embodiment of the present disclosure, in the case where M is an integer greater than 1, the first obtaining unit may include a first obtaining subunit and a second obtaining subunit.

The first obtaining subunit is configured to perform convolution processing on at least one key point feature map corresponding to the m-1 th stage to obtain at least one intermediate key point feature map corresponding to the m-1 th stage.

The second obtaining subunit is used for carrying out feature fusion on at least one middle key point feature map corresponding to the mth stage to obtain at least one key point feature map corresponding to the mth stage;

according to an embodiment of the present disclosure, M is an integer greater than 1 and less than or equal to M.

According to an embodiment of the present disclosure, the second obtaining subunit may be configured to: for T _m And obtaining the key point feature map corresponding to the ith parallel hierarchy according to the other intermediate key point feature maps corresponding to the ith parallel hierarchy and the intermediate key point feature map corresponding to the ith parallel hierarchy in the ith parallel hierarchy.

According to an embodiment of the present disclosure, the other intermediate keypoint feature map corresponding to the ith parallel hierarchy is the one corresponding to T _m And at least part of parallel hierarchies except the ith parallel hierarchy corresponds to the middle key point feature map. i is greater than or equal to 1 and less than or equal to T _m Is an integer of (a).

According to an embodiment of the present disclosure, the first obtaining sub-module may include a third obtaining unit.

And the third obtaining unit is used for carrying out feature extraction of N cascade levels on the dial image to obtain a key point feature map of at least one scale. N is an integer greater than 1.

According to an embodiment of the present disclosure, the second obtaining sub-module may include a fourth obtaining unit and a fifth obtaining unit.

And the fourth obtaining unit is used for obtaining a thermodynamic diagram corresponding to at least one key point according to the key point characteristic diagram of at least one scale. The pixel values of the pixels in the thermodynamic diagram characterize the probability values that the pixels are keypoints.

And a fifth obtaining unit, configured to determine scale detection information according to a thermodynamic diagram corresponding to at least one key point.

According to an embodiment of the present disclosure, the second obtaining sub-module may include a sixth obtaining unit.

And the sixth obtaining unit is used for processing the key point feature map of at least one scale based on the regression position method to obtain scale detection information.

According to an embodiment of the present disclosure, the second obtaining module 730 may include a third obtaining sub-module, a fourth obtaining sub-module, and a fifth obtaining sub-module.

And the third obtaining sub-module is used for obtaining the first middle pointer identification information according to the image segmentation information obtained by carrying out image segmentation on the table disc image. The first intermediate pointer identification information includes at least one pointer region.

And the fourth obtaining submodule is used for carrying out skeleton extraction on the first intermediate pointer identification information to obtain second intermediate pointer identification information. The second intermediate pointer identification information includes pointers corresponding to at least one pointer region.

And a fifth obtaining sub-module, configured to perform straight line fitting on the second intermediate pointer information to obtain pointer identification information.

According to an embodiment of the present disclosure, the meter reading identification device 700 may further include a fourth obtaining module, a fifth obtaining module, and a sixth obtaining module.

And the fourth obtaining module is used for extracting the characteristics of the dial image to obtain a backbone characteristic diagram.

And a fifth obtaining module, configured to obtain semantic segmentation information and pixel feature characterization according to the backbone feature map.

And a sixth obtaining module, configured to obtain image segmentation information according to the semantic segmentation information and the pixel feature representation.

According to an embodiment of the present disclosure, the sixth obtaining module may include a sixth obtaining sub-module, a first determining sub-module, a seventh obtaining self-building module, an eighth obtaining sub-module, and a ninth obtaining sub-module.

And a sixth obtaining sub-module, configured to obtain an object region feature representation according to the semantic segmentation information and the pixel feature representation.

And the first determining submodule is used for determining a relation matrix between the semantic segmentation information and the characteristic characterization of the object region.

And a seventh obtaining submodule, configured to obtain an object context feature representation according to the relation matrix and the object region feature representation.

And an eighth obtaining submodule, configured to fuse the pixel feature representation and the object context feature representation to obtain an object enhancement context feature representation.

And a ninth obtaining sub-module, configured to obtain image segmentation information according to the object enhancement context feature representation.

According to an embodiment of the present disclosure, the ninth obtaining sub-module may include a seventh obtaining unit and an eighth obtaining unit.

And the seventh obtaining unit is used for carrying out cavity convolution processing on the dial image to obtain cavity characteristic characterization.

And the eighth obtaining unit is used for obtaining image segmentation information according to the object enhancement context characteristic representation and the cavity characteristic representation.

According to an embodiment of the present disclosure, the meter reading recognition device 700 may further include a seventh obtaining module, an eighth obtaining module, a ninth obtaining module, and a tenth obtaining module.

And a seventh obtaining module, configured to perform feature extraction on the dial image, to obtain a second backbone feature map.

And an eighth obtaining module, configured to obtain a first intermediate feature map of at least one scale according to the second backbone feature map.

And a ninth obtaining module, configured to obtain a second intermediate feature map according to the second backbone feature map and the intermediate feature map of at least one scale.

And a tenth obtaining module, configured to obtain image segmentation information according to the second intermediate feature map.

According to an embodiment of the present disclosure, the first determination module 710 may include a tenth acquisition sub-module, an eleventh acquisition sub-module, a twelfth acquisition sub-module, a second determination sub-module, and a third determination sub-module.

And a tenth obtaining sub-module, configured to perform feature extraction on the meter image of the pointer meter, and obtain a meter feature map of at least one scale.

And the eleventh obtaining sub-module is used for fusing the instrument characteristic diagrams of at least one scale to obtain fused characteristic diagrams.

And a twelfth obtaining sub-module, configured to obtain target detection information according to the fusion feature map. The target detection information includes location information, category information, and confidence of at least one region. The confidence level characterizes the confidence level of the category information. The category information includes dial areas and non-dial areas.

And the second determining submodule is used for determining dial areas from at least one area according to the confidence and the category information.

And the third determining submodule is used for determining the dial image from the instrument image according to the position information of the dial area.

The meter reading identification device 700 may further include an eleventh obtaining module according to an embodiment of the present disclosure.

An eleventh obtaining module for correcting the dial image according to the scale detection information to obtain a corrected dial image in the case where it is determined that the pointer instrument is a predetermined type of pointer instrument.

According to an embodiment of the present disclosure, the second obtaining module 730 may include a thirteenth obtaining sub-module.

A thirteenth obtaining sub-module is configured to obtain pointer identification information according to image segmentation information obtained by performing image segmentation on the corrected dial image.

According to an embodiment of the present disclosure, the eleventh obtaining module may include a fourteenth obtaining sub-module, a fourth determining sub-module, a fifth determining sub-module, and a fifteenth determining sub-module.

And a fourteenth obtaining sub-module, configured to perform ellipse fitting on the position information of at least one key point, so as to obtain a target ellipse.

And the fourth determination submodule is used for determining a circumscribed circle corresponding to the target ellipse. The center of the circumscribing circle is consistent with the center of the target ellipse, and the diameter of the circumscribing circle is consistent with the long axis of the target ellipse.

And the fifth determining submodule is used for determining a transformation matrix according to the target ellipse and the circumscribed circle.

And a fifteenth obtaining sub-module, configured to correct the dial image according to the transformation matrix, and obtain a corrected dial image.

According to an embodiment of the present disclosure, the fifth determination sub-module may include a ninth obtaining unit and a determining unit.

And a ninth obtaining unit, configured to determine intersection points of the major axis and the minor axis of the target ellipse and the circumscribing circle respectively, so as to obtain a plurality of intersection points.

And a determining unit configured to determine a transformation matrix based on the first position information and the second position information corresponding to the plurality of intersections. The first position information is position information where the intersection point is located in the target ellipse. The second position information is position information where the intersection point is located in the circumscribed circle.

According to an embodiment of the present disclosure, the pointer identification information includes at least one pointer.

According to an embodiment of the present disclosure, the meter reading recognition apparatus 700 may further include a second determination module, a twelfth obtaining module, and a thirteenth obtaining module.

And the second determining module is used for determining the center of the pointer according to the circle center of the circumscribed circle.

And a twelfth obtaining module, configured to translate the pointer straight line to pass through the center of the pointer, so as to obtain a corrected pointer straight line, where the pointer straight line is the straight line where the pointer is located.

A thirteenth obtaining module, configured to obtain corrected pointer identification information according to the corrected pointer straight line;

according to an embodiment of the present disclosure, the third obtaining module 740 may include a sixteenth obtaining sub-module.

The sixteenth obtaining sub-module is used for obtaining instrument indication identifying information according to the scale detecting information, the corrected pointer identifying information and the scale indication information corresponding to the scale detecting information.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described above.

According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform a method as above.

According to an embodiment of the present disclosure, a computer program product comprising a computer program which, when executed by a processor, implements a method as above.

Fig. 8 schematically illustrates a block diagram of an electronic device suitable for implementing a meter reading identification method according to an embodiment of the disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 8, the electronic device 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to the bus 804.

Various components in electronic device 800 are connected to I/O interface 805, including: an input unit 806 such as a keyboard, mouse, etc.; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, or the like. The communication unit 809 allows the electronic device 800 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.

The computing unit 801 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The calculation unit 801 performs the respective methods and processes described above, for example, the meter reading recognition method. For example, in some embodiments, the meter reading identification method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 800 via the ROM 802 and/or the communication unit 809. When the computer program is loaded into RAM 803 and executed by computing unit 801, one or more steps of the meter reading identification method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the meter reading identification method in any other suitable manner (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel or sequentially or in a different order, provided that the desired results of the technical solutions of the present disclosure are achieved, and are not limited herein.

The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims

1. An instrument indication recognition method, comprising:

according to target detection information obtained by detecting an instrument image of a pointer instrument, determining a dial image from the instrument image;

performing key point detection on the dial image to obtain scale detection information, wherein the scale detection information comprises position information of at least one key point, the key point represents a point of a target scale, the target scale is provided with at least one key point corresponding to the target scale, the target scale is a scale with an indication, the number of the target scales is determined according to at least one target number, and the target number refers to the number of target scales included in a pointer instrument of a preset instrument type;

Obtaining pointer identification information according to image segmentation information obtained by carrying out image segmentation on the dial image; and

obtaining instrument indication identification information according to the scale detection information, the pointer identification information and scale indication information corresponding to the scale detection information;

wherein, according to the scale detection information, the pointer identification information and the scale indication information corresponding to the scale detection information, obtaining instrument indication identification information comprises:

determining the scale pointed by the pointer according to the pointer pointing direction corresponding to at least one pointer included in the pointer identification information; and

and under the condition that the scale is the target scale, obtaining the instrument indication identification information according to the indication corresponding to the target scale in the scale indication information.

2. The method of claim 1, wherein the performing the keypoint detection on the dial image to obtain scale detection information includes:

extracting features of the dial image to obtain a key point feature map of at least one scale; and

and obtaining the scale detection information according to the key point feature map of the at least one scale.

3. The method of claim 2, wherein the feature extraction of the dial image to obtain a key point feature map of at least one scale comprises:

extracting features of M stages from the dial image to obtain at least one key point feature map corresponding to an M-th stage; and

obtaining a key point feature map of at least one scale according to at least one key point feature map corresponding to the M-th stage;

wherein the mth stage has T _m The image resolutions of the key point feature maps of the same parallel hierarchy are the same, and the image resolutions of the key point feature maps of different parallel hierarchies are different;

wherein M is an integer greater than or equal to 1, M is an integer greater than or equal to 1 and less than or equal to M, T _m Is an integer greater than or equal to 1.

4. The method according to claim 3, wherein, in the case where M is an integer greater than 1, the performing feature extraction of M stages on the dial image, to obtain at least one key point feature map corresponding to an mth stage, includes:

carrying out convolution processing on at least one key point feature map corresponding to the m-1 stage to obtain at least one intermediate key point feature map corresponding to the m stage; and

Feature fusion is carried out on at least one middle key point feature map corresponding to the mth stage, and at least one key point feature map corresponding to the mth stage is obtained;

wherein M is an integer greater than 1 and less than or equal to M.

5. The method of claim 4, wherein the feature fusion of the at least one intermediate keypoint feature map corresponding to the mth stage to obtain the at least one keypoint feature map corresponding to the mth stage comprises:

for said T _m The ith parallel level of the parallel levels,

obtaining a key point feature map corresponding to the ith parallel hierarchy according to other intermediate key point feature maps corresponding to the ith parallel hierarchy and intermediate key point feature maps corresponding to the ith parallel hierarchy;

wherein the other intermediate key point feature map corresponding to the ith parallel hierarchy is the same as the T _m At least part of parallel hierarchies except the ith parallel hierarchy corresponds to an intermediate key point characteristic diagram, i is greater than or equal to 1 and less than or equal to T _m Is an integer of (a).

6. The method of claim 2, wherein the feature extraction of the dial image to obtain a key point feature map of at least one scale comprises:

And carrying out feature extraction of N cascade levels on the dial image to obtain a key point feature map of the at least one scale, wherein N is an integer greater than 1.

7. The method according to any one of claims 2-6, wherein the obtaining the scale detection information according to the at least one scale key point feature map includes:

obtaining a thermodynamic diagram corresponding to the at least one key point according to the key point feature diagram of the at least one scale, wherein the pixel value of a pixel in the thermodynamic diagram represents the probability value that the pixel is the key point; and

and determining the scale detection information according to a thermodynamic diagram corresponding to the at least one key point.

8. The method according to any one of claims 2-6, wherein the obtaining the scale detection information according to the at least one scale key point feature map includes:

and processing the key point feature map of at least one scale based on a regression position method to obtain the scale detection information.

9. The method according to any one of claims 1 to 8, wherein the obtaining pointer identification information from image division information obtained by image division of the dial image includes:

Obtaining first intermediate pointer identification information according to image segmentation information obtained by carrying out image segmentation on the dial image, wherein the first intermediate pointer identification information comprises at least one pointer area;

performing skeleton extraction on the first intermediate pointer identification information to obtain second intermediate pointer identification information, wherein the second intermediate pointer identification information comprises pointers corresponding to the at least one pointer region; and

and performing straight line fitting on the second intermediate pointer information to obtain the pointer identification information.

10. The method of any one of claims 1-9, further comprising:

extracting features of the dial image to obtain a first backbone feature map;

according to the first backbone feature map, semantic segmentation information and pixel feature characterization are obtained; and

and obtaining the image segmentation information according to the semantic segmentation information and the pixel characteristic representation.

11. The method of claim 10, wherein the deriving the image segmentation information from the semantic segmentation information and the pixel feature characterization comprises:

obtaining object region feature representation according to the semantic segmentation information and the pixel feature representation;

Determining a relation matrix between the semantic segmentation information and the object region feature characterization;

obtaining object context characteristic representation according to the relation matrix and the object region characteristic representation;

fusing the pixel characteristic representation and the object context characteristic representation to obtain an object enhanced context characteristic representation; and

and obtaining the image segmentation information according to the object enhancement context characteristic representation.

12. The method of claim 11, wherein the deriving the image segmentation information from the object enhancement context feature characterization comprises:

carrying out cavity convolution processing on the dial image to obtain cavity feature characterization; and

and obtaining the image segmentation information according to the object enhancement context characteristic representation and the cavity characteristic representation.

13. The method of any one of claims 1-9, further comprising:

extracting features of the dial plate image to obtain a second backbone feature map;

obtaining a first intermediate feature map of at least one scale according to the second backbone feature map;

obtaining a second intermediate feature map according to the second backbone feature map and the intermediate feature map of at least one scale; and

And obtaining the image segmentation information according to the second intermediate feature map.

14. The method according to any one of claims 1 to 13, wherein the determining a dial image from the meter image of the pointer meter based on target detection information obtained by measuring the meter image includes:

extracting features of the instrument images of the pointer instrument to obtain an instrument feature map of at least one scale;

fusing the instrument feature graphs of at least one scale to obtain a fused feature graph;

obtaining target detection information according to the fusion feature map, wherein the target detection information comprises position information, category information and confidence level of at least one area, the confidence level represents the credibility of the category information, and the category information comprises dial areas and non-dial areas;

determining the dial area from the at least one area according to the confidence and the category information; and

and determining the dial image from the instrument image according to the position information of the dial area.

15. The method of any one of claims 1-8, further comprising:

correcting the dial image according to the scale detection information under the condition that the pointer instrument is determined to be a pointer instrument of a preset type, so as to obtain a corrected dial image;

The method for obtaining pointer identification information according to the image segmentation information obtained by carrying out image segmentation on the dial image comprises the following steps:

and obtaining the pointer identification information according to the image segmentation information obtained by carrying out image segmentation on the corrected dial image.

16. The method of claim 15, wherein the correcting the dial image according to the scale detection information to obtain a corrected dial image comprises:

performing ellipse fitting on the position information of the at least one key point to obtain a target ellipse;

determining an circumscribed circle corresponding to the target ellipse, wherein the center of the circumscribed circle is consistent with the center of the target ellipse, and the diameter of the circumscribed circle is consistent with the long axis of the target ellipse;

determining a transformation matrix according to the target ellipse and the circumscribed circle; and

and correcting the dial image according to the transformation matrix to obtain the corrected dial image.

17. The method of claim 16, wherein the determining a transformation matrix from the target ellipse and the circumscribed circle comprises:

respectively determining intersection points of the major axis and the minor axis of the target ellipse and the circumscribed circle to obtain a plurality of intersection points; and

And determining the transformation matrix according to first position information and second position information corresponding to the plurality of intersection points, wherein the first position information is the position information of the intersection points positioned on the target ellipse, and the second position information is the position information of the intersection points positioned on the circumscribed circle.

18. The method of claim 17, wherein the pointer identification information includes at least one pointer;

the method further comprises the steps of:

determining the center of the pointer according to the circle center of the circumscribed circle;

translating a pointer straight line to pass through the center of the pointer to obtain a corrected pointer straight line, wherein the pointer straight line is the straight line where the pointer is located; and

obtaining corrected pointer identification information according to the corrected pointer straight line;

and obtaining the instrument indication identification information according to the scale detection information, the corrected pointer identification information and the scale indication information corresponding to the scale detection information.

19. An instrument registration recognition device, comprising:

the first determining module is used for determining a dial image from the instrument image according to target detection information obtained by detecting the instrument image of the pointer instrument;

the first obtaining module is used for carrying out key point detection on the dial plate image to obtain scale detection information, wherein the scale detection information comprises position information of at least one key point, the key point represents a point of a target scale, the target scale is provided with at least one key point corresponding to the target scale, the target scale is a scale with an indication number, the number of the target scales is determined according to at least one target number, and the target number refers to the number of target scales included by a pointer instrument of a preset instrument type;

the second obtaining module is used for obtaining pointer identification information according to image segmentation information obtained by carrying out image segmentation on the dial image; and

the third obtaining module is used for obtaining instrument indication identification information according to the scale detection information, the pointer identification information and scale indication information corresponding to the scale detection information;

wherein the third obtaining module is configured to:

20. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 18.

21. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-18.