Nothing Special   »   [go: up one dir, main page]

CN111767883B - Question correction method and device - Google Patents

Question correction method and device Download PDF

Info

Publication number
CN111767883B
CN111767883B CN202010645304.9A CN202010645304A CN111767883B CN 111767883 B CN111767883 B CN 111767883B CN 202010645304 A CN202010645304 A CN 202010645304A CN 111767883 B CN111767883 B CN 111767883B
Authority
CN
China
Prior art keywords
detection
region
detection sub
sub
image description
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010645304.9A
Other languages
Chinese (zh)
Other versions
CN111767883A (en
Inventor
杨万里
郭常圳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Ape Power Future Technology Co Ltd
Original Assignee
Beijing Ape Power Future Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Ape Power Future Technology Co Ltd filed Critical Beijing Ape Power Future Technology Co Ltd
Priority to CN202010645304.9A priority Critical patent/CN111767883B/en
Publication of CN111767883A publication Critical patent/CN111767883A/en
Application granted granted Critical
Publication of CN111767883B publication Critical patent/CN111767883B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/40Document-oriented image-based pattern recognition
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/044Recurrent networks, e.g. Hopfield networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10Character recognition

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Multimedia (AREA)
  • Electrically Operated Instructional Devices (AREA)
  • Character Input (AREA)

Abstract

The application provides a question correction method and a device, wherein the question correction method comprises the following steps: receiving a picture to be identified, wherein the picture to be identified comprises a topic to be modified; performing target detection on the picture to be identified, and determining a first detection area and a second detection area corresponding to the topic to be modified; performing image description and identification on the first detection area to obtain image description information of the first detection area, and performing text description and identification on the second detection area to obtain text information of the second detection area; and determining the correction result of the questions to be corrected according to the image description information and the text information, comparing the image description information identifying the first detection area and the text information identifying the second detection area, and directly correcting the graphics questions, wherein the correction execution efficiency is high, the consumption of calculation resources is less, and the final correction accuracy of the questions is higher.

Description

Question correction method and device
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a method and apparatus for topic modification, a computing device, and a computer readable storage medium.
Background
Along with the development of computer technology, online education is rapidly developed, and corresponding teaching tool products are also generated, so that technical support and help in education and coaching are provided for students, teachers and parents, and a plurality of teaching tool products can provide the function of correcting the photographing of the questions.
The existing tool for performing the correction function on the questions is capable of intelligently solving the problem types of the formulas in the primary school stage, and can not directly process the correction of the graphics questions such as abacus, and most of the problem correction of the problem types is realized by replacing the problem types with a scheme of searching the pictures, but the correction of the questions can be performed only when the problem types corresponding to the problem types exist in the question bank by using the method of searching the pictures, and the problem types of the graphics are huge in number and complex in category.
Therefore, how to solve the above-mentioned problems and improve the correction efficiency of graphic subjects is a problem to be solved by technicians.
Disclosure of Invention
In view of the foregoing, embodiments of the present application provide a method and apparatus for topic modification, a computing device and a computer readable storage medium to solve the technical drawbacks in the prior art.
According to a first aspect of an embodiment of the present application, there is provided a topic modification method, including:
receiving a picture to be identified, wherein the picture to be identified comprises a topic to be modified;
performing target detection on the picture to be identified, and determining a first detection area and a second detection area corresponding to the topic to be modified;
performing image description and identification on the first detection area to obtain image description information of the first detection area, and performing text description and identification on the second detection area to obtain text information of the second detection area;
and determining the correction result of the questions to be corrected according to the image description information and the text information.
Optionally, performing target detection on the picture to be identified, and determining a first detection area and a second detection area corresponding to the topic to be modified, where the determining includes:
inputting the picture to be identified into a target detection model to carry out target detection, and determining a first detection area corresponding to the topic to be modified;
And inputting the picture to be identified into a text box detection model to carry out target detection, and determining a second detection area corresponding to the topic to be modified.
Optionally, the first detection region includes at least one first detection sub-region;
performing image description identification on the first detection area to obtain image description information of the first detection area, wherein the image description information comprises the following steps:
and carrying out image description identification on the first detection areas through an image description model, and obtaining image description information corresponding to each first detection sub-area.
Optionally, the second detection region comprises at least one second detection sub-region;
performing text description recognition on the second detection area to obtain text information of the second detection area, wherein the text description recognition comprises the following steps:
and carrying out image description recognition on the second detection areas through a text recognition model, and obtaining text information corresponding to each second detection sub-area.
Optionally, the first detection region includes at least one first detection sub-region, and the second detection region includes at least one second detection sub-region;
determining a correction result of the to-be-corrected question according to the image description information and the text information, wherein the correction result comprises the following steps:
And determining the correction result of the questions to be corrected according to the image description information corresponding to each first detection subarea and the text information corresponding to each second detection subarea.
Optionally, determining a correction result of the to-be-corrected question according to the image description information corresponding to each first detection sub-region and the text information corresponding to each second detection sub-region includes:
matching a corresponding second detection subarea for each first detection subarea;
comparing the image description information of each first detection sub-region with the text information of a second detection sub-region corresponding to the first detection sub-region, and determining a modification result corresponding to the title of each first detection sub-region.
Optionally, matching a corresponding second detection sub-region for each first detection sub-region includes:
acquiring a first target point coordinate, a first length and a first width corresponding to each first detection sub-region, and acquiring a second target point coordinate, a second length and a second width corresponding to each second detection sub-region;
determining first detection sub-region position information corresponding to each first detection sub-region according to first target point coordinates, first length and first width corresponding to each first detection sub-region, and determining second detection sub-region position information corresponding to each second detection sub-region according to second target point coordinates, second length and second width corresponding to each second detection sub-region;
And determining a second detection sub-region corresponding to each first detection sub-region according to the position information of each first detection sub-region and the position information of each second detection sub-region.
Optionally, the method further comprises:
and returning a question type disagreement prompt of the questions to be modified under the condition that the first detection area corresponding to the questions to be modified is not detected.
According to a second aspect of embodiments of the present application, there is provided a topic modification apparatus, including:
the receiving module is configured to receive a picture to be identified, wherein the picture to be identified comprises a subject to be modified;
the detection module is configured to carry out target detection on the picture to be identified and determine a first detection area and a second detection area corresponding to the topic to be modified;
the identification module is configured to perform image description identification on the first detection area to obtain image description information of the first detection area, and perform text description identification on the second detection area to obtain text information of the second detection area;
and the determining module is configured to determine the correction result of the to-be-corrected problem according to the image description information and the text information.
According to a third aspect of embodiments of the present application, there is provided a computing device comprising a memory, a processor and computer instructions stored on the memory and executable on the processor, the processor implementing the steps of the topic correction method when executing the instructions.
According to a fourth aspect of embodiments of the present application, there is provided a computer readable storage medium storing computer instructions which, when executed by a processor, implement the steps of the topic modification method.
According to the title correction method, the picture to be identified is received, wherein the picture to be identified comprises the title to be corrected; performing target detection on the picture to be identified, determining a first detection area and a second detection area corresponding to the questions to be modified, and determining the positions of the questions and the answering positions; performing image description and identification on the first detection area to obtain image description information of the first detection area, performing text description and identification on the second detection area to obtain text information of the second detection area, and obtaining question information and answer information; and determining an correcting result of the questions to be corrected according to the image description information and the text information, comparing the image description information identifying the first detection area and the text information identifying the second detection area, and directly correcting the graphics questions without using a question bank, thereby omitting a complicated searching process for searching the pictures by using pictures, having high execution efficiency, less consumption of calculation resources and higher accuracy of correcting the final questions.
Drawings
FIG. 1 is a block diagram of a computing device provided by an embodiment of the present application;
FIG. 2 is a flow chart of a topic modification method provided in an embodiment of the present application;
fig. 3 is a block diagram of a YoloV3 network provided in an embodiment of the present application;
FIG. 4 is a flow chart of a method for topic modification according to another embodiment of the present application;
FIGS. 5a to 5g are schematic diagrams illustrating a method for topic modification according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a theme correction apparatus according to an embodiment of the present disclosure.
Detailed Description
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is, however, susceptible of embodiment in many other ways than those herein described and similar generalizations can be made by those skilled in the art without departing from the spirit of the application and the application is therefore not limited to the specific embodiments disclosed below.
The terminology used in one or more embodiments of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of one or more embodiments of the application. As used in this application in one or more embodiments and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present application refers to and encompasses any or all possible combinations of one or more of the associated listed items.
It should be understood that, although the terms first, second, etc. may be used in one or more embodiments of the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, a first may also be referred to as a second, and similarly, a second may also be referred to as a first, without departing from the scope of one or more embodiments of the present application. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.
First, terms related to one or more embodiments of the present invention will be explained.
Image description model (Image capture): is a very classical neural network model of two-dimensional information recognition of an image, the input of the image description is the image, the output is the description of the image, and the encoder and decoder are typically included in the image description model.
Encoder (Encoder): the image description model is used for encoding an input image to extract characteristic information, and a convolutional neural network is commonly used for characteristic extraction, so that the information of input sequence data can be compressed into a context vector with a fixed length, and a desired representation vector can better contain the whole input information.
Decoder (Decoder): the image description model is used for receiving characteristic information output by the encoder, and the decoder processes the characteristic information by using a cyclic neural network, a long-term and short-term memory network and the like to obtain a series of output sentences, and the encoder can initialize context vectors and then output the converted vectors.
Attention mechanism (Attention): the attention mechanism can be roughly understood in deep learning as to how much attention is paid to a certain vector, which may represent a certain local area in an image or a certain word in a sentence, and we use the attention vector to estimate the strength of the relationship between the part of interest and other elements, and take the values of the different parts and the result weighted by the attention vector as the approximate value of the target.
Embedding (Embedding): an object, such as a word, or a commodity, is represented by a low-dimensional vector, and the embedded vector has the property that the objects corresponding to the vectors with similar distances have similar meanings, and the embedded vector can encode the object by the low-dimensional vector and can keep the characteristics of the meanings, so that the embedded vector is very suitable for deep learning.
In the present application, a method and apparatus for topic modification, a computing device, and a computer-readable storage medium are provided, and are described in detail in the following embodiments.
FIG. 1 illustrates a block diagram of a computing device 100, according to an embodiment of the present application. The components of the computing device 100 include, but are not limited to, a memory 110 and a processor 120. Processor 120 is coupled to memory 110 via bus 130 and database 150 is used to store data.
Computing device 100 also includes access device 140, access device 140 enabling computing device 100 to communicate via one or more networks 160. Examples of such networks include the Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the internet. The access device 140 may include one or more of any type of network interface, wired or wireless (e.g., a Network Interface Card (NIC)), such as an IEEE802.11 Wireless Local Area Network (WLAN) wireless interface, a worldwide interoperability for microwave access (Wi-MAX) interface, an ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a bluetooth interface, a Near Field Communication (NFC) interface, and so forth.
In one embodiment of the present application, the above-described components of computing device 100, as well as other components not shown in FIG. 1, may also be connected to each other, such as by a bus. It should be understood that the block diagram of the computing device shown in FIG. 1 is for exemplary purposes only and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as desired.
Computing device 100 may be any type of stationary or mobile computing device including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), mobile phone (e.g., smart phone), wearable computing device (e.g., smart watch, smart glasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. Computing device 100 may also be a mobile or stationary server.
Wherein the processor 120 may perform the steps of the topic modification method shown in fig. 2. Fig. 2 shows a flowchart of a topic modification method according to an embodiment of the present application, including steps 202 to 208.
Step 202: and receiving a picture to be identified, wherein the picture to be identified comprises a theme to be modified.
The picture to be identified is a picture including the questions to be modified, which is sent by the user through the equipment terminal, such as a picture of an exercise book shot by the user through a mobile phone, a picture obtained by scanning the exercise book by the user through application software, and the like.
The picture to be identified comprises questions to be corrected, wherein the questions to be corrected are graphic questions, such as abacus questions, drawing recognition questions and the like, and a user needs to correct answers of the questions to be corrected to judge whether the answers are correct.
In a specific embodiment provided in the application, taking a picture to be identified as an exercise book photo shot by a user through a mobile phone as an example, the picture to be identified is a bead calculation question with a picture type to be modified, and the number is determined by observing the number of beads on the bead calculation.
In another specific embodiment provided by the application, taking a picture to be identified as a picture obtained by a user through scanning an exercise book by application software as an example, identifying picture questions with to-be-modified questions being graphic types in the picture to be identified, and determining that a plurality of sheep are in the picture by observing the picture.
Step 204: and carrying out target detection on the picture to be identified, and determining a first detection area and a second detection area corresponding to the topic to be modified.
The first detection area is an image area corresponding to the title to be modified, the first detection area is a title part in the title to be modified, and the first detection area is an area of the abacus chart by taking abacus as an example.
The second detection area is an answer area corresponding to the questions to be corrected, the second detection area is an answer part in the questions to be corrected, and the second detection area is an answer area of the bead calculation chart by taking the bead calculation questions as examples.
In practical application, target detection is required to be performed on the picture to be identified, and a first detection area and a second detection area corresponding to the to-be-corrected question are determined, namely, an image area and a question answering area corresponding to the to-be-corrected question are determined.
Optionally, performing target detection on the picture to be identified, and determining a first detection area and a second detection area corresponding to the topic to be modified, where the determining includes: inputting the picture to be identified into a target detection model to carry out target detection, and determining a first detection area corresponding to the topic to be modified; and inputting the picture to be identified into a text box detection model to carry out target detection, and determining a second detection area corresponding to the topic to be modified.
The target detection model is used for solving the target detection problem, namely, an image is given, targets in the image are found, the positions of the targets are found, the targets are classified, the target detection model is usually trained on a group of fixed training sets, the target detection model is required to determine the position information of the targets in the image and classify the targets, the image to be identified is identified through the target detection model, the first detection area can be accurately identified, the detection purpose is realized through a neural network model, the identification accuracy is effectively improved, and the image to be identified does not need to be searched from a question bank.
The target detection model may be a model of fast R-CNN, SSD or YoloV3, etc., and in the present application, the specific architecture of the target detection model is not limited.
For further explanation of the present application, taking the object detection model as the YoloV3 model as an example, the backbone network used by the YoloV3 model is a dark-53 network, and the front 52 layers in the network structure of the dark-53 network are shown in fig. 3 below, where fig. 3 shows the structure of the YoloV3 network, and DBL is a basic component of YoloV3, and is a convolution+bn+leak relu, for YoloV3, BN and leak relu are already separable parts from the convolution layer, which together form the minimum component.
N in resn represents a number, res1, res2, …, res8, etc., indicating how many res_units are contained in the res_block. Is a large component of YoloV3, yoloV3 began to borrow from the residual structure of ResNet, which can be used to make the network structure deeper.
Concat is tensor stitching, and the dark intermediate layer and the up-sampling of a later layer are stitched. The operation of stitching is not the same as the operation of the residual layer add, which expands the tensor dimension, whereas the direct addition of add only does not result in a change in tensor dimension.
As shown in fig. 3, yolv 3 outputs 3 feature images Y1, Y2, and Y3 with different scales, where the depths of Y1, Y2, and Y3 are 255, the side length rule is 13:26:52, 3 prediction frames are output in each feature image, and a prediction frame with the highest probability of having a target is found from the 9 prediction frames and is used as the first detection area corresponding to the to-be-corrected question.
Optionally, returning a question type disagreement prompt of the to-be-modified question under the condition that the first detection area corresponding to the to-be-modified question is not detected.
In practical application, if the first detection area corresponding to the to-be-modified question is not detected in the to-be-identified picture, the to-be-modified question of the graphic class is not included in the to-be-identified picture, and prompt information that the to-be-modified question is not in a question type is returned.
The text box detection model is used for detecting the text box in the picture to be identified, and as the target detection model, the text box detection model also outputs the position information of the text in the picture to be identified and classifies the text, such as handwriting text, printed digital text, printed text, and the like, and the text box detection model is preferably a combination of YoloV < 3+ > feature pyramids (Feature Pyramid Networks, FPN). The Feature Pyramid (FPN) solves the problem of multiple scales in target detection of pictures, namely, for the same picture to be identified, a large target can be detected, and a small target can be detected. The text box detection model identifies the text box in a neural network model mode, and is high in speed and accuracy.
In a specific embodiment provided by the application, a picture to be identified is input to a text box detection model to detect text boxes, and the text boxes can identify different text boxes in the picture to be identified, and the text boxes correspond to the same class, for example, the text box class corresponding to handwriting text is 101, the text box class corresponding to printing digital text is 102, the text box class corresponding to printing text is 103, and the like.
Step 206: and carrying out image description and identification on the first detection area to obtain image description information of the first detection area, and carrying out text description and identification on the second detection area to obtain text information of the second detection area.
After the first detection area and the second detection area are determined, image description and identification are needed to be carried out on the images in the first detection area to obtain image description information of the image area, and text identification is needed to be carried out on the answer area in the second detection area to obtain text information of the answer area.
Optionally, the first detection region includes at least one first detection sub-region; performing image description identification on the first detection area to obtain image description information of the first detection area, wherein the image description information comprises the following steps: and carrying out image description identification on the first detection areas through an image description model, and obtaining image description information corresponding to each first detection sub-area.
In practical application, an image problem has a plurality of small problems, such as a plurality of pictures for identification in an image problem, a plurality of pictures for identification in an abacus problem, and a plurality of first detection sub-areas in a first detection area, wherein each first detection sub-area is a small problem.
And carrying out image description identification on each first detection subarea through an image description model, and obtaining image description information corresponding to each first detection subarea.
An Image description (Image description) model is a sequence-to-sequence structure and comprises an encoder and a decoder, wherein the encoder and the decoder are used for receiving an Image, then, a text or sentence describing the Image is output through recognition analysis on the Image, in the application, the Image of each first detection subarea is input into the Image description model, and a corresponding digital result is output through recognition of the Image description model.
The encoder of the image description model uses a convolutional neural network (Resnet) with a residual structure, the input image is subjected to convolution operation, visual characteristic information of the image is extracted, an encoded vector is generated, the encoded vector is subjected to embedding processing, the original characteristic information is subjected to dimension reduction processing, the processed encoded vector is input to a decoder of the image description model for decoding, the decoder comprises a bidirectional long-short-term memory (LSTM) and an Attention mechanism (Attention), the encoded vector is subjected to decoding processing by the decoder, and description information corresponding to the image is output.
In a specific embodiment provided in the application, taking an image of a small problem in the bead calculation problem in the first detection subarea as an example, wherein there are 2 beads on hundred positions, 3 beads on ten positions, and 1 bead on one position, the image is input into an image description model for processing, and a corresponding digital result is obtained as 231.
In another specific embodiment provided by the application, taking an image with the first detection subarea as a small question of the recognition question as an example, 3 sheep are in the image, and the images are input into an image description model for processing, so that a corresponding digital result is 3.
Optionally, the second detection region comprises at least one second detection sub-region; performing text description recognition on the second detection area to obtain text information of the second detection area, wherein the text description recognition comprises the following steps: and carrying out image description recognition on the second detection areas through a text recognition model, and obtaining text information corresponding to each second detection sub-area.
The number of the second detection subareas in the second detection area is the same as that of the first detection subareas, and each second detection subarea corresponds to one first detection subarea, namely, each detection subarea has a corresponding answer result. And the second detection subareas are corresponding answer results, and corresponding text information is obtained by carrying out image description and identification on each second detection subarea.
The text recognition model is based on a Convolutional Neural Network (CNN) and a long-short-term memory network (LSTM), wherein a residual neural network structure is applied in the convolutional neural network and is used for extracting characteristic information of an image from the image of the second detection subarea, and the extracted characteristic information is input into the long-short-term memory network for classification and recognition, so that the text information in the second detection subarea is finally obtained.
In a specific embodiment provided in the application, taking the handwritten numeral "57" as an example, the content in the second detection sub-area is input into a text recognition model for processing, and corresponding text information is obtained as "57".
In another specific embodiment provided in the application, taking the case that the content in the second detection subarea is the handwritten capital number of fifty-seven as an example, the handwritten capital number of fifty-seventeen is input into a text recognition model for processing, and the corresponding text information is obtained as fifty-seven.
Step 208: and determining the correction result of the questions to be corrected according to the image description information and the text information.
Optionally, the first detection region includes at least one first detection sub-region, and the second detection region includes at least one second detection sub-region; determining a correction result of the to-be-corrected question according to the image description information and the text information, wherein the correction result comprises the following steps: and determining the correction result of the questions to be corrected according to the image description information corresponding to each first detection subarea and the text information corresponding to each second detection subarea.
The first detection region comprises at least one first detection sub-region, the second detection region comprises at least one second detection sub-region, and each first detection sub-region corresponds to one second detection sub-region.
In practical application, a correction result of the topic to be corrected is determined according to the image description information corresponding to each first detection subarea and the text information corresponding to each second detection subarea, and the correction result comprises steps S2082-S2084:
s2082, matching the corresponding second detection subareas for each first detection subarea.
Specifically, a first target point coordinate, a first length and a first width corresponding to each first detection sub-region are obtained, and a second target point coordinate, a second length and a second width corresponding to each second detection sub-region are obtained; determining first detection sub-region position information corresponding to each first detection sub-region according to first target point coordinates, first length and first width corresponding to each first detection sub-region, and determining second detection sub-region position information corresponding to each second detection sub-region according to second target point coordinates, second length and second width corresponding to each second detection sub-region; and determining a second detection sub-region corresponding to each first detection sub-region according to the position information of each first detection sub-region and the position information of each second detection sub-region.
The first target point coordinates may be coordinates of any one vertex of the first detection sub-region, such as an upper left corner coordinate or a lower right corner coordinate, the first length is a length of the first detection sub-region, the first width is a width of the first detection sub-region, and similarly, the second target point coordinates may be coordinates of any one vertex of the second detection sub-region, the second length is a length of the second detection sub-region, and the second width is a width of the second detection sub-region.
And determining the position information of the first detection sub-region according to the first target point, the first length and the first width, and determining the position information of the second detection sub-region according to the second target point, the second length and the second width.
And according to the position information of the first detection sub-region and the position information of the second detection sub-region, the second detection sub-region corresponding to the first detection sub-region in a matching way can be obtained.
In the embodiment provided in the application, explanation is given taking that 3 first detection subareas are in the first detection area and 3 second detection subareas are in the second detection area as an example.
Taking the top left corner vertex of each first detection sub-region as a first target point, wherein the top left corner vertex coordinates of the 3 first detection sub-regions are (5, 10), (30, 10) and (55,9), the first length is 10, the first width is 10, and the first detection sub-region position information of the three first detection sub-regions is A 1 (5,10,10,10)、A 2 (30,10,10,10)、A 3 (55,9,10,10)。
Taking each second testThe vertex of the upper left corner of the sub-region is the second target point, the coordinates of the upper left corners of the 3 second detection sub-regions are (8, 21), (32, 20), (53, 21), the second length is 8, and the second width is 5, the second detection sub-region position information of the three second detection sub-regions is B 1 (8,21,8,5)、B 2 (32,20,8,5)、B 3 (53,21,8,5)。
Wherein the first detection subarea position information A 1 And second detection sub-area position information B 1 The difference of the X axis of the coordinate points of (2) is 3, which is smaller than the preset threshold value, the difference of the Y axis is 11, which is also smaller than the preset threshold value, and the width (10) of the first detection area, the first detection sub-area A can be determined 1 And a second detection subarea B 1 Correspondingly, and so on, the first detection subarea A 2 And a second detection subarea B 2 Correspondingly, the first detection subarea A 3 And a second detection subarea B 3 Corresponding to each other.
The corresponding second detection subareas are matched for each first detection subarea, so that the subsequent comparison process is more targeted, and the comparison efficiency is higher.
S2084, comparing the image description information of each first detection sub-region with the text information of the second detection sub-region corresponding to the first detection sub-region, and determining the modification result corresponding to the title of each first detection sub-region.
After the first detection subarea and the corresponding second detection subarea are determined, comparing the image description information of the first detection subarea with the text information of the corresponding second detection subarea, if the image description information is consistent with the text information, the question answer in the first detection subarea is correct, and if the image description information is inconsistent with the text information, the question answer in the first detection subarea is wrong.
It should be noted that, when comparing the image description information and the text information, it is necessary to perform normalization processing such as bracket removal, unit removal, recognition result replacement, and the like on the image description information and the text information. The standardization processing effectively reduces interference information, unifies different image description information and text information into the same format, and is convenient for improving comparison efficiency in subsequent comparison.
In a specific embodiment provided in the application, the image description information is "057", the text information is "seventy-seven", the image description information is required to be simplified, the image description information is determined to be "57", the text information is replaced by the identification result, the text information is converted into "57", the image description information and the text information are compared, and the correction result of the problem is determined to be correct.
There are many forms of displaying the correction result, for example, the correction result is correctly indicated by "v", and the error is indicated by "x"; or the correction result is correctly represented by a green frame, the error is marked by a red frame, and in the application, the display form of the correction result is not limited, so that the correction result is in control of practical application.
According to the title correction method, the picture to be identified is received, wherein the picture to be identified comprises the title to be corrected; performing target detection on the picture to be identified, determining a first detection area and a second detection area corresponding to the questions to be modified, and determining the positions of the questions and the answering positions; performing image description and identification on the first detection area to obtain image description information of the first detection area, performing text description and identification on the second detection area to obtain text information of the second detection area, and obtaining question information and answer information; and determining an correcting result of the questions to be corrected according to the image description information and the text information, comparing the image description information identifying the first detection area and the text information identifying the second detection area, and directly correcting the graphics questions without using a question bank, thereby omitting a complicated searching process for searching the pictures by using pictures, having high execution efficiency, less consumption of calculation resources and higher accuracy of correcting the final questions.
Fig. 4 illustrates a method for question modification according to an embodiment of the present application, which is described by taking question modification of the abacus questions as an example, and includes steps 402 to 414.
Step 402: and receiving a picture to be identified, wherein the picture to be identified comprises a theme to be modified.
In the embodiment provided in the present application, referring to fig. 5a, fig. 5a is a picture to be identified provided in the embodiment of the present application, where the topic to be modified is a beading topic.
Step 404: and inputting the picture to be identified into a target detection model to carry out target detection, and determining at least one first detection subarea corresponding to the topic to be modified.
In the embodiment provided in the present application, as shown in fig. 5b, fig. 5b shows a schematic diagram of a first detection sub-area provided in the embodiment of the present application, and a picture to be identified is input to a target detection model to perform target detection, so as to determine 4 first detection sub-areas with types 301 corresponding to the topic to be identified.
Step 406: and carrying out image description identification on the first detection areas through an image description model, and obtaining image description information corresponding to each first detection sub-area.
In the embodiment provided in the present application, as shown in fig. 5c, fig. 5c shows image description information corresponding to the first detection sub-area provided in the embodiment of the present application. Inputting each first detection sub-region into an image description model for image description recognition, and recognizing that the image description information corresponding to each first detection sub-region is 0120, 01203, 05300 and 02050 respectively from left to right.
Step 408: and inputting the picture to be identified into a text box detection model to carry out target detection, and determining at least one second detection subarea corresponding to the topic to be modified.
In the embodiment provided in the present application, referring to fig. 5d, fig. 5d shows a schematic diagram of a second detection sub-area provided in the embodiment of the present application, a picture to be identified is input to a text box detection model to perform target detection, and 4 second detection sub-areas with types 102 corresponding to the topic to be identified are determined.
Step 410: and carrying out image description recognition on the second detection areas through a text recognition model, and obtaining text information corresponding to each second detection sub-area.
In the embodiment provided in the present application, referring to fig. 5e, fig. 5e shows text information corresponding to the second detection sub-area provided in the embodiment of the present application. And inputting each second detection sub-region into the text recognition model to perform image description recognition, and acquiring 1220, 1203, 5300 and 2050 text information corresponding to each second detection sub-region from left to right.
It should be noted that the execution sequence between the steps 404 to 406 and the steps 408 to 410 is not necessarily sequential, and may be executed simultaneously.
Step 412: and matching the corresponding second detection subarea for each first detection subarea.
In the embodiment provided in the present application, referring to fig. 5f, fig. 5f shows a schematic diagram of matching a first detection sub-area with a second detection sub-area provided in the embodiment of the present application, as shown in fig. 5f, a first detection sub-area 0 corresponds to a second detection sub-area 0, a first detection sub-area 1 corresponds to a second detection sub-area 1, a first detection sub-area 2 corresponds to a second detection sub-area 2, and a first detection sub-area 3 corresponds to a second detection sub-area 3.
Step 414: comparing the image description information of each first detection sub-region with the text information of a second detection sub-region corresponding to the first detection sub-region, and determining a modification result corresponding to the title of each first detection sub-region.
In the embodiment provided by the application, referring to fig. 5g, fig. 5g shows a schematic diagram of comparison between the image description information of the first detection sub-area and the text information of the second detection sub-area provided by the embodiment of the application, as shown in fig. 5g, the image description information of the first detection sub-area 0 is 0120, the text information of the second detection sub-area 0 is 1220, after comparison, 1220=0120 is determined, and then, the correction result corresponding to the title in the first detection sub-area 0 is determined to be correct, and by analogy, the correction result corresponding to the title in the first detection sub-area 1 is correct, the correction result corresponding to the title in the first detection sub-area 2 is correct, and the correction result corresponding to the title in the first detection sub-area 3 is correct.
According to the title correction method, the picture to be identified is received, wherein the picture to be identified comprises the title to be corrected; performing target detection on the picture to be identified, determining a first detection area and a second detection area corresponding to the questions to be modified, and determining the positions of the questions and the answering positions; performing image description and identification on the first detection area to obtain image description information of the first detection area, performing text description and identification on the second detection area to obtain text information of the second detection area, and obtaining question information and answer information; and determining an correcting result of the questions to be corrected according to the image description information and the text information, comparing the image description information identifying the first detection area and the text information identifying the second detection area, and directly correcting the graphics questions without using a question bank, thereby omitting a complicated searching process for searching the pictures by using pictures, having high execution efficiency, less consumption of calculation resources and higher accuracy of correcting the final questions.
Corresponding to the above method embodiment, the present application further provides an embodiment of the topic modification apparatus, and fig. 6 shows a schematic structural diagram of the topic modification apparatus according to one embodiment of the present application. As shown in fig. 6, the apparatus includes:
A receiving module 602 configured to receive a picture to be identified, wherein the picture to be identified includes a subject to be modified;
the detection module 604 is configured to perform target detection on the picture to be identified, and determine a first detection area and a second detection area corresponding to the topic to be modified;
the identifying module 606 is configured to perform image description identification on the first detection area to obtain image description information of the first detection area, and perform text description identification on the second detection area to obtain text information of the second detection area;
a determining module 608 is configured to determine a modification result of the to-be-modified question according to the image description information and the text information.
Optionally, the detecting module 604 is further configured to input the picture to be identified into a target detection model for target detection, and determine a first detection area corresponding to the topic to be modified; and inputting the picture to be identified into a text box detection model to carry out target detection, and determining a second detection area corresponding to the topic to be modified.
Optionally, the first detection region includes at least one first detection sub-region;
the identifying module 606 is further configured to identify the image description of the first detection area through an image description model, so as to obtain image description information corresponding to each first detection sub-area.
Optionally, the second detection region comprises at least one second detection sub-region;
the recognition module 606 is further configured to perform image description recognition on the second detection areas through a text recognition model, so as to obtain text information corresponding to each second detection sub-area.
Optionally, the first detection region includes at least one first detection sub-region, and the second detection region includes at least one second detection sub-region;
the determining module 608 is further configured to determine a correction result of the to-be-corrected question according to the image description information corresponding to each of the first detection sub-areas and the text information corresponding to each of the second detection sub-areas.
Optionally, the determining module 608 is further configured to match, for each of the first detection sub-regions, a corresponding second detection sub-region; comparing the image description information of each first detection sub-region with the text information of a second detection sub-region corresponding to the first detection sub-region, and determining a modification result corresponding to the title of each first detection sub-region.
Optionally, the determining module 608 is further configured to obtain a first target point coordinate, a first length and a first width corresponding to each of the first detection sub-areas, and obtain a second target point coordinate, a second length and a second width corresponding to each of the second detection sub-areas; determining first detection sub-region position information corresponding to each first detection sub-region according to first target point coordinates, first length and first width corresponding to each first detection sub-region, and determining second detection sub-region position information corresponding to each second detection sub-region according to second target point coordinates, second length and second width corresponding to each second detection sub-region; and determining a second detection sub-region corresponding to each first detection sub-region according to the position information of each first detection sub-region and the position information of each second detection sub-region.
Optionally, the apparatus further includes:
and the return module is configured to return the question type disagreement prompt of the questions to be modified under the condition that the first detection area corresponding to the questions to be modified is not detected.
The title correcting device provided by the embodiment of the application receives the picture to be identified, wherein the picture to be identified comprises the title to be corrected; performing target detection on the picture to be identified, determining a first detection area and a second detection area corresponding to the questions to be modified, and determining the positions of the questions and the answering positions; performing image description and identification on the first detection area to obtain image description information of the first detection area, performing text description and identification on the second detection area to obtain text information of the second detection area, and obtaining question information and answer information; and determining an correcting result of the questions to be corrected according to the image description information and the text information, comparing the image description information identifying the first detection area and the text information identifying the second detection area, and directly correcting the graphics questions without using a question bank, thereby omitting a complicated searching process for searching the pictures by using pictures, having high execution efficiency, less consumption of calculation resources and higher accuracy of correcting the final questions.
An embodiment of the present application further provides a computing device including a memory, a processor, and computer instructions stored on the memory and executable on the processor, where the processor implements the steps of the topic correction method when executing the instructions.
An embodiment of the present application also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the topic modification method as described above.
The above is an exemplary version of a computer-readable storage medium of the present embodiment. It should be noted that, the technical solution of the storage medium and the technical solution of the above-mentioned subject matter correcting method belong to the same conception, and the details of the technical solution of the storage medium which are not described in detail can be referred to the description of the technical solution of the above-mentioned subject matter correcting method.
The foregoing describes specific embodiments of the present application. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
The computer instructions include computer program code that may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth. It should be noted that the computer readable medium contains content that can be appropriately scaled according to the requirements of jurisdictions in which such content is subject to legislation and patent practice, such as in certain jurisdictions in which such content is subject to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.
It should be noted that, for the sake of simplicity of description, the foregoing method embodiments are all expressed as a series of combinations of actions, but it should be understood by those skilled in the art that the present application is not limited by the order of actions described, as some steps may be performed in other order or simultaneously in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily all necessary for the present application.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments.
The above-disclosed preferred embodiments of the present application are provided only as an aid to the elucidation of the present application. Alternative embodiments are not intended to be exhaustive or to limit the invention to the precise form disclosed. Obviously, many modifications and variations are possible in light of the teaching of this application. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best understand and utilize the invention. This application is to be limited only by the claims and the full scope and equivalents thereof.

Claims (6)

1. A method for topic modification comprising:
receiving a picture to be identified, wherein the picture to be identified comprises a topic to be modified;
inputting the picture to be identified into a target detection model to carry out target detection, and determining a first detection area corresponding to the topic to be modified; inputting the picture to be identified into a text box detection model for target detection, and determining a second detection area corresponding to the topic to be modified, wherein the first detection area comprises at least one first detection subarea, and the second detection area comprises at least one second detection subarea;
Performing image description recognition on the first detection region through an image description model to obtain image description information corresponding to each first detection sub-region, and performing image description recognition on the second detection region through a text recognition model to obtain text information corresponding to each second detection sub-region;
matching a corresponding second detection subarea for each first detection subarea; comparing the image description information of each first detection sub-region with the text information of a second detection sub-region corresponding to the first detection sub-region, and determining a modification result corresponding to the title of each first detection sub-region.
2. The method of topic modification of claim 1, wherein matching each of the first detection subregions with a corresponding second detection subregion comprises:
acquiring a first target point coordinate, a first length and a first width corresponding to each first detection sub-region, and acquiring a second target point coordinate, a second length and a second width corresponding to each second detection sub-region;
determining first detection sub-region position information corresponding to each first detection sub-region according to first target point coordinates, first length and first width corresponding to each first detection sub-region, and determining second detection sub-region position information corresponding to each second detection sub-region according to second target point coordinates, second length and second width corresponding to each second detection sub-region;
And determining a second detection sub-region corresponding to each first detection sub-region according to the position information of each first detection sub-region and the position information of each second detection sub-region.
3. The method for topic modification of claim 1, further comprising:
and returning a question type disagreement prompt of the questions to be modified under the condition that the first detection area corresponding to the questions to be modified is not detected.
4. A device for modifying a subject, comprising:
the receiving module is configured to receive a picture to be identified, wherein the picture to be identified comprises a subject to be modified;
the detection module is configured to input the picture to be identified into a target detection model for target detection, and determine a first detection area corresponding to the topic to be modified; inputting the picture to be identified into a text box detection model for target detection, and determining a second detection area corresponding to the topic to be modified, wherein the first detection area comprises at least one first detection subarea, and the second detection area comprises at least one second detection subarea;
the identification module is configured to carry out image description identification on the first detection areas through an image description model, acquire image description information corresponding to each first detection sub-area, carry out image description identification on the second detection areas through a text identification model, and acquire text information corresponding to each second detection sub-area;
A determining module configured to match, for each of the first detection sub-regions, a corresponding second detection sub-region; comparing the image description information of each first detection sub-region with the text information of a second detection sub-region corresponding to the first detection sub-region, and determining a modification result corresponding to the title of each first detection sub-region.
5. A computing device comprising a memory, a processor, and computer instructions stored on the memory and executable on the processor, wherein the processor, when executing the instructions, implements the steps of the method of any of claims 1-3.
6. A computer readable storage medium storing computer instructions which, when executed by a processor, implement the steps of the method of any one of claims 1-3.
CN202010645304.9A 2020-07-07 2020-07-07 Question correction method and device Active CN111767883B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010645304.9A CN111767883B (en) 2020-07-07 2020-07-07 Question correction method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010645304.9A CN111767883B (en) 2020-07-07 2020-07-07 Question correction method and device

Publications (2)

Publication Number Publication Date
CN111767883A CN111767883A (en) 2020-10-13
CN111767883B true CN111767883B (en) 2024-04-12

Family

ID=72723974

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010645304.9A Active CN111767883B (en) 2020-07-07 2020-07-07 Question correction method and device

Country Status (1)

Country Link
CN (1) CN111767883B (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112507879B (en) * 2020-12-08 2024-08-23 科大讯飞股份有限公司 Review method and device, electronic equipment and storage medium
CN112686263B (en) * 2020-12-29 2024-04-16 科大讯飞股份有限公司 Character recognition method, character recognition device, electronic equipment and storage medium
CN112818975B (en) * 2021-01-27 2024-09-24 北京金山数字娱乐科技有限公司 Text detection model training method and device, text detection method and device
CN112712070A (en) * 2021-03-26 2021-04-27 北京世纪好未来教育科技有限公司 Question judging method and device for bead calculation questions, electronic equipment and storage medium
CN113095315A (en) * 2021-04-13 2021-07-09 第六镜科技(成都)有限公司 Optical character recognition method and device based on deep neural network
CN113392756A (en) * 2021-06-11 2021-09-14 北京猿力未来科技有限公司 Method and device for identifying picture book
CN114895829B (en) * 2022-07-15 2022-09-27 广东信聚丰科技股份有限公司 Display state optimization method and system based on artificial intelligence

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20150027634A (en) * 2013-09-04 2015-03-12 삼성전자주식회사 Method and Apparatus for scoring test
CN106775241A (en) * 2016-12-15 2017-05-31 网易(杭州)网络有限公司 Information demonstrating method, equipment and computer-readable recording medium
CN107798321A (en) * 2017-12-04 2018-03-13 海南云江科技有限公司 A kind of examination paper analysis method and computing device
KR101956714B1 (en) * 2018-06-12 2019-03-11 (주)아나키 Method and system for checking answers
CN109670504A (en) * 2018-12-28 2019-04-23 杭州大拿科技股份有限公司 Method and device is corrected in a kind of hand-written answer identification
CN110414563A (en) * 2019-06-27 2019-11-05 深圳中兴网信科技有限公司 Total marks of the examination statistical method, system and computer readable storage medium
CN110490180A (en) * 2019-07-05 2019-11-22 平安国际智慧城市科技股份有限公司 Work correction method, apparatus, storage medium and server based on image recognition

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108932508B (en) * 2018-08-13 2022-03-18 杭州大拿科技股份有限公司 Method and system for intelligently identifying and correcting subjects

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20150027634A (en) * 2013-09-04 2015-03-12 삼성전자주식회사 Method and Apparatus for scoring test
CN106775241A (en) * 2016-12-15 2017-05-31 网易(杭州)网络有限公司 Information demonstrating method, equipment and computer-readable recording medium
CN107798321A (en) * 2017-12-04 2018-03-13 海南云江科技有限公司 A kind of examination paper analysis method and computing device
KR101956714B1 (en) * 2018-06-12 2019-03-11 (주)아나키 Method and system for checking answers
CN109670504A (en) * 2018-12-28 2019-04-23 杭州大拿科技股份有限公司 Method and device is corrected in a kind of hand-written answer identification
CN110414563A (en) * 2019-06-27 2019-11-05 深圳中兴网信科技有限公司 Total marks of the examination statistical method, system and computer readable storage medium
CN110490180A (en) * 2019-07-05 2019-11-22 平安国际智慧城市科技股份有限公司 Work correction method, apparatus, storage medium and server based on image recognition

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
基于深度学习的数学书面作业智能批阅研究;黄松豪;徐建华;杜佳玲;;电脑知识与技术(第12期);全文 *
题库评分图像识别系统的研究;杨小艳;丁艳秋;;电子元器件与信息技术(第11期);全文 *

Also Published As

Publication number Publication date
CN111767883A (en) 2020-10-13

Similar Documents

Publication Publication Date Title
CN111767883B (en) Question correction method and device
US20210020059A1 (en) Answer evaluation method, answer evaluation system, electronic device, and medium
CN111950528B (en) Graph recognition model training method and device
CN114495129B (en) Character detection model pre-training method and device
CN110363084A (en) A kind of class state detection method, device, storage medium and electronics
CN109299663A (en) Hand-written script recognition methods, system and terminal device
Imran et al. Dataset of Pakistan sign language and automatic recognition of hand configuration of urdu alphabet through machine learning
CN110750624A (en) Information output method and device
CN113221718B (en) Formula identification method, device, storage medium and electronic equipment
CN112347997A (en) Test question detection and identification method and device, electronic equipment and medium
CN113989577B (en) Image classification method and device
CN113033721B (en) Title correction method and computer storage medium
CN118445578B (en) Multi-mode online evaluation data processing method and system
Chillaron et al. Face detection and recognition application for Android
CN110659562A (en) Deep learning (DNN) classroom learning behavior analysis method and device
CN113822302B (en) Training method and device for target detection model
CN116311322A (en) Document layout element detection method, device, storage medium and equipment
CN115937660A (en) Verification code identification method and device
Chandrasekaran American sign language recognition and translation using deep learning and computer vision
CN114973362A (en) Dynamic extension coding micro-expression recognition method applied to social robot
CN113435441A (en) Bi-LSTM mechanism-based four-fundamental operation formula image intelligent batch modification method
CN113947801A (en) Face recognition method and device and electronic equipment
Moon et al. An improved custom convolutional neural network based hand sign recognition using machine learning algorithm
CN110543238A (en) Desktop interaction method based on artificial intelligence
Shaikh et al. Identification and Recognition of Text and Face Using Image Processing for Visually Impaired

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant