Nothing Special   »   [go: up one dir, main page]

WO2023272608A1 - 一种绘本辅助阅读方法和电子设备 - Google Patents

一种绘本辅助阅读方法和电子设备 Download PDF

Info

Publication number
WO2023272608A1
WO2023272608A1 PCT/CN2021/103689 CN2021103689W WO2023272608A1 WO 2023272608 A1 WO2023272608 A1 WO 2023272608A1 CN 2021103689 W CN2021103689 W CN 2021103689W WO 2023272608 A1 WO2023272608 A1 WO 2023272608A1
Authority
WO
WIPO (PCT)
Prior art keywords
picture book
electronic device
user
reading
picture
Prior art date
Application number
PCT/CN2021/103689
Other languages
English (en)
French (fr)
Inventor
张明云
Original Assignee
东莞市小精灵教育软件有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 东莞市小精灵教育软件有限公司 filed Critical 东莞市小精灵教育软件有限公司
Priority to PCT/CN2021/103689 priority Critical patent/WO2023272608A1/zh
Publication of WO2023272608A1 publication Critical patent/WO2023272608A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B5/00—Electrically-operated educational appliances
    • G09B5/06—Electrically-operated educational appliances with both visual and audible presentation of the material to be studied

Definitions

  • the invention relates to the field of education technology, in particular to a picture book aided reading method and electronic equipment.
  • Reading paper picture books requires parents to explain, and there are cases where the explanations are unprofessional. In most cases, infant users or school-age children need to read the same picture book, which consumes a lot of energy for parents.
  • the scheme of reading picture books through the mobile phone/tablet camera can only broadcast the audio of each page of the picture book when the page is turned, and it is difficult for infant users or school-age children to understand the storyline and content in the picture book.
  • the purpose of the present invention is to provide a picture book reading aid method and electronic equipment to help users understand and read picture books, reduce the energy-consuming situation of parents assisting reading, and avoid vision loss caused by electronic reading, greatly improving the picture book reading experience.
  • the invention provides a picture book auxiliary reading method, comprising the steps of:
  • the picture book is placed flatly under the reflector, and the reflector is installed above the camera.
  • the steps include:
  • the identifying the position coordinates of the reading area indicated by the end of the pointer from the picture book image includes the steps of:
  • the pixel coordinates remain unchanged for a preset duration, it is determined that the user has an intention to learn point reading, and the position coordinates of the point reading area are calculated according to the pixel coordinates.
  • said acquiring and broadcasting corresponding explanation data according to the position coordinates of the point-reading area and said picture book image comprises steps:
  • Broadcasting is performed according to the explanation data, so as to explain the reading area indicated by the user.
  • the broadcasting according to the explanation data to explain the reading area indicated by the user includes steps before:
  • the broadcasting according to the explanation data to explain the reading area indicated by the user includes the steps of:
  • the age group belongs to the school age group, and the user's line of sight is not on the screen of the electronic device, control the speaker voice to play the explanation text content, and control the display screen to play the explanation picture content;
  • the age group belongs to the school age group, and the user's line of sight is on the screen of the electronic device, control the speaker to play the explanatory text content, and control the display screen to play the explanatory picture content;
  • the explanation data includes explanation text content and explanation picture content.
  • the present invention also provides an electronic device, comprising:
  • the obtaining module is used to control the camera to capture the picture book page currently read by the user in the projection area of the reflector, so as to obtain the picture book image taken on the picture book page;
  • a recognition module configured to recognize the position coordinates of the reading area indicated by the end of the pointer from the picture book image
  • a control module used to acquire and broadcast corresponding explanation data according to the position coordinates of the reading area and the picture book image
  • the picture book is placed flatly under the reflector, and the reflector is installed above the camera.
  • the starting module is used to obtain the starting instruction for starting the point reading function, and start the camera according to the starting instruction.
  • the identification module includes:
  • An extraction unit configured to extract target contour features matching preset indicator features from the picture book image
  • a calculation unit configured to calculate and obtain the pixel coordinates of the end of the indicator according to the contour features of the target
  • the processing unit is configured to determine that the user has an intention to learn point reading if the pixel coordinates remain unchanged for a preset period of time, and calculate the position coordinates of the point reading area according to the pixel coordinates.
  • control module includes:
  • a comparison unit configured to compare the picture book image with a preset sample image library, and query the page number information corresponding to the picture book image;
  • a query unit configured to query and obtain explanation data corresponding to the target of the point reading area according to the position coordinates and page number information of the point reading area;
  • the execution unit is configured to broadcast according to the explanation data, so as to explain the reading area indicated by the user.
  • control module also includes:
  • An identification unit configured to identify the age group of the user who reads the picture book page
  • a detection unit configured to detect whether the user's line of sight is focused on the screen of the electronic device
  • the execution unit is also used to control the speaker to play the explanatory text content, and control the display screen to play the explanatory picture content if the age group belongs to early childhood;
  • the execution unit is further configured to control the speaker to play the explanatory text content by voice, and control the display screen to play the explanatory picture content if the age group belongs to the school age group and the user's sight line is not on the screen of the electronic device;
  • the executing unit is further configured to control the speaker to play the explanatory text content by voice, and control the display screen to play the explanatory picture content if the age group belongs to the school age group, and the user's line of sight is on the screen of the electronic device;
  • the explanation data includes explanation text content and explanation picture content.
  • the picture book aided reading method and electronic equipment provided by the present invention can assist users to understand and read picture books, reduce the energy consumption of parents' aided reading, avoid vision loss caused by electronic reading, and greatly improve the picture book reading experience. Moreover, it can also help young users or school-age children immerse themselves in picture book reading, avoid distraction from playing other apps in electronic devices, and cultivate children's excellent reading habits.
  • Fig. 1 is the flowchart of an embodiment of a kind of picture book aided reading method of the present invention
  • Fig. 2 is a schematic diagram of a scene of a picture book assisted reading method of the present invention
  • Fig. 3 is another schematic diagram of a picture book assisted reading method of the present invention.
  • Fig. 4 is another schematic diagram of a picture book aided reading method of the present invention.
  • FIG. 1 One embodiment of the present invention, as shown in Figure 1, a picture book aided reading method, comprising:
  • S100 controls the camera 21 to capture the 3 pages of the picture book currently read by the user in the projected area of the reflector, so as to obtain the picture book image taken on the 3 pages of the picture book;
  • picture book 3 (Picture Book) was called a picture book in the early days.
  • the popular term for picture book 3 is a story book with pictures, simple theme, and short plot content.
  • the publications are mainly aimed at young children and children, and most of them are oriented towards reading content suitable for young children and school-age children.
  • Most of the picture book 3 books on the market today are mainly read by young children and school-age children, and the content involves literature, education, popular science, etc.
  • the electronic device 2 can be, but not limited to, a smart tablet such as a tablet, a mobile phone, a learning machine, a phone watch, or the like.
  • the electronic device 2 includes a device body and a camera 21 disposed on the device body.
  • the reflector structure 1 may be a magnetically attracted reflector structure 11 or a clip-on reflector structure 12 .
  • the reflective mirror structure 1 is positioned above the camera 21 of the electronic device 2, and the reflective surface of the reflective mirror of the reflective mirror structure 1 is set at a certain inclination angle with the camera 21, so that the image information on the reflective surface of the reflective mirror can be completely reflected to the Camera 21 on.
  • the magnetic reflector structure 11 includes a casing and a reflector, and the casing has a built-in first magnetic element.
  • the top of the device body of the electronic device 2 is provided with a second magnetic element 2222 adapted to the first magnetic element.
  • the first magnetic element is used to attach the shell to the second magnetic element 2222 of the device body
  • the reflector is arranged on the shell, and the reflector is set toward the camera 21 of the device body, and is used to reflect image information to the camera of the device body twenty one.
  • the clip-on reflector structure 12 includes a housing, and a mounting groove for embedding the housing on the electronic device 2 is provided in the middle of the housing, and a fixing bracket and a reflector for strengthening and fixing the housing are provided in the mounting groove.
  • the reflection principle of the mirror can be used to expand the shooting range of the camera 21, so that the user does not need to move the picture book 3 back and forth frequently when using it, and only needs to spread the picture book 3 under the reflector, and the content on the picture book 3 can be reflected in the mirror. Because the reflective mirror is facing the camera 21, the camera 21 can obtain the image information in the projected area of the reflective mirror, that is, the camera 21 can successfully obtain the image information on the picture book 3 to shoot and obtain the picture book image.
  • the electronic device 2 includes a front and a back opposite to the front, the front is a side with a display screen, the camera 21 can be arranged on the top of the electronic device 2, and the camera 21 can be The front camera 21 or the rear camera 21.
  • the tutoring machine is an Android tablet that provides high-quality educational resources.
  • a magnetic suction mirror structure 11 is installed on the front camera 21 of the tutoring machine to form a flat base (not shown in the figure) + flat panel + magnetic attraction
  • the photo search solution of the reflective mirror structure 11 + paper picture book 3 specifically, the tablet is placed vertically on the inclined tablet base, the magnetic suction mirror structure 11 is installed above the front camera 21 of the tablet, and the magnetic suction installation On the top of the Android tablet, the picture book 3 is placed in front of the tablet, and the top of the picture book 3 is aligned with the base of the tablet.
  • the picture book 3 placed in front of the tablet can be photographed to obtain a picture book image.
  • the magnetic suction mirror structure 11 or snap-in mirror structure 12 supports the picture book 3 placed on the front of the electronic device 2 to reflect and collect light and send it to the camera 21.
  • the picture book 3 in front of the device 2 shoots, collects a plurality of picture book images and takes pictures, then, the electronic device 2 performs recognition and analysis according to the picture book images to obtain the question area that the user intends to ask, that is, the point reading area of the present invention, and then the electronic device 2 reads according to the point reading area.
  • the location coordinates of the area are queried for corresponding explanation data, and the explanation data is broadcast through the display screen or speaker of the electronic device 2 .
  • the present invention does not require manual adjustment by the user, and only sets the reflective surface of the reflective mirror toward the camera 21. In this way, the image information in the projected area of the reflective mirror can be captured by the camera 21, which is conducive to improving the efficiency of the camera 21 in recognizing images and texts, and giving users more Good product experience.
  • S200 recognizes the position coordinates of the reading area indicated by the end of the indicator from the picture book image
  • the indicator includes, but is not limited to, a finger, a point reading pen, a writing pen (pen or pencil), and may even be a bar-shaped item (such as an arithmetic stick).
  • the end of the human finger refers to the fingertip, and the end of the reading pen is the nib.
  • the pointer is generally the user's finger. Because users who use their fingers do not need to hold peripheral devices such as point reading pens or writing pens, users can directly point and read on the picture book 3 with their fingers without using peripheral devices to operate, which provides great convenience for users. The convenience greatly improves the convenience and experience of use.
  • the electronic device 2 After the electronic device 2 acquires the picture book image through the above method, it can use the image recognition algorithm to identify the end of the pointer from the picture book image, and then calculate the end of the pointer indicated by the end of the pointer according to the recognized end of the pointer through the calculation principle of coordinate transformation. The location coordinates of the reading area.
  • S300 acquires and broadcasts corresponding explanation data according to the position coordinates of the reading area and the picture book image;
  • the picture book 3 is flatly placed under the reflector, and the reflector is installed above the camera 21 .
  • the electronic device 2 can perform a local query to obtain explanation data.
  • the electronic device 2 can also perform a network-wide query to obtain the explanation data, and the server returns the explanation data to the electronic device 2 .
  • the electronic device 2 controls the playback device installed and set by itself to broadcast the explaining data.
  • the electronic device 2 can also be connected to an external device through a data interface, and the electronic device 2 controls the external device to broadcast and explain data.
  • the explanatory data refers to the data that explains the content corresponding to the reading area, and the format of the explanatory data includes but is not limited to audio, video, picture and other formats.
  • the explanation data may be a recording or video pre-recorded by parents and teachers explaining the reading area.
  • the present invention uses the camera 21 and the reflector to take picture book images, inquires about the explanation data corresponding to the reading area, and plays according to the explanation data.
  • the present invention realizes the point reading learning function (including explaining the content of the indicated point reading area, or explaining the content of the indicated point reading area after reading aloud) ), which can effectively assist the user to understand the picture book 3, reduce the energy consumption of parents assisting in reading the picture book 3, and avoid the problem that the user uses the electronic device 2 to read the picture book 3 and damages vision health.
  • the most important thing is to avoid the situation of realizing a single point reading function (that is, reading aloud the content of the point reading area indicated), that is, when the user uses the electronic device 2 to read the picture book 3, the audio of the pointed position is played only (that is, only the pointed position is played).
  • the explanation data corresponding to the reading area can be queried and broadcast, which can help users to effectively understand the picture book 3, avoid the situation of cramming characters into memory, and further greatly improve the user's reading experience and learning interest in the paper picture book 3.
  • An embodiment of the present invention a picture book aided reading method, comprising:
  • S010 obtains the starting instruction for starting the point reading function, and starts the camera 21 according to the starting instruction
  • this embodiment is an optimized embodiment of the above-mentioned embodiment, and the parts in this embodiment that are the same as those in the above-mentioned embodiment refer to the above-mentioned embodiment, and details are not repeated here.
  • the electronic device 2 is installed with a little reading learning APP, and the operation interface of the little reading learning APP is provided with a control button of "point reading". If the user has a demand for point reading, then the control button of "point reading" can be clicked to generate a start instruction In order to start the camera 21 to shoot.
  • the electronic device 2 can also say to the electronic device 2 a sentence that meets the preset trigger keywords, such as "I want to read picture book 3", so that the electronic device 2 will generate a start command to start the camera 21 for shooting .
  • S100 controls the camera 21 to capture the picture book page 3 currently read by the user in the projection area of the reflector, so as to obtain the picture book image taken on the picture book page 3;
  • S210 extracts from the picture book image the target contour features matching the preset indicator features
  • the user can use the pointer to point to the current page of the picture book 3, so that the picture book image captured by the camera 21 of the electronic device 2 includes the end of the pointer and the current page of the picture book 3 indicated by the end of the pointer.
  • the preset indicator features are extracted from the training based on the sample images of the indicators.
  • the preset indicator features of indicators such as fingers and pointing pens are bar outline features.
  • the electronic device 2 After the electronic device 2 obtains the picture book image through the above method, it uses an image recognition algorithm to detect and extract all the image features from the picture book image, and inputs the image features into the pre-trained indicator recognition model, and screens out the picture book image that matches The target profile feature for the pointer profile.
  • the machine learning model includes but is not limited to a fully connected neural network model, a convolutional neural network model, a recurrent neural network model, and the like.
  • the reading area indicated by the end of the pointer does not require the user to touch the end of the pointer with the current page of the picture book 3 placed under the mirror structure 1 .
  • the user can choose to touch the end of the pointer to the intended query position on the current page of Picture Book 3.
  • you can also choose to hang the end of the indicator to point to the intended query position on the current page of Picture Book 3.
  • the electronic device 2 can count the length of time that the pixel coordinates at the end of the pointer remain unchanged (hereinafter referred to as the indication time for ease of description) .
  • the electronic device 2 can calculate the pixel coordinates of the target outline feature, and then convert and calculate the position of the end of the pointer on the world coordinate system according to the pixel coordinates coordinate. If the indicated duration does not reach the preset duration, it means that the user only has a single reading intention. If the indicated duration reaches the preset duration, it means that the user has an intention to learn by point-and-click. At this time, since the end of the pointer is in contact with the paper of the picture book 3, the gap between the paper of the picture book 3 and the end of the pointer is almost negligible.
  • the electronic device 2 can use the position coordinates of the end of the pointer on the world coordinate system as a point The location coordinates of the read area. Calculate the pixel coordinates of the target outline features, and then perform conversion calculations based on the pixel coordinates to obtain the position coordinates of the end of the indicator on the world coordinate system. Specifically: Obtain the pixel coordinates of the end of the indicator from the picture book image, and use it as a reference The pixel coordinates of the picture book 3 page border.
  • the pixel coordinates can be obtained from the picture book image, the relative position of the finger in the picture book image relative to the picture book 3 page frame can be obtained, combined with the actual position of the picture book 3 page frame, the end of the pointer can be calculated in the world coordinate system position coordinates on .
  • the electronic device 2 can calculate the corresponding position of the pointer end.
  • One pixel coordinate, and the second pixel coordinate corresponding to the head end of the indicator and calculate the plane equation of the plane where the current page of the picture book 3 is located, and then, the electronic device 2 respectively performs conversion according to the first pixel coordinate and the second pixel coordinate.
  • the first position coordinates of the end of the pointer on the world coordinate system and the second position coordinates of the head end of the pointer on the world coordinate system are obtained through calculation.
  • the electronic device 2 obtains the straight line equation corresponding to the indicator according to the first position coordinate and the second position coordinate, and can solve the calculation according to the straight line equation corresponding to the indicator and the plane equation of the current page of the picture book 3 to obtain the position at the end of the indicator.
  • the location coordinates of the indicated reading area are not limited to the preset duration.
  • S300 acquires and broadcasts corresponding explanation data according to the position coordinates of the reading area and the picture book image;
  • the picture book 3 is flatly placed under the reflector, and the reflector is installed above the camera 21 .
  • the present invention uses the camera 21 and the reflector to take picture book images, inquires about the explanation data corresponding to the reading area, and plays according to the explanation data.
  • the present invention realizes the point reading learning function (including explaining the content of the indicated point reading area, or explaining the content of the indicated point reading area after reading aloud) ), which can effectively assist the user to understand the picture book 3, reduce the energy consumption of parents assisting in reading the picture book 3, and avoid the problem that the user uses the electronic device 2 to read the picture book 3 and damages vision health.
  • the most important thing is to avoid the situation of realizing a single point reading function (that is, reading aloud the content of the point reading area indicated), that is, when the user uses the electronic device 2 to read the picture book 3, the audio of the pointed position is played only (that is, only the pointed position is played).
  • the explanation data corresponding to the reading area can be queried and broadcast, which can help users to effectively understand the picture book 3, avoid the situation of cramming characters into memory, and further greatly improve the user's reading experience and learning interest in the paper picture book 3.
  • the present invention keeps the pixel coordinates at the end of the indicator in a constant state, performs timing to obtain the indicated duration, and compares the indicated duration with the preset duration.
  • the electronic device 2 is compatible with the point reading learning function and the single point reading function Under the premise, it can be judged whether the user has a point-reading learning intention or a single point-reading intention, and then performs broadcast operations according to the corresponding intention, improving the intelligence of the electronic device 2 and adding simple and fast functions to the electronic device 2. learning aids.
  • the camera 21 of the electronic device 2 is combined with the reflector, so that when the user reads the paper picture book 3, in addition to turning the page and reading, the user can respond to any place in the picture book 3 that has images or text, and assists the user in reading the picture book 3 , to deepen the understanding of picture book 3, without the help of parents to explain, users can understand the meaning of the content by pointing at the pointer when reading picture book 3, helping users deepen their understanding and satisfying users' curiosity.
  • the present invention enables users to return to the reading of paper picture books 3, without hurting the eyes or affecting the user's concentration, and realizes the function of pointing to where to read and pointing to where to speak in addition to turning pages to read.
  • the text and images in 3 have professional explanations combined with the story situation of picture book 3, making the reading process of picture book 3 vivid and easy to understand.
  • An embodiment of the present invention a picture book aided reading method, comprising:
  • S010 obtains the starting instruction for starting the point reading function, and starts the camera 21 according to the starting instruction
  • S100 controls the camera 21 to capture the picture book page 3 currently read by the user in the projection area of the reflector, so as to obtain the picture book image taken on the picture book page 3;
  • S210 extracts from the picture book image the target contour features matching the preset indicator features
  • S310 compares the picture book image with a preset sample image library, and queries the page number information corresponding to the picture book image;
  • this embodiment is an optimized embodiment of the above-mentioned embodiment, and the parts in this embodiment that are the same as those in the above-mentioned embodiment refer to the above-mentioned embodiment, and details are not repeated here.
  • the cover of the picture book 3 is basically the same, but the version is different.
  • the content of the picture book 3 will be slightly changed. Therefore, in addition to obtaining the picture book 3 name, it is necessary to further obtain the version number in order to determine which picture book 3 it is.
  • the cover image of the picture book 3 can be captured by the camera 21, so as to facilitate the subsequent search for the corresponding target point to read the picture book 3, that is, it can be determined which picture book 3 the user is currently reading.
  • searching for the target point to read the picture book 3 specifically includes: identifying the text information or barcode information in the cover image to obtain the name and version number of the picture book 3 .
  • the electronic device 2 compares the picture book image with the preset sample image library, and queries the page number information corresponding to the picture book image. In this way, the electronic device 2 can find the picture book 3 read by the user according to the page number information and the location coordinates of the reading area.
  • the current picture book 3 page which is to determine which page the user is currently looking at.
  • S330 broadcasts according to the explanation data, so as to explain the reading area indicated by the user;
  • the picture book 3 is flatly placed under the reflector, and the reflector is installed above the camera 21 .
  • the electronic device 2 can perform a local query according to the position coordinates and page number information of the point reading area to obtain the explanation data.
  • the electronic device 2 can also send the location coordinates and page number information of the point reading area to the server, and the server can query the entire network to obtain the explanation data, and the server returns the explanation data to the electronic device 2.
  • the electronic device 2 controls the playback device installed and set by itself to broadcast the explaining data.
  • the electronic device 2 can also be connected to an external device through a data interface, and the electronic device 2 controls the external device to broadcast and explain data.
  • the present invention uses the camera 21 and the reflector to take picture book images, inquires about the explanation data corresponding to the reading area, and plays according to the explanation data.
  • the present invention uses the camera 21 of the electronic device 2 combined with the reflector to read the paper picture book 3, and realizes the function of pointing or pointing to where to explain where, which can effectively assist users to understand the picture book 3 and reduce the energy-consuming energy for parents to assist in reading the picture book 3 It also avoids the problem that the user uses the electronic device 2 to read the picture book 3 and damages the health of eyesight.
  • the audio of the pointed position is played only (that is, only the reading audio of the text content of the pointed position is played).
  • the explanation data corresponding to the reading area can be queried and broadcast, which can help users to effectively understand the picture book 3, avoid the situation of cramming characters into memory, and further greatly improve the user's reading experience and learning interest in the paper picture book 3.
  • the present invention can assist the user to learn the picture book 3 through the electronic device 2, and the user only needs to use the pointer to point to the page of the picture book 3, and then the reading assistance of the electronic device 2 can be obtained, helping children understand the doubts encountered in the process of reading and learning the picture book 3, To help children develop good reading habits.
  • the present invention obtains the data that needs to be explained by collecting the images of the user's picture book, and the user does not need too many language descriptions, and only needs to point the finger to the corresponding position, which greatly improves the user experience, and the electronic device 2 is also more intelligent.
  • An embodiment of the present invention a picture book aided reading method, comprising:
  • S110 obtains the starting instruction for starting the point reading function, and starts the camera 21 according to the starting instruction
  • S210 extracts from the picture book image the target contour features matching the preset indicator features
  • S310 compares the picture book image with a preset sample image library, and queries the page number information corresponding to the picture book image;
  • S325 identifies the age group of the user who reads the picture book 3 pages, and detects whether the user's sight is focused on the screen of the electronic device 2;
  • this embodiment is an optimized embodiment of the above-mentioned embodiment, and the parts in this embodiment that are the same as those in the above-mentioned embodiment refer to the above-mentioned embodiment, and details are not repeated here.
  • the electronic device 2 is set in advance that the infant period is from 1 to 3 years old, and the school age is from 4 to 7 years old.
  • the first way for the electronic device 2 to identify the user's age group is: the key points of the face located by the SDM algorithm include: eyebrows, eyes, nose, mouth and facial contours.
  • face detection and face key point positioning can also be realized by other algorithms. What needs to be known is to select a preset number of people in the age group as the first human face image sample set in early childhood, perform face key point positioning on all the first human face image samples in the first human face image sample set, and calculate The first dimension data in all the first human face image samples (including the preset spacing, length or width of the key points of the human face), the first dimension data in all the first human face image samples is calculated as the mean value to obtain the infant stage The corresponding first preset size range.
  • the second face image sample set of school age select a preset number of people's age groups as the second face image sample set of school age, perform face key point positioning on all second face image samples in the second face image sample set, and calculate all the second face image samples.
  • the second size data in the two-person face image sample (including the preset spacing, length or width of the key points of the face), the average value of the second size data in all the second face image samples is calculated to obtain the corresponding to the school age Second preset size range.
  • the electronic device 2 After the electronic device 2 sets the relationship between the preset size range and the size data in the above manner, it calculates and obtains the size data to be matched of any one or more key points of the face of the user, wherein the size data to be matched includes the size data of the key points of the face. spacing, length or width. Then, the electronic device 2 matches the user's size data to be matched with the preset size range to obtain the target size range. Since different preset size ranges correspond to different age groups, the electronic device 2 can determine the user's age group according to the matched target size range.
  • the second way for the electronic device 2 to identify the user's age group is to bind the authentication information (such as face, iris, fingerprint) used to prove the user's identity information with the user's date of birth, so that once the electronic device 2 2 After querying the explanation data, the user is prompted to perform identity authentication, and the electronic device 2 obtains the collected authentication information, queries the user's birth date according to the user's authentication information, and calculates the user's actual age according to the birth date, and then The actual age of the user is matched with the early childhood and school age to obtain the age group of the user.
  • the authentication information such as face, iris, fingerprint
  • the way for the electronic device 2 to detect whether the user's sight is focused on the screen of the electronic device 2 is to detect the user's eye movement track, and the electronic device 2 can obtain the focus position of the user's sight line according to the eye movement track. Then, the electronic device 2 judges whether it is located in the screen area of the display screen of the electronic device 2 according to the focus position of the line of sight. If the focus position of the line of sight is within the screen area, it means that the user's line of sight is on the screen of the electronic device 2. Otherwise, it means that the user's line of sight is not on the screen of the electronic device 2 .
  • the electronic device 2 judges that the user's age group belongs to early childhood through the above process, then, regardless of whether the user's line of sight is on the screen of the electronic device 2, the electronic device 2 directly controls the speaker to play the explanatory text content, and controls the display
  • the content of the picture can be played on the screen to explain the content of the picture, so that the attention of the early childhood users can be attracted by the combination of sound and image, and the reading interest of the early childhood users can be improved.
  • the reading process becomes vivid and easy to understand.
  • the electronic device 2 judges that the age group of the user belongs to the school-age group through the above-mentioned process, and the user's line of sight is not on the screen of the electronic device 2, then the electronic device 2 only controls the speaker to play the explanatory text content, so as to avoid school-age
  • the display screen of the electronic device 2 is always in a bright state, so that the power consumption of the electronic device 2 can be effectively reduced.
  • the electronic device 2 judges that the age group of the user belongs to the school age through the above process, and the user's line of sight is not on the screen of the electronic device 2, then the electronic device 2 controls the speaker to play the explanatory text content, and controls the display screen to play Explain the content of the picture, so that it can intuitively and effectively learn for school-age users, making the reading process of picture book 3 vivid and easy to understand.
  • the explanation data includes the explanation text content and the explanation picture content.
  • the picture book 3 is flatly placed under the reflector, and the reflector is installed above the camera 21 .
  • the present invention uses the camera 21 and the reflector to take picture book images, inquires about the explanation data corresponding to the reading area, and plays according to the explanation data.
  • the present invention uses the camera 21 of the electronic device 2 combined with the reflector to read the paper picture book 3, and realizes the function of pointing or pointing to where to explain where, which can effectively assist users to understand the picture book 3 and reduce the energy-consuming energy for parents to assist in reading the picture book 3 It also avoids the problem that the user uses the electronic device 2 to read the picture book 3 and damages the health of eyesight.
  • the audio of the pointed position is played only (that is, only the reading audio of the text content of the pointed position is played).
  • the explanation data corresponding to the reading area can be queried and broadcast, which can help users to effectively understand the picture book 3, avoid the situation of cramming characters into memory, and further greatly improve the user's reading experience and learning interest in the paper picture book 3.
  • the execution subject of the embodiment of the present invention may be an electronic device 2 such as a smart phone or a tablet computer.
  • the user can start the camera 21 of the electronic device 2 by clicking the "point-to-read" control button on the operation interface, so as to use the camera 21 of the electronic device 2 to take pictures to obtain the point-to-read Object
  • the present invention controls the broadcast state of the electronic device 2 by identifying the identification result of the user's age group and the detection result of the user's line of sight, so as to prevent the display screen of the electronic device 2 from always being in the In the bright screen state, the power consumption of the electronic device 2 can be effectively reduced.
  • the present invention also provides an electronic device 2, comprising:
  • the acquisition module is used to control the camera 21 to capture the picture book 3 pages currently read by the user in the projection area of the reflector, so as to obtain the picture book images taken on the picture book 3 pages;
  • picture book 3 (Picture Book) was called a picture book in the early days.
  • the popular term for picture book 3 is a story book with pictures, simple theme, and short plot content.
  • the publications are mainly aimed at young children and children, and most of them are oriented towards reading content suitable for young children and school-age children.
  • Most of the picture book 3 books on the market today are mainly read by young children and school-age children, and the content involves literature, education, popular science, etc.
  • the electronic device 2 can be, but not limited to, a smart tablet such as a tablet, a mobile phone, a learning machine, a phone watch, or the like.
  • the electronic device 2 includes a device body and a camera 21 disposed on the device body.
  • the reflector structure 1 may be a magnetically attracted reflector structure 11 or a clip-on reflector structure 12 .
  • the reflector structure 1 supports reflective lighting for the picture book 3 placed on the front of the electronic device 2 and sends it to the camera 21, and the electronic device 2 drives the camera 21 to take pictures of the picture book 3 placed in front of the electronic device 2.
  • a picture book image is taken, and then, the electronic device 2 performs identification and analysis according to the picture book image to obtain the problem area that the user intends to ask, that is, the point reading area of the present invention, and then, the electronic device 2 queries the corresponding explanation data according to the position coordinates of the point reading area, through The display screen or the loudspeaker of the electronic device 2 broadcasts and explains the data.
  • the present invention does not require manual adjustment by the user, and only sets the reflective surface of the reflective mirror toward the camera 21. In this way, the image information in the projected area of the reflective mirror can be captured by the camera 21, which is conducive to improving the efficiency of the camera 21 in recognizing images and texts, and giving users more Good product experience.
  • the identification module is used to identify the position coordinates of the reading area indicated by the end of the pointer from the picture book image
  • the indicator includes, but is not limited to, a finger, a point reading pen, a writing pen (pen or pencil), and may even be a bar-shaped item (such as an arithmetic stick).
  • the electronic device 2 After the electronic device 2 acquires the picture book image through the above method, it can use the image recognition algorithm to identify the end of the pointer from the picture book image, and then calculate the end of the pointer indicated by the end of the pointer according to the recognized end of the pointer through the calculation principle of coordinate transformation. The location coordinates of the reading area.
  • a control module used to acquire and broadcast corresponding explanation data according to the position coordinates of the reading area and the picture book image
  • the picture book 3 is flatly placed under the reflector, and the reflector is installed above the camera 21 .
  • the electronic device 2 can perform a local query to obtain explanation data.
  • the electronic device 2 can also perform a network-wide query to obtain the explanation data, and the server returns the explanation data to the electronic device 2 .
  • the electronic device 2 controls the playback device installed and set by itself to broadcast the explaining data.
  • the electronic device 2 can also be connected to an external device through a data interface, and the electronic device 2 controls the external device to broadcast and explain data.
  • the present invention uses the camera 21 and the reflector to take picture book images, inquires about the explanation data corresponding to the reading area, and plays according to the explanation data.
  • the present invention realizes the point reading learning function (including explaining the content of the indicated point reading area, or explaining the content of the indicated point reading area after reading aloud) ), which can effectively assist the user to understand the picture book 3, reduce the energy consumption of parents assisting in reading the picture book 3, and avoid the problem that the user uses the electronic device 2 to read the picture book 3 and damages vision health.
  • the most important thing is to avoid the situation of realizing a single point reading function (that is, reading aloud the content of the point reading area indicated), that is, when the user uses the electronic device 2 to read the picture book 3, the audio of the pointed position is played only (that is, only the pointed position is played).
  • the explanation data corresponding to the reading area can be queried and broadcast, which can help users to effectively understand the picture book 3, avoid the situation of cramming characters into memory, and further greatly improve the user's reading experience and learning interest in the paper picture book 3.
  • the starting module is used to obtain the starting instruction for starting the point reading function, and start the camera 21 according to the starting instruction.
  • the electronic device 2 is installed with a reading learning APP, and the operation interface of the learning APP is provided with a "reading" control button. If the user has a demand for reading, then the control button of "reading” can be clicked, Generate a starting command to start the camera 21 to take pictures. Of course, if the user has a demand for point reading, he can also say to the electronic device 2 a sentence that meets the preset trigger keywords, such as "I want to read picture book 3", so that the electronic device 2 will generate a start command to start the camera 21 for shooting .
  • the identification module includes:
  • the extraction unit is used to extract the target contour features matching the preset indicator features from the picture book image
  • the user can use the pointer to point to the current page of the picture book 3, so that the picture book image captured by the camera 21 of the electronic device 2 includes the end of the pointer and the current page of the picture book 3 indicated by the end of the pointer.
  • the preset indicator features are extracted from the training based on the sample images of the indicators.
  • the preset indicator features of indicators such as fingers and pointing pens are bar outline features.
  • the electronic device 2 After the electronic device 2 obtains the picture book image through the above method, it uses an image recognition algorithm to detect and extract all the image features from the picture book image, and inputs the image features into the pre-trained indicator recognition model, and screens out the picture book image that matches The target profile feature for the pointer profile.
  • a calculation unit configured to calculate and obtain the pixel coordinates of the end of the indicator according to the contour features of the target
  • the processing unit is configured to determine that the user has an intention to learn point reading if the pixel coordinates remain unchanged for a preset period of time, and calculate the position coordinates of the point reading area according to the pixel coordinates.
  • the reading area indicated by the end of the pointer does not require the user to touch the end of the pointer with the current page of the picture book 3 placed under the mirror structure 1 .
  • the user can choose to touch the end of the pointer to the intended query position on the current page of Picture Book 3.
  • you can also choose to hang the end of the indicator to point to the intended query position on the current page of Picture Book 3.
  • the electronic device 2 can count the length of time that the pixel coordinates at the end of the pointer remain unchanged (hereinafter referred to as the indication time for ease of description) .
  • the present invention keeps the pixel coordinates at the end of the indicator in a constant state, performs timing to obtain the indicated duration, and compares the indicated duration with the preset duration.
  • the electronic device 2 is compatible with the point reading learning function and the single point reading function. Next, it can be judged whether the user has the intention of point-reading learning, or has a single point-reading intention, and then performs the broadcast operation according to the corresponding intention, improves the intelligence of the electronic device 2, and adds a simple and fast learning method for the electronic device 2 Accessibility.
  • the camera 21 of the electronic device 2 is combined with the reflector, so that when the user reads the paper picture book 3, in addition to turning the page and reading, the user can respond to any place in the picture book 3 that has images or text, and assists the user in reading the picture book 3 , to deepen the understanding of picture book 3, without the help of parents to explain, users can understand the meaning of the content by pointing at the pointer when reading picture book 3, helping users deepen their understanding and satisfying users' curiosity.
  • the invention enables users to return to the reading of paper picture books 3 without hurting the eyes or affecting the user's concentration, and realizes that in addition to turning pages and reading, it can also combine the text and images in picture books 3
  • the professional explanation of the situation makes the reading process of picture book 3 vivid and easy to understand.
  • control module includes:
  • the comparison unit is used to compare the picture book image with the preset sample image library, and query the page number information corresponding to the picture book image;
  • the picture book 3 has basically the same cover but different versions.
  • the content of picture book 3 will be slightly changed. Therefore, in addition to obtaining the picture book 3 name, it is necessary to further obtain the version number in order to determine which picture book 3 it is.
  • the cover image of the picture book 3 can be captured by the camera 21, so as to facilitate the subsequent search for the corresponding target point to read the picture book 3, that is, it can be determined which picture book 3 the user is currently reading. .
  • searching for the target point to read the picture book 3 specifically includes: identifying the text information or barcode information in the cover image to obtain the name and version number of the picture book 3 .
  • the electronic device 2 compares the picture book image with the preset sample image library, and queries the page number information corresponding to the picture book image. In this way, the electronic device 2 can find the picture book 3 read by the user according to the page number information and the location coordinates of the reading area.
  • the current picture book 3 page which is to determine which page the user is currently looking at.
  • the query unit is used to query and obtain the explanation data corresponding to the target of the point reading area according to the position coordinates and the page number information of the point reading area;
  • the execution unit is used for broadcasting according to the explanation data, so as to explain the reading area indicated by the user.
  • the electronic device 2 can perform a local query according to the position coordinates and page number information of the point reading area to obtain the explanation data.
  • the electronic device 2 can also send the location coordinates and page number information of the reading area to the server, and the server can search the entire network to obtain the explanation data, and the server returns the explanation data to the electronic device 2 .
  • the electronic device 2 controls the playback device installed and set by itself to broadcast the explaining data.
  • the electronic device 2 can also be connected to an external device through a data interface, and the electronic device 2 controls the external device to broadcast and explain data.
  • the present invention can assist the user to learn the picture book 3 through the electronic device 2, and the user only needs to use the pointer to point to the page of the picture book 3, and then the reading assistance of the electronic device 2 can be obtained, helping children understand the doubts encountered in the process of reading and learning the picture book 3, To help children develop good reading habits.
  • the present invention obtains the data that needs to be explained by collecting the images of the user's picture book, and the user does not need too many language descriptions, and only needs to point the finger to the corresponding position, which greatly improves the user experience, and the electronic device 2 is also more intelligent.
  • control module also includes:
  • the identification unit is used to identify the age group of the user who reads the picture book 3 pages;
  • a detection unit configured to detect whether the user's line of sight is focused on the screen of the electronic device 2;
  • the electronic device 2 pre-sets the infant period as 1 year old to 3 years old, and the school age as 4 years old to 7 years old.
  • the electronic device 2 identifies the user's age group and detects whether the user's line of sight is focused on the screen of the electronic device 2. Refer to the above-mentioned embodiments, and details will not be repeated here.
  • the execution unit is also used to control the voice of the speaker to play the explanation text content, and control the display screen to play the explanation picture content if the age group belongs to the early childhood;
  • the electronic device 2 judges that the user's age group belongs to early childhood through the above process, then, regardless of whether the user's line of sight is on the screen of the electronic device 2, the electronic device 2 directly controls the speaker to play the explanatory text content, and controls the display
  • the content of the picture can be played on the screen to explain the content of the picture, so that the attention of the early childhood users can be attracted by the combination of sound and image, and the reading interest of the early childhood users can be improved.
  • the reading process becomes vivid and easy to understand.
  • the execution unit is also used to control the speaker to play the explanatory text content and control the display screen to play the explanatory picture content if the age group belongs to the school age and the user's line of sight is not on the screen of the electronic device 2 ;
  • the electronic device 2 judges that the age group of the user belongs to the school-age group through the above process, and the user's line of sight is not on the screen of the electronic device 2, then the electronic device 2 only controls the speaker to play the explanatory text content, so as to avoid school-age
  • the display screen of the electronic device 2 is always in a bright state, so that the power consumption of the electronic device 2 can be effectively reduced.
  • the execution unit is also used to control the speaker to play the explanatory text content, and control the display screen to play the explanatory picture content if the age group belongs to the school age and the user's line of sight is on the screen of the electronic device 2;
  • the explanation data includes explanation text content and explanation picture content.
  • the electronic device 2 judges that the age group of the user belongs to the school age through the above process, and the user's line of sight is not on the screen of the electronic device 2, then the electronic device 2 controls the speaker to play the explanatory text content, and controls the display screen to play Explain the content of the picture, so that it can intuitively and effectively learn for school-age users, making the reading process of picture book 3 vivid and easy to understand.
  • the execution subject of the embodiment of the present invention may be an electronic device 2 such as a smart phone or a tablet computer.
  • the user can start the camera 21 of the electronic device 2 by clicking the "point-to-read" control button on the operation interface, so as to use the camera 21 of the electronic device 2 to take pictures to obtain the point-to-read Object
  • the present invention controls the broadcast state of the electronic device 2 by identifying the identification result of the user's age group and the detection result of the user's line of sight, so as to prevent the display screen of the electronic device 2 from always being in the In the bright screen state, the power consumption of the electronic device 2 can be effectively reduced.
  • the picture book 3 reading app takes pictures of the picture book 3 currently being read by the user, locates the coordinates of the user's fingertips through the fingertip positioning technology, and uploads the captured picture and fingertip coordinates to the cloud of the picture book 3 reading app, and Picture book 3 reading picture book 3 library resource matching in the cloud, locates which page and which page of which picture book 3 belongs to which user is currently reading;
  • Content matching match the picture book 3 with the content resources corresponding to the clicked position (that is, the explanation data of the present invention) according to the book, page and location clicked by the user, and return to the picture book 3 to read the APP display and broadcast and end.
  • the image and text of each page in the picture book 3 are divided into blocks in combination with the story situation of the picture book 3, so as to realize what to say and what to assist Users understand picture book 3, reduce the energy consumption of parents' assisted reading, and can greatly improve the reading experience of paper picture book 3.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units. It should be noted that the above embodiments can be freely combined as required. The above is only a preferred embodiment of the present invention, it should be pointed out that, for those of ordinary skill in the art, without departing from the principle of the present invention, some improvements and modifications can also be made, and these improvements and modifications can also be made. It should be regarded as the protection scope of the present invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • Human Computer Interaction (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

一种绘本辅助阅读方法和电子设备，其方法包括：控制摄像头拍摄反光镜投射区域内用户当前阅读的绘本页面，以获取对绘本页面拍摄的绘本图像（S100）；从所述绘本图像中识别出指示物末端所指示的点读区域的位置坐标（S200）；根据点读区域的位置坐标和所述绘本图像，获取并播报对应的讲解数据（S300）；其中，绘本平摊放置于所述反光镜的下方，所述反光镜安装于摄像头的上方。本方法辅助用户理解阅读绘本，降低家长辅助阅读耗费精力的情况，且避免电子阅读导致的视力下降，大大提升了绘本阅读的体验。

Description

一种绘本辅助阅读方法和电子设备 技术领域
本发明涉及教育技术领域，尤指一种绘本辅助阅读方法和电子设备。
背景技术
随着时代的进步,社会对教育越来越重视,教育对于一个国家非常重要，教育水平已经是一个国家是否强大的最重要依据。目前，对处于2岁至6岁年龄阶段的幼儿教育，以及7岁至18岁年龄阶段的学龄儿童教育也越来越普及。为了提高对幼儿、学龄儿童对学习的兴趣，通常使用绘本阅读的方式来实施教育。而且，随着技术的不断发展，智能手机、平板电脑等移动终端的功能越来越丰富，在日常生活中不再扮演着单纯的通信工具，而是成为了人们工作、休闲、娱乐的必需品。
目前绘本阅读有多种方式：
1、直接购买纸质绘本阅读，家长辅助讲解。
2、使用绘本APP购买电子绘本，在电子设备上阅读。
3、借助手机/平板的摄像头结合额外的反光镜，将纸质绘本放在摄像头可视范围，通过手机/平板中的绘本阅读APP实现翻页即读。
但是上述绘本阅读方式存在如下问题：
1、阅读纸质绘本需要家长讲解，存在讲解不专业的情况，且多数情况幼儿用户或学龄儿童需要阅读同一本绘本，非常耗费家长的精力。
2、使用电子设备阅读存在伤眼的问题，且因为电子设备应用APP繁多，导致幼儿用户或学龄儿童无法沉浸于绘本阅读，反而分心去玩电子设备中的其它应用APP。
3、通过手机/平板摄像头阅读绘本的方案，只能翻页即读播报每一页 绘本的音频，幼儿用户或学龄儿童较难理解绘本中的故事情节和内容。
发明内容
本发明的目的是提供一种绘本辅助阅读方法和电子设备，实现辅助用户理解阅读绘本，降低家长辅助阅读耗费精力的情况，且避免电子阅读导致的视力下降，大大提升了绘本阅读的体验。
本发明提供的技术方案如下：
本发明提供一种绘本辅助阅读方法，包括步骤：
控制摄像头拍摄反光镜投射区域内用户当前阅读的绘本页面，以获取对绘本页面拍摄的绘本图像；
从所述绘本图像中识别出指示物末端所指示的点读区域的位置坐标；
根据点读区域的位置坐标和所述绘本图像，获取并播报对应的讲解数据；
其中，绘本平摊放置于所述反光镜的下方，所述反光镜安装于摄像头的上方。
进一步的，所述控制摄像头拍摄反光镜投射区域内用户当前阅读的绘本页面之前包括步骤：
获取启动点读功能的启动指令，根据所述启动指令启动摄像头。
进一步的，所述从所述绘本图像中识别出指示物末端所指示的点读区域的位置坐标包括步骤：
从所述绘本图像中提取出与预设指示物特征匹配的目标轮廓特征；
根据所述目标轮廓特征计算得到所述指示物末端的像素坐标；
若所述像素坐标持续不变的时长达到预设时长，确定用户具有点读学习意图，并且根据所述像素坐标计算得到所述点读区域的位置坐标。
进一步的，所述根据点读区域的位置坐标和所述绘本图像，获取并播 报对应的讲解数据包括步骤：
将所述绘本图像与预设样本图像库进行比对，查询所述绘本图像对应的页码信息；
根据所述点读区域的位置坐标和页码信息，查询得到对应于所述点读区域目标的讲解数据；
根据所述讲解数据进行播报，以对用户所指示的点读区域进行讲解。
进一步的，所述根据所述讲解数据进行播报，以对用户所指示的点读区域进行讲解之前包括步骤：
所述根据所述讲解数据进行播报，以对用户所指示的点读区域进行讲解包括步骤：
识别阅读所述绘本页面的用户的年龄段，并且检测所述用户的视线是否聚焦于电子设备的屏幕上；
若所述年龄段属于幼儿期，控制扬声器语音播放讲解文字内容，以及控制显示屏画面播放讲解图片内容；
若所述年龄段属于学龄期，且所述用户的视线不在电子设备的屏幕上，控制扬声器语音播放讲解文字内容，且控制显示屏画面播放讲解图片内容；
若所述年龄段属于学龄期，且所述用户的视线在电子设备的屏幕上，控制扬声器语音播放讲解文字内容，且控制显示屏画面播放讲解图片内容；
其中，所述讲解数据包括讲解文字内容和讲解图片内容。
本发明还提供一种电子设备，包括：
获取模块，用于控制摄像头拍摄反光镜投射区域内用户当前阅读的绘本页面，以获取对绘本页面拍摄的绘本图像；
识别模块，用于从所述绘本图像中识别出指示物末端所指示的点读区 域的位置坐标；
控制模块，用于根据点读区域的位置坐标和所述绘本图像，获取并播报对应的讲解数据；
其中，绘本平摊放置于所述反光镜的下方，所述反光镜安装于摄像头的上方。
进一步的，还包括：
启动模块，用于获取启动点读功能的启动指令，根据所述启动指令启动摄像头。
进一步的，所述识别模块包括：
提取单元，用于从所述绘本图像中提取出与预设指示物特征匹配的目标轮廓特征；
计算单元，用于根据所述目标轮廓特征计算得到所述指示物末端的像素坐标；
处理单元，用于若所述像素坐标持续不变的时长达到预设时长，确定用户具有点读学习意图，并且根据所述像素坐标计算得到所述点读区域的位置坐标。
进一步的，所述控制模块包括：
比对单元，用于将所述绘本图像与预设样本图像库进行比对，查询所述绘本图像对应的页码信息；
查询单元，用于根据所述点读区域的位置坐标和页码信息，查询得到对应于所述点读区域目标的讲解数据；
执行单元，用于根据所述讲解数据进行播报，以对用户所指示的点读区域进行讲解。
进一步的，所述控制模块还包括：
识别单元，用于识别阅读所述绘本页面的用户的年龄段；
检测单元，用于检测所述用户的视线是否聚焦于电子设备的屏幕上；
所述执行单元，还用于若所述年龄段属于幼儿期，控制扬声器语音播放讲解文字内容，以及控制显示屏画面播放讲解图片内容；
所述执行单元，还用于若所述年龄段属于学龄期，且所述用户的视线不在电子设备的屏幕上，控制扬声器语音播放讲解文字内容，且控制显示屏画面播放讲解图片内容；
所述执行单元，还用于若所述年龄段属于学龄期，且所述用户的视线在电子设备的屏幕上，控制扬声器语音播放讲解文字内容，且控制显示屏画面播放讲解图片内容；
其中，所述讲解数据包括讲解文字内容和讲解图片内容。
通过本发明提供的一种绘本辅助阅读方法和电子设备，能够辅助用户理解阅读绘本，降低家长辅助阅读耗费精力的情况，且避免电子阅读导致的视力下降，大大提升了绘本阅读的体验。而且，还能帮助幼儿用户或学龄儿童沉浸于绘本阅读，避免分心玩电子设备中的其它应用APP，培养孩子优良的阅读习惯。
附图说明
下面将以明确易懂的方式，结合附图说明优选实施方式，对一种绘本辅助阅读方法和电子设备的上述特性、技术特征、优点及其实现方式予以进一步说明。
图1是本发明一种绘本辅助阅读方法的一个实施例的流程图；
图2是本发明一种绘本辅助阅读方法的一个场景示意图；
图3是本发明一种绘本辅助阅读方法的另一个场景示意；
图4是本发明一种绘本辅助阅读方法的另一个场景示意。
具体实施方式
以下描述中，为了说明而不是为了限定，提出了诸如特定系统结构、技术之类的具体细节，以便透彻理解本申请实施例。然而，本领域的技术人员应当清楚，在没有这些具体细节的其他实施例中也可以实现本申请。在其他情况中，省略对众所周知的系统、装置、电路以及方法的详细说明，以免不必要的细节妨碍本申请的描述。
应当理解，当在本说明书和所附权利要求书中使用时，术语“包括”指示所述描述特征、整体、步骤、操作、元素和/或组件的存在，但并不排除一个或多个其他特征、整体、步骤、操作、元素、组件和/或集合的存在或添加。
为使图面简洁，各图中只示意性地表示出了与本发明相关的部分，它们并不代表其作为产品的实际结构。另外，以使图面简洁便于理解，在有些图中具有相同结构或功能的部件，仅示意性地绘示了其中的一个，或仅标出了其中的一个。在本文中，“一个”不仅表示“仅此一个”，也可以表示“多于一个”的情形。
还应当进一步理解，在本申请说明书和所附权利要求书中使用的术语“和/或”是指相关联列出的项中的一个或多个的任何组合以及所有可能组合，并且包括这些组合。
另外，在本申请的描述中，术语“第一”、“第二”等仅用于区分描述，而不能理解为指示或暗示相对重要性。
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对照附图说明本发明的具体实施方式。显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图，并获得其他的实施方式。
本发明的一个实施例，如图1所示，一种绘本辅助阅读方法，包括：
S100控制摄像头21拍摄反光镜投射区域内用户当前阅读的绘本3页 面，以获取对绘本3页面拍摄的绘本图像；
具体的，绘本3(Picture Book)早期被称作为图画书，绘本3的通俗说法就是有图画、主题简单、情节内容简短的故事书。主要是针对幼儿、儿童所设定的出版品，多以适合幼儿、学龄儿童阅读的内容为取向。如今市面上的绘本3图书大多以低幼儿童、学龄儿童阅读为主，内容涉及文学、教育、科普等。
电子设备2可为但不局限于平板、手机、学习机、电话手表等智能平板。电子设备2包括设备本体及设于设备本体的摄像头21，反光镜结构1可以是磁吸式反光镜结构11，也可以是卡接式反光镜结构12。反光镜结构1位于该电子设备2的摄像头21的上方，且反光镜结构1的反光镜的反射面与摄像头21成一定倾斜角度设置，以使反光镜的反射面上的图像信息能够完全反射至摄像头21上。其中，磁吸式反光镜结构11包括壳体和反光镜，壳体内置第一磁性元件，此时，电子设备2的设备本体顶部设置有适配于第一磁性元件的第二磁性元件2222。第一磁性元件用于将壳体吸附于设备本体的第二磁性元件2222上，反光镜设于壳体上，且反光镜朝向设备本体的摄像头21设置，用于反射图像信息至设备本体的摄像头21。卡接式反光镜结构12包括壳体、壳体中间设有将壳体嵌在电子设备2上的安装凹槽，安装凹槽内设有对壳体进行加强固定的固定支架以及反光镜。
如图2所示，通过设置磁吸式反光镜结构11于电子设备2的摄像头21前，或者如图3所示，通过设置卡接式反光镜结构12于电子设备2的摄像头21前。这样，可以利用反光镜的反射原理扩大摄像头21的拍摄范围，使得用户在使用时，无需经常来回移动绘本3，只需要将绘本3平摊在反光镜的下方，绘本3上的内容即可倒映在反光镜里。因为反光镜正对着摄像头21，这样摄像头21即可获取反光镜投射区域内的图像信息，即摄像头21可顺利获得绘本3上的图像信息拍摄获取绘本图像。
示例性的，以电子设备2为平板为例，电子设备2包括正面和与正面相对的 背面，正面为设有显示屏的一面，该摄像头21可以设置在电子设备2的顶端，摄像头21可以是前置摄像头21或者是后置摄像头21。如图4所示，家教机是提供优质教育资源的Android平板，通过在家教机的前置摄像头21安装磁吸式反光镜结构11，形成平板底座(图中未示出)+平板+磁吸式反光镜结构11+纸质绘本3的拍照搜索解决方案，具体为平板竖着放在倾斜的平板底座上，磁吸式反光镜结构11安装在平板的前置摄像头21的上方，磁吸安装在Android平板的顶部，绘本3放置在平板的前面，绘本3顶部和平板底座对齐。通过磁吸式反光镜结构11和平板上的前置摄像头21，可以拍摄放在平板前面的绘本3得到绘本图像。
本实施例中磁吸式反光镜结构11或卡接式反光镜结构12支持针对放在电子设备2前部的绘本3进行反射采光送入摄像头21，电子设备2驱动摄像头21以对放置在电子设备2前面的绘本3进行拍摄，采集多个绘本图像摄像，然后，电子设备2根据绘本图像进行识别分析得到用户意图询问的问题区域即本发明的点读区域，然后，电子设备2根据点读区域的位置坐标查询对应的讲解数据，通过电子设备2的显示屏或者扬声器进行播报讲解数据。本发明无需用户手动调整，仅仅将反光镜的反射面朝向摄像头21设置，如此，反光镜投射区域内的图像信息可以被摄像头21拍摄到，有利于提高摄像头21识别图文的效率，给予用户更好的产品体验。
S200从绘本图像中识别出指示物末端所指示的点读区域的位置坐标；
具体的，指示物包括但是不限于手指、点读笔、书写笔(钢笔或者铅笔)，甚至还可以是条状物品(例如算术棒)。例如，人体手指的末端就指的是手指尖，点读笔的末端就是笔尖。优选的，指示物一般为用户的手指。因为使用手指是的用户进行点读学习时无需手持点读笔或者书写笔等外围设备，用户通过手指直接可以在绘本3上进行点读，不必使用外围设备进行操作，给用户使用提供了极大的方便，大大地提升了使用的便利性 和使用体验。电子设备2通过上述方式获取到绘本图像后，可以使用图像识别算法从绘本图像中识别出指示物末端，进而根据识别出的指示物末端，通过坐标转换计算原理，计算得到指示物末端所指示的点读区域的位置坐标。
S300根据点读区域的位置坐标和所述绘本图像，获取并播报对应的讲解数据；
其中，绘本3平摊放置于反光镜的下方，反光镜安装于摄像头21的上方。
具体的，电子设备2通过上述方式计算得到点读区域的位置坐标后，根据点读区域的位置坐标和所述绘本图像，可以由电子设备2进行本地查询得到讲解数据。当然，也可以将绘本图像和点读区域的位置坐标发送到服务器，由服务器进行全网查询得到讲解数据，服务器再将讲解数据返回给电子设备2。电子设备2获取到讲解数据后，电子设备2自行控制自身已经安装设置的播放设备进行播报讲解数据。当然，也可以将电子设备2通过数据接口连接外接设备，由电子设备2控制外接设备进行播报讲解数据。讲解数据是指对点读区域所对应内容进行解释说明的数据，讲解数据的格式包括但是不限于音频、视频、图片等格式的文件。示例性的，讲解数据可以是家长、老师预先录制的对点读区域进行解释说明的录音或视频。
本发明利用摄像头21、反光镜拍摄绘本图像后，查询对应于点读区域的讲解数据，并根据讲解数据进行播放。本发明通过电子设备2的摄像头21结合反光镜阅读纸质绘本3的基础上，实现点读学习功能(包括讲解所指示的点读区域的内容，或者朗读后讲解所指示的点读区域的内容)，这样能够有效辅助用户理解绘本3，降低家长辅助阅读绘本3耗费精力的情况，也避免用户使用电子设备2阅读绘本3损害视力健康的问题。最重要 的是，也避免实现单一点读功能(即朗读所指示的点读区域的内容)的情况，即用户使用电子设备2阅读绘本3时单一播放所指位置的音频(即只播放所指位置对应的朗读音频)。通过本发明能够查询对应于点读区域的讲解数据并播报，能够帮助用户有效理解绘本3，避免填鸭式进行文字记忆的情况，进而大大提升了用户对纸质绘本3阅读的体验以及学习兴趣。
本发明的一个实施例，一种绘本辅助阅读方法，包括：
S010获取启动点读功能的启动指令，根据启动指令启动摄像头21；
具体的，本实施例是上述实施例的优化实施例，本实施例中与上述实施例相同的部分参见上述实施例，在此不再一一赘述。电子设备2上安装有点读学习APP，点读学习APP的操作界面上设置有“点读”的控件按钮，如果用户具有点读需求，那么就可以点击“点读”的控件按钮，生成启动指令以便启动摄像头21进行拍摄。当然，如果用户具有点读需求，也可以对着电子设备2说符合预设触发关键词的语句，例如“我要读绘本3”，这样电子设备2会生成启动指令，以便启动摄像头21进行拍摄。
S100控制摄像头21拍摄反光镜投射区域内用户当前阅读的绘本3页面，以获取对绘本3页面拍摄的绘本图像；
S210从绘本图像中提取出与预设指示物特征匹配的目标轮廓特征；
具体的，用户可以使用指示物指着绘本3的当前页面，这样，电子设备2的摄像头21采集的绘本图像就包括指示物末端，以及指示物末端所指示绘本3的当前页面。预设指示物特征是根据指示物的样本图像进行训练提取到的，通常手指，点读笔等指示物的预设指示物特征就是条形轮廓特征。
电子设备2通过上述方式获取到绘本图像后，使用图像识别算法从绘本图像中，检测提取出所有的图像特征，并将图像特征输入至预先训练的指示物识别模型中，从绘本图像筛选出符合指示物轮廓的目标轮廓特征。
示例性的，通过大量带有手指的图片并人工标注指甲区域的位置，进而对机器学习模型进行训练，得到手指识别模型，将绘本图像输入手指识别模型，可以确定绘本图像中符合指甲的目标轮廓特征。其中，机器学习模型包括但不限于全连接神经网络模型、卷积神经网络模型、循环神经网络模型等。
S220根据所述目标轮廓特征计算得到所述指示物末端的像素坐标；
S230若所述像素坐标持续不变的时长达到预设时长，确定用户具有点读学习意图，并且根据所述像素坐标计算得到所述点读区域的位置坐标；
具体的，需要知道的是，指示物末端所指示的点读区域，并不强制要求用户将指示物末端与放置在反光镜结构1下方的绘本3的当前页面接触。用户可以选择将指示物末端与绘本3当前页面上意图询问的位置进行接触。当然，也可以选择将指示物末端悬空指向绘本3当前页面上意图询问的位置。由于每张绘本图像具有时间戳，因此，根据多张绘本图像及其对应的时间戳，电子设备2可统计指示物末端的像素坐标持续不变的时长(下文为了便于叙述，简称为指示时长)。
如果指示物末端与绘本3当前页面上意图询问的位置接触，那么，电子设备2可以计算出目标轮廓特征的像素坐标，再根据这个像素坐标进行转换计算得到指示物末端在世界坐标系上的位置坐标。如果指示时长未达到预设时长，说明用户只是具有单一点读意图。如果指示时长达到预设时长，说明用户具有点读学习意图。此时，由于指示物末端与绘本3纸质接触，绘本3纸质与指示物末端接触的间隙几乎可忽略不计，因此，电子设备2可将指示物末端在世界坐标系上的位置坐标作为点读区域的位置坐标。计算出目标轮廓特征的像素坐标，再根据这个像素坐标进行转换计算，得到指示物末端在世界坐标系上的位置坐标具体为：从绘本图像中获取该指示物末端的像素坐标，以及作为参照物的绘本3书页边框的像素坐标。由于 像素坐标均可以从绘本图像中获得，那么就可以获得绘本图像中手指相对于绘本3书页边框的相对位置，再结合绘本3书页边框的实际位置，便可计算获得指示物末端在世界坐标系上的位置坐标。
当然，如果指示物末端悬空指向绘本3当前页面上意图询问的位置，即指示物末端与绘本3当前页面上意图询问的位置不接触，那么，电子设备2可以计算出指示物末端所对应的第一像素坐标，以及指示物的首端所对应的第二像素坐标，并且计算得到绘本3当前页面所在平面的平面方程，然后，电子设备2根据第一像素坐标、第二像素坐标，分别进行转换计算得到指示物末端在世界坐标系上的第一位置坐标，以及指示物的首端在世界坐标系上的第二位置坐标。如果指示时长未达到预设时长，说明用户只是具有单一点读意图。如果指示时长达到预设时长，说明用户具有点读学习意图。然后，电子设备2根据第一位置坐标、第二位置坐标得到指示物所对应的直线方程，根据指示物所对应的直线方程和绘本3当前页面的平面方程，可以进行求解计算得到指示物末端所指示的点读区域的位置坐标。
S300根据点读区域的位置坐标和所述绘本图像，获取并播报对应的讲解数据；
其中，绘本3平摊放置于反光镜的下方，反光镜安装于摄像头21的上方。
本发明利用摄像头21、反光镜拍摄绘本图像后，查询对应于点读区域的讲解数据，并根据讲解数据进行播放。本发明通过电子设备2的摄像头21结合反光镜阅读纸质绘本3的基础上，实现点读学习功能(包括讲解所指示的点读区域的内容，或者朗读后讲解所指示的点读区域的内容)，这样能够有效辅助用户理解绘本3，降低家长辅助阅读绘本3耗费精力的情况，也避免用户使用电子设备2阅读绘本3损害视力健康的问题。最重要的是，也避免实现单一点读功能(即朗读所指示的点读区域的内容)的情 况，即用户使用电子设备2阅读绘本3时单一播放所指位置的音频(即只播放所指位置对应的朗读音频)。通过本发明能够查询对应于点读区域的讲解数据并播报，能够帮助用户有效理解绘本3，避免填鸭式进行文字记忆的情况，进而大大提升了用户对纸质绘本3阅读的体验以及学习兴趣。
此外，本发明对指示物末端的像素坐标保持不变的状态，进行计时得到指示时长，并且将指示时长与预设时长进行比较，这样，电子设备2在兼容点读学习功能和单一点读功能的前提下，能够判断出用户到底是具有点读学习意图，还是具有单一点读意图，进而根据对应的意图进行播报操作，提高电子设备2的智能性的同时，为电子设备2新增简单快捷的学习辅助功能。本发明通过电子设备2摄像头21结合反光镜，使得用户阅读纸质绘本3时除了翻页即读外，用户点击绘本3任何一个有图像或文字的地方都可响应的方式，辅助用户阅读绘本3，加深对绘本3的理解，不需要家长辅助讲解，用户在阅读绘本3时通过指示物指一指就能了解所指内容的意思，帮助用户加深理解，满足用户的好奇心。本发明使得用户回归到纸质绘本3的阅读中，既不伤眼又不影响用户的专注力，而且实现除了翻页即读，还能指哪儿读哪儿，指哪儿讲哪儿的功能，对于绘本3中的文字和图像都有结合绘本3故事情境的专业讲解，让绘本3阅读过程变得生动易理解。
本发明的一个实施例，一种绘本辅助阅读方法，包括：
S010获取启动点读功能的启动指令，根据启动指令启动摄像头21；
S100控制摄像头21拍摄反光镜投射区域内用户当前阅读的绘本3页面，以获取对绘本3页面拍摄的绘本图像；
S210从绘本图像中提取出与预设指示物特征匹配的目标轮廓特征；
S220根据所述目标轮廓特征计算得到所述指示物末端的像素坐标；
S230若所述像素坐标持续不变的时长达到预设时长，确定用户具有点 读学习意图，并且根据所述像素坐标计算得到所述点读区域的位置坐标；
S310将绘本图像与预设样本图像库进行比对，查询绘本图像对应的页码信息；
具体的，本实施例是上述实施例的优化实施例，本实施例中与上述实施例相同的部分参见上述实施例，在此不再一一赘述。在实际使用过程中，绘本3因为版权及改版等问题，会存在大量封面基本相同，但是版本不一样的问题，这时绘本3的内容会有少量更改。因此，除了获得绘本3名称外，还需要进一步获取版本号，以便确定具体是哪本绘本3。在开始阅读之前，或者在拍摄绘本图像之前，可先通过摄像头21采集绘本3的封面图像，便于后续查找到对应的目标点读绘本3，也就是能够确定用户当前阅读的是哪一本绘本3。其中，根据封面图像查找目标点读绘本3具体为：识别封面图像中的文字信息或条码信息，以获取绘本3名称和版本号。电子设备2将绘本图像与预设样本图像库进行比对，查询绘本图像对应的页码信息，这样，电子设备2可以根据页码信息和点读区域的位置坐标，可查找到用户所阅读绘本3的当前绘本3页面，即确定用户当前是看的是哪一页。
S320根据点读区域的位置坐标和页码信息，查询得到对应于点读区域目标的讲解数据；
S330根据讲解数据进行播报，以对用户所指示的点读区域进行讲解；
其中，绘本3平摊放置于反光镜的下方，反光镜安装于摄像头21的上方。
具体的，电子设备2通过上述方式获取到点读区域的位置坐标和页码信息后，可以由电子设备2根据点读区域的位置坐标和页码信息，进行本地查询得到讲解数据。当然，电子设备2也可以将点读区域的位置坐标和页码信息发送到服务器，由服务器进行全网查询得到讲解数据，服务器再 将讲解数据返回给电子设备2。电子设备2获取到讲解数据后，电子设备2自行控制自身已经安装设置的播放设备进行播报讲解数据。当然，也可以将电子设备2通过数据接口连接外接设备，由电子设备2控制外接设备进行播报讲解数据。
本发明利用摄像头21、反光镜拍摄绘本图像后，查询对应于点读区域的讲解数据，并根据讲解数据进行播放。本发明通过电子设备2的摄像头21结合反光镜阅读纸质绘本3的基础上，实现指哪儿或者点哪儿解释哪儿的功能，这样能够有效辅助用户理解绘本3，降低家长辅助阅读绘本3耗费精力的情况，也避免用户使用电子设备2阅读绘本3损害视力健康的问题。最重要的是，也避免单一实现指哪儿读哪儿的情况，即用户使用电子设备2阅读绘本3时单一播放所指位置的音频(即只播放所指位置的文字内容的朗读音频)。通过本发明能够查询对应于点读区域的讲解数据并播报，能够帮助用户有效理解绘本3，避免填鸭式进行文字记忆的情况，进而大大提升了用户对纸质绘本3阅读的体验以及学习兴趣。
本发明可以通过电子设备2辅助用户学习绘本3，只需用户使用指示物指着绘本3页面，就可以获得电子设备2的阅读辅助，帮助孩子理解阅读学习绘本3过程中遇到的疑惑地方，以帮助孩子培养良好的阅读习惯。本发明通过采集用户的绘本图像获取需要讲解数据，用户无需过多语言描述，只需手指指向相应位置即可，大大提高了用户体验，该电子设备2的智能程度也更高。
本发明的一个实施例，一种绘本辅助阅读方法，包括：
S110获取启动点读功能的启动指令，根据启动指令启动摄像头21；
S210从绘本图像中提取出与预设指示物特征匹配的目标轮廓特征；
S220根据所述目标轮廓特征计算得到所述指示物末端的像素坐标；
S230若所述像素坐标持续不变的时长达到预设时长，确定用户具有点 读学习意图，并且根据所述像素坐标计算得到所述点读区域的位置坐标；
S310将绘本图像与预设样本图像库进行比对，查询绘本图像对应的页码信息；
S320根据点读区域的位置坐标和页码信息，查询得到对应于点读区域目标的讲解数据；
S325识别阅读绘本3页面的用户的年龄段，并且检测所述用户的视线是否聚焦于电子设备2的屏幕上；
具体的，本实施例是上述实施例的优化实施例，本实施例中与上述实施例相同的部分参见上述实施例，在此不再一一赘述。电子设备2事先设定幼儿期为1周岁到满3周岁，学龄期为4周岁～7周岁。
电子设备2识别用户的年龄段的第一种方式为：通过SDM算法定位的人脸关键点包括：眉毛、眼睛、鼻子、嘴巴和面部轮廓。当然人脸检测和人脸关键点定位也可以通过其他算法来实现。需要知道的是，选取预置数目的人物年龄段为幼儿期的第一人脸图像样本集，对第一人脸图像样本集中的所有第一人脸图像样本进行人脸关键点定位，并计算所有第一人脸图像样本中第一尺寸数据(包括人脸关键点的预设之间的间距、长度或宽度)，将所有第一人脸图像样本中第一尺寸数据进行均值计算得到幼儿期对应的第一预设尺寸范围。同理，选取预置数目的人物年龄段为学龄期的第二人脸图像样本集，对第二人脸图像样本集中的所有第二人脸图像样本进行人脸关键点定位，并计算所有第二人脸图像样本中第二尺寸数据(包括人脸关键点的预设之间的间距、长度或宽度)，将所有第二人脸图像样本中第二尺寸数据进行均值计算得到学龄期对应的第二预设尺寸范围。
电子设备2通过上述方式设定预设尺寸范围和尺寸数据之间的关系后，计算得到用户的任意一个或者多个人脸关键点的待匹配尺寸数据，其中待匹配尺寸数据包括人脸关键点之间的间距、长度或宽度。然后，电子 设备2将用户的待匹配尺寸数据与预设尺寸范围进行匹配得到目标尺寸范围。由于不同预设尺寸范围对应有不同的年龄段，这样，电子设备2根据匹配得到的目标尺寸范围，可以确定用户的年龄段。
电子设备2识别用户的年龄段的第二种方式为：将用于证明用户的身份信息的认证信息(例如人脸、虹膜、指纹)与用户的出生年月进行绑定，这样，一旦电子设备2查询出讲解数据后，就提示用户进行身份认证，电子设备2获取采集到的认证信息，根据用户的认证信息查询用户的出生年月，以根据出生年月计算得到用户的实际年龄，进而将用户的实际年龄与幼儿期和学龄期匹配得到用户的年龄段。
电子设备2检测所述用户的视线是否聚焦于电子设备2的屏幕上的方式为检测用户的眼球运动轨迹，电子设备2根据眼球运动轨迹可以获取用户视线聚焦位置。然后，电子设备2根据视线聚焦位置判断是否位于电子设备2显示屏的屏幕区域，如果视线聚焦位置位于屏幕区域内，则说明用户的视线在电子设备2的屏幕上，如果视线聚焦位置位于屏幕区域外，则说明用户的视线不在电子设备2的屏幕上。
S331若年龄段属于幼儿期，控制扬声器语音播放讲解文字内容，以及控制显示屏画面播放讲解图片内容；
具体的，电子设备2通过上述流程判断出用户的年龄段属于幼儿期，那么，不论用户的视线是否在电子设备2的屏幕上，电子设备2均直接控制扬声器语音播放讲解文字内容，并且控制显示屏画面播放讲解图片内容，这样可以通过声图结合的方式，吸引幼儿期用户的注意力，提高幼儿期用户的阅读兴趣的前提下，能够更加直观、有效地为幼儿期用户进行学习，让绘本3阅读过程变得生动易理解。
S332若年龄段属于学龄期，且用户的视线不在电子设备2的屏幕上，控制扬声器语音播放讲解文字内容，且控制显示屏画面播放讲解图片内 容；
具体的，电子设备2通过上述流程判断出用户的年龄段属于学龄期，且用户的视线不在电子设备2的屏幕上，那么，电子设备2就只控制扬声器语音播放讲解文字内容，这样避免学龄期用户在使用点读功能的过程中，电子设备2的显示屏一直处于亮屏状态，从而可以有效降低电子设备2的功耗。
S333若年龄段属于学龄期，且用户的视线在电子设备2的屏幕上，控制扬声器语音播放讲解文字内容，且控制显示屏画面播放讲解图片内容；
具体的，电子设备2通过上述流程判断出用户的年龄段属于学龄期，且用户的视线不在电子设备2的屏幕上，那么，电子设备2控制扬声器语音播放讲解文字内容，并且控制显示屏画面播放讲解图片内容，这样，可以直观、有效地为学龄期用户进行学习，让绘本3阅读过程变得生动易理解。
其中，讲解数据包括讲解文字内容和讲解图片内容其中，绘本3平摊放置于反光镜的下方，反光镜安装于摄像头21的上方。
本发明利用摄像头21、反光镜拍摄绘本图像后，查询对应于点读区域的讲解数据，并根据讲解数据进行播放。本发明通过电子设备2的摄像头21结合反光镜阅读纸质绘本3的基础上，实现指哪儿或者点哪儿解释哪儿的功能，这样能够有效辅助用户理解绘本3，降低家长辅助阅读绘本3耗费精力的情况，也避免用户使用电子设备2阅读绘本3损害视力健康的问题。最重要的是，也避免单一实现指哪儿读哪儿的情况，即用户使用电子设备2阅读绘本3时单一播放所指位置的音频(即只播放所指位置的文字内容的朗读音频)。通过本发明能够查询对应于点读区域的讲解数据并播报，能够帮助用户有效理解绘本3，避免填鸭式进行文字记忆的情况，进而大大提升了用户对纸质绘本3阅读的体验以及学习兴趣。
本发明实施例的执行主体可以是诸如智能手机或平板电脑等的电子设备2。其中，在用户使用电子设备2的点读功能时，用户可以通过点击操作界面上的“点读”控件按钮来启动电子设备2的摄像头21，以使用电子设备2的摄像头21拍摄得到待点读对象，本发明通过识别用户的年龄段的识别结果，以及用户的视线的检测结果，控制电子设备2的播报状态，可以避免用户在使用点读功能的过程中，电子设备2的显示屏一直处于亮屏状态，从而可以有效降低电子设备2的功耗。
本发明还提供一种电子设备2，包括：
获取模块，用于控制摄像头21拍摄反光镜投射区域内用户当前阅读的绘本3页面，以获取对绘本3页面拍摄的绘本图像；
具体的，绘本3(Picture Book)早期被称作为图画书，绘本3的通俗说法就是有图画、主题简单、情节内容简短的故事书。主要是针对幼儿、儿童所设定的出版品，多以适合幼儿、学龄儿童阅读的内容为取向。如今市面上的绘本3图书大多以低幼儿童、学龄儿童阅读为主，内容涉及文学、教育、科普等。
电子设备2可为但不局限于平板、手机、学习机、电话手表等智能平板。电子设备2包括设备本体及设于设备本体的摄像头21，反光镜结构1可以是磁吸式反光镜结构11，也可以是卡接式反光镜结构12。
本实施例中反光镜结构1支持针对放在电子设备2前部的绘本3进行反射采光送入摄像头21，电子设备2驱动摄像头21以对放置在电子设备2前面的绘本3进行拍摄，采集多个绘本图像摄像，然后，电子设备2根据绘本图像进行识别分析得到用户意图询问的问题区域即本发明的点读区域，然后，电子设备2根据点读区域的位置坐标查询对应的讲解数据，通过电子设备2的显示屏或者扬声器进行播报讲解数据。本发明无需用户手动调整，仅仅将反光镜的反射面朝向摄像头21设置，如此，反光镜投射区域内的图像信息可以被摄像头21拍摄到，有利于提高摄像头21识别图文的效率，给予用户更好的产品体验。
识别模块，用于从绘本图像中识别出指示物末端所指示的点读区域的位置坐标；
具体的，指示物包括但是不限于手指、点读笔、书写笔(钢笔或者铅笔)，甚至还可以是条状物品(例如算术棒)。电子设备2通过上述方式获取到绘本图像后，可以使用图像识别算法从绘本图像中识别出指示物末端，进而根据识别出的指示物末端，通过坐标转换计算原理，计算得到指示物末端所指示的点读区域的位置坐标。
控制模块，用于根据点读区域的位置坐标和所述绘本图像，获取并播报对应的讲解数据；
其中，绘本3平摊放置于反光镜的下方，反光镜安装于摄像头21的上方。
具体的，电子设备2通过上述方式计算得到点读区域的位置坐标后，根据点读区域的位置坐标和所述绘本图像，可以由电子设备2进行本地查询得到讲解数据。当然，也可以将绘本图像和点读区域的位置坐标发送到服务器，由服务器进行全网查询得到讲解数据，服务器再将讲解数据返回给电子设备2。电子设备2获取到讲解数据后，电子设备2自行控制自身已经安装设置的播放设备进行播报讲解数据。当然，也可以将电子设备2通过数据接口连接外接设备，由电子设备2控制外接设备进行播报讲解数据。
本发明利用摄像头21、反光镜拍摄绘本图像后，查询对应于点读区域的讲解数据，并根据讲解数据进行播放。本发明通过电子设备2的摄像头21结合反光镜阅读纸质绘本3的基础上，实现点读学习功能(包括讲解所指示的点读区域的内容，或者朗读后讲解所指示的点读区域的内容)，这样能够有效辅助用户理解绘本3，降低家长辅助阅读绘本3耗费精力的情况，也避免用户使用电子设备2阅读绘本3损害视力健康的问题。最重要 的是，也避免实现单一点读功能(即朗读所指示的点读区域的内容)的情况，即用户使用电子设备2阅读绘本3时单一播放所指位置的音频(即只播放所指位置对应的朗读音频)。通过本发明能够查询对应于点读区域的讲解数据并播报，能够帮助用户有效理解绘本3，避免填鸭式进行文字记忆的情况，进而大大提升了用户对纸质绘本3阅读的体验以及学习兴趣。
基于前述实施例，还包括：
启动模块，用于获取启动点读功能的启动指令，根据启动指令启动摄像头21。
具体的，电子设备2上安装有点读学习APP，点读学习APP的操作界面上设置有“点读”的控件按钮，如果用户具有点读需求，那么就可以点击“点读”的控件按钮，生成启动指令以便启动摄像头21进行拍摄。当然，如果用户具有点读需求，也可以对着电子设备2说符合预设触发关键词的语句，例如“我要读绘本3”，这样电子设备2会生成启动指令，以便启动摄像头21进行拍摄。
基于前述实施例，识别模块包括：
提取单元，用于从绘本图像中提取出与预设指示物特征匹配的目标轮廓特征；
具体的，用户可以使用指示物指着绘本3的当前页面，这样，电子设备2的摄像头21采集的绘本图像就包括指示物末端，以及指示物末端所指示绘本3的当前页面。预设指示物特征是根据指示物的样本图像进行训练提取到的，通常手指，点读笔等指示物的预设指示物特征就是条形轮廓特征。
电子设备2通过上述方式获取到绘本图像后，使用图像识别算法从绘本图像中，检测提取出所有的图像特征，并将图像特征输入至预先训练的指示物识别模型中，从绘本图像筛选出符合指示物轮廓的目标轮廓特征。
计算单元，用于根据所述目标轮廓特征计算得到所述指示物末端的像素坐标；
处理单元，用于若所述像素坐标持续不变的时长达到预设时长，确定用户具有点读学习意图，并且根据所述像素坐标计算得到所述点读区域的位置坐标。
具体的，需要知道的是，指示物末端所指示的点读区域，并不强制要求用户将指示物末端与放置在反光镜结构1下方的绘本3的当前页面接触。用户可以选择将指示物末端与绘本3当前页面上意图询问的位置进行接触。当然，也可以选择将指示物末端悬空指向绘本3当前页面上意图询问的位置。由于每张绘本图像具有时间戳，因此，根据多张绘本图像及其对应的时间戳，电子设备2可统计指示物末端的像素坐标持续不变的时长(下文为了便于叙述，简称为指示时长)。
本发明对指示物末端的像素坐标保持不变的状态，进行计时得到指示时长，并且将指示时长与预设时长进行比较，这样，电子设备2在兼容点读学习功能和单一点读功能的前提下，能够判断出用户到底是具有点读学习意图，还是具有单一点读意图，进而根据对应的意图进行播报操作，提高电子设备2的智能性的同时，为电子设备2新增简单快捷的学习辅助功能。本发明通过电子设备2摄像头21结合反光镜，使得用户阅读纸质绘本3时除了翻页即读外，用户点击绘本3任何一个有图像或文字的地方都可响应的方式，辅助用户阅读绘本3，加深对绘本3的理解，不需要家长辅助讲解，用户在阅读绘本3时通过指示物指一指就能了解所指内容的意思，帮助用户加深理解，满足用户的好奇心。本发明使得用户回归到纸质绘本3的阅读中，既不伤眼又不影响用户的专注力，而且实现除了翻页即读，还能对于绘本3中的文字和图像都有结合绘本3故事情境的专业讲解，让绘本3阅读过程变得生动易理解。
基于前述实施例，控制模块包括：
比对单元，用于将绘本图像与预设样本图像库进行比对，查询绘本图像对应的页码信息；
具体的，在实际使用过程中，绘本3因为版权及改版等问题，会存在大量封面基本相同，但是版本不一样的问题，这时绘本3的内容会有少量更改。因此，除了获得绘本3名称外，还需要进一步获取版本号，以便确定具体是哪本绘本3。在开始阅读之前，或者在拍摄绘本图像之前，可先通过摄像头21采集绘本3的封面图像，便于后续查找到对应的目标点读绘本3，也就是能够确定用户当前阅读的是哪一本绘本3。其中，根据封面图像查找目标点读绘本3具体为：识别封面图像中的文字信息或条码信息，以获取绘本3名称和版本号。电子设备2将绘本图像与预设样本图像库进行比对，查询绘本图像对应的页码信息，这样，电子设备2可以根据页码信息和点读区域的位置坐标，可查找到用户所阅读绘本3的当前绘本3页面，即确定用户当前是看的是哪一页。
查询单元，用于根据点读区域的位置坐标和页码信息，查询得到对应于点读区域目标的讲解数据；
执行单元，用于根据讲解数据进行播报，以对用户所指示的点读区域进行讲解。
具体的，电子设备2通过上述方式获取到点读区域的位置坐标和页码信息后，可以由电子设备2根据点读区域的位置坐标和页码信息，进行本地查询得到讲解数据。当然，电子设备2也可以将点读区域的位置坐标和页码信息发送到服务器，由服务器进行全网查询得到讲解数据，服务器再将讲解数据返回给电子设备2。电子设备2获取到讲解数据后，电子设备2自行控制自身已经安装设置的播放设备进行播报讲解数据。当然，也可以将电子设备2通过数据接口连接外接设备，由电子设备2控制外接设备 进行播报讲解数据。
本发明可以通过电子设备2辅助用户学习绘本3，只需用户使用指示物指着绘本3页面，就可以获得电子设备2的阅读辅助，帮助孩子理解阅读学习绘本3过程中遇到的疑惑地方，以帮助孩子培养良好的阅读习惯。本发明通过采集用户的绘本图像获取需要讲解数据，用户无需过多语言描述，只需手指指向相应位置即可，大大提高了用户体验，该电子设备2的智能程度也更高。
基于前述实施例，控制模块还包括：
识别单元，用于识别阅读绘本3页面的用户的年龄段；
检测单元，用于检测所述用户的视线是否聚焦于电子设备2的屏幕上；
具体的，电子设备2事先设定幼儿期为1周岁到满3周岁，学龄期为4周岁～7周岁。电子设备2识别用户的年龄段的，以及检测所述用户的视线是否聚焦于电子设备2的屏幕上参见上述实施例，在此不再一一赘述。
执行单元，还用于若年龄段属于幼儿期，控制扬声器语音播放讲解文字内容，以及控制显示屏画面播放讲解图片内容；
具体的，电子设备2通过上述流程判断出用户的年龄段属于幼儿期，那么，不论用户的视线是否在电子设备2的屏幕上，电子设备2均直接控制扬声器语音播放讲解文字内容，并且控制显示屏画面播放讲解图片内容，这样可以通过声图结合的方式，吸引幼儿期用户的注意力，提高幼儿期用户的阅读兴趣的前提下，能够更加直观、有效地为幼儿期用户进行学习，让绘本3阅读过程变得生动易理解。
执行单元，还用于若年龄段属于学龄期，且用户的视线不在电子设备2的屏幕上，控制扬声器语音播放讲解文字内容，且控制显示屏画面播放讲解图片内容；
具体的，电子设备2通过上述流程判断出用户的年龄段属于学龄期， 且用户的视线不在电子设备2的屏幕上，那么，电子设备2就只控制扬声器语音播放讲解文字内容，这样避免学龄期用户在使用点读功能的过程中，电子设备2的显示屏一直处于亮屏状态，从而可以有效降低电子设备2的功耗。
执行单元，还用于若年龄段属于学龄期，且用户的视线在电子设备2的屏幕上，控制扬声器语音播放讲解文字内容，且控制显示屏画面播放讲解图片内容；
其中，讲解数据包括讲解文字内容和讲解图片内容。
具体的，电子设备2通过上述流程判断出用户的年龄段属于学龄期，且用户的视线不在电子设备2的屏幕上，那么，电子设备2控制扬声器语音播放讲解文字内容，并且控制显示屏画面播放讲解图片内容，这样，可以直观、有效地为学龄期用户进行学习，让绘本3阅读过程变得生动易理解。
本发明实施例的执行主体可以是诸如智能手机或平板电脑等的电子设备2。其中，在用户使用电子设备2的点读功能时，用户可以通过点击操作界面上的“点读”控件按钮来启动电子设备2的摄像头21，以使用电子设备2的摄像头21拍摄得到待点读对象，本发明通过识别用户的年龄段的识别结果，以及用户的视线的检测结果，控制电子设备2的播报状态，可以避免用户在使用点读功能的过程中，电子设备2的显示屏一直处于亮屏状态，从而可以有效降低电子设备2的功耗。
示例性的，本发明的关键流程如下：
1、搭建装置：搭建好绘本3阅读装置，将绘本3摆放到家教机前面，对着家教机说“小布小布，我要读绘本3”即可开始阅读绘本3；
2、翻页点读：用户阅读的过程中，指着绘本3中的图像或文字持续一段时间(≥200毫秒)，家教机中的绘本3阅读APP即响应用户的点读 学习意图，进入点读学习搜索流程；
3、图像定位：绘本3阅读APP拍摄当前用户阅读的绘本3，并通过指尖定位技术定位到用户手指指尖的坐标，将拍摄的图片和指尖坐标上传到绘本3阅读APP的云端，和绘本3阅读云端的绘本3图库资源匹配，定位到用户当前阅读绘本3属于哪一本书的哪一页的哪一个位置；
4、内容匹配：根据用户点击的书、页和位置匹配该绘本3在点击位置对应的内容资源(即本发明的讲解数据)，并返回给绘本3阅读APP展示播报后结束。
本发明通过在平板前置摄像头21结合反光镜阅读纸质绘本3的基础上，将绘本3中每一页的图像和文字，结合绘本3的故事情境按区块分割，实现点哪讲哪辅助用户理解绘本3，降低家长辅助阅读耗费精力的情况，而且能够大大提升了纸质绘本3阅读的体验。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详细描述或记载的部分，可以参见其他实施例的相关描述。
另外，在本申请各个实施例中的各功能单元可能集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。应当说明的是，上述实施例均可根据需要自由组合。以上所述仅是本发明的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本发明原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本发明的保护范围。

Claims (10)

  1. 一种绘本辅助阅读方法，其特征在于，包括步骤：
    控制摄像头拍摄反光镜投射区域内当前阅读的绘本页面，以获取对绘本页面拍摄的绘本图像；
    从所述绘本图像中识别出指示物末端所指示的点读区域的位置坐标；
    根据点读区域的位置坐标和所述绘本图像，获取并播报对应的讲解数据；
    其中，绘本页面平摊放置于所述反光镜的下方，所述反光镜安装于摄像头的上方。
  2. 根据权利要求1所述的绘本辅助阅读方法，其特征在于，所述控制摄像头拍摄反光镜投射区域内用户当前阅读的绘本页面之前包括步骤：
    获取启动点读功能的启动指令，根据所述启动指令启动摄像头。
  3. 根据权利要求1所述的绘本辅助阅读方法，其特征在于，所述从所述绘本图像中识别出指示物末端所指示的点读区域的位置坐标包括步骤：
    从所述绘本图像中提取出与预设指示物特征匹配的目标轮廓特征；
    根据所述目标轮廓特征计算得到所述指示物末端的像素坐标；
    若所述像素坐标持续不变的时长达到预设时长，确定用户具有点读学习意图，并且根据所述像素坐标计算得到所述点读区域的位置坐标。
  4. 根据权利要求1-3任一项所述的绘本辅助阅读方法，其特征在于，所述根据点读区域的位置坐标和所述绘本图像，获取并播报对应的讲解数据包括步骤：
    将所述绘本图像与预设样本图像库进行比对，查询所述绘本图像对应的页码信息；
    根据所述点读区域的位置坐标和页码信息，查询得到对应于所述点读区域目标的讲解数据；
    根据所述讲解数据进行播报，以对用户所指示的点读区域进行讲解。
  5. 根据权利要求4所述的绘本辅助阅读方法，其特征在于，所述根据所述讲解数据进行播报，以对用户所指示的点读区域进行讲解之前包括步骤：
    识别阅读所述绘本页面的用户的年龄段，并且检测所述用户的视线是否聚焦于电子设备的屏幕上；
    所述根据所述讲解数据进行播报，以对用户所指示的点读区域进行讲解包括步骤：
    若所述年龄段属于幼儿期，控制扬声器语音播放讲解文字内容，以及控制显示屏画面播放讲解图片内容；
    若所述年龄段属于学龄期，且所述用户的视线不在电子设备的屏幕上，控制扬声器语音播放讲解文字内容，且控制显示屏画面播放讲解图片内容；
    若所述年龄段属于学龄期，且所述用户的视线在电子设备的屏幕上，控制扬声器语音播放讲解文字内容，且控制显示屏画面播放讲解图片内容；
    其中，所述讲解数据包括讲解文字内容和讲解图片内容。
  6. 一种电子设备，其特征在于，包括：
    获取模块，用于控制摄像头拍摄反光镜投射区域内用户当前阅读的绘本页面，以获取对绘本页面拍摄的绘本图像；
    识别模块，用于从所述绘本图像中识别出指示物末端所指示的点读区域的位置坐标；
    控制模块，用于根据点读区域的位置坐标和所述绘本图像，获取并播报对应的讲解数据；
    其中，绘本平摊放置于所述反光镜的下方，所述反光镜安装于摄像头的上方。
  7. 根据权利要求6所述的电子设备，其特征在于，还包括：
    启动模块，用于获取启动点读功能的启动指令，根据所述启动指令启动摄像头。
  8. 根据权利要求6所述的电子设备，其特征在于，所述识别模块包括：
    提取单元，用于从所述绘本图像中提取出与预设指示物特征匹配的目标轮廓特征；
    计算单元，用于根据所述目标轮廓特征计算得到所述指示物末端的像素坐标；
    处理单元，用于若所述像素坐标持续不变的时长达到预设时长，确定用户具有点读学习意图，并且根据所述像素坐标计算得到所述点读区域的位置坐标。
  9. 根据权利要求6-8任一项所述的电子设备，其特征在于，所述控制模块包括：
    比对单元，用于将所述绘本图像与预设样本图像库进行比对，查询所述绘本图像对应的页码信息；
    查询单元，用于根据所述点读区域的位置坐标和页码信息，查询得到对应于所述点读区域目标的讲解数据；
    执行单元，用于根据所述讲解数据进行播报，以对用户所指示的点读区域进行讲解。
  10. 根据权利要求9所述的电子设备，其特征在于，所述控制模块还包括：
    识别单元，用于识别阅读所述绘本页面的用户的年龄段；
    检测单元，用于检测所述用户的视线是否聚焦于电子设备的屏幕上；
    所述执行单元，还用于若所述年龄段属于幼儿期，控制扬声器语音播放讲解文字内容，以及控制显示屏画面播放讲解图片内容；
    所述执行单元，还用于若所述年龄段属于学龄期，且所述用户的视线不在电子设备的屏幕上，控制扬声器语音播放讲解文字内容，且控制显示屏画面播放讲解图片内容；
    所述执行单元，还用于若所述年龄段属于学龄期，且所述用户的视线在电子设备的屏幕上，控制扬声器语音播放讲解文字内容，且控制显示屏画面播放讲解图片内容；
    其中，所述讲解数据包括讲解文字内容和讲解图片内容。
PCT/CN2021/103689 2021-06-30 2021-06-30 一种绘本辅助阅读方法和电子设备 WO2023272608A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/103689 WO2023272608A1 (zh) 2021-06-30 2021-06-30 一种绘本辅助阅读方法和电子设备

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/103689 WO2023272608A1 (zh) 2021-06-30 2021-06-30 一种绘本辅助阅读方法和电子设备

Publications (1)

Publication Number Publication Date
WO2023272608A1 true WO2023272608A1 (zh) 2023-01-05

Family

ID=84692117

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/103689 WO2023272608A1 (zh) 2021-06-30 2021-06-30 一种绘本辅助阅读方法和电子设备

Country Status (1)

Country Link
WO (1) WO2023272608A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117765779A (zh) * 2024-02-20 2024-03-26 厦门三读教育科技有限公司 基于孪生神经网络的儿童绘本智能化导读方法及系统
CN118711568A (zh) * 2024-08-26 2024-09-27 厦门三读教育科技有限公司 基于tts技术的幼儿绘本辅助阅读方法及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109756676A (zh) * 2019-01-16 2019-05-14 广东小天才科技有限公司 一种图像处理方法及电子设备
CN110597450A (zh) * 2019-09-16 2019-12-20 广东小天才科技有限公司 防误触识别方法、装置、点读设备及其点读识别方法
CN111078829A (zh) * 2019-06-09 2020-04-28 广东小天才科技有限公司 一种点读控制方法及系统
CN111077989A (zh) * 2019-05-27 2020-04-28 广东小天才科技有限公司 一种基于电子设备的屏幕控制方法及电子设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109756676A (zh) * 2019-01-16 2019-05-14 广东小天才科技有限公司 一种图像处理方法及电子设备
CN111077989A (zh) * 2019-05-27 2020-04-28 广东小天才科技有限公司 一种基于电子设备的屏幕控制方法及电子设备
CN111078829A (zh) * 2019-06-09 2020-04-28 广东小天才科技有限公司 一种点读控制方法及系统
CN110597450A (zh) * 2019-09-16 2019-12-20 广东小天才科技有限公司 防误触识别方法、装置、点读设备及其点读识别方法

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117765779A (zh) * 2024-02-20 2024-03-26 厦门三读教育科技有限公司 基于孪生神经网络的儿童绘本智能化导读方法及系统
CN117765779B (zh) * 2024-02-20 2024-04-30 厦门三读教育科技有限公司 基于孪生神经网络的儿童绘本智能化导读方法及系统
CN118711568A (zh) * 2024-08-26 2024-09-27 厦门三读教育科技有限公司 基于tts技术的幼儿绘本辅助阅读方法及系统

Similar Documents

Publication Publication Date Title
CN103763453B (zh) 一种图文采集识别装置
US10438080B2 (en) Handwriting recognition method and apparatus
US8538087B2 (en) Aiding device for reading a printed text
EP2591466B1 (en) Method and system for book reading enhancement
CN109409234B (zh) 一种协助学生进行问题定位学习的方法及系统
CN105590486A (zh) 一种基于机器视觉的座台式点读机及相关系统装置与方法
CN109376612B (zh) 一种基于手势协助定位学习的方法和系统
WO2023272608A1 (zh) 一种绘本辅助阅读方法和电子设备
CN109243215B (zh) 一种基于智能装置的交互方法、智能装置及系统
CN107742446A (zh) 图书阅读器
CN112013294B (zh) 一种智能听写台灯及其辅助听写的方法
Arakeri et al. Assistive technology for the visually impaired using computer vision
US20200013406A1 (en) Control method for human-computer interaction device, human-computer interaction device and human-computer interaction system
Stearns et al. The design and preliminary evaluation of a finger-mounted camera and feedback system to enable reading of printed text for the blind
CN111156441A (zh) 用于辅助学习的台灯、系统和方法
CN111753715B (zh) 点读场景下试题拍摄的方法、装置、电子设备和存储介质
CN110853424A (zh) 一种具有视觉识别的语音学习方法、装置及系统
CN113469035A (zh) 一种绘本辅助阅读方法和电子设备
JP2016045724A (ja) 電子機器
CN211289676U (zh) 用于辅助学习的台灯和系统
CN111077993B (zh) 一种学习场景的切换方法、电子设备及存储介质
US20110294522A1 (en) Character recognizing system and method for the same
CN109948600A (zh) 一种智能文字识别笔及文字识别系统
von Agris et al. Signum database: Video corpus for signer-independent continuous sign language recognition
CN116088675A (zh) 虚拟形象交互方法及相关装置、设备、系统和介质

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21947562

Country of ref document: EP

Kind code of ref document: A1