CN108665555A - A kind of autism interfering system incorporating real person's image - Google Patents
A kind of autism interfering system incorporating real person's image Download PDFInfo
- Publication number
- CN108665555A CN108665555A CN201810464235.4A CN201810464235A CN108665555A CN 108665555 A CN108665555 A CN 108665555A CN 201810464235 A CN201810464235 A CN 201810464235A CN 108665555 A CN108665555 A CN 108665555A
- Authority
- CN
- China
- Prior art keywords
- image
- personage
- autism
- plot
- point cloud
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
- G16H20/70—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mental therapies, e.g. psychological therapy or autogenous training
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/41—Medical
Landscapes
- Engineering & Computer Science (AREA)
- Computer Graphics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Computer Hardware Design (AREA)
- Psychology (AREA)
- Medical Informatics (AREA)
- Primary Health Care (AREA)
- Public Health (AREA)
- Architecture (AREA)
- General Health & Medical Sciences (AREA)
- Epidemiology (AREA)
- Social Psychology (AREA)
- Psychiatry (AREA)
- Hospice & Palliative Care (AREA)
- Developmental Disabilities (AREA)
- Child & Adolescent Psychology (AREA)
- Processing Or Creating Images (AREA)
Abstract
The present invention provides a kind of autism interfering systems incorporating real person's image, including：Figure image analogue unit, for simulating figure image according to including being carried out the 3D that simulation generates and real person is alike by personage's real pictures including intervening object；Social context simulation unit is simulated figure image, social scene and plot, is instructed to personage's driving unit sending action according to the growth requirement of plot for rendering；Personage's behavior driving unit drives simulation figure image to complete corresponding action for receiving action command according to action command.The present invention associates the image of other associated real persons by autism children and with it, it is dissolved among the fictitious situation of interfering system, help autism children are got involved in the vision and the mode of thinking of " first person " and " second person " in all directions among fictitious situation, it improves the self-consciousness of autism children and them is helped to establish the association of virtual world, finally improve their social skill.
Description
Technical field
The present invention relates to Education Technology fields, and in particular to a kind of autism intervention system incorporating real person's image
System.
Background technology
Autism-spectrum obstacle, abbreviation autism are a kind of nervous system disorders betiding children's early stage, not just
Normal social contact ability is one of its main symptom.According to the statistics of U.S.'s the Centers for Disease Control and Prevention, every 68 children
In i.e. have 1 autistic patients.There has been no the drugs for curing autism and other effective medical procedures, education of interference at present is
Treat the main path of childhood autism.In view of existing notable individual difference between autism children, American National research committee
Member can suggest at least needing weekly the one-to-one education of interference carried out 25 hours or more to autism children.This one-to-one religion
It is huge to the consumption of the educational resources such as qualified teachers, equipment, place to educate intervention.U.S. government's special throwing in autism rehabilitation every year
Enter to be up to more than 600 hundred million dollars, can not still ensure that all autism children obtain timely education of interference；Although China is to orphan
The rehabilitation pay attention to day by day of only disease children, but since starting evening, radix are big, current Specialized education resource is also lonely far from meeting
The rehabilitation demands of disease children.It is both domestic and external numerous to enable more autism children to obtain timely, effective education of interference
Scholar has carried out the research of computer assisted childhood autism interference method.
The childhood autism interfering system of current main-stream computer technology auxiliary can be roughly divided into class humanoid robot, virtual
Reality, multimedia animation/game three classes.This three classes interfering system respectively has advantage：Class humanoid robot provides a kind of approximation
The pattern of " people-people " interaction, and relatively simple facial expression and body language reduce fear and the row of autism children
Reprimand；Virtual reality technology has built not only safety but also close reality using its property immersed, interactivity and imagination as autism children
The environment of life；Multimedia animation/game then can easily create the various learning activities based on social context, and be easy to
It is deployed in the common computer and mobile device on the ground such as special education mechanism, family.However, there is also respective for this three classes interfering system
The shortcomings that：Humanoid robot is a kind of product integrating a variety of new and high technologies such as artificial intelligence, electronics, machinery, current
Intervention of the extremely limited interactive mode for autism children, and tens of thousands of to hundreds thousand of members one can only be provided under technical merit
The high cost of platform hinders it in the universal of special education mechanism；Virtual reality device and resources making exist and humanoid robot one
The problem of sample involves great expense, and need to wear the row that the invasive devices such as the helmet, data glove easily cause autism children
Reprimand；Although conventional Multi Media animation/game cost many cheaper than humanoid robot and virtual reality, can not provide strong
Feeling of immersion.
In addition, present invention applicant has found, existing interfering system is generally autism children not in virtual world
The social object in autism children and real-life is hinted obliquely at, i.e., is all in as autism children in the way of " third person "
Now intervene situation.This disadvantage makes autism children can not be established well in digital world to be contacted with real-life.Example
Such as, as soon as section with the cartoon that " Xiao Ming " is role, " Xiao Ming " wants to play with other children and politely ask in animation：" it may I ask me
It can together be played with you", " Xiao Ming " just giocoso plays with other children together quickly later.It is dynamic to play this section
After picture is seen to normal development children, oneself can be updated in animation by they in consciousness (replaces with " Xiao Ming " certainly
Oneself), it then can learn the interpersonal skill into animation and be used in real life；This section of cartoon is played to autism
After infant is seen, " Xiao Ming " still " Xiao Ming " will not be replaced with oneself by " Xiao Ming " very maximum probability in their eyes, can not be excited certainly
I realizes.This is because there is " mirror neuron system (MNS) " defect in autism children, it is autism not in interfering system
Children intuitively, visually hint obliquely at the social activity in autism children and real-life with " first person " or " second person "
Object, they are difficult to understand character relation and social context in interfering system, learn with can not be successfully to social skill, more talk
On not into real-life migration and it is extensive.
Invention content
In view of the drawbacks of the prior art, the present invention provides a kind of autism interfering system incorporating real person's image,
Purpose is, autism children is helped to get involved in void in all directions with the vision and the mode of thinking of " first person " and " second person "
Among quasi- situation, improves the self-consciousness of autism children and them is helped to establish the association of virtual world, finally improve them
Social skill.
A kind of autism interfering system incorporating real person's image, including：
Figure image analogue unit, for according to including being carried out simulation generation by personage's real pictures including intervening object
Figure image is simulated with the alike 3D of real person；
Social context simulation unit simulates figure image, social scene and plot, according to plot for rendering
Growth requirement to personage's driving unit sending action instruct；
Personage's behavior driving unit drives simulation figure image to complete phase for receiving action command according to action command
The action answered.
Further, the figure image analogue unit includes：
2D face -3D number of people conversion modules, for 2D human face photos to be transformed to 3D headforms；
Personage's concatenation module is used for 3D headforms and 3D character physical's model splicings, and then is rendered into complete mould
Anthropomorphic figure image；
Pretest module, simulates figure image for rendering, and whether personage's body can be accurately identified by intervention object with test
Part, if cannot accurately identify, start figure image adjustment module；
Figure image adjusts module, for being adjusted to simulating figural details, until piece identity can by by
Intervene object to accurately identify.
Further, the specific implementation of the 2D faces -3D number of people conversion modules is：
(S1011) position of face is detected on 2D human face photos；
(S1012) locating human face's characteristic point on the face detected；
(S1013) it according to the rotation of the human face characteristic point of positioning, scaling face, is aligned with standard faces；
(S1014) depth convolutional neural networks are used to complete the mapping between 2D faces and 3D point cloud, wherein depth convolution
The parameter of neural network trains to obtain by the 2D faces of magnanimity standard with 3D point cloud；
(S1015) texture rendering is carried out to 3D point cloud, generates 3D headforms.
Further, the 3D headforms indicate that wherein D is 3D shapes using two tuple ＜ D, C ＞, are one group
The coordinate value of 3D point cloud；C is color model, is the corresponding color value of each 3D points；
D is expressed as in 3D headforms：WhereinE is to pass through a large amount of 3D numbers of people sample 3D point clouds respectively
It is reconstruction parameter to count obtained mean value and eigenvectors matrix, λ.
Further, the training of the depth convolutional neural networks used in the 2D faces -3D number of people conversion module steps
Method is：
(S10141) prepare sample setWherein IjFor human face photo, DjFor IjCorresponding 3D point cloud data, M
For sample size；
(S10142) 3D point cloud of each sample is expressed as：It collects human face photo and corresponds to 3D point cloud
Reconstruction parameterE be the mean value counted respectively by a large amount of 3D numbers of people sample 3D point clouds and feature to
Moment matrix；
(S10143) deep neural network by facial recognition data collection pre-training is selected, and output layer is changed to
Export reconstruction parameter λ；
(S10144) it usesIterate the deep neural network of construction in fine tuning (S10143), when fine tuning
Loss function is：WhereinFor the reconstruction parameter of current depth neural network output.
Further, the point cloud for the 3D headforms that personage's concatenation module is generated using 3D number of people conversion modules is replaced
The point cloud for falling corresponding position on 3D character physical's models, completes the splicing of personage.
Further, the 3D character physicals model carries skeleton, to facilitate personage's behavior driving unit to drive personage's body
Body posture changes.
Further, the social context simulation unit includes：
Plot setup module for selecting or creating plot, and selects or creates scenes for plot；
Figure image input module, the orphan that the personage's list and personage that plot is related to for rendering are intervened with receiving
The social interaction relationship of only disease children inputs the real pictures of corresponding personage for figure image analogue unit according to personage's list
It generates 3D and simulates figure image；
Social context emulation module, scenes for plot and plot for rendering, according to plot growth requirement to
Personage's driving unit sending action instructs.
Further, the action command includes body posture instruction, facial expression instruction, visual angle instruction, sound instruction
In it is one or more.
The advantageous effects of the present invention are embodied in：
The present invention associates the image of other associated real persons by autism children and with it, is dissolved into interfering system
Among fictitious situation, autism children is helped to be intervened in all directions with the vision and the mode of thinking of " first person " and " second person "
To among fictitious situation, improves the self-consciousness of autism children and them is helped to establish the association of virtual world, it is final to improve
Their social skill.
Description of the drawings
Fig. 1 is a kind of autism interfering system preferred embodiment structure composition signal incorporating real person's image of the present invention
Figure；
Fig. 2 is the present inventor's figure image analogue unit preferred embodiment module composition schematic diagram.
Specific implementation mode
In order to make the purpose , technical scheme and advantage of the present invention be clearer, with reference to the accompanying drawings and embodiments, right
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.As long as in addition, technical characteristic involved in the various embodiments of the present invention described below
It does not constitute a conflict with each other and can be combined with each other.
Fig. 1 shows a preferred embodiment of the present invention.A kind of autism interfering system incorporating real person's image, packet
It includes：Figure image analogue unit 10, social context simulation unit 11, personage's behavior driving unit 12, wherein
Figure image analogue unit 10 according to including by intervene object including the progress of personage's real pictures-simulation generate with
The alike simulation figure image of real person.In a preferred manner, first by real person's facial photo be transformed to
Alike 3D headforms, then by 3D headforms and 3D character physical's model splicings be complete simulation figure image.
Scene, simulation personage and the story of social context is presented in social context simulation unit 11 in the form of animation or game etc.
Plot.
The driving simulation figure image of personage's behavior driving unit 12 sends out action behavior, and including but not limited to body posture becomes
Change, facial expression variation, visual angle change, sound variation etc..
Preferably, the figure image analogue unit includes：2D face -3D number of people conversion modules, for shining 2D faces
Piece is transformed to 3D headforms；Personage's concatenation module is used for 3D headforms and 3D character physical's model splicings, and then is rendered
Completely to simulate figure image；Whether pretest module, simulates figure image for rendering, can be accurate by intervention object with test
Really identification piece identity starts figure image adjustment module if cannot accurately identify；Figure image adjust module, for pair
It simulates figural details to be adjusted, until piece identity accurately identifies.
In the present embodiment, it in order to ensure the figural fidelity of simulation of the generation of figure image analogue unit 10, adopts
Use the high-resolution human face photo in front as the input of figure image analogue unit 10；Using the people of other postures and resolution ratio
Face can also be generated simulation figure image more true to nature by method provided by the invention.Preferably, the 2D faces -3D numbers of people become
Mold changing block 101 calculating step be：
(S1011) position of face is detected on 2D human face photos.In the present embodiment, added using Haar-like features
Hierarchical AdaBoost graders detect face.
(S1012) locating human face's characteristic point on the face detected.In the present embodiment, use condition returns random gloomy
Woods (Conditional Regression Forests) carrys out 21 characteristic points on locating human face.
(S1013) it according to the rotation of the human face characteristic point of positioning, scaling face, is aligned with standard faces；
(S1014) depth convolutional neural networks are used to complete the mapping between 2D faces and 3D point cloud, wherein depth convolution
Neural network trains to obtain by the 2D faces of magnanimity standard with 3D point cloud；
(S1015) texture rendering is carried out to 3D point cloud, generates 3D headforms.
In the present embodiment, 3D headforms are indicated using two tuple ＜ D, C ＞, and wherein D is the coordinate of one group of 3D point cloud
Value, C are the corresponding color value of each 3D points.For the ease of generating 3D headforms from 3D photos, by D in 3D headforms into one
Step is expressed as：WhereinE is the mean value counted respectively by a large amount of 3D numbers of people sample 3D point clouds and spy
Vector matrix is levied, λ is reconstruction parameter.
Preferably, the training of the depth convolutional neural networks used in 2D faces -3D numbers of people conversion module (S1014) step
Method is：
(S10141) prepare sample setWherein IjFor human face photo, DjFor IjCorresponding 3D point cloud data, M
For sample size.DjFace can be scanned to obtain by 3D scanners.50000 samples have been used in the present embodiment
This.
(S10142) 3D point cloud of each sample is expressed as：It collects human face photo and corresponds to 3D point cloud
Reconstruction parameter
(S10143) deep neural network by facial recognition data collection pre-training is selected, selected in the present embodiment
For VGG-Face (O.M.Parkhi, A.Vedaldi, A.Zisserman：Deep face recognition,
Proceedings of the British Machine Vision Conference, 2015), and output layer is changed to export
Reconstruction parameter λ.
(S10144) it usesIterate the deep neural network of construction in fine tuning (S10143), when fine tuning
Loss function is：WhereinFor the reconstruction parameter of current depth neural network output.
Preferably, the color model C in 3D headforms, can design one group of age, gender, different style color modelPreferably, step (S1015) can be fromA color model is selected to complete the rendering of head model.Face
Color model C is by the benefit manually set：(1) the slightly simple color model of texture can be set convenient for autism children's
Understand；(2) animation of the generation of social context simulation unit 11 or the style of game are conveniently adapted to, indisposed sense is avoided.
Pretest module tests the autism children for receiving to intervene before social context is presented, to ensure that it can be with
The simulation figure image that easily identification figure image analogue unit 10 generates.It is true to nature (i.e. and real character to simulate figure image
Appearance picture, identification are high) help to improve the substitution sense of autism children.In the present embodiment, using touch screen and voice dialogue
Mode autism children are tested, such as generate the simulation figure image for the autism children for receiving to intervene
Afterwards, which is mixed with 3 with other people cotemporary simulation figure image of gender, then voice
" which is * * (names for receiving the autism children intervened) here for enquirement”.If receiving the autism youngster intervened
Child can select correct answer before the deadline, then it is assumed that by once testing；3 times or more tests are continued through to recognize
True to nature for the simulation figure image of generation, otherwise figure image analogue unit 10 needs to adjust the simulation figure image generated.
Preferably, the 3D point cloud number on the head of 3D character physicals model is generated with 2D face -3D number of people conversion modules
The 3D point cloud number of 3D headforms is identical.
Preferably, the point cloud for the 3D headforms that personage's concatenation module is generated using 3D number of people conversion modules replaces
The point cloud of corresponding position, completes the splicing of personage on 3D character physical's models.The 3D character physicals model carries skeleton, with side
Person who happens to be on hand for an errand's object behavior driving unit driving character physical's posture changes.
Preferably, figure image adjustment module provides a series of shape of face and face 3D point cloud is candidate, for artificial selection
Replace corresponding shape of face and face on the 3D headforms that 2D face -3D number of people conversion modules generate.
Preferably, social context simulation unit includes：Plot setup module, for selecting or creating plot,
And select or create scenes for plot；Figure image input module, the personage's list and personage that plot is related to for rendering
With the social interaction relationship for the autism children for receiving to intervene, the real pictures of corresponding personage are inputted for people according to personage's list
Figure image analogue unit generates 3D and simulates figure image；Social context emulation module, scenes for plot and plot for rendering,
It is instructed to personage's driving unit sending action according to the growth requirement of plot.
The social interaction of the personage and the autism children for receiving to intervene in personage's list and list involved in situation
Relationship, including but not limited to set membership, mother-child relationship (MCR), teacher-student relationship, classmate's relationship, friends etc..It is lonely in order to reduce
The cognition pressure of disease children, personage and character relation involved in each social context should not be excessively complicated.In the present embodiment
Personage involved in each social context is generally 2~3 (including receiving the autism children intervened).Operating personnel's root
Prepare the facial photo of corresponding personage according to personage's list, input figure image analogue unit 10 is online or generates corresponding mould offline
Anthropomorphic figure image.
Preferably, personage's behavior driving unit 12 is appointed according to what different intervention objects and social context simulation unit 11 exported
Corresponding simulation figure image is read in business list.In the present embodiment, different intervention objects is identified using face recognition technology.
Example：
Below using one autism children with Expression Recognition obstacle of training as example, the present invention is done further
Explanation.It is diagnosed as autism when the infant five years old, three years old, cannot distinguish other people facial expressions substantially, it cannot be to teacher and family
The expression of people gives a response, and seldom has the variation of facial expression in daily life, can carry out of short duration expression in the eyes pair with people
Depending on, but often have eye dodge situation appearance.Using technology provided by the invention, the image of the infant interfering system has been incorporated into
In, 12 sections of cartoon making teaching animations have been made for glad, surprised, sad, angry four kinds of expressions, daily life situation has been melted
Enter into the intervention teaching of facial expression.Before receiving to intervene, the facial photo of the children is inputted, by figure image analogue unit
10 generate its 3D head model, and simulation figure image is generated with the 3D character physicals model splicing to prestore in system, after rendering, is deposited
Enter into interfering system.When receiving to intervene, the simulation figure image that system chooses the infant is dissolved into system, by personage's row
For behaviors such as 12 drive actions of driving unit, facial expressions, situation animation is generated by social context simulation unit 11.The infant exists
When receiving to intervene, it is oneself that can recognize in interfering system at a glance, to improve self meaning of " originally I should do so "
Know.After the intervention of 32 class hour, the human facial expression recognition ability of the infant is significantly improved.
In the present embodiment, due to needing to use deep neural network, figure image analogue unit 10 is deployed in equipped with GPU
Server on, personage's behavior driving unit 12, social context simulation unit 11 pass through network and figure image analogue unit 10
Communication.It can be appreciated that these units are deployed on same or different computing device, it is included in protection scope of the present invention
Within.
As it will be easily appreciated by one skilled in the art that within the spirit and principles in the present invention by unit of the present invention or
Module is split, is recombinated, and should be included within protection scope of the present invention.
The hardware of this example includes mobile phone, tablet computer, smart television, personal computer etc..
As it will be easily appreciated by one skilled in the art that the foregoing is merely illustrative of the preferred embodiments of the present invention, not to
The limitation present invention, all within the spirits and principles of the present invention made by all any modification, equivalent and improvement etc., should all include
Within protection scope of the present invention.
Claims (9)
1. a kind of autism interfering system incorporating real person's image, which is characterized in that including：
Figure image analogue unit, for according to including by personage's real pictures including intervening object carry out simulation generate with it is true
The alike 3D of real personage simulates figure image；
Social context simulation unit simulates figure image, social scene and plot, according to the hair of plot for rendering
Exhibition demand is instructed to personage's driving unit sending action；
Personage's behavior driving unit drives simulation figure image to complete corresponding for receiving action command according to action command
Action.
2. the autism interfering system according to claim 1 for incorporating real person's image, which is characterized in that the personage
Visual simulation unit includes：
2D face -3D number of people conversion modules, for 2D human face photos to be transformed to 3D headforms；
Personage's concatenation module is used for 3D headforms and 3D character physical's model splicings, and then is rendered into complete simulation people
Figure image；
Pretest module, simulates figure image for rendering, and whether piece identity can be accurately identified by intervention object with test, if
It cannot accurately identify, then start figure image adjustment module；
Figure image adjusts module, for being adjusted to simulating figural details, until piece identity can be intervened
Object accurately identifies.
3. the autism interfering system according to claim 1 or 2 for incorporating real person's image, which is characterized in that described
The specific implementation of 2D face -3D number of people conversion modules is：
(S1011) position of face is detected on 2D human face photos；
(S1012) locating human face's characteristic point on the face detected；
(S1013) it according to the rotation of the human face characteristic point of positioning, scaling face, is aligned with standard faces；
(S1014) depth convolutional neural networks are used to complete the mapping between 2D faces and 3D point cloud, wherein depth convolutional Neural
The parameter of network trains to obtain by the 2D faces of magnanimity standard with 3D point cloud；
(S1015) texture rendering is carried out to 3D point cloud, generates 3D headforms.
4. the autism interfering system according to claim 3 for incorporating real person's image, which is characterized in that the 3D people
Head model indicates that wherein D is 3D shapes using two tuple ＜ D, C ＞, is the coordinate value of one group of 3D point cloud；C is color mould
Type is the corresponding color value of each 3D points；
D is expressed as in 3D headforms：WhereinE is to be counted respectively by a large amount of 3D numbers of people sample 3D point clouds
Obtained mean value and eigenvectors matrix, λ are reconstruction parameter.
5. the autism interfering system according to claim 3 or 4 for incorporating real person's image, which is characterized in that described
The training method of the depth convolutional neural networks used in 2D face -3D number of people conversion module steps for：
(S10141) prepare sample setWherein IjFor human face photo, DjFor IjCorresponding 3D point cloud data, M are sample
This quantity；
(S10142) 3D point cloud of each sample is expressed as：Collect the reconstruction of human face photo and corresponding 3D point cloud
ParameterE is the mean value counted respectively by a large amount of 3D numbers of people sample 3D point clouds and feature vector square
Battle array；
(S10143) deep neural network by facial recognition data collection pre-training is selected, and output layer is changed to export
Reconstruction parameter λ；
(S10144) it usesIterate the deep neural network of construction in fine tuning (S10143), and when fine tuning loses letter
Number is：WhereinFor the reconstruction parameter of current depth neural network output.
6. the autism interfering system according to claim 2 for incorporating real person's image, which is characterized in that the personage
The point cloud for the 3D headforms that concatenation module is generated using 3D number of people conversion modules replaces and corresponds to position on 3D character physical's models
The point cloud set, completes the splicing of personage.
7. the autism interfering system of involvement real person's image according to claim 2 or 6, which is characterized in that described
3D character physical's models carry skeleton, to facilitate personage's behavior driving unit driving character physical's posture to change.
8. the autism interfering system according to claim 1 or 2 for incorporating real person's image, which is characterized in that described
Social context simulation unit includes：
Plot setup module for selecting or creating plot, and selects or creates scenes for plot；
Figure image input module, the autism that the personage's list and personage that plot is related to for rendering are intervened with receiving
The social interaction relationship of children inputs the real pictures of corresponding personage so that figure image analogue unit generates according to personage's list
3D simulates figure image；
Social context emulation module, scenes for plot and plot for rendering, according to the growth requirement of plot to personage
Driving unit sending action instructs.
9. the autism interfering system according to claim 8 for incorporating real person's image, which is characterized in that the action
Instruction includes one or more in body posture instruction, facial expression instruction, visual angle instruction, sound instruction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810464235.4A CN108665555A (en) | 2018-05-15 | 2018-05-15 | A kind of autism interfering system incorporating real person's image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810464235.4A CN108665555A (en) | 2018-05-15 | 2018-05-15 | A kind of autism interfering system incorporating real person's image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108665555A true CN108665555A (en) | 2018-10-16 |
Family
ID=63779733
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810464235.4A Pending CN108665555A (en) | 2018-05-15 | 2018-05-15 | A kind of autism interfering system incorporating real person's image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108665555A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109409274A (en) * | 2018-10-18 | 2019-03-01 | 广州云从人工智能技术有限公司 | A kind of facial image transform method being aligned based on face three-dimensional reconstruction and face |
| CN110838357A (en) * | 2019-11-19 | 2020-02-25 | 上海青鸿教育科技有限公司 | Attention holographic intelligent training system based on face recognition and dynamic capture |
| CN110916689A (en) * | 2019-11-29 | 2020-03-27 | 上海青鸿教育科技有限公司 | Cognitive and attention-strengthening intelligent evaluation training system and method for autism |
| CN110969705A (en) * | 2019-11-29 | 2020-04-07 | 上海青鸿教育科技有限公司 | Training system, method and medium for preschool mental enhancement combined with intelligent evaluation |
| CN111081371A (en) * | 2019-11-27 | 2020-04-28 | 昆山杜克大学 | Virtual reality-based early autism screening and evaluating system and method |
| CN113096805A (en) * | 2021-04-12 | 2021-07-09 | 华中师范大学 | Autism emotion cognition and intervention system |
| CN115870970A (en) * | 2022-08-24 | 2023-03-31 | 哈尔滨工业大学（深圳） | Picture exchange communication method and system based on interactive intervention robot |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102054291A (en) * | 2009-11-04 | 2011-05-11 | 厦门市美亚柏科信息股份有限公司 | Method and device for reconstructing three-dimensional face based on single face image |
| CN103430218A (en) * | 2011-03-21 | 2013-12-04 | 英特尔公司 | Method of augmented makeover with 3d face modeling and landmark alignment |
| CN103927747A (en) * | 2014-04-03 | 2014-07-16 | 北京航空航天大学 | Face matching space registration method based on human face biological characteristics |
| US20140267614A1 (en) * | 2013-03-15 | 2014-09-18 | Seiko Epson Corporation | 2D/3D Localization and Pose Estimation of Harness Cables Using A Configurable Structure Representation for Robot Operations |
| CN106067190A (en) * | 2016-05-27 | 2016-11-02 | 俞怡斐 | A kind of fast face threedimensional model based on single image generates and alternative approach |
| CN107680158A (en) * | 2017-11-01 | 2018-02-09 | 长沙学院 | A kind of three-dimensional facial reconstruction method based on convolutional neural networks model |
| CN107785061A (en) * | 2017-10-10 | 2018-03-09 | 东南大学 | Autism-spectrum disorder with children mood ability interfering system |
-
2018
- 2018-05-15 CN CN201810464235.4A patent/CN108665555A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102054291A (en) * | 2009-11-04 | 2011-05-11 | 厦门市美亚柏科信息股份有限公司 | Method and device for reconstructing three-dimensional face based on single face image |
| CN103430218A (en) * | 2011-03-21 | 2013-12-04 | 英特尔公司 | Method of augmented makeover with 3d face modeling and landmark alignment |
| US20140267614A1 (en) * | 2013-03-15 | 2014-09-18 | Seiko Epson Corporation | 2D/3D Localization and Pose Estimation of Harness Cables Using A Configurable Structure Representation for Robot Operations |
| CN103927747A (en) * | 2014-04-03 | 2014-07-16 | 北京航空航天大学 | Face matching space registration method based on human face biological characteristics |
| CN106067190A (en) * | 2016-05-27 | 2016-11-02 | 俞怡斐 | A kind of fast face threedimensional model based on single image generates and alternative approach |
| CN107785061A (en) * | 2017-10-10 | 2018-03-09 | 东南大学 | Autism-spectrum disorder with children mood ability interfering system |
| CN107680158A (en) * | 2017-11-01 | 2018-02-09 | 长沙学院 | A kind of three-dimensional facial reconstruction method based on convolutional neural networks model |
Non-Patent Citations (3)
| Title |
|---|
| ANH TUAN TRAN ET AL.: "Regressing Robust and Discriminative 3D Morphable Models with a Very Deep Neural Network", 《2017 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)》 * |
| NKIRUKA UZUEGBUNAM ET AL: "MEBook: Multimedia Social Greetings Intervention for Children with Autism Spectrum Disorders", 《IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES》 * |
| 张孟地: "基于Avatar技术的孤独症儿童面部表情识别的干预与实证研究", 《中国优秀硕士学位论文全文数据库 社会科学Ⅱ辑》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109409274A (en) * | 2018-10-18 | 2019-03-01 | 广州云从人工智能技术有限公司 | A kind of facial image transform method being aligned based on face three-dimensional reconstruction and face |
| CN110838357A (en) * | 2019-11-19 | 2020-02-25 | 上海青鸿教育科技有限公司 | Attention holographic intelligent training system based on face recognition and dynamic capture |
| CN111081371A (en) * | 2019-11-27 | 2020-04-28 | 昆山杜克大学 | Virtual reality-based early autism screening and evaluating system and method |
| CN110916689A (en) * | 2019-11-29 | 2020-03-27 | 上海青鸿教育科技有限公司 | Cognitive and attention-strengthening intelligent evaluation training system and method for autism |
| CN110969705A (en) * | 2019-11-29 | 2020-04-07 | 上海青鸿教育科技有限公司 | Training system, method and medium for preschool mental enhancement combined with intelligent evaluation |
| CN110969705B (en) * | 2019-11-29 | 2023-11-14 | 上海青鸿教育科技有限公司 | Training system, method and medium for intelligent evaluation of pre-school mental reinforcement |
| CN113096805A (en) * | 2021-04-12 | 2021-07-09 | 华中师范大学 | Autism emotion cognition and intervention system |
| CN113096805B (en) * | 2021-04-12 | 2024-02-13 | 华中师范大学 | Autism emotion cognition and intervention system |
| CN115870970A (en) * | 2022-08-24 | 2023-03-31 | 哈尔滨工业大学（深圳） | Picture exchange communication method and system based on interactive intervention robot |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108665555A (en) | A kind of autism interfering system incorporating real person's image | |
| Adamo-Villani et al. | SMILE: an immersive learning game for deaf and hearing children | |
| Mozumder et al. | Technological roadmap of the future trend of metaverse based on IoT, blockchain, and AI techniques in metaverse education | |
| Pearson | A machine for playing in: Exploring the videogame as a medium for architectural design | |
| CN106652608A (en) | Auxiliary teaching method in virtue of virtual reality and software research and development technologies | |
| Di Tore | Perception Of Space, Empathy And Cognitive Processes: Design Of A Video Game For The Measurement Of Perspective Taking Skills. | |
| El Ghoul et al. | Virtual reality for educating Sign Language using signing avatar: The future of creative learning for deaf students | |
| De Paolis et al. | Augmented Reality, Virtual Reality, and Computer Graphics: 4th International Conference, AVR 2017, Ugento, Italy, June 12-15, 2017, Proceedings, Part I | |
| Bajaj et al. | Design and development of digital humans in virtual exhibition space | |
| Wu et al. | Exploration of mathematics education by metaverse technology | |
| US20230162619A1 (en) | Systems and methods for accessible computer-user interactions | |
| Jacobson | Ancient architecture in virtual reality: does immersion really aid learning? | |
| Trampas | Extracting learning analytics from game based learning sessions generated by multimodal data sources. | |
| Velaora | Methods of teaching architecture in virtual immersion: Experimentations for the design studio | |
| Lindgren et al. | Inter-identity technologies for learning | |
| Lyk et al. | Creating a more Immersive and" VR-like" 360-Degree Video Experience-Development of an Immersive and Interactive Alcohol Resistance Training Tool | |
| De Paolis et al. | Augmented Reality, Virtual Reality, and Computer Graphics: 7th International Conference, AVR 2020, Lecce, Italy, September 7–10, 2020, Proceedings, Part I | |
| Liu et al. | Development of Web3D education platform suitable for schoolchild | |
| Crooks | Virtual Reality for Fashion Education | |
| Wang | Research on the influence of high and low spatial ability on VR panoramic video learning task | |
| Huang | The effects of visual realism on cognitive constructs, spatial memory and learning in virtual reality | |
| Colella et al. | The Implementation of a Mobile Game for Social Inclusion in Multicultural School Contexts | |
| Hagen | Virtual reality for remote collaborative learning in the context of the COVID-19 crisis | |
| Papke | Creating Project Contrast: a Video Game exploring Consciousness and Qualia | |
| Kastemaa | Recognizing compound facial expressions of virtual characters in augmented reality |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181016 |