CN115494971B - Electronic touch pen and input system thereof - Google Patents
Electronic touch pen and input system thereof Download PDFInfo
- Publication number
- CN115494971B CN115494971B CN202211281031.XA CN202211281031A CN115494971B CN 115494971 B CN115494971 B CN 115494971B CN 202211281031 A CN202211281031 A CN 202211281031A CN 115494971 B CN115494971 B CN 115494971B
- Authority
- CN
- China
- Prior art keywords
- module
- information
- gesture
- image
- pen
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 241000180587 Pinnidae Species 0.000 claims abstract description 21
- 230000005540 biological transmission Effects 0.000 claims abstract description 20
- 238000007781 pre-processing Methods 0.000 claims description 9
- 230000008859 change Effects 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 6
- 230000003068 static effect Effects 0.000 claims description 6
- 238000005452 bending Methods 0.000 claims description 3
- 238000011156 evaluation Methods 0.000 claims description 3
- 210000001145 finger joint Anatomy 0.000 claims description 3
- 239000003086 colorant Substances 0.000 claims 1
- 230000003993 interaction Effects 0.000 abstract description 14
- 238000000034 method Methods 0.000 description 16
- 230000008569 process Effects 0.000 description 16
- 230000006870 function Effects 0.000 description 8
- 230000006872 improvement Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 4
- 206010016256 fatigue Diseases 0.000 description 3
- 239000000835 fiber Substances 0.000 description 3
- 230000001149 cognitive effect Effects 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 208000019914 Mental Fatigue Diseases 0.000 description 1
- 241001422033 Thestylus Species 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000012905 input function Methods 0.000 description 1
- 230000015654 memory Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
- 230000003936 working memory Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0354—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
- G06F3/03545—Pens or stylus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/0414—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using force sensing means to determine a position
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/0416—Control or interface arrangements specially adapted for digitisers
- G06F3/04162—Control or interface arrangements specially adapted for digitisers for exchanging data with external devices, e.g. smart pens, via the digitiser sensing hardware
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
- Position Input By Displaying (AREA)
Abstract
The invention discloses an electronic stylus and an input system thereof, wherein an image acquisition module, a pressure sensing module, an adjusting module and a transmission module are arranged in a hybrid pen shell, when the electronic stylus is used, the image acquisition module acquires gesture information to determine operation information, the adjusting module adjusts the proper length of the electronic stylus and the angle of the image acquisition module according to the operation information and gestures, so that a user can conveniently operate the electronic stylus, the pressure sensing module can receive the pressure information to determine a touch area and line thickness, the transmission module transmits the pressure information and track information formed by a pen point in the touch area to electronic equipment and displays corresponding input information in the touch input module, and the space gestures and the touch are combined to realize quick input of various input types, so that the interaction flexibility is improved, the problem of low touch recognition accuracy caused by different hand structures is effectively solved, and the application range of the electronic stylus and the experience of the user are improved.
Description
Technical Field
The invention belongs to the technical field of electronics, and particularly relates to an electronic stylus and an input system thereof.
Background
At present, manually operated pen type interaction and multi-point touch interaction have the characteristics of visual and natural operation, are always considered as a very promising interaction mode, and are integrated with information input modes such as pen pressure sensing, pen holding gesture and the like of an electronic pen on the basis of a traditional physical pen direct mode, so that the input bandwidth of a pen channel is widened. The finger is directly utilized in the information input link without other additional input media, so that a user can directly obtain more tactile feedback in the interaction process, and the inherent attribute in the touch interaction brings more real experience to the user. In the daily man-machine interaction process, a user usually needs to process a plurality of complex work tasks and directly and repeatedly switch between different task types so as to meet specific operation requirements. Due to the limitation of input conditions, the operation tasks which can be realized in parallel are often forced to be completed in a sequential manner, so that the task completion time is prolonged, the fatigue of hands is caused, the operation performance of touch is influenced, the problems of finger fattening and shielding are solved, misoperation is often generated in the process of clicking or selecting small targets by fingers directly on touch equipment, and the experience of users is reduced.
Disclosure of Invention
In view of the above, the present invention provides an electronic stylus and an input system thereof that improve operation flexibility, recognition accuracy and expand application range, so as to solve the above technical problems.
In a first aspect, the present invention provides an electronic stylus for inputting information to an electronic device through a touch input module, the electronic stylus comprising:
A pen shell;
the pen point is arranged at the first end of the pen shell;
The image acquisition module is arranged at the second end of the pen shell and is used for acquiring gesture images and generating operation information of the electronic touch pen according to the gesture images, wherein the operation information comprises a writing mode, a drawing mode and a control mode;
The adjusting module is connected with the image acquisition module and used for generating adjusting information according to the operation information and the gesture image, wherein the adjusting information comprises the length of the electronic stylus and the angle of the image acquisition module;
the pressure sensing module is arranged in the pen point and is used for acquiring a plurality of pieces of pressure information generated by the pen point contacting the touch input module when the electronic touch pen executes the operation information according to the adjustment information, and the plurality of pieces of pressure information are respectively in one-to-one correspondence with at least two contact areas of the electronic touch pen contacting the touch input module;
the transmission module is arranged in the pen shell and is used for sending a plurality of pressure information to the electronic equipment, and the electronic equipment generates corresponding input information according to the track information of the operation information executed by the electronic touch pen in contact with the touch input module and the pressure information.
As a further improvement of the above technical solution, the pressure sensing module includes:
The setting unit is used for setting a first line corresponding to the pressure information when the change value of the pressure information received by the pressure sensing module is smaller than a preset pressure value when the operation information is in a writing mode; when the operation information is in a drawing mode and the variation value of the pressure information is larger than or equal to the preset pressure, the setting unit sets a second line corresponding to the pressure information, and the width of the second line is larger than that of the first line.
As a further improvement of the above technical solution, the pressure sensing module includes:
the conversion unit is connected with the setting unit and used for converting the pressure information into a pressure value and sending the pressure value to the setting unit;
And the power supply unit is connected with the conversion unit and is used for supplying power to each module in the pen shell.
As a further improvement of the above technical solution, the image acquisition module is configured to acquire a gesture image and generate operation information of the electronic stylus according to the gesture image, and includes:
acquiring the number of fingers and the positions of the palms in the acquired gesture image, and when the palms do not appear in the gesture image, enabling the operation information to be in a writing mode; when the number of the fingers is a single finger, the operation information is a control mode; when the number of fingers is plural, the operation information is a drawing mode.
As a further improvement of the above technical solution, the track information includes a track formed by moving or clicking a contact area of the pen tip on the touch input module, the input information includes an input type and a display color, and the input type includes at least one of text, a picture or a page.
The invention also provides an electronic stylus input system, which comprises an electronic stylus and electronic equipment which are in wireless connection, wherein the electronic equipment comprises a gesture image preprocessing module, a touch input module and a control module, the gesture image preprocessing module receives gestures acquired by an image acquisition module and transmitted by a transmission module, the gestures are filtered to obtain gesture images, the control module generates operation information according to the gesture images, the control module transmits the operation information to the touch input module to display and adjust the electronic stylus, the pressure sensing module receives pressure information generated by the contact of a pen point and the touch input module and transmits the pressure information to the electronic equipment through the transmission module, and the control module generates corresponding input information according to the track information and the pressure information.
As a further improvement of the above technical solution, the executing process of the gesture image preprocessing module includes:
the gray value of each point gray value on the neighborhood of each pixel point of the calculated image is preset as f (i, j), (i, j) as the pixel point on the image by adopting a mean value filtering algorithm, and the expression corresponding to the mean value filtering is as follows Wherein M is the value range of the image pixel point covered by the calculation template, n is the number of pixels contained in the neighborhood of the point (i, j) covered by the calculation template operator.
As a further improvement of the above technical solution, when the image acquisition module detects the gesture, a cartesian coordinate system is established, the origin of coordinates is at the central position of the control module, the x-axis and the z-axis of the coordinate system are on the horizontal plane of the touch input module, the x-axis is parallel to the long side and the z-axis is parallel to the short side of the touch input module, and the y-axis of the coordinate system is perpendicular to the plane of the touch input module and is upward in the positive direction;
each frame of image acquisition of the image acquisition module calculates and updates a data set according to the current image frame data, each frame of data comprises a data list, and when the image acquisition module detects a gesture, the control module distributes a first identifier to the current gesture;
If the image acquisition module does not acquire the gesture, the control module allocates a second identifier for the current gesture image, reads the hand attribute according to the first identifier and the second identifier, and determines gesture information.
As a further improvement of the technical scheme, the gesture information comprises static gestures and dynamic gestures, the static gestures adopt the combination of all finger joints and bending states of the hand to realize gesture recognition, and the dynamic gestures recognize different gestures through the spatial position change condition of the whole hand.
As a further improvement of the above technical solution, reading the hand attribute according to the first identifier and the second identifier and determining the gesture information includes:
Adopting a three-dimensional gesture tracking evaluation algorithm to track the average Euclidean distance between the endpoint of the gesture and the real key point in the gesture image, wherein the expression of the endpoint error is as follows Wherein the default single hand has 21 joint points, i is the joint label number, D is the sample number of the data set, s is the sample number, w and h are the width and height of the original image respectively, y si represents the tag realism value,Representing the predicted value;
the percentage of true keypoints is PCK, i.e., the percentage of true keypoints with Euclidean distance errors below a threshold, expressed as Wherein δ is a threshold, if the predicted value is less than the distance of the real value, the indication function δ is set to 1, otherwise, it is set to 0.
The invention provides an electronic stylus and an input system thereof, wherein an image acquisition module, a pressure sensing module, an adjusting module and a transmission module are arranged in a hybrid pen shell, when the electronic stylus is used, the image acquisition module acquires gesture information to determine operation information, the adjusting module adjusts the proper length of the electronic stylus and the angle of the image acquisition module according to the operation information and gestures, so that a user can conveniently operate the electronic stylus, the pressure sensing module can receive the pressure information to determine a touch area and line thickness, the transmission module transmits the pressure information and track information formed by a pen point in the touch area to electronic equipment and displays corresponding input information in the touch input module, and the space gestures and the touch are combined to realize quick input of various input types, so that the interaction flexibility is improved, the problem of low touch recognition accuracy caused by different hand structures is effectively solved, and the application range of the electronic stylus and the experience of the user are improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a block diagram of an electronic stylus provided by the present invention;
FIG. 2 is a block diagram of an electronic stylus input system provided by the present invention;
FIG. 3 is a block diagram of a pressure sensing module according to the present invention;
fig. 4 is a diagram illustrating a usage process of the electronic stylus according to the present invention.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the invention.
It will be understood that when an element is referred to as being "fixed to" another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may also be present. In contrast, when an element is referred to as being "directly on" another element, there are no intervening elements present. The terms "vertical," "horizontal," "left," "right," and the like are used herein for illustrative purposes only.
In the present invention, unless explicitly specified and limited otherwise, the terms "mounted," "connected," "secured," and the like are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communicated with the inside of two elements or the interaction relationship of the two elements. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.
Referring to fig. 1, the present invention provides an electronic stylus for inputting information to an electronic device through a touch input module, the electronic stylus comprising:
A pen shell;
the pen point is arranged at the first end of the pen shell;
The image acquisition module is arranged at the second end of the pen shell and is used for acquiring gesture images and generating operation information of the electronic touch pen according to the gesture images, wherein the operation information comprises a writing mode, a drawing mode and a control mode;
The adjusting module is connected with the image acquisition module and used for generating adjusting information according to the operation information and the gesture image, wherein the adjusting information comprises the length of the electronic stylus and the angle of the image acquisition module;
the pressure sensing module is arranged in the pen point and is used for acquiring a plurality of pieces of pressure information generated by the pen point contacting the touch input module when the electronic touch pen executes the operation information according to the adjustment information, and the plurality of pieces of pressure information are respectively in one-to-one correspondence with at least two contact areas of the electronic touch pen contacting the touch input module;
the transmission module is arranged in the pen shell and is used for sending a plurality of pressure information to the electronic equipment, and the electronic equipment generates corresponding input information according to the track information of the operation information executed by the electronic touch pen in contact with the touch input module and the pressure information.
In this embodiment, the pressure sensing module includes a setting unit, a converting unit, and a power supply unit, where when the operation information is in a writing mode and a variation value of the pressure information received by the pressure sensing module is smaller than a preset pressure value, the setting unit sets a first line corresponding to the pressure information; when the operation information is in a drawing mode and the variation value of the pressure information is larger than or equal to the preset pressure, the setting unit sets a second line corresponding to the pressure information, and the width of the second line is larger than that of the first line; the conversion unit is connected with the setting unit and is used for converting the pressure information into a pressure value and sending the pressure value to the setting unit; the power supply unit is connected with the conversion unit and is used for supplying power to each module in the pen shell. The pressure sensing module can comprise at least two pressure fiber bodies, the pressure fiber bodies can deform according to the received pressure, the deformation degree of the pressure fiber bodies corresponds to the pressure, and the pressure sensing module obtains at least two pressure information according to at least two pressure sensors. The image acquisition module can be installed on the top end of the pen shell, the pressure sensing module is arranged at one end close to the pen point, the image acquisition module can acquire gestures of a user so as to set an operation mode of the electronic stylus, and the electronic stylus can be used for writing, drawing or controlling functions such as a mouse.
Referring to fig. 2, it should be noted that, the image acquisition module is a miniature camera, the miniature camera is disposed at the tail end of the pen shell of the pen holder in a structure embedding manner, two ends of the miniature camera are connected to two sides of the pen holder through rotating shafts, the rotating shafts are connected with the transmission motor through two meshing gears, the angle of the camera is adjusted through controlling and displaying the motor, and the electronic stylus can be properly adjusted according to the use habit or operation mode of a user. The transmission module can be a WeChat controller or a wireless communication function such as Bluetooth, and can display the pressure information received by the pressure sensing module and the track formed by the pen point on the touch input module, so that quick and convenient interaction is realized. The dynamic space gesture is realized in a waving mode, the position data of the palm center is recorded once after the frame data of fixed times are refreshed in advance, the current dynamic gesture type is determined through the change of the palm position in the two frame data, the dynamic gesture is recorded once, the change of the drawing color is realized through waving the palm left and right, and the change of the brush color is triggered once by the dynamic gesture to the left or right each time.
It should be understood that when the user uses the electronic stylus, the operation information can be set according to the gesture recognition of the image acquisition module, so that the electronic stylus is applicable to different scenes, can write or draw, can also be simply controlled, and is convenient to carry. The touch input module is a touch screen, and the finger easily shields the cursor due to the physiological structure of the finger, so that the relative position between the cursor and the base line cannot be observed in the drawing process, the fingertip position is finely adjusted, the drawing offset is slightly larger than the pen input, and meanwhile, the drawing time is slightly shorter than the pen input. In the control mode, the long-distance continuous sliding can cause uncomfortable feeling of the contact surface of the finger and influence the active experience of operation due to friction force between the finger and the screen, the electronic touch pen is suitable for accurate operation due to combination of space gestures and touch, and hand uncomfortable feeling caused by long-distance touch sliding operation and long-time continuous space gesture operation and influence on operation performance can be avoided.
Optionally, the image acquisition module is configured to acquire a gesture image and generate operation information of the electronic stylus according to the gesture image, and includes:
acquiring the number of fingers and the positions of the palms in the acquired gesture image, and when the palms do not appear in the gesture image, enabling the operation information to be in a writing mode; when the number of the fingers is a single finger, the operation information is a control mode; when the number of fingers is plural, the operation information is a drawing mode.
In this embodiment, the track information includes a track formed by moving or clicking a contact area of the pen tip on the touch input module, the input information includes an input type and a display color, and the input type includes at least one of text, a picture or a page. The electronic stylus is single-point input, the input bandwidth is relatively high, the combination of a plurality of finger positions can express rich semantic information, the palm position can be used as a reference personally discuss to determine gesture specific information, in the related work of drawing or design, the electronic stylus mostly needs to be used for pen input by an interactive digital screen or a digital panel, drawing, clicking or color-block selection of a pen brush is carried out on the digital screen, a folding menu is clicked and the thickness of the painting brush is selected, and the operation tasks mainly comprise drawing tasks and command operation tasks. The user needs to move the input position of the pen, the user switches between the drawing area and the command operation area for a plurality of times, the command operation in the drawing process breaks the current drawing task, when the drawing task with higher accuracy requirement is carried out, the user needs to accurately adjust the positions of the hand and the pen for a plurality of times before returning to the drawing state after the drawing process of the user is broken by the command operation, thus causing serious waste in time and reducing the input efficiency. If multiple command operations, such as changing the color of a stroke, are required during the drawing process, the adjustment process before each switch back to the drawing task will add up to a kind of fatigue that causes fatigue to the upper arm and the hand, and also causes mental fatigue to the user, thereby affecting the ability of the drawing operation and the command operation.
When the operation information is in a writing mode, the proper length is adjusted according to the hand structure of a user, when the operation information is in a drawing mode, the angle of a camera can be adjusted so as to conveniently and rapidly capture gesture images, interface switching or color selection and the like are realized, when the operation information is in a control mode, the length of an electronic stylus can be adjusted so as to conveniently click or slide to replace the function of a mouse, the writing can replace the simple input function of a keyboard, when the electronic stylus is actually used, the user can select one hand to operate the electronic stylus, and can operate the electronic stylus with two hands, the operation hands are distributed according to the accuracy degree of operation and the cognitive load, the cognitive load is related with the working memory, and the brain needs to mobilize resources for short-time memory storage. The information processing and the action control have completed the corresponding tasks. Under the touch operation without limiting the area, if the touch operation and the pen operation are combined on the right hand, the pen point moves due to the operation of the touch instruction gesture, and the gesture of holding the pen can be readjusted by a user for the touch operation of the right hand, so that the use convenience of the electronic touch pen is improved.
Referring to fig. 2, the invention further provides an electronic stylus input system, which comprises an electronic stylus and electronic equipment which are connected in a wireless manner, wherein the electronic equipment comprises a gesture image preprocessing module, a touch input module and a control module, the gesture image preprocessing module receives gestures acquired by an image acquisition module and transmitted by a transmission module, the gestures are filtered to obtain gesture images, the control module generates operation information according to the gesture images, the control module transmits the operation information to the touch input module to display and adjust the electronic stylus, the pressure sensing module receives pressure information generated by the contact of a pen point and the touch input module and transmits the pressure information to the electronic equipment through the transmission module, and the control module generates corresponding input information according to the track information and the pressure information.
In this embodiment, the executing process of the gesture image preprocessing module includes: the gray value of each point gray value on the neighborhood of each pixel point of the calculated image is preset as f (i, j), (i, j) as the pixel point on the image by adopting a mean value filtering algorithm, and the expression corresponding to the mean value filtering is as follows Wherein M is the value range of the image pixel point covered by the calculation template, n is the number of pixels contained in the neighborhood of the point (i, j) covered by the calculation template operator. The using process of the electronic touch pen comprises the following steps:
S1: acquiring a gesture image and generating operation information of the electronic stylus according to the gesture image, wherein the operation information comprises a writing mode, a drawing mode and a control mode;
S2: generating adjustment information according to the operation information and the gesture image, wherein the adjustment information comprises the length of the electronic stylus and the angle of the image acquisition module;
S3: when the electronic stylus executes the operation information according to the adjustment information, acquiring a plurality of pressure information generated by the nib contacting the touch input module, wherein the pressure information corresponds to at least two contact areas of the electronic stylus contacting the touch input module one by one;
S4: and sending a plurality of pieces of pressure information to the electronic equipment, and generating corresponding input information by the electronic equipment according to the track information of the operation information executed by the electronic stylus touching the touch input module and the plurality of pieces of pressure information.
It should be noted that, for a user who inputs a dominant hand as a right hand, the dominant hand input by the pen is the right hand, and the touch and space gestures may be performed using the left hand or the right hand. Taking an electronic touch pen as a main input channel, ensuring that a pen holding hand is the right hand of a mobile phone for a user, and realizing mixed input by matching with a left hand operation touch or space gesture on the basis of realizing accurate input by using the right hand holding pen, wherein the electronic touch pen comprises two distribution modes: the electronic touch pen can be connected with the electronic equipment through Bluetooth or other wireless modes, and operation keys are further arranged on the electronic touch pen, so that a mouse function can be better replaced, operation hands can be distributed according to the moving distance of the pen point, when the electronic equipment can clearly distinguish touch and pen input behaviors, the position of touch operation can be not limited, and better input can be realized by utilizing the touch gesture. When the touch control and the pen are mixed for input, the input attribute of the pen is controlled by the touch control, the area of the operation space is the operation surface of the interactive digital screen, namely, a two-dimensional plane with a fixed size, the current operation plane is divided, the number of the input operable attributes can be expanded, the electronic equipment can store the interaction result which is written or controlled by the user for the first time, and the interaction result can be pushed when the user inputs the interaction result next time. The user can also switch pages or fast play pages of the electronic equipment through the electronic stylus, so that the application range of the electronic stylus is enlarged.
Optionally, when the image acquisition module detects the gesture, a cartesian coordinate system is established, the origin of the coordinates is at the central position of the control module, the x-axis and the z-axis of the coordinate system are on the horizontal plane of the touch input module, the x-axis is parallel to the long side of the touch input module, the z-axis is parallel to the short side, and the y-axis of the coordinate system is perpendicular to the plane of the touch input module and is in the positive direction upwards;
each frame of image acquisition of the image acquisition module calculates and updates a data set according to the current image frame data, each frame of data comprises a data list, and when the image acquisition module detects a gesture, the control module distributes a first identifier to the current gesture;
If the image acquisition module does not acquire the gesture, the control module allocates a second identifier for the current gesture image, reads the hand attribute according to the first identifier and the second identifier, and determines gesture information.
In this embodiment, the gesture information includes a static gesture and a dynamic gesture, where the static gesture uses a combination of each finger joint and a bending state of the hand to identify the gesture, and the dynamic gesture identifies different gestures according to a spatial position change condition of the whole hand. Reading the hand attribute according to the first identifier and the second identifier and determining gesture information, including: adopting a three-dimensional gesture tracking evaluation algorithm to track the average Euclidean distance between the endpoint of the gesture and the real key point in the gesture image, wherein the expression of the endpoint error is as followsWherein the default single hand has 21 joint points, i is the joint label number, D is the sample number of the data set, s is the sample number, w and h are the width and height of the original image respectively, y si represents the tag realism value,Representing the predicted value; the percentage of true keypoints is PCK, i.e., the percentage of true keypoints with Euclidean distance errors below a threshold, expressed asWherein δ is a threshold, if the predicted value is less than the distance of the real value, the indication function δ is set to 1, otherwise, it is set to 0.
In another possible embodiment, the electronic stylus further includes a microprocessor, a wireless module and a storage unit, wherein the microprocessor mainly completes clock and instruction control of the flash memory, the storage unit is used for storing all static information and dynamic process outage files, and the wireless module is used for achieving real-time wireless transmission function by being connected with the electronic device. The electronic touch pen storage process comprises the following steps: firstly, information of the electronic equipment receives the information through a data line or receives a wireless carrier signal through a wireless antenna to a wireless module to finish adjustment, then the information is transmitted to a memory card interface control chip, so that normal reading and writing of a clock and data are controlled, and the control chip stores an information file into a storage unit of an electronic touch pen through a storage interface. The transmission process of the touch pen is as follows: inserting a touch control device to be directly connected with the electronic equipment, and transmitting information between the touch control pen and the electronic equipment in a moving or copying and pasting mode; or the touch pen is operated on a touch input module of the electronic equipment, namely a screen to control the touch pen to send or receive data, when the touch pen finishes the gesture operation on the screen, the control module of the electronic equipment judges whether to send or receive instructions, when the touch pen sends, the control instruction from the electronic equipment is needed to be received firstly, and then an information document needed by a user in the touch pen is sent to the electronic equipment; upon receipt, the user may transmit the editing complete text, picture, or application process selection to the stylus.
According to the electronic stylus input system provided by the invention, the image acquisition module, the pressure sensing module, the adjusting module and the transmission module are arranged in the miscellaneous pen shell, when the electronic stylus is used, the image acquisition module acquires gesture information to determine operation information, the adjusting module adjusts the proper length of the electronic stylus and the angle of the image acquisition module according to the operation information and gestures, so that a user can conveniently operate the electronic stylus, the pressure sensing module can receive the pressure information to determine a touch area and line thickness, the transmission module transmits the pressure information and track information formed by a pen point in the touch area to electronic equipment and displays corresponding input information in the touch input module, the spatial gestures and the touch are combined to realize quick input of various input types, the interaction flexibility is improved, the problem of low touch recognition accuracy caused by different hand structures is effectively solved, and the application range of the electronic stylus and the experience of the user are improved.
Any particular values in all examples shown and described herein are to be construed as merely illustrative and not a limitation, and thus other examples of exemplary embodiments may have different values.
It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures.
The above examples merely represent a few embodiments of the present invention, which are described in more detail and are not to be construed as limiting the scope of the present invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention.
Claims (7)
1. An electronic stylus for inputting information to an electronic device through a touch input module, the electronic stylus comprising:
A pen shell;
the pen point is arranged at the first end of the pen shell;
The image acquisition module is arranged at the second end of the pen shell and is used for acquiring gesture images and generating operation information of the electronic touch pen according to the gesture images, wherein the operation information comprises a writing mode, a drawing mode and a control mode;
the adjusting module is connected with the image acquisition module and used for generating adjusting information according to the operation information and the gesture image, and the adjusting information is used for adjusting the length of the electronic stylus and the angle of the image acquisition module;
the pressure sensing module is arranged in the pen point and is used for acquiring a plurality of pieces of pressure information generated by the pen point contacting the touch input module when the electronic touch pen executes the operation information according to the adjustment information, and the plurality of pieces of pressure information are respectively in one-to-one correspondence with at least two contact areas of the electronic touch pen contacting the touch input module;
The transmission module is arranged in the pen shell and is used for sending a plurality of pressure information to the electronic equipment, and the electronic equipment generates corresponding input information according to the track information of the operation information executed by the electronic stylus contacting the touch input module and the plurality of pressure information;
The setting unit is used for setting a first line corresponding to the pressure information when the change value of the pressure information received by the pressure sensing module is smaller than a preset pressure value when the operation information is in a writing mode; when the operation information is in a drawing mode and the variation value of the pressure information is larger than or equal to the preset pressure, the setting unit sets a second line corresponding to the pressure information, and the width of the second line is larger than that of the first line; the pressure sensing module includes: the conversion unit is connected with the setting unit and used for converting the pressure information into a pressure value and sending the pressure value to the setting unit;
the power supply unit is connected with the conversion unit and is used for supplying power to each module in the pen shell;
The track information comprises a track formed by moving or clicking a pen point in a contact area on the touch input module, the input information comprises an input type and display colors, and the input type comprises at least one of texts, pictures or pages; the image acquisition module is a miniature camera which is arranged at the tail end of a pen shell of the pen holder in a structure embedding mode, two ends of the miniature camera are connected to two sides of the pen holder through rotating shafts, the rotating shafts are connected with a transmission motor through two meshing gears, and the angle of the camera is adjusted through control and display of the motor.
2. The electronic stylus of claim 1, wherein the image acquisition module is configured to acquire a gesture image and generate operation information of the electronic stylus according to the gesture image, comprising:
acquiring the number of fingers and the positions of the palms in the acquired gesture image, and when the palms do not appear in the gesture image, enabling the operation information to be in a writing mode; when the number of the fingers is a single finger, the operation information is a control mode; when the number of fingers is plural, the operation information is a drawing mode.
3. An electronic stylus input system according to any one of claims 1-2, comprising an electronic stylus and an electronic device that are connected wirelessly, wherein the electronic device comprises a gesture image preprocessing module, a touch input module and a control module, the gesture image preprocessing module receives gestures acquired by an image acquisition module and transmitted by a transmission module, the gestures are filtered to obtain gesture images, the control module generates operation information according to the gesture images, the control module transmits the operation information to the touch input module to display and adjust the electronic stylus, the pressure sensing module receives pressure information generated by a pen point in contact with the touch input module and transmits the pressure information to the electronic device through the transmission module, and the control module generates corresponding input information according to the track information and the pressure information.
4. The electronic stylus input system of claim 3, wherein the gesture image preprocessing module is configured to perform the steps of:
Calculating gray values of all points on the neighborhood of each pixel point of the image by adopting a mean value filtering algorithm, presetting an image as f (i, j), (i, j) as the pixel point on the image, and adopting a mean value filtering corresponding expression as follows Wherein M is the value range of the image pixel point covered by the calculation template, n is the number of pixels contained in the neighborhood of the point (i, j) covered by the calculation template operator.
5. The electronic stylus input system of claim 3, wherein when the image acquisition module detects the gesture, a cartesian coordinate system is established, an origin of coordinates is at a central position of the control module, an x-axis and a z-axis of the coordinate system are on a horizontal plane of the touch input module, the x-axis is parallel to a long side of the touch input module, the z-axis is parallel to a short side of the touch input module, and a y-axis of the coordinate system is perpendicular to a plane of the touch input module and faces upwards in a positive direction;
the image acquisition module calculates each acquired frame of image to obtain current image frame data and updates a data set corresponding to the image frame data, each frame of data comprises a data list, and when the image acquisition module detects a gesture, the control module distributes a first identifier to the current gesture;
If the image acquisition module does not acquire the gesture, the control module allocates a second identifier for the current gesture image, reads the hand attribute according to the first identifier and the second identifier, and determines gesture information.
6. The electronic stylus input system of claim 5, wherein the gesture information comprises static gestures that employ a combination of individual finger joints and bending states of the hand to achieve gesture recognition, and dynamic gestures that recognize different gestures through spatial position changes of the entire hand.
7. The electronic stylus input system of claim 5, wherein reading the hand attribute and determining the gesture information based on the first identifier and the second identifier comprises:
Adopting a three-dimensional gesture tracking evaluation algorithm to track the average Euclidean distance between the endpoint of the gesture and the real key point in the gesture image, wherein the expression of the endpoint error is as follows Wherein the default single hand has 21 joint points, i is the joint label number, D is the sample number of the data set, s is the sample number, w and h are the width and height of the original image respectively, y si represents the tag realism value,Representing the predicted value;
the percentage of true keypoints is PCK, i.e., the percentage of true keypoints with Euclidean distance errors below a threshold, expressed as Wherein δ is a threshold, if the predicted value is less than the distance of the real value, the indication function δ is set to 1, otherwise, it is set to 0.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211281031.XA CN115494971B (en) | 2022-10-19 | 2022-10-19 | Electronic touch pen and input system thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211281031.XA CN115494971B (en) | 2022-10-19 | 2022-10-19 | Electronic touch pen and input system thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115494971A CN115494971A (en) | 2022-12-20 |
| CN115494971B true CN115494971B (en) | 2024-11-05 |
Family
ID=84474829
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211281031.XA Active CN115494971B (en) | 2022-10-19 | 2022-10-19 | Electronic touch pen and input system thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115494971B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118210390B (en) * | 2024-04-03 | 2024-10-18 | 东莞市意志强塑胶五金有限公司 | Touch monitoring method and system of capacitive stylus |
| CN118394229A (en) * | 2024-04-22 | 2024-07-26 | 深圳市左木科技有限公司 | Intelligent touch pen interaction method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105278704A (en) * | 2014-07-17 | 2016-01-27 | 致伸科技股份有限公司 | Stylus |
| CN109960404A (en) * | 2019-02-15 | 2019-07-02 | 联想(北京)有限公司 | A kind of data processing method and device |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9575573B2 (en) * | 2014-12-18 | 2017-02-21 | Apple Inc. | Stylus with touch sensor |
| KR102543380B1 (en) * | 2015-04-09 | 2023-06-14 | 가부시키가이샤 와코무 | Method, system, sensor controller, and active stylus using active stylus and sensor controller |
| KR20160148959A (en) * | 2015-06-17 | 2016-12-27 | 엘지전자 주식회사 | Mobile terminal and method for controlling the same |
| US10146124B2 (en) * | 2016-02-23 | 2018-12-04 | Xtal, Inc. | Full chip lithographic mask generation |
| CN213545237U (en) * | 2020-12-15 | 2021-06-25 | 维沃移动通信有限公司 | Touch control pen and electronic equipment |
-
2022
- 2022-10-19 CN CN202211281031.XA patent/CN115494971B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105278704A (en) * | 2014-07-17 | 2016-01-27 | 致伸科技股份有限公司 | Stylus |
| CN109960404A (en) * | 2019-02-15 | 2019-07-02 | 联想(北京)有限公司 | A kind of data processing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115494971A (en) | 2022-12-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115494971B (en) | Electronic touch pen and input system thereof | |
| US8604364B2 (en) | Sensors, algorithms and applications for a high dimensional touchpad | |
| US20060119588A1 (en) | Apparatus and method of processing information input using a touchpad | |
| US20070110287A1 (en) | Remote input method using fingerprint recognition sensor | |
| KR20130088104A (en) | Mobile apparatus and method for providing touch-free interface | |
| CN105824531A (en) | Method and device for adjusting numbers | |
| US20070177806A1 (en) | System, device, method and computer program product for using a mobile camera for controlling a computer | |
| KR20160097410A (en) | Method of providing touchless input interface based on gesture recognition and the apparatus applied thereto | |
| CN114816135B (en) | Cross-device drawing system | |
| CN114816088A (en) | Online teaching method, electronic equipment and communication system | |
| CN210573714U (en) | Erasing device of electronic whiteboard | |
| US20070216653A1 (en) | Pointing method and system for quickly moving an object to a designated area | |
| CN100394373C (en) | Application execution method and apparatus employing pointing device | |
| JP5165661B2 (en) | Control device, control method, control program, and recording medium | |
| WO2023024536A1 (en) | Drawing method and apparatus, and computer device and storage medium | |
| CN112565844B (en) | Video communication method and device and electronic equipment | |
| CN108780383A (en) | Based on second input selection the first numeral input behavior | |
| CN103092381A (en) | Electronic device with multiple touch interfaces and method for controlling three-dimensional image | |
| CN105389031A (en) | Man-machine interaction method and device | |
| Signer | Pen-based interaction | |
| CN108803898B (en) | Combined intelligent wireless finger input device, system and control method thereof | |
| JP7273098B2 (en) | Method of operation | |
| JP6603325B2 (en) | Input terminal device | |
| US20230333731A1 (en) | Display apparatus, display system, display method, and non-transitory recording medium | |
| JP5899819B2 (en) | Learning system and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |