DE102014119419A1 - Display a transition from gesture-based input to touch surfaces - Google Patents

Abstract

A system and method for indicating a transition from a gesture-based input to a touch surface is provided herein. The system includes a detection monitor coupled to a recognition device to detect the transition; and an alarm transmitter to alert an output device based on the detection. The gesture-based input is a contactless interface associated with an electronic system, and a touch surface is a contact interface associated with the electronic system.

Description

priority

This patent application claims priority over provisional pending US Patent Application No. 61 / 920,993, filed December 26, 2013, entitled "Indicating a Transition from Gesture Based Inputs to Touch Surfaces". This patent application contains the entire detailed description of US Patent Application No. 61 / 920,993.

General state of the art

Interfaces are becoming ever more complex and versatile. Providing various ways to interact with a system is becoming more commonplace. Accordingly, a user of a system may selectively select a first or second technique for interacting with the system. In certain cases, the transition between a first and second technique may require a toggle switch. In other cases, both the first and second techniques may be simultaneously available to the user.

One such input technique is gesture-based input. Gesture-based input allows movement of a trigger stimulus, such as a body part (usually the hand), to be detected, and a command is triggered based on the detected gesture or gesture. The gesture-based input does not require the user to contact a touch surface, pad, or device.

The gesture is recorded by a video camera or a motion detector. Accordingly, the video camera picks up the motion, equalizes the motion with a stored command center (i.e., a processor and memory device), and translates the motion into action.

Another such input technique is a touch surface. The touch interface allows a user to interact with a touch screen or touchpad to control a system. The system may be any type of electronics-based system, for example, components installed in or around the control console of a vehicle. The touch screen or the touch surface may use, for example, capacitive touch technology or resistive touch technology.

In certain cases, the gesture-based input systems and the touch-based input systems may be integrated together. Thus, a user operating a system can selectively interact with either a touch screen or an area provided for recognizing gestures.

Summary

A system and method for indicating a transition from a gesture-based input to a touch surface is provided herein. The system includes a detection monitor coupled to a recognition device to detect the transition; and an alarm transmitter to alert an output device based on the detection. The gesture-based input is a contactless interface associated with an electronic system, and a touch surface is a contact interface associated with the electronic system.

Description of the drawings

The detailed description refers to the following drawings in which like numerals refer to like elements. Show it:

1 a block diagram illustrating an exemplary computer.

2 an example of a system for displaying a transition from gesture-based inputs to a touch surface.

3 an example of a method for displaying a transition from gesture-based inputs to a touch surface.

4 an example of the system 2 , integrated in a vehicle.

Detailed description

The invention will now be described more fully with reference to the accompanying drawings, in which exemplary embodiments of the invention are shown. However, this invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Instead, these exemplary embodiments are provided so that this disclosure will be thorough and will convey the full scope of the invention to those skilled in the art. It is understood that "at least one of" for the purposes of this disclosure is interpreted as referring to any combination of the elements listed by the corresponding designation, including a combination of several of the listed elements. For example, "at least one of X, Y and Z "is to be construed as only X, only Y, only Z or any combination of two or more elements X, Y and Z (eg XYZ, XZ, YZ, X). It should be understood that like drawing reference numbers throughout the drawings and the detailed description consistently refer to the same elements, features, and structures, unless otherwise described. The relative size and appearance of these elements may be exaggerated for purposes of clarity, illustration, and convenience.

Gesture-based input is used in a variety of situations and contexts. Gesture-based input allows a user or operator to interact with an input or interface without contacting a user interface. Gesture-based input is enabled by a camera or recognition technique that allows a gesture to be taken and a machine or system to be controlled accordingly. The gesture can refer to any part of a body that can be controlled or moved. For example, shaking with one hand or pointing a finger may refer to a gesture.

One difficulty with gesture-based input systems is that a user can not recognize or know if he is in the room required to recognize the gesture. Accordingly, when the user attempts to navigate to an estimated space, a clumsiness phenomenon may occur. Further, even in cases where a touchpad or a touch surface is implemented, the user may not be aware of the area required to operate the touchpad effectively. In cases where the touchpad is implemented in a vehicle, averting the user's gaze or attention from the road to a touchpad can be impractical and insecure.

Interacting with a surface often gives users or operators feedback associated with touch and physical contact. Accordingly, users often experience a haptic perception. Haptic technology or feel is a tactile feedback technology that utilizes the sense of touch by applying forces, vibrations or movements to the user. A haptic actuator can be embedded in an ergonomically selected location in the vehicle. An ergonomic location may refer to a location where the user or operator of the system positions a part of his body (for example, a seat, an arm rest, a wrist rest).

Disclosed herein are systems and methods for indicating a transition from a gesture-based input space to a touch surface. In addition, the systems and methods disclosed herein may be used to indicate any type of approach to a touch surface or gesture-based input space. Accordingly, providing the user with an input system provides the user with an improved user experience. The display may be audible, visual, haptic or a combination thereof.

1 is a block diagram illustrating an exemplary computer 100 represents. The computer 100 includes at least one processor 102 that with a chipset 104 is coupled. The chipset 104 includes a memory control node 120 and an input / output (I / O) control node 122 , A store 106 and a graphics adapter 112 are with the memory control node 120 coupled and an ad 118 is with the graphics adapter 112 coupled. A storage device 108 , a keyboard 110 , a pointing device 114 and a network adapter 116 are with the I / O control node 122 coupled. Other embodiments of the computer 100 can have different architectures.

The storage device 108 is a non-volatile, computer-readable storage medium such as a hard disk, CD-ROM, DVD or solid state storage device. The memory 106 contains instructions and data supplied by the processor 102 be used. The pointing device 114 is a mouse, trackball or other type of pointing device and is used in combination with the keyboard 110 used to data in the computer 100 enter. The pointing device 114 may also be a game system controller or any type of device used to control the gaming system. For example, the pointing device 114 be connected to a video or image capture device that uses biometric scans to recognize a specific user. The specific user can use movement and gestures to the pointing device 114 to instruct different aspects of the computer 100 to control.

The graphics adapter 112 shows pictures and other information on the display 118 at. The network adapter 116 couples the computer system 100 with one or more computer network (s).

The computer 100 is adapted to execute computer program modules for providing functionality described herein. The term "module" as used herein refers to computer program logic used to provide the specified functionality. Thus, a module in hardware, firmware and / or Software to be implemented. In one embodiment, program modules are stored in the storage device 108 stored in the memory 106 loaded and through the processor 102 executed.

The types of computers used by the devices and processes disclosed herein may vary depending on the embodiment and processor performance required by the device. The computer 100 may be a mobile device, tablet, smartphone, or any type of computing element with the items listed above. For example, a data storage device, such as a hard disk, a solid state memory, or a storage device, may be stored in a distributed database system that includes a plurality of blade servers that work together to provide the functionality described herein. The computers may lack some of the components described above, such as keyboards 110 , Graphics adapter 112 and ads 118 ,

The computer 100 may act as a server for the content sharing service disclosed herein (not shown). The computer 100 can work with other computer devices 100 in a cluster network to form the server.

2 is an example of a system 200 to show a transition from gesture-based input to a touch interface. The system 200 includes a recognition monitor module 210 , a caller 220 and an alarm transmitter 230 , The system 200 can be implemented on a device, such as the computer 100 ,

The system 200 can be implemented in any environment or situation in which a touch or gesture input is used. For example, the system may be implemented in a vehicle with the touchpad surface 260 in a control console of a vehicle (or any area where the operator or passenger of the vehicle can interact).

The system 200 can also use a gesture-based input system 250 communicate. As shown, the gesture-based input system includes 250 a camera. In an implementation of the system 200 The camera can be in an area just below and in front of the touchpad surface 260 be placed. When a user touchpad surface 260 approaching the touchpad surface 260 To touch, the gesture-based input system 250 recognize the user's hand accordingly.

The gesture-based input system 250 Can be used alone to detect when a hand in one with the operation of a touchpad surface 260 linked area, or alternatively, the gesture-based input system 250 also serve as an input mechanism.

The system 200 may be based on a toggle command from a user or implementer of the system 200 be selectively implemented. The toggle command may be stored in nonvolatile memory.

The detection monitor module 210 receives an indication from the gesture-based input system 250 in that a user's hand has entered a room in which the gesture-based input system 250 should monitor his configuration accordingly. As described above, this may be a space that is substantially close to an area where a user has a touchpad surface 260 touched, provided. The detection monitor module 210 can store the hint in a persistent store (as any of the above with reference to the storage device 108 listed storage devices).

The caller 220 retrieves an alarm based on the detected alert. The alarm function may be an audio, video, haptic function or combinations thereof. The alarm function serves as an indication of one with the touchpad surface 260 and the gesture-based input system 250 linked user, that a hand or an operating mechanism is about to touch the surface of the touchpad 260 to touch.

The alarm transmitter 230 transmits the retrieved alarm function to the appropriate mechanism. In certain cases, the alarm transmitter 230 transmit the audio alarm to a speaker (for example, a car audio speaker). If the alarm function is a visual indication, the alarm transmitter can 230 transmitted to a display or a light to indicate the retrieved alarm function. In this way, a light hint or visual indication may indicate that a user's hand is interacting with a touchpad surface 260 linked area is located.

In other cases, when the alarm function is a haptic sensation, the alarm function can be sent to a haptic controller 270 be transmitted. The haptic actuator 270 can be installed in an ergonomically determined environment. If the system 200 implemented in a vehicle, the haptic actuator can 270 for example, in an armrest, seat or any portion of the vehicle that a user of a touchpad surface 260 can be touched, implemented. Accordingly, the haptic actuator 270 be configured to respond to that Moving a user's hand into a room or area substantially close to the operating area for a touchpad surface 260 to tremble or vibrate.

In another example, the alarm function may provide an indication of the area of the touchpad surface 260 send that will be touched. If, therefore, a hand or gesture of a touchpad surface 260 approaching, the touchpad surface may be 260 Illuminated or an ad may be in the area of the touchpad surface 260 which will be touched to be displayed.

3 shows a method 300 to show a transition from gesture-based input to a touch interface. The procedure 300 can in a system 200 be implemented.

In process 310 For example, it can be seen that a hand or other limb is used to interact with a touchpad or touch surface. Detection may be accomplished by installing an image or video capture device that is placed below, over, or near the area where a user will touch a touch surface. As the user approaches a touch surface, the device accordingly picks up as the user approaches the touch surface (eg, a hand or limb used to make contact with the touch surface). Using digital signal processing, the device can ensure that this corresponds to a user's hand (or other limb) and record recognition accordingly.

In process 320 an alarm is called. An implementer of the process 300 can store a correlation table with different detections and alarms. The alarm associated with the detection is stored in a memory.

In process 330 Based on the received alarm, the device communicating the alarm is transferred to the corresponding device and accordingly an instruction is sent to generate the alarm. For example, as discussed above, communicating with a speaker would be when the alarm is an audio alarm. In another example, if the alarm is visual, it would communicate with display or lighting elements. In one example, a light shining on the limb may be initiated. In another example, a mark associated with a display on the touch surface may be initiated or illuminated.

In another example, if the alarm is a haptic sensation, it would be communicated with a haptic actuator.

4 shows an example of the system 200 in a vehicle 400 is integrated.

As in 4 shown, a hand enters a camera level 255 one to a touch surface 260 to touch. The level 255 can be predefined as lying in a certain area around the touch surface. The touch surface 260 can with the control of an electronic device 410 be linked. Not shown is a gesture-based input, in addition to the control of the electronic device 410 can be linked.

In response to the penetration of the hand into the area of the camera plane 255 It can be seen that the input mode transitions from a gesture-based control (contactless) to a touch control (contact). The recognition can be made when the hand 270 in the area of the plane 255 entry. Alternatively, the system can 200 recognize that the hand is 270 at a predetermined distance from the touch surface 260 and then detect a transition.

Once a transition has been detected, various sensor devices (such as audio, video or haptics) can be triggered. Each of the sensor devices may be associated with a specific output device. Accordingly, the user associated with the hand is provided with a sense-based interaction with the vehicle-mounted input mechanism.

Certain of these in 1 shown devices include a computer system. The computer system includes a processor (CPU) and a system bus that couples various system components, including a system memory such as read-only memory (ROM) and random access memory (RAM), to the processor. Other system memory may also be available for use. The computing system may include more than one processor or a cluster or cluster of computing systems that are networked together to provide higher processor performance. The system bus may be any of various types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. A simple ROM stored input / output (BIOS) or the like may provide basic routines that help to transfer information between elements in the computer system, such as during startup. The computer system also includes data stores that can manage a database according to known database management systems. The data stores may be in many forms, such as a hard disk drive, a magnetic disk drive, an optical drive, a tape drive, or other type of computer readable medium that can store data that the processor can access, such as magnetic cassettes, flat memory cards, DVDs , Endless tape cassettes, random access memory (RAM) and read-only memory (ROM). The data memories can be connected to the system bus via a driver interface. The data stores provide nonvolatile storage of computer readable instructions, data structures, program modules, and other data for the computer system.

To facilitate interaction with a human user (and in some cases, a machine), the computing device may include an input device such as a voice and audio microphone, a touch screen for gesture or graphic input, a keyboard, a mouse, a motion input, and so on. An output device may include one or more of a plurality of output mechanisms. In some cases, multimodal systems allow a user to provide multiple types of input to communicate with the computer system. A communication interface generally allows the computing device system to communicate with one or more other computing devices using various communications and network protocols.

The above disclosure is directed to a plurality of flowcharts and accompanying descriptions to provide for in 3 represent illustrated embodiments. The disclosed devices, components and systems contemplate the use or implementation of any suitable technique for performing the steps illustrated in these figures. 3 Accordingly, it is for illustrative purposes only and the described or similar steps may be performed at any convenient time, including concurrently, individually or in combination. In addition, many of the steps in these flowcharts may occur concurrently and / or in different orders than shown and described. Further, the disclosed systems may utilize processes and methods with additional, fewer, and / or different steps.

The embodiments disclosed herein may be implemented in digital electronic circuits or in computer software, firmware, or hardware, including the structures and their elements disclosed herein. Some embodiments may be implemented as one or more computer program (s), i. H. one or more modules of computer program instructions encoded on a tangible computer storage medium for execution by one or more processors. A computer storage medium may be or may be included in a computer readable storage device, a computer readable storage substrate, or a random access memory or serial access memory. The computer storage medium may also be or include one or more separate tangible component (s) or media such as multiple CDs, disks, or other storage devices. The computer storage medium contains no transitory signal.

As used herein, the term processor includes all types of devices, devices, and machines for processing data, including, for example, a programmable processor, a computer, a system on a chip or more, or combinations of the foregoing. The processor may include special logic circuits, e.g. As an FPGA (Field Programmable Gate Array) or an ASIC (application-specific integrated circuit). The processor may also include, in addition to hardware, code that generates an execution environment for a particular computer program, e.g. Code constituting processor firmware, a protocol stack, a database management system, an operating system, a cross-platform execution environment, a virtual machine, or a combination of one or more of them.

A computer program (also known as a program, module, engine, software, software application, script or code) may be written in any form of programming language, including compiled or interpreted language, declarative or procedural languages, and the program may be used in any form including as a stand-alone program or as a module, component, subroutine, object or any other device suitable for use in a computing environment. A computer program may correspond to a file in a file system, but this is not mandatory. A computer program may be stored in a portion of a file containing other programs or data (eg, one or more scripts stored in a markup language document) in a single file associated with the particular program or is stored in multiple coordinated files (eg, files that store / store one or more modules, subprograms, or sections of code). A computer program may be used to run on one or more computers that may be located at one site or distributed over multiple sites and through a communications network.

To provide for interaction with a person, the embodiments disclosed herein may be implemented using an interactive display, such as a graphical user interface (GUI). Such GUIs may include interactive functions such as pop-up or drop-down menus or lists, selection items, scannable features, and other features that may receive human input.

The computer system disclosed herein may include clients and servers. A client and server are generally remote from each other and typically interact over a communications network. The relationship between client and server arises because computer systems are running on the respective computers and have a client-server relationship. In some embodiments, a server transmits data (eg, an HTML page) to a client device (eg, for displaying data to a user and receiving user input from a user interacting with the client device). , Data generated in the client device (eg, a result of user interaction) may be received by the client device at the server.

It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Accordingly, the present invention is intended to cover the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents.

Claims (20)

A system for displaying a transition from a gesture-based input to a touch surface, comprising: a detection monitor coupled to a recognition device to detect the transition; and an alarm transmitter for alerting an output device based on the recognition, wherein the gesture-based input is a contactless interface associated with an electronic system, and a touch interface is a contact interface associated with the electronic system. The system of claim 1, wherein the recognition device is a video capture device. The system of claim 2, wherein the video capture device monitors a predetermined area surrounding the touch surface. The system of claim 1, wherein the transition is detected in response to the positioning of a body member associated with the gesture-based input at a predetermined distance from the touch surface. The system of claim 1, further comprising an alert handler to retrieve an alert type, the alert type associated with the dispenser. The system of claim 5, wherein the alarm type is defined as a haptic response. The system of claim 5, wherein the alarm type is defined as an audio noise. The system of claim 5, wherein the alert type is defined as a visual cue. The system of claim 8, wherein the visual indication is an illuminated mark on the touch surface. The system of claim 8, wherein the visual indication is a projected light on a body member associated with the gesture-based input. A method of displaying a transition from a gesture-based input to a touch surface, comprising: Recognizing that a body member associated with the gesture-based input transitions from a non-contact position to a contact position, wherein the non-contact position is associated with the gesture-based input and the contact position is associated with the touch surface; and Transmitting an alarm based on the detection, wherein at least one of recognizing and transmitting is performed on a processor. The method of claim 11, wherein the recognition is performed by a video capture device. The method of claim 12, wherein the video capture device monitors a predetermined area surrounding the touch surface. The method of claim 11, wherein the transition is detected in response to the positioning a predetermined distance from the touch surface. The method of claim 1, further comprising retrieving an alert function retriever to retrieve an alert type, wherein the alert type is associated with the transmitted alert function. The method of claim 15, wherein the alarm type is defined as a haptic response. The method of claim 15, wherein the alarm type is defined as an audio noise. The method of claim 15, wherein the alert type is defined as a visual cue. The method of claim 18, wherein the visual indication is an illuminated mark on the touch surface. The method of claim 18, wherein the visual indication is a projected light on a body member associated with the gesture-based input.

Images