KR20140066258A - Video display modification based on sensor input for a see-through near-to-eye display - Google Patents

Abstract

The present disclosure relates to an NFC device including a near field communication (NFC) enabled electronic device, wherein the wrist wearing NFC enabled electronic device includes a first communication link for communicating with a second NFC enabled electronic device via an NFC protocol, And a second communication link that communicates with the eyepiece via a medium-range communication protocol and receives a control command. The wrist-worn NFC-enabled electronic device facilitates the transfer of data between the eyepiece and the second NFC-enabled electronic device. The eyepiece includes an optical system that enables a perspective display in which data is displayed.

Description

VIDEO DISPLAY MODIFICATION BASED ON SENSOR INPUT FOR SEE-THROUGH NEAR-TO-EYE DISPLAY BACKGROUND OF THE INVENTION [0001]

Cross reference of related application

This application claims priority based on the following US patent applications, each of which is incorporated herein by reference in its entirety:

U. S. Patent Application No. 61 / 539,269, filed September 26,

This application is a continuation-in-part of the following US patent applications, each of which is incorporated herein by reference in its entirety:

U.S. Provisional Application No. 13 / 591,187, filed on August 21, 2012, which claims the benefit of the following provisional applications (each of which is incorporated herein by reference in its entirety): As of August 3, 2012 Filed U.S. Provisional Patent Application No. 61 / 679,522; U.S. Provisional Patent Application No. 61 / 679,558, filed August 3, 2012; U.S. Provisional Patent Application No. 61 / 679,542, filed August 3, 2012; U.S. Provisional Patent Application No. 61 / 679,578, filed August 3, 2012; U.S. Provisional Patent Application No. 61 / 679,601, filed August 3, 2012; U.S. Provisional Patent Application No. 61 / 679,541, filed August 3, 2012; U.S. Provisional Patent Application No. 61 / 679,548, filed August 3, 2012; U. S. Patent Application No. 61 / 679,550, filed August 3, 2012; U.S. Provisional Patent Application No. 61 / 679,557, filed August 3, 2012; And U.S. Provisional Patent Application No. 61 / 679,566, filed August 3, 2012; U.S. Provisional Patent Application No. 61 / 644,078, filed May 8, 2012; U.S. Provisional Patent Application No. 61 / 670,457, filed July 11, 2012; And U.S. Provisional Patent Application No. 61 / 674,689, filed July 23,

U.S. Application Serial No. 13 / 441,145, filed April 6, 2012, which claims the benefit of the following provisional application (each of which is incorporated herein by reference in its entirety): As of February 14, 2012 Filed U.S. Provisional Patent Application No. 61 / 598,885; U.S. Provisional Patent Application No. 61 / 598,889, filed February 14, 2012; U.S. Provisional Patent Application No. 61 / 598,896, filed February 14, 2012; And U. S. Patent Application No. 61 / 604,917, filed February 29,

U.S. Application Serial No. 13 / 429,413, filed March 25, 2012, which claims the benefit of the following provisional application (each of which is incorporated herein by reference in its entirety): As of January 6, 2012 Filed U.S. Provisional Patent Application No. 61 / 584,029.

U.S. Application Serial No. 13 / 341,758, filed December 30, 2011, which claims the benefit of the following provisional application (each of which is incorporated herein by reference): As of November 8, 2011 Filed U.S. Provisional Patent Application No. 61 / 557,289.

U.S. Application Serial No. 13 / 232,930, filed September 14, 2011, which claims the benefit of the following provisional application (each of which is incorporated herein by reference): as of September 14, 2010 Filed U.S. Provisional Patent Application No. 61 / 382,578; U. S. Patent Application No. 61 / 472,491, filed April 6, 2011; U. S. Patent Application No. 61 / 483,400, filed May 6, 2011; U. S. Patent Application No. 61 / 487,371, filed May 18, 2011; And U. S. Patent Application No. 61 / 504,513, filed July 5,

Each of which is incorporated herein by reference in its entirety for all purposes to the benefit of U.S. Provisional Patent Application No. 13 / 037,324, filed February 28, 2011, the benefit of which is hereby incorporated by reference in its entirety, U.S. Provisional Patent Application No. 13 / 037,335, filed on February 28, U.S. Provisional Patent Application No. 61 / 308,973, filed February 28, 2010; U. S. Patent Application No. 61 / 373,791, filed August 13, 2010; U. S. Patent Application No. 61 / 382,578, filed September 14, 2010; U. S. Patent Application No. 61 / 410,983, filed November 8, 2010; U.S. Provisional Patent Application No. 61 / 429,445, filed January 3, 2011; And U. S. Patent Application No. 61 / 429,447, filed January 3,

Field:

The present disclosure relates to an augmented reality eyepiece, associated control techniques, and applications for use, and more particularly to a software application program running on an eyepiece.

The present disclosure relates to thin display technology that uses a switchable mirror in a sequenced pattern to provide images from a waveguide.

BACKGROUND OF THE INVENTION [0002] Head mounted displays having reflecting surfaces are known in the art. A head mounted display having an angled single partial reflecting beam splitter plate is described in U.S. Patent No. 4969714. [ While this approach provides excellent brightness and color uniformity over the display field of view, the optical system is relatively thick due to the angled beam splitter plate.

A head mounted display having an array of partial reflective surfaces to provide a thinner optical system is described in US Patent Nos. 6829095 and 7724441, shown in Fig. 124, and an array of partial reflective surfaces 12408 Is used to provide an image light 12404 over the display field to allow the user to view the displayed image associated with the view of the environment in front of the user. The image light 12404 viewed by the user consists of a combination of light reflected from each of the plurality of partial reflective surfaces 12408. [ Light 12402 from an image source is passed through a number of partial reflective surfaces 12408 where a portion of light 12402 is reflected toward the user's eye to provide image light 12404 must do it. In order to provide a uniform image over the display field of view, the reflection characteristics of the partial reflective surfaces 12408 must be precisely controlled. The reflectivity of the partial reflective surfaces 12408 should be lowest for the surfaces closest to the image light source and highest for the furthest surfaces from the image light source. In general, the reflectivity of the partially reflective surfaces 12408 must increase linearly with distance from the image source. This causes manufacturing and cost problems because the reflectivity of each partial reflective surface 12408 is different from neighboring surfaces and the reflectivity of each surface must be tightly controlled. Accordingly, it is difficult to use an array of partial reflective surfaces to provide images with uniform brightness and hue throughout the display field of view.

As another alternative, a diffractive grating may be used to direct the image light into and out of a waveguide toward the display field, as described in U.S. Patent No. 4,711,512. Is used. However, the diffraction grating is expensive and undergoes color aberration.

Thus, there is a need for a relatively thin optical system for a head mounted display that also provides good image brightness and color uniformity across the display field of view.

The present disclosure also relates to a lightweight frontlight that includes a wire grid polarizer film as a partial reflective surface to deflect illumination light down toward a reflective image source. ).

133, the illumination light 13308 exits the edge light source 13300 and is deflected by the forward illumination 13304 to produce a reflected image light source (e.g., 13302). The illumination light 13308 is then reflected from the reflection image light source 13302 to be an image light 13310. The image light 13310 then passes through the front illumination 13304 again to enter the display optics. Accordingly, the forward illumination 13304 simultaneously deflects the illumination light 13308 incoming from the edge light source 13300 and allows the reflected image light 13310 to pass through without being biased to enter the display optics Wherein the display optics may be dispersive when the display is a flat screen display or may be refractive or diffractive when the display is a near- have. In this embodiment, the display optical system may include a diffuser.

For a reflected image light source such as a liquid crystal on silicon (LCOS) image light source, the illumination light is polarized and the reflected image light source is a quarter wave retardation film that changes the polarization state during reflection from the reflected image light source. ). A polarizer is then included in the display optics to allow the polarization effect imparted by the liquid crystal to form an image as the image light passes through the display optics.

U.S. Patent No. 7,163,330 discloses a method of forming a groove in an upper surface of a front illuminator for deflecting light from an edge light source toward a reflective image light source downwardly between grooves for allowing reflected image light to enter the display optical system Along with a flat section of the front lighting. Fig. 134 shows an example of a front illumination 13400 with a groove 13410 and a flat section 13408. Fig. The illumination light 13402 from the edge light source 13300 is reflected from the groove 13410 and deflected downward to illuminate the reflection image light source 13302. [ The image light 13404 is reflected from the reflected image light source 13302 and passes through the flat section 13408 of the front illumination 13400. Linear and curved grooves 13410 are described. However, in order for grooves 13410 to effectively deflect illumination light 13402, groove 13410 must occupy a significant area of the front illumination, thereby limiting the area of flat section 13408, And deteriorates the image quality provided to the display optical system due to light scattering from the groove when passing through the groove. Front light 13400 is typically formed of a solid plate of material, and may therefore be relatively heavy.

In US 7545571 a polarization beam splitter 13502 is provided for deflecting and polarizing the illumination light 13504 supplied by the edge light source 13500 onto the reflected image light source 13502, a wearable display system including a reflective image light source 13502 having a splitter 13512 as front illumination is proposed. Polarizing beam splitter 13512 is an angled plane in a solid block with a separate curved reflector 13514 that is associated with edge light source 13500. Curved reflector 13514 may be a total internal reflection block 13510 coupled to polarization beam splitter 13512. [ Accordingly, the front lighting disclosed in this patent with the solid block and the total internal reflection block of the polarizing beam splitter provides large and relatively heavy front lighting. In addition, Figure 135 also shows an image light ray 13508. [

There is a need to provide front lighting for a display that provides good image quality with little scattered light and is also small and lightweight and has a reflective image light source.

The disclosure also relates to an optically flat surface made of an optical film. More specifically, the present disclosure provides a method of manufacturing an optically planar beam splitter using an optical film.

Optical films can be obtained for various purposes, including beam splitters, polarizing beam splitters, holographic reflectors, and mirrors. In imaging applications, and particularly in reflective imaging applications, it is important to define the optical film as very flat to maintain the wavefront of the image. Any optical film with a pressure sensitive adhesive on one side is available to allow the optical film to adhere to the substrate for structural support and to help keep the optical film flat. However, the optical film attached to the substrate in this manner tends to have a surface with small undulations and pockmarks (called orange peels), which may result in optical surface flatness optical flatness, and as a result, the reflected image is deteriorated.

In U.S. Patent Application No. 20090052030, a method of manufacturing an optical film in which the optical film is a wire grid polarizer is provided. However, a technique for providing a film having optical flatness is not provided.

U.S. Patent Nos. 4537739 and 4643789 provide a method of attaching artwork to a molded structure using a strip to carry the artwork in a mold. However, these methods do not anticipate specific requirements for optical films.

In U.S. Patent Application No. 20090261490 there is provided a method of making a simple optical article comprising an optical film and a molding. This method relates to the curved surface generated since the method involves a limitation on the ratio of radius of curvature to diameter to avoid wrinkles in the film due to deformation of the film during molding. The specific requirements for producing optically flat surfaces with optical films are not addressed.

In U.S. Patent No. 7820081, a method of laminating a functional film to a lens is provided. This method uses a thermally cured adhesive to bond the functional film to the lens. However, this process involves thermoforming the optical film while the lens is at a high temperature so that the optical film, the adhesive, and the lens are deformed together during the bonding process. Accordingly, this method is not suitable for producing an optically smooth surface.

Therefore, there is a need for a method of using an optical film so that the surface including the optical film has optical flatness.

In embodiments, the eyepiece may include an internal software application running on an integrated multimedia computing facility configured to interact with the 3D augmented reality (AR) content display and the eyepiece. 3D AR software applications may be developed with mobile applications and may be provided as stand-alone applications targeting the eyepiece either through the application store (s) or specifically as an end-use platform, and through a dedicated 3D AR eyepiece store . The internal software application may be accessed via equipment inside and outside the eyepiece (such as those initiated from sensing devices, user motion capture devices, internal processing equipment, internal multimedia processing equipment, other internal applications, cameras, sensors, Through the transceiver, through the tactile interface, with the input and output facilities provided by the eyepiece, from external computing facilities, external applications, events and / or data feeds, external devices, third parties, Commands and control modes that operate in relation to the eyepiece can be initiated by sensing input through the input device, user actions, external device interaction, receiving events and / or data feeds, executing internal applications, executing external applications, have. In embodiments, at least an event and / or data feed, a sense input and / or sense device, a user action capture input and / or output, a user movement and / or action to control and / An application on a platform on which the command can be used to respond to input, a communication and / or connection from an on-platform interface to an external system and / or device, an external device, an external application, There may be a series of steps included in the execution control provided through the internal software application, including the combination of two of the feedback (the external device, the external application, etc.) to the user or the like.

The present disclosure also provides a method of providing a relatively thin optical system that provides images with improved brightness and color uniformity over the display field of view. The present disclosure includes an integral array of narrow switchable mirrors across a display area to provide a display field of view and to reflect portions of light from an image light source to present sequential portions of the image to a user, Switchable mirrors are used sequentially. By quickly switching narrow transparent mirrors from transparent to reflective in a repeating sequence, the user recognizes portions of the image that will be combined into the overall image as presented by the image light source. If each of the narrow switchable mirrors is switched to more than 60 Hz, the user does not recognize flicker in portions of the image.

Various embodiments of arrays of narrowly switchable mirrors are shown. In one embodiment, the switchable mirror is a liquid crystal switchable mirror. In another embodiment, the switchable mirror is a moveable prism element that uses an air gap to provide a convertible internal total reflection mirror.

In an alternative embodiment, not all of the switchable mirrors are used in the sequence, but instead the switchable mirrors are used as the selected group, which changes based on the eye distance of the user.

The present disclosure also provides a small, lightweight frontal illumination that includes the wire grid polarizing film as a partial reflective surface to deflect the illumination light down toward the reflective image light source. The edge light source is polarized, and the wire grid polarizer is oriented so that the illumination light is reflected and the image light enters the display optics. By using a flexible wire grid polarizing film, the present disclosure provides a partially reflective surface that can be curved to focus illumination light onto a reflected image light source, thereby improving efficiency and improving uniformity of image brightness. The wire grid polarizer also has very low light scattering because the image light passes through the front illumination and continues to the display optics, thus maintaining the image quality. In addition, since the partially reflective surface is a wire grid polarizing film, most of the front lighting is made up of air, so that the front lighting is much lighter in weight.

The present disclosure also provides a method of making a surface having optical flatness when using an optical film. In embodiments of the present disclosure, the optical film may comprise a beam splitter, a polarizing beam splitter, a wire grid polarizer, a mirror, a partial mirror, or a holographic film. An advantage provided by the present disclosure is that the surface of the optical film is optically flat so that the wavefront of the light is maintained to provide improved image quality.

In certain embodiments, the present disclosure provides an image display system that includes an optically flat optical film. The optically planar optical film includes a substrate that maintains the optical film optically flat in a display module housing having an image light source and a viewing location. The image provided by the image light source is reflected from the optical film to the viewing position and the substrate with the optical film is replaceable within the display module housing.

In other embodiments of the present disclosure, the optical film is attached to the molded structure, and thus the optical film is part of the display module housing.

In a prior art display 18700 having a reflected image light source 18720 and a solid beam splitter cube frontlight 18718 as shown in Figure 187, light 18712 is incident on a light source 18702 ) Into diffuser 18704 and the light is more uniform in diffuser 18704 to provide illumination light 18714. [ The illumination light 18714 is redirected by the partially reflective layer 18708, thereby illuminating the reflected image light source 18720. The illumination light 18714 is then reflected from the reflected image light source 18720 and becomes an image light 18710 which is then passed through the partial reflective layer 18708 to present an image to an observer. (Not shown). Accordingly, the solid beam splitter cube 18718 simultaneously redirects the illumination light 18714 and allows the reflected image light 18710 to pass undirected so that it can enter the imaging optics, The optical system may be dispersive when the display is a flat screen display or it may be refracting or diffracting when the display is a projector or near vision display.

In the case of a reflective image light source such as a liquid crystal on silicon (LCOS) image light source, the illumination light is polarized, and the reflected image light source is polarized based on the image content provided by the image light source when the illumination light is reflected from the reflected image light source. Thereby forming an image light. An analyzer polarizer is then included which allows the imaging light to pass through the imaging optics and cause the polarization effect provided by the LCOS to form an image when the image is presented to an observer.

U.S. Patent No. 7545571 discloses a wearable display system that includes as a front illumination a reflected image light source having a polarizing beam splitter for deflecting and polarizing illumination light supplied by an edge light source onto a reflected image light source. The polarizing beam splitter is an angular plane in the solid block with a separate curved reflector associated with the edge light source. The curved reflector may be an internal total reflection block connected to the polarization beam splitter. Accordingly, the front lighting disclosed in this patent with the solid block and the total internal reflection block of the polarizing beam splitter provides large and relatively heavy front lighting.

U.S. Patent No. 6,195,136 discloses a series of frontlight illumination methods for use with reflective image light sources. A method of using a curved beam splitter to make the front lighting smaller is disclosed. However, the curved beam splitter is located quite far away from the image light source to reduce the angle of light from the light source (which is later reflected by the beam splitter towards the image light source). Further, the light is provided only on one side of the front light, and therefore the size of the beam splitter must be at least as large as the image light source. As a result, when the overall size of the front illumination is measured along the optical axis, it is still relatively large compared to the illuminated area on the image light source.

There is a need to provide front lighting for a display having a reflective image light source that provides good image quality with little scattered light and is also small, efficient, and lightweight.

The present disclosure provides a small, efficient, and lightweight front light to the display assembly that includes a partial reflective surface for redirecting illumination light from a side light source toward a reflective image light source, The size of the assembly is substantially smaller than the width of the reflected reflection image light source. In certain embodiments, the partial reflective surface may be curved to focus or focus light from the light source onto the reflective image light source. The polarizing beam splitter film can be used as a curved partial reflecting surface so that the light source can be polarized and the illumination light can be redirected and the reflected video light can enter the imaging optics. The polarizing beam splitter film is lightweight and has a very low light scattering because the image light passes through the front illumination and continues to the display optical system, thus the image quality is maintained.

In other embodiments of the present disclosure, a light source is provided on opposite sides of the forward illumination such that light is provided at the opposite edges of the reflective image light source. In this case, the partially reflecting surface consists of two surfaces, one surface deflecting the illumination light from one light source towards one half of the image light source, and the other surface deflecting the light towards the other half of the image light source . In this embodiment, the partially reflective surface may be curved or flat.

In a further embodiment of the present disclosure, the partial reflective surface is a polarizing beam splitter, in which the light source is polarized and thus the light from the light source is first redirected by the polarizing beam splitter, then reflected by the reflected image light source, Lt; / RTI >

In another embodiment, the light from the light source is unpolarized, so that the polarization beam splitter reflects one polarization state of light to illuminate half of the reflected image light source while the other polarization state of light is transmitted. The transmitted polarization state of light enters the opposite side of the front illumination, where light is recirculated. Recirculation of the transmitted polarization state can be done by passing through the quarter wave film and reflecting off the mirror so that light passes again through the quarter wave film and thereby changes the polarization state. After the polarization state of the transmitted and reflected light changes, the light is redirected by the polarization beam splitter to illuminate the other half of the reflected image light source. In an alternative embodiment, the light from the two sidelights of the front illumination functions in a complementary manner, wherein the transmitted polarization state of light from the opposite side is unpolarized when interacting with the diffuser on the opposite side And is thereby recirculated.

In yet another embodiment of the present disclosure, a method is provided for manufacturing a front lighting with a flexible partial reflective film. The flexible film may be either supported at the edges and freestanding on the reflective image light source or the flexible film may be clamped between two or more solid solid pieces that are transparent. The solid side piece may be formed before being placed in contact with the flexible film. The solid single piece can hold the flexible film in a flat shape or a curved shape. In yet another embodiment, the flexible film may be supported at the edge, and then the solid side piece may be cast in place such that the flexible film is embedded in a transparent solid material.

In one embodiment, the system includes an interactive head-mounted eyepiece worn by the user, the eyepiece includes an optical assembly through which a user views the surrounding environment and the displayed content, An integrated processor that processes content for display to a user, an integrated image light source that directs the content into an optical assembly, and the processor is configured to modify the content, the modification being made in response to the sensor input. The content may be a video image. These modifications include adjusting the brightness, adjusting the color saturation, adjusting the color balance, adjusting the color hue, adjusting the video resolution, transparency Adjusting the compression ratio, adjusting the frame rate per second, isolating a portion of the video, stopping playback of the video, pausing the video, or restarting the video It can be one. The sensor input may be a charge-coupled device, a black silicon sensor, an IR sensor, an acoustic sensor, an induction sensor, a motion sensor, an optical sensor, an opacity sensor, a capacitive sensor, a capacitive displacement sensor, a Hall effect sensor, a magnetic sensor, a GPS sensor, a thermal image sensor, a thermocouple, a thermistor, a thermoelectric sensor, an inductive sensor, an eddy current sensor, a passive infrared proximity sensor, Ultrasonic 3D motion sensor, accelerometer, inclinometer, force sensor, piezoelectric sensor, rotary encoder, linear encoder, chemical sensor, ozone sensor, smoke sensor, ultrasonic sensor, ultrasonic sensor, infrared laser sensor, inertial motion sensor, MEMS internal motion sensor, A sensor, a magnetometer, a carbon dioxide detector, a carbon monoxide detector, an oxygen sensor, a glucose sensor, a smoke detector, a metal detector, a rain sensor, an altimeter, (E.g., a billboard), a landmark detector (e.g., a geographical location marker for advertising), a laser range finder, a sonar, a capacitance, an optical response, a heart rate sensor, Or an RF / MIR (micropower impulse radio) sensor. The playback of the content may be interrupted in response to an indication from the accelerometer input that the user's head is moving. Audio sensor input can be generated by at least one participant in the videoconference talking. The visual sensor input may be a video image of at least one participant of the videoconference or a video image of a visual presentation. This modification may be at least one of making the video image more transparent or less transparent in response to an indication from the sensor that the user is moving.

In one embodiment, the system may include an interactive head mounted eyepiece worn by a user, the eyepiece may include an optical assembly through which a user views the surrounding environment and the displayed content, an integrated assembly A processor, an integrated image light source processor for importing content into the optical assembly, the processor configured to modify the content, the modification being made in response to the sensor input; And an integrated video image capturing facility that provides content for recording and displaying the appearance of the surrounding environment.

These and other systems, methods, objects, features, and advantages of the present disclosure will become apparent to those skilled in the art from the following detailed description and drawings of the embodiments.

All documents referred to herein are incorporated herein by reference in their entirety. Unless expressly stated otherwise or clear from the text, reference to an item in singular is to be understood as including a plurality of items, and vice versa. Unless otherwise stated or clear from the context, a grammatical conjunction is to be understood as representing all logical AND combinations of combined clauses, sentences, words, and so on.

The following detailed description of certain embodiments of the present disclosure and the present disclosure can be understood with reference to the following drawings.
1 illustrates an exemplary embodiment of an optical configuration;
Figure 2 shows an RGB LED projector.
3 shows a projector in use;
Figure 4 shows one embodiment of a waveguide and a correction lens disposed in a frame.
5 shows a design for a waveguide eyepiece.
6 illustrates one embodiment of an eyepiece having a see-through lens;
7 shows an embodiment of an eyepiece having a perspective lens;
Figures 8a-8c illustrate embodiments of eyepieces arranged in a flip-up / flip-down configuration.
8D and 8E illustrate embodiments of a snap-fit element of a secondary optic.
Figure 8f illustrates embodiments of flip-up / flip-down electro-optic modules.
9 is a view showing an electrochromic layer of an eyepiece;
Figure 10 illustrates advantages of an eyepiece in real-time image enhancement, keystone correction, and virtual perspective correction.
11 is a plot of the responsivity versus wavelength for three substrates;
12 shows performance of a black silicon sensor.
FIG. 13A is a diagram showing a conventional night vision system, FIG. 13B is a diagram showing a night vision system of the present disclosure, and FIG. 13C is a diagram showing a difference in responsiveness between the two.
14 shows a tactile interface of an eyepiece;
Figure 14a illustrates movement in an embodiment of an eyepiece featuring nod control;
15 shows a ring for controlling an eyepiece;
15aa illustrates, in one embodiment, a ring that controls an eyepiece having an integral camera that allows a user to provide his or her video image as part of a video conference.
15A is a view of a hand mounted sensor in an embodiment of a virtual mouse.
Figure 15B shows a facial actuation sensor mounted on an eyepiece;
15C is a diagram illustrating hand pointing control of an eyepiece;
15D is a view showing hand pointing control of the eyepiece;
15E is a diagram showing an example of eye tracking control;
15F is a view showing hand position determination control of the eyepiece.
Figure 16 illustrates a position-based application mode of an eyepiece.
17 is a diagram showing the difference in image quality between the flexible platform of an uncooled CMOS image sensor capable of VIS / NIR / SWIR imaging and B) an image intensified night vision system.
18 shows an augmented reality-supported custom billboard;
19 illustrates an augmented reality-supported customized advertisement;
20 illustrates augmented reality-supported custom artwork.
20A is a diagram illustrating a method for posting a message to be transmitted when an observer reaches a specific place;
21 is a view showing an alternative configuration of an eyepiece optical system and an electronic device;
22 is a view showing an alternative configuration of an eyepiece optical system and an electronic device;
22A is a view showing an eyepiece having an example of an eyeglow;
22B is a cross-sectional view of an eyepiece having a light control element for reducing glare;
23 is a view showing an alternative configuration of an eyepiece optical system and an electronic device.
24 is a view showing the lock position of the virtual keyboard;
Figure 24A illustrates one embodiment of an image that is virtually projected on a portion of a human body;
25 is a view showing a detailed view of the projector.
26 is a view showing a detailed view of an RGB LED module;
27 shows a game network;
28 is a view showing a method of playing a game using an augmented reality glasses.
29 illustrates an exemplary electronic circuit diagram for an augmented reality eyepiece;
29A is a diagram showing a control circuit for eye tracking control of an external apparatus.
29B illustrates a communication network between users of an augmented reality eyepiece;
30 shows partial image removal by an eyepiece;
31 is a flowchart of a method for identifying a person based on a person's voice captured by a microphone of an augmented reality device;
32 illustrates a typical camera for use in a video telephone or video conference;
33 shows an embodiment of a block diagram of a video telephone camera;
Figures 34A-34E illustrate embodiments of eyepieces for optical or digital stabilization.
35 illustrates one embodiment of a classic cassegrain configuration;
36 is a view showing a configuration of a micro-cassegrain telescoping folded optical camera;
37 is a view showing a swipe process by a virtual keyboard;
38 shows a target marker process for a virtual keyboard;
Figure 38A illustrates one embodiment of a visual word translator.
39 is a view of glasses for capturing biometric data according to an embodiment;
40 is a view showing iris recognition using biometric data capturing glasses according to one embodiment;
41 illustrates face and iris recognition according to one embodiment;
Figure 42 illustrates the use of a dual omni-microphone in accordance with one embodiment.
Figure 43 is a diagram illustrating directionality improvement by a plurality of microphones;
Figure 44 illustrates the use of an adaptive array to tune an audio capture facility in accordance with one embodiment;
45 illustrates a mosaic finger and palm registration system in accordance with one embodiment;
46 shows a conventional optical scheme used by other finger and palm printing systems;
Figure 47 illustrates a method used by a mosaic sensor in accordance with one embodiment;
48 illustrates a device layout of a mosaic sensor according to one embodiment;
49 is a view showing the number of camera view fields and cameras used in the mosaic sensor according to another embodiment;
50 illustrates a bio-phone and a tactical computer in accordance with an embodiment;
Figure 51 illustrates the use of a biophone and tactical computer for capturing latent fingerprints and palm prints according to one embodiment;
52 illustrates a typical DOMEX collection;
53 is a diagram illustrating a relationship between biometric images captured using a biophone and a tactical computer and a biometric watch list according to an embodiment;
54 illustrates a pocket bio-kit according to one embodiment.
55 illustrates components of a pocket biotic kit according to one embodiment.
56 illustrates a fingerprint, a long portal, a geographic location, and a POI registration device, according to one embodiment.
57A-57E illustrate a multi-mode biometric collection, identification, geolocation, and POI registration system, according to one embodiment.
58 illustrates a fingerprint, a long pass, a geographic location, and a POI registered forearm wearable device, according to one embodiment.
59 shows a mobile folding biometric registration kit according to one embodiment;
60 is a high-level system diagram of a biometric registration kit according to an embodiment;
61 is a system diagram of a folding biometric registration device according to one embodiment;
62 illustrates a thin film fingerprint and long-range sensor according to one embodiment;
63 illustrates a biometric collection device for finger, palm, and registration data collection, according to one embodiment.
64 illustrates acquisition of a two stage palm print according to one embodiment;
65 illustrates capture of a fingertip tap in accordance with one embodiment;
Figure 66 illustrates acquisition of a slap and roll print according to one embodiment;
67 shows a system for collecting non-contact fingerprints, long or other biometric prints;
68 shows a process for collecting non-contact fingerprints, long or other biometric prints;
69 shows an embodiment of a clock controller;
Figures 70A-70D illustrate exemplary cases for an eyepiece including a charging function and an integral display.
71 illustrates one embodiment of a ground stake data system;
72 is a block diagram of a control mapping system including an eyepiece;
73 shows a biometric flashlight;
74 shows a helmet-mounted version of an eyepiece;
75 illustrates an embodiment of a situation-aware eyeglass.
FIG. 76A is a view showing an assembled 360 ° imager, and FIG. 76B is a broken view of a 360 ° imager.
77 is an exploded view of a multi-coincident view camera;
78A and 78B are diagrams showing a flight eye;
79 is an exploded top view of an eyepiece;
80 shows a disassembled electro-optical assembly;
81 is an exploded view of a shaft of an electro-optical assembly;
82 illustrates one embodiment of an optical display system that utilizes a planar lighting fixture with a reflective display;
83 shows a structural embodiment of a planar illumination optical system;
84 shows an embodiment assembly of a planar lighting fixture and a reflective display with laser spot suppression components;
85 shows an embodiment of a planar lighting fixture having a grooved feature for redirecting light;
86 illustrates one embodiment of a planar lighting fixture having a pair of grooved features and a " anti-grooved " feature to reduce image aberration.
87 shows one embodiment of a planar lighting fixture made of a laminate structure;
88 illustrates one embodiment of a planar lighting fixture having a wedged optic assembly for redirecting light;
89 is a block diagram of a lighting module according to one embodiment of the present disclosure;
90 is a block diagram of an optical frequency converter according to one embodiment of the present disclosure;
91 is a block diagram of a laser illumination module in accordance with one embodiment of the present disclosure;
92 is a block diagram of a laser illumination system in accordance with another embodiment of the present disclosure;
93 is a block diagram of an imaging system in accordance with one embodiment of the present disclosure;
94A and 94B are top and side views, respectively, of a lens having a photochromic element and a heater element;
95 illustrates one embodiment of a LCoS forward lighting design;
96 is a view of an optically bonded prism optically bonded to a polarizer;
97 is a view showing a prism optically bonded to a polarizer;
98a-c illustrate multiple embodiments of a LCoS frontal lighting design.
99 shows a wedge + OBS overlaid on a LCoS;
Figures 100a and 100b show two versions of the wedge.
101 shows a curved PBS film on an LCoS chip;
102A illustrates one embodiment of an optical assembly;
102B illustrates one embodiment of an optical assembly having an in-line camera.
103 shows an embodiment of an image light source;
104 shows an embodiment of an image light source;
105 shows embodiments of an image light source;
106 is a high-level block diagram depicting a software application program facility and market in connection with the function and control aspects of the eyepiece in one embodiment of the present disclosure;
107 is a functional block diagram of an eyepiece application development environment in an embodiment of the present disclosure;
108 is a view of a platform elements development stack for a software application of an eyepiece in an embodiment of the present disclosure;
109 illustrates a head-mounted display with see-through capability, in accordance with an embodiment of the present disclosure;
110 is a view showing a view of an unlabeled scene viewed through the head mounted display shown in FIG. 109; FIG.
Figure 111 shows a view of the scene of Figure 110 with 2D labels overlaid.
112 is a view showing the 3D label of FIG. 111 displayed on the left eye of the observer; FIG.
113 is a view showing the 3D label of FIG. 111 displayed on the observer's right side; FIG.
114 shows left and right 3D labels of FIG. 111 overlapped with each other to show discrepancies; FIG.
115 shows a view of the scene of FIG. 110 with a 3D label; FIG.
116 is a view showing a captured stereoscopic image of the scene of FIG. 110; FIG.
117 shows the overlaid left and right stereoscopic images of FIG. 116 showing discrepancies between images; FIG.
118 shows a scene of FIG. 110 showing an overlaid 3D label; FIG.
119 is a flow chart for a depth cue method embodiment of the present disclosure that provides a 3D label;
120 is a flowchart for another depth cue method embodiment of the present disclosure that provides a 3D label;
121 is a flow chart for another depth cue method embodiment of the present disclosure that provides a 3D label;
122 is a flow chart for another depth cue method embodiment of the present disclosure that provides a 3D label;
123a illustrates a processor that provides display sequential frames for image display through a display component;
123B shows a display interface configured to omit the display driver;
124 is a schematic view of a prior art waveguide having a plurality of partial reflectors;
125 is a schematic view of a waveguide having a plurality of electrically switchable mirrors in a first position;
125A is a view of a waveguide assembly having an electrical connection;
126 is a schematic view of a waveguide having a plurality of electrically switchable mirrors in a second position;
127 is a schematic view of a waveguide having a plurality of electrically switchable mirrors in a third position;
128 is a schematic view of a waveguide having a plurality of mechanically switchable mirrors in a first position;
128a is a schematic diagram of a waveguide assembly having a microactuator and associated hardware;
129 is a schematic view of a waveguide having a plurality of mechanically switchable mirrors in a second position;
130 is a schematic view of a waveguide having a plurality of mechanically switchable mirrors in a third position;
131A and 131B show a waveguide display with a switchable mirror on the face of the user;
Figures 132a-132c show display areas provided for users with different eye gaps.
133 is a schematic view of a reflected image light source having an edge light source and a forward illumination, showing the passage of light rays of light;
134 is a schematic view of a prior art front lighting comprising a groove;
135 is a schematic diagram of a prior art front illumination including a planar polarizing beam splitter and a curved reflector in a solid block;
136 is a schematic diagram of one embodiment of the present disclosure having a single edge light source and a curved wire grid polarizing film.
137 is a schematic diagram of one embodiment of the present disclosure having two edge light sources and a curved wire grid polarizing film.
138 is a schematic view of a side frame holding a flexible wire grid polarizing film in a desired curved shape;
139 is a flow chart of a method of the present disclosure.
140 is a schematic diagram of a near-eye imaging system having a beam splitter;
141 is a schematic diagram of an optical system module for a near vision system;
Figure 142 shows a pellicle style optical plate.
143 is a view of an insert molded module housing having an embedded optical plate;
144 shows compression molding of a laminate style optical plate;
Figures 145a-c illustrate attachment of an optical film within a molded module housing.
146 is a schematic front perspective view of an AR eyepiece (without eyepiece leg portion) according to one embodiment of the present disclosure;
147 is a schematic rear perspective view of the AR eyepiece of FIG. 146;
Figure 148 is a schematic rear perspective view of the right side of the wearer of the AR eyepiece of Figure 146;
149 is a schematic rear perspective view of the right side of the wearer of the AR eyepiece of FIG. 146;
150 is a schematic perspective view of the components of the AR eyepiece shown in FIG. 146 to support one of the projection screens;
151 is a schematic perspective view of the adjustment platform of the AR eyepiece shown in FIG. 146;
152 is a schematic perspective view of the components of the lateral adjustment mechanism of the AR eyepiece shown in FIG. 146;
Figure 153 is a schematic perspective view of the components of a tilt adjustment mechanism of the AR eyepiece shown in Figure 146;
154 is a chart showing dark adaptation curves for human eyes;
FIG. 155 is a chart showing the effect of progressively reducing the illuminance on the dark adaptation curve to the human eye. FIG.
156 shows a head-mounted display having a perspective function;
157 is a graph showing the relationship between display brightness and time when entering a dark environment;
158 is a flowchart of a dark adaptation method;
Figure 159 shows a virtual keyboard presented in the field of view of the user.
160 shows an example of a display system having an optically flat reflective surface;
161 is a diagram showing an example of a near vision display module;
Figure 162 illustrates an example of an optical system associated with one type of head mounted display.
Figure 163 shows an example in which a baffle is added between the illumination beam splitter and the lens in the interior of the housing;
164 illustrates an example of another embodiment of the present disclosure in which a baffle is added to the entrance surface of the lens;
165 illustrates an example of another embodiment of the present disclosure in which a baffle is added to the exit of the lens;
Figure 166 illustrates an example of another embodiment of the present disclosure in which the baffle is attached to the housing between the lens and the image beam splitter.
Figure 167 illustrates an example of a further embodiment of the present disclosure in which an absorbent coating is applied to a side wall of the housing.
168 shows an illustration of another source of stray light in a head mounted display wherein the stray light comes directly from the edge of the light source.
169 is a view showing that a stray light is reflected from an arbitrary reflective surface or an edge of a lens in a housing;
170 shows an illustration of another further embodiment of the present disclosure in which a baffle is provided adjacent to a light source;
Figure 171 illustrates that an absorbent coating with ridges may be used wherein a series of small ridges or stairs form a series of < RTI ID = 0.0 > It functions as a baffle.
172 shows a further embodiment of a tape or sheet comprising a carrier sheet and ridges that can be used to block reflected light;
Figure 173 is an exploded view of an embodiment of a pair of glasses.
Figure 174 shows a wiring design of a spectacle and a wire guide;
175 is an enlarged view of a wiring design of a spectacle and a wire guide;
176 (a) shows a wiring design of glasses and a broken view of a wire guide.
Fig. 176b shows a wiring design of glasses and a broken view of a wire guide; Fig.
Fig. 176c is a diagram showing the wiring design of glasses and a complete view of the wire guide; Fig.
177 is a view showing a U-shaped accessory for fixing glasses.
Figure 178 shows an embodiment of a cable-tensioned system for securing glasses to a user's head.
Figures 179a and 179b illustrate an embodiment of a cable-tensioning system for securing glasses to a user's head with a curved configuration.
180 illustrates one embodiment of a cable-tensioning system for securing glasses to a wearer's head;
Figure 181 shows an embodiment of a system for securing glasses to a user's head;
Figure 182 illustrates an embodiment of a system for securing glasses to a user's head.
Figure 183 shows an embodiment of a system for fixing glasses to a user's head.
Figure 184 illustrates an embodiment of a system for securing glasses to a user's head.
185a shows an embodiment of an optical train;
185B shows a sample ray trace for light in one embodiment of a light train;
Figure 186 illustrates one embodiment of an LCoS + ASIC package.
Figure 187 schematically illustrates a prior art front illumination using a single light source and a beam splitter cube;
Figure 188 schematically depicts a prior art frontlight using a single light source and a reflective beam splitter layer;
189 schematically shows a front light using a single light source, in which a flat reflective beam splitter layer is arranged at a reduced angle.
FIG. 190 schematically shows a front light using a single light source, wherein the reflective beam splitter layer is curved. FIG.
191 schematically shows a front light using a dual light source, wherein a curved reflective beam splitter film with a flat surface is disposed in a transparent solid body.
FIG. 192 schematically shows a front light using a dual light source, wherein a bent free-standing reflective beam splitter film having a flat surface is used.
Figure 193 schematically shows front illumination using a dual light source, wherein a curved free standing reflective beam splitter film with a curved surface is used.
Figure 194 is a schematic representation of front illumination using a dual light source, in which a curved reflective beam splitter film with a curved surface is disposed in a transparent medium.
195 is a schematic view of a front light using a single light source having opposing mirrors and a quarter wave film to recycle a portion of the polarized light, wherein the curved reflective beam splitter film having a flat surface is transparent Provided within the entity.
196 schematically depicts a frontal illumination using a single light source having opposite mirrors and quarter wave films to recycle a portion of the polarized light, wherein the free standing reflective polarizer beam splitter film with a flat surface Is provided.
197 is a schematic representation of a front light using a single light source having opposite mirrors and quarter wave films to recycle a portion of the polarized light, wherein the free standing reflective polarizer beam splitter film having a curved surface Is provided.
198 is a schematic view of a method of manufacturing a front illumination in which a curved reflective beam splitter film having a flat surface as shown in FIG. 197 is disposed in a transparent intermediate body, wherein a reflective beam splitter film is formed and arranged The upper and lower film holders are used and a portion of the polarized light is recycled.
199 is a schematic illustration of a front light, for use with a dual light source, produced using the method illustrated in FIG. 198 and in which a portion of the polarized light is recirculated;
200 schematically depicts a curved free standing reflective beam splitter film supported at the edge in a first step of a method of casting a solid frontlight;
201 is a schematic view of a hole for injecting transparent casting material and venting air in a method for casting solid front lighting.
202 schematically illustrates the casting of the upper portion of the cast solid front light.
Figure 203 is a schematic diagram showing the use of a flat transparent sheet to planarize the top of cast solid front lighting.
204 is a flowchart of a method of fabricating solid front lighting by assembly.
205 is a flowchart for a method of manufacturing solid front lighting by casting;
Figure 206 is a flow chart of a method for manufacturing a solid film holder using a multistage molding process.
207 shows an embodiment of a local communication clock;
Figure 208 illustrates one embodiment of a telecommunications clock that interfaces with a point-of-service (POS) device that supports point-to-point communication.
Figure 209 illustrates an embodiment of a point-of-service (POS) device that supports near-field communication and a short-range communication clock that interfaces with a user's smartphone.

The present disclosure relates to eyepiece electro-optics. The eyepiece may include a projection optics suitable for projecting an image on a see-through or translucent lens to allow the wearer of the eyepiece to view the displayed image as well as the surrounding environment. A projection optics, also referred to as a projector, may include an RGB LED module using a field sequential color. In a field sequential color, a single full color image is displayed on a color basis based on the primary colors of red, green, and blue, which are individually imaged by a liquid crystal on silicon (LCoS) Can be decomposed into color fields. Since each color field is imaged by the optical display 210, the corresponding LED color is turned on. When these color fields are displayed in rapid succession, a full color image can be seen. In the field sequential color illumination, the projected image on the eyepiece can be adjusted for any chromatic aberration by shifting the red image for the blue and / or green image. The image can then be reflected into a two surface freeform waveguide, wherein the image light is reflected by the inner total reflection (TIR) until the user reaches the active viewing area of the lens viewing the image, ≪ / RTI > A processor, which may include memory and an operating system, may control the LED light source and the optical display. The projector may also include or be optically coupled to a display coupling lens, a condenser lens, a polarization beam splitter, and a field lens.

Referring to Figures 123A and 123B, a processor 12302 (e.g., a digital signal processor) is operatively coupled to display sequential frames for display of images through the display component 12328 of the eyepiece 100 (e.g., an LCOS display component) (12324). In embodiments, the sequential frames 12324 may be generated with or without the display driver 12312 as an intermediate component between the processor 12302 and the display component 12328. [ For example, referring to FIG. 123A, a processor 12302 may include a frame buffer 12304 and a display interface 12308 (e.g., mobile industry processor interface (MIPI)) along with a display serial interface (DSI) . Display interface 12308 may provide per pixel RGB data 12310 to display driver 12312 as an intermediate component between processor 12302 and display component 12328, RGB data 12310 per pixel to generate separate full frame display data for red 12318, green 12320 and blue 12322 and thus display sequential frames 12324 To the display component 12328. The display driver 12312 may provide a timing signal for synchronizing the transfer of the entire frames 12318, 12320 and 12322 to the display component 12328 as display sequential frames 12324, have. In another example, referring to Figure 123B, display interface 12330 may display full frame display data for red 12334, green 12338, and blue 12340 as display sequential frames 12324 as a display component 12328 in order to bypass the display driver 12312. In addition, a timing signal 12332 may be provided directly from the display interface 12330 to the display component. This configuration can provide significantly lower power consumption by eliminating the need for a display driver. Not only can this direct panel information eliminate the need for drivers, but it also simplifies the overall logic of the configuration and reshapes the panel information from the pixels, generating pixel information from the frame, It is possible to eliminate redundant memory.

Referring to Figure 186, in embodiments, to improve the yield of the LCoS + ASIC package 18600, an ASIC may be mounted on a flexible printed circuit (FPC) 18604 having a stiffener on the top side . The upper side stiffening plate does not increase the thickness of the entire package when the length is the same as the ASIC. The FPC is connected to a standard LCoS (not shown) such as LCoS on a glass fiber reinforced epoxy laminate (FR4) (18608) through a connector 18602 such as a zero insertion force (ZIF) connection or a board to board connector for more pins. Can be connected to the package. Pressure sensitive adhesives can be used to bond the ASIC gusset plate (s) and LCoS to the FPC.

Referring to FIG. 1, an exemplary embodiment of an augmented reality eyepiece 100 may be illustrated. It will be appreciated that while embodiments of the eyepiece 100 may not include all of the elements shown in FIG. 1, other embodiments may include additional or different elements. In embodiments, the optical elements may be embedded in the arm portions 122 of the eyepiece frame 102. Images can be projected onto at least one lens 104 disposed at the opening of the frame 102 using the projector 108. [ One or more projectors 108, such as nano projectors, pico projectors, micro projectors, femto projectors, laser-based projectors, holographic projectors, etc., may be disposed in the arm portion of the eyepiece frame 102. In embodiments, while both lenses 104 may be transparent or translucent, in other embodiments, only one lens 104 is translucent and the other is opaque or none. In embodiments, two or more projectors 108 may be included in the eyepiece 100.

1, eyepiece 100 also includes at least one articulating ear bud 120, a wireless transceiver 118, and an LED light engine (not shown). And may include a heat sink 114 that absorbs the heat of the heat source to cool it and allow it to operate at full brightness. There are also at least one open multimedia applications processors (TI OMAP4) 112 and a flex cable 110 with an RF antenna, all of which will be further described herein.

In one embodiment, referring to FIG. 2, the projector 200 may be an RGB projector. The projector 200 may include a housing 202, a heat sink 204, and an RGB LED engine or module 206. The RGB LED engine 206 may include an LED, a dichroic, a concentrator, and the like. A digital signal processor (DSP) (not shown) converts an image or video stream to control signals such as voltage drop / current correction, pulse width modulation (PWM) signals, etc. to control the intensity, duration, can do. For example, the DSP can control the duty ratio of each PWM signal to control the average current flowing through each LED generating a plurality of colors. The still image co-processor of the eyepiece can utilize noise filtering, image / video stabilization, and face detection, and can perform image enhancement. The audio back-end processor of the eyepiece can utilize buffering, SRC, equalization, and the like.

The projector 200 may include an optical display 210, such as an LCoS display, and a number of components such as shown. In embodiments, the projector 200 may be designed with a single panel LCoS display 210, but a three panel display may also be possible. In a single panel embodiment, the display 210 is sequentially illuminated in red, blue, and green (also referred to as field sequential color). In other embodiments, the projector 200 may be a back-lit liquid crystal display, a front-lit LCD, a transflective LCD, an organic light emitting diode (OLED) Alternative optical display technologies such as FED (field emission display), FLCOS (ferroelectric LCoS), liquid crystal technology mounted on sapphire, transparent liquid crystal microdisplay, quantum dot display,

In various embodiments, the display includes a CMOS-style pixel sensor at the junction between the 3D display, LCD, thin film transistor LCD, LED, LCOS, ferroelectric liquid crystal on silicon (FLCOS) display, CMOS display, OLED, QLED, (OLED) array, transmissive LCoS display, CRT display, VGA display, SXGA display, QVGA display, display with video based gaze tracker, display with exit pupil expanding technology, An asahi film display, a free form optics display, an XY polynomial combiner display, a light guide transfer display, an amoled display, and the like. In embodiments, the display may be a holographic display that allows the eyepiece to display an image from the image light source as a hologram. In embodiments, the display may be a liquid crystal reflective microdisplay. Such a display can include a polarization optic and can improve brightness compared to a particular OLED microdisplay. In embodiments, the display may be a free form prism display. The free-form prism display can achieve the 3D stereoscopic function. In embodiments, the display may be similar or identical to the display described in Cannon Patent No. 6,384,983 and / or Olympus Patent No. 6,181,475. In still other embodiments, the display may include a video based gaze tracker. In embodiments, the light beam of the infrared light source may be split and expanded into the exit pupil expander (EPE) to produce a collimated beam from the EPE towards the eye. A miniature video camera can capture the cornea and the eye line direction can be calculated by locating the glint of the pupil and infrared beam. After user calibration, the data from the gaze tracker can reflect the user's focus point in the displayed image that can be used as an input device. Such a device may be similar to that provided by the Nokia Research Center in Tampere, Finland. In addition, in embodiments, the display may include an exit pupil expander that enlarges the exit pupil to deliver the image to a new location. Thus, only a thin transparent plate may be positioned in front of the user's two eyes, and the image light source may be located elsewhere. In still other embodiments, the display may be an off axis optics display. In embodiments, such a display may not coincide with the mechanical center of the aperture. This can prevent the secondary optical element, the instrument package and / or the sensor from interfering with the primary aperture, and provide access to the instrument package and / or sensors at the focus. For example, an amoled (active-maxtrix organic light-emitting diode) display can use a pixel design called Nouvoyance's PenTile, which transmits more light in two ways. First, red, blue, and green subpixels are larger than in conventional displays. Second, one subpixel is transparent for every four pixels. This means that the backlight uses less power and can emit brighter. A smaller number of subpixels will usually mean a lower resolution, but the PenTile display uses a sub-pixel of about 1/3 of the RGB stripe panel, Pixels. The PenTile display also uses image processing algorithms to determine the brightness of the scene and automatically dims the backlight for darker images.

In order to overcome the limitations of the prior art described above, the present disclosure provides an integrated array of switchable mirrors to the waveguide that can be used sequentially to provide a progressive scan of the portions of the image over the display field of view . By rapidly switching mirrors sequentially from reflection to transmission, images can be provided to the user without perceptible flicker. Since each switchable mirror is in a more transmissive state than the reflective state, the array of switchable mirrors appears to the user as transparent and also presents the displayed image to the user.

Presenting light from an image light source by a wave guide is well known to those skilled in the art and will not be discussed herein as such. An exemplary discussion of delivering light from waveguides and image light sources to the display area is provided in U.S. Patent Nos. 5076664 and 6829095. The present disclosure includes a method and apparatus for redirecting image light in a waveguide to provide an image to a user, wherein the image light in the waveguide is provided from an image light source.

Figure 125 illustrates a waveguide display 12508 having an integral array of convertible mirrors 12508a through 12508c for redirecting light from an image light source 12502 that is transmitted through waveguide 12510 to provide a user with image light 12504, Device 12500 in accordance with an embodiment of the present invention. In the present disclosure, three switchable mirrors 12508a through 12508c are shown, but the array may include a different number of switchable mirrors. The switchable mirrors shown in Figure 125 are electrically switchable mirrors including liquid crystal switchable mirrors. A cover glass 12512 is provided to include the liquid crystal material in the thin layers shown as switchable mirrors 12508a through 12508c. Figure 125 also shows power wires 12514 and 12518. [

The integral array of waveguide 12510 and switchable mirrors 12508a through 12508c may be made of plastic or glass material, as long as it is reasonably flat. Thickness uniformity is not as important as in most liquid crystal devices, since a switchable mirror has a high reflectivity. The structure of the convertible liquid crystal mirror is described in U.S. Patent No. 6,999,649.

Figures 126 and 127 show sequential aspects of the present disclosure in that only one of the switchable mirrors in the array at a time is in a reflective state and the other switchable mirrors in the array are then in a transmissive state. Figure 124 illustrates that the first switchable mirror 12508a is in a reflective state and thus diverts the light from the image light source 12502 into an image light 12504 that presents a portion of the image to the user. The other switchable mirrors 12508b and 12508c are in the transmissive state. Figure 124 further illustrates a waveguide 12410. [

In Figure 126, switchable mirrors 12508a and 12508c are in the transmissive state, while switchable mirror 12508b is in the reflective state. This condition provides the user with an image light 12600 having a portion of the image associated therewith. Finally, in Figure 127, switchable mirrors 12508a and 12508b are in a transmissive state, while switchable mirror 12508c is in a reflective state. This last condition provides the user with video light 12700 having a portion of the video associated therewith. After this last condition, this sequence is repeated as shown in FIG. 124, followed by what is shown in FIG. 125, and then as shown in FIG. 126, to provide progressive scanning of the image . This sequence continues to be repeated while the user is viewing the displayed image. As such, all of the light from the image light source 12502 is redirected by a single switchable mirror at any given time in the sequence. The image light source may operate continuously while the switchable mirrors provide progressive scanning of the image light 12504 over the field of view. If the image light is brighter or there is a different color balance for the different switchable mirrors, then the image light source may be adjusted for compensation, or the brightness or color balance of the image light source may be synchronized with the switching sequence of the array of switchable mirrors Lt; / RTI > In another embodiment of the present disclosure, the switching order of the switchable mirrors is changed to provide the user with an interlaced image such as 1, 3, 2, 4 repeatedly for the array of four switchable mirrors. .

Figure 128 shows another embodiment of the present disclosure in which an integral array of mechanically driven switchable mirrors is provided. In this case, the switchable mirrors in waveguide display device 12800 include prisms 12804a through 12804c, respectively, which are moved to provide alternating optical contact with voids or surfaces 12810a through 12810c . As shown in FIG. 128, prism 12804a has been moved downward to provide air gap so that surface 12810a is a reflective surface that is operated by total internal reflection. Simultaneously, prisms 12804b and 12804c were pushed upward to provide optical contact with surfaces 12810b and 12810c, respectively, such that surfaces 12810b and 12810c are transmissive. This condition redirects the light from the image light source 12502 to image light 12802 that presents a portion of the image to the user. In this embodiment, the switchable mirror moves to an internal total reflection with a reflectance of almost 100% from an optical contact with a transmittance of almost 100%. 128 also shows a power line 12812, a mount and a common ground connection 12814, and micro actuators 12818a through 12818c.

129 and 130 show other conditions in the sequence for the mechanically driven switchable mirrors in the switchable mirror array. In Figure 129, prisms 12804a and 12804c are pushed upward to provide optical contact with surfaces 12810a and 12810c, respectively, thereby causing a transmission state for light from image light source 12502 to provide. At the same time, the light from the image light source 12502 is redirected and the prism 12804b is moved downward to create a void in the surface 12810b to become the image light 12900 that presents the relevant portion of the image to the user . In the final step of the sequence shown in Figure 130, prisms 12804a and 12804b (respectively) are provided to provide optical contact with surfaces 12810a and 12810b, respectively, so that light from the image source passes through surface 12810c ) Is pushed upward. To provide a cavity in surface 12810c such that surface 12810c becomes a reflective surface with internal total reflection and light from image light source 12502 is redirected to image light 13000 having a portion of the image associated therewith The prism 12804c is moved downward.

In the previous discussion, the conditions for total internal reflection are based on the optical properties of the material and air of the waveguide 12808, as is known to those skilled in the art. In order to obtain a 90 degree reflection as shown in Figures 128 to 130, the refractive index of the waveguide 12808 must be greater than 1.42. The surfaces of the prisms 12804a through 12804c are spaced apart from the surfaces of the surfaces 12810a through 12810c by a distance between the surfaces of the prisms 12804a through 12804c and the surfaces 12810a through 12810c, Lt; / RTI > Finally, in order for the light from the image light source 12502 to travel through the waveguide 12808 and the prisms 12804a to 12804c without being deflected at the interfaces, the refractive index of the prisms 12804a to 12804c, Should be within about 0.1 of the index of refraction of the light source 12808.

131A and 131B illustrate examples of waveguide assembly 13102 having an array of convertible mirrors as included in this disclosure. Figure 131a shows a side view of what the waveguide assembly 13102 is on the user's head where the long axis of the array of switchable mirrors is oriented vertically such that the video light 13100 is directed to the user's eye. Figure 131b shows an overhead view of the waveguide assembly 13102 at the user's head where the short axis of the array of switchable mirrors 13104 can be seen and the image light 13100 And is provided to the user's eye 13110. 131A and 131B, the view provided by the video light 13100 can be clearly seen. In Figure 131b, the respective portions of the image, such as those provided by the different switchable mirrors in the array, may also be seen. Figure 131b also illustrates one embodiment of a waveguide assembly 13102 that includes an image light source 13108 wherein the image light source 13108 is a light source that is optically coupled to a light source such as a LCOS display or a miniature display such as an LCD display The light is later transmitted to the switchable mirrors by a waveguide, where the light is redirected by the switchable mirrors to become the image light 13100 presented to the user's eye 13110 I have.

In order to reduce the perception of the image flicker by the user since the switchable mirrors are operated to provide sequential portions of the image to the user, the switchable mirror sequence is preferably operated faster than 60 Hz. In this case, each of the n switchable mirrors in the array is in the (1/60) X (n-1) / n-second reflective state for each cycle of the sequence, and is in the transmission state for n seconds. As such, each switchable mirror is in a transmission state for a greater portion of each cycle in the sequence than it is in the reflective state, so that the user perceives the array of switchable mirrors as relatively transparent.

In another embodiment of the present disclosure, the integral array of switchable mirrors has more switchable mirrors than are needed to cover the display area. Additional switchable mirrors are used to provide control for different users with different eye gaps (also called interpupillary distances). In this case, the switchable mirrors used to present the image to the user are adjacent to each other to provide a continuous image area. Switchable mirrors at the edge of the array are used depending on the user's eye spacing. As an example illustrated in Figures 132A-132C, the array 13200 has seven switchable mirrors each 3 mm wide. During use, five adjacent switchable mirrors are used to provide a 15 mm wide display area 13202a-13202c with an adjustment of +/- 3 mm for the eye distance. In the case of the narrow eye pitch shown in Figure 132A, five switchable mirrors are used towards the inner edge for display, whereas two outer switchable mirrors are not used. In the case of the wide eye interval shown in Figure 132c, five switchable mirrors are used towards the outer edge for display, whereas two inside switchable mirrors are not used. The middle case is shown in Figure 132B, in which case five central switchable mirrors are used and the outer and inner switchable mirrors are not used. In this description, the term "unused" refers to that the switchable mirror is maintained in the transmissive state, while other switchable mirrors are used between the transmissive state and the reflective state in the repeat sequence.

Yes

In the first example, a liquid crystal switchable mirror with a fast response as provided by Kent Optronics Inc. of Hopewell Junction, NY is used (http://www.kentoptronics.com/). The waveguide is made of glass or plastic, and the liquid crystal is contained in the space between the layers so that the liquid crystal is 5 micrometers thick. The cover glass comprises liquid crystal on the outer surface. The response time is 10 milliseconds, and the reflectance is 87% in the reflected state and the transmittance in the transmitted state is 87%. Three switchable mirrors can be driven in a sequence operating at 30 Hz. If the switchable mirrors are 5 mm wide, then a 15 mm wide display area corresponding to a field of view of 38 degrees with an eye at 10 mm from a waveguide with an 8 mm wide eyebox is provided.

In a second example, an array of mechanically driven prisms made of glass or plastic with a refractive index of 1.53 is provided, and the waveguide is made of the same material with a refractive index of 1.53. The surfaces of the prisms are polished to provide a flatness of less than one micrometer and a piezoelectric micro-actuator is used to move the prisms from the transmissive state to the reflective state by about 10 micrometers. The waveguide is shaped to provide a flatness of less than one micrometer on the surfaces mated with the prisms. Five switchable mirrors can be driven by the piezoelectric actuator to operate at 100 Hz in the sequence. Piezoelectric micro-actuators are available from Steiner & Martins Inc. of Miami, Florida, USA (http://www.steminc.com/piezo/PZ_STAKPNViewPN.asp?PZ_SM_MODEL=SMPAK155510D10), micro actuators in a 5X5X10mm package driven at 150V Providing a 10 micrometer shift with a force of over 200 pounds. An array of five prisms each 5 mm wide is used to provide a display area of 25 mm wide corresponding to a field of view of 72 degrees with an eye at 10 mm from a waveguide having an 8 mm wide eye box. As another alternative, a display area of 15 mm width (view of 38 degrees) can be moved by +/- 5 mm in the lateral direction to adjust the spacing between different binaries for different users Only three prisms are used at a time to provide.

In embodiments, the waveguide display system may include an image light source that provides image light from the displayed image, a waveguide that transmits the image light to the display area, and a display that allows the user to view the displayed image Lt; RTI ID = 0.0 > mirrors. ≪ / RTI > In embodiments, the switchable mirrors may be electrically driven. In embodiments, the switchable mirrors may be mechanically driven. In further embodiments, a micro-actuator may be used to mechanically drive the switchable mirrors. In addition, the micro-actuator may be piezoelectric. The switchable mirrors can be switched between the transmissive state and the reflective state to provide a portion of the image light through the display area in a progressive scan.

In embodiments, a method of providing an image displayed from a waveguide includes providing image light from an image light source to the waveguide, providing an integrated array of convertible mirrors in the waveguide over the display area, and providing a portion of the image light And sequentially moving the switchable mirrors between the transmissive state and the reflective state for providing across the display area with progressive scanning.

In yet another embodiment, a waveguide display system with interpupillary adjustment comprises an image light source providing image light from the displayed image, a waveguide delivering the image light to the display area, and an image light from the waveguide Lt; RTI ID = 0.0 > a < / RTI > display to a display. In addition, the array of switchable mirrors may have more switchable mirrors than needed to cover the display area, and the switchable mirrors at the edges of the array may be used to provide a display area that matches the user & .

The eyepiece can be powered by any power supply, such as battery power, solar power, line power, and the like. The power source may be integral to the frame 102 or disposed outside the eyepiece 100 and may be in electrical communication with the powered elements of the eyepiece 100. For example, a solar energy collector may be placed on frame 102, on a belt clip, or on a guitar. Battery charging can be done in a wall charger, in a wall charger, in a belt clip, in an eyepiece case, in a guitar, using a wall charger.

The projector 200 may include a heat sink 204 and a heat sink 204 to ensure non-vibration mountings on the LED light engine, the hollow tapered light tunnel 220, the diffuser 212, And may include an LED light engine 206 that may be mounted on a holder 208. The hollow tunnel 220 helps to homogenize rapidly varying light from the RGB LED light engine. In one embodiment, the hollow light tunnel 220 comprises a silver coating. The diffuser lens 212 further homogenizes and mixes the light before it reaches the condenser lens 214. The light exits the condenser lens 214 and then enters a polarizing beam splitter (PBS) 218. In the PBS, the LED light is propagated and split into polarization components before being refracted into the viewing lens 216 and the LCoS display 210. The LCoS display provides images to the microprojector. The image is then reflected from the LCoS display and passed through the polarizing beam splitter again and then reflected 90 degrees. Thus, the image exits the micro projector 200 in the middle of the micro projector. The light then travels to a coupling lens 504, which is described below.

2 illustrates one embodiment of a projector assembly with other supporting figures such as those described herein, it will be appreciated by those skilled in the art that other configurations and optical techniques may be utilized. For example, rather than using a reflective optical system, a transparent structure, such as having a sapphire substrate, can be used to implement the optical path of the projector system, thus changing optical components such as a beam splitter, And / or omitted. The system may have a backlight system, where the LED RGB triplet may be a light source that is directed to pass light through the display. As a result, the backlight and display may be mounted adjacent to the waveguide, or there may be columnizing / directing optics after display to allow light to enter the optical system properly. If there is no direct optical system, the display may be mounted on the top, side or the like of the waveguide. In one example, a small transparent display may be implemented on a transparent substrate (e.g., sapphire) with a silicon active backplane, transparent electrodes controlled by a silicon active backplane, a liquid crystal material, a polarizer, and the like. The function of the polarizer may be to correct the polarization of the light passing through the system to improve the contrast of the display. In another example, the system may be configured to provide a membrane-mirror light shutter in any form of optical path, such as a micro-channel spatial light modulator based on a micro-electromechanical system (MEMS) A spatial light modulator that imposes a spatially varying modulation of the spatial light modulator. The system also includes a tunable optical filter (e.g., having a deformable membrane actuator), a high angular deflection micro-mirror system, an individual phase optical element discrete phase optical elements, and the like.

In other embodiments, the eyepiece may utilize an OLED display, a quantum dot display, or the like that provides higher power efficiency, brighter display, less expensive components, and the like. In addition, display technologies such as OLEDs and quantum dot displays can enable flexible display and thus enable greater packaging efficiency that can reduce the overall size of the eyepiece. For example, OLED and quantum dot display materials may be printed on a plastic substrate through stamping techniques, thus creating a flexible display component. For example, an OLED (organic LED) display can be a flexible low power display that does not require backlighting. This can be a curved surface as in an ordinary spectacle lens. In one embodiment, the OLED display may be a transparent display or may provide it. In embodiments, the high modulation transfer function enables a combination of resolution levels and device size (e.g., eyeglass frame thickness) that were previously not achievable.

82, the eyepiece may utilize a planar lighting fixture 8208 in conjunction with reflective display 8210, wherein light source (s) 8202 is coupled to the edge of planar lighting fixture 8208, Where the planar side of the flat light fixture 8208 illuminates a reflective display 8210 that provides an image of the content to be presented to the wearer's eye 8222 via transfer optics 8212. [ In embodiments, reflective display 8210 may be a liquid crystal display (LCD), a liquid crystal on silicon (LCoS), a cholesteric liquid crystal, a guest-host liquid crystal, a polymer dispersed liquid crystal, a phase delay liquid crystal, Technology. In other embodiments, the reflective display 8210 may be formed of any suitable material, including, but not limited to, electrophoretic, electrofluidic, electrowetting, electrokinetic, cholesteric liquid crystals, And may be a bistable display such as any other bistable display. Reflective display 8210 may also be a combination of LCD technology and bistable display technology. In embodiments, a coupling 8204 between the light source 8202 and the 'edge' of the planar lighting fixture 8208 is made through the other surfaces of the planar lighting fixture 8208, followed by the initial top surface, Surface, or the like, into the plane of the planar lighting fixture 8208. [ For example, light may enter the planar lighting fixture from the top surface, but enter the 45 ° facet so that the light is bent into the plane direction. In an alternative embodiment, such bending in the direction of the light may be realized with an optical coating.

In one example, the light source 8202 may be an RGB LED light source (e.g., an LED array) that is coupled 8204 directly to the edge of the planar lighting fixture. The light entering the edge of the planar lighting fixture can then be directed towards the reflective display for imaging, as described herein. The light may enter the reflective display to form an image and then be redirected through the planar lighting fixture again, such as by a reflective surface at the back of the reflective display. The light then passes through the image to the wearer's eye 8222, e.g., through the lens 8214, is reflected by the beam splitter 8219 onto the reflective surface 8220, and then through the beam splitter 8218, 8222 to the delivery optics 8212. [ Although transmission optics 8212 are described in connection with 8214, 8218, and 8220, those skilled in the art will appreciate that transmission optics 8212 may be used in a variety of other configurations, including more complex or simpler configurations than those described herein, It will be appreciated that any transmission optics configuration may be included. For example, due to the different focal lengths in the field of view lens 8214, the beam splitter 8218 will be able to bend the image directly into the eye, thus omitting the curved mirror 8220 and achieving a simpler design implementation . In embodiments, light source 8202 may be an LED light source, a laser light source, a white light source, or the like, or any other light source known in the art. The optical coupling mechanism 8204 may be a direct coupling between the light source 8202 and the planar lighting fixture 8208 or may be through a coupling medium or mechanism such as a waveguide, an optical fiber, a light pipe, a lens, . Planar lighting fixture 8208 can receive light and redirect it to the planar side of its structure through interference gratings, optical defects, scatter features, reflective surfaces, refractive elements, and the like. Planar lighting fixture 8208 may be a cover glass on reflective display 8210 for reducing the combined thickness of reflective display 8210 and planar lighting fixture 8208, and the like. The planar lighting fixture 8208 includes a planar lighting fixture 8208 that is disposed on the side closest to the delivery optics 8212 to extend the cone angle of the image light as it travels through the planar lighting fixture 8208 to the delivery optics 8212. [ Lt; RTI ID = 0.0 > diffuser < / RTI > Transfer optics 8212 may include a plurality of optical elements, such as a lens, a mirror, a beam splitter, or any other optical transfer element known in the art.

83 provides one embodiment of an optical system 8302 for an eyepiece 8300 and includes an initial diverging lens 8312 for presenting an image to an eye box 8320 where the wearer's eye receives the image, A planar illumination facility 8310 and a reflective display 8304 that are mounted on a substrate 8304 that interfaces through a transmissive optical system 8212 that includes a spherical mirror 8314 and a spherical mirror 8314. In one example, the flat beam splitter 8314 may be a wire grid polarizer, a metal partially transmitting mirror coating, and the like, and the spherical reflector 8318 may be a series of dielectric coatings Lt; / RTI > In another embodiment, the coating on the spherical mirror 8318 may be a thin metal coating that provides a partially transmissive mirror.

84 includes a configuration utilizing a ferroelectric light-wave circuit (FLC) 8404, including a configuration that utilizes coupling of a laser light source 8402 to a planar lighting fixture 8408 via waveguide wavelength converters 8420 and 8422. In one embodiment, Where the planar lighting fixture 8408 utilizes a grating technique to provide light coming from the edge of the planar lighting fixture to a planar surface facing the reflective display 8410 . The image light from the reflective display 8410 is then redirected through the planar lighting fixture 8408 through the hole 8412 in the support structure 8414 to the delivery optics. Because this embodiment uses laser light, FLC also uses optical feedback to reduce spots from the laser by extending the laser spectrum, as described in U.S. Patent No. 7265896. [ In this embodiment, the laser light source 8402 is an IR laser light source, wherein the FLC combines the beams into RGB, and the back reflection, which hopes the laser light to produce an extended bandwidth, provides spot suppression. In this embodiment, spot suppression occurs in waveguide 8420. Laser light from the laser light source 8402 is coupled to the planar illumination facility 8408 through a multi-mode interference combiner (MMI) Each laser light source port is arranged such that the light traversing the MMI coupler is superimposed on one output port to the planar illumination facility 8408. [ The grating of the planar lighting fixture 8408 produces uniform illumination for the reflective display. In embodiments, the grating elements may be of a very fine pitch (e.g., interference measurement (e.g., to measure light) to produce illumination for the reflective display, which is reflected back to the very low scattering from the grating as the light travels through the planar lighting fixture interferometric) can be used. That is, the light is aligned so that the grating is almost completely transparent. Note that the optical feedback used in this embodiment is due to the use of a laser light source, and when the LED is used, spot suppression may not be necessary since the LED is already sufficiently wideband.

In one embodiment of the optical system, the use of a planar lighting fixture 8502 comprising an optically defective configuration-in this case, a "grooved" configuration-is shown in FIG. In this embodiment, the light source (s) 8202 is directly coupled (8204) to the edge of the planar illumination facility 8502. The light then travels through the planar lighting fixture 8502 and meets the small grooves 8504A through 8504D in the planar lighting fixture material-for example, grooves in a piece of PMMA (Poly-methyl methacrylate). In embodiments, the grooves 8504A through 8504D may be spaced as they move away from the input port (e.g., less progressive from 8504A to 8504D, ' aggressive ' , The pitch can change, and so on. The light is then redirected by the grooves 8504A through 8504D towards the reflective display 8210 as an array of non-coherent light sources to produce sectoral rays traveling toward the reflective display 8210, The light source 8210 is far enough away from the grooves 8504A through 8504D to create an illumination pattern from each of the overlapping grooves to provide uniform illumination of the area of the reflective display 8210. [ In other embodiments, there may be an optimal spacing for the grooves, where the number of grooves per pixel on the reflective display 8210 may be increased (to fill more) to make the light more coherent Which, in turn, produces a lower contrast in the image provided to the wearer, so that more grooves interfere in the provided image. While this embodiment has been discussed in the context of grooves, there may be other optical defects, such as dots.

In embodiments, referring to FIG. 86, counter-ridges 8604 (or 'anti-ridge') 8604, such as in a 'snap-on' ridge assembly 8602, -groove '] may be attached within the grooves of the planar lighting fixture. Opposing ridges 8604 are disposed in the grooves 8504A through 8504D such that there is a gap between the groove side wall and the opposing ridge side wall. This pore provides a defined refractive index change perceived by the light as it travels through the planar lighting fixture, which enhances the reflection of light at the groove sidewalls. The attachment of the opposing ridges 8604 reduces the aberrations and deflections of the image light caused by the grooves. That is, the image light reflected from the reflective display 8210 is refracted by the groove sidewalls, thereby changing direction due to Snell's law. By providing opposing ridges in the grooves, when the sidewall angle of the groove coincides with the sidewall angle of the opposing ridge, the refraction of the image light is compensated, and the image light is redirected toward the transmission optical system 8212.

87, the planar lighting fixture 8702 may be a laminate structure produced with a plurality of laminate layers 8704, wherein the laminate layers 8704 have alternately different refractive indices. For example, the flat lighting fixture 8702 can be cut along two diagonal planes 8708 of the laminated sheet. In this manner, the groove-like structure shown in Figs. 85 and 86 is replaced by a laminate structure 8702. Fig. For example, the laminate sheet may consist of a similar material (PMMA 1 to PMMA 2: where the difference is the molecular weight of the PMMA). As long as the layers are quite thick, they can have no interference effect and can function as a transparent plastic sheet. In the configuration shown, diagonal lamination will redirect a small proportion of the light source 8202 towards the reflective display, where the pitch of the lamination is chosen to minimize aberrations.

In one embodiment of the optical system, FIG. 88 shows a planar lighting fixture 8802 using a "wedge" configuration. In this embodiment, the light source (s) are directly coupled (8204) to the edge of the planar lighting fixture 8802. The light then travels through the planar lighting fixture 8802 and meets the sloped surface of the first wedge 8804 where the light travels toward the reflective display 8210 and then back to the lighting fixture 8802, And is then diverted onto the transfer optics through the wedge 8804 and the second wedge 8812. In addition, a multi-layer coating 8808, 8810 can be applied to the wedge to improve transfer characteristics. In one example, the wedge may consist of PMMA, the dimensions are 1/2 mm high and 10 mm wide, spanning the entire reflective display, with an angle of 1 to 1.5, and so on. In embodiments, light may be reflected several times within the wedge 8804 before passing through the wedge 8804 to illuminate the reflective display 8210. [ When the wedge 8804 is coated with the highly reflective coatings 8808 and 8810, the light beam can be reflected several times within the wedge 8804 before returning to the light source 8202 again. However, by using multilayer coatings 8808 and 8810 (made of SiO ₂ , niobium pentoxide, and the like) on the wedge 8804, light can be directed to illuminate the reflective display 8210. Coatings 8808 and 8810 can be designed to reflect light over a wide angular range of light at a given wavelength, yet transmit light within a specific angular range (e.g., a set-out angle). In embodiments, this design may allow light to be reflected in the wedge until it reaches the transmission window to be presented to the reflective display 8210, in which case the coating is then allowed to transmit . The angle of the wedge directs light from the LED illumination system to uniformly irradiate the reflective image display to produce an image reflected through the illumination system. By providing light from a light source 8202 such that broad cone angle light enters the wedge 8804, different light rays of light will reach the transmission window at different locations along the length of the wedge 8804, Uniform illumination of the surface of the wearer's eye 8210 is provided so that the image provided to the wearer's eye is determined by the image content in the image.

In embodiments, a see-through optics system including a planar illumination fixture 8802 and a reflective display 8210 as described herein may be used with an eyepiece or the like as described herein Helmets (e.g., military helmets, pilot helmets, bicycle helmets, motorcycle helmets, deep sea helmets, space helmets, etc.), as well as any head-worn devices known in the art, Goggles, eyeglasses, underwater diving masks, dust masks, respirators, hazmat head gears, virtual reality headgear, simulation devices, and the like. In addition, the optical system and protective cover associated with the tofu-wearing device may include a plurality of optical systems and protective covers in addition to the optical system and cover conventionally associated with the tofu-wearing device, Optical system. ≪ RTI ID = 0.0 > For example, the optical system may be included as a separate unit in the ski goggles to provide the user with the projected content, but the optical system may include a see-through covering of the ski goggles (e.g., Such as a transparent or tinted plastic cover that keeps the wind and eyes from getting into the user's eyes). As an alternative, the optical system may replace, at least in part, a specific optical system that is conventionally associated with a head-worn gear. For example, certain optical elements of delivery optics 8212 may replace the outer lens of an eyeglass application. In one example, the beam splitter, lens or mirror of transmission optics 8212 may replace the front lens for eyewear applications (e.g., sunglasses), and thus the curved reflective mirror 8220 may be extended to cover the glasses When the cover lens is not necessary, the front lens of the glasses is not necessary. In embodiments, a perspective optical system including a planar illumination facility 8208 and a reflective display 8210 may be positioned in the head-worn gear so as not to interfere with the function and appearance of the head-worn gear. For example, in the case of eyewear or more specifically an eyepiece, the optical system may be located proximate to the upper portion of the lens (such as the upper portion of the frame).

In embodiments, the optical assembly may be used in configurations such as a head or helmet mounted display, and / or may be a single lens, a binocular, a holographic binocular, a helmet visor, an integrated helmet and display aiming system, a helmet monolithic display aiming system, a link advanced head mounted display (AHMD), and multiple microdisplay optics. In embodiments, the optical assembly may include a telephoto lens. These lenses may be spectacle mounted or otherwise mounted. Such an embodiment may be beneficial for people with visual impairment. In embodiments, a wide-field Keplerian telescope of Eli Peli may be built into the spectacle lens. This design can use an embedded mirror inside the carrier lens to bend the optical path and power elements for higher magnification. This allows the wearer to simultaneously view the magnified view and the non-magnified view within the eyeglass format. In embodiments, the optical assembly may be used in configurations having a Q-Sight helmet mount display developed by BAE Systems, London, UK. This configuration can provide heads-up and eyes-out functions that convey situational awareness. In addition, various embodiments may use any of the optical assemblies in the configurations discussed above.

Planar lighting fixtures (also referred to as lighting modules) can provide light in a plurality of colors, including Red-Green-Blue (RGB) light and / or white light. The light from the lighting module may be directed to a 3LCD system, a Digital Light Processing (DLP®) system, a liquid crystal on silicon (LCoS) system, or other microdisplay or micro projection system. The lighting module can use wavelength combining and nonlinear frequency conversion with nonlinear feedback to the light source to provide a high brightness, long life, reduced spot or spot free light source. have. Various embodiments of the present disclosure may provide light in a plurality of colors, including Red-Green-Blue (RGB) light and / or white light. The light from the lighting module may be directed to a 3LCD system, a Digital Light Processing (DLP) system, a liquid crystal on silicon (LCoS) system, or other microdisplay or micro projection system. The illumination module described herein may be used in an optical assembly for the eyepiece 100.

One embodiment of the present disclosure includes a system comprising a laser, an LED or other light source configured to produce an optical beam of a first wavelength, a laser coupled to the laser, A planar lightwave circuit configured and configured to receive the light beam of the first wavelength and to convert the light beam of the first wavelength into the output light beam of the second wavelength, coupled to the planar lightwave circuit, And a waveguide optical frequency converter. The system can provide the laser with optically coupled feedback that is non-linearly dependent on the power of the light beam of the first wavelength.

Another embodiment of the present disclosure includes a system comprising a substrate, a light source such as a laser diode array or one or more LEDs arranged to emit a plurality of light beams of a first wavelength, A planar lightwave circuit disposed on the substrate and coupled to the light source and configured to couple the plurality of lightwaves to produce a combined light beam of the first wavelength and a planar lightwave circuit disposed on the substrate and coupled to the planar lightwave circuit, And a nonlinear optical element configured to convert the combined light beam of the first wavelength into the light beam of the second wavelength using frequency conversion. The system can provide optically coupled feedback to the laser diode array that is non-linearly dependent on the power of the combined light beam of the first wavelength.

Another embodiment of the present disclosure includes a system comprising a light source, such as a semiconductor laser array or one or more LEDs, configured to generate a plurality of light beams of a first wavelength, a light source coupled to the light source, An arrayed waveguide grating coupled to the arrayed waveguide grating and configured to output a combined light beam of a first wavelength, an arrayed waveguide grating coupled to the arrayed waveguide grating and configured to generate a combined light beam of a first wavelength using a second harmonic generation, Phase matching wavelength-converting waveguide configured to generate an output light beam of the second wavelength based on the quasi-phase matching wavelength-converting waveguide.

Power can be obtained from within the wavelength converter and fed back to the light source. The feedback power has a nonlinear dependence on the input power provided to the wavelength converter by the light source. The nonlinear feedback can convert the sensitivity of the output power from the wavelength converter to the variation of the nonlinear coefficient of the device because the feedback power increases as the nonlinear coefficient decreases. The increased feedback tends to increase the power supplied to the wavelength converter and thus alleviates the effect of the reduced nonlinear coefficient.

109A and 109B, a processor 10902 (e.g., a digital signal processor) is coupled to display sequential frames 10302 for image display through a display component 10928 of the eyepiece 100 (e.g., an LCOS display component) Lt; RTI ID = 0.0 > 10924 < / RTI > In embodiments, the sequential frames 10924 may be generated with or without the use of a display driver 10912 as an intermediate component between the processor 10902 and the display component 10928. For example, referring to FIG. 109A, a processor 10902 may include a frame buffer 10904 and a display interface 10908 (e.g., mobile industry processor interface (MIPI)) with a display serial interface (DSI) . Display interface 10908 may provide per pixel RGB data 10910 to display driver 10912 as an intermediate component between processor 10902 and display component 10928 where display driver 10912 may provide pixel data And generates separate full frame display data for red 10918, green 10920, and blue 10922, thereby generating display sequential frames 10924 To the display component 10928. The display driver 10912 may provide a timing signal for synchronizing the transfer of the entire frames 10918, 10920 and 10922 to the display component 10928 as display sequential frames 10924 have. 109b, display interface 10930 displays the full frame display data for red 10934, green 10938, and blue 10940 as display sequential frames 10924 as a display component (e.g., 10928 by directly supplying the display driver 10912 to the display driver 10912. In addition, a timing signal 10932 may be provided directly from the display interface 10930 to the display component. This configuration can provide significantly lower power consumption by eliminating the need for a display driver. Not only can this direct panel information eliminate the need for drivers, but it also simplifies the overall logic of the configuration and reshapes the panel information from the pixels, generates pixel information from the frame, You can remove the redundant memory you need.

89 is a block diagram of a lighting module according to an embodiment of the present disclosure; Illumination module 8900 includes an optical source, a combiner, and an optical frequency converter, according to one embodiment of the present disclosure. Light sources 8902 and 8904 emit optical radiation 8910 and 8914 towards input ports 8922 and 8924 of coupler 8906. Coupler 8906 has a coupler output port 8926 that emits combined radiation 8918. The combined radiation 8918 is received by an optical frequency converter 8908 and the optical frequency converter 8908 provides output light radiation 8928. The optical frequency converter 8908 may also provide feedback radiation 8920 to the combiner output port 8926. The coupler 8906 splits the feedback radiation 8920 to provide the source feedback radiation 8912 emitted from the input port 8922 and the light source feedback radiation 8916 emitted from the input port 8924 do. The light source feedback radiation 8912 is received by a light source 8902 and the light source feedback radiation 8916 is received by a light source 8904. Light emission 8910 and light source feedback radiation 8912 between light source 8902 and coupler 8906 can be used to create any of the free space and / or any of a guiding structure (e.g., optical fiber or any other light waveguide) Can be propagated in combination. Light radiation 8914, light source feedback radiation 8916, combined radiation 8918 and feedback radiation 8920 may also propagate in any combination of free space and / or guide structure.

Suitable light sources 8902 and 8904 include one or more LEDs or any light emitting light source having an emission wavelength that is affected by optical feedback. Examples of light sources include lasers and may be semiconductor diode lasers. For example, light sources 8902 and 8904 may be elements of an array of semiconductor lasers. Light sources other than lasers may also be used (e.g., an optical frequency converter may be used as the light source). 89. Although two light sources are shown in Fig. 89, the present disclosure may also be embodied as three or more light sources. Coupler 8906 is shown as a three port device generally having ports 8922, 8924, and 8926. Although ports 8922 and 8924 are referred to as input ports and port 8926 is referred to as a combiner output port, these ports may be bidirectional and may also receive or emit light radiation, as discussed above .

Coupler 8906 may comprise optical elements that define wavelength dispersive elements and ports. Suitable wavelength dispersion elements include an arrayed waveguide grating, a reflective diffraction grating, a transmissive diffraction grating, a holographic optical element, an assembly of wavelength selective filters, and a photonic band gap structure. As such, combiner 8906 may be a wavelength combiner, where each of the input ports has a corresponding non-overlapping input port wavelength range for efficient coupling to the combiner output port.

Harmonic generation, sum frequency generation (SHG), second harmonic generation (SHG), difference frequency generation, parametric generation, parametric amplification, , Parametric oscillation, three-wave mixing, four-wave mixing, stimulated Raman scattering, stimulated Brillouin scattering, stimulated emission, acoustic Various optical processes can be performed in the optical frequency converter 8908 including, but not limited to, acousto-optic frequency shifting and / or electro-optic frequency shifting.

In general, optical frequency converter 8908 receives the optical input of the optical wavelengths of the input set and provides the optical output of the optical wavelengths of the output set, where the output set is different from the input set.

The optical frequency converter 8908 may be made of a material selected from the group consisting of lithium niobate, lithium tantalate, potassium titanyl phosphate, potassium niobate, quartz, silica, silicon oxynitride, gallium Non-linear optical materials such as arsenic, lithium borate, and / or beta-barium borate. Optical interaction in the optical frequency converter 8908 can take place in a variety of structures including bulk structures, waveguides, quantum well structures, quantum wire structures, quantum dot structures, photonic bandgap structures, and / or multiple component waveguide structures.

If the optical frequency converter 8908 provides a parametric nonlinear optical process, then this nonlinear optical process is preferably phase-matched. This phase matching may be birefringent phase-matching or quasi-phase-matching. Pseudo-phase matching may include a method disclosed in Miller, U.S. Patent No. 7,116,468, the disclosure of which is incorporated herein by reference.

The optical frequency converter 8908 may also include a wavelength selective reflector for wavelength selective output coupling, a wavelength selective reflector for wavelength selective resonance, and / or a wavelength that controls the spectral response of the transducer An optional loss factor, and the like.

In embodiments, a plurality of illumination modules, such as those described in Figure 89, may be associated to form a compound illumination module.

One component of the illumination module may be a diffraction grating or a grating, which is further described herein. The diffraction grating plate may be less than 1 mm thick, but may still be sufficiently rigid to be permanently bonded in place or to replace the cover glass of the LCOS. One advantage of using a grating in an illumination module is the use of a laser illumination light source to increase efficiency and reduce power. The grating may have essentially less stray light and will allow additional options to filter out eye glow with less reduction of see through brightness due to narrow band.

90 is a block diagram of an optical frequency converter according to one embodiment of the present disclosure; 90 shows how feedback radiation 8920 is provided by an exemplary optical frequency converter 8908 that provides parametric frequency conversion. Combined radiation 8918 provides forward radiation 9002 in an optical frequency transducer 8908 that propagates to the right in Figure 90 and provides a forward radiation 9002 that is parametric radiation 9004 is generated in the optical frequency converter 8908 and emitted as output light radiation 8928 from the optical frequency converter 8908. Typically there is a net power transfer from the omnidirectional radiation 9002 to the parametric radiation 9004 as the interaction progresses (i.e., in this example, as the radiation propagates to the right). A reflector 9008 that may have a wavelength dependent transmittance to reflect (or partially reflect) the omnidirectional radiation 9002 to provide a backward radiation 9006 is disposed in the optical frequency converter 8908 , And may be disposed outside the optical frequency converter 8908 after the endface 9010. [ The reflector 9008 can be a grating, an internal interface, a coated or uncoated end face, or any combination thereof. The reflectance level of the reflector 9008 is preferably greater than 90%. A reflector located at the input interface 9012 provides purely linear feedback (i.e., feedback independent of process efficiency). The reflector located at end face 9010 provides maximum nonlinear feedback because the dependence of forward power on process efficiency is maximized at the output interface (assuming phase-matched parametric interaction).

91 is a block diagram of a laser illumination module in accordance with one embodiment of the present disclosure; In this embodiment, it is to be appreciated that while lasers are used, other light sources such as LEDs may also be used. The laser illumination module 9100 includes an array of diode lasers 9102, waveguides 9104 and 9106, star couplers 9108 and 9110, and an optical frequency translator 9114. The array of diode lasers 9102 is coupled to waveguides 9104 that serve as input ports (ports 8922 and 8924 in Figure 89) to a planar waveguide star coupler 9108 And has lasing elements. Star coupler 9108 is coupled to another planar waveguide star coupler 9110 by waveguides 9106 having different lengths. The combination of star couplers 9108 and 9110 and waveguides 9106 may be an arrayed waveguide grating and may include a wavelength coupler (e.g., coupler 8906 of FIG. 89), which provides combined radiation 8918 to waveguide 9112 )]. The waveguide 9112 provides the combined radiation 8918 to the optical frequency converter 9114. Within the optical frequency converter 9114, the optional reflector 9116 provides back reflection of the combined radiation 8918. As previously discussed with respect to FIG. 90, this back reflection provides non-linear feedback in accordance with embodiments of the present disclosure. One or more of the elements described with reference to Figure 91 may be fabricated on a common substrate using a planar coating method and / or a lithographic method, in order to reduce cost, parts count and alignment requirements .

The second waveguide can be arranged so that its core is very close to the core of the waveguide in the optical frequency converter 8908. [ As is known in the art, this configuration of waveguides serves as a directional coupler, and thus the radiation in the waveguide can provide additional radiation in the optical frequency converter 8908. [ Significant coupling may be prevented by providing radiation at wavelengths other than the wavelengths of the omnidirectional radiation 9002 or additional radiation may be coupled to the optical frequency converter 8908 at a location where the omnidirectional radiation 9002 is depleted have.

While a standing wave feedback configuration is useful where the feedback power propagates in the opposite direction along the same path along which the input power follows, a traveling wave feedback configuration may also be used. In the progressive feedback configuration, the feedback re-enters the gain medium at a different location from where the input power is emitted.

92 is a block diagram of a compound laser illumination module according to another embodiment of the present disclosure; Composite laser illumination module 9200 includes one or more laser illumination modules 9100 as described with reference to FIG. Although FIG. 92 illustrates a composite laser illumination module 9200 including three laser illumination modules 9100 for the sake of simplicity, the composite laser illumination module 9200 may include more or fewer laser illumination modules 9100 . The array of diode lasers 9120 may be an array of laser diodes, a diode laser array, and / or a semiconductor laser array that is configured to emit light radiation within the infrared spectrum (i.e., shorter wavelengths and longer wavelengths than visible light) One or more arrays 9102 of diode lasers.

The laser array output waveguides 9220 are coupled to the diode lasers in the array of diode lasers 9210 and direct the outputs of the array of diode lasers 9210 to the star couplers 9108A through 9108C. Laser array output waveguides 9220, arrayed waveguide gratings 9230, and optical frequency converters 9114A through 9114C can be fabricated on a single substrate using planar lightwave circuits, and the silicon oxynitride waveguide and / RTI > and / or a lithium tantalate waveguide.

Arrayed waveguide gratings 9230 include star couplers 9108A through 9108C, waveguides 9206A through 9206C, and star couplers 9110A through 9110C. Waveguides 9112A through 9112C provide combined radiation to optical frequency converters 9114A through 9114C and feedback radiation to star couplers 9110A through 9110C, respectively.

Optical frequency converters 9114A through 9114C may include nonlinear optical (NLO) elements (e.g., optical parametric oscillator elements and / or pseudo phase matching optical elements).

The composite laser illumination module 9200 may produce output light emission of a plurality of wavelengths. The plurality of wavelengths may be within a visible spectrum (i.e., shorter than infrared and have a longer wavelength than ultraviolet). For example, waveguide 9240A may similarly provide output light radiation between about 450 nm and about 470 nm, and waveguide 9240B may provide output light radiation between about 525 nm and about 545 nm And waveguide 9240C may provide output light radiation between about 615 nm and about 660 nm. These ranges of output light emission can be re-selected to provide satisfactory visible wavelengths (e. G., Blue, green and red wavelengths) to the human observer and recombined to produce a white light output .

Waveguides 9240A through 9240C may be fabricated on the same planar lightwave circuit as laser array output waveguides 9220, arrayed waveguide gratings 9230, and optical frequency converters 9114A through 9114C. In some embodiments, the output light radiation provided by each of waveguides 9240A through 9240C can provide optical power in the range of about 1 watts to about 20 watts.

The optical frequency converter 9114 includes a pseudo phase matched wavelength conversion waveguide configured to perform a second harmonic generation (SHG) on the combined radiation of the first wavelength to generate a second wavelength of radiation . The pseudo phase-matched wavelength conversion waveguide pumped the optical parametric oscillator integrated with the pseudo-phase matched wavelength conversion waveguide to generate a second (" third " May be configured to use wavelength radiation. The pseudo-phase matched wavelength converting waveguide can also generate feedback radiation propagating through the arrayed waveguide grating 9230 via waveguide 9112 to the array of diode lasers 9210, thereby causing the array of diode lasers 9210 Allowing each placed laser to operate at an individual wavelength determined by the corresponding port on the arrayed waveguide grating.

For example, the composite laser illumination module 9200 may be an array of diode lasers operating at wavelengths of about 830 nm nominally to produce output light radiation in the visible spectrum corresponding to any of the red, green, And may be configured using the second interface 9210.

Composite laser illumination module 9200 may optionally be configured to directly illuminate spatial light modulators without intermediate optics. In some embodiments, the compound laser illumination module 9200 operates nominally at a single first wavelength to simultaneously produce output light emissions of a plurality of second wavelengths, such as wavelengths corresponding to red, green, and blue May be constructed using an array of diode lasers (9210). Each different second wavelength may be generated by one instance of the laser illumination module 9100.

Composite laser illumination module 9200 can be configured to couple diffuse limited white light by combining multiple output wavelengths of light at a single wavelength using, for example, waveguide-selective taps (not shown) lt; / RTI > white light.

An array of diode lasers 9210, laser array output waveguides 9220, arrayed waveguide gratings 9230, waveguides 9112, optical frequency converters 9114 and frequency converter output waveguides 9240 Can be fabricated on a common substrate using manufacturing processes such as coating and lithography. A beam shaping element 9250 is coupled to the complex laser illumination module 9200 by waveguides 9240A through 9240C, as described with reference to FIG.

Beam shaping element 9250 may be disposed on the same substrate as composite laser illumination module 9200. [ The substrate may comprise, for example, a thermally conductive material, a semiconductor material, or a ceramic material. The substrate may comprise copper-tungsten, silicon, gallium arsenide, lithium tantalate, silicon oxynitride, and / or gallium nitride and may be processed using a semiconductor fabrication process including coating, lithography, etching, .

An array of diode lasers 9210, laser array output waveguides 9220, arrayed waveguide gratings 9230, waveguides 9112, optical frequency converters 9114, waveguides 9240, beam shaping elements 9250, and various related planar lightwave circuits may be passively coupled and / or aligned, and in some embodiments may be passively aligned by height on a common substrate. Each of waveguides 9240A through 9240C may be coupled to a different instance of beam shaping element 9250 rather than being coupled to a single element as shown.

Beam shaping element 9250 may be configured to shape the output light emission from waveguides 9240A through 9240C into a substantially rectangular diffraction limited light beam and also output light emission from waveguides 9240A through 9240C Can be configured to have a brightness uniformity of greater than about 95% over a nearly rectangular beam shape.

Beam shaping element 9250 may comprise an aspheric lens such as a "top-hat" microlens, a holographic element, or a light grating. In some embodiments, the diffraction limited light beam output by beam shaping element 9250 may produce substantially reduced spots or may not produce spots. The light beam output by beam shaping element 9250 can provide optical power in the range of about 1 to about 20 watts, and a substantially planar phase front.

93 is a block diagram of an imaging system in accordance with one embodiment of the present disclosure; The imaging system 9300 includes a light engine 9310, a light beam 9230, a spatial light modulator 9330, a modulated light beam 9340, and a projection lens 9350. Light engine 9310 may include a compound optical illumination module, such as a plurality of illumination modules described in FIG. 89, a composite laser illumination module 9200, described with reference to FIG. 92, or FIG. 93 May be a laser illumination system 9300 as described in reference. Spatial light modulator 9330 may be a 3LCD system, a DLP system, a LCoS system, a transmissive liquid crystal display (e.g., transmissive LCoS), a liquid-crystal-on-silicon (LCoS) array, a grating- Or other microdisplay or microprojection system or a reflective display.

The spatial light modulator 9330 may be configured to spatially modulate the light beam 9320. Spatial light modulator 9330 may be configured to provide a spatial light modulator 9330 with a video image such as that which can be displayed by a television or computer monitor on light beam 9320 to produce a modulated light beam 9340 And may be coupled to an electronic circuit configured to modulate. In some embodiments, the modulated light beam 9340 may be output from the spatial light modulator on the same side as the spatial light modulator receives the light beam 9320 using the optical reflection principle. In other embodiments, the modulated light beam 9340 may be output from the spatial light modulator on the opposite side of the spatial light modulator receiving the light beam 9320 using the optical transmission principle. The modulated light beam 9340 may optionally be coupled to a projection lens 9350. Projection lens 9350 is configured to project the modulated light beam 9340 onto a display, such as a video display screen.

The method of illuminating a video display may be performed using a composite illumination module such as a plurality of illumination modules 8900, a composite laser illumination module 9100, a laser illumination system 9200, or an imaging system 9300 . A diffraction limited output light beam is generated using a combined illumination module, a combined laser illumination module 9100, a laser illumination system 9200, or a light engine 9310. An output light beam is directed using a spatial light modulator, such as spatial light modulator 9330, and optionally a projection lens 9350. The spatial light modulator can project an image onto a display such as a video display screen.

The illumination module may be configured to emit any number of wavelengths, including one, two, three, four, five, six, or more, the wavelengths being spaced apart from one another by a variable amount, And have different power levels. The illumination module may be configured to emit a single wavelength for each light beam or to emit multiple wavelengths for each light beam. The lighting module may also include a power circuit, such as a polarization controller, a polarization rotator, a power supply, a power FET, an electronic control circuit, a thermal management system, a heat pipe, and a safety interlock And may include additional components and functionality. In some embodiments, the lighting module may be coupled to an optical fiber or a light guide-glass (e.g., BK7).

Some options for LCoS frontal lighting design include the following: 1) Wedge with MLC (Multi Layer Coating). This concept uses MLC to define specific reflection and transmission angles; 2) a wedge with a polarizing beam splitter coating. This concept works like a regular PBS cube, but works at a much lower angle (shallow angle). This can be PBS coating or wire grid film; 3) PBS Prism bar [(PBS Prism bar); These are similar to option # 2], but have a seam under the center of the panel; 4) wire grid polarizer plate beam splitter [similar to PBS wedge, but only plate and therefore generally air instead of solid glass]; And 5) a flexible film such as a 3M polarizing beam splitter consisting of alternating different plastic layers with an index of refraction that is matched in one plane direction but not otherwise in an in-plane direction polarizing beamsplitter). In the unmatched direction, a highly reflective quarter-wave stack is formed, while in the matched direction, the film functions like a transparent plastic slab. The film is laminated between glass prisms to form a wide angle PBS that provides high performance over high-speed beams throughout the visible range. The MLC wedge can be rigid and can be firmly bonded in place without voids for condensation or thermal deflection. Which can operate on a broadband LED light source. In embodiments, the MLC wedge may replace the cover glass of the LCOS for the finished module. The MLC wedge may be less than about 4 mm thick. In one embodiment, the MLC wedge may be less than 2 mm thick.

It is to be understood that the present disclosure provides for the use of frontlighting systems such as those described herein in all types of optical configurations that may (but need not necessarily) include an augmented reality eyepiece You will know. The front lighting system can be used as a component in any type of optical system as a light source of direct or indirect illumination and is particularly suitable for illumination of any type or type of optical element, optical surface, or optical sensor And is most preferred for having, for example, selectively configurable light paths, such as LCoS or liquid crystal displays, and / or for reflecting light. In some embodiments, at least a portion of the light generated by the front lighting system will be reflected to continue to pass through a portion of the front lighting system back to its final destination (e.g., eye, light sensor, etc.) In the example, none of the generated light continues to pass through the front lighting system again to its final destination. For example, the front lighting system can be redirected through the components of the front lighting system and then passed through one or more additional optical systems that condition the image light that the user's eye ultimately receives An optical device such as LCoS can be illuminated to generate an image. Any other optical system may be, or may include, one or more of a waveguide (which may be a free-form waveguide), a beam splitter, a collimator, a polarizer, a mirror, a lens, and a diffraction grating.

95 illustrates one embodiment of a LCoS frontal illumination design. In this embodiment, the light from the RGB LED 9508 illuminates the front illumination 9504, which may be a wedge, a PBS, or the like. The light impinges on polarizer 9510, is transmitted through its S state and goes to LCoS 9502, where it is reflected as image light in its P state and passes through aspheres 9512 again. An inline polarizer 9512 can again polarize the image light and / or rotate it in half-wave to the S state. The image light then hits the wire grid polarizer 9520 and is reflected toward the curved (spherical) partial mirror 9524 and passes through the 1/2 wave retarder 9522 and continues. The image light is reflected from the mirror toward the user's eye 9518 and passes through the 1/2 wave retarder 9522 and the wire grid polarizer 9520 one more time. Various examples of the forward lighting 9504 will now be described.

In embodiments, the optical assembly includes a partially reflective, partially transmissive optical element that reflects portions of the image light from the image light source and transmits scene light from a see-through view of the environment So that a combined image composed of the reflected image light and the transmitted portions of the scene light is provided to the user's eyes.

In portable display systems, it is important to provide a display that is bright, small and lightweight. Portable display systems include cellphones, laptop computers, tablet computers, and head-mounted displays.

The present disclosure provides a compact lightweight front light source for a portable display system comprising a curved or other nonplanar wire grid polarizing film as a partial reflector to efficiently deflect light from the edge light source and illuminate the reflected image light source. Wire grid polarizers are known to provide efficient reflection for one polarization state while allowing other polarization states to pass through at the same time. Although glass plate wire grid polarizers are well known in the industry and hard wire grid polarizers can be used in this disclosure, in a preferred embodiment of the present disclosure, flexible wire grid polarizers are used for curved wire grid polarizers. A suitable wire grid polarizing film is available from Asahi-Kasei E-materials Corp of Tokyo, Japan.

The edge light source provides a small form of illumination on the display, but because it is located at the edge of the image light source, the light must be deflected by 90 degrees to illuminate the image light source. In one embodiment of the present disclosure, a curved wire grid polarizing film is used as the partial reflective surface to deflect the light provided by the edge light source downward to illuminate the reflected image light source. A polarizer is provided adjacent to the edge light source to polarize the illumination light provided to the curved wire grid polarizer. Polarizers and wire grid polarizers are oriented such that light passing through the polarizers is reflected by the wire grid polarizers. Due to the quarter wave retarder film contained in the reflected image light source, the polarization of the reflected image light is in the opposite polarization state compared to the illumination light. Thereby, the reflected image light passes through the wire grid polarizing film and continues to the display optical system. By using a flexible wire grid polarizing film as a partial reflector, the partially reflective surface can be a curved surface in a lightweight structure, where the wire grid polarizer performs a dual role: a reflector for illumination light and a transparent member for imaging light. The advantage provided by the wire grid polarizing film is that it can receive the image light over a wide range of incident angles and thus the curved surface does not interfere with the image light going to the display optics. In addition, since the wire grid polarizing film is thin (for example, less than 200 micrometers), the curved shape does not distort the image light much when the image light passes through the display optical system. Finally, the wire grid polarizer has a very low tendency to scatter light, thus a high image contrast can be maintained.

136 shows a schematic diagram of a frontlighted image source 13600 of the present disclosure. The edge light source 13602 provides illumination light passing through the polarizer 13614 so that the illumination light 13610 is polarized where the polarizer 13614 can be an absorptive polarizer or a reflective polarizer. have. The polarizer state of the illumination light 13610 is oriented such that light is reflected by the curved wire grid polarizer 13608 and thereby deflects the illumination light 13610 down toward the reflective image light source 13604. Thus, the passing axis of the polarizer 13614 is perpendicular to the pass axis of the wire grid polarizer 13608. It should be noted by those skilled in the art that although FIG. 136 shows a horizontally oriented front illuminated image light source 13600, other orientations are equally possible. As mentioned earlier, a reflected image light source, such as an LCOS image light source, typically includes a quarter wave delay film, so that the polarization state of the illumination light is changed by the reflected image light source during reflection, And generally has an opposite polarization state as compared with illumination light. This change in polarization state is fundamental to the operation of all liquid crystal based displays, as is known to those skilled in the art and as described in U.S. Patent No. 4,398,805. For individual portions of the image, the liquid crystal element of the reflected image light source 13604 will cause more or less change in polarization state, and thus the reflected image light 13612, before passing through the curved wire grid polarizer , And has a mixed elliptically polarized state. After passing through the curved wire grid polarizer 13608 and any additional polarizers that may be included in the display optics, the polarization state of the video light 13612 is determined by the curved wire grid polarizer 13608, Determines the local intensity of the video light 13612 in the video displayed by the portable display system.

The flexible nature of the wire grid polarizing film used in curved wire grid polarizer 13608 allows it to be shaped into a shape that focuses illumination light 13610 onto reflective image light source 13604. The shape of the curved surface of the curved wire grid polarizer is chosen to provide uniform illumination of the reflected image light source. 136 shows a curved wire grid polarizer 13608 having a parabolic shape but may be curved to have a constant radius of curvature in order to uniformly deflect the illumination light 13610 onto the reflected image light source 13604 according to the nature of the edge light source 13602. [ radiused curves, complex spline surfaces or planes are also possible. Experiments have shown that both parabolic curves, curved surfaces of constant radius, and curved surfaces of complex splines provide more uniform illumination than flat surfaces. However, in some very thin front illuminated image light sources, a flat wire grid polarizing film can be effectively used to provide a lightweight portable display system. 138 by a side frame forming slots of suitable curved surfaces to hold the wire grid polarizing film in place, as shown in Figure 138, which shows a schematic of the front illuminated image light source assembly 13800. [ The shape of the polarizing film can be maintained. A side frame 13802 with a curved slot 13804 is shown to hold the flexible wire grid polarizing film in a desired curved shape. Although only one side frame 13802 is shown in Figure 138, two side frames 13802 will be used to support the curved shape on both sides along the other components of the front illuminated image light source. In either case, the weight is substantially lower than in prior art front lighting systems, since the majority of the front illuminated image light source of this disclosure is air and the wire grid polarizing film is very thin.

137, two or more edge light sources 13702 are arranged along two or more edges of the reflected image light source 13604, as shown in Figure 137. In a further embodiment of the present disclosure, a front illuminated image light source 13700 Is provided. Polarizers 13712 are provided adjacent to each edge light source 13702 to polarize the illumination light 13708. [ The illumination light 13708 is deflected by the curved wire grid polarizer 13704 to illuminate the reflected image light source 13604. The reflected image light 13710 then passes through the curved wire grid polarizer 13704 and continues to the display optics. The advantage of using two or more edge light sources 13702 is that more light can be applied to the reflective image light source 13604 and thereby provide brighter images.

The edge light source may be a fluorescent light, an incandescent light, an organic light emitting diode, a laser, or an electroluminescent light. In a preferred embodiment of the present disclosure, the edge light source is an array of three or more light emitting diodes. In order to uniformly illuminate the reflected image light source, the edge light source must have a substantially conical angle (e.g., the edge light source may be a Lambertian light source). In the case of a laser light source, the cone angle of the light will need to be expanded. By using an array of light sources or a plurality of edge light sources, the distribution of light onto the reflected image light source can be adjusted to provide more uniform illumination, resulting in a more uniform brightness of the displayed image.

The image light provided by the front illuminated image light source of the present disclosure enters into the display optics of the portable display system. Various display optical systems are possible depending on how the displayed image is used. For example, the display optics may be decentralized when the display is a flat screen display or, alternatively, may be refractive or diffractive when the display is a near-vision display or a head-mounted display.

Figure 139 is a flow chart of the method of the present disclosure for a portable display system having a reflected image light source. At step 13900, polarized illumination light is provided at one or more edges of the reflected image light source. In step 13902, the curved wire grid polarizer receives the illumination light and deflects it to illuminate the reflected image light source, wherein the curved surface of the wire grid polarizer is selected to enhance the uniformity of illumination of the area of the reflected image light source . In step 13904, the reflected image light source receives the illumination light, reflects the illumination light, and at the same time changes the polarization state of the illumination light corresponding to the displayed image. The image light then passes through the curved wire grid polarizer in step 13908 into the display optics. In step 13910, an image is displayed by the portable display system.

In embodiments, a lightweight portable display system having a reflective liquid crystal image light source for displaying an image includes one or more edge light sources providing polarized illumination light adjacent one or more edges of the reflective liquid crystal image light source, A curved wire grid polarizer partial reflector capable of receiving and deflecting light to illuminate a reflective liquid crystal image light source and a display optical system for receiving the image light reflected from the reflective liquid crystal image light source and displaying the image, . ≪ / RTI > In addition, the one or more edge light sources may include light emitting diodes. In embodiments, the wire grid polarizer may be a flexible film, and the flexible film may be retained in a curved shape by a side frame. In embodiments, the curved wire grid polarizer of the display system may be a curved surface of a parabolic, constant radius, or complex spline. In addition, the reflective liquid crystal image light source of the display system may be an LCOS. In embodiments, the display optics of the display system may include a diffuser, and the display system may be a flat screen display. In embodiments, the display optics of the display system may include refractive or diffractive elements, and the display system may be a near vision display or a head mounted display.

In embodiments, a method of providing images in a lightweight portable display system having a reflective liquid crystal imaging light source includes providing polarized illumination light to one or more edges of the reflective liquid crystal imaging light source, receiving the illumination light with a curved wire grid polarizer, Illuminating a reflective liquid crystal image light source by polarizing the light; reflecting the illumination light to change its polarization state with respect to an image to be displayed as a reflective liquid crystal image light source to provide image light; converting the image light into a curved wire grid polarizer Receiving the image light by the display optical system, and displaying the image. In embodiments of the method, the curved surface shape of the curved wire grid polarizer may be selected to enhance the uniformity of illumination of the reflected liquid crystal image light source. In addition, the one or more edge light sources may include light emitting diodes. In embodiments, the wire grid polarizer may be a flexible film. In addition, the flexible film can be held in a curved shape by the side frame. In embodiments of the method, the curved wire grid polarizer may be a curved surface of a parabola, a constant radius, or a complex spline. In addition, in embodiments of the method, the reflected liquid crystal image light source may be an LCOS. In embodiments, the display optics may include a diffuser, and the display system may be a flat screen display. In embodiments of the above method, the display optical system may comprise a refracting or diffractive element, and the display system may be a near vision display or a head mounted display.

96 shows an embodiment of a front illumination 9504 that includes prisms optically conjugated with a polarizer. The prisms look like a rectangular solid of two having a substantially transparent interface 9602 between them. The entities of each rectangle are diagonally bisected and a polarizing coating 9604 is disposed along the interface of the bisection. The lower triangle formed by the nested portion of the entity in the rectangle may optionally consist of a single piece 9608. The prisms may consist of B-7 or equivalent. In this embodiment, the entity in the rectangle has a square end of the size 2 mm x 2 mm. In this embodiment, the length of the middle entity is 10 mm. In an alternate embodiment, the nutrient site includes a 50% mirror 9705 surface, and the interface between the entities of the two rectangles includes a polarizer 9702 that is capable of passing light in the P state.

98 shows three versions of the LCoS forward lighting design. 98 (a) shows a wedge having an MLC (multi-layer coating). This concept uses MLC to define specific reflection and transmission angles. In this embodiment, video light in the P or S polarization state is observed by the user's eyes. 98b shows a PBS with a polarizer coating. Here, only the S-polarized image light is transmitted toward the user's eye. 98c shows a rectangular prism that removes most of the material of the prism so that the image light can be transmitted through the air as S-polarized light.

99 shows a wedge + PBS in which the polarizing coating 9902 is layered on the LCoS 9904.

100 shows two embodiments of a prism A in which light enters a short end and a prism B in which light enters along a long end. In Fig. 100a, a wedge is formed by offset bisecting an entity in a rectangle to form at least one 8.6 degree angle at the nutrient site interface. In this embodiment, by offset natures, a segment with a height of 0.5 mm on the side and another segment with 1.5 mm are obtained, through which the RGB LED 10002 transmits light. Along the nutrient site, a polarizing coating 10004 is disposed. 100b, a wedge is formed by offsetting the entity in the rectangle to form at least one 14.3 degree angle at the nutrient site interface. In this embodiment, by offset natures, a segment having a height of 0.5 mm and another segment having a thickness of 1.5 mm are obtained on the side surface, and the RGB LED 10008 transmits light through them. Along the nutrient site, a polarizing coating 10010 is disposed.

101 shows a curved PBS film 10104 illuminated by an RGB LED 10102 disposed on an LCoS chip 10108. Fig. The PBS film 10104 reflects the RGB light from the LED array 10102 onto the surface 10108 of the LCOS chip but causes the light reflected from the image chip to go towards the optical assembly and ultimately toward the user's eye without interference. The films used in this system include Asahi films, which are tri-acetate cellulose or cellulose acetate substrates (TAC). In embodiments, the film may have a calendared coating built on ridges that may be angled for a 100 nm UV embossed corrugation and incident angle of light. Asahi films can come in rolls of 20 cm wide x 30 cm long and have BEF characteristics when used in LCD lighting. Asahi films can support wavelengths from visible to IR, and can be stable up to 100 ° C.

In another embodiment, Figures 21 and 22 show an exploded view of an alternative configuration of a waveguide and a projector. In this configuration, the projector is positioned immediately behind the hinge of the arm of the eyepiece and is vertically oriented to be vertical until the initial progress of the RGB LED signal is redirected by the reflective prism to enter the waveguide lens have. The vertically arranged projection engine may have a central PBS 218, a bottom RGB LED array, a hollow tapered tunnel with a thin film diffuser to mix colors for focusing in the optical system, and a condenser lens. The PBS may have a pre-polarizer on the entrance face. The full-deflector may be arranged to transmit light of a specific polarization such as p-polarized light and to reflect (or absorb) the opposite polarized light such as s-polarized light. The polarized light can then pass through the PBS to the visual field lens 216. The purpose of the viewing lens 216 may be to produce near telecentric illumination of the LCoS panel. The LCoS display can be highly reflective and reflects the colors sequentially at the correct timing, thus displaying the image properly. Light can be reflected from the LCoS panel, and can be rotated with s-polarization for bright areas of the image. The light can then be refracted through the field lens 216 and reflected off the inner interface of the PBS to exit the projector and onto the coupling lens. The hollow tapered tunnel 220 may replace a homogenizing lenslet from other embodiments. By vertically orienting the projector and centering the PBS, space is saved and the projector can be placed in the hinge space with little chance of the moment arm being caught in the waveguide.

Light reflected or scattered from the image source of the eyepiece or from the associated optical system can go out into the environment. These light losses are perceived as 'glare' or 'night glow' by an external observer, in which case a portion of the lens or the area surrounding the eyepiece appears to glow in a darkened light environment. In the specific case of the glare as shown in Fig. 22A, the image displayed when the external observer looks from the outside can be viewed as the observable image 2202A in the display area. In terms of maintaining the privacy of the viewing image, it is also desirable to reduce glare in order to maintain the privacy of the viewing experience for the user, even in terms of making the user invisible when using the eyepiece in a dark lighting environment. The method and apparatus can reduce glare through a light control element (e.g., by a partial reflective mirror in an optical system associated with the image light source, by a polarizing optical system, or the like). For example, the light entering the waveguide can be polarized (e.g., s-polarized). The light control element may comprise a linear polarizer. The linear polarizer in the light control element is oriented with respect to the linearly polarized image light so that the second part of the linearly polarized image light passing through the partial reflection mirror is blocked and the glare is reduced. In embodiments, a lens, such as snap-fit optics, described herein, which is polarized (e.g., in this case, p-polarized) as opposed to light reflected from the user's eye, Or attached to the frame, the glare can be minimized or eliminated.

In embodiments, the light control element may comprise a second quarter wave film and a linear polarizer. The second quarter wave film converts the second portion of the circularly polarized image light into linearly polarized image light having a polarization state that is blocked by a linear polarizer in the light control element, thereby reducing glare. For example, when the light control element includes a linear polarizer and a quarter wave film, 50% of the light is blocked as unpolarized scene light coming from the external environment in front of the user is converted into linearly polarized light . The first portion of the scene light passing through the linear polarizer is linearly polarized light, which is converted into circularly polarized light by a quarter wave film. A third portion of the scene light reflected from the partially reflecting mirror inverts the circularly polarized light, which is then converted into linearly polarized light by the second quarter wave film. The linear polarizer then blocks the reflected third portion of the scene light, thereby reducing escaping light and reducing glare. 22B shows an example of a perspective display assembly having a light control element in a spectacle frame. The eyeglass section 2200B shows the components of the perspective display assembly in the eyeglass frame 2202B. The light control element is responsible for the entire perspective view the user is viewing. Support members 2204B and 2208B are shown supporting a partial reflective mirror 2210B and a beam splitter layer 2212B in the field of view of the user's eye 2214B, respectively. The support members 2204B and 2208B, together with the light control element 2218B, are connected to the eyeglass frame 2202B. Other components such as a folding mirror 2220B and a first quarter wave film 2222B are also connected to the support members 2204B and 2208B so that the combined assembly is structurally stable .

Stray light in small optical systems, such as head mounted displays, is typically caused by scattering from the side of the housing or other structures where the light meets the surface at a steep angle. This type of stray light creates a bright region of scattered light surrounding the displayed image.

There are two ways to reduce this type of stray light. One approach is to darken or roughen the sidewalls or other structures to reduce the reflectivity of the light. However, while this increases the absorbance at the surface, the reflected light scattering from the surface may still be noticeable. Another approach is to provide a baffle for blocking or clipping stray light. Blocking or clipping the reflected light scattered from the surface greatly reduces the effect of this stray light. In a head mounted display, it is advantageous to use both of these ways to reduce stray light, since the bright areas around the displayed image are removed and the contrast of the displayed image is increased.

U.S. Patent No. 5949583 provides a visor on top of a head mounted display to block stray light from coming from above. However, this does not address the need for control to reduce stray light from the inside of a head mounted display system.

U.S. Patent No. 6369952 provides two masks that block light from around the edges of a liquid crystal display image source in a head mounted display. The first mask is located on the input side of the liquid crystal image light source adjacent the backlight while the second mask is located on the output side of the liquid crystal display. Because both masks are located close to the liquid crystal display, "both the first mask 222 and the second mask 224 can be made to have openings or windows 232 and 234, which are substantially identical and co- Quot; (Column 15, lines 15-19). By placing the mask close to the image light source, the mask may have little effect on light emitted by the image light source from a region of the image light source closer to the center of the active region of the image light source to a wide cone angle. This wide cone angle light can be reflected from the side walls of the housing in a variety of ways, thereby providing stray light in the form of bright areas and reduced contrast.

Therefore, a method for reducing the stray light from the light source inside the head mounted display is needed.

Figure 160 shows an example of a display system having an optically flat reflective surface that is a beam splitter made of an optical film on a substrate, wherein the display system is a near vision display 16002. [ In this example, the image light source 16012 includes a projection system (not shown) that provides the image light with an optical layout that includes a curved optical axis 16018 located in the near vision display 16002. An optical system along optical axis 16018 may include a lens that focuses the image light to provide a focused image from image light source 16012 to the user's eye 16004. [ The beam splitter 16008 causes the optical axis 16018 from the image light source 16012 to bend toward the spherical or aspheric reflector 16010. The beam splitter 16008 may be a partial reflective mirror or a polarizing beam splitter. The beam splitter 16008 in the near vision display 16002 is oriented at an angle that redirects at least a portion of the image light from the image light source 16012 towards the reflector 16010. [ From the reflector 16010, at least an additional portion of the image light is reflected back toward the user's eye 16004. The reflected additional portion of the image light passes again through the beam splitter 16008 and is focused on the user's eye 16004. The reflector 16010 may be a mirror or a partial mirror. The scene light from the scene in front of the near vision display 16002 can be combined with the video light so that the video light and the scene light 16014 along the axis 16018 can be combined with the video light, To the eye 16004 of the user. The combined image light 16020 provides a combined image of the scene and the overlaid image from the image light source to the user's eye 16004.

Fig. 161 shows an example of the near vision display module 200. Fig. The module 200 comprises a reflector 16104, an image light source module 16108, and a beam splitter 16102. The module may be open at the sides by attachment between at least some of the joining edges between the reflector 16104, the image light source module 16108 and the beam splitter 16102 . Alternatively, the module 200 may be enclosed at the sides by sidewalls to provide an enclosed module to prevent dust, contaminants, and moisture from reaching the interior surface of the module 200. The reflector 16104, the image light source module 16108 and the beam splitter 16102 may be fabricated separately and then joined together, or at least some of the pieces may be fabricated together into a combined subassembly. In the module 200, an optical film may be used on the beam splitter 16102 or the reflector 16104. In Figure 161 beam splitter 16102 is shown as a flat surface while reflector 16104 is shown as a spherical surface. In the near vision display module 200, both the reflector 16104 and the beam splitter 16102 are used to provide images to the user's eye, as shown in Figure 160, so that the surfaces are optically flat It is important that it is optically uniform.

When the image light source 16108 includes a projection system having a light source with a wide cone angle of light, the image brightness also has a wide cone angle. As a result, the image light interacts with the sidewalls of the module 200, and this interaction can be reflected in the form of a bright area and provide scattered light, which is bright Region. This bright area can be very annoying to the user because it can look like a halo surrounding the displayed image. In addition, scattered light can degrade the contrast in the displayed image by providing a random low level of light over the image.

Figure 162 shows an example of an optical system associated with one type of head mounted display 16200. In the optical system, the light source 16204 provides a wide cone angle of the ray including the central ray 16202 and the edge ray 16224. Light source 16204 may provide polarized light. The light beam passes from the light source 16204 to the illumination beam splitter 16210, which reflects a portion of its light towards the reflected image light source 16208, which may be an LCOS display. The first portion of the light is reflected by the image light source 16208 corresponding to the image content being displayed and the polarization state is changed. The second portion of light then passes through the illumination beam splitter 16210 and then through one or more lenses 16212 that extend the cone angle of the light beam. The third portion of the light is reflected by the image beam splitter 16220 toward the spherical (or hemispherical) partial mirror 16214 at an angle. The partial mirror 16214 causes the light to converge while reflecting the fourth portion of light to focus the image on the user's eye 16228. [ After the fourth portion of the light is reflected by the partial mirror 16214 a fifth portion of the light passes through the image beam splitter 16220 and continues toward the user's eye 16228, An enlarged version of the resulting image is provided to the user ' s eyes 16228. [ In a see-through head mounted display, light 16218 (or scene light) from the environment passes through a partial mirror 16214 and an image beam splitter 16220, through image. The user is then provided with a combined image consisting of a displayed image from the image light source and a perspective image of the environment.

The central ray 16202 passes through the center of the optical system of the head-mounted display along the optical axis of the optical system. The optical system includes an illumination beam splitter 16210, an image light source 16208, an image beam splitter 16220, and a partial mirror 16214. 162, edge ray 16224 may follow the sides of housing 16222 and may then interact with the side walls of housing 16222, in which case edge ray 16224 may move along side walls As shown in FIG. This reflected or scattered light from edge ray 16224 is visible to the user as a bright area surrounding the displayed image or as a reduction in contrast in the image. The present disclosure provides a method of reducing bright areas by reducing reflected and scattered light from sidewalls by blocking or clipping the reflected or scattered light.

Figure 163 shows an example of a first embodiment of the present disclosure in which a baffle 16302 is added between the illumination beam splitter 16210 and the lens 16212 inside the housing 16222. [ The baffle 16302 blocks or clips the edge ray 16224 before it enters the lens 16212. The baffle 16302 may be of any opaque material such that the edge ray 16224 is blocked or clipped. In a preferred embodiment, the baffle 16302 may be made of a black material with a matte finish so that the incident light is absorbed by the baffle. The baffle 16302 may be comprised of a sheet of flat material having an opening disposed in the housing 16222 or the baffle 16302 may be part of the housing 16222. [ Because the baffle 16302 is located remotely from the image light source 16208 and the image light is diverging, the image provided by the image light source 16208 can be captured by the surrounding baffle 16302 so that it is not clipped by the edges of the baffle The resulting aperture is larger than the active area of the image light source 16208 and as a result, the entire image provided by the image light source 16208 is visible to the user, as shown in Figure 163. In addition, the baffle preferably has a thin cross section (shown in Figure 163) or sharp edges so that light is not scattered from the edges of the baffle.

Figure 164 shows an example of another embodiment of the present disclosure in which a baffle 16402 is added to the entrance surface of the lens 16212. [ Baffle 16402 may be fabricated as part of housing 16222 or baffle 16402 may be attached as a mask on lens 16212. [ In either case, the baffle 16402 should be opaque to block and absorb incident light, and should preferably be black with a matte finish.

165 shows an example of an embodiment of the present disclosure, similar to the embodiment shown in FIG. 164, but located on the output side of the lens 16212. FIG. In this embodiment, baffle 16502 is provided for blocking or clipping edge ray 16224 after edge ray 16224 has passed through lens 16212.

Figure 166 shows an example of another embodiment of the present disclosure in which a baffle 16602 is attached to the housing 16222 between a lens 16212 and an image beam splitter 16220. The baffle 16602 may be part of the housing 16222 or the baffle 16602 may be a separate structure disposed in the housing 16222. [ The baffle 16602 blocks or clips the edge ray 16224 so that a bright region is not provided in the user's eye 16228 around the displayed image.

Figure 167 illustrates an example of a further embodiment of the present disclosure in which an absorbing coating 16702 is applied to the side walls of the housing 16222 to reduce reflection and scattering of incident light and edge light 16224 . The absorbent coating 16702 can be coupled to the baffle 16302, 16402, 16502, or 16602.

Figure 168 shows an example of another source of stray light in a head mounted display wherein the stray light 16802 comes directly from the edge of the light source 16204. This stray light 16802 can be particularly bright because it first comes out of the light source 16204 without first being reflected from the illumination beam splitter 16210 and then reflected from the image light source 16208. [ 169 shows an example of another source of stray light 16902 from light source 16204 where stray light 16902 is reflected from the surface of image light source 16208 where the polarization state is changed and stray light 16902 is then And can pass through the illumination beam splitter at a relatively steep angle. As shown in FIG. 169, this stray light 16902 can then be reflected from any reflective surface in the housing or from the edge of the lens 16212. Figure 170 illustrates an example of another further embodiment of the present disclosure in which a baffle 17002 is provided adjacent to a light source 16204. [ The baffle 17002 is opaque and extends from the light source 16204 so that the stray light 16802 and 16902 are not blocked by the light source 16204 or clipped to reach the user's eye 16228. [

163 to 167, 169 and 170 to further reduce stray light in the head mounted display and thereby reduce the bright area surrounding the displayed image or to increase the contrast in the displayed image, in a further embodiment, The baffle or coating shown in FIG. A plurality of baffles may be used between the light source 16204 and the image beam splitter 16220. 171, an absorbent coating having ridges 17102 can be used, wherein a series of small ridges or staircases can be used as a series of blocking or clipping edge rays over the entire sidewall area of the housing As shown in FIG. The ridges 17102 may be part of the housing 16222 or may be attached as a separate layer to the inner wall of the housing 16222. [

172 shows a further embodiment of a tape or sheet 17210 that includes a carrier sheet 17212 and ridges 17214 that can be used to block reflected light as shown in Figure 171. [ The ridges 17214 are obliquely inclined at one side and are steeply inclined at the other side, so that incident light approaching from a sharply inclined side is struck. The ridges 17214 may be a solid ridge having a triangular cross section with a sharp edge as shown in Figure 172, or may be a thin inclined scale affixed to one edge, It can be an inclined fiber attached to one end and thus the surface is angled with respect to the side wall and the incident light is blocked. The advantage of the tape or sheet 17210 is that the ridges 17214 can be relatively thin and the ridges covering a substantial area of the housing 16222. [ A further advantage of the tape or sheet 17210 is that the ridges 17214 can be manufactured more easily than the ridges shown in Figure 171, which can be difficult to mold as part of the housing.

In all embodiments, the surrounding baffles can create an aperture having a magnitude corresponding to the distance they are located along the optical axis from the image light source, so that the image light can diverge along the optical axis, The user may provide an undipped view of the user's eye 16228. [

In one embodiment, an absorption polarizer in the optical assembly is used to reduce stray light. The absorption polarizer may comprise an anti-reflective coating. An absorption polarizer can be placed after the focusing lens of the optical assembly to reduce the light passing through the optically flat film of the optical assembly. The light from the image light source can be polarized to increase the contrast.

In one embodiment, an anti-reflective coating in the optical assembly can be used to reduce stray light. The antireflective coating may be disposed on the retarder film of the polarizer or optical assembly of the optical assembly. The retardation film may be a quarter wave film or a half wave film. The anti-reflection coating may be disposed on the outer surface of the partial reflective mirror. The light from the image light source can be polarized to increase the contrast.

Referring to FIG. 102A, an image light source 10228 directs image light to a beam splitter layer of an optical assembly. 103 shows an enlarged view (blow-up) of the image light source 10228. Fig. In this particular embodiment, the image light source 10228 transmits light through a diffuser 10304 and a full polarizer 10308 to a curved wire grid polarizer 10310 where light is reflected toward the LCoS display 10312 And a light source (LED bar 10302) that directs light toward the - side. The image light from the LCoS is then reflected back to the beam splitter layer of the optical assembly 10200 through the curved wire grid polarizer 10310 and the 1/2 wave film 10312. In embodiments, an optical assembly comprising optical components 10204, 10210, 10212, 10212, 10230 may be used as a sealable optical assembly-detachable (e.g., snap-on and snap-off), interchangeable, And the image light source 10228 can be provided as an integral component in the frame of the eyepiece. This allows the sealed optical assembly to be made waterproof, dust proof, interchangeable, customizable, and the like. For example, a given encapsulated optical assembly may have corrective optics for one person, and may include a second encapsulated optical assembly for another person having different corrective optical system requirements (e.g., different prescriptions) Can be replaced. In embodiments, there may be applications where both eyes do not need to receive input from the eyepiece. In this example, a person can simply detach one side and use only a single side to project the content. In this manner, the user will now have an unobstructed optical path to the eye from which the assembly is removed, and the eyepiece will save battery life, such as by operating only half of the system.

The optical system assembly is divided into discrete portions depending on which portion is sealed-for example, an image generation facility 10228 and a directive optics facility 10228 as shown in FIG. 10204, 10210, 10212, and 10230). In a further example, FIG. 147 illustrates a configuration of an embodiment of an eyepiece that illustrates the directing optics as a 'projection screen' 14608a and 14608b. 102A also shows a portion of an eyepiece electronic device and projection system 14602, wherein this portion of the projection system can be referred to as an imaging generator. The image generation facility and the directed optical system facility may be a sealed subassembly, for example to protect the optical system therein from contaminants of the surrounding environment. In addition, the directing optical system may be configured to receive a non-destructive forced removal (e. G., To break away from the body of the eyepiece without impinging upon the deflecting optical system) to remove, for example, , ≪ / RTI > and others. In embodiments, the present disclosure may include an interactive head-mounted eyepiece worn by a user, wherein the eyepiece comprises an optical assembly through which a user views the surrounding environment and the displayed content through the eyepiece, Wherein the optical assembly includes an image generation facility mounted within a frame of the eyepiece and a directing optics arrangement disposed in front of the user's eye and separable from a frame of the eyepiece, The installation is enclosed in a frame to reduce contamination from the surrounding environment. In embodiments, the seal may be a sealed optical window. As described herein, the eyepiece may further include a treatment facility, a power management facility, a detachment sensor, a battery, etc., where the power management facility may provide a detachment indication from the separation sensor It is possible to detect the separation of the directing optical system equipment and selectively reduce the power to the components of the eyepiece to reduce the power consumed by the battery. For example, a component with reduced power may be an image light source (e.g., reducing the brightness of the image light source and turning off power to the image light source, etc.) And the power consumption of the image light source can be returned to the pre-separation operation level. If the directional optical system equipment is inadvertently forcibly disconnected, it can be separated in such a way that the optical system of the optical system is removed so that the eyepiece is not damaged. Directional optical system equipment can be separated via connection mechanisms such as magnets, pins, rails, snap-on connectors, and the like. The directing optics facility may provide a user's vision correction that requires correction glasses, in which case the orientation optics facility is interchangeable to change the vision correction prescription of the eyepiece. The eyepiece may have two separate detachable optical assemblies for each eye wherein one of the individual optical assemblies is removed to enable monocular usage by the rest of the individual optical assemblies. For example, monocular use may be firearms sighting usage, wherein the side of the eyepiece having a detachable optical system is used to aim the firearm so that a user with an unobstructed visual path can use the firearm sighting And keep the equipment provided to the other eye by the eyepiece. The directing optics facility can be separated to enable exchange between a direct optical system facility adapted for indoor use and a directed optical system facility adapted for outdoor use. For example, there may be different filters, field of view, contrast, shielding, etc. for indoor versus outdoor use. The directing optics system may be configured to accept additional elements such as optical elements, mechanical elements, control elements, and the like. For example, an optical element can be inserted to adjust the optical prescription of the user. The directional optical system equipment can also be replaced to change the field of view provided by, for example, replacing a directional optical system having a first field of view with a directional optical system having a second field of view.

Referring to Figure 104, the LED provides unpolarized light. The diffuser spreads the light from the LED and homogenizes it. The absorption full-beam photon converts the light into S polarized light. The S polarized light is then reflected towards the LCOS by a curved wire grid polarizer. The LCOS reflects the S polarized light and converts it into P polarized light according to the local image content. The P polarized light passes through the curved wire grid polarizer and becomes P polarized image light. The 1/2 wave film converts the P-polarized image light into S-polarized image light.

Referring again to Figure 102a, the beam splitter layer 10204 is a polarizing beam splitter, or the image light source provides polarized image light 10208, and the beam splitter layer 10204 is a polarizing beam splitter, 10208 is linearly polarized light, and this embodiment and associated polarization control is shown in Figure 102a. When the image light source provides linearly polarized image light and the beam splitter layer 10204 is a polarizing beam splitter, the polarization state of the image light is aligned according to the polarizing beam splitter and thus the image light 10208 is reflected by the polarizing beam splitter do. 102A shows that the reflected image light has S-state polarized light. When the beam splitter layer 10204 is a polarizing beam splitter, a first quarter wave film 10210 is provided between the beam splitter layer 10204 and the partial reflective mirror 10212. The first quarter wave film 10210 converts the linearly polarized image light into circularly polarized image light (S in FIG. 102A is shown as being converted to CR). The reflected first portion of the video light 10208 is also circularly polarized, with the circularly polarized state reversed (shown as CL in FIG. 102A), and after passing through the quarter wave film, The polarization state of the reflected first portion of the image light 10208 is reversed (with P polarization) compared to the polarization state (shown as S) of the image light 10208 provided by the image light source. As a result, the reflected first portion of the video light 10208 passes through the polarizing beam splitter without return loss. When the beam splitter layer 10204 is a polarizing beam splitter and the perspective display assembly 10200 includes a first quarter wave film 10210, the light control element 10230 may include a second quarter wave film and a linear Polarizer 10220 may be included. In embodiments, the light control element 10230 includes a controllable darkening layer 10214. The second quarter wave film 10218 is used to split the second portion of the circularly polarized image light 10208 into linearly polarized image light having a polarization state blocked by the linear polarizer 10220 in the light control element 10230 10208) (CR is shown as being converted to S), thereby reducing glare.

When the light control element 10230 includes a linear polarizer 10220 and a quarter wave film 10218, the unpolarized scene light 10222 coming from the external environment in front of the user is converted to linearly polarized light (Shown in Figure 102a as P-polarized state) and 50% of its light is blocked. The first portion of scene light 10222 passing through linear polarizer 10220 is linearly polarized light, which is converted into circularly polarized light by a quarter wave film (in Figure 102a P is converted to CL Lt; / RTI > A third portion of the scene light reflected from the partially reflective mirror 10212 inverts the circular polarization (shown in Figure 102a to convert CL to CR), which is then reflected by the second quarter wave film 10218 Linearly polarized light (CR in Figure 102a is shown as being converted to S polarized light). The linear polarizer 10220 then blocks the reflected third portion of the scene light, thereby reducing escaping light and reducing glare.

As shown in Figure 102A, the reflected first portion of the video light 10208 and the transmitted second portion of the scene light have the same circular polarization state (shown as CL) and therefore they are combined and the first 1/4 Converted to linearly polarized light by wave film 10210 (shown as P) which passes through the beam splitter when the beam splitter layer 10204 is a polarizing beam splitter. The linearly polarized combined light 10224 is then provided to a user ' s eye 10202 located behind the perspective display assembly 10200, wherein the combined image is displayed on the display screen, And an overlaid portion of the perspective view of the external environment at the front of the vehicle.

The beam splitter layer 10204 includes an optically planar film such as the Asahi TAC film discussed herein. The beam splitter layer 10204 may be disposed at an angle to the front of the user's eye to reflect and transmit portions of the image light and to transmit scene light from a perspective view of the surrounding environment, A combined image composed of portions of the transmitted scene light is provided to the user's eyes. The optically planar film may be a polarizer such as a wire grid polarizer. The optically planar film may be laminated to a transparent substrate. The optically planar film may be molded, over-molded, glued, or otherwise formed on or in the surface of the optical surface of one of the optical surfaces of the eyepiece, such as beam splitter 10202, . The optically planar film may be disposed less than 40 degrees from vertical. The curved polarizing film may have a ratio of the height of the light source to the width of the illuminated area less than 1: 1. The highest point of the curved film is lower than the length of the narrowest axis of the display. In embodiments, if the optically thin film (s) are on a beam splitter, additional optical systems, such as calibration optics, prescriptions, etc., are added to the surface, etc., to keep the film flat in the sandwich layer therebetween .

The present disclosure also provides a method of providing an optical film on an optically smooth surface. Optical films are a convenient way of forming optical structures with optical properties that are very different from the rest of the imaging device. In order to provide a function to the imaging device, an optical film needs to be attached to the optical device. When the optical film is used in a reflective manner, it is important that the reflective surface is optically flat, otherwise the wavefront of the light reflected from the reflective surface will not be preserved and the image quality will deteriorate. The optically planar surface can be defined as a uniform surface within five wavelengths of light per inch of the surface when the imaging device is measured for the wavelength of light used and compared to a flat surface or a desired optical surface.

Optically planar surfaces, including optical films such as those described in this disclosure, may be included in display systems, including projectors, projection televisions, near vision displays, head mounted displays, perspective displays, and the like.

Figure 140 shows an example of a display system having an optically flat reflective surface that is a beam splitter made of an optical film on a substrate, wherein the display system is a near vision display 14000. [ In this example, the image light source 14010 includes a projection system (not shown) that provides the image light with an optical layout that includes a curved optical axis 14014 located in the near vision display 14000. [ An optical system along the optical axis 14014 may include a lens that focuses the image light to provide an image focused from the image light source 14010 to the user's eye 14002. [ The beam splitter 14004 causes the optical axis 14014 from the image light source 14010 to bend toward the spherical or aspheric reflector 14008. The beam splitter 14004 may be a partial reflective mirror or a polarizing beam splitter layer. The beam splitter 14004 in the near vision display 14000 is oriented at an angle that diverts at least a portion of the image light from the image light source 14010 toward the reflector 14008. [ From the reflector 14008, at least an additional portion of the image light is reflected back toward the user's eye 14002. The reflected additional portion of the image light passes again through the beam splitter 14004 and is focused on the user's eye 14002. The reflector 14008 may be a mirror or a partial mirror. When the reflector 14008 is a partial mirror, scene light from a scene in front of the near vision display 14000 can be combined with the image light, thereby causing the image light along the axis 14014 and the axis 14012 And provides the combined image light 14018 consisting of the following scene light toward the eyes 14002 of the user. The combined image light 14018 provides a combined image of the overlaid image from the scene and the image light source to the user's eye.

Fig. 141 shows an example of the near vision display module 14100. Fig. The module 14100 comprises a reflector 14104, an image light source module 14108, and a beam splitter 14102. The module may be open at the sides by an attachment between at least some of the joining edges between the reflector 14104, the image light source module 14108 and the beam splitter 14102 . Alternatively, the module 14100 may be closed at the sides by sidewalls to provide a hermetic module to prevent dust, contaminants, and moisture from reaching the interior surface of the module 14100. The reflector 14104, the image light source module 14108 and the beam splitter 14102 may be fabricated separately and then joined together, or at least some of the fragments may be fabricated together as a combined subassembly. In module 14100, an optical film may be used on the beam splitter 14102 or on the reflector. In Figure 141 beam splitter 14102 is shown as a flat surface while reflector 14104 is shown as a spherical surface. In the near vision display module 14100, both the reflector 14104 and the beam splitter 14102 are used to provide an image to the user's eye, as shown in Figure 140, so that the surfaces are optically flat It is important that it is optically uniform.

Figure 142 shows a schematic view of a pellicle style film assembly 14200, which is one embodiment of the present disclosure. The pellicle style film assembly 14200 includes a frame 14202 comprising upper and lower frame members 14202a and 14202b. The optical film 14204 is held between the frame member 14202a and the frame member 14202b by an adhesive or a fastener. In order to improve the flatness of the optical film 14204, the optical film 14204 may be stretched in one or more directions while an adhesive is applied and the frame members 14202a and 14202b are bonded to the optical film 14204. After the optical film 14204 is bonded to the frame 14202, the edges of the optical film can be trimmed to provide a smooth surface to the outer edge of the frame 14204.

In some embodiments of the present disclosure, the optical film 14204 is a folded film consisting of a series of optically planar surfaces, and the interface of the frame members 14202a and 14202b is a curved And has a matching folded shape. The curved film is then stretched along the direction of the curvature, and frame members 14202a and 14202b hold the optical film 14204 in a curved configuration and position each of the series of optically planar surfaces to be held in place Respectively.

In all cases, after the frame members 14202a and 14202b are bonded to the optical film 14204, the resulting pellicle-style film assembly 14200 is optically coupled to the optics 14204, such as a near-vision display module 14100, Is a rigid assembly that can be positioned within a device. In this embodiment, the pellicle style film assembly 14200 is a replaceable beam splitter 14102 assembly in the near vision display module 14100. The sidewalls in the near vision display module 14100 may have grooves in which the frame 14202 is fitted or alternatively may be provided with a flat surface connecting the sidewalls and the frame 14202 may be on top of the flat surface .

Figure 143 shows an example of an insert molded assembly 14300 comprising an optical film 14302. In this embodiment, the optical film 14302 is positioned in the mold, and the plastic fills the mold cavity and forms a molded structure 14304 adjacent the optical film 14302 and behind the optical film 14302 A viscous plastic material is injected into the mold through a molding gate 14308. [ When the plastic material is cured in the mold, the mold is opened along the parting line 14310 and the insert-molded assembly 14300 is removed from the mold. The optical film 14302 is then embedded in the insert molded assembly 140000 and attached thereto. In order to improve the optical flatness of the optical film 14302 in the insert molded assembly 14300, the inner surface of the mold where the optical film 14302 is positioned against is an optically flat surface. In this manner, the viscous plastic material forces the optical film 14302 against the optically planar surface of the mold during the molding process. This process can be used to provide an optically smooth surface having flat or desired optical curvature as previously described. In a further embodiment, the optical film 14302 may have an adhesive layer or a tie layer to increase the adhesive force between the optical film 14302 and the molded structure 14304.

In another embodiment, the optical film 14302 is located within the mold, and there is a protective film between the mold surface and the optical film 14302. The protective film may be attached to the optical film 14302 or the mold. The protective film may be smoother or more flatter than the mold surface to provide a smoother or smoother surface to the optical film 14302 to be molded. Accordingly, the protective film may be any material such as, for example, plastic or metal.

144 shows an example of a lamination process for producing a laminated plate having an optical film 14400. Fig. In this embodiment, upper and lower press plates 14408a and 14408b are used to laminate the optical film 14400 to the substrate 14404. An adhesive 14402 may optionally be used to bond the substrate 14404 to the optical film 14400. [ In addition, one or more of the compression plates 14408a and 14408b may be heated or the substrate 14404 may be heated to provide a higher level of adhesion between the substrate 14404 and the optical film 14400 . Heating one or more of the squeeze plates 14408a and 14408b and the substrate may also be used to soften the substrate 14404 thereby providing more uniform pressure to the back of the optical film 14400, The smoothness or flatness of the optical film 14400 can be improved. As described above for the pellicle style film assembly 14200, a laminated plate having the optical film 14400 of this embodiment can be used as a replaceable beam splitter in the near vision optical module 14100. [

145A to 145C show examples of an application process for producing a molded structure 14502 having an optical surface including an optical film 14500. Fig. In this embodiment, an optical film 14500 is attached to an optically planar surface 14504 in a structure 14502 molded by a rubber applicator 14508. In this embodiment, An adhesive layer may be applied to the optically planar surface 14504 of the molded structure 14502 or the lower surface of the optical film 14500 to adhere the optical film 14500 to the molded structure 14502. [ The rubber applicator 14508 may be a relatively soft rubber material having a curved surface so that the central portion of the optical film 14500 first contacts the optically smooth surface 14504 of the molded structure 14502. [ As the rubber applicator 14508 is further pushed downwardly there is a gap between the optical film 14500 and the optically planar surface 14504 of the molded structure 14502 as shown in Figures 145a, 145b and 145c The size of the contact area increases. This gradual attachment process provides a very uniform pressure application that allows air to escape at the interface during the attachment process. As shown in Figure 145c, the incremental deposition process is performed with an optically planar optical film 14500 attached to the inner surface of the molded structure 14502, along with the optically planar surface 14504 of the formed structure 14502. [ ). An adhesive layer used to bond the optical film 14500 to the molded structure 14502 may be attached to the optically film 14500 or to the optically planar surface 14504 on the inside of the molded structure 14502. [ Those skilled in the art will appreciate that this attachment process may likewise be used to attach the optical film to the outer surface of the molded structure. In addition, the optically planar surface can be a flat surface or a surface with a desired optical curvature, or a series of optically planar surfaces, and the rubber applicator is shaped to provide a progressive pressure application when the optical film is attached.

In embodiments, the image display system may include an optically planar optical film comprising a display module housing, wherein the housing comprises an optically flat optical film, an image light source, and a substrate having a viewing position, Is reflected from the optical film toward the viewing position. In embodiments, the optical film of the image display system may be molded into the display module. In embodiments, an optical film may be attached to the display module. In addition, in embodiments, the optical film of the display system may be a wire grid polarizer, a mirror, a partial mirror, a holographic film, or the like. In embodiments, the image display system may be a near vision display. In embodiments, the optical film may be retained against the optically smooth surface when the optical film is molded in the display module, or alternatively when the optical film is molded into the display module. In embodiments, the optical film of the image display system may include optical flatness of five wavelengths of light per inch.

In one embodiment, an image display system that includes an optically flat optical film can include an optically flat optical film, a display module housing, an image light source, and a substrate having viewing locations, wherein the image provided by the image light source Can be reflected from the optical film to the viewing position, and the substrate with the optical film can be replaced in the display module housing. In such embodiments, the substrate of the image display system may be a frame, the optical film may remain tensioned by the frame, the substrate may be a plate molded behind the film, and / or the substrate may be laminated Plate. In addition, the optical film of the image display system can be a beam splitter, a polarizing beam splitter, a wire grid polarizer, a mirror, a partial mirror, a holographic film, and the like. In addition, the image display system may be a near vision display. In embodiments, the optical film of the image display system can be retained against an optically smooth surface when the plate is molded behind the optical film. In addition, in embodiments, the optical film of the image display system can be held against an optically flat surface when the plate is laminated to the optical film. In various embodiments, the optical film of the image display system may include optical flatness of five wavelengths of light per inch.

In one embodiment, all of the components in FIG. 102A are gathered to form an electro-optic module. The angle of the optical axis associated with the display may be more than 10 degrees forward from the vertical. This tilt angle refers to how the upper portion of the optical module is inclined forward. This allows the beam splitter angle to be reduced, which makes the optical module thinner.

The ratio of the height of the curved polarizing film to the width of the reflective image display is less than 1: 1. The curved surface on the polarizing film determines the width of the illuminated area on the reflective display and the tilt of the curved area determines the placement of the illuminated area on the reflective display. The curved polarizing film reflects the illumination light in the first polarization state onto the reflective display, which changes the polarization of the illumination light to generate image light, and the curved polarizing film passes the reflected image light. The curved polarizing film includes a portion parallel to the reflective display on the light source. The height of the image light source may be at least 80%, 3.5 mm or more, or less than 4 mm of the display active area width.

In portable display systems, it is important to provide a display that is bright, small and lightweight. Portable display systems include cell phones, laptop computers, tablet computers, near vision displays and head-mounted displays.

The present disclosure provides a compact lightweight front light source for a portable display system that is redirected from an edge light source to a partially reflective film for illuminating a reflected image light source. The partially reflecting film may be a partial mirror beam splitter film or a polarizing beam splitter film. The polarizing beam splitter film may be a multilayer dielectric film or a wire grid polarizing film. Polarizing beam splitter films are known to provide efficient reflection for one polarization state while allowing other polarization states to pass through at the same time. Multilayer dielectric films are available from 3M under the name DBEF, Minneapolis, Minn. The wire grid polarizing film is available from Asahi-Kasei E-Materials, Tokyo, Japan under the name WGF.

The edge light provides a small light source for the display, but because it is located at the edge of the image light source, the light must be turned 90 degrees to illuminate the image light source. When the image light source is a reflection image light source such as a liquid crystal on silicon (LCOS) image light source, the illumination light must be polarized. The polarized light is reflected by the surface of the image light source, and the polarization state of the light is changed corresponding to the image content being displayed. The reflected light then passes through the backlight again.

Figure 187 shows a schematic illustration of a prior art display assembly 18700 having a solid beam splitter cube 18718 as front illumination. The display assembly includes front lighting, one or more light sources, and an image light source. In display assembly 18700, one or more light sources 18702 are included to provide light shown as light rays 18712. [ The light source may be an LED, a fluorescent lighting device, an OLED, an incandescent lighting device, or a solid state light. A light beam 18712 passes through the diffuser 18704 to diffuse the light laterally for more uniform illumination. When the diffused light is polarized, the diffuser comprises a linear polarizer. The diffused light beam 18714 is emitted towards the partial reflective layer 18708 through the solid beam splitter cube 18718 where it partially reflects towards the reflected image light source 18720. Diffused light 18714 is then reflected by reflective image light source 18720, thereby forming image light 18710 that is transmitted by partial reflective layer 18708. [ The image light 18710 may then enter the associated image optics (not shown) to present the image to the observer. However, as can be seen in Figure 187, the height of the illuminated area of the light source, shown here as diffuser 18704, is equal to the width of the reflected image light source 18720 being illuminated. The partial reflective layer 18708 is disposed at an angle of 45 degrees to provide an image ray 18710 that travels straight or vertically into the associated imaging optics. As a result, the front lighting shown in Figure 187 is relatively large in size.

In a typical imaging system, it is important to maintain the wavefront from the image light source to provide a high quality image with good resolution and contrast. Accordingly, as is known to those skilled in the art, a high quality image is provided to the observer so that the image light 18710 must travel vertically from the reflective image light source 18720 to provide a uniform wavefront to the lock- do. Accordingly, the diffused light beam 18714 has to be redirected vertically to the reflected image light source 18720 by the partial reflective film 18708, so that this light beam is reflected (as shown in Figures 187 to 198 ) Vertically into the associated imaging optics.

Figure 188 shows another prior art display assembly 18802 that includes a partially reflective film 18804 that is supported at an edge and is free standing on a reflective image light source 18720. This display assembly operates in a manner similar to the display assembly shown in Figure 187 except that due to the absence of a solid beam splitter cube 18718 the display assembly 18802 is lighter than the display assembly 18700 will be. As can be seen in Figure 188, the height of the diffuser 18704, when reflected by the reflected image light source 18720, to provide image light 18808 traveling vertically into the associated imaging optics, Is equal to the width of the image light source 18720.

Figure 189 shows a schematic illustration of what happens to light in the display assembly 18902 when the partially reflective film 18804 is placed at a subtended angle of less than 45 degrees. In this case, portions of the reflected image light source 18720 are not uniformly illuminated. The light rays illuminating a portion of the reflected image light source farthest from the diffuser do not proceed straight to the associated imaging optics (as in the case of ray 18904) or change polarization states (as in the case of ray 18908) , Which is then passed through the film when the partially reflective film is a polarizing beam splitter film (also referred to as a reflective polarizer film). Accordingly, when the partial reflective film 18804 is disposed at an angle of less than 45 degrees when the related image optical system can use only the image light proceeding straight from the reflective image light source 18720, the reflected image light source 18720, The dark region of the image is generated correspondingly.

In one embodiment of the present disclosure shown in FIG. 190, the diffused light 19010 provided by the light source 18702 is redirected downward to illuminate the reflective image light source 18720, 19004) are provided. The curved partial reflective surface 19004 may be a thin and flexible polarizing beam splitter film. In this case, diffuser 18704 includes a linear polarizer such that light 18712 is diffused and then linearly polarized, and thus diffused light 19010 is polarized. The linear polarizer and polarizing beam splitter film 19004 in diffuser 18704 are oriented such that light passing through the linear polarizer is reflected by the polarizing beam splitter film. In this manner, when the reflected image light source 18720 changes the polarization of the diffused light 19010, the polarization of the reflected image light 19008 is in the opposite polarization state as compared to the diffused light 19010. The reflected image light 19008 then passes through the partial reflecting film 19004 and continues to the display optical system. By using a flexible polarizing beam splitter film as the partially reflective surface 19004, the partially reflective surface 19004 can be curved and lightweight. The polarizing beam splitter film performs a dual role of being a reflector for the diffused light 19010 illuminating the reflected image light source 18720 and a transparent member for the reflected image light 19008. As is known to those skilled in the art, the advantage provided by the polarizing beam splitter film is that the curved surface can receive light over a wide range of angles of incidence so as not to interfere with the light entering the film. In addition, since the polarizing beam splitter film is thin (e.g., less than 200 micrometers), the curved shape does not significantly distort the image light 19008 as the image light 19008 enters the display optics through the film. Finally, the polarizing beam splitter film has a low tendency to scatter light, and therefore a high image contrast can be maintained.

The flexible nature of the polarizing beam splitter film allows the film to be formed into a curved shape that redirects the light from the diffuser and focuses it onto the reflected image light source. The shape of the curved surface of the polarizing beam splitter film can be selected based on the light distribution provided by the diffuser to provide uniform illumination of the reflected image light source. 190 shows a curved partial reflection film 19004 having a parabolic shape, it is possible to uniformly distribute the light 19010 diffused according to the nature of the light source 18702 and the effectiveness of the diffuser 18704 onto the reflection image light source 18720 A curved surface of constant radius, a complex spline curved surface, a relatively flat curved surface, a flat surface or a segmented planar surface are also possible for redirecting and focusing. Experiments show that the curved surface on the partial reflective surface 19004 tends to focus the diffused light 19010 at the center of the reflective image light source 18720 and thus the diffuser 18704 provides a distribution of brighter light at the edges Curved surfaces were found to be best used. Alternatively, from experiments it has been found that a relatively flat surface on the partial reflective surface 19004 is best used when the diffuser 18704 provides a distribution of brighter light in the center. When the partially reflective surface 19004 is made of a flexible film, its shape is such that the flexible film has slots of curved surfaces suitable for retaining the flexible film as a free standing film in place, Can be maintained by the side frame. Two side frames are used to support the curved shape on either side of the display assembly 190002 with other components. Since the majority of the display assembly 19002 is air and the partial reflective surface 19004 is thin, the weight is substantially lower than the prior art display assembly 18700 shown in Figure 187. In addition, as can be seen in Figure 190, the width of the reflected image light source 18720, which is illuminated so that the display assembly 19002 is smaller than the prior art display assembly shown in Figure 188, It is bigger than the height.

Figure 191 illustrates another embodiment of the present disclosure in which a dual light source 19104 is used in a display assembly 19102 in which two relatively flat partially reflective surfaces are disposed back to back. The configuration shown in Figure 191 provides a solid film holder 19120 in the front lighting with two sides so that the display assembly 19102 has two display assemblies, as shown in Figure 187, It is similar to using. In Figure 191, although the rays are shown only on one side, the elements and rays on the other side are symmetrical with the side shown. The solid film holder 19120 has a partially reflecting film 19110 that extends continuously between two side surfaces. The solid film holder 19120 is also continuous between the two sides so that the video light 19112 is not disturbed or deflected by the seam line between the two sides of the display assembly 19102. [ The solid film holder 19120 and the partial reflective film 19110 together provide a constant optical thickness and thus the image light is not deflected or distorted. Accordingly, the image light 19112 having a continuous image quality can be provided while being illuminated by the light from the two light sources 19104. Each light source 19104 provides light 19114 to a diffuser 19108 which diffuses light 19114 to provide diffused light 19118 for illuminating half of the reflected image light source 18720 ) In the lateral direction. The solid film holder 19120 holds the partially reflecting film 19110 in a desired shape. Most importantly, when compared to the illuminated width of the reflected image light source 18720, the height of the diffuser 19108 is reduced to half of the prior art diffuser 18704 shown in Figure 187 for the display assembly 18700 will be.

Figure 192 shows a schematic illustration of a display assembly 19202 having a dual light source 19104 and a free standing partial reflective film 19204 supported only at the edges. In Figure 192, although the rays are shown only on one side, the elements and rays on the other side are symmetrical with the side shown. 191, a further advantage is that since most of the display assembly 19202 is made of air, the display assembly 19202 is mounted on the display assembly 19202 19102).

Figure 193 shows a display assembly 19302 having a dual light source 19104 and a free-standing partial reflective film 19308, wherein the film is supported at its edges to provide two curved surfaces. In Figure 193, although the rays are shown only on one side, the elements and rays on the other side are symmetrical with the side shown. The partially reflecting film 19308 is continuous across both sides and has a similar curved surface on both sides. The curved surface is selected to reflect and focus the diffused light 19312 provided by the diffuser onto the reflective image light source 18720. [ Reflected image light source 18720 reflects diffused light 19312 and thereby forms image light 19310. The height of the diffuser 19304 is less than one half the prior art diffuser 18704 shown in Figure 187, and therefore the front lighting and display assembly 19302 is very compact.

Figure 194 shows a schematic illustration of a display assembly 19402 having a continuous partial reflective film 19308 inside a solid film holder 19404, which is otherwise similar to the display assembly 19302 shown in Figure 193. In Figure 194, although the rays are shown only on one side, the elements and rays on the other side are symmetrical with the side shown. A solid film holder 19404 is used on either side of the partial reflective film 19308 to hold the film in a defined two-sided curved surface and also to protect the partially reflective film 19308. In order to additionally avoid providing a seam line that will interfere with the video light 19310 at the center of the image, the two sides of the solid film holder 19404 are relatively thin sections < RTI ID = 0.0 > Respectively.

In preferred embodiments of the present disclosure, the partially reflecting film in the display assembly shown in Figures 191 to 194 is a polarizing beam splitting film. In these embodiments, the diffuser includes a linear polarizer such that the diffused light is polarized. The linear polarizer is aligned with the polarizing beam splitter film so that the diffused light has a polarization state that is reflected by the polarizing beam splitter film. The polarizing beam splitter film also functions as an analyzer for image light. The advantage of using diffused light polarized by a polarizing beam splitter film in forward illumination is that because all of the polarized diffused light is reflected by the polarizing beam splitter film towards the reflected image light source and is converted to image light there, The stray light is reduced. If the diffused light is not polarized, the polarization state of the unreflected diffused light will be transmitted through the polarizing beam splitter film, and if this light is not controlled, it will provide scattered light to the image light, Lt; RTI ID = 0.0 > a < / RTI >

195 shows a schematic illustration of a display assembly 19502 with polarization control for effectively illuminating a single light source 19104 on one side and a reflected image light source 18720 from both sides. In this case, light source 19104 provides unpolarized light 19114 and non-polarized diffused light 19508. [ The partially reflecting film is a polarizing beam splitter film 19504 in the solid film holder 19514. The polarizing beam splitter film 19504 transmits another polarization state (shown as ray 19518) while reflecting one polarization state of the diffused light (shown as ray 19510). The polarizing beam splitter film 19504 is curved and continuous so that light having a different polarization state 19518 passes through both sides of the curved polarizing beam splitter film 19504. This light 19518 then passes through a quarter wave retardation film 19524 which changes the polarization state from linear to circular. The circularly polarized light is then incident on a 1/4 wave retardation film 19524 that is reflected by mirror 19528 and changes the polarization state from circularly polarized light in one polarization state to linearly polarized light (shown as ray 19520) And thus the light 19520 is then reflected by the polarizing beam splitter film 19504 towards the reflected image light source 18720. [ As such, the light provided by the light source 19104 in the display assembly 19502 illuminates the reflected image light source 18720 on both sides using light in the same polarization state. Since diffused light 19508 is not polarized and both polarization states 19510 and 19518 are used to illuminate reflective image light source 18720, essentially all of the light provided by the light source is reflected by image light 19512, 19522). The image lights 19512 and 19522 are provided straight to the relevant image optical system. Again, the height of the diffuser 19108 is half the diffuser 18704 shown in Figure 187, thus providing a compact and efficient front lighting and display assembly.

Figure 196 shows a display assembly 19602 having a geometry similar to that shown in Figure 195, but the polarizing beam splitter film 19604 is free standing and is only supported at the edges to reduce the weight of the front lighting, Lt; RTI ID = 0.0 > diffuser < / RTI >

Figure 197 illustrates another embodiment of the present disclosure including a display assembly 19702 having dual light sources 19704 and 19708 and a curved polarizing beam splitter film 19714 wherein the curved polarizing beam splitter film 19714) are curved surfaces. Light 19718 and 19720 from light sources 189704 and 19708 are not polarized and diffusers 19710 and 19712 do not contain polarizers and thus diffused light 19722 and 19724 are also unpolarized . The angled sides of the curved surface of the polarizing beam splitter film 19714 directs one of the polarizations of the diffused light 19728,19730 toward the reflected image light source 18720 and also the light 19728,19730 Onto the image area of the light source 18720. In this display assembly, the dual light sources 19704, 19708 and the curved polarizing beam splitter 19714 operate in a complementary manner because the polarizing beam splitter film 19714 is continuous. Accordingly, the unpolarized diffused light 19722, 19724 is provided on each side of the display assembly 19702, respectively, and the first polarization state (typically S) is reflected by the polarization beam splitter film 19714, While light 19740, 19738 with a different polarization state (typically P) is transmitted by the polarizing beam splitter film 19714, while it is redirected toward the light source 18720. The transmitted light 19740 and 19638 with different polarization states pass through both sides of the curved polarizing beam splitter film 19714 and thus reach diffusers 19712 and 19710, respectively, on the opposite sides. When light 19740 and 19738 respectively affect the diffuser 19712 and 19710 on the opposite side, the light is diffusely reflected by the diffuser and is not polarized in this process. A reflector may be added to the light sources 19704, 19708 and the surrounding area to increase the reflection of light 19740, 19738. This diffuse reflected non-polarized light is then mixed with the diffused light 19722, 19724 provided by the light sources 19704, 19708 on its side and then back to the polarization beam splitter film 19714, The light 19730, 19728 having the state is reflected toward the reflection image light source, the light 19738, 19740 having the other polarization state is transmitted, and this process is repeated continuously. Accordingly, in this embodiment of the present disclosure, the light in the other polarization state is continuously recirculated, thereby increasing the efficiency of the display assembly 19702 because the light source (not shown) provided by the dual light sources 19704, 19708 Since both polarization states of light 19718 and 19720 are used to illuminate reflective image light source 18720. The increased diffuse reflection of the recycled light also improves the uniformity of the illumination light provided to the reflected image light source 18720. Image light (19732, 19734) can be provided straight to the relevant imaging optics.

197 and similar to that described above may be used in other embodiments having a display assembly having a flat surface on the sides of the curved polarizing beam splitter film. In this embodiment, since the sides of the reflective polarizer film are flat, the light from the side illumination maintains the illumination uniformity provided by the diffuser.

In a further embodiment of the display assembly shown in Figure 197, a solid film holder may be used, wherein light of another polarization state is recycled to improve efficiency. In this embodiment, the sides of the curved polarizing beam splitter film may be flat or curved.

FIG. 198 shows a schematic illustration of a method for manufacturing a front reflector (19902) as shown in FIG. 199 with a curved reflective beam splitter film (19808) and a dual light source on the sides. In Figure 198, dual light sources are not shown because they may be part of other assembly steps or may be in a peripheral module. A flow chart of the assembly method is provided in FIG. In this method, at step 20402, an upper film holder 19810 and a lower film holder 19812 are provided. The upper and lower film holders 19810 and 19812 can be made of any transparent material by diamond turning, injection molding, compression molding or grinding. A combination of materials and fabrication techniques is selected to provide a top film holder 19810 and a bottom film holder 19812 with low birefringence. Suitable low birefringent materials for the film holders (19810, 19812) include Zeonex F52 from Zeon Chemicals, APL5514 from Mitsui, or OKP4 from Osaka Gas. The surfaces in the upper and lower film holders that will be in contact with the curved polarizing beam splitter film 19808 are matched to retain the film 19808 in place with the desired shape and angle, Therefore, the image light can pass through the front light 19902 without being much deflected. In step 20404 the lower film holder 19812 holds the lower film holder 19812 either by adhesive binding or in a constant relationship (with contact or at a specified distance) to the reflective image light source 18720 And is attached to reflective image light source 18720 by providing a peripheral structure. In step 20408, the polarizing beam splitter film is curved. Subsequently, in step 20410, the curved polarizing beam splitter film 19808 is positioned in the lower film holder 19812 and the upper film holder 19810 is positioned on top, thereby forming the polarizing beam splitter film 19808 into the upper Conform to the matched surfaces of film holder 19810 and lower film holder 19812. In an alternative embodiment of this method, an adhesive is applied to the upper film holder 19810 or the lower film holder 19812 such that the polarizing beam splitter film 19808 is bonded to the upper film holder 19810 or the lower film holder 19812. [ . At step 20412, diffusers 19802 and 19804 are attached to the sides of the lower film holder 19812. A schematic illustration of an assembled front light 19902 is shown in FIG. Similar methods can be used to produce the front illumination shown in Figures 191, 194, and 195. The order of the assemblies can be changed within the scope of the present disclosure.

In an alternative embodiment to the previously described method, before the film holders 19810, 19812 are attached to the diffusers 19802, 19804 or the reflected image light source 18720 or any other single piece, Assembled with a beam splitter film (19808). Steps 20402, 20408 and 20410 are then successively performed to produce a solid film holder having a curved polarizing beam splitter film 19808 therein, as shown in Figures 191, 194 and 195 similarly. Is done. Reflected image light source 18720 and diffusers 19802 and 19804 are later attached (steps 20404 and 20412).

Various methods can be used to retain the reflective beam splitter film in place between the upper film holder and the lower film holder. The film may be bonded in place to the upper or lower film holder. The upper or lower film holder may be attached to the peripheral structural piece (not shown) or to an associated imaging optical system (not shown). When the reflective beam splitter film is a polarizing beam splitter film with a wire grid polarizer, the performance of the wire grid polarizer may deteriorate if an adhesive is used on the side of the wire grid structure. In this case, the polarizing beam splitter film may be bonded to the upper or lower film holder, depending on which side is adjacent to the wire grid structure, on the opposite side of the wire grid structure. The adhesive used to bond the polarizing beam splitter film to the film holder must be transparent and low birefringence. Examples of suitable adhesives include UV curing adhesives or pressure sensitive adhesives.

Figures 200 to 203 show a series of schematic illustrations of alternative methods of manufacturing frontal illumination with dual side illumination. 205 is a flowchart listing the steps of the method. In this method, the upper and lower film holders are cast into place around the curved reflective beam splitter film. In step 20502, the polarizing beam splitter film 20008 is bent. In step 20504, a curved polarizing beam splitter film 20008 is inserted into side frames having slots or mating pieces for holding the polarizing beam splitter film 20008 in a desired shape for front illumination (See the dual curved shape shown in Fig. In step 20508, the side frames are then attached to the reflected image light source 18720. At step 20510, spreaders 20002 and 20004 are attached to the sides of the side frames. At this point, the curved polarizing beam splitter film 20008 has its sides surrounded by side frames and diffusers 20002, 20004 and the bottom surrounded by a reflective image light source 18720. 200 shows a schematic illustration of a reflected image light source 18720 with attached diffusers 20002 and 20004 and a freestanding reflective beam splitter film 20008 supported at its edges to impart a desired shape to the reflective beam splitter film 20008. [ .

FIG. 201 shows holes in the side frames or surrounding structures used to introduce a transparent casting material under the curved reflective beam splitter film. As shown, a larger aperture 20102 near the reflective image light source 18720 is used to introduce the transparent casting material, while smaller holes 20104 are used to reflect the air into the reflective beam splitter film 20008, It is used to allow exhaust from below. In this way, a curved reflective beam splitter film 20008 forms a sealed cavity surrounded by diffusers 20002, 20004 and side frames or surrounding structures on a reflective image light source 18720. [ When the transparent casting resin is gradually injected into the hole 20102, the air from the sealed cavity escapes from the smaller holes 20104. When the cavity is filled, a portion of the transparent casting material flows out of the holes 20104, thereby preventing the formation of pressure below the reflective beam splitter film 20008 that will distort the shape of the film. The holes 20102 and 20104 are then closed to prevent leakage of the transparent casting material.

At step 20512, a transparent liquid casting material 20202 is poured onto the polarizing beam splitter film 20008, as shown in FIG. At step 20514, a transparent top sheet or plate 20302 is then attached to provide a flat top surface to the material 20202, as shown in Fig. Care must be taken to ensure that no air is trapped beneath the sheet when a flat sheet of transparent material is attached to the transparent casting material. A stop may be provided to the surrounding structure to hold the sheet of flat transparent material parallel to the reflected image light source.

The transparent liquid casting material may be any transparent liquid casting material such as epoxy, acrylic or urethane. The same transparent liquid casting material as the lower film holder should be used for the upper film holder so that the image light is exposed to the solid block of uniform optical thickness and the image light is not deflected by the surfaces of the bent polarizing beam splitter film . Transparent liquid casting materials can be cured by allowing curing time, by exposure to UV, or by exposure to heat. Curing of the transparent casting material can be done in a single step or in multiple steps. The curing of the lower portion as shown in Fig. 201 can be performed before casting of the upper portion shown in Fig. As another alternative, the curing of the entire casted front lighting can be done after the step shown in Fig.

The advantage of the method shown in Figures 200 to 203 is that close contact is obtained between the transparent casting material and the reflective beam splitter film so that light can pass through parts of the frontlight without interference. A casting method may also be used for the solid top or bottom film holder so that only the top or bottom film holder is cast. Figures 200 to 203 show that front lighting with curved surfaces is produced, but this method can also be used to produce front lighting with a flat surface.

In a further embodiment, one of the film holders is fabricated as a solid single piece and the other film holder is cast with the curved polarizing beam splitter film in place. Prior to casting the other film holder in place, the curved polarizing beam splitter film can be bonded to the solid piece. In this way, the cast film holder will have intimate contact with the surface of the polarizing beam splitter film. The material used in the solid film holder must have the same refractive index as the cast film holder in order to avoid deflecting the image light when the image light travels from the reflected image light source to the associated image optical system. An example of moderately matched materials is APEC 2000 from Bayer, which can be injection molded with a refractive index of 1.56, and EpoxAcast 690 from Smooth-On, which can be cast with a refractive index of 1.565.

In another further embodiment of this method, a solid film holder is fabricated using a multistage molding process as shown in the flowchart of Figure 206. In step 20602, the lower film holder is molded. Suitable molding techniques include injection molding, compression molding or casting. In step 20604, the polarizing beam splitter film is curved. In step 20608, a curved polarizing beam splitter film is placed on the molded bottom film holder and then placed in a mold for the top film holder as an insert. In step 20610, the upper film holder is then molded onto the curved polarizing beam splitter film and the lower film holder. The end result is a solid film holder with an internally curved polarizing beam splitter film as shown in Figures 191, 194 and 195. The advantage of this multistage molding technique is that the curved polarizing beam splitter film is in close contact with the surface of the bottom film holder and the bottom film holder is in close contact with the curved polarizing beam splitter film. In a preferred embodiment, the refractive indices of the upper and lower film holders are the same within 0.03. In a further preferred embodiment, the glass transition point of the material for the lower film holder is higher than the glass transition point for the material of the upper film holder, or the polarizing beam splitter film and lower film The material for the lower film holder is crosslinked so that the lower film holder is not deformed when formed on the holder. Examples of suitable combinations of injection-moldable materials are Tg of 139C and Zeonex E48R from Zeon Chemicals having a refractive index of 1.53 and Tg of 177C and cyclic olefin such as Topas 6017 from Topas Advanced Polymers with a refractive index of 1.53. Material.

It will be appreciated that certain embodiments of the AR eyepiece of this disclosure have a high modulation transfer function that allows for a combination of resolution levels and device size (e.g., eyeglass frame thickness) that were not previously achievable. For example, in some embodiments, the virtual image pixel resolution level presented to the user may be in the range of about 28 to about 46 pixels per degree.

105A to 105C, the angle of the curved wire grid polarizer controls the direction of the image light. The curved surface of the curved wire grid polarizer controls the width of the image light. The curved surface allows the use of a narrow light source because curved surfaces diffuse light when the light impinges on the curved surface and then bends / reflects the light to uniformly illuminate the image display. Again, the video light passing through the wire grid polarizer is not disturbed. As such, curved surfaces also enable miniaturization of the optical assembly.

21 and 22, the augmented reality eyepiece 2100 includes a frame 2102 and left and right earpieces or glasses leg piece 2104. A protective lens 2106 such as a ballistic lens is mounted in front of the frame 2102 in order to protect the user's two eyes or in the case of a prescription lens, have. The front portion of the frame may also be used to mount a camera or video sensor 2130 and one or more microphones 2132. [ Although not shown in FIG. 21, waveguides are mounted behind the protective lens 2106 in the frame 2102, one on each side of the center or adjustable nose bridge 2138. The front cover 2106 may be interchanged and thus the tint or prescription may be changed immediately for a particular user of the augmented reality device. In one embodiment, each lens is quickly interchangeable, which allows different prescriptions for each eye. In one embodiment, as discussed elsewhere herein, the lens is quickly interchangeable by a snap-fit. Certain embodiments may have only a projector and a waveguide combination on one side of the eyepiece, while the other side may be filled with a regular lens, a reading lens, a prescription lens, or the like. Each of the left and right earphones 2104 may be mounted vertically with a projector or micro-projector 2114 or other image light source on top of a spring-loaded hinge 2128 for easier assembly and vibration / . Each eyeglass leg portion also includes an eyeglass leg housing 2116 for mounting an associated electronic device to the eyepiece, each having an elastomeric head grip pad (not shown) for better holding on the user 2120). Each eyeglass leg piece also includes an orifice 2126 for mounting a wrap around earphone 2112 and a headstrap 2142 extending therefrom.

As will be seen, the spectacle leg housing 2116 includes an electronic device associated with an augmented reality eyepiece. The electronic device includes several circuit boards as shown for a microprocessor and radio portion 2122, a communications on-chip (SOC) 2124 and an open multimedia applications processor (OMAP) processor board 2140, . A communication system on a chip (SOC) may be one or more communications, including a wide local area network (WLAN), a BlueTooth communication, a frequency modulation (FM) wireless unit, a global positioning system (GPS), a triaxial accelerometer, one or more gyroscopes, Lt; RTI ID = 0.0 > function. ≪ / RTI > In addition, the right eyeglass leg piece may include an optical trackpad (not shown) on the eyepiece and on the outside of the eyeglass leg piece for user control of one or more applications.

In one embodiment, a digital signal processor (DSP) is programmed and / or configured to receive video feed information to configure the video feed to drive whatever type of image light source is being used in the optical display . A DSP may include a bus or other communication mechanism for communicating information, and an internal processor coupled to the bus to process the information. A DSP is a random access memory (RAM) or other dynamic storage device (e.g., dynamic RAM, static RAM, and synchronous DRAM) coupled to a bus for storing information and instructions to be executed, Etc. < / RTI > The DSP may be a non-volatile memory (e.g., read only memory (ROM) or other static storage device (e.g., PROM (programmable ROM), erasable PROM), and EEPROM (electrically erasable PROM)], and the like. The DSP may be a special purpose logic device (e.g., an application specific integrated circuit) or a configurable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array . ≪ / RTI >

The DSP may include at least one computer readable medium or memory for holding instructions programmed to drive the optical display and for containing data structures, tables, records, or other data necessary to drive the optical display . Examples of computer readable media suitable for the application of this disclosure include, but are not limited to, a compact disk, a hard disk, a floppy disk, a tape, a magneto optical disk, a PROM (EPROM, EEPROM, Flash EPROM), a DRAM, a SRAM, Other physical media having a pattern of punch cards, paper tapes, or holes, a carrier wave (described below), or a computer readable medium, such as a computer readable medium, But may be any other medium. Various forms of computer readable media may be involved in carrying out one or more sequences of one or more instructions with respect to an optical display for execution. The DSP may also include a communication interface that provides data communication coupling to a network link that may, for example, be connected to a local area network (LAN) or other communication network such as the Internet. A wireless link may also be implemented. In any such implementation, a suitable communication interface may transmit and receive electrical, electromagnetic, or optical signals that convey a digital data stream representing various types of information (such as video information) to the optical display.

The eyepiece may perform context-aware video acquisition that adjusts the video capture parameters based on the observer's movements, where the parameters may be video resolution, video compression, frames per second rate, etc. . Recording video through an eyepiece to a wearer, by recording an image taken via an integrated camera or transmitted from an external video device, by playing back the video through an eyepiece (by means of a method and system as described herein) Streaming live video from a video camera (e.g., video conferencing, live broadcast news feeds, video streams from other eyepieces) or from an integrated camera (e.g., from a non-line-of-sight camera) An eyepiece can be used for video applications. In embodiments, the eyepiece can accommodate multiple video applications being presented to the wearer at one time-for example, viewing an external video link being streamed while playing a video file stored on the eyepiece have. The eyepiece can provide a 3D viewing experience, such as by providing an image to either eye, or, alternatively, provide a reduced 3D experience, such as providing a reduced amount of content to either eye of either eye. have. The eyepiece can be a text-enhanced video, such as when the audio situation is too loud to hear the included audio, when the audio is in a foreign language to the user, when the user wants to record a text transcription of the audio, enhanced video.

In embodiments, the eyepiece may provide context aware video applications such as adjusting at least one parameter of video capture and / or viewing as a function of the wearer's environment. For example, a wearer of an eyepiece may be presented with an eyepiece in an external environment where the wearer needs to focus more on the outside environment than on video, in which case the presentation is less distracting (E.g., adjustment of spatial resolution; adjustment of frames per second; replacement of a video presentation with a static image representing the content of the video, a photograph of the person being stored, a single frame from video, etc.) do. In other cases, the video can be captured by the integrated camera on the eyepiece in situations where the wearer is moving (e.g., walking, jumping, riding, driving), and in this case, At least one parameter is adjusted to aid in the adjustment (e.g., adjusting during the time the eyepiece detects a fast movement of the video to be blurred, adjusting during the wearer is walking or slowly moving, .

In embodiments, the at least one parameter may include at least one of a spatial resolution parameter (e.g., a number of pixels per unit area, a number of specific color pixels per unit area, only one (& Number of frames presented per unit time, data compression, periods not recorded / presented, and so on.

In embodiments, at least one parameter may be determined based on an input sensed by the eyepiece, such as a motion detection input (e.g., to determine a head motion, e.g., a fast head motion, to determine slow head motion) Movement in the environment over the processing of images received through the integrated camera or of the ambient environment in which the video is captured to determine relative motion between the wearer and the environment, (Described herein), ambient light, and / or acoustic conditions to determine if the wearer is distracted.

In embodiments, the eyepiece may be associated with reducing movement or environmental effects on the quality of the wearer ' s video experience, such as being stored upon capturing video for light motion, bouncing, Image processing; Adjustment of background illumination and / or acoustic environment by adjusting color mixing, brightness, and the like. The choice of process may be a function of the sensed input, environmental conditions, video content, and the like. For example, in some cases, a high quality image may be preferred, and therefore a reduction in quality under certain circumstances is not tolerated, thereby causing the video to pause under that circumstance. In other cases, video and / or audio compression may be applied if a situation is determined to render capture of a reasonable level of quality impossible, but where certain continuity of capture is still desired. The treatment is also different for each eye of the eyepiece, for example for the dominant eye of the wearer, with respect to different environmental conditions experienced in one eye and in the other eye, Can be applied. In order to check the ambient light level for possible adjustments to the display of the content - for example, to determine which color channel compression and / or manipulation should be performed based on the environment, to make the surroundings look or not well visible To modify the color curve / palette, to change the color depth, to change the color curve, to change how the colors are compressed, etc. - If an embedded sensor is used, Can compensate for a bright environment.

In embodiments, the eyepiece may be configured to go into the screen shot mode while continuing the audio portion of the video when the condition is exceeded, such that motion of the eyepiece exceeds a predetermined amount as a result of the sensed condition, Stopping video shooting if it is enough to degrade the quality level, triggering changes in the video presentation when the level of motion in the received video is exceeded, and so on.

In embodiments, the eyepiece may initiate operation as a result of receipt of a control signal. The control signal may be based on the position of the eyepiece, what the eyepiece is currently viewing, or on a user gesture. This action may be to upload or download video captured by the eyepiece from a storage location. This operation may be initiated only upon receipt of the control signal itself or by receipt of a control signal and a confirmation control signal initiated by the user. This action may be the initiation of a process to move to a specific location within the video being displayed by the glasses, bookmarking a specific location in the video being displayed by the glasses, and the like.

In embodiments, adjustments made as a result of the sensed condition may be controlled through user preferences, organizational policies, state or federal rules, and the like. For example, whatever the sensed input represents, it may be desirable to always provide a specific quality, resolution, compression, and the like.

In one example, the wearer of the eyepiece may be in an environment where his head, and therefore the integrated camera of the eyepiece, is rapidly shaking while the eyepiece is recording the video. In this case, the eyepiece can adjust at least one parameter to reduce the degree to which shaken video is captured-for example, increasing the compression applied to the video, reducing the number of frames captured per unit period For example, capturing frames every few seconds), discarding frames with large changes in image per frame, reducing spatial resolution, and the like.

In one example, the wearer of the eyepiece may be videoconferencing through the eyepiece, wherein the eyepiece senses that the wearer is moving through the motion sensor. As a result, for a video of one of the other participants or the video of the user being sent to other members, the static video can replace the participant's video feed during this move. In this way, the effect of distracting the attention of the wearer ' s movement to the wearer and / or other participants in the videoconference can be reduced.

In one example, the wearer can start driving the car while watching the video, which can be a safety issue if the wearer continues to watch the video currently being displayed. In this example, the eyepiece detects the movement of the environment, such as indicating that it is in the car, and the wearer's eye movements cause the wearer's eye movements to move quickly between the visual line of sight (driving direction) In the case of representing something, the viewing experience can be changed to make distractions less noticeable. The eyepiece may, for example, present the observer with options to stop and continue the video. The eyepiece can also detect and adjust the movement of the environment, such as being in an automobile, riding a bicycle, walking, or distinguishing between others.

In one example, a wearer may need assistance in moving to a place, whether in a car, on a bicycle, or walking. In this example, the eyepiece will display a video navigation application to the user. The navigation commands displayed to the user by the eyepiece may be selected by the control signal. The control signal may be generated by the location designated by the wearer, by the one currently being displayed on the glasses, or based on the destination the wearer is talking about. The place may be one of meal / drinking, education, ceremony, exercise, house, outdoors, retail store, traffic facility location,

In one example, the wearer may be capturing video in a situation where the environment distracts or degrades the quality of the video at some point (e.g., due to color contrast, blending, depth, resolution, brightness, etc.). The eyepiece can adjust conditions in the guitar under different lighting conditions, under poor acoustic conditions, when the wearer is outdoors and indoors. In this case, the eyepiece can adjust the image and sound recorded to produce a video product that is a more effective representation of the content being captured.

In embodiments, the eyepiece may be a computer, such as a monitor, display, TV, keyboard, mouse, memory storage device (e.g., external hard disk, optical drive, solid state memory), network interface And may provide an external interface to the peripheral device. For example, the external interface can be a direct connection to an external computer peripheral device (e.g., directly connected to the monitor), an external computer peripheral device (e.g., via a central external peripheral interface), a wired connection, Lt; / RTI > In one example, the eyepiece may be coupled to a central, external peripheral interface that provides connection to an external peripheral device, wherein the external peripheral interface includes a computer processor, memory, operating system, peripheral driver and interface, USB port, An external display interface, a network port, a speaker interface, a microphone interface, and the like. In embodiments, the eyepiece may be connected to a central external peripheral interface by a wired connection, a wireless connection, a cradle directly, or the like, and when the eyepiece is connected, the eyepiece may be provided with computing equipment similar or identical to a personal computer . In embodiments, the device selected to be controlled by the eyepiece may be selected by the user viewing the eyepiece, pointing to the eyepiece, selecting from the user interface displayed on the eyepiece, and the like. In other embodiments, the eyepiece may display the user interface of the device when the user is looking at or pointing to the device.

The frame 2102 has a general shape of wraparound sunglasses. The sides of the glasses include a shape memory alloy strap 2134 such as a nitinol strap. Nitinol or other shape memory alloy straps are tailored to the user of the augmented reality eyepiece. The strap is worn by the user and adjusted to maintain its trained or preferred shape when closer to body temperature. In embodiments, the feet of the eyepiece may provide user eye width alignment techniques and measurements. For example, the arrangement and / or alignment of the display projected onto the wearer of the eyepiece can be adjusted to accommodate the different eye widths of different wearers. Arrangement and / or alignment may be by detecting the position of the wearer's binocular through an automatic-optical system (e.g., by iris or pupil detection), by passive-wearer, or the like.

Other features of this embodiment include a detachable noise canceling earphone. As shown in this figure, the earphone is adapted to be connected to the control of the augmented reality eyepiece for delivering sound to the ear of the user. The sound may include input from a wireless Internet or communication function of the augmented reality eyepiece. The earphone may also include a flexible, deformable plastic or foam portion, thus protecting the inner ear of the user in a manner similar to an earplug. In one embodiment, the earphone limits the input to the user's ear to about 85 dB. This allows the wearer to hear normally while providing protection from gun noise or other explosive noise and listening in a high background noise environment. In one embodiment, the control of the noise canceling earphone has automatic gain control for very quick adjustment of the elimination feature in protecting the wearer's ear.

23 shows the layout of the projector 2114 arranged vertically in the eyepiece 2300 where the illumination light passes through one side of the PBS from the bottom to the top and is mounted on a display and imager board (which may be silicon backed) and is refracted as image light. In this case, it collides with the inner interface of the triangular prism constituting the polarizing beam splitter, is reflected from the projector, and enters the waveguide lens. In this example, the dimensions of the projector are shown to be 11 mm wide, 11 mm wide, the distance from the end of the imager board to the centerline of the image is 10.6 mm, and the distance from the image centerline to the end of the LED board is about 11.8 mm have.

A detailed assembled view of the components of the projector discussed above can be seen in Fig. This figure shows how compact the microprojector 2500 is, for example, when assembled near the hinge of an augmented reality eyepiece. The microprojector 2500 includes a holder 2508 for mounting a part of the housing and the optical parts. When each color field is imaged by optical display 2510, the corresponding LED color is turned on. An RGB LED light engine 2502 mounted on the heat sink 2504 is shown in the lower portion. The holder 2508 is mounted on top of the LED light engine 2502 and the holder is equipped with a light tunnel 2520, a diffuser lens 2512 (to remove the hot spot), and a condenser lens 2514 . The light travels from the condensing lens into the polarization beam splitter 2518 and then to the view lens 2516. The light then refracts onto a liquid crystal on silicon (LCoS) chip 2510, where an image is formed. The light for the image is then reflected back through the field lens 2516, polarized, reflected at 90 ° and passed through the polarizing beam splitter 2518. The light then exits the microprojector and is transmitted to the optical display of the glasses.

4 V)이 필요할 수 있다.FIG. 26 shows an exemplary RGB LED module 2600. FIG. In this example, the LED is a 2x2 array with one red die, one blue die, and two green dies, and the LED array has four cathodes and a common anode. The maximum current can be 0.5 A per die, and for green and blue dies the maximum voltage (

4 V) may be required.

In embodiments, the system may utilize an optical system capable of producing a monochrome display to the wearer, which may provide advantages such as image sharpness, image resolution, frame rate, and the like. For example, the frame rate can be three times (than the RGB system), which can be useful in night vision and similar situations where the camera is shooting the surroundings, in which case the images can be processed and displayed as content. The image may be brighter (e.g., three times brighter when three LEDs are used), or may provide space savings with only one LED. When multiple LEDs are used, the LEDs may be the same color or may be different (RGB). The system may be a switchable monochromatic / color system (RGB is used), but when the wearer wishes to have a single color, an individual LED or multiple LEDs may be selected. To generate white light, all three LEDs may be used simultaneously, not sequencing. Using three LEDs in a manner other than sequencing can be similar to any other white light with a three times higher frame rate. Depending on the application being executed, a "switch" between monochrome and color may be done manually or manually (e.g., by physical buttons, GUI interface selection). For example, the wearer may go to night vision mode or fog clearing mode, and the processing portion of the system may require the eyepiece to be in a monochrome high refresh rate mode It is automatically determined that there is.

Fig. 3 shows an embodiment in which horizontally arranged projectors are in use. The projector 300 may be disposed on the arm portion of the eyepiece frame. The LED module 302 under the processor control 304 may emit one color at a time in rapid succession. The emitted light travels down the light tunnel 308 and through at least one homogenization lenslet 310 and then to the polarization beam splitter 312 and to the LCoS display 314 where the full color image is displayed Can be biased. The LCoS display can have a resolution of 1280 x 720p. The image is then reflected back upwards through the polarizing beam splitter, reflected from a fold mirror 318, and continues through the collimator to exit the projector and into the waveguide. The projector may include a diffractive element to remove the aberration.

In one embodiment, the interactive head-mounted eyepiece includes an optical assembly in which a user views the surrounding environment and the displayed content, wherein the optical assembly includes a corrective element for correcting a user view of the ambient environment, a freeform optical waveguide that enables internal reflection, and a coupling lens that is arranged to direct the image from the optical display, such as an LCoS display, towards the optical waveguide. The eyepiece further includes an integrated image source, such as one or more integrated processors for processing content for display to a user and projector equipment for introducing the content into the optical assembly. In embodiments where the image light source is a projector, the projector facility includes a light source and an optical display. Light from a light source such as an RGB module is emitted under the control of the processor and passes through a polarization beam splitter where light is polarized and then reflected from an optical display, such as an LCoS display or an LCD display (in certain other embodiments) And enters the optical waveguide. The surface of the polarizing beam splitter can reflect the color image from the optical display into the light pipe. The RGB LED module may sequentially emit light to form a color image that is reflected from the optical display. The correction factor may be a see-through correction lens attached to the optical waveguide to allow the ambient light to be properly viewed regardless of whether the image light source is on or off. The correction element may be a wedge-shaped correcting lens, prescription, colored, coated or otherwise. A free-form optical waveguide that can be described by a higher-order polynomial may include a dual freeform surface that allows curvature and magnification of the waveguide. The curvature and size adjustment of the waveguide makes it possible to place it in the frame of the interactive head-mounted eyepiece. This frame may be sized to fit the user's head in a manner similar to sunglasses or glasses. Other components of the optical assembly of the eyepiece include a homogenizer-a light from the light source is propagated through the homogenizer so that the beam of light is uniform, and a collimator that improves the resolution of the light entering the light pipe.

In embodiments, the prescription lens may be mounted inside or outside the eyepiece lens. In some embodiments, prescription power may be distributed to prescription lenses mounted on the outside and inside of the eyepiece lens. In embodiments, prescription correction is provided by a calibration optics attached to an eyepiece lens or component of the optical assembly (such as a beam splitter) by surface tension or the like. In embodiments, the calibration optical system may be provided at one location in the optical path and at some other location in the optical path. For example, one half of the correcting optics may be provided on the outside of the converging surface of the beam splitter, and the other half may be provided on the inside of the converging surface. In this way, calibration can be provided for the image light from the internal light source and for the scene light differently. That is, the light from the light source can be corrected only by the part of the correcting optical system inside the converging lens, since the image is reflected toward the user's eye, and the scene light is transmitted through the beam splitter And thus can be calibrated through both parts, since they are exposed to different optical corrections. In other embodiments, the optical assembly associated with the beam splitter may be a sealed assembly, for example, to make the assembly waterproof, dustproof, etc., wherein the inner surface of the encapsulated optical assembly has a correcting optical system And the outer surface of the encapsulated optical assembly has another portion of the correcting optics. At least a suitable optical system available as Prisms (also referred to as Fresnel Prisms), Aspheric Minus Lenses, Aspheric Plus Lenses, and Bifocal Lenses is available from 3M Lt; / RTI > Press-On Optics. The calibration optical system includes a user detachable and interchangeable diopter configured to detachably attach to a position between the user's eye and the displayed content to correct the visual acuity of the user with respect to the displayed content and the surrounding environment diopter calibration facility. The diopter correction facility may be configured to be mounted on the optical assembly. The diopter correction device may be configured to be mounted on a head-mounted eyepiece. A diopter correction device can be mounted using a friction fit. A diopter correction device can be mounted using a magnetic attachment facility. The user can select from a plurality of different diopter correction equipments depending on the user's visual acuity.

In embodiments, the present disclosure is directed to an apparatus and method for correcting a user ' s visual acuity with respect to displayed content and the surrounding environment, It is possible to provide a correcting optical system that " snap-on " the eyepiece as if it was configured to be detachably attached. The diopter correction device may be configured to be mounted on an optical assembly, on a head mounted eyepiece, or on a guitar. The diopter correction apparatus can be mounted using friction pits, magnetic attachment facilities, and the like. The user can select from a plurality of different diopter correction equipments depending on the user's visual acuity.

Referring to FIG. 4, the image light that can be polarized and collimated selectively passes through the waveguide 414 through the display coupling lens 412 - which may or may not be additional to the collimator itself or the collimator. . In embodiments, waveguide 414 may be a free-form waveguide, wherein the surface of the waveguide is described by a polynomial equation. The waveguide may be rectilinear. The waveguide 414 may include two reflective surfaces. When entering the waveguide 414, the image light may collide with the first surface at a larger incident angle - a total internal reflection (TIR) occurs if the critical angle - exceed it. The image light can undergo a TIR bounce between the first and second facing surfaces and ultimately reach the active viewing area 418 of the composite lens. In one embodiment, the light can make a TIR bounce at least three times. The thickness of the composite lens 420 may not be uniform because the waveguide 414 is tapered to allow the TIR bounce to ultimately exit the waveguide. Distortion through the viewing area of the composite lens 420 may be achieved by placing a wedge-shaped orthodontic lens 410 along the length of the free-form waveguide 414 to provide a uniform thickness across at least the viewing area of the lens 420 , Can be minimized. The calibrating lens 410 may be a prescription lens, a tinted lens, or the like, mounted on the inside or outside of the eyepiece lens, or in some embodiments, mounted on both the inside and outside of the eyepiece lens. A polarized lens, a ballistic lens, or the like.

In some embodiments, the optical waveguide may have a first surface and a second surface that allow for total internal reflection of light entering the waveguide, while the waveguide may have an internal angle of incidence, . ≪ / RTI > The eyepiece may include a mirrored surface on the first surface of the light pipe to reflect the displayed content toward the second surface of the light pipe. Thus, the mirror surface enables total reflection of light entering the light pipe or reflection of at least a portion of the light entering the light pipe. In embodiments, the surface may be 100% mirrored or a lower proportion of the mirrored surface. In some embodiments, instead of a mirror surface, the gap between the waveguide and the correction element can cause reflection of light entering the waveguide at an angle of incidence that will not cause TIR.

In one embodiment, the eyepiece comprises an integrated image light source, such as a projector, for introducing content for display from the side of the light pipe adjacent to the arm of the eyepiece to the optical assembly. Unlike prior art optical assemblies in which image injection occurs from the upper side of the light pipe, this disclosure provides an image injection from the side of the wave guide to the wave guide. The aspect ratio of the displayed content is approximately square to approximately rectangular, and the major axis is approximately horizontal. In embodiments, the aspect ratio of the displayed content is 16: 9. In embodiments, achieving a rectangular aspect ratio for displayed content when the long axis is approximately horizontal can be done through rotation of the injected image. In other embodiments, this may be done by stretching the image until the image reaches the desired aspect ratio.

Figure 5 shows a design for a waveguide eyepiece showing sample dimensions. For example, in this design, the width of the coupling lens 504 may be between 13 and 15 mm, and the optical display 502 is optically coupled in series. These elements can be arranged on the arm or on both arms of the eyepiece in an overlapping manner. The image light from the optical display 502 is projected through the coupling lens 504 into the free-form waveguide 508. The thickness of the composite lens 520 including the waveguide 508 and the corrective lens 510 may be 9 mm. In this design, the waveguide 502 allows an exit pupil diameter of 8 mm if the eye clearance is 20 mm. The resulting see-through view 512 may be about 60 to 70 mm. The distance (dimension a) from the pupil to the image light path when the image light enters the waveguide 502 can be about 50 to 60 mm, which can accommodate most of the human head width. In one embodiment, the field of view may be larger than the pupil. In embodiments, the field of view may not fill the lens. It will be appreciated that these dimensions are for a particular exemplary embodiment and should not be construed as limiting. In one embodiment, waveguides, snap-on optics, and / or calibrating lenses may include optical plastic. In other embodiments, the waveguide, snap-on optics, and / or calibrating lens may be formed of glass, marginal glass, bulk glass, metal glass, palladium-enriched glass, Suitable glasses may be included. In embodiments, the waveguide 508 and the corrective lens 510 may be made of different materials that are chosen so as to cause little or no chromatic aberration. These materials may include diffraction gratings, holographic gratings, and the like.

In the embodiments as shown in FIG. 1, when two projectors 108 are used for the left and right images, the projected image may be a stereoscopic image. To enable stereo viewing, the projectors 108 may be arranged at adjustable distances to allow adjustment based on the same spatial distance from each other to the individual eyewear wearers. For example, a single optical assembly may include two independent electro-optic modules with individual adjustments for horizontal, vertical, and tilt placement. As another alternative, the optical assembly may include only a single electro-optic module.

146 to 149 schematically illustrate an embodiment of an augmented reality (AR) eyepiece 14600 (without eyewear leg part) in which placement of images can be adjusted. 146 and 147 are front and rear perspective views of the AR eyepiece 14600, respectively. In this embodiment, the electronic devices and portions (all combined 14602) of the projection systems are located above the lenses 14604a, 14604b. The AR eyepieces 14600 have two projection screens 14608a, 14608b that are adjustably suspended on an adjustment platform 14610 on the wearer's side of the lenses 14604a, 14604b. The adjustment platform 14610 has mounted thereon a mechanism that independently adjusts the lateral position of the AR eyepiece 14600 to the nose pads 14612 and the tilt of each of the projection screens 14608a, 14608b.

The mechanism for adjusting the position of one or both of the display screens may be manually controlled by a manual-activated (e.g., button) or software-activated motor, a manual control device (thumbwheel, A lever arm, etc.), or by a combination of both a motorized device and a manual device. The AR eyepiece 14600 uses a passive device that will now be described. Those skilled in the art will appreciate that the adjustment mechanism is designed to separate the lateral adjustment and the tilt adjustment.

Figure 148 shows a rear perspective view of a portion of the left side of the wearer of the AR eyepiece 14600 where the adjustment mechanism 14614 on the adjustment platform 14610 for the projection screen 14608a is more clearly shown. The projection screen 14608a is mounted on a frame 14618 which is fixedly attached to (or is part of) a movable carriage 14620. On its nose pads 14612 side, the carriage 14620 is rotatably and slidably mounted on a carriage shaft 14622 in an arcuate groove of a first block 14624 attached to a control platform 14610 . On the side of his eyeglass leg, the carriage 14620 is supported by a yoke 14628 rotatably and slidably. 150, a yoke 14628 has a shaft portion 14630 fixedly attached to a carriage 14620 and coaxial with the carriage shaft 14622 to provide a rotational axis to the carriage 14620. [ The yoke 14628 is slidably and rotatably supported on the arcuate groove of the second support block 14632 attached to the adjustment platform 14610 (see Figure 151).

The yoke 14628 also has two parallel arms 14634a, 14634b extending radially outwardly from the shaft portion 146320. [ The free ends of the arms 14634a and 14634b have holes (e.g., holes 14638 in the arm 14634b) for capturing and capturing the shaft 14678 therebetween, (See FIG. 149). The arm 14634a has another portion 14640 to which the arm 14634a is attached to the shaft portion 14630 of the yoke 14628. [ The anchor portion 14640 has a through-hole 14642 for slidably capturing the pin 14660, as discussed below (see Figure 152).

Referring again to Figure 148, the adjustment mechanism has a first thumbwheel 14644 for controlling the lateral position of the projection screen 14608a and a second thumbwheel 14648 for controlling the tilt of the projection screen 14608a. The first thumbwheel 14644 partially extends through the slot 14650 in the adjustment platform 14610 and is threadably engaged and supported by a first threaded shaft 14652. The first threaded shaft 14652 is slidably supported in the through holes in the third and fourth support blocks 14654 and 14658 (see FIG. 151). The sides of the third and fourth blocks 14654, 14658 and / or the slot 14650 serve to prevent the first thumbwheel 14644 from moving in the lateral direction. Thus, rotating the thumb wheel 14644 about its axis (indicated by arrow A) causes the first threaded shaft 14652 to move in the lateral direction (indicated by arrow B). As best seen in Figure 152, the first threaded shaft 14652 has a pin 14660 extending radially outwardly from its nose bridge-side end. [Note that the screw of the first threaded shaft 14652 is not shown in the figure, but may be a single or multiple pitch screw.] The pin 14660 is secured to the anchor 14634a of the arm 14634a of the yoke 14628, And is slidably captured by the vertically oriented through hole 14642 of the portion 14640. The pin 14660 pushes the side of the nose pads 14612 of the through hole 14642 when the first thumbwheel 14644 is rotated in the direction to advance the first screw shaft 14652 in the lateral direction toward the nose pads 14612 Which in turn causes both yoke 14628, carriage 14620, frame 14618, and first projection screen 14608a to move laterally toward nose pads 14612 (see arrow C). Similarly, when the first thumbwheel 14644 is rotated in the opposite direction, the first projection screen 14608a moves laterally away from the nose pads 14612.

The second thumbwheel 14648 is used to control the tilt of the first projection screen 14608a about an axis defined by the carriage shaft 14622 and the yoke shaft portion 14630. Referring now to FIG. 153, the second thumb wheel 14648 is fixedly attached to a narrow portion 14662 of a hollow flanged shaft 14664. The flange portion 14668 of the flanged shaft 14664 receives the threaded shaft portion 14670 of the eyehook 14672 in a threaded manner. [Note that the screw of the threaded shaft portion 14670 is not shown in the figure, but may be a single or multiple pitch screw.) In use, the narrow portion 14662 of the flanged shaft 14664 is adjusted Rotatably passes a countersunk hole 14674 in platform 14610 (see Figure 151), whilst thumb wheel 14648 is on the lower side of control platform 14610 and eye hook 14672 The flange portion 14668 of the flanged shaft 14664 is on the upper portion and is captured in the countersunk portion of the dish-shaped hole 14674. Referring again to Figure 149, the eye of the eye hook 14672 is slidably engaged about a shaft 14678 which is captured in the holes in the free ends of the yoke arms 14634a, 14634b. As such, rotating the second thumbwheel 14644 about its axis (indicated by arrow D) causes the flanged shaft 14664 to rotate therewith, which causes the threaded shaft portion 14670 of the eye hook 14672 (Indicated by arrow E), which causes the eye of the eye hook 14672 to push the shaft 14678, which in turn causes the yoke 14628 to move about its axis Thereby tilting the first projection screen 14608a away from or towards the wearer (indicated by arrow F).

Referring again to Figure 148, it is noted that the electronics and parts of the projection system 14602a are located on a platform 14680 that is secured to the top of the carriage 14620. [ As such, the spatial relationship between the projection screen 14608a and its associated electronics and portions thereof in the projection system 14602a remains substantially unchanged by any lateral or tilt adjustments made to the projection screen 14608a have.

The AR glasses 14600 also include an adjustment mechanism similar to the adjustment mechanism 14614 just described for laterally positioning and tilting the second projection screen 14608b located to the right of the wearer of the AR eyepiece 14600 .

In one embodiment, the eyepiece may include a slanted or curved guide rail for IPD adjustment to further maintain the optical module in the curved frame. In some embodiments, the display operates in conjunction with such tilted or curved guide rails.

In embodiments, the display screen or screens of the AR eyepiece are arranged parallel to the line connecting the two eyes of the user. In some embodiments, the display screen or screens are rotated about their vertical axis, and thus their edges near the nose are angled at an angle in the range of about 0.1 to about 5 degrees from parallel to the line connecting the user & (I.e., "toe-in"). In some of these latter embodiments, the toe-in angle is permanently fixed, while in other embodiments the toe angle is user-adjustable. In some of the user adjustable embodiments, the adjustability is indicative of two or more predetermined positions (e.g., near convergence, medium distance convergence, and distant convergence) Location]. In other embodiments, the controllability is continuous. Preferably, in embodiments of the AR glasses that also include automatic vergence correction as disclosed herein, the amount of toe-in is taken into account in the version calibration. In the embodiments in which the toein is permanently fixed, the amount of toe in can be directly incorporated into the automatic version calibration without the need for a position sensor, but in user adjustable embodiments, preferably, A position sensor is used to communicate the amount of toyne present to the processor. In embodiments in which the toe angle is user adjustable, the adjustment can be made manually - for example, either directly or indirectly (e.g. via a drive train) about one or both of the display screens about its vertical axis Or may be motorized to achieve selectable rotation when operated by a user via a user interface or control switch.

In some cases, in order for the user's two eyes to rest on the user's eyes for long periods of time during which the user's two eyes are kept at a particular focus distance (e.g., reading, watching, playing ball, Feature can be used. The toein feature described above can be used to adjust the user's dynamic distance by effectively rotating the display screen to better align with the user ' s eyes.

In embodiments, the present disclosure provides a method and system for adjusting the mechanical pupil distance when the optical assembly of the eyepiece is configured to be adjustable in user position within the eyeglass frame, such as, for example, allowing the user to change the position of the optical assembly relative to the user & Can be provided. The position adjustment can control the horizontal position, vertical position, tilt, etc. of the optical assembly within the spectacle frame.

In embodiments, the present disclosure provides a method and system for determining the pupil alignment calibration factor to be used in an arrangement of other display content, for example, When performing a pupil alignment procedure that allows adjustment of the placement of the displayed content, digital pupil distance adjustment can be provided. The correction factor may include horizontal and / or vertical adjustment of the displayed content within the field of view. The correction factors may include a plurality of correction factors, each representing a distance correction factor from the real world object to be used when placing the content within the field of view based on the calculation of the distance to the real world object. The calibration factor may include a calibration process based on a plurality of calibration factors, each representing a distance calibration factor from the real world object to be used when placing the content within the field of view based on the calculation of the distance to the real world object. Placing the image may be adjusted on the display to move the image within the field of view. Moving the two images farther apart will make the imaged object look more distant, while moving the images closer together will make the object appear closer. The disparity of the object's position within the field of view for each eye is called the disparity. The parallax is related to the perceived distance the object is away from the user.

Referring now to Figure 173, an exploded view of the glasses is shown. An electronic device 17302, including a CPU, a display driver, a camera, a wireless unit, a processor, a user interface, etc., is above the binocular in the front frame of the glasses. The optics module 17308 is attached to the frame with a lens 17304 - which may be optional - covering it. The lens 17304 may be tinted or tintable. Although stereoscopic embodiments are shown here, it will be appreciated that a single optical system module 17308 can also be used. The electronic device 17302 is sealed with a cover 17314 that includes a physical user interface 17310 that may be a button, touch interface, rollerball, switch, or any other physical user interface. The physical user interface 17310 can control various aspects of the glasses, such as the functionality of the glasses, the applications running on the glasses, or the application programs controlling the external devices. The user can easily use the control feature by touching the lower feature of the frame to stabilize the frame while touching the control feature / UI at the top of the frame. Arms 17312 may rest on the ear and may include a strap for securing the glasses, a jack for the audio / earphone function or an external audio device, a battery 17318 or a power function. An optional battery 17318, as described herein, which may include any available battery type, may be located on either arm. The strap can be an ear band consisting of Nitinol or other shape memory alloy. The earbands may be in band form, or, as in Figure 177, earbands 17702 may be of the bent wire type to be thin, lightweight, and low in cost. For the sake of appearance, the frame may be of any color, the lens may be of any color, and the eyepiece arm or at least the tip of the arm may be colored. For example, the nitinol forming the tip of the arm may be colored.

Referring now to Figure 174, a battery is connected to an electronic device in the front frame, even through an operable hinge 17408, using a minimum number of wires and a wiring design that passes through the hinge in the wire guide 17404, Can be supplied. The wiring design may include a wire 17402 extending from the front frame electronics to an earphone located on the arm. FIG. 175 shows an enlarged version of FIG. 174 focused on wire 17402 passing through wire guide 17404. FIG. Figures 176a through 176c illustrate cut out a wire guide having various parts of the operating frame and inner glasses. This view shows the hinge from the user side of the frame. Figure 176a shows the most part cut out, Figure 176b shows the next most cut out, while Figure 176c shows the original appearance of the glasses.

6 illustrates an embodiment of an eyepiece 600 having a perspective or translucent lens 602. [ The projected image 618 can be seen on the lens 602. [ In this embodiment, there is a case where the image 618 being projected onto the lens 602 is an augmented reality version of the scene the wearer is looking at, and a tagged point of interest (POI) Is displayed. The augmented reality version may be enabled by a forward facing camera (not shown in Figure 6) that is embedded in the eyepiece that images what the wearer is seeing and identifies the location / POI. In one embodiment, the output of a camera or optical transmitter can be sent to an eyepiece controller or memory for storage, for transmission to a remote location, or for viewing by a person wearing an eyepiece or glasses have. For example, the video output may be streamed to a virtual screen that the user is viewing. The video output may thus be used to help determine the location of the user, or may be remotely transmitted to others to aid in locating the wearer or for any other purpose. Other detection techniques (GPS, RFID, manual input, etc.) can be used to determine the wearer's position. Using the location or identification data, the database can be accessed by the eyepiece to see if there is information that can be displayed overlay, projection or otherwise in conjunction with what is being viewed. Augmented reality applications and techniques will be further described herein.

7, an embodiment of an eyepiece 700 with a translucent lens 702 displaying streaming media (email application) and an incoming call notification 704 is shown. In this embodiment, it will be appreciated that although the media covers a portion of the viewing area, the displayed image can be placed anywhere in the field of view. In embodiments, the media may be more transparent or less transparent.

In one embodiment, the eyepiece may receive input from any external source, such as an external converter box. The source can be represented by a lens in the eyepiece. In one embodiment, when the external source is a telephone, the eyepiece can be used to position the phone to display a location-based augmented reality that includes a marker overlay from a marker-based AR application Can be used. In embodiments, a VNC client running on a processor of an eyepiece or associated device may be coupled to the computer and used to control the computer, wherein the wearer sees the display of the computer in the eyepiece. In one embodiment, content from any source, such as a display from a panoramic camera carried on a vehicle, a user interface to the device, an image from an unmanned airplane or helicopter, etc., may be streamed to the eyepiece. For example, a gun mounted camera can make it possible to shoot a target that is not in direct line of sight when the camera feed is sent to the eyepiece.

The lens may be a chromatic lens, such as a photochromic lens or an electrochromic lens. The electrochromic lens may include an integral discoloring material or discoloring coating that changes the opacity of at least a portion of the lens in response to a burst of charge applied by the processor over the discoloring material. For example, referring to FIG. 9, the discolored portion 902 of the lens 904 may be used to provide greater visibility by the wearer of the eyepiece when the portion is displaying content displayed to the wearer Is shown dark for. In embodiments, a plurality of discoloration areas that can be independently controlled, most of the lens, a sub-portion of the projected area, a programmable area of the lens and / or the projected area (controlled at the pixel level) May be on the lens. Activation of the discoloring material may be controlled through control techniques further described herein, or may be controlled by certain applications (e.g., streaming video applications, solar tracking applications, ambient brightness sensors, cameras that track the brightness in the field of view) Can be automatically enabled in response to a UV sensor built into the system. In embodiments, the electrochromic layer may be located between the optical elements and / or on the surface of the optical element on the eyepiece (on a correcting lens, on a ballistic lens, etc.). In one example, the electrochromic layer may consist of a stack of ITO (Indium Tin Oxide) coated PET / PC film and two electrochromic (EC) layers therebetween, The PET / PC layer can be omitted, thereby reducing the reflection (e.g., the layer laminate can include PET / PC - EC - PET / PC - EC - PET / PC). In embodiments, an electrically controllable optical layer may be provided as a liquid crystal based solution having a binary state of tint. In other embodiments, this tint (e-tint) of an alternative to forming a plurality of liquid crystal layers or optical layers such that certain layers or segments of the optical layer can be turned on or off in steps provides a variable hue Can be used. Electrochromic layers can generally be used for any of the electrically controlled transparencies in an eyepiece, including SPD, LCD, electrowetting, and the like.

In embodiments, the lens may have an angular sensitive coating that is capable of transmitting light waves having a low angle of incidence and reflecting light having a high angle of incidence (such as s-polarized light). The discoloration coating can be controlled, in part or in whole, by control techniques, etc., as described herein. The lens may be a variable contrast, and the contrast may be under the control of a push button or any other control technique described herein. In embodiments, the user may wear an interactive head-mounted eyepiece wherein the eyepiece includes an optical assembly through which the user views the surrounding environment and the displayed content. The optical assembly may include a correction element for correcting a user view of the environment, an integrated processor for processing content for display to a user, and an integrated image light source for introducing the content into the optical assembly. The optical assembly may include an electrochromic layer that provides display property adjustment that is dependent on displayed content requirements and environmental conditions. In embodiments, the display characteristics may be brightness, contrast, and the like. The ambient condition may be a level of brightness that would make the displayed content less noticeable to the wearer of the eyepiece if there is no display characteristic adjustment, wherein the display characteristic adjustment may be applied to the area of the optical assembly in which the content is being displayed .

In embodiments, the eyepiece may control brightness, contrast, spatial resolution, etc. for the eyepiece projection area (e.g., alter and improve the user view of the projected content for a bright or dark environment). For example, the user may be using the eyepiece under bright daylight conditions, and the display area may need to be changed in brightness and / or contrast so that the user can clearly see the displayed content. As another alternative, the viewing area surrounding the display area may be changed. In addition, the modified areas, whether or not they are within the display area, can be spatially oriented or controlled depending on the application being implemented. For example, only a small portion of the display area needs to be changed, such as when a small portion of the display area deviates from a predetermined or predetermined contrast ratio between the display area of the display area and the surrounding environment. In embodiments, it is possible to adjust the brightness of the displayed content such that the illumination condition of the surrounding environment and / or the brightness of the displayed content, which is adjusted only with a portion of the lens, fixed to include the entire display area, Some of the brightness, contrast, space range, resolution, etc. may change. The spatial extent (e.g., the area affected by the change) and resolution (e.g., display optical resolution) may vary across different portions of the lens, including high resolution segments, low resolution segments, single pixel segments, Different segments may be combined to achieve the viewing purpose of the program (s). In embodiments, techniques that implement variations in brightness, contrast, spatial range, resolution, etc. may be used in electrochromic materials, LCD technology, beads embedded in optical systems, flexible displays, suspension particle devices (SPDs) Techniques, colloidal techniques, and the like.

In embodiments, there may be various electrochromic layer activation modes. For example, a user may enter a sunglass mode in which the compound lens appears to be only slightly darkened, or the user may enter a "Blackout" mode in which the compound lens appears to be completely blackened .

One example of a technique that can be used to implement changes in brightness, contrast, spatial range, resolution, etc. may be electrochromic materials, films, inks, and the like. Electrochromism is a phenomenon represented by some materials that alter their appearance when electrical charge is applied. Depending on the particular application, various types of materials and structures may be used to construct the electrochromic device. For example, electrochromic materials include tungsten oxide (WO ₃ ), which is a major chemical used in the manufacture of electrochromic windows or smart glass. In embodiments, the electrochromic coating may be used on a lens of an eyepiece in implementing the modification. In another example, an electrochromic display can be used to implement an 'electronic paper' designed to mimic the appearance of a normal paper, where the electronic paper displays reflected light like normal paper. In embodiments, a gyricon (consisting of a polyethylene sphere embedded in a transparent silicone sheet, each sphere floating in the form of a bubble of oil and thus free to rotate), an electrophoretic display Electro-wetting, electro-fluidic, an interferometric modulator, a flexible substrate (e.g., an image is formed by rearranging charged pigment particles using an applied electric field), electronic ink technology, electro- Electrochromic devices can be implemented in a wide variety of applications and materials, including organic transistors embedded in the nano-chromics display (NCD), and the like.

Another example of a technique that may be used to implement changes in brightness, contrast, spatial range, resolution, etc. may be a suspended particle device (SPD). When a small voltage is applied to the SPD film, its microscopic particles, which are randomly dispersed in a steady state, are aligned and allow light to pass through. The response can be immediate and uniform, and has a stable color throughout the film. Adjustment of the voltage can allow the user to control the amount of light, glare and heat that passes through. The response of the system can range from a deep blue appearance to the complete block of light in its off state and to its transparency in its on state. In embodiments, the SPD technique may be that an emulsion is applied on a plastic substrate to produce an active film. This plastic film may be laminated (as a single pane of glass), floating between two sheets of glass, plastic or other transparent material, or the like.

8A-8C, in certain embodiments, the electro-optic system may be mounted in a monocular or binocular flip-up / flip-down configuration with two parts as follows : 1) Electro-optic system; And 2) a corrective lens. Figure 8a illustrates a two part eyepiece included within a module 802 that may be electrically connected to an eyepiece 804 through an electrical connector 810 (plug, pin, socket, wire, etc.) eyepiece. In this configuration, the lens 818 in the frame 814 may be entirely a corrective lens. The interpupillary distance (IPD) between the two halves of the electro-optic module 802 may be adjusted in the nose pads 808 to accommodate various IPDs. Similarly, the arrangement of the display 812 can be adjusted via the nose pads 808. [ 8B shows a binocular electro-optical module 802 in which one half is flipped up and the other half is flipped down. The nose pads can be fully adjustable and made of elastomeric material. This enables three-point mounting on the nose pads and ears with head straps to ensure the stability of the image from the user's eyes, unlike the instability of the helmet-mounted optics moving on the scalp. 8C, the lens 818 may be an ANSI compatible hardcoat scratch-resistant polycarbonate ballistic lens, may be a color changing lens, may have an angular sensitive coating, or may include a UV sensitive material Or other. In this configuration, the electro-optic module may include a CMOS-based VIS / NIR / SWIR black silicon sensor for night vision functions. The electro-optic module 802 may feature user flexibility, field replacement, and quick disconnect functionality for upgrades. The electro-optic module 802 may be characterized by an integrated power dock.

As shown in FIG. 79, the flip-up / flip-down lens 7910 may include a light block 7908. A detachable elastomeric night adapter / light dam / optical block 7908 can be used to shield the flip-up / flip-down lens 7910 for night-time operation, The exploded top view of the eyepiece also shows a head strap 7900, a frame 7904, and an adjustable nose pad 7902. 80 shows an exploded view of the electro-optical assembly in front view (A) and side angle view (B). The holder 8012 has a see-through optic with a correcting lens 7910. [ An O-ring 8020 and a screw 8022 fix the holder to the shaft 8024. The spring 8028 provides a spring-mounted connection between the holder 8012 and the shaft 8024. The shaft 8024 is connected to an attachment bracket 8014 that is secured to the eyepiece using a thumbscrew 8018. [ Shaft 8024 functions as an IPD adjustment tool using a pivot and IPD adjustment knob 8030. As shown in FIG. 81, the knob 8030 rotates along an adjustment thread 8134. Shaft 8024 also features two set screw grooves 8132.

In embodiments, a photochromic layer may be included as part of the optical system of the eyepiece. Photochromism is a reversible transformation of chemical species between two forms by the absorption of electromagnetic radiation, where the two forms are the same as the reversible changes such as color, degree of darkness upon exposure to light of a given frequency And have different absorption spectra. In one example, a photochromic layer may be included between the waveguide of the eyepiece and the correcting optical system on the outside of the correcting optical system or the like. In embodiments, a photochromic layer (such as that used as a darkening layer) may be activated with a UV diode or other photochromic responsive wavelength known in the art. When the photochromic layer is activated with UV light, the eyepiece optics may also include a UV coating outside the photochromic layer to prevent UV light from the sun unintentionally activating it.

Photochromism now changes rapidly from light to dark, but slowly from dark to light. This is due to the molecular changes involved in changing the photochromic material from transparent to dark. The photochromic molecules are again vibrated after UV light (such as UV light from the sun) is removed and becomes transparent. If the vibration of the molecule is increased by exposure to heat, the optical system will become more transparent. The rate at which the photochromic layer travels from dark to bright may be temperature dependent. For military applications where sunglasses users often go from a bright external environment to a darker interior environment, it is particularly important that they change rapidly from dark to bright, and it is important that they can be seen quickly in an internal environment.

The present disclosure provides a photochromic film device having an attached heater that is used to accelerate the transition from dark to clear in a photochromic material. This method relies on the relationship between the transition speed of the photochromic material from dark to clear, where the transition is faster at higher temperatures. To allow the heater to rapidly increase the temperature of the photochromic material, the photochromic material is provided as a thin layer with a thin heater. By keeping the thermal mass of the photochromic film device low per unit area, the heater only needs to provide a small amount of heat in order to generate a large temperature change of the photochromic material. Since the photochromic material only has to be at a higher temperature during the transition from dark to clear, the heater needs only to be used for a short period of time, and therefore the power requirements are low.

The heater may be a thin and transparent heater element, such as an ITO heater or any other transparent and electrically conductive film material. When the user needs to make the eyepiece fast transparent, the user can activate the heater element by any of the control techniques discussed herein.

In one embodiment, a heater element may be used to calibrate the photochromic element to compensate for cold ambient conditions when the lens may be self-dimming.

In another embodiment, a thin coat of photochromic material may be deposited on a thick substrate, over which a heater element may be layered. For example, a sunglass cover lens may include an accelerated electrochromic solution and a separate electrochromic patch that can be selectively controlled with or without UV light is still on the display area Lt; / RTI >

94A shows a photochromic film device having a serpentine heater pattern, and FIG. 94B shows a side view of the photochromic film device, wherein the photochromic film device is a lens for sunglasses. The photochromic film device is shown as not contacting the protective cover lens at the top in order to reduce the thermal mass of the device.

U.S. Patent No. 3,152,215 describes a heater layer coupled with a photochromic layer for heating a photochromic material to reduce the transit time from dark to clear. However, the photochromic layer is arranged in a wedge shape, which greatly increases the thermal mass of the device thereby reducing the rate at which the heater can change the temperature of the photochromic material, or, alternatively, the temperature of the photochromic material Will greatly increase the power required to change the current.

The present disclosure includes the use of a thin carrier layer onto which a photochromic material is applied. The carrier layer may be glass or plastic. As is known in the art, the photochromic material can be applied by vacuum coating, by dipping, or by thermal diffusion into the carrier layer. The thickness of the carrier layer may be 150 micrometers or less. The choice of the thickness of the carrier layer is selected based on the desired darkness of the photochromic film device in the dark state and the desired transition speed between the dark state and the transparent state. Thicker carrier layers may be darker in a dark state, but may be slower to heat to higher temperatures due to having more thermal mass. Alternatively, a thinner carrier layer may be less dark in a dark state, while it may be faster to heat to higher temperatures due to having a lower thermal mass.

The protective layer shown in Fig. 94 is separated from the photochromic film device in order to keep the thermal mass of the photochromic film device low. In this way, the protective layer can be made thicker to provide higher impact strength. The protective layer may be glass or plastic (e.g., the protective layer may be polycarbonate).

The heater may be a transparent conductor patterned with a relatively uniform conductive path so that the heat generated over the length of the patterned heater is relatively uniform. One example of a transparent conductor that can be patterned is titanium dioxide. A larger area is provided at the end of the heater pattern for the electrical contact as shown in Fig.

As discussed in the discussion of Figures 8A-8C, the augmented reality glasses may include a lens 818 for each eye of the wearer. The lens 818 can be made to fit easily into the frame 814, so that each lens can be tailored to the person to wear the glasses. As such, the lens may be a corrective lens, may also be colored for use as sunglasses, or have other characteristics suitable for the intended environment. Thus, the lens may be colored yellow, dark or other suitable color, or may be photochromic, thus reducing transparency of the lens when exposed to brighter light. In one embodiment, the lens may also be designed to snap fit within the frame or on the frame (i.e., a snap on lens is one embodiment). For example, the lens may be made of high quality Schott optical glass and may include a polarizing filter.

Of course, the lens need not be a corrective lens, but can simply serve as a sunglass or as a shield to the optical system in the frame. It is important to note that in non-flip-up / flip-down configurations, the outer lens is important to help protect the much more expensive waveguides, observation systems and electronics in augmented reality glasses There is no. At the very least, the outer lens provides protection from scratches by the user's environment (sand, spiny, visible, and other environments, flying debris, bullets and shotguns) in one environment. In addition, the outer lens may be decorative for functioning to change the appearance of the compound lens to attract attention, perhaps in accordance with the user's personality or fashion sense. The outer lens can also help one individual user to distinguish his glasses from others, for example, when many users are gathered together.

It is preferable that the lens is suitable for impact such as ballistic impact. Accordingly, in one embodiment, the lens and frame meet the ANSI standard Z87.1-2010 for ballistic resistance. In one embodiment, the lens also meets the ballistic standard CE EN166B. In another embodiment, for military applications, the lens and frame may meet standard 3.5.1.1 or standard 4.4.1.1, which are standards of MIL-PRF-31013. Each of these standards has slightly different requirements for ballistic performance, and each is designed to protect the user's eyes from impacts due to high-speed bullets or debris. Certain substances are not specified, but polycarbonates such as certain Lexan® grades are usually sufficient to pass the tests specified in the standard.

In one embodiment, as shown in Fig. 8D, the lens is snap in from the outside of the frame, not from the inside of the frame, for better impact resistance, This is because a shock is expected. In this embodiment, the interchangeable lens 819 has a plurality of snap fit arms 819a that fit into the depression 820a of the frame 820. In this embodiment, The engagement angle 819b of the arm is greater than 90 degrees, while the engaging angle 820b of the depression is also greater than 90 degrees. Making the angle larger than the right angle has an actual effect that makes it possible to separate the lens 819 from the frame 820. [ When a person's vision changes or another lens is desired for some reason, the lens 819 needs to be separated. The design of the snap foot has a slight compression or bearing load between the lens and the frame. That is, the lens can be firmly held within the frame by, for example, slightly fitting an interference fit within the frame.

The cantilever snap fit of Figure 8d is not the only possible way to detachably snap fit the lens and frame. For example, an annular snap fit can be used, in which case a continuous sealing lip of the frame engages the enlarged edge of the lens, which then snaps into the lip or possibly onto the lip. These snap pits are typically used to bond a cap to an ink pen. This configuration can have the advantage of a more robust joint where little dust and contaminant particles are likely to enter. A possible disadvantage is that a fairly stringent tolerance is required throughout the perimeter of both the lens and the frame, and requires dimensional integrity in all three dimensions over time.

It is also possible to use a much simpler interface that can still be considered a snap fit. The grooves can be molded in the outer surface of the frame, and the lens has a protruding surface that can be regarded as a tongue that fits into the groove. If the groove is semi-cylindrical, such as from about 270 ° to about 300 °, the tongue will snap into the groove and be firmly retained and still be removable through the gap remaining in the groove. 8E, a lens or a replacement lens or cover 826 with a tongue 828 can be inserted into the groove 827 in the frame 825, It does not. Because the pits are similar, the pit will function as a snap fit and keep the lens firmly in the frame.

In another embodiment, the frame may be made of two piece portions (lower portion and upper portion, etc.) having a conventional tongue-and-groove fit. In another embodiment, the design may also use a standard fastener to ensure that the frame holds the lens firmly. This design should not require dismantling anything on the inside of the frame. As such, the snap-on or other lens or cover must be assembled on or separated from the frame without the need to enter the interior of the frame. As seen in other parts of the present disclosure, augmented reality glasses have many components. Some of the assemblies and subassemblies may require careful alignment. Moving and impacting these assemblies can be detrimental to its function, such as moving and impacting frames and outer or snap-on lenses or covers.

In embodiments, the flip-up / flip-down arrangement enables a modular design for the eyepiece. For example, not only can the eyepiece be equipped with a monocular or binocular module 802, but the lens 818 can be replaced. In embodiments, additional features associated with one or both displays 812 may be included with the module 802. 8F, the monocular version or the binocular version of the module 802 may be a display-only [852 (monocular), 854 (binocular)] or a forward-looking camera 858 (monocular) and 860 862 (both eyes). In some embodiments, the module may have additional integrated electronic devices such as GPS, a laser range finder, and the like. In an embodiment 862 that allows for 'urban-tactical response (awareness & visualization)', the binocular electro-optic module 862 includes a stereoscopic front camera 870, a GPS and a laser distance meter (Not shown). These features may allow the ultravision embodiment to have a panoramic night vision, and a laser range finder and a panoramic night vision with geographic location.

In one embodiment, the electro-optic properties may be, but are not limited to, the following:

In one embodiment, the projector characteristics may be as follows:

In another embodiment, the augmented reality eyepiece may comprise a lens that is electrically controlled as part of a microprojector or as part of an optical system between a microprojector and a waveguide. Fig. 21 shows an embodiment having such a liquid lens (liquid lens) 2152. Fig.

The glasses may also include at least one camera or light sensor 2130 that can provide images or images for viewing by the user. Images can be formed for transmission to the waveguide 2108 on its side by the microprojector 2114 on each side of the glasses. In one embodiment, a variable focus lens 2152, which is an additional optical element, may also be provided. This lens can be electrically adjusted by the user so that the image seen at waveguide 2108 is focused on the user. In embodiments, the camera can be a multi-lens camera, such as an 'array camera', wherein the eyepiece processor combines multiple viewpoints of data from multiple lenses with a single high quality Images can be created. This technique is called computational imaging because the software is used to process the image. Computer image processing can provide image processing advantages such as making it possible to process a composite image as a function of individual lens images. For example, since each lens can provide its own image, the processor can provide image processing to produce an image with a particular focus, such as foveal imaging Where the focus from one of the lens images is clear, high resolution, etc., and the remaining images are defocused, low resolution, and so on. The processor can also select portions of the composite image to be stored in the memory, while the rest can be deleted when the memory storage device is limited and only a fraction of the composite image is so important to store. In embodiments, the use of an array camera may provide a function to change the focus of an image after the image is taken. In addition to the image processing advantages of array cameras, array cameras can provide a thinner mechanical profile than conventional single lens array cameras, and therefore easier to integrate into eyepieces.

The variable lens may comprise a so-called liquid lens provided by Varioptic, SA of Lyon, France, or by LensVector, Inc. of Mountain View, California, USA. Such a lens may include a central portion having two immiscible liquids. Typically in these lenses, the path of light passing through the lens (i.e., the focal length of the lens) is changed or focused by applying a potential between the electrodes immersed in the liquid. At least one of the liquids is affected by the resulting electric potential or magnetic field potential. As such, electrowetting can occur, as described in U.S. Patent Application No. 2010/0007807, assigned to LensVector, Inc. Other techniques are described in LensVector's patent application publications 2009/021331 and 2009/0316097. All three of these disclosures are incorporated herein by reference, such that each page and drawing is incorporated herein by reference.

Another patent document from Varioptic, S.A., describes other devices and techniques for variable focus lenses that can also operate through electro-wetting phenomena. These documents include U.S. Patent Nos. 7,245,440 and 7,894,440 and U.S. Patent Application Publication Nos. 2010/0177386 and 2010/0295987, each of which is also incorporated herein by reference in its entirety, , Incorporated herein by reference. In these documents, the two liquids usually have different refractive indices and different electrical conductivity (e.g., one liquid is conductive (such as an aqueous liquid) and the other liquid is an insulating (oily liquid ), Etc.). Applying a potential changes the thickness of the lens, changes the path of light passing through the lens, and thus changes the focal length of the lens.

An electrically adjustable lens can be controlled by the controls of the glasses. In one embodiment, focus adjustment is done by calling a menu from the controls and adjusting the focus of the lens. The lenses can be individually controlled or can be controlled at one time. This adjustment can be made by physically turning the control knob or by displaying it as a gesture or by voice command. In other embodiments, the augmented reality glasses may also include a range finder, and the focus of the electrically adjustable lens may be automatically controlled by directing a range finder, such as a laser range finder, to a target or object a desired distance away from the user .

As shown in the above-mentioned U.S. Patent No. 7,894,440, a variable lens can also be applied to the augmented reality glasses or the outer lens of the eyepiece. In one embodiment, the lens can simply replace the correcting lens. A variable lens with electrically adjustable control may be used instead of or in addition to a lens mounted on the image light source or mounted on the projector. The corrective lens insert provides a correcting optics for the outside world, which is the user's environment, regardless of whether the waveguide display is active or not.

It is important to stabilize the image presented to the wearer of the augmented reality eyewear or eyepiece (s) (i.e., the image seen in the waveguide). The presented view or image goes from one or two digital cameras mounted on an eyepiece to a digital circuit, the image is processed in a digital circuit and, if desired, is stored as digital data before appearing on the display of the glasses. Anyway, as discussed above, digital data is then used to form an image, such as by using an LCOS display and a series of RGB light emitting diodes. A light image is processed using a series of lenses, a polarizing beam splitter, an electro-actuated liquid calibration lens, and at least one transition lens from the projector to the waveguide.

The process of gathering and presenting images involves several mechanical and optical connections between the components of the augmented reality glasses. Therefore, it seems clear that some form of stabilization is needed. This may include the optical stabilization of the camera itself, which is the most direct cause, since the camera is mounted on a mobile platform, in which the glasses - themselves are movably mounted on the mobile user - to be. Accordingly, camera stabilization or correction may be required. In addition, at least some stabilization or correction must be used for the liquid variable lens. Ideally, the stabilization circuit at that point would be able to correct any aberrations and vibrations from the liquid lens as well as from many parts of the circuit upstream from the liquid lens, including the image light source. One advantage of this system is that many commercially available cameras are highly advanced and typically have at least one image stabilization feature or option. As such, there may be many embodiments of the present disclosure having the same or different methods of stabilizing each image or very fast video stream, as will be discussed below. The term optical stabilization is commonly used herein to mean physically stabilizing a camera, camera platform, or other physical object, while image stabilization refers to data manipulation and processing.

One image stabilization technique is performed on a digital image when a digital image is formed. This technique can use pixels outside the boundaries of a visible frame as a buffer for unwanted motion. As an alternative, this technique may use other relatively stable regions or bases in successive frames. This technique is applicable to video cameras that transition electronic images every frame of video in a manner sufficient to accommodate the movements. This technique does not rely on a sensor, but directly stabilizes the image by reducing motion distracting vibrations and other distractions from moving cameras. In some techniques, the speed of the image may be slowed to add a stabilization process to the rest of the digital process, and may require more time per image. These techniques can determine the direction of stabilization using a global motion vector calculated from the motion difference per frame.

Optical stabilization of an image uses gravity or an electrical drive mechanism to move or adjust the optical element or image sensor to correspond to ambient vibration. Another way to optically stabilize the displayed content is to provide gyroscopic correction or sensing of a platform (e.g., a user) having augmented reality glasses. As previously noted, the sensors that are available and used in augmented reality glasses or eyepieces include MEMS gyroscopic sensors. These sensors can be used as feedback to capture movements and movements in three dimensions in very small increments and to correct the image transmitted from the camera in real time. It is clear that at least a majority of unwanted and undesirable movements are caused by movement of the user and the camera itself. These larger movements may include a user ' s overall shift (e.g., walking or running, riding in the vehicle). Vibration in the components at the electrical and mechanical connections that form a path from the camera (input) to the image (output) at the waveguide - can also occur within the augmented reality glasses. For example, it may be more important to calibrate or consider these global movements, rather than independent and small movements at the connections of components downstream from the projector. In embodiments, gyroscopic stabilization may stabilize the image when the image experiences periodic motion. For this periodic motion, the gyroscope can determine the periodicity of the user's movement and send the information to the processor to correct for the placement of the content in the user's view. The gyroscope may use a rolling average of two or three or more cycles to determine the periodicity. Such as an accelerometer, a position sensor, a distance sensor, a distance sensor, a biological sensor, a geodetic sensor, an optical sensor, a video sensor, a camera, an infrared sensor, an optical sensor, a photocell sensor, Other sensors of the system can also be used to stabilize the image or to position the image correctly in the user's field of view. When the sensor detects a user's head or eye movement, the sensor provides an output to the processor, which can determine the direction, speed, amount, and rate of the user's head or eye movement. The processor may convert this information to an appropriate data structure for further processing by a processor (which may be the same processor) that controls the optical assembly. The data structure may be one or more vector quantities. For example, the direction of the vector may define the orientation of the motion, and the length of the vector may define the rate of motion. Using the processed sensor output, the display of the content is adjusted accordingly.

Motion detection can thus be used to correct the image being captured and processed after sensing motion, such as in optical stabilization, to detect motion and correct it, or in image stabilization. An apparatus for detecting movement and correcting an image or data is shown in FIG. 34A. In this device, one or more kinds of motion sensors may be used, including an accelerometer, an angular position sensor or a gyroscope (MEMS gyroscope, etc.). The data from the sensors is fed back to a suitable sensor interface such as an analog-to-digital converter (ADC) or other suitable interface such as a digital signal processor (DSP). The microprocessor then processes this information, as discussed above, and sends the image stabilized frame to the display driver and then to the perspective display or waveguide discussed above. In one embodiment, the display begins with an RGB display in a microprojector of an augmented reality eyepiece.

In another embodiment, a video sensor or augmented reality glasses, or other device having a video sensor, may be mounted on the vehicle. In this embodiment, the video stream may be communicated to a person in the vehicle via a communication function or an Internet function. One application could be local tourism or touring. Other embodiments may be exploration or scouting or even patrolling of the area. In these embodiments, rather than applying gyroscopic correction to the digital data representing the image or image, the gyroscope stabilization of the image sensor will be helpful. One embodiment of this technique is shown in Figure 34B. In this technique, a camera or an image sensor 3407 is mounted on the vehicle 3401. One or more motion sensors 3406, such as a gyroscope, are mounted within the camera assembly 3405. A stabilizing platform 3403 receives information from the motion sensor and stabilizes the camera assembly 3405, thus minimizing jitter and wobbles while the camera is operating. This is true optical stabilization. Alternatively, a motion sensor or gyroscope may be mounted on or within the stabilization platform itself. This technique will actually provide optical stabilization to stabilize the camera or image sensor, as opposed to digital stabilization, which later corrects the image by computer processing of the data taken by the camera.

In one technique, the key to optical stabilization is to apply stabilization or correction before the image sensor converts the image to digital information. In one technique, feedback from a sensor, such as a gyroscope or an angular velocity sensor, is encoded and transmitted to an actuator that moves the image sensor as the autofocus mechanism adjusts the focus of the lens. The image sensor is moved in such a way as to maintain the projection of the image onto the image plane - a function of the focal length of the lens used. Perhaps autoranging and focal length information from an interactive head-mounted eyepiece's range finder can be obtained through the lens itself. In another technique, an angular velocity sensor (sometimes referred to as a gyroscope sensor) may be used to detect horizontal and vertical motion, respectively. The detected motion can then be fed back to the electromagnet to move a floating lens of the camera. However, this optical stabilization technique will need to be applied to each lens considered and as a result it will be quite costly.

Stabilization of the liquid lens is discussed in U.S. Patent Application Publication No. 2010/0295987, assigned to Varioptic, S. A., Lyon, France. Theoretically, the control of the liquid lens is relatively simple, since there is only one variable to control - for example, by using the lens housing and cap as electrodes, the electrodes in the conductive and non- Level of voltage to be applied -. Applying a voltage causes a change or tilt to the liquid-liquid interface through the electrowetting effect. This change or tilt adjusts the focus or output of the lens. Most fundamentally, the control scheme with feedback will then apply the voltage and verify the effect of the applied voltage on the result (i.e., focus or astigmatism of the image). The voltage may be applied in various patterns (e.g., positive and negative voltages of the same magnitude but opposite, positive voltages of different magnitudes, negative voltages of different magnitudes, etc.). Such a lens is called an electrically variable optical lens or an electro-optic lens.

The voltage can be applied to the electrodes in a variety of patterns for a short period of time and a focus or astigmatism check can be made. This inspection can be done, for example, by an image sensor. In addition, a camera or a sensor on the lens in this case can detect the movement of the camera or the lens. The motion sensor may include an accelerometer, a gyroscope, an angular velocity sensor, or a piezoelectric sensor mounted on a portion of a liquid lens or a lens optic train in the immediate vicinity of the liquid lens. In one embodiment, a table, such as a calibration table, then consists of the applied voltage and the degree of calibration or voltage required for a given level of motion. More sophistication can also be added, for example, by using segmented electrodes at different parts of the liquid so that four rather than two voltages can be applied. Of course, when four electrodes are used, four voltages can be applied in more patterns than in only two electrodes. These patterns may include positive and negative voltages that are the same for opposite segments but opposite. One example is shown in Figure 34c. Four electrodes 3409 are mounted in a liquid lens housing (not shown). The two electrodes are mounted in or near the nonconductive liquid, and two are mounted in or near the conductive liquid. Each electrode is independent in terms of the possible voltages that can be applied.

A search table or calibration table may be constructed and placed in the memory of the augmented reality glasses. During use, an accelerometer or other motion sensor will sense movement of the glasses (i.e., the glasses or the camera on the lens itself). A motion sensor, such as an accelerometer, will, in particular, sense a movement of a small vibration type that interferes with the smooth transmission of the image to the waveguide. In one embodiment, the image stabilization techniques described herein can be applied to electrically controllable liquid lenses so that images from the projector are immediately assured. This will stabilize the output of the projector, at least partially correcting at least some movement of the user, as well as the vibration and movement of the augmented reality eyepiece. There may also be manual control to adjust the gain or other parameters of the correction. It should be noted that this technique can also be used to calibrate myopia or hyperopia of individual users in addition to the focus adjustment already discussed by image sensor control and discussed as part of an adjustable-focus projector It is.

Another variable focus element uses a tunable liquid crystal cell to focus the image. These are disclosed, for example, in U.S. Patent Application Publication Nos. 2009/0213321, 2009/0316097 and 2010/0007807, the entire contents of which are incorporated herein by reference. In this method, the liquid crystal material is contained in a transparent cell (preferably having a matching refractive index). The cell includes a transparent electrode made of ITO (indium tin oxide). Using one spiral electrode and a second spiral electrode or a planar electrode, a spatially non-uniform magnetic field is applied. Other shapes of electrodes may be used. The shape of the magnetic field determines the rotation of the molecules in the liquid crystal cell to achieve a refractive index and thus a change in focus of the lens. Adjustable by changing the refractive index of the liquid crystal The liquid crystal is thus electromagnetically operated to make the liquid crystal cell function as a lens.

In the first embodiment, adjustable liquid crystal cell 3420 is shown in Figure 34d. This cell includes an inner liquid crystal layer 3421 and a thin layer of orientation material 3423, such as polyimide. This material helps orient the liquid crystal in the desired direction. The transparent electrode 3425 is on each side of the orienting material. The electrode may be planar, or it may be helical, as shown on the right side of Figure 34d. The transparent glass substrate 3427 includes a substance in the cell. The electrodes are formed to impart a shape to the magnetic field. As we have seen, in one embodiment, a helical electrode on one side or both sides is used so that the two are not symmetrical. A second embodiment is shown in Figure 34E. Adjustable liquid crystal cell 3430 includes a central liquid crystal material 3431, a transparent glass substrate wall 3433, and a transparent electrode. The lower electrode 3435 is planar while the upper electrode 3437 is helical. The transparent electrode may be made of indium tin oxide (ITO).

Additional electrodes may be used for rapid return of the liquid crystal to a non-shaped or natural state. Thus, a small control voltage is used to dynamically change the refractive index of the material through which the light passes. The voltage generates a spatially nonuniform magnetic field of the desired shape, allowing the liquid crystal to function as a lens.

In one embodiment, the camera includes a black silicon short wave infrared (SWIR) CMOS sensor as described elsewhere in this patent. In another embodiment, the camera is a 5 megapixel (MP) optically-stabilized video sensor. In one embodiment, the control includes a 3 GHz microprocessor or microcontroller and also includes a 633 MHz digital signal processor with a 30 M polygon / second graphics accelerator for real time image processing of the image from the camera or video sensor can do. In one embodiment, the augmented reality glasses may include wireless Internet, wireless or communication capabilities for broadband, a personal area network (PAN), a local area network (LAN), a wide local area network (WLAN) And may include reach-back communications. The equipment provided in one embodiment includes a Bluetooth function compliant with IEEE 802.15. In one embodiment, the augmented reality glasses include an encryption system such as a 256 bit AES (Advanced Encryption System) encryption system or other suitable encryption program for secure communication.

In one embodiment, the wireless communication may include functionality for a 3G or 4G network, and may also include wireless Internet functionality. For extended life, the augmented reality eyepiece or glasses may also include at least one lithium ion battery and a charging function, as discussed above. The charging plug may include an AC / DC power converter and may use multiple input voltages, such as 120 or 240 VAC. In one embodiment, the control for adjusting the focus of the adjustable focus lens includes a 2D or 3D wireless air mouse or other non-contact control responsive to the user's gesture or movement. 2D mice are available from Logitech, Fremont, CA, USA. 3D mice are described herein, or others such as Cideko AVK05 available from Cideko of Taiwan may be used.

In one embodiment, the eyepiece may comprise an electronic device suitable for controlling the optical system and related systems, including a central processing unit, a non-volatile memory, a digital signal processor, a 3D graphics accelerator, and the like. The eyepiece is equipped with an inertial navigation system, camera, microphone, audio output, power supply, communication system, sensor, stopwatch or chronometer function, thermometer, vibratory temple motor, , A UV sensor that allows contrast and dimming using a photochromic material, and the like, as well as other electronic components or features.

In one embodiment, the central processing unit (CPU) of the eyepiece may be OMAP 4 with a dual 1 GHz processor core. The CPU may include a 633 MHz DSP that provides the capability for a CPU of 30 M polygons / second.

The system can also provide a dual micro-SD (secure digital) slot to provide additional removable non-volatile memory.

The onboard camera offers 1.3 MP color and can record up to 60 minutes of video footage. Video recorded to empty the footage can be transmitted wirelessly or using a mini-USB transmission device.

The communication system-on-a-chip (SOC) can operate as a wide local area network (WLAN), Bluetooth version 3.0, GPS receiver, FM radio, and the like.

The eyepiece can be operated with a 3.6 VDC Li-ion rechargeable battery for long battery life and ease of use. Additional power may be provided through the solar cell outside the frame of the system. These solar cells can supply power and can also charge lithium ion batteries.

The total power consumption of the eyepiece can be approximately 400 mW, but it is variable depending on the features and applications used. For example, processor-intensive applications with significant video graphics will require more power and will be closer to 400 mW. Less simpler, less video-intensive applications will use less power. The operating time by a single charge may also vary depending on the application and feature usage.

A micro-projector lighting engine (also referred to herein as a projector) may include a plurality of light emitting diodes (LEDs). To provide a life-like color, Osram red LED, Cree green LED, and Cree blue LED are used. These are die-based LEDs. The RGB engine provides an adjustable color output that allows the user to optimize the viewing of various programs and applications.

In embodiments, light may be added to the glasses or may be controlled through various means. For example, an LED illumination or other illumination may be embedded in the frame of the eyepiece, for example, around the nose ring, around the composite lens, or in the eyeglass leg.

The intensity of the illumination and the color of the illumination can be modulated. Through various control techniques described herein, modulation can be achieved through various application programs, filtering and magnification.

By way of example, the illumination can be modulated through various control techniques described herein, such as through control of the control knob, gestures, eye movements, or voice commands. When the user intends to increase the intensity of the illumination, the user can adjust the control knob or glasses, or adjust the control knob in the user interface displayed on the lens or by other means. The user can use the eye movement to control the knob displayed on the lens, or control the knob by other means. The user may adjust the illumination through the hand or other body movement so that the intensity or color of the illumination changes based on the movement being performed by the user. The user can also adjust the illumination through voice commands, such as by saying a syntax requesting an increase or decrease in illumination, or by requesting another color to be displayed. In addition, illumination modulation may be achieved through any control technique described herein or by other means.

In addition, the illumination can be modulated depending on the particular application being executed. As an example, an application may automatically adjust the intensity of illumination or the color of the illumination based on an optimal setting for the application. If the current illumination level is not at an optimal level for a running application, a message or command may be sent to provide illumination adjustment.

In embodiments, illumination modulation can be accomplished through filtering or through magnification. For example, filtering techniques may be used that allow the intensity and / or color of the light to be varied such that optimal or desired illumination is achieved. Also, in embodiments, the intensity of the illumination can be modulated by applying greater or less magnification to reach the desired illumination intensity.

The projector can be connected to the display to output video and other display elements to the user. The display used may be a SVGA 800 x 600 dot / inch SYNDIANT liquid crystal on silicon (LCoS) display.

The target MPE dimension for the system may be 24 mm x 12 mm x 6 mm.

The focus can be adjusted, allowing the user to fine-tune the projector output to suit its needs.

The optical system may be contained within a housing made of 6061-T6 aluminum and glass-filled ABS / PC.

In one embodiment, the weight of the system is estimated to be 3.75 ounces or 95 grams.

In one embodiment, the eyepiece and associated electronics provide a night vision function. This night vision function can be enabled by a black silicon SWIR sensor. Black silicon is a CMOS (complementary metal-oxide silicon) processing technique that improves the photo response of silicon more than 100 times. The spectral range extends deeper into the SWIR (short wave infra-red) wavelength range. In this technique, an absorption and anti-reflection layer with a depth of 300 nm is added to the glasses. This layer provides improved responsiveness, as shown in FIG. 11, where the responsiveness of black silicon is much greater than the responsiveness of silicon over the visible and NIR ranges and extends deeper into the SWIR range. This technology is an improvement over current technology, which suffers from very high cost, performance and high volume manufacturability problems. Incorporating this technology into night vision optics brings the economic advantages of CMOS technology to the design.

Unlike current night vision goggles (NVGs) that amplify starlight or other ambient light from the visible spectrum, the SWIR sensor captures individual photons and, similarly to digital photography, And converts it into an electric signal. Photons can be generated from natural recombination of oxygen atoms and hydrogen atoms in the atmosphere at night (also called "night glow"). Shortwave infrared devices look at objects at night by detecting reflected starlight, city lights, or invisible short-wave infrared radiation within the moon. This will work in daylight or in fog, mist or smoke, while current NVG Image Intensifier infrared sensors will be overwhelmed by heat or brightness. Since a shortwave infrared device captures invisible radiation at the edge of the visible spectrum, the SWIR image is an image generated by visible light with the same shading and contrast and facial detail (only black and white ), Dramatically improves cognition, and therefore does not look like a blob that people often look like in people and are often seen in thermal imagers. One of the important SWIR functions is to provide a sighting laser on the battlefield. The aiming laser (1.064um) is not visible as a current night vision goggle. With SWIR electro-optics, soldiers will be able to see all the aiming lasers in use, including those used by enemy forces. Unlike thermal imagers that can not penetrate the windows of a vehicle or a building, visible / near-infrared / short-wave infrared sensors can view them day or night and provide important strategic advantages to the user.

Certain advantages include the use of active illumination only when needed. In some cases, there may be enough natural lighting at night, such as when a full moon appears. In this case, an artificial night vision using active illumination may not be necessary. With a black silicon CMOS based SWIR sensor, active illumination may or may not be needed during these conditions, thus improving battery life.

In addition, the black silicon image sensor can have a signal-to-noise ratio greater than eight times that seen in expensive indium-gallium arsenide image sensors under the night sky conditions. Better resolution is also provided by this technique and provides much higher resolution than is available using current technology for night vision. Generally, long wavelength images generated by CMOS-based SWIRs are difficult to interpret and have good thermal detection but poor resolution. This problem is solved by a black silicon SWIR image sensor that relies on much shorter wavelengths. For this reason, SWIR is highly desirable for night vision glasses on the battlefield. 12 shows a) dust; b) fog, and c) smoke, both before and after the image. The images in FIG. 12 show the performance of a new VIS / NIR / SWIR black silicon sensor. In embodiments, the image sensor can distinguish changes in the natural environment, such as degraded vegetation, damaged ground, and the like. For example, an enemy soldier may have recently installed a demolition device on the ground, so the ground on the explosive will be the "damaged ground" and the image sensor (along with the treatment facilities inside or outside the eyepiece) The damaged ground can be distinguished from the surrounding ground. In this way, a soldier can detect the possibility of installing an underground explosive device (e.g., an improvised explosive device) from a distance.

In previous night vision systems, there was a "bloom" caused by a bright light source such as a streetlight. This "bloom" is particularly strong in image enhancement technology and is also associated with loss of resolution. In some cases, a video enhancement technology system requires a cooling system, which increases weight and shortens battery power life. 17 shows the difference in image quality between the flexible platform of an uncooled CMOS image sensor capable of A) VIS / NIR / SWIR imaging and B) image intensified night vision system.

13 shows the difference in structure between the current or conventional vision enhancement technique 1300 and the uncooled CMOS image sensor 1307. [ The conventional platform (FIG. 13A) is limited in installation due to cost, weight, power consumption, spectral range, and reliability issues. Conventional systems typically include a front lens 1301, a photocathode 1302, a micro channel plate 1303, a high voltage power supply 1304, a phosphorous screen 1305, And an eyepiece 1306, as shown in FIG. This is in contrast to the flexible platform of the uncooled CMOS image sensor 1307 (Fig. 13B), which is capable of VIS / NIR / SWIR imaging with low cost, power consumption and weight. These much simpler sensors include a front lens 1308 and an image sensor 1309 with a digital image output.

These advantages are derived from CMOS-compatible processing techniques that improve the light response of silicon by a factor of 10 or more and extend the spectrum range deep into the short-wave infrared range. The difference in responsiveness is illustrated in FIG. 13C. Although conventional night vision goggles are limited to about 1100 nm (1.1 micrometers) in the UV, visible and near infrared (NIR) range, newer CMOS image sensor ranges are also limited to 2000 nm (2 micrometers) It also includes a short-wave infrared (SWIR) spectrum.

Black silicon core technology can provide a significant improvement over current night vision glasses. Femtosecond laser doping can enhance the photodetecting properties of silicon over a broad spectrum. In addition, the optical response can be improved by 100 to 10,000 times. Black silicon technology is a very inexpensive, fast and scalable CMOS compatible technology compared to current night vision systems. Black silicon technology can also provide a low operating bias (typically 3.3 V). In addition, uncooled performance up to 50 ° C may be possible. Cooling requirements for current technology increase both weight and power consumption, and also cause inconvenience to the user. As we have seen, black silicon core technology provides a high-resolution alternative to current image enhancer technology. Black silicon core technology can provide high-speed electronic shuttering at speeds of up to 1000 frames / second with minimal crosstalk. In certain embodiments of night vision eyepieces, OLED displays may be preferable to other optical displays such as LCoS displays.

Eyepieces with VIS / NIR / SWIR black silicon sensors can provide better SAAS (situational awareness) monitoring and real-time image enhancement.

In some embodiments, the VIS / NIR / SWIR black silicon sensor may be included in a form factor suitable only for night vision, such as night vision goggles or night vision helmets. Night vision goggles may include features that make them suitable for military markets such as ruggedization and alternative power supplies, while other form factors may be suitable for consumer or toy markets . In one example, night vision goggles may have an extended range, such as 500 to 1200 nm, and may also be used as a camera.

In some embodiments, VIS / NIR / SWIR black silicon sensors as well as other outboard sensors may be included in the mounted camera, which may be mounted on a transport or combat vehicle, The real time feed can be transmitted to the driver of the vehicle or other passenger by superimposing the video on the forward view. The driver can better see where he or she is going, the gunner can better see the threat of the tenure or the target, and the navigator can detect threats and better detect situational awareness (SAAS). have. The feed may also be sent to an off-site location, such as memory / storage at the parent headquarters for later use in target selection, navigation, monitoring, data mining, etc., as desired.

An additional advantage of the eyepiece can include a stable connection. This connection enables downloading and transmission using Bluetooth, Wi-Fi / Internet, cellular, satellite, 3G, FM / AM, TV, and UVB transceivers to transmit / receive huge amounts of data quickly. For example, a very high data rate LPI / LPD (low-probability) data link to connect weapon sight, weapon-mounted mouse / controller, E / O sensor, medical sensor, -of-intercept / low-probability-of-detection A UWB transceiver may be used to create a Wireless Personal Area Network (WPAN). In other embodiments, WPANs may be generated using other communication protocols. For example, a WPAN transceiver may be a COTS-compliant module front end to increase responsiveness of the combat radio's power management and to prevent degradation of the stability of the transceiver. By integrating the UWB transceiver, baseband / MAC and encryption chip onto the module, a physically small, dynamic and configurable transceiver is achieved that addresses a number of operational requirements. A WPAN transceiver generates a low-power encrypted wireless personal area network (WPAN) between military wear devices. WPAN transceivers can be attached or embedded in almost any fielded military device with a network interface (handheld computer, combat display, etc.). The system can support many users, ideal for combat that provides low probabilities of interception and detection (LPI / LPD) as well as stable to AES encryption, jamming and RF interference. The WPAN transceiver eliminates the volume, weight and "snagability" of the solder's data cables. The interface includes USB 1.1, USB 2.0 OTG, Ethernet 10-, 100 Base-T and RS232 9-pin D-Sub. The power output can be -10, -20 dBm output for a variable range of up to 2 meters. The data capacity may be greater than 768 Mbps. The bandwidth can be 1.7 GHz. The encryption may be 128 bit, 192 bit or 256 bit AES. The WPAN transceiver may include optimized MAC (Message Authentication Code) generation. The WPAN transceiver can comply with MIL-STD-461F. The WPAN transceiver may be in the form of a connector dust cap and may be attached to any field military device. The WPAN transceiver enables simultaneous video, voice, still, text and chat, eliminates the need for data cables between electronic devices, and enables hands-free control of multiple devices without distracting attention. -free control, featuring an adjustable connectivity range, interfacing with Ethernet and USB 2.0, adjustable frequency 3.1 to 10.6 GHz and 200 mw peak draw and nominal standby nominal standby.

For example, a WPAN transceiver is capable of generating a WPAN between an eyepiece 100 in the form of a GSE stereoscopic heads-up battle display glasses, a computer, a remote computer controller, and a biometric registration device such as that shown in Figure 58 You can do it. In another example, the WPAN transceiver may include an eyepiece in the form of a flip-up / flip-down head up display battle glasses, a HUD CPU (if external), a weapon fore- grip controller, It can make it possible to create a WPAN between similar forearm computers.

The eyepiece can provide its own cellular connection by, for example, via a personal wireless connection with the cellular system. The personal wireless connection may be used only by the wearer of the eyepiece, or may be available to a plurality of nearby users, such as in a Wi-Fi hotspot (e.g., WiFi) where the eyepiece may be a local hotspot to provide. These proximity users may be users of other eyecare wearers or any other wireless computing device-such as a mobile communication facility (e.g., a cellular phone). Through this personal wireless connection, the wearer may not need another cellular or internet wireless connection to connect to the wireless service. For example, if there is no personal wireless connection integrated into the eyepiece, the wearer may have to find a WiFi connection point or tether for his mobile communication setup to establish a wireless connection. In embodiments, the eyepiece may replace the need to have a separate mobile communication device, such as a mobile phone, mobile computer, etc., by integrating these functions and a user interface into the eyepiece. For example, the eyepiece may have an integral WiFi connection or hotspot, a physical or virtual keyboard interface, a USB hub, a speaker or speaker input connection (e.g., to stream music), an integral camera, an external camera, In embodiments, the external device may provide a single unit with a personal network connection (e.g., WiFi, cellular connection), a keyboard, a control pad (e.g., a touchpad), etc., with respect to the eyepiece.

Communication from the eyepiece may include a communication link for a special purpose. For example, an ultra-wide bandwidth communication link may be used when transmitting and / or receiving large amounts of data in a short period of time. In other cases, an NFC having a very limited transmission range can be used for posting information to be transmitted to the person when the person is very close, such as for strategic reasons, for local direction, for warning, a near-field communications link may be used. For example, a soldier can safely post / possess information and send information only to those who are in close proximity to those who need to know or need to use it. In other cases, a wireless personal area network (PAN) may be used to connect to, for example, an inorganic sight, an on-board mouse / controller, an electro-optical sensor, a medical sensor, an audio-

The eyepiece may include a MEMS based inertial navigation system such as a GPS processor, an accelerometer (e.g., which enables head control of the system and other functions), a gyroscope, an altimeter, an inclinometer, a speedometer / odometer, a laser range finder, and a magnetometer , Which also enables image stabilization.

The eyepiece may include integral headphones such as an articulating earphone 120 that provides audio output to a user or wearer.

In one embodiment, a forward camera (see FIG. 21) integrated with the eyepiece can enable a basic augmented reality. In augmented reality, an observer can visualize what is being viewed and then layer on top of the enhanced view, enhanced, edited, tagged, or analyzed version. In the alternative, the related data may be displayed on a basic image or with a basic image. If two cameras are provided and are mounted at the correct co-spatial distance for the user, a stereoscopic video image can be generated. This function can be useful for people who need vision assistance. Many people have barriers to his vision, such as nearsightedness, primacy, A camera and a very close virtual screen such as that described herein provide these people with "video ", which video is adjustable closer or further in relation to focus, Is fully controlled by the. This function may also be useful for people suffering from eye diseases such as cataracts, pigmented retinitis, and the like. As long as any organic vision function is maintained, the augmented reality eyepiece can help people see more clearly. Embodiments of the eyepiece may feature one or more of magnification, increased brightness, and the ability to still map the content to the area of the healthy eye. Embodiments of eyepieces may be used as bifocal or magnifying glasses. The wearer may increase the zooming in the field of view or increase the zooming in the partial field of view. In one embodiment, the associated camera can create an image of the object and then present the zoomed-in photo to the user. The user interface can allow the wearer to point to the area to which the wearer wishes to zoom, for example, by the control techniques described herein, and thus, unlike simply zooming in on everything in the camera's field of view Can continue.

In a further embodiment, a rear-facing camera (not shown) may also be included in the eyepiece. In this embodiment, the rear camera can enable eye control of the eyepiece, and the user makes an application or feature selection by directing his or her eye to the particular item displayed on the eyepiece.

A further embodiment of an apparatus for capturing biometric data relating to an individual may include a micro-catenary telephoto bending optical camera in the apparatus. The Micro-Cassegrain telephoto bending optical camera can be mounted on handheld devices such as bio-print devices, biophones, and also used as part of a bio-kit to acquire biometric data. Lt; RTI ID = 0.0 > glasses. ≪ / RTI >

The Cassegrain reflector is the combination of a primary concave mirror and a secondary convex mirror. These reflectors are often used in optical telescopes and wireless antennas because they provide good light collecting (or sound collecting) functionality in shorter and smaller packages.

In a symmetric case grain, both mirrors are aligned about the optical axis, and the primary mirror usually has a hole in the center so that the light can reach the eyepiece or camera chip or a light detecting device (such as a CCD chip) I will. An alternative design, often used in radio telescopes, places the final focus in front of the primary reflector. An additional alternative design may be to tilt the mirror so as not to disturb the primary mirror or secondary mirror, and may not require a hole in the primary mirror or secondary mirror. The micro-Cassegrain telephoto bending optical camera can use any of the above variations, and the final choice is determined by the desired size of the optical device.

The classic Caspian grain configuration (3500) uses a parabolic reflector as the main mirror and a hyperbolic mirror as the submirror. Additional embodiments of the micro-Cassegrain telephoto bending optical camera may use hyperbolic main mirrors and / or spherical or ellipsoidal mirrors. To explain the operation, as shown in Fig. 35, a classical cassense grain with a parabolic main mirror and a hyperbolic mirror reflects light back down through a hole in the main mirror. Bending the optical path makes the design more compact and, at "micro" size, makes it suitable for use in the bioprint sensors and bio print kits described herein. In a bending optical system, the beam is bent to make the optical path much longer than the physical length of the system. One common example of a bending optical system is a prismatic binocular. In a camera lens, the sub-mirror can be mounted on an optically flat, optically transparent glass plate that closes the lens tube. This support eliminates the "star-shaped" diffraction effect caused by the straight-vaned support spider of the straight vane. This enables a sealed closed tube and protects the main mirror, but there is some loss of light collecting power.

The Cassegrain design also uses special properties of parabolic reflectors and hyperbolic reflectors. A concave parabolic reflector will reflect all incoming rays parallel to its symmetry axis to a single focal point. The convex hyperbolic reflector has two focal points and reflects all rays directed at one focal point to the other focal point. The mirrors in this type of lens are designed and arranged to share a single focus, and the second focus of the hyperbolic mirror is located at the same point as the image is observed (usually just outside the eyepiece). The parabolic mirror reflects the parallel rays coming into the lens to its focal point coinciding with the focus of the hyperbolic mirror. The hyperbolic mirror then reflects the rays to the other focal point, where the camera records the image.

36 shows a configuration of a microcassegrain telescoping folded optical camera. The camera may be mounted on augmented reality glasses, bio-phones or other bio-collection devices. Assembly 3600 has a number of telescoping segments that allow the camera to extend to a caustrine optical system that provides a longer optical path. A thread 3602 allows the camera to be mounted on a device (such as augmented reality glasses or other biological collection devices). Although the embodiment shown in Figure 36 uses screws, other mounting schemes such as a bayonet mount, knob, or press-fit may also be used. The first telescoping section 3604 also functions as an outer housing when the lens is in a fully retracted position. The camera may also include a motor that drives the extension and retraction of the camera. A second telephoto section 3606 may also be included. Other embodiments may include a variable number of telescopic sections depending on the length of the optical path required for the selected task or data to be collected. The third telephoto section 3608 includes a lens and a reflecting mirror. If the camera is designed according to the classic camera design, the reflective mirror may be the main reflector. The sub-mirror may be included in the first telephoto section 3604.

Additional embodiments can still use a microscopic mirror to form a camera while still providing a longer optical path through the use of folded optics. The same principles of Cassegrain design are used.

Lens 3610 provides an optical system for use in conjunction with a bending optical system of the Cassegrain design. The lens 3610 may be selected from various types and may vary depending on the application. The screws 2602 allow various cameras to be exchanged according to the requirements of the user.

Eye control of feature and option selections can be controlled and activated by object recognition software loaded on the system processor. Object recognition software enables augmented reality, combines recognition output with querying the database, combines recognition output with a calculation tool to determine dependencies / probabilities, and so on.

In a further embodiment including a 3D projector, a three dimensional view is also possible. Two stacked pico projectors (not shown) may be used to produce a three-dimensional image output.

Referring to FIG. 10, a plurality of digital CMOS sensors having redundant micro and DSP for each sensor array and projector may be used for real-time image enhancement 1002, real-time keystone correction 1004, Near-infrared, and short-wave infrared light to enable manual daytime and nighttime operation such as, for example, The eyepiece may include visible imaging for monitoring the visible scene (e.g., for coordinated imaging with biometrics, gesture control, 2D / 3D projected maps), IR / UV imaging for scene enhancement A digital CMOS image sensor such as those described herein for processing (e.g., viewing through smoke, smoke, in the dark), acoustic direction sensing (e.g., direction of fire or explosion, voice detection) A directional microphone (e.g., a microphone array) may be used. In embodiments, each of these sensor inputs may be fed to a digital signal processor (DSP) for processing, e.g., inside an eyepiece or interfacing with an external processing facility. The outputs of the DSP processing of each sensor input stream may then be algorithmically combined in such a way as to generate intelligence data. For example, the system may be useful for combining real-time face recognition, real-time voice detection, and analysis via a link to a database, and may include a remote area of interest (e.g., a known path or mountain path, Especially in the case of monitoring and correcting distortions and simultaneous GPS positioning for soldiers and service personnel. In one embodiment, the acoustic direction sensor input to the DSP can be processed to create one or more of a visual, audible, or vibration queue for the user of the eyepiece to indicate the direction of the sound. For example, if hearing protection is used to block the sound of a large explosion or fire to protect the soldier's hearing, or if the soldier can not tell where the explosion is from, If the ears are so loud that the soldier can not hear anything, the visual, auditory, or vibrational cue for the operator can be used to signal the direction of the original threat.

The augmented reality eyepiece or glasses can be powered by any stored energy system, such as battery power, solar power, line power, and the like. A solar energy collector may be disposed on the frame 102, on the belt clip, and the like. Battery charging can be done in an outlet charger, using a car charger, in a belt clip, in a glasses case, or in a guitar. In one embodiment, the eyepiece may be charged and may be equipped with a mini USB connector for charging. In another embodiment, the eyepiece is available from Powercast, LIGION, PA; And Fulton Int'l. Of Ada, Michigan, USA. Equipped with equipment for remote inductive recharging by one or more remote inductive power conversion techniques such as those provided by the supplier (also owned by another supplier, Splashpower, Inc., Cambridge, England) .

The augmented reality eyepiece also includes any interface needed to connect the camera and camera to the circuit. The output of the camera can be stored in memory and can also be displayed on a display available to the wearer of the glasses. A display driver may also be used to control the display. The augmented reality device also includes a power supply such as a battery as shown, a power management circuit, and a circuit for charging the power supply. As we have seen elsewhere, charging can be done via a hard connection (e. G., A mini USB connector) or by an inductor, a solar panel input, or the like.

A control system for an eyepiece or glasses may include a control algorithm to save power when the power source of the battery or the like exhibits low power. This reduction algorithm may involve powering off applications such as lighting, cameras, or energy consuming applications, such as sensors requiring high levels of energy, such as, for example, any sensor requiring a heater have. Other saving steps include slowing down the power used for the sensor or camera (e.g., lowering the sampling or frame rate, going to a slower sampling or frame rate when power is low); Or lowering the sensor or camera at a much lower level. Thus, there are at least three operating modes-normal mode, depending on the available power; Conserve power mode; And an emergency or shutdown mode.

The application of the present disclosure can be controlled through the wearer's movement and direct action (such as movement of the user's hand, finger, foot, head, eyes, etc.) , A GPS sensor, etc.) and / or through equipment worn by or worn by a wearer (e.g., a sensor control facility mounted on the body). In this way, the wearer can directly control the eyepiece through his / her body movement and / or motion without using a traditional handheld remote controller. For example, a wearer may have a sensing device (such as a position sensing device) mounted on one or both hands (e.g., on at least one finger, palm, on the back, etc.) Provides position data of the hand, and provides wireless communication of the position data to the eyepiece as command information. In embodiments, the sensing device of the present disclosure may include a gyroscope device (e.g., an electronic gyroscope, a MEMS gyroscope, a mechanical gyroscope, a quantum gyroscope, a ring laser gyroscope, a fiber optic gyroscope) An accelerometer, a MEMS accelerometer, a velocity sensor, a force sensor, a pressure sensor, an optical sensor, a proximity sensor, an FID, and the like. For example, the wearer can mount the position sensing device on his right index finger, and this device can sense the movement of the finger. In this example, the user can activate the eyepiece through some switching mechanism on the eyepiece or through a certain sequence of movements of the finger (e.g., fast-moving fingers, tapping a hard surface with fingers, etc.) have. Notably, tapping the hard surface through sensing by an accelerometer, force sensor, pressure sensor, or the like can be interpreted. The position sensing device can then transmit the motion of the finger (e.g., moving the finger in the air to move the cursor across the displayed or projected image, moving in quick motion to indicate selection, etc.) as command information have. In embodiments, the position sensing device may send sensed command information directly to the eyepiece for command processing, or the command processing circuit may be co-located with the position sensing device (in this example, the sensor of the position sensing device Which is mounted on the finger as part of an assembly comprising the < RTI ID = 0.0 > The command information may be accompanied by a visual indicator. For example, the cursor may change colors when interacting with different content. For example, a visual indication of command information may be displayed in the glasses to know where the finger is when the external device is being used to control the glasses.

In embodiments, the wearer may be mounting a plurality of position sensing devices on his body. For example, continuing with the previous example, the wearer may be mounting the position sensing devices at a plurality of points in the hand (e.g., the individual sensors are on different fingers or as a collection of devices, such as in a glove) . In this way, the entire sense command information from the collection of sensors at different locations of the hand can be used to provide more complex command information. For example, a wearer may use a sensor device glove to play the game, where the glove may be used in a ball, bat, racket, or the like to use the present disclosure in simulating and playing a simulated game It detects the grasp and movement of the user's hand. In embodiments, a plurality of position sensing devices may be mounted on different parts of the body, allowing the wearer to transmit complex movements of the body to the eyepiece for use by the application.

In embodiments, the sensing device may have a force sensor, a pressure sensor, etc. for detecting when the sensing device comes into contact with an object, and so on. For example, the sensing device may include a force sensor at the end of the wearer's finger. In this case, the wearer may tap, multiple tap, sequence tap, swipe, touch, etc. to generate a command for the eyepiece. The force sensor may also be used to indicate the degree of touch, grip, push, etc., and the predetermined or learned threshold value determines the different command information. In this manner, the instructions can be delivered as a series of sequential instructions that continually update the instruction information being used in the application via the eyepiece. In one example, the wearer may be performing a simulation such as a game application, a military application, a commercial application, etc. In this case, movement due to, for example, at least one of the plurality of sensing devices, Lt; / RTI > is fed to the eyepiece as a command that affects the simulation being displayed. For example, a sensing device may be included in a pen controller, where the pen controller may have a force sensor, a pressure sensor, an inertial measurement unit, and the like, May be used to control the cursor associated with the display of the computer mouse, to serve as a computer mouse, to provide control commands through physical movement and / or contact, and the like.

In embodiments, the sensing device may include an optical sensor or optical transmitter as a manner in which motion is interpreted as a command. For example, the sensing device may include a light sensor mounted in the wearer's hand, and the eyepiece housing may include an optical transmitter, so that when the user moves his or her hands past the optical transmitter on the eyepiece, Motion can be interpreted as a command. Motion detected through the optical sensor may include swiping to a guitar at a different rate, in repeated motion, in combination with dwelling and movement. In embodiments, the optical sensor and / or transmitter may be located on the eyepiece, or may be mounted on the wearer (e.g., in the hands, feet, gloves, clothing), or may be used in combination between the wearer and different areas of the eyepiece Or other.

In one embodiment, a number of sensors useful for monitoring a wearer or a person's condition in proximity to the wearer are mounted within the augmented reality glasses. With advances in electronics, the sensors have become much smaller. Signal transducing and signal processing techniques have also made significant advances in the direction of size reduction and digitization. Accordingly, it is possible to have not only the temperature sensor but also the entire sensor array in the AR glasses. These sensors, as we have seen, include temperature sensors and also pulse oximeters; Beat-to-beat heart variability; EKG or ECG; Respiration rate; Core body temperature; Heat flow from the body; Galvanic skin response (GSR); EMG; EEG; EOG; Blood pressure; Body fat; Hydration level; Activity level; Oxygen consumption; Glucose or blood glucose level; Body position; And a sensor for detecting UV radiation exposure or absorption. In addition, there may also be a retinal sensor and a blood oxygenation sensor (SpO ₂ sensor, etc.) among others. These sensors are commercially available from Vermed of Bellows Falls, Vermont, USA; VTI from Vantaa, Finland; And ServoFlow, Lexington, Mass., USA.

In some embodiments, it may be more useful for the sensors to be mounted on a person or equipment of a person rather than on the glasses themselves. For example, it may be useful to have an accelerometer, a motion sensor, and a vibration sensor mounted on a person, on a person's clothing, or on equipment the person wears. These sensors can maintain continuous or periodic contact with the controller of the AR glasses via a Bluetooth® wireless transmitter or other wireless device compliant with the IEEE 802.11 standard. For example, if a physician wishes to monitor a movement or an impact experienced by a patient during a foot race, the sensors may be mounted directly above the patient's skin rather than mounted on the glasses, Sensors can be more useful when mounted on a T-shirt. In these cases, a more accurate reading can be obtained by a sensor located on the person or on the clothing rather than on the glasses. These sensors do not need to be as small as sensors suitable for mounting in the glasses themselves, and, as you know, can be more useful.

The AR glasses or goggles also include environmental sensors or sensor arrays. These sensors are mounted on glasses and sample air or air in the vicinity of the wearer. These sensors or sensor arrays may be sensitive to the concentration of a particular substance or substance. For example, sensors and arrays can measure the concentration of carbon monoxide, nitrogen oxides ("NOx"), temperature, relative humidity, noise level, volatile organic chemical (VOC), ozone, particulate, hydrogen sulfide, It is available for measurement. Sellers and Manufacturers of Sensares in Kroll, France; Cairpol in Ales, France; Critical Environmental Technologies of Canada, Delta, BC; Chinese Shen? Apollo Electronics Co .; And AV Technology Ltd., Cheshire Stockport, UK. Many other sensors are known. If these sensors are mounted on a person or on a person's clothing or equipment, they can also be useful. These environmental sensors may include radiation sensors, chemical sensors, toxic gas sensors, and the like.

In one embodiment, the environmental sensor, the health monitoring sensor, or both are mounted on the frame of the augmented reality glasses. In another embodiment, the sensors may be mounted on a person or on a person's clothing or equipment. For example, a sensor that measures the electrical activity of a wearer's heart may be implanted with an accessory suitable for converting and transmitting a signal indicative of a person's cardiac activity.

This signal can be transmitted over a very short distance via an IEEE 802.15.1 compliant Bluetooth® wireless transmitter or other wireless device. Other frequencies or protocols may be used instead. This signal can then be processed by the signal monitoring and processing equipment of augmented reality glasses, recorded and displayed on a virtual screen available to the wearer. In another embodiment, the signal may also be transmitted to the wearer's friend or squad leader via the AR glasses. As such, human health and well-being can be monitored by humans and by others, and can be tracked over time.

In another embodiment, the environmental sensors may be mounted on a person or on a person's equipment. For example, a radiation or chemical sensor may be more useful when worn on a person's coat or web-belt rather than mounted directly on the glasses. As previously noted, the signals from the sensors can be monitored locally by the person through the AR glasses. The sensor readings may also be transmitted on request or automatically, perhaps at set intervals (e.g., every 25 minutes or every 30 minutes). Thus, for tracking or trending, a history of sensor readings can be made, whether for human body readings or environmental readings.

In one embodiment, a RF / MIR (micropower impulse radio) sensor may be associated with the eyepiece and may serve as a short-range medical radar. The sensor can operate in an ultra-wide band. The sensor may include an RF / impulse generator, a receiver, and a signal processor and may be useful for detecting and measuring heart signals by measuring ion flow in cardiac cells within 3 mm from the skin. The receiver may be a phased array antenna which allows to determine the position of the signal in a certain spatial region. Sensors can be used to detect and identify cardiac signals through obstructions such as walls, water, concrete, contaminants, metals, trees, and the like. For example, a user can use sensors to determine how many people are in a concrete structure by how many heartbeats are detected. In another embodiment, the detected heart rate can serve as a unique identifier for a person so that the person can be recognized in the future. In one embodiment, the RF / impulse generator may be embedded in one device, such as an eyepiece or some other device, whereas a different device, such as another eyepiece or device, has a built-in receiver. In this way, a virtual "tripwire" can be generated when the heart rate is detected between the transmitter and the receiver. In one embodiment, the sensor may be used as an in-field diagnostic or self-diagnosis tool. The EKG can be analyzed and stored as a biometric identifier for future use. The user may be alerted of the sensed heart rate signal and how many heart beats are present as the displayed content in the eyepiece.

29 shows an embodiment 2900 of an augmented reality eyepiece or glasses with various sensors and communication equipment. One or more environmental or health sensors are connected to the sensor interface locally or remotely via short-range wireless circuits and antennas, as shown. The sensor interface circuit includes both devices that detect, amplify, process, and transmit or transmit signals detected by the sensor (s). The remote sensor may include, for example, an implanted heart rate monitor or other body sensor (not shown). Other sensors may include an accelerometer, an inclinometer, a temperature sensor, a sensor suitable for detecting one or more chemicals or gases, or any other health or environmental sensor discussed in this disclosure. The sensor interface is coupled to a microprocessor or microcontroller of the augmented reality device and information gathered from this point is stored in a memory such as random access memory (RAM) or permanent memory, read only memory (ROM) Lt; / RTI >

In one embodiment, the sensing device enables simultaneous field sensing through an eyepiece. Electric field (EF) detection is a proximity detection method that allows a computer to detect, evaluate, and work on objects in its vicinity. Physical contact with the skin, such as a handshake with another person, or any other physical contact with a conductive or non-conductive device or object, is sensed as a change in the electric field and allows data transfer to or from the eyepiece, . ≪ / RTI > For example, the video captured by the eyepiece may be stored in the eyepiece until the wearer of the eyepiece having the embedded field-sensitive transceiver touches the object and initiates data transmission from the eyepiece to the receiver. The transceiver distinguishes the transmission mode and the reception mode by detecting both the transmission data and the reception data, and transmits the control signal in response to the two modes to detect the data which enables the bidirectional transmission And a transmitter including a circuit. A handshake or other contact can cause a momentary private network between two people. Data may be transmitted between the eyepiece of the user and the data receiver or eyepiece of the second user. To improve the private network, additional security measures such as face or audio recognition, detection of eye contact, fingerprint detection, biometric input, iris or retina tracking, etc. may be used.

In embodiments, access to the function of the eyepiece, such as access to the displayed or projected content, access to the restricted projected content, the ability of the eyepiece itself (e.g., via a login to access the function of the eyepiece) , ≪ / RTI > enabling, in whole or in part, < RTI ID = 0.0 > and / or < / RTI > Authentication may be provided via recognition of the wearer's voice, iris, retina, fingerprint, or other biometric identifier. For example, the eyepiece or associated controller may have an IR, ultrasonic or capacitive tactile sensor that receives control input related to authentication or other eyepiece function. Capacitance sensors can detect fingerprints, activate applications, or otherwise control eyepiece functions. Each finger has a different fingerprint, and thus each finger can be used to control different eyepiece functions or to quickly activate different applications or to provide different levels of authentication. The capacitance does not work for gloves, but the ultrasonic sensor operates and can be used in the same way to provide biometric or control. An ultrasonic sensor useful in an eyepiece or related controller is Sonavation's SonicSlide, which operates by imaging the fingerprint in 256 gray shades and measuring the ridges and valleys of the fingerprint acoustically to distinguish the finest fingerprint detail Includes Sonavation's SonicTouch ™ technology, which is used in the sensor. The main imaging component of the SonicSlide ™ is a ceramic micro-electro mechanical system (MEMS) piezoelectric transducer array consisting of a ceramic composite material.

The authentication system provides a database of biometric inputs for a plurality of users so that access control may be provided for use of the eyepiece based on policies and associated access privileges for each of the users entered into the database can do. The eyepiece can provide an authentication process. For example, the authentication facility can detect when the user takes off the eyepiece and may require re-authentication when the user wears the eyepiece again. This further ensures that the eyecare is available only to authorized users and that the wearer only provides access to authorized rights. In one example, when wearing an eyepiece, the authentication facility can detect the presence of the user's eyes or head. In the first level of access, the user can only access low-sensitivity items until authentication is completed. During authentication, the authentication facility can identify the user and navigate his access privileges. Once these rights are verified, the authentication facility can provide the appropriate access to the user. In the case where an unauthorized user is detected, the eyepiece may maintain access to less sensitive items, further restrict access, completely deny access, or otherwise.

In one embodiment, a receiver may be associated with the object to enable control of the object through a touch by the wearer of the eyepiece, wherein the touch enables transmission or execution of a command signal on the object. For example, the receiver may be associated with a car door lock. When the wearer of the eyepiece touches the car, the car door can be unlocked. In another example, the receiver may be embedded in a medicine bottle. When the wearer of the eyepiece touches the vial, an alarm signal may be initiated. In another example, the receiver may be associated with a wall along a sidewalk. When the wearer of the eyepiece passes the wall or touches the wall, the advertisement can be launched on the eyepiece or on the video panel of the wall.

In one embodiment, when the wearer of the eyepiece initiates a physical contact, the exchange of WiFi information with the receiver may provide an indication that the wearer is connected to an online activity, such as a game, Verification can be provided. In this embodiment, the expression of a person in response to a touch can change color or indicate some other visual indication.

In embodiments, the eyepiece may include a tactile interface as in Fig. 14 for enabling haptic control of the eyepiece, e.g., by swiping, tapping, touching, clicking, clicking, . For example, the tactile interface 1402 may be mounted on a frame of the eyepiece 1400, for example, one arm, both arms, a nose piece, an upper portion of the frame, a lower portion of the frame, In embodiments, tactile interface 1402 may include controls and functions similar to a computer mouse having left and right buttons, a 2D position control pad as described herein, and the like. For example, the tactile interface may be mounted on the eyepiece near the user's eyepiece bridge and may function as a 'eyeglass leg mouse' controller for eyepiece-projected content for the user, selector and an enter button. In another example, the tactile interface may be one or more vibrating eyeglass motors capable of vibrating to warn or notify a user, such as a danger left, a danger right, a medical condition, have. The tactile interface may be mounted on a controller remote from the eyepiece, such as a worn controller, a hand-carried controller, and the like. If the controller has an accelerometer, the accelerometer can sense user tapping on the keyboard, in the user's hand (tapping with a hand with a controller or with a hand with a controller), etc. The wearer then places the tactile sense in a plurality of ways that will be interpreted as an instruction by the eyepiece (e.g., tapping the interface once or several times, touching the interface with the finger, pressing and holding two or more interfaces at once) You can touch the interface. In embodiments, the tactile interface may be attached to the wearer's body (e.g., a hand, an arm, a leg, a torso, a neck), a wearer's clothing, an attachment to a garment, a ring 1400, a bracelet, . For example, an interface may be attached to the body (e.g., behind the wrist), wherein touching a different portion of the interface may be performed using different command information (e.g., touching the front portion, the rear portion, Present, tapping, swiping, etc.). In embodiments, the user touch to the tactile interface may be interpreted via force, pressure, motion, and the like. For example, the tactile interface may include a resistive touch technology, a capacitive touch technology, a proportional pressure touch technology, and the like. In one example, the tactile interface may utilize a separate resistive touch technology, wherein the application needs to be simple, robust, low power, and so on. In another example, the tactile interface may utilize capacitive touch technology, where more functionality (e.g., movement, swipe, multi-point contact, etc.) is required through the interface. In another example, the tactile interface may utilize pressure-sensitive technology, for example, when a variable pressure command is needed. In embodiments, any of these or similar touch techniques may be used in any tactile interface such as those described herein.

In one embodiment, a handheld accessory can be used to control the virtual keyboard for input to the glasses. For example, if the handheld device has a touch screen, the user may interact with a touch screen configured to present an on-screen keyboard or allow the user to interact with a device that provides input to the glasses in cooperation with the virtual keyboard. Lt; / RTI > For example, a virtual keyboard may be presented to the glasses, but instead of selecting an item in the air, the user may configure the touch screen device to receive input corresponding to the virtual keyboard. The device may track the finger as the finger slides across the capacitive module, and the click of the device will provide a key strike sensation. The device may have one or more motion buttons that have a front touch surface and allow the user to click back or up to select the user without lifting his finger from the touch surface. Characters selected by the user can be highlighted. The user will still be able to swipe the text, lift the finger to end the word, insert a blank, tap twice to insert a period, and so on. Figure 159 shows a virtual keyboard 15902 presented in the user's field of view. On the keyboard, two keys 'D' and 'Enter' are highlighted. In the drawing, a touch screen accessory device 15904 is used to provide this input to a keyboard, which is then transmitted as input to the glasses. As such, a visual indicator is provided that indicates that an input or control command has been executed using a virtual interface or an actual touch screen on the external device.

In embodiments, the eyepiece may include a haptic communication interface that uses a magnetic field to transmit and / or receive commands, telemetry, information, etc., between the eyepiece and the external device or directly to / from the user. For example, a user may be applying a patterned magnetic material directly to any part of his body (e.g., skin, nails, body interior, etc.) where the patterned magnetic material is generated by a haptic communication interface (E.g., vibration, force, motion, etc.) to an oscillating magnetic field. The vibrating magnetic field can transmit information through modulation of the magnetic field (e.g., through the amplitude of the signal, the time-wise variance of the signal, the frequency of the signal, etc.). The information conveyed may include an indication associated with the eyepiece application for the purpose of providing haptic feedback from the eyepiece to the user, an indication of the incoming call, an indication of entertainment, an indication of communication, Lt; / RTI > Different commands can cause different stimulus effects on the patterned magnetic material for different commands or indicators. For example, different stimulus effects can be detected in different frequency and / or sequence patterns for incoming calls from different people in the user's contact list, different intensities for different alert levels, patterns of interest for entertainment, . ≪ / RTI >

The haptic communication interface may include a coil for transmitting and / or receiving an oscillating magnetic signal. The magnetic material may be a ferromagnetic material, a paramagnetic material, or the like and may be applied as power, ink, tattoo, decal, tape, rub-on, sprayed-on, In embodiments, the magnetic material may be demagnetized when the user is not using the eyepiece, and may be unmagnetized when the magnetic material is in a location free of magnetic fields from the eyepiece, have. The applied magnetic material may be applied in a spatial pattern that functions, for example, in response to a particular communication signal modulation, has a specific impedance, responds to a particular frequency, and so on. The applied magnetic material may be a visible image, an invisible image, a tattoo, a marking, a label, a symbol, and the like. The applied magnetic material may include a pattern that uses an incoming magnetic signal to generate a signal that is transmitted back to the eyepiece haptic communication interface along with an identifier for the user as a signal indicating proximity between the eyepiece and the magnetic material . For example, the identifier may be a user ID that is compared to an ID stored on the eyepiece to verify that the user is an authorized user of the eyepiece. In another example, the magnetic material may only generate a signal that is transmitted back to the eyepiece, if the magnetic material is near the eyepiece. For example, a user may apply magnetic material to the nail and the user may provide the command indicator to the eyepiece by bringing his finger close to the user's tactile interface.

In another example, the wearer may have an interface on the ring, on a hand piece, on a guitar as shown in FIG. 15, where the interface is a tactile interface with wireless command connection to the eyepiece , A position sensor device, and the like. In one embodiment, ring 1500 includes a computer mouse, such as a button 1504 (e.g., functioning as a one-button, multi-button, and similar mouse function), a 2D position control 1502, a scroll wheel, You can have controls to reflect. Button 1504 and 2D position control 1502 may be as shown in Figure 15 wherein the button is on the side facing the thumb and the 2D position control is on the top. As an alternative, the button and 2D position controls may be in different configurations (e.g., all on the thumb side, all on the upper surface, or any other combination). The 2D position control 1502 may include a 2D button position controller (e.g., a TrackPoint pointing device embedded in any laptop keyboard to control the position of the mouse), pointing stick, joystick, optical track pad An optical touch wheel, a touch screen, a touchpad, a trackpad, a scroll trackpad, a trackball, any other location or pointing controller, and the like. In embodiments, the control signal from the tactile interface (such as the ring tactile interface 1500) may be provided to the eyepiece by a wired or wireless interface, wherein the user may place the control input in his / her hand, thumb, Can be provided. In embodiments, the ring may be expanded to fit any finger or may be retracted to better fit. For example, the ring may have a customizable strap or a spring-mounted hinge. For example, a user can clearly express control with his / her thumb, wherein the ring is worn on the index finger of the user. In embodiments, the method or system may provide an interactive head-mounted eyepiece worn by a user, wherein the eyepiece comprises an optical assembly through which a user views the surrounding environment and displayed content, content for display to a user An integrated projector facility for projecting content to an optical assembly, and at least one control component operated by a user, and wherein the control command from the operation of the at least one control component is sent to the processor And a control device worn on the user's body, such as a user's hand, provided as a command instruction. The command indication may relate to the manipulation of the content for display to the user. The control device can be worn on the first finger of the user's hand and the at least one control component can be operated by the second finger of the user's hand. The first finger may be the index finger, the second finger may be the thumb, and the first and second fingers may be in the same hand of the user. The control device may have at least one control component mounted on the index finger side facing the thumb. At least one control component may be a button. At least one control component may be a 2D position controller. The control device may have at least one button operation control component mounted on the index finger side facing the thumb, and a 2D position controller operation control component mounted on the upper side facing the side of the index finger. The control component may be mounted on at least two fingers of the user's hand. The control device can be worn as a glove in the user's hand. The control device can be worn on the user's wrist. At least one control component can be worn on at least one finger of the hand, and the transmission facility can be worn separately in the hand. Transmission equipment can be worn on the wrist. Transmission equipment can be worn on the back of the hand. The control component may be at least one of a plurality of buttons. The at least one button may provide a function substantially similar to a conventional computer mouse button. Two of the plurality of buttons may function substantially similar to the main button of a conventional two-button computer mouse. The control component may be a scroll wheel. The control component may be a 2D position control component. The 2D position control component can be a button position controller, a pointing stick, a joystick, an optical trackpad, an optical touch wheel, a touch screen, a touch pad, a track pad, a scroll trackpad, a trackball, The 2D position control component can be controlled by the user's thumb. The control component may be a touch screen capable of implementing touch control including functions similar to buttons and 2D manipulation functions. The control component may be activated when the user places the pointing and control device on the projected processor content. The ring controller can be powered by an onboard battery, which can be disposable, rechargeable, solar or other.

In embodiments, the wearer may have an interface mounted on a ring 1500AA that includes a camera 1502AA, as shown in Fig. 15aa. In embodiments, the ring controller 1502AA may include a control interface type (e.g., button 1504, 2D position control 1502, 3D position control (e.g., using an accelerometer, gyro), etc.) as described herein . ≪ / RTI > The ring controller 1500AA may then be used to control functions within the eyepiece, such as controlling the operation of the display content projected to the wearer. In embodiments, the control interfaces 1502 and 1504 may provide control aspects (e.g., on / off, zooming, panning, focusing, recording still pictures, video recording, etc.) to the embedded camera 1502AA, Can be provided. Alternatively, the functions may be controlled via other control aspects of the eyepiece (e.g., through voice control, other tactile control interfaces, eye detection such as described herein), or the like. The camera can also enable automatic control functions such as autofocus, timer function, face detection and / or tracking, automatic zooming, and the like. For example, ring controller 1500AA with integrated camera 1502AA can be used to view wearer 1508AA during a videoconference enabled via an eyepiece, where wearer 1508AA attaches camera 1502AA to his A face controller may be provided to provide a view of the face to be transmitted to at least one other participant participating in the videoconference (e. G., Worn on the finger). Alternatively, the wearer may take off the ring controller 1500AA and lower it onto the surface 1510AA (e.g., on the table top surface) so that the camera 1502AA sees the wearer. The image of wearer 1512AA may then be displayed on display area 1518AA of the eyepiece, along with images 1514AA of other participants present in the videoconference call, and may be sent to other persons attending the videoconference . In embodiments, camera 1502AA may provide manual or automatic FOV 1504AA adjustment. For example, the wearer may place the ring controller 1500AA on the surface 1510AA for use in a videoconference call, and the FOV 1504AA may be manipulated manually to direct the FOV 1504AA of the camera towards the wearer's face (E.g., via button controls 1502, 1504, voice control, other tactile interfaces) or automatically (e.g., via face recognition). FOV 1504AA may change as the wearer moves (e.g., by tracking by face recognition). The FOV 1504AA may also be zoomed in / out to adjust for changes in the position of the wearer's face. In embodiments, a camera 1502AA may be used for a plurality of still and / or video applications where a view of the camera is provided to the wearer on the display area 1518AA of the eyepiece, A storage device may be used in the eyepiece to store video / video that may be transmitted, delivered, etc. to a user, a web-application, or the like. In embodiments, as shown in FIG. 32 and FIG. 33, a plurality of cameras, such as a wrist watch 3202 with a built-in camera 3200, may be used, such as a camera worn on an arm, a hand, May be included in different mobile devices. As with ring controller 1502AA, any of these mobile devices may include manual and / or automatic functions such as those described for ring controller 1502AA. In embodiments, the ring controller 1502AA may have additional sensors such as a fingerprint scanner, tactile feedback, and an LCD screen, accelerometer, Bluetooth, etc., embedded functions, control features, . For example, the ring controller can provide synchronized monitoring between an eyepiece and other control components such as those described herein.

In embodiments, the eyepiece may provide a system and method for presenting a wearer ' s image to a video conference participant through the use of an external mirror, wherein the wearer sees himself in the mirror, Lt; / RTI > The captured image can be used directly or the image can be inverted to compensate for the image reversal of the mirror. In one example, the wearer can enter a videoconference with a plurality of other people, where the wearer can view live video images of others via the eyepiece. By using an ordinary camera in an ordinary mirror and an eyepiece, a user can see himself in a mirror, capture an image by an integrated camera, and provide his or her own image to a videoconference. This image can also be used by the wearer as an image projected on the eyepiece, in addition to the images of other people participating in the video conference, for example.

In embodiments, a surface sensing component in a control device that detects motion across a surface may also be provided. The surface sensing component can be placed on the palm side of the user's hand. The surface may be a hard surface, a soft surface, a surface of the user's skin, a surface of the user's clothing, or the like. Providing control commands may be sent by wireless, via a wired connection, or by others. The control device may control the pointing function associated with the displayed processor content. The pointing function controls the cursor position; Selecting displayed content, selecting and moving displayed content; Control of zooming, panning, viewing, size, and position of the displayed content; Others. The control device can control the pointing function associated with the observed environment. The pointing function may be to position the cursor on the observed object in the environment. The position of the observed object can be determined by the processor along with the camera integrated with the eyepiece. The identification of the observed object can be determined by the processor with the camera integrated with the eyepiece. The control device can control the function of the eyepiece. This function may be associated with the displayed content. This function can be the mode control of the eyepiece. The control device can be folded for storage convenience when the user is not wearing it. In embodiments, a control device may be used for an external device, such as for controlling an external device with an eyepiece. Audio can be an entertainment device, an audio device, a portable electronic device, a navigation device, a weapon, a car control, and the like.

In embodiments, a body wear control device (e. G., Worn on a finger, hand held on a palm, or on an arm, leg, torso, etc.) may provide 3D position sensor information to the eyepiece. For example, the control device may function as an 'air mouse', wherein a 3D position sensor (eg, an accelerometer, gyro, etc.) may, for example, be a button click, a voice command, a visually detected gesture, When the user orders it, it provides the location information. The user can use this feature to search 2D or 3D images being projected to the user through an eyepiece projection system. In addition, the eyepiece can provide an external relay of the image for display or projection to others, as in the case of a presentation. The user can change the mode of the control device between 2D and 3D to accommodate different functions, application programs, user interfaces, and the like. In embodiments, multiple 3D control devices may be used for a particular application, such as a simulation application.

In embodiments, the system includes an interactive head-mounted eyepiece eyepiece worn by a user that includes an optical assembly through which a user views the surrounding environment and the displayed content, and the optical assembly includes a correction An integrated processor for processing content for display to a user, and an integrated image light source for introducing the content into an optical assembly; And a tactile control interface mounted on the eyepiece that receives control input from the user through at least one of the user touching the interface and the user proximate to the interface.

In embodiments, control of the eyepiece, and in particular control of the cursor associated with the displayed content to the user, is enabled by the worn device 1500, as in FIG. 15, to the virtual computer mouse 1500A, For example, by hand control, or by other means. For example, the worn device 1500 may transmit commands via a physical interface (e.g., button 1502, scroll wheel 1504), and the virtual computer mouse 1500A may send commands to the user's thumb, fist, And the like, and can detect the motion and the motion of the command. In computing, a physical mouse is a pointing device that serves to detect two-dimensional motion on its support surface. A physical mouse consists of an object that is traditionally held under the hand of one of the user's hands, with one or more buttons. This often features "wheels" that allow the user to perform various system-dependent operations, or other elements such as additional buttons or features that can add more control or dimension input. The movement of the mouse is converted into the movement of the cursor on the display, which allows fine control of the graphical user interface. In the case of an eyepiece, the user can use a physical mouse, a virtual mouse, or a combination of the two. In embodiments, the virtual mouse may include one or more sensors attached to a user's hand, such as, for example, a thumb 1504A, a finger 1504A, a palm 1508A, a wrist 1510A, The eyepiece receives a signal from the sensors and converts the received signal into a cursor motion on the eyepiece display for the user. In embodiments, a signal may be received via an external interface, such as a tactile interface 1402, via a receiver inside the eyepiece, at an auxiliary communication interface, via an associated physical mouse or a worn interface, . The virtual mouse may also include an actuator or other output type element attached to the user's hand for haptic feedback to the user via, for example, vibration, force, pressure, electrical impulse, temperature, . The sensors and actuators may be attached to the user's hand through wraps, rings, pads, gloves, and the like. Accordingly, the eyepiece virtual mouse can allow the user to convert the movement of the hand into movement of the cursor on the eyepiece display, where 'motion' includes slow motion, rapid motion, sudden motion, position, And allow the user to work in three dimensions, including some or all of the six degrees of freedom, without the need for a physical surface. It should be noted that since a 'virtual mouse' may be associated with multiple portions of a hand, a virtual mouse may be implemented as a plurality of 'virtual mouse' controllers, or as controllers distributed across multiple control members of a hand It is possible. In embodiments, the eyepiece may provide the user with a plurality of virtual mice, e.g., one for each hand of the user, one or more feet of the user's feet, and one for the guitar.

In embodiments, the eyepiece virtual mouse does not require a physical surface to manipulate, but rather includes a plurality of accelerometer types (e.g., a tuning fork, a piezoelectric, a shear mode, a strain mode, Such as one of a magnetic field, a magnetic field, a magnetic field, a magnetic field, a magnetic field, a capacitance, a heat, a resistance, an electromechanical, a resonance, To determine the translational and angular displacement of any portion of < RTI ID = 0.0 > For example, an accelerometer can produce an output signal that is proportional to the translational acceleration of the hand in three directions. The accelerometer pair may be configured to detect rotational acceleration of a hand or a portion of a hand. The translational velocity and displacement of the hand or portion of the hand may be determined by integrating the accelerometer output signal and the rotational speed and displacement of the hand may be determined by integrating the difference between the output signals of the pair of accelerometers. Alternatively, other sensors such as ultrasonic sensors, imagers, IR / RF, magnetometers, gyro magnetometers, etc., may be used. Because an accelerometer or other sensor may be mounted on various parts of the hand, the eyepiece can be used to move from simple movements usually associated with computer mouse movements to highly complex movements (such as the interpretation of complex hand movements in a simulation application) The motion of the plurality of hands can be detected. In embodiments, the user may only require a small translation or rotation operation to cause these actions to be translated into a motion associated with the user's intentional motion on the eyepiece projection for the user.

In embodiments, the virtual mouse may have physical switches associated therewith (hand, eyepiece, or on / off switch mounted on another part of the body, etc.) to control the device. The virtual mouse may also have on / off control through predefined movements or actions of the hand, and so on. For example, the motion of the virtual mouse may be enabled through fast and forth motion of the hand. In another example, the virtual mouse may be disabled through movement of a hand passing through the eyepiece, e.g., in front of the eyepiece. In embodiments, a virtual mouse for an eyepiece may include a plurality of motions that are typically associated with physical mouse control and thus are used to perform operations familiar to an untrained user (e.g., one-click, two-click, Click, left click, click and drag, combination click, roller wheel movement, etc.). In embodiments, the eyepiece may provide gesture recognition, such as in interpreting the hand gesture through a mathematical algorithm.

In embodiments, gesture control awareness can be provided through techniques that use a capacitive change resulting from a change in distance between a user's hand and a conductor element that is part of a control system of the eyepiece, You will not need any devices. In embodiments, the conductor may be mounted as part of an eyepiece, for example as an external interface mounted on an arm or other part of the frame, or on the user's body or clothing. For example, the conductor may be an antenna, where the control system behaves in a manner similar to a touch-less instrument, called a theremin. Although telemin uses the heterodyne principle to generate an audio signal, in the case of an eyepiece, the signal can be used to generate a control input signal. The control circuit may include a plurality of radio frequency oscillators, such as where one oscillator operates at a fixed frequency and the other oscillator is controlled by the user's hand, Change. In this technique, the user's hand functions as a grounded plate (a connection of the user's body to the ground) of the variable capacitor at an inductance-capacitance (L-C) circuit (which determines the frequency of the oscillator as part of the oscillator). In another example, the circuit may use a single oscillator, two pairs of heterodyne oscillators, and the like. In embodiments, there may be a plurality of different conductors used as control inputs. In embodiments, this type of control interface may be ideal for control inputs that vary over a range of ranges, such as volume control, zooming control, and the like. However, this type of control interface can also be used for more discrete control signals (e.g., on / off control) where a predetermined threshold determines the state change of the control input.

In embodiments, the eyepiece may interface with a physical remote control device such as a wireless trackpad mouse, a handheld remote control, a body mounted remote control, a remote control mounted on an eyepiece, or the like. The remote control device may be mounted on external equipment for personal use, games, professional use, military use, and the like. For example, a remote control may be mounted on a weapon for a soldier [e.g., a hand grip, a muzzle shroud, a front handle, etc.) The remote control may be removably mounted on the eyepiece.

In embodiments, the remote control for the eyepiece may be activated and / or controlled via a proximity sensor. The proximity sensor may be a sensor capable of detecting the presence of a nearby object without any physical contact. For example, a proximity sensor may emit an electromagnetic field or an electrostatic field or an electromagnetic radiation beam (e.g., infrared), and search for changes in the field or return signal . Objects that are sensed are often referred to as proximity sensor targets. Different proximity sensor targets may require different sensors. For example, a capacitive or photoelectric sensor may be suitable for plastic targets; Inductive proximity sensors require metal targets. Other examples of proximity sensor technologies include capacitive displacement sensors, eddy currents, magnetic, photovoltaic (reflective), laser, passive thermal infrared, passive optics, CCD, and reflection of ionizing radiation. In embodiments, the proximity sensor may be any of the control embodiments described herein, including a physical remote control, a virtual mouse, an interface mounted on an eyepiece, controls (e.g., game controllers, weapons) It can be integrated into one.

In embodiments, sensors may be used to control the eyepiece or to measure the user's body movement as an external input (e.g., an inertial measurement unit (IMU), a three-axis magnetometer, a three-axis gyro, a three-axis accelerometer, etc.) . For example, the sensor may be mounted on the user's hand (s), thereby making it possible to use the signal from the sensor for control of the eyepiece as described herein. In other cases, the sensor signal may be received and interpreted by the eyepiece to assess and / or utilize the user ' s body movement for purposes other than control. In one example, the user's respective legs and sensors mounted on each arm can provide a signal to the eyepiece that allows the eyepiece to measure the user's posture. The user's posture can then be used in turn to monitor the user's posture over time (e.g., to monitor changes in body behavior, improvement during physical therapy, changes due to head trauma, etc.). In the case of monitoring head trauma, the eyepiece first determines the baseline gait profile of the user, and then, after a lapse of time (e.g., after a physical event (e.g., a sports related crash, an explosion, The user can be monitored. In the case of an athlete or person in physical therapy, the eyepiece may be used periodically to measure the user's attitude and to keep the measurements in the database for analysis. To monitor the user's posture for indication of physical trauma, physical improvement, etc., a running posture time profile may be generated.

In embodiments, the control of the eyepiece, and in particular the control of the cursor associated with the displayed content to the user, is facilitated by the movement of the facial features of the user wearing the eyepiece, facial muscle strain, Clicking through the teeth, movement of the jaw, and the like. For example, as shown in Fig. 15B, the eyepiece may have a facial activity sensor as an extension from the guitar, from the eyepiece ear 1508B, from the eyepiece earphone assembly 1504B, and the facial activity sensor The force, vibration, etc. associated with the movement of the facial feature can be detected. The facial activity sensor may also be mounted separately (e.g., as part of a stand-alone earpiece) from the eyepiece assembly and the sensor output of the earpiece and facial activity sensor may be wireless or wired (e.g., Or other communication protocol) to the eyepiece. Facial activity sensors can also be attached to the ears, mouth, face, neck, guitar. The facial activity sensor may also be comprised of a plurality of sensors, for example to optimize the sensed movement of different faces or internal movements or movements. In embodiments, the facial activity sensor may detect motion and interpret it as a command, or a raw signal may be transmitted to the eyepiece for analysis. The command may be a command for control of the eyepiece functions, a cursor provided as part of the display of the content to the user, or a control associated with the pointer, and so on. For example, a user may crack one or two times to indicate a one-click or two-click, such as typically associated with a click of a computer mouse. In another example, the user may relax the facial muscles to indicate a command, such as a selection, associated with the projected image. In embodiments, the facial activity sensor may use noise reduction processing to minimize background motions of the face, head, etc., e.g., through adaptive signal processing techniques. For example, a voice activity sensor may also be used to reduce interference from the user, from other people in the vicinity, from the surrounding environment, and the like. In one example, the facial activity sensor may also improve communications by detecting vibrations in the user's cheek during speech, e.g., using multiple microphones to identify background noise and remove it through noise cancellation, volume increase, etc. And can eliminate noise.

In embodiments, the user of the eyepiece may obtain information about any environmental features, locations, objects, etc. that are viewed through the eyepiece by lifting his or her hands into the field of view of the eyepiece and pointing to the object or location. For example, a pointing finger of a user may point to an environmental feature, where the finger is not only in the view of the eyepiece, but also in the view of the built-in camera. The system can now correlate the position of the pointed finger with the position of the environmental feature that the camera sees. In addition, the eyepiece may have a position and orientation sensor, such as a GPS and a magnetometer, to allow the system to know the user's position and line of sight. From this, the present system can provide the position information to the user, for example, in order to overlay the position of the environment information on the 2D or 3D map in order to provide the position information to the user, For example, the location information of the environmental feature may be extrapolated to further correlate, correlate with, for example, the address, the name of the person at the address, the name of the company at that address, the coordinates of the location. Referring to Figure 15C, in one example, the user is viewing through an eyepiece 1502C and points to a home 1508C within his field of view with a hand 1504C, wherein the built-in camera 1510C includes a hand 1504C He has both houses in his field of view (1508C). In this example, the system may determine the location of the house 1508C and provide a 3D map superimposed on the location information 1514C and the user view of the environment. In embodiments, the information associated with the environmental feature may be provided by an external facility (e.g., communicated via a wireless communication connection, stored inside the eyepiece (e.g., downloaded to the eyepiece for the current location) , Other]. In embodiments, the information provided to the wearer of the eyepiece may include geographic information, point of interest information, social networking information (e.g., a person who is augmented around a person (e.g., "floating" around a person) (Such as those stored in the wearer's contact list), history information, consumer information, product information, retail information, safety information, advertising information, And may include any of a plurality of information related to a scene viewed by the wearer, such as commercial information, security information, game related information, humorous comments, news related information, and the like.

In embodiments, a user may view his or her view of a 3D projected image associated with an external environment, a 3D projected image retrieved and retrieved, a 3D projected image (e.g., downloaded for viewing) (view perspective). For example, referring back to Fig. 15C, the user can change the view perspective of the 3D displayed image 1512C by, for example, turning his head, wherein the user rotates his head, , The live external environment and the 3D displayed image are still present when playing guitar. In this way, the eyepiece can provide an augmented reality by overlaying information (e.g., overlaid 3D displayed map 1512C, location information 1514C, etc.) on the external environment that the user sees, The displayed map, information, and the like may change. In other cases, in the case of a 3D movie or a 3D converted movie, the observer's perspective may be changed to place the observer "in" the cinematic environment by any control of the viewing perspective, And the view is changed corresponding to the changed head position, where the user can walk into the image " into the image " when the user physically walks forward, and the user moves the gazing view of his eye The perspective can change when you do it, and so on. In addition, additional image information can be provided on the side of the user view or the like that can be accessed by turning the head.

In embodiments, a user of one eyepiece may synchronize the user view of the projected image or video with at least the view of the second user of the eyepiece or other video display device. For example, two separate eyepiece users may want to view the same 3D map, game projection, point of interest projection, video, etc., where the two observers not only see the same projected content, / RTI > In one example, two users may want to view a 3D map of an area together, and the image is synchronized so that one user can point to a location on a 3D map that other users can view and interact with. Two users can move to and from the 3D map, share the virtual-physical interaction between the two users and the 3D map, and so on. In addition, a group of eyewear wearers can interact with the projection as a group together. In this way, two or more users can have an integrated augmented reality experience through coordination-synchronization of their eyepieces. Position information (e.g., absolute position information, relative position information, translational and rotational position information from the position sensors (e.g., gyroscope, IMU, GPS, etc.) Etc.) can be provided for synchronization of two or more eyepieces. Communication between the eyepieces may be direct, through the Internet network, through the cell network, through the satellite network, or others. The processing of position information that contributes to synchronization may all be performed in a master processor within a single eyepiece of a group of eyepieces, at a remote server system, at other, or any combination thereof. In embodiments, a coordinated and synchronized view of the projected content between a plurality of eyepieces may provide an augmented reality experience from an individual to a plurality of individuals, wherein a plurality of individuals may benefit from group augmented reality experience . For example, a group of people going to a concert can synchronize their eyepieces with a feed from a concert maker so that visual effects or audio can be pushed to people with eyepieces by concert producers, performers, or other audience members. In one example, a player may have a master eyepiece and control the transmission of content to audience members. In one embodiment, the content may be a player view of the surrounding environment. The player may also be using a master eyepiece for applications (e.g., controlling an external lighting system, interacting with an augmented reality drum kit or sampling board, calling song lyrics, etc.).

In embodiments, the image or video displayed on the eyepiece may be displayed on a connected device having a communication link to the eyepiece or may be synchronized with an image or video captured by it or coming directly from a feed of the remote camera. This feed may be selected, or other operations may be initiated by a control signal received from a sensor input or one of the connected devices, metadata transmitted by one of the other connected devices, and so on. Other video display devices may be other eyepieces, desktop computers, laptop computers, smart phones, tablet computers, televisions, and the like. The eyepiece, device, and remote camera may be connected by a wide area network (WAN), a local area network (LAN), a metropolitan area network (MAN), a personal area network (PAN), and a cloud network communication link. The sensor input may be an audio sensor input, a video sensor input, or the like. Other operations that may be initiated by sensor input or receipt of a control signal include initiating operations such as tracking a target, sending a message, or initiating video synchronization, as described elsewhere in this disclosure. Or < / RTI > For example, when a face recognition application recognizes a person of interest in a video feed from a guard's eyepiece, the video captured by the guard's eyepiece at a remote checkpoint or screening location is supervisor ) On the eyepiece of the camera.

In embodiments, the eyepiece may utilize a sound projection technique to recognize the direction of the sound to the wearer of the eyepiece by a surround sound technique or the like. Recognition of the direction of the sound to the wearer may include reproduction of the sound from the direction of the origin in real time or as reproduction. This may include visual or auditory indicators to provide direction to the source of sound. An acoustical projection technique may be useful to a person with a hearing impairment or a hearing impairment, for example, due to a user experiencing hearing loss, a user wearing a headphone, a user wearing a hearing protection, etc. . In this example, the eyepiece can provide improved 3D audio reproduction. In one example, the wearer could be wearing headphones and shooting was done. In this example, the eyepiece can reproduce a 3D acoustic profile of the sound of the shot, thereby allowing the wearer to know where the sound is and respond to the shot. In another example, the wearer of the guitar in a noisy environment with headphones, hearing loss, etc. may not know what they are talking about and / or the direction of the person speaking, and provides a 3D sound enhancement from the eyepiece (E. G., The wearer is listening to another close individual via headphones and therefore has no directional information). In another example, the wearer may be in a noisy environment or in an environment where periodic loud noises may occur. In this example, the eyepiece can block the noisy sound to protect the hearing of the wearer, or the sound can be too loud to tell the wearer where the sound is coming from, and besides, the ears of the wearer are now too loudly clenched, You will not hear it. To help with this situation, the eyepiece can provide the wearer with visual, auditory, vibration cues, etc. to indicate the direction of the sound source. In embodiments, the eyepiece provides an "augmented " hearing using an earphone to generate a reproduction of the sound that replaces the wearer ' s ears but is not in the natural world to protect the user's ear from loud noises . This artificial sound, which the operator can not hear in nature, can then be used to provide directionality to wirelessly transmitted communications.

In embodiments, an example of a configuration for setting the directionality of a sound source may be a different point microphone in a different direction. For example, at least one microphone may be used for the wearer's voice, and at least one microphone may be used for the surroundings, at least one pointing down to the ground and possibly a plurality of different individual directions. In this example, the microphone pointing below can be deducted to separate 3D sound surround and other sounds that can be combined with enhancement hearing techniques such as those described herein.

In an example of a sound augmented system that is part of an eyepiece, a number of users with eyepieces, such as in a noisy environment where all users are 'in the ear', as implemented by artificial noise cancellation through the eyepiece earphone, have. One of the wearers can say what equipment is needed. Because of all the ambient noise and the hearing protection that the eyepiece produces, no one may hear requests for equipment. Here, the wearer who has made a verbal request will have a filtered microphone near his mouth and will be able to wirelessly transmit the request to others, where his eyepiece will be able to send the acoustic signal to the other user's eyepiece and to the correct You can relay to your ears on the side, and others will see left or right to know who made the request. The system could be further enhanced with a "virtual" surround sound system (SRS True Surround Technology, etc.) that uses two earphones to provide geographical location for all wearers and awareness of 3D space.

In embodiments, an auditory queue may also be computer generated so that the communicating user does not need to express his communication in words and can select him from the list of conventional commands, And the like, and so on. In one example, the wearer may be in a situation where he does not want a display in front of his eyes and wants to have an earphone in his ear. In this case, if you want to notify someone in the group to come up, you can just click the controller a certain number of times, or provide a visual hand gesturer with a camera, IMU, and so on. The system can select the 'Follow Me' command and send it to other users along with the location of the communicating user, in order to deceive other users as if the 3D system is actually heard from invisible. In embodiments, direction information may be determined and / or provided via location information from a user of the eyepiece.

The eyepiece may include a facility that provides a user with a sense of vibration through a vibrating actuator in a frame or arm of an eyepiece structure, such as a mechanical vibration motor, a piezoelectric vibration actuator, an ultrasonic vibration actuator, and the like. Vibration may be provided as an indicator for a user with a visual impairment (e.g., due to darkness, smoke, clouds, blindness), as part of a game, as part of a simulation, or as a guitar to inform the user of a message display. For example, to help create a 3D visual-acoustic-vibration virtual reality environment for games, simulations, etc., a vibrating actuator can be used separately or with a speaker in the side arm of the eyepiece. For example, when the application presents a bullet flying over the left side of the user's head, the vibrating actuator is moved to each side of the eyepiece so that the left vibrating actuator is set to vibrate in a manner simulating the feeling that the bullet actually flew over the user May be mounted on the arm. In addition, the speaker on the side arm can apply sound similar to the sound emitted when the bullet flies over the user's head in synchronism therewith. The vibration and / or the speaker may be mounted on the eyepiece in a manner that provides the user with a 3D vibration-audio experience to augment the visual experience provided through visually displayed content, such as in visually displayed content in 3D have. In this way, the user may be surrounded by a multi-sensory virtual 3D environment. In embodiments, the present disclosure may include an interactive head-mounted eyepiece worn by a user, wherein the eyepiece includes an optical assembly through which a user views the surrounding environment and the displayed content, And a processing facility configured to manage the function of the eyepiece, wherein the head-mounted eyepiece comprises a frame for viewing a surrounding environment through which the user views the surroundings, and a left- And a right side arm, and a vibrating actuator in each of the left and right side arms, wherein each vibrating actuator is responsive to vibration commands from the treatment facility independently. The vibration command may initiate a vibration in one of the vibration actuators in response to a virtual bullet, a virtual explosion, a message indication, a visual cue, an alert, etc., that are part of the displayed content. As part of the user performing a simulation, a gaming application, a utility application, etc., the displayed content can be provided. An application that invokes a vibration command may be running on an eyepiece locally, partly or wholly, through an external platform - in this case, the eyepiece has communication interconnections with the external platform, and so on. In addition, the eyepiece may comprise, for example, an integral speaker, such as that described herein, on each of the left and right side arms, wherein the vibration command is a sound command from a speaker on the same side arm To initiate vibration in one of the vibration actuators in time synchronization with the audible command.

In embodiments, the eyepiece may be used to acquire signal intelligence for devices and users that are close to the wearer of the eyepiece, such as in the use of existing WiFi, 3G, Bluetooth and other communication signals, RTI ID = 0.0 > (SIGINT). ≪ / RTI > These signals include other eyepieces for collecting information about other known friendly users; Other eyepieces acquired by an unauthorized person (e.g., through a signal generated when an unauthorized user attempts to use the eyepiece); Other communication devices (e.g., a radio, a cell phone, a pager, a walkie-talkie, etc.); Electronic signals emitted from devices that may not be used directly for communication, and the like. The information collected by the eyepiece may be orientation information, position information, motion information, number of communications and / or speed, and the like. In addition, information can be gathered through the cooperative operation of multiple eyepieces, such as in triangulation of signals to determine the position of the signal.

15D, in embodiments, the user of eyepiece 1502D may determine a field of view (FOV) 1508D of camera 1510D for a see-through view, such as for an augmented reality application, Finger points from his / her hand 1504D to define the hand / finger points. For example, in the illustrated example, the user is using the first finger and the thumb to adjust the FOV 1508D of the camera 1510D of the eyepiece 1502D. The user may select the FOV 1508D by, for example, a combination of fingers, fingers and thumbs, a combination of fingers and thumbs from both hands, use of the palm (s), wrist hand Other combinations can be used to adjust. The use of multiple hand / finger points may allow the user to change the FOV 1508 of the camera 1510D in much the same way as a user of the touch screen, where different points of the hand / The points of the FOV are set. However, in this case, no physical contact is made between the user's hand (s) and the eyepiece. Here, the camera may be instructed to associate portions of the user's hand (s) with the setting or changing of the FOV of the camera. The commands may include, but are not limited to, hand movements in the FOV of the camera, commands associated with the physical interface on the eyepiece, commands associated with motion detected near the eyepiece, commands received from the command interface on any portion of the user, ), ≪ / RTI > or any other type of instruction described herein. The eyepiece can recognize the finger / hand movement as an instruction, as in any repetitive motion. In embodiments, the user can also use this technique to adjust any portion of the projected image, where the eyepiece relates the image the camera sees to some aspect of the projected image (e.g., hand / Associating finger points with the user's projected image). For example, the user may be viewing the external environment and the projected image at the same time, and the user may use this technique to change the projected viewing area, area, enlargement, and so on. In embodiments, the user can zoom in or zoom out from the scene observed in the live environment, zoom in or out from the observed portion of the projected image, change the viewing area assigned to the projected image, Changing the perspective view of the FOV, etc. < / RTI >

In embodiments, the eyepiece may enable simultaneous FOV. For example, simultaneous wide, medium and narrow camera FOVs can be used, where the user can have different FOVs at the same time (i. E., Wide to show the entire field of view (maybe static) (Perhaps moving with the eye or with the cursor) to focus on.

In embodiments, the eyepiece may track the eye through the light reflected from the user's eye, thereby determining the movement of the user's eye or of the user's eye. This information can then be used to help correlate the user's line of sight to the projected image, the camera view, the external environment, etc., and can be used in control techniques such as those described herein. For example, the user can take a look at the location on the projected image (e.g., by an external remote control or by some detected eye movement (e.g., blinking)). 15E, the transmitted light 1508E, such as infrared light, is reflected 1510E from the eye 1504E and illuminated by an optical display 502 (e.g., with a camera or other optical sensor) ). ≪ / RTI > This information can then be analyzed to extract eye rotation from changes in reflections. In embodiments, the eye tracing facility uses corneal reflections and pupil centers as features to track over time; Using the reflection from the front of the cornea and from the back of the lens as a tracing feature; Imaging features from the interior of the eye, such as retinal vessels, and following these features as the eye rotates; Perform other. As an alternative, the eyepiece may use other techniques to track the movement of the eye, for example, by the elements surrounding the eye that are mounted on the contact lens in the eye. For example, a user may be provided with a special contact lens having a built-in optical component, such as a mirror, a magnetic field sensor or the like, for measuring the motion of the eye. In other cases, a steady electric potential field from the eye may be used as a dipole (e.g., a positive pole in the cornea and a negative pole in the retina) The potentials can be measured and monitored. In this case, an electrical signal can be derived on the skin around the eye, on the frame of the contact electrode, using the contact electrode disposed on the guitar. When the eye moves from the central position toward the periphery, the retina approaches one electrode while the cornea approaches the opposite electrode. Due to this orientation change of the dipole and thus the electric potential field, the measured signal changes. By analyzing these changes, eye movements can be tracked.

Another example of how the user ' s eye direction and associated control can be applied is to reduce the clutter in a narrow portion of the user ' s field of view around the gaze direction in which the highest visual input of the eye is present, (By the eyepiece) of the visual indicator in the user ' s peripheral view and an optional selection (by the user). Since the brain is limited in how much information it can process at a time and the brain is most concerned with the visual content close to the direction of gaze, the eyecam is a cue that provides the user with a visual indicator projected on the periphery of the field of view can do. As such, the brain may only have to deal with the detection of the indicator, not the information associated with the indicator, thus reducing the likelihood of over-provisioning the information to the user. The indicator may be an icon, a picture, a color, a symbol, a blinking object, etc. and may indicate an alarm, an e-mail arrival, an incoming call, a calendar schedule, an internal or external processing facility requiring user attention, If the visual indicator is in the periphery, the user can be aware of it without being distracted by it. The user may then decide to raise the content associated with the visual cue, optionally to view additional information (e.g., to look at the visual indicator and thereby open its content). For example, an icon representing an incoming email may indicate that an email is being received. The user may decide to ignore the icon and ignore it (e.g., the icon will disappear after a period of time if not activated by the stare or some other control facility). As an alternative, the user may be aware of the visual indicator and may "activate" it by striking the direction of the visual indicator. In the case of e-mail, when the eyepiece detects that the user's eye line matches the position of the icon, the eyepiece can open the e-mail and display its contents. In this manner, the user maintains control over what information is being watched, and as a result, minimizes distractions and maximizes content usage efficiency.

In embodiments, in connection with the particular optical configuration described herein, such as front illuminated LCoS, feedback between two or more displays may ensure that the display has the same brightness and contrast. In embodiments, a camera in each display may be used. For example, by selecting LEDs of similar quality, output and color (e.g., from similar bins), the current to the LEDs can be controlled and color balance can be achieved, and left and right PWM (pulse width modulation) And periodic calibration can be performed. In embodiments, calibration of the power spectrum can be achieved. When the display is weakened due to high external brightness, the user can know the calibration for each display. In embodiments, the same brightness, color saturation, color balance, hue, etc. may be generated between the two displays. This can prevent the user's brain from ignoring one display. In embodiments, a feedback system may be created from a display that allows the user or other person to adjust the brightness, etc., so that each display has a constant and / or consistent brightness, color saturation, balance, hue, In embodiments, a brightness sensor, which may be a color, RGB, white sensor, sensor for total light, etc., may be on each display. In embodiments, the sensor may be a power sensor that is passed to the LED or monitors or inspects the power consumed by the LED. The user or other person can adjust one or more displays by raising or lowering the power to the LEDs. This can be done during manufacture and / or can be done during the life of the eyepiece and / or periodically. In embodiments, there may be a dynamic range aspect. There may be power algorithms that can be fine-tuned on one display so that both displays have consistent brightness when the LED and / or power is weak. In embodiments, the user and / or manufacturer or eyepiece may adjust the LED to follow the same brightness curve when power is changed. There can be an RGB LED, and the LED curves can be matched between the two displays. Accordingly, brightness, color saturation, color balance, hue, etc. can be controlled over the dynamic range. In embodiments, these items can be measured and controlled during manufacture, during the dynamic range, during the lifetime of the glasses, and others. In embodiments, the same brightness, color saturation, color balance, hue, etc. may be generated between two displays or may be generated to be perceived by the user based on the difference between the eyes of the user. In various embodiments, adjustment of brightness, color saturation, color balance, hue, etc. may be performed by the user, manufacturer, and / or performed automatically by the eyepiece based on feedback, various program algorithms, In embodiments, the sensor feedback may cause automatic and / or manual adjustment in at least one of brightness, color saturation, color balance, hue, and the like.

In embodiments, the system may include an interactive head mounted eyepiece worn by a user, wherein the eyepiece includes an optical assembly through which a user views the surrounding environment and the displayed content, and an integrated assembly Wherein the optical assembly comprises two or more displays and wherein at least one of the brightness, color saturation, color balance, and hue of the two or more displays is balanced against each other within a predetermined range, At least one of brightness, color saturation, color balance and color tone is adjusted for at least one of the colors. In embodiments, the adjustment may include making at least one of the brightness, color saturation, color balance, hue, etc. of two or more displays within a predetermined range for each other. In embodiments, adjustment of at least one of brightness, color saturation, color balance, hue, etc. may be performed based on detection of power delivered to the integrated image light source. In embodiments, the adjustment may be based on a power algorithm such that at least one of brightness, color saturation, color balance, hue, etc. is consistent between two or more displays. In further embodiments, the adjustment may be based on a sensor of total optical sensor feedback. In embodiments, during manufacture, at least one of brightness, color saturation, color balance, hue, etc. may be adjusted in at least one of the others during the dynamic range of the output produced by the integrated image light source. In embodiments, the system may be configured to automatically check for at least one of the brightness, color saturation, color balance, hue, etc. of two or more displays periodically over the life of the eyepiece. In embodiments, the system may automatically check at least one of the brightness, color saturation, color balance, tint, etc. of two or more displays against each other and determine the brightness, color saturation, color balance, May be configured to selectively set at least one of the first and second threshold values to a predetermined value. In addition, one embodiment of the present system can automatically check at least one of the brightness, color saturation, color balance, hue, etc. of two or more displays against each other and determine the brightness, color saturation , Color balance, color tone, and the like to a predetermined value.

In embodiments, with respect to the specific optical configuration described herein, such as front illuminated LCoS, the contrast between two or more displays may be adjusted to be the same or equally recognized by the user. In embodiments, the contrast can be checked on each display and adjusted accordingly, adjusted during the manufacturing process to calibrate and adjust the display, and the contrast can be adjusted in the manufacturing process, over the dynamic range, It can be measured during guitar. In embodiments, the contrast of the system can be automatically corrected as compared to the outside world, as well as between the two displays. In embodiments, the user can compensate for differences between his or her eyes. The contrast can be adjusted as needed to compensate for the user ' s visual and / or sensory disturbances. In embodiments, the contrast ratio may be a function of how the optical system module is assembled. As described herein, reducing stray light can solve the technique of assembling to provide a high contrast ratio. In embodiments, various types of single pixel brightness detectors and / or multiple pixel color detectors may be inserted within the light train to sample some or all of the light that does not continue to the eye box of the display. In embodiments, depending on where the detectors are located in the optical path, the system can be used for assembly tolerance, LED and LCoS panel yield, binning tolerance, hot and cold panel compensation and cold panel compensation and / or to maintain individual user calibration. In embodiments, the brightness and contrast of the display can be managed through good manufacturing practice. In addition, during manufacture, quality analysis can be done to inspect the display, if necessary, calibrate, and compensate as needed. In addition, corrections can be made to the search table for compensation values as the components are worn, or as the system is heated and cooled during use, over the lifetime of the system. In various embodiments, adjustment of brightness, color saturation, color balance, hue, contrast, etc. may be performed by the user, manufacturer, and / or performed automatically by the eyepiece based on feedback, various program algorithms, . In embodiments, the sensor feedback may cause automatic and / or manual adjustment in at least one of brightness, color saturation, color balance, hue, contrast, and the like.

In embodiments, the system may include an interactive head mounted eyepiece worn by a user, wherein the eyepiece includes an optical assembly through which a user views the surrounding environment and the displayed content, and an integrated assembly Wherein the optical assembly includes two or more displays wherein contrast is adjusted for at least one of the displays such that the contrast of the two or more displays is balanced against each other within a predetermined range. In further embodiments, the contrast can be adjusted so that the contrast is equal between two or more displays. In embodiments, the contrast can be adjusted during the manufacturing process, during the dynamic range of the output produced by the integrated image light source, and others. In embodiments, the system may be configured to automatically check the contrast of two or more displays periodically against each other over the life of the eyepiece. In embodiments, the system may be configured to automatically check the contrast of two or more displays against each other and optionally to set the contrast of two or more displays to a predetermined value. In embodiments, the system may be configured to automatically check the contrast of two or more displays against each other, and optionally to set the contrast of two or more displays to a predetermined value based on the sensor feedback measurements. In embodiments, the contrast may be adjusted to compensate for the user's deficit. In embodiments, the contrast can be adjusted as a function of at least one of the stray light and the light generated by the integrated image light source. In embodiments, the contrast may be adjusted based on feedback from the detectors in the optical path of the system. In addition, the detectors may include at least one of a single pixel brightness detector and a multi-pixel color detector. In embodiments, the system may be provided with real-time feedback to compensate for at least one of assembly tolerance, LED and LCoS panel yield, binning tolerance, high temperature and low temperature panel compensation, and to maintain individual user calibration. In embodiments, the calibration of the contrast may be adjusted based on the search table for one or more compensation values.

In one embodiment, the particular optical configuration described herein, such as a front illuminated LCoS, allows the camera to be inserted into many locations along the optical train, to place the camera directly above the axis with the eye . For example, a camera sensor, such as camera 10232 in Figure 102B, may be placed adjacent to the LCoS. This in turn makes it possible to measure the position, diameter, velocity and direction of the pupil and to image the iris directly. Such measurements and imaging may include detecting a health condition by loading a secure login or user setting, measuring the size and / or thickness of capillary blood vessels, locating a placeholder based on the area last looked at in the book, / Set a bookmark, and so on. The data collected by the camera on the various components of the eye can be used to control the user interface, to determine the stress level, to monitor alertness, to detect responses to external or projected stimuli , Can be used for guitar. Since the front lighting optical system is sharp and compact, a camera having extremely small pixels can be disposed in the optical train, so that the overall size of the optical system can be kept small and a high-resolution image can be ensured. In embodiments, the camera can be placed in many parts of the optical path by inserting the beam splitter as in Figure 185, but it can also be done by placing the camera on the LCoS PCB, directly buried in the LCoS silicon substrate, Or other optical train placement.

In embodiments in which the camera lies directly above the axis with the eye, the camera can see or detect the eye or just inside the eye. In embodiments, the system can track eye movements, detect pupil dilation, measure pupil location, diameter, velocity, and direction, and can image iris directly. In embodiments, the camera may determine whether the user is browsing the environment or the user is controlling the eyepiece. By way of example only, the camera can sense a pattern of eye movements, thereby transmitting a signal to track the eye movements so that the user can sense certain control commands that his eyes may perform. By way of example, the camera can recognize that the user's eyes are reading something on the user interface, based on the pattern of the user's eye movements. In this case, the camera initiates detection of a specific series of eye commands to be transmitted to the eyepiece to perform a specific function, such as opening an email. In embodiments, the camera may be configured to allow the user to control the eyepiece in a predetermined manner to control the eyepiece, such as by focusing on the item for an extended period of time, focusing on the item, quickly moving the eye, It may be detected that the object may be focused on the object. When the camera detects such motion patterns, the camera can signal the eyepiece to perform a specific function. For example, focus, focus, and focus again can cause the camera to signal to the eyepiece that the user is about to "double-click" an item in the display. Of course, any such pattern and / or algorithm may be used to control the device through the user's eye movements. In embodiments, the camera can detect a specific motion pattern, and if such a motion is detected when a particular application is in use, the camera can send a specific signal to the eyepiece based on this combination. For example, if an email program is open and the user's eyes indicate a pattern that matches the reading, the camera may signal the eyepiece to open a particular email that the user's eyes are focusing on. In various embodiments, an instruction to control the eyepiece based on detection of the camera may be initiated.

In embodiments, the camera may enable security measures such as detecting pupil location, diameter, velocity and direction, imaging the retina and / or iris directly, and the like. For example, when a user wears an eyepiece, the camera may perform a retinal scan that identifies the user in contrast to a database stored on or remote from the eyepiece. In embodiments, if the user is identified as the owner of the glasses or as a user of the glasses, the glasses may open the application and provide access to the user. If the glasses do not recognize the user, the glasses may lock or disable all or some of the functions. In embodiments, the user may not need such a password, and the eyepiece may perform this function automatically. In embodiments, when the user is not recognized, the camera may obtain identification information about the wearer in case the wearer stole the eyepiece.

In embodiments, the eyepiece may perform user diagnostics based on detection of eye movement, detection of pupil location, diameter, velocity and direction, direct imaging of the retina and / or iris, and the like. For example, the diagnosis may be based on pupil dilation. For example, if the pupil of the user is expanded in a manner compatible with the sick person, the camera and / or eyepiece can detect that the user is sick. In addition, if the user is suffering from a concussion, the pupil may change in size despite the amount of light entering the eye. The eyepiece can alert the user if the user has a concussion. In embodiments, an eyepiece may be provided to them when a soldier, athlete, or the like quits physical activity, and the eyepiece may be used, for example, to diagnose the user as having a concussion. The eyepiece may have a database of users on the board or separate from the eyepiece, which may store information related to various users. In one embodiment, when a competitor exits the playground to the sideline, the player performs a retina scan to identify the user through the database and then detects the pupil size of the user and compares it with the expected pupil size for a given light state You can wear glasses to diagnose or inspect users. If the user ' s data is outside the expected range, the glasses may inform the user that his pupil is consistent with the concussion. Similar uses can be used, such as detecting possible drug intoxication, detecting retinal damage, detecting eye conditions, and the like.

In embodiments, organic light emitting diodes (OLEDs) may be used in applications to the microdisplay and / or sensors herein and may be used in Fraunhofer systems such as OLEDCam or in other ways in the detection of eye movements, Can be used in other ways, such as in an eyepiece illuminating the eyes of a person. In embodiments, an apparatus for detecting eye movement may be disposed along a light train on an axis with the user's eye. In embodiments, the micro-light emitter and receiver may be integrated within the same chip. They can be implemented as an array-like structure as bidirectional or unidirectional microdisplays. In embodiments, the device may simultaneously present and / or capture the image. The microdisplay can be the basis of the system for personalized information, presenting information to the user and recognizing the interaction by the user. The user can perceive the environment as usual, and additional information can be presented through an eyepiece equipped with a bi-directional display. The visual information may be adapted to the context of the operation of the system, and the user may interact with the movement or motion of the eye. In embodiments, a CMOS chip is a nested active matrix that can include a microdisplay and a camera on one substrate, the central element consisting of an OLED pixel and a photodiode. In embodiments, the pixel cell may comprise a red-green-blue and a red-green-blue-photodiode pixel cell or the like.

In embodiments, the system may include an interactive head-mounted eyepiece worn by a user, wherein the eyepiece includes an optical assembly through which the user views the ambient environment and the displayed content, And a camera disposed in the optical assembly along the optical axis to allow the camera to view at least a portion of the user's eye. In embodiments, the camera may be configured to capture images of an eye, a pupil, a retina, an eyelid, and / or an eyelash. In embodiments, an instruction to control the eyepiece based on at least one image captured by the camera may be initiated. In embodiments, the user's diagnosis may be based on at least one image captured by the camera. The identification of the user may also be based on at least one image captured by the camera. By way of example, the diagnosis may include a diagnosis of a concussion. In embodiments of the system, the identification of the user may be installed as a security aspect of the eyepiece. In embodiments, the integrated image light source may illuminate the eye during image capture by the camera. In addition, light from the image light source can be modulated during image capture by the camera. In embodiments, the camera may include one or more OLEDs (organic light emitting diodes). In embodiments, the user ' s eye and other areas listed herein (including iris, pupil, eyelid, eyelash, etc.) may be illuminated by various lighting devices, LEDs, OLEDs, In embodiments, illumination of the user's eye may be used for imaging techniques, for capturing eye data, for identification, and so on.

In one embodiment, the system may include an interactive head-mounted eyepiece worn by a user, wherein the eyepiece includes an optical assembly through which the user views the surrounding environment and the displayed content, An integrated image light source configured to detect an eye movement, and an apparatus for detecting an eye movement. In embodiments, the device for detecting eye movement may include a micro-light emitter and receiver that are integrated within the same chip. In embodiments, the apparatus may comprise a CMOS chip comprising a microdisplay and a camera on one substrate. In embodiments, an apparatus for detecting eye movement may be disposed along a light train on an axis with the user's eye.

In embodiments, the camera is disposed in the optical assembly along the optical axis so that the camera can view at least a portion of the user ' s eye and image one or more of the eye, pupil, retina, eyelid, and eyelashes. The integrated processor and camera track the eye movement of the user; Measuring at least one of pupil dilation, pupil position, pupil diameter, pupil velocity, and pupil direction; Distinguishing user eye movements intended as controls or commands from user eye movements for reading or gazing; The user's eye movement is used by the processor as an instruction to control the function of the integrated processor or interactive head-mounted eyepiece; And is configured to use the user's eye movement as an instruction to control an apparatus external to the user and external to the interactive head-mounted eyepiece. The diagnosis or identification of the user, such as a concussion, may be based on at least one image captured by the camera. The identification of the user may be provided as a security aspect of the eyepiece. The system may include a user input interface for controlling or signaling an external device based on eye movement from a user. The camera may be configured to capture images of both eyes, where the image is compared to a database containing different images of both eyes to indicate diagnosis. The optical axis of the integrated image light source and the optical axis of the camera may be different. The optical axis of the integrated image light source and at least a portion of the optical axis of the camera may be the same.

Devices such as a camera in an augmented reality eyepiece, a micro-light emitter and receiver incorporated in the same chip, or a CMOS chip including a microdisplay and a camera on a single substrate can detect a user's eye movement. The integrated image light source is configured to perform at least one of modulating light from the image light source and illuminating the eye during image capture by the camera. The camera may include one or more organic light emitting diodes (OLEDs). The device for detecting eye movement may be along the optical train on the axis with the user's eye or on an axis different from the user's eye. The integrated processor may be configured to interpret the user's eye movement as an instruction to operate the device in an interactive head-mounted eyepiece or external device.

A method of detecting a user's eye movement includes wearing a head mounted eyepiece wherein the head mounted eyepiece includes an optical assembly through which a user views the surrounding environment and the displayed content through the user, an integrated processor, The method comprising the steps of: detecting an eye movement of a user with a camera and an integrated processor, and controlling the device through an eye movement and an integrated processor, wherein the camera comprises: Detects motion of at least one eye of the user and interprets the motion as a command. The integrated processor can distinguish between eye movement as an instruction and eye movement as a stare. The method may include interpreting the determined eye movement as an instruction to perform a particular function. The method may include scanning at least one eye of the user to determine an identification of the user. The method may include scanning at least one eye of the user to diagnose a user ' s health condition. The camera may include at least one organic light emitting diode (OLED). A particular eye movement can be interpreted as a specific command. Eye movement, blinking, repetitive flicker, flicker rate, flicker rate, eye-cold (slow flicker), eye tracking, side to side, up and down, left to right, Eye movement of one eye, stopping time at one position, gazing at a fixed object, and gazing through a particular portion of the lens of the head-mounted eyepiece. The method may include controlling the device through eye movement and a user input interface. The method may include capturing a view of the environment with a camera or a second camera for display to a user.

In embodiments, the eyepiece may be a subconscious perception of the image of the wearer's surroundings, a scene viewed by an observer, a subconscious perception, or a subconscious perception, Side can be used. For example, a wearer may be aware that a wearer may not be aware of the wearer, but may be able to increase the level of attention to the subconsciously presented content (e.g., reminders, alerts (E.g., a warning that the eyepiece detects something in the wearer's sight that the eyewearer may be interested in for some time and the indication is of interest to the wearer) It is possible to present the image through the eyepiece at a speed at which it is recognized. In other cases, the eyepiece can provide an indicator to the wearer through a brain activity monitoring interface, where an electrical signal is generated within the brain before the human notices that the brain has recognized the image. For example, the brain activity monitoring interface may include an EEG (electroencephalogram) sensor (or other) to monitor brain activity when the wearer is viewing the current environment. When the eyecamper detects, through the brain activity monitoring interface, that the wearer has "recognized" the elements of the surrounding environment, the eyepiece can provide the wearer with conscious level feedback to make the wearer more aware of the element. For example, the wearer may be unconsciously perceived to see familiar faces (e.g., friends, suspects, celebrities) in the crowd, and the eyepiece may be visually or audiophile to make the wearer more consciously interested in the person Provide a mark to the wearer. In another example, the wearer may see a product that raises his interest in the subconscious level, and the eyecamer may provide the wearer with a conscious indication, additional information about the product, an improved view of the product, and links to additional information about the product can do. In embodiments, the ability of the eyepiece to extend the wearer ' s reality to unconscious levels may allow the eyepiece to provide the wearer with an augmented reality that exceeds the wearer ' s normal conscious experience with the world around the wearer.

In embodiments, the eyepiece may have a plurality of modes of operation, in which the control of the eyepiece is at least partially controlled by the position, shape, motion, etc. of the hand. To provide this control, the eyepiece may use a hand recognition algorithm to detect the shape of the hand / finger and then associate the hand configurations, perhaps in combination with the motion of the hand, as an instruction. In reality, these hand configurations may need to be re-used depending on the mode of operation of the eyepiece, since there may only be a limited number of hand structures and movements available for commanding the eyepiece. In embodiments, certain hand structures or movements may be assigned to switch the eyepiece from one mode to the next, thereby enabling reuse of hand movements. For example, referring to FIG. 15F, a user's hand 1504F may be moved to where the camera on the eyepiece is visible, which may then be moved to a different command (e.g., rotational movement 1508F, , Left and right motion 1512F, and the like). In a simple example, assume that there are two modes of operation-mode 1 for panning the view from the projected image and mode 2 for zooming the projected image. In this example, the user may wish to use a left-to-right finger-pointed hand motion to command a panning motion to the right. However, the user may also wish to use left-to-right finger pointing movements to command the zooming of the image to larger magnification. To enable dual use of this hand movement for both command types, the eyepiece may be configured to interpret hand movements differently according to the mode in which the eyepiece is currently present, where certain hand movements are assigned for mode switching . For example, a clockwise rotational motion may indicate a transition from a panning mode to a zooming mode, and a counterclockwise rotational motion may indicate a transition from a zooming mode to a panning mode. This example is for illustration and not limitation, and one of ordinary skill in the art will appreciate that this general technique may be used with various types of commands / modes, such as using hand (s) and finger (s) You will see how it can be used to implement structures.

In embodiments, the system includes an interactive head-mounted eyepiece eyepiece worn by a user that includes an optical assembly through which a user views the surrounding environment and the displayed content, and the optical assembly includes a correction An integrated processor for processing content for display to a user, and an integrated image light source for introducing the content into an optical assembly; And an integrated camera facility for imaging the gesture, where the integrated processor identifies the gesture and interprets it as a commanded instruction. A control instruction may provide manipulation of content for display, instructions to be delivered to an external device, and so on.

In embodiments, control of the eyepiece may be enabled through eye movement, eye movement, and the like. For example, there may be a camera on the eyepiece for viewing the wearer ' s eyes (the two eyes) toward the rear, where the eye movement or motion is blinking, repetitive flickering, flickering rate, flickering rate, eye- To a fixed position (e.g., the corner of the lens of the eyepiece), to a specific position through a series of positions, side to side, up and down, left to right, eye movement to a specific position, , Or by looking at a real-world object, or the like. In addition, the eye control can allow the observer to focus on a particular point on the displayed image from the eyepiece, and because the camera can correlate the observing direction of the eye with the point on the display, (E.g., flicker, touching the interface device, motion of the position sensing device, etc.) by the wearer. For example, an observer can view an object on the display and select the object through motion of the finger enabled through the position sensing device.

In some embodiments, the glasses may be equipped with an eye tracking device that tracks the movement of the user's or preferably both eyes; As an alternative, the glasses may be equipped with sensors for motion tracking of six degrees of freedom (i.e., tracking head movement). These devices or sensors are available, for example, from Chronos Vision GmbH, Berlin, Germany, and ISCAN, Woburn, Mass., USA. Retina scanners are also available to track eye movements. Retinal scanners can also be installed in augmented reality glasses, and are available from a variety of companies including Tobii, Stockholm, Sweden, SMI, Teltov, Germany, and ISCAN.

The augmented reality eyepiece also includes a user input interface, as shown, to allow the user to control the device. The input used to control the device may include any of the sensors discussed above, and may also include a trackpad, one or more function keys, and any other suitable local or remote device. For example, an eye tracking device may be used to control other devices such as video games or external tracking devices. As an example, FIG. 29A shows a user with an augmented reality eyepiece equipped with an eye tracking device 2900A discussed elsewhere in this document. The eye tracking device allows the eyepiece to track the direction of the user's or preferably both eyes and transmit the motion to the controller of the eyepiece. The control system includes an augmented reality eyepiece and a control device for the weapon. The movement may then be transmitted to a control device for the weapon controlled by the control device, which may be within the user's field of view. The movement of the user's two eyes is then converted into a signal that controls the movement of the weapon, such as the quadrant (range) and azimuth (direction), by appropriate software. Additional controls such as the user ' s trackpad or function keys may be used with eye tracking. The weapon may be a large-sized, such as a howitzer or a mortar, or may be a small-sized, such as a machine gun.

The movement of the user's two eyes is then converted into a signal by the appropriate software to control the movement of the weapon, such as the quadrant (range) and azimuth (direction) of the weapon. Additional controls, such as the user's trackpad or function keys, may be used for single or sequential firing of the weapon. Alternatively, the weapon may be stationary, non-directional (such as buried mines or shape-charge), secured by a safety device (e.g., ). ≪ / RTI > The user of the augmented reality device can activate the weapon by transmitting appropriate codes and commands without using the eye tracking feature.

In embodiments, control of the eyepiece may be enabled through the wearer's gesture. For example, the eyepiece may have a camera looking outside (e.g., front, side, bottom) and interprets the gesture or movement of the wearer's hand as a control signal. Hand signals may include passing a hand across a camera, hand position or sign language in front of the camera, pointing to a real world object (e.g., to activate an enhancement of an object), and the like. Hand motion can also be used to manipulate objects displayed inside a translucent lens, such as moving an object, rotating an object, deleting an object, opening or closing a screen or window in an image have. While hand movements have been used in the previous examples, any part of the body that the body or wearer holds or wears for gesture recognition by the eyepiece can also be used.

In embodiments, hand motion control may be used to transmit an instruction to the eyepiece, wherein a motion sensor, such as an accelerometer, gyro, or any other sensor described herein, is attached to the head of the wearer, , On a helmet, on a guitar. Referring to FIG. 14A, head movements may include sudden movement of the head in a forward and / or backward motion 1412, upward and / or downward movement 1410, in leftward and rightward motion, such as a nod, It may include rapid movement of the head, such as being in a position such as a side, moving and in position. The motion sensor may be integrated in the eyepiece, mounted on the user's head or by a wired or wireless connection to the eyepiece, or mounted on a head cover (e.g., a hat, helmet), or the like. In embodiments, the user may wear an interactive head-mounted eyepiece wherein the eyepiece includes an optical assembly through which the user views the surrounding environment and the displayed content. The optical assembly may include a correction element for correcting a user view of the environment, an integrated processor for processing content for display to a user, and an integrated image light source for introducing the content into the optical assembly. At least one of the plurality of head motion detection control devices that provide control instructions to the processor as commanded instructions based on the detection of the predefined head motion characteristics may be integrated in or associated with the eyepiece. The head movement characteristic may be a nod of the user's head, and thus the nod is a different head movement than the normal head movement. Explicit movements can be sudden movements of the head. Control instructions may provide manipulation of content for display, may be communicated to control an external device, or may be other. Head motion control may be used with other control mechanisms, such as activating commands and using head motion to perform other control mechanisms as discussed herein to execute them. For example, the wearer may wish to move the object to the right and select the object and activate the head movement control through eye control as discussed herein. Subsequently, by tilting his head to the right, the object can be commanded to move to the right, and the command can be terminated via eye control.

In embodiments, the eyepiece may be controlled via audio (e.g., via a microphone). The audio signal may include speech recognition, voice recognition, sound recognition, sound detection, and the like. Audio can be detected through an eyepiece microphone, a throat microphone, a jaw bone microphone, a boom microphone, a headphone with a microphone, an earphone, and the like.

In embodiments, the command inputs include turning on / off the eyepiece projector, turning the audio on / off, turning the camera on / off, turning the augmented reality projection on and off, turning the GPS on and off , Interacting with the display (e.g., selecting / accepting displayed functions, playing a captured image or video, etc.), interacting with the real world (e.g., capturing video or video, (E.g., forwarding, subsequent results, etc.), e-mail control (e. G., Sending < / RTI > (E.g., e-mail reading, text display, text-to-speech conversion, creation, selection, etc.), GPS and navigation control It may provide a plurality of control features such as location list). In embodiments, the eyepiece or a component part thereof may be automatically turned on / off via an eye tracing detection facility, for example via a sensor indication from an IR sensor, an accelerometer, a force sensor, a microswitch, a capacitive sensor, Off. For example, a capacitive sensor (e.g., in the nose of the user's nose) that senses that the eyepiece is no longer in physical contact with the user's skin when the user removes the eyepiece from his head, Off. Those skilled in the art will recognize other similar configurations for detecting when the eyepiece is removed. In embodiments, the eyepiece may sense when the divergent component is attached to / detached from the eyepiece and may use this detection to turn on / off the sides of the eyepiece. For example, a portion of the optical system can be split, and when a portion of the optical system is split, power to that portion of the eyepiece system is cut off to save battery power. The present disclosure may include a power management facility where the power management facility controls the power provided to select the components of the eyepiece in response to the sensor. The eyepiece may be mounted on a frame having a nose paddle and a foldable arm, wherein the hinge for the frame attaches the foldable arm, wherein the sensor is mounted on the nose ring of the frame, on the arm, on the hinge, Can be. The selected component may be an image light source, a processor, or the like. The power management facility may be in a sleep mode when the user is not wearing an eyepiece wherein the power saving mode may include periodic reading of the sensor wherein the power management facility is configured to allow the user to wear the eyepiece , It switches to the wake mode and turns on the power of the eyepiece. The power management facility can reduce power to the components based on the use of the eyepiece function, power remaining in the integral battery, network availability, power consumption rate, and the like. The reduction in power may be based on a user preference profile. The user can ignore the power reduction through the command. The user can be provided with an indication through the user interface of the eyepiece when power is being reduced. The electrochromic density in the optical assembly can be increased if the brightness level of the image light source is reduced as a result of reducing power to the image light source.

In embodiments, the eyepiece may be a stereoscopic, auto-stereoscopic, computer generated holography, volumetric display image, stereogram / stereoscope, an electro-holographic display, a parallax " two view " display, a parallax panoramagram, a re-imaging system, etc. To provide a 3D display image to the user, for example, by generating a perception of 3D depth to the observer. Displaying the 3D image to the user may utilize different images presented in the left and right eyes of the user, such as when the left and right optical paths have certain optical components that differentiate the image, Project different images on left eye and right eye, and do guitar. The optical path from the projector facility to the user's eye through the optical path may include a graphic display device that forms a visual representation of the object in three physical dimensions. A processor, such as an integrated processor in an eyepiece or a processor in an external facility, may provide 3D image processing as at least one step in the generation of a 3D image for a user.

In embodiments, a holographic projection technique, such as computer-generated holography (CGH), which is a method of digitally generating a holographic interference pattern, can be used to present 3D video effects to a user. For example, a holographic image may be projected by a holographic 3D display, such as a display, that operates based on interference of coherent light. Computer generated holograms have the advantage that the object they are trying to show need not have any physical reality at all (ie they can be generated entirely as 'synthetic holograms'). There are a number of different ways to calculate interference patterns for CGH, including computing and quantization techniques, as well as the field of holographic information and computing reduction. For example, Fourier transform methods and point source holograms are two examples of computing techniques. The Fourier transform method can be used to simulate the propagation of each plane of depth of an object to the hologram plane, where reconstruction of the image can take place in the far field. In an exemplary process, there may be two steps, where a light field in the far observer plane is first calculated and the field of view is again Fourier transformed into the lens plane , Where the wavefront to be reconstructed by the hologram is a superposition of the Fourier transform of each object plane in depth. In another example, the target image may be multiplied by a phase pattern to which an inverse Fourier transform is applied. The intermediate holograms can then be generated by shifting this image product and combined to create the final set. The final set of holograms can then be approximated to form a kinoform for sequential display to a user, where the kinoform is a surface-relief profile in which the phase modulation of the object wavefront is < RTI ID = 0.0 >Lt; / RTI > In the point light source hologram method, an object is decomposed into a self-luminous point, wherein an elementary hologram is calculated for all point light sources, and the final hologram is synthesized by superimposing all the element holograms .

In one embodiment, a 3D or holographic image may be made possible by a dual projector system, where the two projectors are stacked one over the other for 3D image output. And may enter the holographic projection mode by capturing an image or signal, such as a hand, SKU, RFID readout, etc., spread out by a control mechanism as described herein or with the palm of the hand. For example, the wearer of the eyepiece may see the letter "X" on the cardboard that causes the eyepiece to enter the holographic mode to turn on the second stacked projector. Choosing which hologram to display may be done by control techniques. The projector can project the hologram onto the letter 'X' on the cardboard. Related software can track the location of the letter 'X' and move the projected image with movement of the letter 'X'. In another example, the eyepiece may scan a SKU, such as an SKU on a toy assembly kit, and a 3D image of the completed toy assembly may be accessed from an online source or non-volatile memory. Interaction with the hologram, such as rotating the hologram, zooming in / out, etc., can be done using the control mechanism described herein. Scanning can be enabled by the associated barcode / SKU scan software. In another example, a keyboard may be projected into space or onto a surface. A holographic keyboard may be used to control or control any of the related applications / functions.

In embodiments, the eyepiece assembly may provide for locking the position of the virtual keyboard below the actual environmental object (e.g., table, wall, vehicle dashboard, etc.), so that when the wearer moves his head, Does not move. In one example, referring to FIG. 24, a user may be sitting at a table and wearing an eyepiece 2402, and may wish to input text into an application such as a word processing application, a web browser, a communication application, or the like. A user may have a virtual keyboard 2408 or other interactive control element (e.g., virtual mouse, calculator, touch screen, etc.) displayed for use in typing. The user command may provide an instruction to display the virtual keyboard 2408 and may use the hand gesture 2404 to indicate the fixed location of the virtual keyboard 2408. [ The virtual keyboard 2408 may then remain fixed in space relative to the external environment (e.g., fixed at a location on the table 2410), where the eyepiece assembly may also be mounted on the table 2410, And maintains the position of the virtual keyboard 2408. That is, the eyepiece 2402 can compensate the user's head movement to maintain a user view of the virtual keyboard 2408 located on the table 2410. [ In embodiments, the user may wear an interactive head-mounted eyepiece wherein the eyepiece includes an optical assembly through which the user views the surrounding environment and the displayed content. The optical assembly may include a correction element for correcting a user view of the environment, an integrated processor for processing content for display to a user, and an integrated image light source for introducing the content into the optical assembly. An integrated camera facility may be provided that identifies a user's hand gesture as an interactive control element position command, such as a hand-finger configuration, such as being moved in a specific manner, imaged in a particular manner, imaging the surrounding environment, and the like. The position of the interactive control element may then remain fixed in place relative to the object in the environment, irrespective of changes in the viewing direction of the user, in response to the interactive control element position command. In this way, the user can use the virtual keyboard in much the same way as using a physical keyboard, where the virtual keyboard stays in the same position. However, in the case of a virtual keyboard, there is no 'physical limitation' such as gravity that limits where the user can position the keyboard. For example, the user can stand next to the wall, position the keyboard position on the wall, and so on. One of ordinary skill in the art will appreciate that the " virtual keyboard " technology may be applied to any controller such as a virtual mouse, virtual touch pad, virtual game interface, virtual phone, virtual calculator, virtual paintbrush, virtual drawing pad, It can be applied. For example, the virtual touch pad can be visualized to the user through an eyepiece, can be placed by a user by use of a hand gesture, etc., and can be used instead of a physical touch pad.

In embodiments, the eyepiece facility may include an object (e.g., a virtual keyboard, a keypad, a calculator, a notepad, a keyboard, etc.) on the surface by applying distortions such as parallax, keystone, You can use visual techniques to render the projection of the joystick, control panel, book. For example, the appearance of a keyboard projected on the table top in front of a user with a suitable perspective can be assisted by applying a keystone effect, where the projection provided to the user through the eyepiece lies on the surface of the table It is distorted to appear to exist. In addition, these techniques can be applied dynamically to provide a suitable perspective even if the user moves around with respect to the surface.

In embodiments, the eyepiece facility may provide gesture recognition that may be used to provide a keyboard and mouse experience with the eyepiece. For example, with images of a keyboard, a mouse, and a finger overlaid on a lower portion of the display, the system can track the finger position in real time to enable a virtual desktop. With gesture recognition, tracing can be done without wires and external powered devices. In other cases, the fingertip position may be tracked through gesture recognition through the eyepiece without wires and external power by gloves or the like having a passive RFID chip at each fingertip. In this case, each RFID chip may have its own response characteristic which allows a plurality of fingers of the fingers to be read simultaneously. The RFID chip may be paired with the glasses and thus be distinguishable from other RFID chips that may be operating in the vicinity. The glasses may provide a signal to activate the RFID chip and may have two or more receive antennas. Each receive antenna may be coupled to a phase measurement circuit element, which in turn provides an input to a position determination algorithm. The location algorithm can also provide speed and acceleration information and, ultimately, algorithms that can provide keyboard and mouse information to the eyepiece operating system. In embodiments, with two receive antennas, the azimuthal position of each fingertip can be determined as the phase difference between the receive antennas. The relative phase difference between the RFID chips can then be used to determine the radial position of the fingertip.

In embodiments, the eyepiece facility may use visual techniques to render previously projected medical scans (e.g., x-rays, ultrasound, MRI, PET scans, etc.) have. For example, referring to Fig. 24A, an eyepiece can access an x-ray image of a wearer's hand. The eyepiece can then use its integral camera to view the wearer ' s hand 2402A and overlay the projected image 2404A of the x-ray on the hand. In addition, the eyepiece can maintain the image overlay as the wearer moves his hands and gaze towards the other. In embodiments, this technique may also be implemented while the wearer is looking into the mirror, where the eyepiece reverses the image to come onto the reflected image. This technique can be used for other purposes, as part of diagnostic procedures, for rehabilitation during physical therapy, to encourage exercise and diet, to diagnose or describe the condition to the patient. The image can be a wearer's image, a general image from a database of images of health conditions, and the like. A typical overlay can show some type of internal problem that indicates a physical condition, a projection of how the body will look when following a certain routine over a period of time, and so on. In embodiments, an external control device such as a pointer controller may enable manipulation of the image. In addition, an overlay of the images can be synchronized among a large number of people each wearing an eyepiece, as described herein. For example, both the patient and the physician can project an image into the patient's hand, where the physician can now describe the physical ailment, while the patient has a synchronized image of the projected scan, .

In embodiments, the eyepiece assembly may provide for removing a portion of the virtual keyboard projection when an intervening obstacle is present (e.g., the user's hand is interfering, in which case the keyboard is projected into the user's hand Is not desirable). In one example, referring to FIG. 30, eyepiece 3002 may provide a projected virtual keyboard 3008 to a wearer (e.g., on a table top surface). The wearer may then 'reach' on the virtual keyboard 3008 for typing. Since the keyboard is just a projected virtual keyboard rather than a physical keyboard, the projected virtual computer will be projected 'on the user's hand' if there is no compensation for the projected image. However, as in this example, the eyepiece may provide compensation for the projected image, and thus a portion of the wearer's hand 3004 that interferes with the intended projection of the virtual keyboard onto the table may be removed from the projection . That is, it may not be desirable for a portion of the keyboard projection 3008 to be visualized in the user's hand, so that the eyepiece excludes a portion of the virtual keyboard projection that is co-located with the wearer's hand 3004. In embodiments, the user may wear an interactive head-mounted eyepiece wherein the eyepiece includes an optical assembly through which the user views the surrounding environment and the displayed content. The optical assembly may include a correction element for correcting a user view of the environment, an integrated processor for processing content for display to a user, and an integrated image light source for introducing the content into the optical assembly. The displayed content may include interactive control elements (e.g., virtual keyboard, virtual mouse, calculator, touch screen, etc.). An integrated camera arrangement may image a body part of the user when the body part of the user interacts with the interactive control element, wherein the processor is configured to display a dialogue that is determined to be co- Removes a portion of the interactive control element by excluding a portion of the control element. In embodiments, this technique of partially projected image removal can be applied to other projected images and obstructions, and should not be viewed as being limited to this example of a hand on a virtual keyboard.

In embodiments, the eyepiece facility may provide an intervening blockage for any virtual content displayed on "real" world content. In the event that any reference frame that positions the content somewhat away is determined, any object passing between the virtual image and the observer will be able to display the displayed content < RTI ID = 0.0 > . ≪ / RTI > In embodiments, a variable focus technique may also be used to increase the perception of the distance hierarchy in the observed content.

In embodiments, the eyepiece facility may provide the functionality to determine the intended text input from a series of character contacts that are swiped across the virtual keypad, for example, using a finger, stylus, hand gesture, or the like. For example, referring to FIG. 37, the eyepiece may be projecting a virtual keyboard 3700, where the user wishes to input the word 'wind'. Normally, the user would press the key position individually for 'w', then 'i', then 'n' and finally 'd', and the equipment associated with the eyepiece (camera, Accelerometer, etc.) will interpret each position as a character to that position. However, the system may also monitor the movement or swipe of the user's finger or other pointing device over the virtual keyboard and determine the best match for the pointer movement. In this figure the pointer has passed a path 3704 starting at the letter w and passing through the letters e, r, t, y, u, i, k, n, b, v, f and d, It stopped. The eyepiece observes this sequence and determines the sequence through, for example, an input path analyzer, feeds the detected sequence to a word-matching facility, selects the best word (in this case, 'wind' Can be output as the text 3708. In embodiments, the eyepiece may monitor the movement of the pointing device over the keypad and determine the word more directly (e.g., through autocomplete word matching, pattern recognition, object recognition, etc.), where a 'separator ) Represents a space between words such as a pause in the movement of the pointing device, a tap of the pointing device, a swirling motion of the pointing device, and the like. For example, in a pattern or object recognition algorithm, an entire swipe path can be used to associate an entire word with an individual pattern formed by that finger when the user's finger passes through each character to form a word, The pause between words is a distinction between words. The eyepiece can provide the most appropriate word, a list of the most suitable words, and so on. In embodiments, the user may wear an interactive head-mounted eyepiece wherein the eyepiece includes an optical assembly through which the user views the surrounding environment and the displayed content. The optical assembly may include a correction element for correcting a user view of the environment, an integrated processor for processing content for display to a user, and an integrated image light source for introducing the content into the optical assembly. The displayed content may include an interactive keyboard control element (e.g., a virtual keyboard, a calculator, a touch screen, etc.), wherein the keyboard control element is associated with an input path analyzer, a word matching facility, and a keyboard input interface. A user may enter text by sliding a pointing device (e.g., a finger, a stylus, etc.) across the character keys of the keyboard input interface with a sliding movement through an approximate sequence of words the user wishes to input as text, The path analyzer determines the characters to be touched in the input path, and the word matching facility finds the best word match for the sequence of touched characters and inputs the best word match as the input text. In embodiments, if the displayed reference content is a sketch pad for freehand text or a 4-way joystick pad for controlling a game or an actual robot and an airplane It may be something other than a keyboard, such as another interface reference. Another example may be a virtual trump kit with a color pad or the like that the user "taps " to make a sound. The ability of the eyepiece to interpret the pattern of movement across the surface can make it possible to project the reference content in order to provide something to the user and provide visual and / or audio feedback to the user. In embodiments, the 'motion' detected by the eyepiece may be the movement of the user's eye as the user looks at the surface. For example, the eyepiece may have a facility for tracking the user's eye movement, and both the content display position of the projected virtual keyboard and the gaze direction of the user's eye, , And then interpret the motion as a word as described herein.

In embodiments, the eyepiece may include a hand gesture 'air lettering', such as air swiping letters, words, etc., visible from the built-in eyepiece camera using the wearer's fingers And the eyepiece interprets finger movements as letters, words, and symbols for commands, signatures, writing, emailing, texting, and the like. For example, a wearer may use this technique to sign documents using an 'air signature'. The wearer can use this technique to create text, such as in e-mail, text, documents, and so on. The wearer's eyepiece can recognize a symbol made by hand motion as a control command. In embodiments, as described herein, an inertial measurement unit (IMU) mounted on a device on a user's finger, hands, or the like, through hand gesture recognition interpreted by an image captured through an eyepiece camera, Air lettering can be implemented through other input control devices,

In embodiments, the eyepiece facility may provide for presenting displayed content corresponding to an identified marker indicative of an intent to display the content. That is, the eyepiece may be instructed to display a specific content in response to detecting a predetermined external visual cue. Visual clues can be images, icons, photographs, face recognition, hand composition, body composition, and so on. The displayed content may be an interface device displayed for use, a navigation aid to help a user find a location when the user goes to a travel destination, an advertisement when the eyepiece is viewing the target image, an information profile, and the like. In embodiments, the visual marker clue for display and its associated content may be stored in memory on the eyepiece, stored in an external computer storage facility and used as needed (e.g., by geographic location, proximity to a trigger target, ) Imported, generated by a third party, or the like. In embodiments, the user may wear an interactive head-mounted eyepiece wherein the eyepiece includes an optical assembly through which the user views the surrounding environment and the displayed content. The optical assembly may include a correction element for correcting a user view of the environment, an integrated processor for processing content for display to a user, and an integrated image light source for introducing the content into the optical assembly. An integrated camera facility for imaging external visual cues may be provided wherein the integrated processor interprets the external visual cue as an instruction to identify the visual cue and display the content associated with the visual cue. 38, in embodiments, the visual cue 3812 may be included in a signboard 3814 in the environment, wherein the projected content is associated with an advertisement. The billboard can be a bulletin board, and the advertisement for the personalized advertisement can be based on the user's preference profile. The visual cues 3802 and 3808 may be hand gestures and the projected content may be a projected virtual keyboard 3804 and 3810. [ For example, the hand gesture may be the thumb and index finger gesture 3802 of the first user's hand, and the virtual keyboard 3804 may be projected onto the palm of the first user's hand, You can type on the virtual keyboard with your hand. The hand gesture 3808 may be a thumb and index finger gesture combination of both user hands and the virtual keyboard 3810 may be projected between the user's hands as configured in the hand gesture, ≪ / RTI > can be typed on the virtual keyboard using the thumb of the keyboard. Visual clues can provide an automated resource to the wearer of the eyepiece that associates predetermined external visual cues with the desired result that is interfering with the projected content, thereby relieving the wearer of searching for clues by himself.

In embodiments, the eyepiece may include a visual recognition language translation facility for providing translations of visually presented content, e.g., road signs, menus, bulletin boards, store signage, books, magazines, Visual Recognition Language translation facilities can use optical character recognition to identify characters from content and match strings of characters with words and phrases through a translation database. This functionality may be entirely contained within the eyepiece (e.g., in off-line mode), or at least partially within an external computing device (e.g., on an external server). For example, the user may be in a foreign country, where the wearer of the eyecare does not understand signs, menus, etc., but the eyecamer can provide a translation thereof. This translation may appear to the user as a comment, and may replace a foreign language word (e.g., on a billboard) with a translation or the like provided to the user via audio translation. In this way, the wearer will not have to try to search for a translation of the words, but rather they will be provided automatically. In one example, the user of the eyepiece may be Italian, and when going to the US, it is necessary to interpret most road signs to drive safely. Referring to FIG. 38A, an Italian user of an eyepiece is viewing a US stop sign 3802A. In this example, the eyepiece identifies the characters on the sign, translates the word 'stop' into the Italian 'arresto' for 'stop', and makes the stop sign 3804A appear to be the word 'arresto' rather than 'stop'. In embodiments, the eyepiece may also provide a simple translation message to the wearer, provide audio translation, provide a translation dictionary to the wearer, and so on. The present disclosure provides an interactive head mounted eyepiece eyepiece worn by a user includes an optical assembly in which a user views the surrounding environment and the displayed content therethrough and an integrated image light source configured to introduce the content into the optical assembly -; An integrated camera for imaging text visible in the surrounding environment; And an optical character recognition facility for correlating one or more characters from a visible text with one or more characters of a first language and correlating one or more characters of a first language with one or more characters of a second language, And presents the displayed content as one or more characters of the two languages, and the displayed content is locked for one or more characters from the visible text. The presentation of one or more characters of the second language may appear to the user as annotations and may be placed as displayed content for the originally viewed text. The presentation of one or more characters of the second language can be superimposed on the visible location of the originally viewed text (e.g., the presentation of one or more characters of the second language overlaid on the originally seen text coincides with the font characteristics of the originally viewed text) do). The visible text can be a signboard, a printed document, a book, a road sign, a bulletin board, a menu, and the like. Optical character recognition equipment may be included in the eyepiece, provided outside the eyepiece, or provided in a combination of the interior and exterior. One or more characters may be words, phrases, strings of alphanumeric characters, and the like. One or more characters of the second language may be tagged to be available in a second eyepiece stored in an external facility and viewing the same text (e.g., the tagging includes a geographic location indication, an object identifier, etc.). In addition, the presentation of one or more characters of the second language may be stored such that when the view of the text moves out of the view of the eyepiece, the view of the text is re-invoked for presentation when the text moves back into the view of the eyepiece .

In one example, an eyepiece can be used in an adaptive environment, e.g., for a blind user. In embodiments, the result of face recognition or object identification may be processed to obtain an audible result and presented as audio to the wearer of the glasses via the associated earphone / headphone. In other embodiments, the results of face recognition or object identification may be converted to haptic vibrations in the glasses or the associated controller. In one example, if someone is standing in front of the user of the adaptive eyeglasses, the camera images the person, and the images are processed into an integrated processor for processing by the face recognition software, or facial recognition software running on the server or in the cloud Lt; / RTI > The results of facial recognition may be presented as written text on a display of glasses for a particular individual, but for a blind or amblyopic user, the results may be processed to obtain audio. In other examples, object recognition may determine that the user is approaching a road curb, exit, or other object, and the glasses or controller will warn the user audibly or haptically. For an amblyopic user, the text on the display may be enlarged, or the contrast may be increased.

In embodiments, a GPS sensor may be used to determine the position of the user wearing the adaptive display. A GPS sensor can be accessed by the navigation application program to audibly inform the user of the point of interest as it approaches or reaches various points of interest. In embodiments, a user may be audibly guided to a destination by a navigation application.

The eyepiece can be useful for a variety of applications and markets. It will be appreciated that the control mechanisms described herein may be used to control the functionality of the applications described herein. The eyepiece can run one application program at a time, or multiple applications can be executed at a time. The switching between the application programs can be done by the control mechanism described herein. GPS navigation (location, direction, speed and ETA), mobile TV, athletics (pacing, ranking, view of the time of the game); Receive coaching], telemedicine, industrial inspection, airline, shopping, inventory control tracking, firefighting (possible with fog, haze, VIS / NIRSWIR sensor visibility), outdoor activities / . In one embodiment, the eyepiece may be used for e-mail, Internet, web browsing, sports score viewing, video chatting, etc., such as GMAIL in FIG. In one embodiment, an eyepiece can be used for educational / training purposes, such as by displaying a step-by-step guide, such as hands-free, wireless maintenance and repair instructions. For example, video manuals and / or instructions may be displayed in the field of view. In one embodiment, an eyepiece can be used in fashion, health, and beauty. For example, a potential costume, hairstyle or makeup may be projected onto the user's mirror image. In one embodiment, eyepieces may be used in business intelligence, conferences, and conferences. For example, in order to acquire personal information, the user's name tag may be scanned, the user's face may pass through the face recognition system, or the name spoken by the user's voice may be retrieved from the database. Name tags, faces, and dialogues scanned for later viewing or for filing can be recorded.

In one embodiment, "mode" may be entered by the eyepiece. In this mode, a specific application program can be used. For example, the consumer version of the eyepiece may have a travel mode, an education mode, an internet mode, a TV mode, a game mode, a motion mode, a stylist mode, a personal assistant mode,

A user of the augmented reality glasses may wish to participate in a video call or video conference while wearing glasses. Many computers (both desktops and laptops) have integrated cameras to facilitate the use of videophones and videoconferencing. Typically, a software application is used to integrate the use of a camera having a telephone or conference function. By providing augmented reality glasses with most of the functionality of laptops and other computing devices, many users may want to wear augmented reality glasses and use videophones and video conferencing while on the move.

In one embodiment, a video telephony or video conferencing application program may operate as a WiFi connection, or may be part of a 3G or 4G call network associated with the user's mobile phone. Cameras for video telephony or video conferencing are located on a device controller, such as a wrist watch or other discrete electronic computing device. It is not feasible to place a videophone or videoconference camera on the augmented reality goggles because this arrangement will provide the user only with his or her own view and will not display other participants in the conference or call. However, the user may choose to use the front camera to display his or her surroundings or others during a videoconference.

32 shows a typical camera 3200 for use in a video telephone or videoconference. Such a camera is typically small and may be mounted on a wrist watch 3202, a mobile phone or other portable computing device (including a laptop computer), as shown in FIG. The videophone operates by connecting the device controller to a cellular phone or other communication device. The device uses software that is compatible with glasses and communication devices or with the operating system of the computing device. In one embodiment, the screen of the augmented reality glasses may display a list of options for placing a call, and the user may gesture using the pointing control device to select a videophone option on the screen of the augmented reality glasses, Lt; / RTI > can be used.

33 shows an embodiment 3300 of a block diagram of a video telephone camera. The camera includes a lens 3302, a CCD / CMOS sensor 3304, an analog-to-digital converter 3306 for a video signal, and an analog-to-digital converter 3314 for an audio signal. The microphone 3312 collects audio input. Both of these analog-to-digital converters 3306 and 3314 transmit its output signal to the signal enhancement module 3308. [ The signal enhancement module 3308 conveys an enhanced signal, which is a composite of both the video signal and the audio signal, to the interface 3310. The interface 3310, together with the control module 3316, is connected to an IEEE 1394 standard bus interface.

In operation, a video telephone camera relies on signal acquisition to convert incident light as well as incident light into electrons. In the case of light, this process is performed by a CCD or CMOS chip 3304. The microphone converts the sound into an electrical impulse.

The first step in the process of generating an image for a videophone is to digitize the image. A CCD or CMOS chip 3304 separates the image and converts it to a pixel. If the pixel collects a lot of photons, the voltage will be high. When the low-pixel photon is collected, the voltage will be low. This voltage is an analog value. During the second phase of digitization, the voltage is converted to a digital value by an analog-to-digital converter 3306, which is responsible for image processing. At this point, a raw digital image is available.

The audio captured by the microphone 3312 is also converted to a voltage. This voltage is sent to an analog-to-digital converter 3314 where the analog value is converted to a digital value.

The next step is to improve the signal so that the signal can be transmitted to the observer of the video telephony or videoconference. The signal enhancement includes generating color on the image using a color filter located in front of the CCD or CMOS chip 3304. [ This filter is red, green or blue and changes its color per pixel, and in one embodiment may be a color filter array or a Bayer filter. These raw digital images are then enhanced by a filter to meet aesthetic requirements. Audio data can also be improved for a better call experience.

In the final stage before transmission, in one embodiment, the video and audio data is compressed and output as a digital video stream using a digital video camera. When a photo camera is used, a single image may be output, and in a further embodiment, a voice comment may be attached to the file. Enhancement of the raw digital data is done away from the camera and, in one embodiment, can be done in a device controller or computing device that communicates with augmented reality glasses during a video call or videoconference.

Additional embodiments may provide portable cameras for use in industry, medical, astronomy, microscopy, and other fields requiring special camera use. These cameras often output raw digital images without signal enhancement. These cameras may be mounted in other electronic devices or in the user's hands for ease of use

The camera interfaces with augmented reality glasses and device controllers or computing devices using an IEEE 1394 interface bus. The interface bus transmits time critical data, such as video and data, that are critical to integrity, including parameters or files for manipulating data or delivering images.

In addition to the interface bus, the protocol defines the behavior of devices associated with video telephony or video conferencing. Cameras for use with augmented reality glasses may, in embodiments, use one of the following protocols: AV / C, DCAM, or SBP-2.

AV / C is a protocol for audio video control and defines the behavior of digital video devices, including video cameras and video recorders.

DCAM is a 1394-based digital camera specification, which defines the behavior of a camera that outputs uncompressed image data without audio.

SBP-2 refers to the Serial Bus Protocol and defines the behavior of mass storage devices such as hard drives or disks.

Devices using the same protocol can communicate with each other. Thus, in the case of a videophone using augmented reality glasses, the same protocol can be used by the device controller and the video camera on the augmented reality glasses. Since augmented reality glasses, device controllers and cameras use the same protocol, data can be exchanged between these devices. Files that may be transferred between devices include video and audio files, video and audio data flows, parameters controlling the camera, and the like.

In one embodiment, a user attempting to initiate a video call can select a video call option from the screen presented when the call process is initiated. The user makes a selection by performing a gesture using the pointing device or by performing a gesture signaling the selection of the video call option. The user then places the camera located on the device controller, wrist watch or other removable electronic device so that the user's image is captured by the camera. The images are processed through the process described above and then streamed to the augmented reality glasses and other participants for display to the users.

In embodiments, the camera may be mounted on a cell phone, a personal digital assistant (PDA), a wrist watch, a pendant, or other small portable device that can be carried, worn or mounted. An image or video captured by the camera can be streamed to the eyepiece. For example, when a camera is mounted on a rifle, the wearer can image a target that is not in the line of sight and wirelessly receive the image as a stream of displayed content to the eyepiece.

In embodiments, the present disclosure may provide the wearer with GPS-based content reception as in FIG. As discussed, the augmented reality glasses of the present disclosure may include a memory, a global positioning system (GPS), a compass or other orientation device, and a camera. A GPS-based computer program available to the wearer may include a number of applications typically available from Apple Inc.'s App Store for use with the iPhone. Similar versions of these programs are available for other brands of smartphones and can be applied to embodiments of the present disclosure. These programs include, for example, SREngine (scene recognition engine), Nearest Tube, TAT Augmented ID, Yelp, Layar, and Twittound as well as other more specialized applications such as RealSki.

SREngine is a scene recognition engine that can identify the objects that the user's camera sees. It is a software engine that can recognize static scenes such as scenes of buildings, structures, pictures, objects, rooms, and so on. This can then automatically attach a virtual "label" to the structure or object as he perceives it. For example, when viewing a distance scene as in FIG. 6, the program may be called by the user of the present disclosure. Using augmented reality glasses camera, this engine will recognize the fountain of Concorde Square in Paris. This program will then invoke the virtual label shown in Figure 6 as part of the virtual image 618 projected onto the lens 602. [ This label may be text only, as seen from the bottom of the image 618. Other labels that can be affixed to this scene may include the name of a building with a "fountain", a "hotel", or a column at the rear. Other programs of this type can include Wikitude AR Travel Guide, Yelp and many others.

NearestTube uses the same technology to guide users, for example, to the nearest subway station in London, and other programs can perform the same or similar functions in different cities. Layar is a camera, compass or direction, and other applications that use GPS data to identify the user's location and view. With this information, an overlay or label may appear virtually to help inform the user of the location and guide the user. Yelp and Monocle perform similar functions, but their databases are somewhat more specialized, helping to guide users to restaurants or other service providers in a similar way.

The user can control the glasses and invoke these functions using any of the controls described in this patent. For example, the glasses may be equipped with a microphone to pick up a voice command from the user and use it to process it using software contained in the memory of the glasses. The user can then also respond to a prompt from a small speaker or earphone included in the eyeglass frame. The glasses may also be equipped with a small trackpad similar to that on a smartphone. The trackpad may allow a user to move a pointer or indicator on a virtual screen within the AR glasses similar to a touch screen. When the user reaches a desired point on the screen, the user presses the trackpad to announce his selection. As such, the user can call a program (e.g., a tour guide) and find his way through several menus (perhaps by selecting a country, city and then a category). The category selection can include, for example, hotels, shopping, museums, restaurants, and the like. The user makes a selection and is then guided by the AR program. In one embodiment, the glasses also include a GPS locator, and the current country and city provide a base location that can be replaced.

In one embodiment, the object recognition software of the eyepiece may process the image received by the forward camera of the eyepiece to confirm that it is in the field of view. In other embodiments, the GPS coordinates of the position determined by the GPS of the eyepiece may be sufficient to confirm that it is in view. In other embodiments, an RFID or other beacon in the environment may be broadcasting the location. Any one or combination of the above may be used by the eyepiece to identify the position and the ID of what is in sight.

When an object is recognized, the resolution of imaging the object can be increased or the image or video can be captured with low compression. In addition, in order to reduce the required bandwidth, the resolution for other objects in the user's view can be reduced or captured with a higher compression ratio.

Once determined, content related to points of interest in the field of view can be overlaid on real-world images such as social networking content, interactive travel, local information, and the like. Information and content related to movies, local information, weather, restaurants, restaurant availability, local events, local taxis, music, etc., can be accessed by the eyepiece and projected onto the eyepiece lens for viewing and interacting by the user. For example, when the user views the Eiffel Tower, the front camera can take an image and transmit it to the associated processor of the eyepiece for processing. The object recognition software can determine that the structure in the wearer's field of view is the Eiffel Tower. Alternatively, GPS coordinates determined by the GPS of the eyepiece can be retrieved from the database to determine whether the coordinates coincide with the coordinates of the Eiffel Tower. In either case, content regarding Eiffel tower visitor information, restaurants in the vicinity of the Eiffel Tower itself, local weather, local metro information, local hotel information, other nearby attractions, and the like can then be retrieved. Interacting with the content may be enabled by the control mechanisms described herein. In one embodiment, GPS-based content reception may be enabled when entering the traveler mode of the eyepiece.

In one embodiment, an eyepiece may be used to view the streaming video. For example, the video can be identified by searching by GPS position, by object recognition of objects in view, by voice search, by holographic keyboard search, and the like. Continuing with the example of the Eiffel Tower, if it is determined to be a structure in view, the video database can be searched through the GPS coordinates of the Eiffel Tower or by the term " Eiffel Tower ". The search results may include geo-tagged video or video associated with the Eiffel Tower. Video may be scrolled or flipped using the control techniques described herein. Video of interest may be played using control techniques described herein. The video can be overlaid on the real-world scene or displayed on the lens out of view. In one embodiment, the eyepiece may be dimmed through the mechanisms described herein to enable a higher contrast view. In another example, to provide a videoconference streaming function to the wearer, the eyepiece may utilize a camera and network connection, as described herein. The streamed video may be video, visual presentation, etc. of at least one other video conference participant. The streamed video may be automatically uploaded to the video storage location upon capture, without the user interaction of the eyepiece. The streamed video may be uploaded to a physical or virtual storage location. The virtual storage location may be located in a single physical location or cloud storage location. The streamed video of the videoconference may also be modified by the eyepiece, where modification may be based on the sensor input. The sensor input may be a visual sensor input or an audio sensor input. The visual sensor input may be video, visual presentation, etc. of other participants of the videoconference. The audio sensor input may be the voice of a particular participant of the videoconference.

In embodiments, the eyepiece may be a wireless streaming media (e.g., video, audio, text, etc.) from an external facility such as a smartphone, tablet, personal computer, entertainment device, portable music and video device, home theater system, home entertainment system, Messages, phone calls, and calendar alerts). Wireless streaming media may be wireless, such as Bluetooth, WiFi, wireless home network connectivity, wireless local area network (WLAN), wireless home digital interface (WHDI), cellular mobile communication, A communication system and a protocol. The eyepiece can also be used in a number of wireless communication systems, such as for streaming high data rate media (e.g., video), for low data rate media (e.g., text messages), for command data between external equipment and eyepieces, Can be used. For example, high data rate video may be streamed over Bluetooth for low data rate applications such as WiFi Digital Living Network Alliance (DLNA) interfaces and text messaging. In embodiments, the external facility may include an application program that supports an interface with an eyepiece. For example, a mobile application program may be made available to the user to interface the user's smartphone with the eyepiece. In embodiments, the external facility may have transmission facilities to interface with the eyepiece. For example, a transmitter dongle may be provided to interface the user's smartphone with the eyepiece. Because streaming media from an external device can place most of the processing requirements on external devices, the eyepiece may require less onboard processing power to accommodate streaming media. For example, one embodiment of an eyepiece for responding to streaming media includes receiving streaming media, buffering data, providing streaming media to an optical assembly through which a user sees the surrounding environment and displayed content, and so on Lt; / RTI > That is, one embodiment of an eyepiece that receives streaming media may be a simplified version (e.g., serving as a display for external equipment) of other embodiments of the eyepiece described herein. In one example, a user may stream video from his smartphone to a 'simplified version' of the eyepiece. However, those skilled in the art will appreciate that versions that include the full range of capabilities described herein from the simplest version of the eyepiece (e.g., serving only as a display interface to an external facility) (e.g., a wireless streaming interface, It will be appreciated that any of the additional functions described herein may also be included to produce embodiments versions of the eyepiece that are only one of a plurality of functions and capabilities provided by the eyepiece. For example, control techniques such as those described herein, power saving techniques, application programs, driving one or both displays with streaming media, displaying in 3D mode, etc., It may also be useful in a simpler version of the eyepiece to help with battery management for increased life, optional media viewing mode, and so on. As an alternative, a quick version of the eyepiece may provide an embodiment that minimizes the cost and complexity of the eyepiece (e.g., the interface between the external facility and the eyepiece is a wired interface). For example, one embodiment of an eyepiece may provide a wired interface between a user's smartphone or tablet and an eyepiece, where the processing capabilities of the eyepiece can now include a streaming media to view the content on the eyepiece lens (s) But can be limited to the processing required to present it to the assembly.

In other embodiments, an app running on a smartphone may function as a remote input device for glasses. For example, a user interface, such as a keyboard, may allow a user to type characters through a smartphone. The app will make the phone look like a Bluetooth keyboard. The app may simply be a full screen blank app that sends the touch to the pseudo touch-screen driver running on the glasses, so that the user can perform these moves and provide tactile feedback on the user's hands and visual feedback on the glasses To get it, you could pinch and drag using your smartphone as a real physical place. As such, the more common apps running on glasses using these types of input gestures will work well for users using the smartphone touchscreen. The command information may be accompanied by a visual indicator. For example, to know where a finger is when using an external device to control the glasses or glasses app, a visual indication of the command information may be displayed in the glasses (e.g., a highlighted trace of finger movement) . The present disclosure provides an interactive head mounted eyepiece eyepiece worn by a user includes an optical assembly in which a user views the surrounding environment and the displayed content therethrough and an integrated image light source configured to introduce the content into the optical assembly -; Integrated processor; A physical user interface, and an external device having an application program for converting an external device into a user interface for an eyepiece operable through an integrated processor, wherein the physical interaction with the external device is displayed on the displayed content. In embodiments, the external device may be a smart phone, tablet, mobile navigation device, or the like. The physical user interface may be a keypad, a touchpad, a control interface, or the like. For example, the physical interface may be an iPhone, and the displayed content may be a virtual keyboard that displays the user's actions on the iPhone keypad as an action on the virtual keyboard that is displayed content on the eyepiece (e.g., Showing the highlighted keys, keystroke indicators on the virtual keypad when physically interacting with the iPhone's physical keypad). The finger action may be one of selection of the content and movement of the displayed content. The manipulation may be multiple finger operations on the touchpad (e.g., pinch manipulation to resize the displayed content on the eyepiece).

As we have seen, augmented reality users can receive content from rich sources. Visitors or travelers may wish to restrict their choice to a local business or organization; On the other hand, an entrepreneur looking for a visitor or traveler may want to limit his suggestion or solicitation to those in his area or place, but who are visiting rather than a local resident. As such, in one embodiment, a visitor or traveler may limit his search to a local business entity (i.e., a business within a particular geographic boundary). These boundaries can be set by way of a GPS reference or by manually indicating a geographical limitation. For example, a person may need to limit the streaming content or the source of the advertisement to those within a certain radius (set number or km or miles) from that person. Alternatively, this criterion may require that the source be limited to being within a particular city or province. These boundaries can be set by the augmented reality user as if the user of the computer at home or office were to restrict his search using a keyboard or a mouse; The input of the augmented reality user is done by simply voice, by hand movements, or in any other manner as described in portions of the present disclosure discussing control elsewhere.

In addition, the available content selected by the user may be limited or limited by the type of provider. For example, a user may restrict the selection to a web site (.gov) run by a government agency or to a website (.org) run by a nonprofit or organization. In this way, travelers or visitors who may be more interested in visiting government offices, museums, heritage sites, etc., will find that their choices are cleaner. It is easier to make a decision when the available options have been reduced to a more reasonable number. It is desirable to be able to rapidly reduce the options available in urban areas such as Paris or Washington with many options.

The user controls the glasses in any of the modes or modes described elsewhere in this patent. For example, the user may call the desired program or application by voice or by displaying the selection on a virtual screen of the augmented reality glasses. As described above, the augmented reality glasses can respond to track pads mounted on frames of glasses. As another alternative, the glasses may be responsive to one or more movements or position sensors mounted on the frame. The signal from the sensor is then transmitted to a microprocessor or microcontroller in the glasses and the glasses also provide any necessary signal conversion or processing. Once the selected program is started, the user may select by any of the methods discussed herein, such as signaling "yes" or "no" with head movement, hand gesture, trackpad press, And enters a response.

At the same time, the content provider (i. E., Advertiser) may also wish to limit his offer to people within a particular geographic area (e.g., its city boundary). At the same time, an advertiser (perhaps a museum) may not want to provide content to local people and may want to offer it to visitors or outsiders. In another example, an advertisement may not be presented when the user is at home, but may be presented when the user is traveling or is away from home. The augmented reality device discussed herein is preferably equipped with an integrated processor that implements both geographically based rules for advertising and presentation of both GPS and communication functions. It would be a simple matter for a museum to provide streaming content within a restricted area by limiting its broadcast power. However, museums can provide content over the Internet, and their content can be used globally. In this case, the user can receive content through the augmented reality device that informs the museum that it can be opened and viewed today.

The user can respond to the content by the augmented reality equivalent of clicking on the link to the museum. Augmented reality equivalents may be voice representations of the user's choice, hand or eye movements, or other sensory indicia, or using the associated body-mounted controller. The museum then receives the user's identity or at least a cookie indicating the user's ISP (internet service provider). If the cookie indicates or implies an Internet service provider other than the local provider, the museum server may respond with an ad or suggestion tailored to the visitor. The cookie may also include an indication of a communication link (e.g., a telephone number). If the telephone number is not a local number, this is an additional clue that the responding person is a visitor. The museum or other institution may then continue with the content requested or suggested by his marketing department.

Other applications of the augmented reality eyepiece utilize the user to be able to control the eyepiece and its tools using a minimal amount of user's hand and instead using voice commands, gestures or movements. As previously noted, the user may request that the augmented reality eyepiece be searched for information. This information may already be stored in the eyepiece's memory, but may instead be located remotely, such as in a database accessible via the Internet or possibly via an intranet accessible only by employees of a particular company or organization. As such, the eyepiece can be likened to a computer or display screen that can be seen and heard at very short distances and can generally be controlled and used with a minimum of human hands.

As such, an application may include providing information to a mechanical or electronic technician on the fly. The technician can wear the glasses, for example, when searching for information regarding the particular structure or problem encountered when repairing the engine or power supply. Using voice commands, the technician can access the database and retrieve specific information (e.g., manual or other repair and maintenance documents) within the database. In this way, desired information can be immediately accessed and applied with minimal effort, allowing the technician to perform the necessary repairs or maintenance more quickly and re-use the equipment. In addition to reducing repair or maintenance costs, these time savings can also save lives for mission critical equipment.

The information provided may include repair manuals, etc., but may also include a full range of audio-visual information (i.e., the eyepiece screen may include a video descriptor of how to perform a particular task at the same time as attempting to perform the task) Or to a mechanic). The augmented reality device also includes communication capabilities, so the technician can also ask others for help if there are any problems or unexpected difficulties in the work. This educational aspect of the present disclosure is not limited to maintenance and repair, but may apply to any educational endeavor, such as secondary school or post secondary school, continuing education courses or subjects, seminars, and the like.

In one embodiment, the Wi-Fi enabled eyepiece may execute a location based application for the geographic location of the participating users. Users can join by logging into the application on their phone and enabling the broadcast of their location or by enabling their geographical location in their own eyepiece. When the wearer of the eyepiece scans people, and therefore his participating device, the application can identify the participating users and send an indication to the projector to project the augmented reality indicator to the participating users in the user's field of view. For example, a green ring can be placed around people who have participated in making their location visible. In another example, the yellow ring may represent people who have joined but do not meet certain criteria (e.g., people who do not have a Facebook account, people who have a Facebook account, but friends of each other).

Some social networking, career networking, and dating applications can work with location-based applications. The software in the eyepiece can organize data from networking and friend site and location-based applications. For example, Twittound is one such program that uses a camera mounted to detect and label location-stamped tweets from other nearby tweeters. This allows a person using this disclosure to find other nearby Twitter users. As an alternative, users may have to configure their device to organize information from various networking and dating sites. For example, a wearer of an eyepiece may wish to view all E-HARMONY users broadcasting their location. If the participating user is identified by the eyepiece, the augmented reality indicator can be overlaid on the participating user. If the user has something in common with the wearer, has a lot in common with the user, or the like, the indicator can take on a different appearance. For example, referring to FIG. 16, the wearer sees two persons. Both of these are identified as the following monitor users by the ring being arranged around them. However, a woman represented by a solid line ring has more than one item in common with a wearer, while a woman represented by a dashed ring does not have any items in common with the wearer. Any available profile information can be accessed and displayed to the user.

In one embodiment, the wearer is directed to a user having a networking account such as FACEBOOK, TWITTER, BLIPPY, LINKEDIN, GOOGLE, WIKIPEDIA, When directing the eyepiece, the user's recent post or profile information may be displayed to the wearer. For example, as noted above for TwittARound, recent status updates, "tweets", "blips", and the like can be displayed. In one embodiment, when the wearer is aiming the eyepiece towards the direction of the target user, if the eyepiece is aimed for a period of time and / or when a gesture, head, eye or audio control is activated, I do not know. The target user can receive an indication of interest in his phone or in his glasses. If the target user marks the wearer as interested but waits for the wearer to first express interest, an indication of the target user's interest may pop up immediately on the eyepiece. A control mechanism may be used to capture the image and store the target user's information in the associated non-volatile memory or in an online account.

In other applications for social networking, face recognition programs such as TAT Augmented ID from TAT (The Astonishing Tribe) of Malmo, Sweden can be used. Such a program can be used to identify a person by his facial features. The software uses face recognition software to identify people. Other application programs, such as photo identification software from Flickr, can be used to identify people in a particular neighborhood and download information from social networking sites that have information about that person. This information may include the name of the person and the profile that the person can use on sites such as Facebook, Twitter, and the like. The application can be used to refresh the user's memory for that person or to collect information about the person as well as to identify the person in the vicinity.

In other applications for social networking, the wearer may use the location-based equipment of the eyepiece to leave notes, explanations, reviews, etc. in various places relating to people, places, products, and the like. For example, you can post a description of where you came from, where postings can then be made available to others via social networks. In another example, the description may be posted at the place of the visit so that the description is available when the other person comes to the place. In this way, the wearer can access comments left by others when they come to the place. For example, a wearer may come to the entrance of a restaurant and have access to a later review of restaurants categorized, for example, by some criteria (e.g., latest, late, author's age, etc.).

The user can select the desired program by selecting from the virtual touch screen, by voice, as described above, by using the trackpad to select the desired program, or by using any of the control techniques described herein The desired program can be started. The menu selection can then be done in a similar or complementary manner. Sensors or input devices (e.g., wrist pads, sensors and track pads mounted on gloves, or even individual devices that are the size of a smart phone or a personal digital assistant) mounted at a convenient location on the user's body Can be used.

The application of the present disclosure may provide the wearer with Internet access, for example, for browsing, searching, shopping, entertainment, etc., via a wireless communication interface to an eyepiece, for example. For example, a wearer may use any component (e.g., a personal computer, a smartphone, a personal digital assistant, a personal digital assistant , A music player) can initiate a web search from a guitar to a control gesture in a furniture (e.g., a chair, a desk, a table, a lamp) near the wearer where the video of the web search is viewed do. The wearer can then view the search through the eyepiece and control the web interaction through the control facility.

In one example, a user may be wearing an embodiment configured as a pair of glasses, and at least a portion of the actual surroundings may be viewed at the same time, and a projected image of an internet web browser is provided through the glasses. In this case, the user may be wearing a motion-sensitive control facility in his hand, where the control facility may control the relative movement of the user's hand with respect to the eyepiece, for example in a similar manner to a mouse in a conventional personal computer configuration, Lt; / RTI > It will be appreciated that the user will be able to perform web operations in a manner similar to conventional personal computer configurations. In this case, while the image of the web search is provided through the eyepiece, the control for the selection of the operation to perform the search is provided through the movement of the hand. For example, the overall motion of the hand may move the cursor within the projected image of the web search, the flick of the finger (s) may provide a selection action, and so on. In this manner, the wearer may perform desired web browsing or any other Internet browser supporting function through one embodiment connected to the Internet. In one example, a user may access a computer program Yelp or Monocle, which is available from the App Store, or NRU ("near you") applications from Zagat to find restaurants or other stores nearby, Google Earth, You may have downloaded a similar product. The person may initiate a search for, for example, a restaurant or other product or service provider (e.g., hotel, repairman, etc.), or information. When desired information is found, the place is displayed or the distance and direction to the desired place are displayed. This display can take the form of a virtual label that is co-located with the real world object in the user view.

Other applications from Layar (Amsterdam, The Netherlands) include various "layers" that are tailored to the specific information the user wants. The hierarchy may include restaurant information, information about a particular company, listing of real estate, gas stations, and the like. Using information provided in a software application such as a mobile application and a user's global positioning system (GPS), information can be provided on the screen of the eyeglasses with the tag having the desired information. Using haptic control or other controls discussed elsewhere in this disclosure, a user can view buildings tagged with virtual tags that pivot his body or otherwise rotate and contain information. If the user is looking for a restaurant, the screen will display restaurant information (name and location, etc.). If the user is looking for a particular address, a virtual tag will appear on the building in the wearer's view. The user can then select by voice, by the trackpad, by the virtual touch screen, or by guitar.

The application of the present disclosure may provide a manner in which the advertisement is delivered to the wearer. For example, an advertisement may be displayed to an observer through an eyepiece while the observer begins a day, browses the Internet, performs a web search, walks the store, or the like. For example, a user may be performing a web search, and through a web search, the advertisement is targeted to the user. In this example, the ad may be projected in the same space as the projected web search and floated to the side, upper, or lower side of the wearer's view angle. In another example, when an ad provisioning facility (perhaps proximate to the wearer) detects the presence of an eyepiece (e.g., via wireless connection, RFID, etc.) and sends the ad to the eyepiece, the ad may be triggered to be delivered to the eyepiece have. In embodiments, the eyepiece may be used to track ad interaction (e.g., to allow a user to view or interact with billboards, promotions, advertisements, etc.). For example, the user behavior for an advertisement can be tracked, such as to provide benefits, rewards, etc. to the user. In one example, each time a user views the billboard, the user may be paid $ 5 in virtual cash. The eyepiece can provide impression tracking based on viewing brand images (e.g., based on time, geographic location). As a result, the proposal may be targeted based on the location and events associated with the eyepiece (e.g., viewed, listened to, or interacted with by the user). In embodiments, ad targeting may be based on past behavior (e.g., the user has interacted in the past, patterns of interaction, etc.).

For example, a wearer may be doing window shopping in Manhattan, where the store is equipped with such an ad provisioning facility. When the wearer walks by the store, the ad serving facility may trigger delivery of the ad to the wearer based on the user's known location determined by an integrated position sensor (GPS, etc.) of the eyepiece. In one embodiment, the user's location may be further subdivided by other integrated sensors, such as a magnetometer, to enable hyperlocal augmented reality advertising. For example, if the magnetometers and GPS readings represent a user in front of a particular store, a user on the first floor of the shopping mall can receive a particular ad. When the user climbs up the mall, the GPS position may remain the same, but the magnetometer reading may indicate a user's altitude change and a new placement of the user in front of a different store. In embodiments, for the purpose of enabling the wearer to block at least part of the advertisement so that the wearer ' s can provide preferences for the advertisement, so that the advertisement provisioning facility can better match the advertisement to the wearer & Personal profile information can be stored. The wearer can also communicate advertising and related discounts to friends. The wearer may be in proximity to them and may direct their own eyepieces to friends who can use them; The wearer can also forward them to a friend of the social network via a wireless Internet connection, for example via e-mail, SMS, or the like. The wearer is in a facility and / or infrastructure that enables delivery of advertisements from the sponsor to the wearer; Feedback from the wearer to advertising facilities, advertising sponsors, etc.; Other users, such as friends and family, or someone close to the wearer; Locally in an eyepiece, for example, at a store, or at a remote site, for example, on the Internet or on a user's home computer. These interconnecting facilities provide the user's position and gazing direction through the use of, for example, a GPS, a three-axis sensor, a magnetometer, a gyro, an accelerometer, etc. to determine the wearer's direction, speed and attitude It is possible to include a facility integrally provided in the eyepiece. The interconnect facility can provide communications facilities such as cellular links, WiFi / MiFi bridges, and the like. For example, the wearer may communicate via guitar via an available WiFi link, via MiFi (or any other individual or group cellular link) that is integral to the cellular system. There may be a facility for the wearer to store advertisements for later use. For example, there may be a facility in the local area that is integrated with the eyepiece of the wearer or that is located at the local computer facility, which enables the caching of the advertisement, wherein the cached advertisement is located near the location where the wearer is associated with the advertisement , It can make it possible to deliver the advertisement. For example, local ads may be stored on a server that includes local advertisements and specials according to geographic location, and such advertisements may be stored on a server, such as a wearer Can be delivered individually to the wearer when approaching a particular place, or a set of ads can be delivered to the wearer in bulk when the wearer enters a geographic area associated with the advertisement. A geographic location may be a city, a portion of a city, a plurality of blocks, a single block, a street, a portion of a street, an Indian, etc., representing a region of a province, a region, a hyper-local. It should be noted that although the discussion above uses the term advertising, those of skill in the art will also appreciate that this may also be used in announcements, broadcasts, circles, commercials, sponsored communications, , Promotions, bulletins, messages, and so on.

18 to 20A illustrate a method of delivering a personalized message to people within a short distance from a facility to which a message such as a retail store is to be transmitted. Referring now to FIG. 18, embodiments may provide a way of viewing a custom billboard, for example, when the wearer of the eyepiece is walking or driving, by the aforementioned application programs searching for a provider of goods and services have. As shown in FIG. 18, billboard 1800 shows an exemplary augmented reality-based advertisement displayed by a seller or service provider. An exemplary advertisement may be about a proposal about drinking with a bar, as shown. For example, two mainstreams can be provided with only one mainstream value. With such augmented reality-based advertisements and suggestions, the wearer's attention can easily go toward the billboard. The billboard can also provide details about the location of the bar, such as street address, floor number, telephone number, and the like. According to other embodiments, several devices other than eyepieces may be used to view the billboard. These devices may include smartphones, iPhones, iPads, windshields, user glasses, helmets, wrist watches, headphones, vehicle mounts, and the like. According to one embodiment, a user (a wearer when the augmented reality technology is embedded in the eyepiece) can automatically receive a suggestion or view a scene of the billboard as the user passes by or drives the road. According to another embodiment, the user may receive a suggestion or view the scene of the billboard based on his request.

19 shows two exemplary roadside billboards 1900 that include suggestions and advertisements from a seller or service provider viewed in an augmented reality manner. The augmented advertisement can provide a live or near-realistic perception to the user or wearer.

As illustrated in FIG. 20, an augmented reality supporting device such as a camera lens provided in an eyepiece can be displayed on a road side or in a graffiti (2000), a slogan, a picture Etc. < / RTI > The roadside billboard and graffiti may have a visual (e.g., code, shape) or wireless indicator that can link the ad or the ad database to the billboard. When the wearer approaches the billboard and views the billboard, projection of the billboard advertisement can then be provided to the wearer. In embodiments, to enable the wearer to block at least some of the advertisements so that the advertiser may better match the needs of the wearer, the wearer may be provided with personal profile information It can also be stored. In embodiments, the eyepiece may have brightness and contrast controls for the eyepiece-projected area of the billboard to improve the readability of the advertisement, e.g., in a bright external environment.

In other embodiments, a user may post information or a message at a specific location based on his or her GPS location or other location indication (e.g., a magnetometer reading). 20A, an intended observer can view a message when the observer is within a certain distance from its position. In the first step 2001 of the method of Figure 20A, the user determines where the message is to be received by the person to whom the message is sent. The message is then posted (2003) and sent to the person or people when the recipient is near the intended "viewing area ". The position of the wearer of the augmented reality eyepiece is continually updated by the GPS system forming part of the eyepiece (2005). When the GPS system determines that the wearer is within a certain distance (e.g., 10 meters) from the viewing area desired, the message is then sent to the observer (2007). In one embodiment, a message may appear in the eyepiece if the message subsequently appears to the recipient as an email or text message, or if the recipient wears an eyepiece. In one sense, a message may be displayed on a building or feature at or near a location designated as "graffiti" because the message is sent to a person based on the location of the person. Certain settings may be used to determine whether all messages passing through the "viewing area" can see the message or only those devices that have a particular person or group of people or a particular identifier can view the message. For example, a soldier searching for a village can virtually mark a house that has been searched by associating a message or identifier with the house (e.g., marking the house with a large X character). The soldier can indicate that only other US soldiers can receive location-based content. When other US soldiers pass through the house, these US soldiers may, for example, by viewing a virtual "X" on the side of the house if they have an eyepiece or some other augmented reality-assisted device, or by indicating that the house has been searched By receiving the message, the display can be automatically received. In another example, content related to safety applications such as alerts, target identification, communication, etc. may be streamed to the eyepiece.

Embodiments may provide a way of viewing information associated with a product, e.g., at a store. The information may include nutritional information on foodstuffs, instruction manuals for apparel products, technical specifications for home appliances, e-coupon promotions, comparison with other similar products, price comparison with other stores, etc. have. This information can be projected onto the guitar, in relation to the product, around the perimeter of the wearer, in relation to the shop layout. The product can be visually identified through SKU, brand tag, etc.; For example, via an RFID tag on the product, by a product package; For example, based on the location of the wearer in the store in relative position relative to the product, may be carried by the store, and so on.

For example, the observer may be walking around in the clothing store, and when walking, information about the garment hanging on the shelf is provided, where the information is provided through the RFID tag of the product. In embodiments, this information may be conveyed as a list of information, as a graphical representation, as an audio and / or video presentation, or otherwise. In another example, the wearer may be in grocery shopping, the ad serving facility may be providing information to the wearer in relation to the product in proximity to the wearer, and the wearer may pick up the product to view the brand, product name, SKU, Information can be provided. In this way, the wearer can be provided with an additional information environment to effectively shop.

One embodiment may allow a user to receive or share information about a shopping or urban area through the use of an augmented reality supporting device, such as a camera lens, installed in an eyepiece of an exemplary sunglass. These embodiments will use augmented reality (AR) software applications such as those mentioned above in connection with searching for providers of goods and services. In one scenario, the wearer of the eyepiece may be walking the street or the market for shopping. In addition, the user can activate various modes that can help define user preferences for a particular scenario or environment. For example, a user may enter a navigation mode where the wearer may be guided in the street and market to shop for their favorite accessories and products. Mode may be selected and various indications may be given by the wearer via various means (e.g., via text, voice, etc.). In one embodiment, the wearer may provide a voice command to select the navigation mode, so that an augmented display may appear in front of the wearer. The augmented information may indicate information about the location of various shops and sellers in the market, suggestions at various stores and by various sellers, current discount time zones, current date and time, and the like. Various types of options can also be displayed to the wearer. The wearer can scroll through the options and walk the guided streets through the navigation mode. Based on the options offered, the wearer can select the best fit for the shopping based on suggestions, discounts and the like. In embodiments, the eyepiece may provide the ability to search, browse, select, store, share, and receive advertisements for purchased items, such as those viewed through an eyepiece. For example, a wearer may search for goods through the Internet and make purchases without telephone calls, for example, through an application store, a commerce application, or the like.

The wearer can provide a voice command to move to that location, and the wearer can then be guided over there. The wearer can also receive advertisements and offers automatically or based on promotions and events at a place of interest, such as a request for a current transaction, a nearby shopping store, or the like. The advertisements, transactions and offers may appear in close proximity to the wearer, and options for purchasing desired products based on advertisements, transactions and offers may be displayed. The wearer can, for example, select the product and purchase him through Google checkout. Similar to that shown in Fig. 7, a message or email with information that the transaction for purchase of the product has been completed may appear in the eyepiece. Product delivery status / information can also be displayed. The wearer can also communicate with friends and relatives about the proposal and the event via the social networking platform, and can also tell them to join and join them.

In embodiments, the user may wear a head-mounted eyepiece, where the eyepiece includes an optical assembly through which the user can view the surrounding environment and the displayed content. The displayed content may include one or more local ads. The position of the eyepiece can be determined by the integrated position sensor, and the local advertisement can be related to the position of the eyepiece. By way of example, the location of the user can be determined via GPS, RFID, manual input, and the like. In addition, the user may be walking next to the coffee shop, and based on the proximity of the user to the coffee shop, it is possible to display a brand 1900 of a store, such as a fast food restaurant or a coffee brand, An advertisement similar to that of the user may appear in the user's field of view. The user can experience similar types of local advertising when traveling around the surrounding environment.

In other embodiments, the eyepiece may include a capacitive sensor that can sense whether the eyepiece is in contact with the human skin. The sensor may be a capacitive sensor, a resistive sensor, an inductive sensor, an electromagnetic sensor, or the like. These sensors or a group of sensors can be placed in the eyepiece and / or eyepiece arm in such a way that they can detect when the user is wearing glasses. In other embodiments, sensors may be used to determine if the eyepiece is in a state ready to be worn by a user, for example, when the earpiece is unbent. In addition, local advertisements can be transmitted only when the eyepiece is in contact with the human skin, when in the wearable position, when they are in combination, when the user is actually wearing the eyepiece. In other embodiments, local advertisements may be transmitted in response to powering on the eyepiece or in response to powering on the eyepiece and wearing the eyepiece. By way of example, an advertiser may send a local ad only when the user is close to a particular facility and only when the user is actually wearing the glasses and the glasses are powered on and the advertiser is able to target the ad to the user at an appropriate time You can choose.

According to other embodiments, the local advertisement may be displayed to the user as a banner advertisement, a two-dimensional graphic, text, or the like. In addition, local advertisements may be associated with the physical aspects of the user view of the surrounding environment. The local advertisement may also be displayed as an augmented reality advertisement, where the advertisement is associated with the physical aspects of the surrounding environment. Such advertisements may be two-dimensional or three-dimensional. By way of example, and as further described in FIG. 18, a local advertisement may be associated with a physical billboard, wherein the displayed content, which is indicative of following a drink from an actual billboard 1800 in the surrounding environment, Can attract attention. The local advertisement may also include sound provided to the user via earpiece, audio device, or other means. In addition, in embodiments, the local advertisement may be animated. For example, the user may see the beverage flow from the billboard onto the adjacent building and optionally into the surrounding environment. Similarly, an ad may display any other type of movement, as desired in the ad. In addition, a local advertisement may be displayed as a three-dimensional object that is associated with or interacts with the surrounding environment. In embodiments in which the advertisement is associated with an object in the user view of the surrounding environment, the advertisement may be associated with or remain in proximity to the object, even when the user turns his head. For example, as shown in FIG. 19, when an advertisement such as a coffee mug is associated with a particular building, even when the user turns his head to look at another object in his environment, And can remain in place on the building.

In other embodiments, a local advertisement may be displayed to a user based on a web search performed by the user, where the advertisement may be displayed in the content of the web search results. For example, a user may search for "discounted time zones" while walking along a street, and local ads that advertise beer prices in a local bar may be displayed in the content of search results.

In addition, the content of the local advertisement can be determined based on the user's personal information. The user's information can be made available to a web application, an advertisement facility, and the like. In addition, the eyepiece of the web application, advertising facility or user may filter the advertisement based on the user ' s personal information. In general, for example, a user may store personal information about what he likes and dislikes, and this information may be used to send the advertisement to the user's eyepiece. As a specific example, a user may store data regarding his or her affinity for a local sports team, and when the ad becomes available, the ad for his favorite sports team may be given priority and pushed to the user . Similarly, what a user dislikes to exclude a particular ad from view can be used. In various embodiments, advertisements may be cached on the server, where the advertisements may be accessed by at least one of an advertising facility, a web application, and an eyepiece and displayed to a user.

In various embodiments, a user may interact with any type of local advertisement in a number of ways. The user request may request additional information related to the local advertisement by performing at least one of eye movement, body movement and other gestures. For example, when an advertisement is displayed to a user, the user may shake his or her hands on the advertisement in his view, or may move his eyes on the advertisement to receive additional information about the advertisement by selecting a particular advertisement . Moreover, a user may choose to ignore the advertisement by any motion or control technique described herein (e.g., through eye movements, body movements, other gestures, etc.). In addition, the user may decide to ignore the advertisement by allowing the advertisement to be essentially ignored by not selecting the advertisement for further interaction within a given period of time. For example, if a user decides not to make a gesture to get additional information from an advertisement within five seconds of the ad being displayed, the ad may be ignored by default and may disappear from the user view. In addition, the user can select such that the local advertisement can not be displayed by the user turning off this feature by selecting this option on the graphical user interface, or through control on the eyepiece.

In other embodiments, the eyepiece may comprise an audio device. Accordingly, the displayed content may include local advertisements and audio, so that the user may also hear messages or other sound effects associated with the local advertisement. By way of example, referring back to Figure 18, while the user is watching that it is following a beer, the user will in fact be able to hear an audio transmission corresponding to the action in the advertisement. In this case, the user can hear the bottle open and then hear the liquid pouring from the bottle onto the roof. In still other embodiments, a description message may be played, general information may be provided as part of the advertisement, or both. In embodiments, any audio may be played as desired for the advertisement.

According to another embodiment, social networking can be facilitated by using an augmented reality supporting device such as a camera lens installed in an eyepiece. This can be used to connect a number of users or others who may not have an augmented reality support device that may share ideas and ideas with one another. For example, a wearer of an eyepiece may be sitting on a school campus with other students. The wearer may be connected to the first student who may be in the coffee shop and send a message to him. The wearer may ask the first student about people who are interested in a particular subject, such as, for example, environmental economics. As other students pass through the wearer's vision, the camera lens installed inside the eyepiece can be compared to a networking database such as 'Google me' that can track students and include a public profile. A profile of interested and relevant persons in the public database may appear in front of the wearer on the eyepiece and pop up. Some of the profiles that may not be relevant may seem to be blocked or blocked by the user. Relevant profiles can be highlighted for quick reference by the wearer. A relevant profile selected by the wearer may be of interest to the subject of environmental boundaries, and the wearer may also be associated with the subject. In addition, the subject may also be associated with the first student. In this way, the social network can be set by the wearer using an eyepiece having the features of an augmented reality. A social network managed by the wearer and a conversation therein can be stored for future reference.

The present disclosure can be applied to real estate scenarios using an augmented reality supporting device such as a camera lens installed in an eyepiece. According to this embodiment, the wearer may wish to obtain information about a place where the user may be at a particular time (e.g., during driving, walking, jogging, etc.). The wearer may want to know, for example, the residential advantages and losses at the place. The wearer may also wish to obtain detailed information about the facilities in the place. Therefore, the wearer can use a map such as a Google online map and find out the real estate that can be listed there for rental or purchase. As previously noted, a user may receive information about a property for sale or lease using a mobile internet application such as Layar. In one such application, information about the building in the user's field of view is projected onto the interior of the glasses for the user to look at. Options for scrolling can be displayed to the wearer on an eyepiece lens, such as in a trackpad mounted on a frame of glasses. The wearer may select and receive information regarding the selected option. The augmented reality support scene of the selected options can be displayed to the wearer and the wearer can view the pictures and view the facility in a virtual environment. The wearer may also receive information about real estate agents and make an appointment with one of them. E-mail notifications or telephone notifications for confirmation of appointments may also be received via the eyepiece. If the wearer realizes that the selected property is worthwhile, the transaction can be made and the wearer can purchase him.

According to another embodiment, the use of an augmented reality supporting device such as a camera lens installed in an eyepiece can improve personalized and sponsored viewing and traveling. For example, a wearer (as a traveler) may arrive in a city such as Paris and receive travel and tourism related information about the place and thereby plan his visit for several consecutive days during his stay. The wearer may wear his eyepiece or operate any other augmented reality assisted device and provide voice or text instructions on his request. Augmented Reality Support The eyepiece can detect the wearer's position and determine the wearer's travel preferences through geo-sensing technology. The eyepiece can receive custom information based on the wearer ' s request and display it on the screen. Customized travel information includes information on art galleries and museums, monuments and sites, shopping complexes, entertainment and nightlife, restaurants and pubs, most popular travel destinations and travel centers / attractions, and most popular local / cultural / local destinations and attractions . ≪ / RTI > Based on user selection of one or more of these categories, the eyepiece may ask the user other questions such as stay time, investment in travel, and the like. The wearer can respond through voice commands and receive customized travel information in the order selected by the wearer. For example, a wearer may prioritize an art gallery rather than a monument. Accordingly, the information can be made available to the wearer. In addition, a map may also appear in front of the wearer with different sets of travel options and with different priorities as follows.

Priority 1: First trip option (Champs-Elysees, Louvre, Rodin, museum, famous café)

Priority 2: Second option

Priority 3: Third option

The wearer can, for example, select the first option because the first option has the highest priority based on the wearer's display preferences. Advertisements related to sponsors can be popped up immediately after selection. Then, a virtual trip can be started in an augmented reality mode that is very close to the actual environment. The wearer may, for example, take a 30-second trip for a special vacation to the Atlantis resort in the Bahamas. Virtual 3D travel can include viewing rooms, beaches, public places, parks, and facilities at a glance. The wearer can also experience shopping facilities in the area and receive suggestions and discounts at the places and shops. After all, the most important thing is that the wearer may be able to sit in his room or hotel and experience a whole day tour. Finally, the wearer can make a decision and schedule his schedule accordingly.

Other embodiments can provide information on automotive repair and maintenance services through the use of an augmented reality support device, such as a camera lens, installed on an eyepiece. The wearer may receive advertisements for auto repair shops and dealers by sending a voice command for the request. The request may include, for example, a request for oil change in the vehicle / motor vehicle. The eyepiece can receive information from the repair shop and display it to the wearer. The eyepiece can invoke the 3D model of the wearer's vehicle and show the amount of oil remaining in the car through the augmented reality support screen / view. The eyepiece may again show other relevant information about the wearer's vehicle (e.g., maintenance requirements on other parts such as a brake pad). The wearer can see a 3D view of the wear brake pad and may be interested in repairing or replacing it. Accordingly, the wearer can make an appointment with the seller to solve the problem using the integrated wireless communication function of the eyepiece. Confirmation may be received via e-mail or an incoming call alert on the eyepiece camera lens.

According to another embodiment, the use of an augmented reality supporting device such as a camera lens installed in an eyepiece can benefit a gift shopping. The wearer can post a request for a gift for any celebration via a text or voice command. The eyepiece may ask the wearer to answer his preferences, such as the type of gift, the age range of the person to receive the gift, the price range of the gift, and the like. Various options may be presented to the user based on the received preferences. For example, options presented to a wearer may be a cookie basket, a wine and cheese basket, a chocolate bar, a golfer's gift basket, and the like.

Available options can be scrolled by the wearer, and the most suitable option can be selected via a voice command or a text command. For example, the wearer may select a golfer's gift basket. A 3D view of a golfer's gift basket can appear along the golf course in front of the wearer. A virtual 3D view of golfers' gift baskets and golf courses made possible by Augmented Reality can be perceived as being very similar to the real world environment. The wearer may finally respond to the address, location and other similar queries prompted by the eyepiece. The confirmation may then be received via an e-mail or an incoming call alert on the eyepiece camera lens.

In various embodiments, the eyeglass platform can be used by various control mechanisms, receive physical and useful inputs, perform processing functions, control on-board and external features and systems (including those based on feedback loops ) Content and execute electronic commerce. While such e-commerce and content scenarios are enormous, some of these scenarios include, but are not limited to, retail shopping environments, educational environments, traffic environments, home environments, event environments, dining / drinking environments, and outdoor environments. While these areas are described herein, various other scenarios will be apparent to those skilled in the art.

In embodiments, a glasses platform may be used in a retail shopping environment. For example, the user may use the glasses to receive content related to the item of interest and / or the environment. The user can receive and / or search for pricing information or alternative proposals, product information such as SKU / barcode, evaluation, advertisement, GroupOn proposal, etc. in a retail shopping environment. In embodiments, a user may search for or obtain location information for a particular article. The user may also obtain information about loyalty program information related to a particular brand, article, and / or shopping environment. In addition, the user can use glasses with cameras, scanners, QR readers, etc. to scan the items and place them in a shopping basket. In addition, the user can use the eyepiece to detect the best item among a group of items. By way of example, a user can activate a feature of a spectacle to visualize the article in a particular way (by a program that determines or senses the density or thickness of the article) to find the best of many. In embodiments, the user may use the glasses to negotiate the price of the article or to speak of his preferred price. For example, after virtually scanning an item with a scanner or the like associated with the glasses, the user may use a gesture, move his eyes, or use a voice command, etc., to say the price to pay for the item . The user can also use the glasses to scan the item and then command the user to pay through the displayed or suggested payment method through the user interface. Such payments may be indicated through hand gestures, eye movements, etc., as described herein. Similarly, a user may pay for a "point" or reward during a shopping trip, for example, receive promotions related to a particular item and / or facility via a GroupOn, have. In addition, the user can use the image recognition by the glasses to recognize the article on one interface and make an order for the article. For example, a program used in glasses may allow a user to use the glasses to recognize the wristwatch in front of the store, thereby triggering an order entry menu for the item in the interface when the article is recognized. In additional embodiments, information can be entered into the glasses platform by scanning bar codes, QR codes, product labels, and the like. Promotional information such as processes, signage, advertisements, coupons, etc., may be scanned or otherwise received or recognized by the glasses when the user is passing through a retail environment or engaged in a retail interface with the glasses have. The user may scan the loyalty card with glasses for use in a transaction or otherwise input such information for use during retail transactions. In embodiments, the glasses can aid navigation and guidance. For example, a user may be presented with a specific map of the store, and may be provided with an aisle content sign that allows the user to better navigate the items and navigate around the retail environment . The user can capture the product image from the real environment or download the product image, and thus, to make a memo on the article, to make an article, to make an evaluation of the article, For that, the image can be used. In addition, the geographic location application of the object image and the glasses can allow the user to receive the nearest location of the item, the local people's review of the item, and so on. In embodiments, the geographic location of the user may allow a particular object image to be generated or more appropriately recognized.

As a more specific example, the system may include an interactive head-mounted eyepiece worn by a user, wherein the eyepiece includes a module for determining whether the eyepiece is in close proximity to the retail environment, , A 3D processing module that recognizes the features of the environment and renders a 3D display of advertising content targeted to the retail location of an eyepiece on an eyepiece eyepiece, and an optical assembly for capturing and processing images of the wearer's environment of the head mounted eyepiece An image processing module, and an integrated image light source for introducing the content into the optical assembly, wherein the integrated image light source renders the 3D display as an overlay on the environment, wherein the integrated image light source is a 3D Present the display. In embodiments, the 3D processing module may lock the display element on the recognized feature of the environment and the content may be presented in relation to the recognized feature in the display. In embodiments, rendering of the 3D display of an advertisement may be a result of scanning at least one of a bar code, QR code, product label, or the like. This may also include purchasing the product, entering images of the product in the eyepiece, entering the retail environment (or moving into a retail environment), fixing the user's two eyes to the product, May be the result of entering the loyalty program information. In embodiments, a second 3D display may be rendered as a result of scanning at least one of the product, purchasing the product, entering the retail environment, or the like. In embodiments, a user may execute an electronic commerce via an eyepiece, which involves scanning an item for purchase, selecting an item based on a comparison with other items, negotiating a price, Paying with loyalty points, redeeming promotions, ordering items, and the like. In embodiments, the advertisement may include the location of the article in proximity to the user. The position of the article can be viewed in relation to the location of the user, and the user can be provided with a direction toward the article. In embodiments, an eyepiece may be used for social networking, and the eyepiece may use another person's face recognition in social networking. In addition, an eyepiece can be used to recognize that a person is present in the environment and to present social networking content related to the relationship between the wearer and the perceived person. In addition, the user may send and / or receive a friend request by gesturing as part of his body. In embodiments, the user may compare the prices of the items through the advertisement. In embodiments, the advertisement may include audio content. In embodiments, recognizing a feature may include automated processing of the image including the feature, pinging the feature to a signal, communicating with the feature, recognizing the feature by processing the position of the feature, Retrieving information about the feature from the database, customizing the feature, and / or the like. In addition, the user can specify a feature to retain the overlay content by interacting with the user interface of the eyepiece. In embodiments, the overlay may present the content on or near the recognized feature, and in further embodiments, the recognized feature may include an item for purchase, an item to sell, a sign, an advertisement, A place at a store, a kiosk, a service counter, a cash register, a television, a screen, a shipping cart, and the like.

In embodiments, glasses may be used in an educational environment. By way of example, the glasses may display e-learning content such as textbooks or elsewhere. The glasses can allow the user to view, select, and review items for examination. In embodiments, the user may be monitored while viewing the test. The glasses can measure the time of the user navigating through the data and can track the user's response to adjust the test as needed based on the user's answer and / or continue the test. In further embodiments, the user may view an augmented reality (AR) overlay via glasses. In embodiments, the AR overlay may include step-by-step guidance in an exercise, in a lecture, and the like. In embodiments, a virtual professor may be displayed to enable interaction via video, audio, and chat. The user can view the black board / whiteboard memo through the glasses, and the user can use the blackboard / whiteboard in the user interface so that the AR memo can be added and / or overlaid when the user views a particular blackboard / When viewing or viewing the actual board, the user can enter additional items on the board that can be shared with other users. In embodiments, the eyeglasses can provide a social networking platform to members of the class or education section, provide social networking content to the class members, and provide social networking content about the class members.

In embodiments, the glasses may be used for commerce in an educational environment. As an example, a user may use the glasses to purchase a program or to track content progress and course credits in other ways. In addition, the user can monitor test and quiz scores, and the dates of upcoming tests and quizzes, and the user can download the course credits / degrees information, and the user will be able to download the course credits / degree information listed in the syllabus , Or others, which can be added to a timetable, and the user can meet a friend or class member by communicating with another person through glasses or by communicating with another person. In embodiments, the user may view his bill and tuition bill for review and tracking. In embodiments, the user may purchase the program to do so, or the program may use the program if it provides advertising in connection therewith.

In further embodiments, the user may use the glasses in an educational environment. The user can scan the test / quiz paper through the glasses for viewing, manipulation or otherwise. Users can scan or otherwise capture data associated with textbook content, manuals and / or worksheets, and blackboard / whiteboard content for notes and homework tracking. The user can scan or capture data related to the poster / signage. As a result, the user can track upcoming student meetings, inventory building descriptions, meeting locations, and the like. In embodiments, the user may capture faces such as classmates, friends, interests, and the like. In embodiments, the glasses may track a user's eye movement to verify interaction with the content. In embodiments, the glasses may enable "Lifestride" or other pen functionality, for example, to take in content. The user can display the note as a gesture, and by moving the pen in communication with the glasses, the glasses can store the user's memo. In other embodiments, the user can make a gesture and the glasses can record a note based on this gesture; in other embodiments, another sensor associated with the user's hand may be able to record a note Can be recorded by the glasses.

In embodiments, the system may include an interactive head mounted eyepiece worn by a user, wherein the eyepiece includes a module for determining whether the eyepiece is close to the educational environment. In addition, the system includes an optical assembly that allows a user to view the surroundings through the user, a processing module that recognizes the characteristics of the environment and renders educational contents related to the environment, and an image capture and processing device of the wearer's environment of the head- An image processing module capable of locking the display element on a recognized feature of the environment, and an integrated image light source for introducing the content into the optical assembly, Rendering the content as an overlay on the environment, where the content may be presented in relation to a feature recognized in the display. In embodiments, the integrated image light source may present a display of traffic content that is related to the traffic environment, and this relationship with the recognized feature may not exist. In embodiments, the rendering of the training content may be a result of scanning bar codes, QR codes, and the like. This may be the result of entering a textbook image in the eyepiece, entering the image of the distribution in the eyepiece, recognizing the marker in the environment, and entering the educational environment. In embodiments, the educational environment may be a classroom, a practice studio, a car garage, an outdoor environment, an auditorium, a laboratory, a factory, a business location, a kitchen, a hospital, In addition, educational content includes text, textbook excerpts, user manuals, video, audio, lab plans, chemical structures, 3D visuals, 3D overlays, classroom worksheets, exams, recipes, lecture notes, Safety instructions, and exercise routines. In embodiments, the training content may be associated with or overlaid on an object in the environment. In embodiments, the object may be a whiteboard, a blackboard, a machine, a car, an airplane, a patient, a textbook, a projector, and the like. In embodiments, the system can be used for social networking, and can also use face recognition of at least one of the classmates, teachers, etc. in the environment. In embodiments, the user may send and / or receive a friend request by gesturing as part of the user's body. In embodiments, the user can use a variety of methods to view the test, to complete the homework, to view the lecture notes, to view the lecture plan, to perform the technique, to track the progress of the course, You can interact with the content for notes, for writing notes, for submitting questions, for other purposes. In embodiments, the overlay may present the content on or near the recognized feature. In addition, the recognized features may be at least one of a poster, a blackboard, a whiteboard, a screen, a machine, a car, an airplane, a patient, a textbook, a projector, a monitor, a desk, a smart board, As an example, a memo with a frame blackboard may appear on the display; A movie may appear at the point where the screen is on the display, a molecular display may appear on the blackboard, and so on. In embodiments, recognizing a feature may include automated processing of the image including the feature, pinging the feature with a signal, communicating with the feature, recognizing the feature by processing the position of the feature, Retrieving information about the feature section by processing the feature information, retrieving information about the feature section from the database, and specifying the feature section. In addition, the user can specify a feature to retain the overlay content by interacting with the user interface of the eyepiece.

In embodiments, glasses may be used in a traffic environment. As an example, a user may search for or capture content such as schedules, availability, delays and cancellations related to traffic. For example, when a user arrives at an airport, the user can view his flight information through the glasses and see if his flight is delayed as scheduled. In embodiments, the user can view his / her seat / grade selection and select snacks and meal preferences. The user can check through the glasses and, in embodiments, switch or update his seat selection through the glasses. In embodiments, the user pilot may be provided with a step-by-step procedure for the pre-flight checklist for the FAA requirements. In embodiments, train crews, pilots, and the like may be provided with guidance and navigational instructions for operating trains, airplanes, and the like. In other embodiments, the user passenger can review the safety information through the eyeglasses (e.g., the user can view the pre-flight safety instruction manual showing how to operate the life-saving equipment, etc.). In embodiments, the user may use the glasses to reserve additional items such as rental cars, hotels, and the like. In embodiments, a user may schedule an in person tour and / or make a virtual tour of the area of interest through the glasses. The user can see the surrounding environment of the destination to be familiar with the area before arrival. In other embodiments, the user can also examine various items and compare the processes. A user may view and / or receive loyalty content such as available reserves, available points and which items are available for a particular account. Users can pay for loyalty points to book flights, rent-a-car, hotels, etc. through glasses. In embodiments, the user may use the glasses for networking in a travel or traffic environment. For example, a user can find out who is on his particular flight or train. The user can also use the glasses to view entertainment content during boarding. As an example, an inflight movie may be transmitted to the user's glasses. In embodiments, the user can view content associated with various locations, view AR landmarks, and the like. By way of example, when a train or windscreen crosses the landscape, the user may see AR overlays of items of interest, such as landmarks, associated with a particular area. In embodiments, the user may receive ad content as it passes through the billboard / signage during boarding. In addition, the user can receive employee information related to the traffic attendant involved in his boarding. As an example, the user may receive information related to the driver of the taxi to view his driver ' s records, or the user may have a pilot's accident < RTI ID = 0.0 > and / can see.

In addition, the user can use the glasses in a traffic environment related to commerce. For example, a user can use glasses for guitars to set up a meal plan during boarding and make a payment for it, to make a reservation for a seat, to pay as a reserve point for him. Users can find, book / pay for flights, rent cars, book hotels, taxis, buses, and so on. The user can form a network with people related to his trip, such as other passengers. In embodiments, the user may navigate the path using glasses. For example, a user may be provided with a bus and taxi map showing the best route and method of exploring the city. The user can pay for the application for it and / or view the advertisement associated with the application thereon. The user can interact with the AR content in and around the landmark during the trip, and interact with advertisements and promotions from AR-based billboards, signs and the like.

In embodiments, the user may enter items into the spectacle platform in a traffic environment. For example, the user may scan his ticket to start the check-in process using the glasses. The user may be provided with a dashboard displaying the speed, fuel and GPS position during his boarding. The glasses can communicate with the vehicle's IT system via Bluetooth to display the instrument panel and to provide information about the vehicle and / or traffic mode. In embodiments, the user may use the glasses to recognize the face of another passenger by entering the image into the glasses and / or to store the image thereof. The user may enter the landmark related content into the glasses to create his database for interaction or for later calling. In embodiments, a user may enter a billboard / signboard, which may or may not be AR based, for its storage and interaction therewith.

In addition, the system may include an interactive head-mounted eyepiece worn by the user, wherein the eyepiece comprises a module for determining whether the eyepiece is close to the traffic environment, an optical assembly for viewing the surroundings through which the user views the feature, A video processing module for capturing and processing the image of the environment of the wearer of the head mounted eyepiece, an image processing module for recognizing the display element as a recognized feature of the environment And an integrated image light source for introducing the content into the optical assembly, wherein the integrated image light source renders the traffic content as an overlay on the environment, wherein the content is associated with a recognized feature in the display . In embodiments, the integrated image light source may present a display of traffic content that is related to the traffic environment, and this relationship with the recognized feature may not exist. In embodiments, the rendering of traffic content may include scanning a bar code, QR code, tickets, etc., inputting images of traffic tickets, trains, train stations, taxi stands, taxis, airports, boats, stations, ≪ / RTI > In embodiments, the traffic content may be text, video, audio, 3D images, 3D overlays, text overlays, directions, schedules, maps, navigation, Checklist, FAA information, flight information, arrival and departure time information, travel itinerary, and the like. In embodiments, the auxiliary resources may be used to make hotel reservations, to make car rental reservations, to make meal reservations, to note personal preferences, to change seating selections, to find local entertainment, , And resources for others. In embodiments, a user may use an eyepiece to purchase a boarding pass for flights, boarding, or boarding a train, and a user may purchase a face value for a subway boarding pass, reserve a taxi, view a traffic timetable, You can compare prices, search for directions, search for traffic routes, view your current location on a map, see an efficient route to traffic patterns, and more. In embodiments, the content may be associated with the vehicle to display information about the vehicle, wherein the information includes emergency exit information, maintenance information, operational information, instrument panel information, model information, and the like. The system can be used for social networking, which can take advantage of face recognition by travelers, operators, etc. in the environment. A user may send and / or receive a friend request by gesturing as part of his body. In embodiments, the eyepiece can be used to recognize that a person is present in the environment and to present social networking content related to the relationship between the wearer and the perceived person. In embodiments, the user may interact with the displayed advertisement to obtain additional information. In embodiments, the content may be augmented (including any of a visual user guide, audio usage guide, visual marker, overlay path planning due to various reasons (including escape of the environment in an emergency) environment (content can enhance the environment). In embodiments, the overlay may present the content on or near the recognized feature. In addition, the recognized feature can be at least one of a poster, a train, an airplane, a taxi, a ship, a subway train, a screen, a kiosk, a map, a window and a wall. In embodiments, recognizing a feature may include automated processing of the image including the feature, pinging the feature with a signal, communicating with the feature, recognizing the feature by processing the position of the feature, Retrieving information about the part, customizing the feature, and / or the like. In addition, the user can specify a feature to retain the overlay content by interacting with the user interface of the eyepiece.

In embodiments, a glasses platform may be used in a home environment. In embodiments, glasses may be used for the content. For example, glasses can be used for entertainment, where the user is viewing the media at home. Glasses can also be used for shipment, for example, to generate a grocery list, etc., and for necessary stored stock items and their inspection. The user may use the glasses for family collaboration, such as paying bills through glasses, creating a checklist of tasks to do for the family, and the like. For example, glasses can be used for doctor's appointments and keeping, upcoming soccer games, and so on. Glasses can be used for procedural guidance. By way of example, the glasses may be used to guide the user in operating appliances such as DVD players, VCRs, remote controllers, and the like. In addition, glasses can be used for security and / or safety. The user can activate the alarm system to keep the alarm system on while he is at home or out of the house. The user can view the home camera while on the go, turn on and off the lights in the house, and play guitar. The user can be provided with an instruction manual for the emergency (for example, the user can be provided with instructions on what to do during a fire, a hurricane, etc.). The user may turn on the features of the glasses as described herein to view smoke or the like. The user can use the glasses to track family members and communicate with them during such an emergency. The glasses can provide lifesaving CPR guidance, 911 calls, and so on.

In embodiments, the glasses may be used for commerce in a home environment. For example, a user can use glasses for guitars to order food delivery, to order dry cleaning, or to order a dry cleaning pickup. Users can order entertainment content such as movies, video games, and the like. In embodiments, the user can find and use information about housewares, pay bills, and so on. A user can view advertisements and / or promotions while at home and act accordingly. For example, if a user is using a blender in a kitchen and the ad is displayed in glasses, the ad will prompt the user to look for more about the new blender and the user will be prompted to find out more about the device Can be selected.

In embodiments, the user may use the glasses in a home environment so that the user enters information into the glasses. As an example, a user may enter document tasks for storage, calling, interaction, and the like. The user can enter shopping lists, invoices, checklists, manuals, mail, and the like. The user can enter advertisements from supported paper mail advertisements for AR, TV, radio, and the like. The user may scan the paper advertisement to view or receive additional AR information associated with the advertisement. The user may enter embedded symbols and / or IDs, for example, to identify household appliances or other hardware. Users can enter Wi-Fi network content into their glasses. In addition, the user can input television content such as screen and smart TV content. Thereby, the user can interact with this content through the eyeglass platform. The user can input a remote control command to the glasses platform so that the user can operate various devices such as a TV, a VCR, a DVD player, a household appliance, and the like. In addition, the user can enter the security system content so that the user can interact with and control the security system, its associated camera, and the like. The user can view various camera feeds associated with the security system so that the user can view various areas around the home environment through the glasses platform. Glasses can be connected to these cameras via Bluetooth, Internet, Wi-Fi connection, and so on. The user can also set alerts associated with the security system, turn off alerts, view alerts, and interact with alerts.

In addition, the system may include an interactive head mounted eyepiece worn by a user, wherein the eyepiece comprises a module for determining whether the eyepiece is close to the home environment, an optical assembly for viewing the surrounding home environment therethrough, Image processing module for capturing and processing the image of the wearer's environment of the head-mounted eyepiece. The image processing module is a module for recognizing the environment of the environment, , And an integrated image light source for introducing the content into the optical assembly, wherein the integrated image light source can render the home related content as an overlay in an environment, wherein the content is characterized by features Can be presented in relation to wealth. In embodiments, the integrated image light source may present a display of content that is related to the environment, and this relationship with recognized features and content may not exist. In embodiments, the rendering of the content may include entering the house, fixing the user ' s eyes to an item at home, recognizing markers in the environment with an eyepiece, manipulating other devices in the home, Can be the result. In an embodiment, the content may include a user interface for manipulating devices such as VCRs, DVDs, satellite receivers, set-top boxes, video-on-demand devices, audio equipment, video game consoles, alarm systems, home computers, heating and cooling systems, . In embodiments, a user may interact with the user interface through eye movement, hand gestures, head nudity, and the like. In embodiments, the content may be provided by a user, such as a user generating a shopping list, reviewing a grocery inventory, paying an invoice, viewing an invoice, activating a device, manipulating lighting, And / or completing tasks such as generating virtual communication to other people, ordering delivery such as dry cleaning, food, etc., acting in accordance with advertising in the environment, and the like. In embodiments, a user may access a home environment through an eyepiece or recognize the face of another person at home. In embodiments, the content may include an instruction manual in an emergency, and the instruction manual may be at least one of audio, video, video instruction manual, and the like. In embodiments, the content may be an augmented environment, or may enhance the environment and include any of the following: a visual user guide, an audio usage guide, a visual marker, an overlay path plan for escaping the environment in an emergency, . In embodiments, content may be generated in response to embedded symbols, television audio and / or video content, advertisements, and the like. In embodiments, the content may be retrieved from a user manual stored in the eyepiece, downloaded from the Internet, or the like. The content may include 3D advertisements, audio, video, text, and the like. In embodiments, recognizing a feature may include automated processing of the image including the feature, pinging the feature with a signal, communicating with the feature, recognizing the feature by processing the position of the feature, Retrieving information about the part, customizing the feature, and / or the like. In embodiments, a user may designate a feature to retain the overlay content by interacting with a user interface of the eyepiece. In embodiments, the overlay may present the content on or near the recognized feature. In addition, the recognized features may be at least one of a home appliance, a memo station, a notepad, a calendar, a wall, an electronic device, a security system, a room, a door, an entrance, a key holder, and a built-in.

In embodiments, the user may use the glasses in an event environment. In various event environments, a user can interact with content using a glasses platform. By way of example, a user may view the availability of a schedule, ticketing information, and / or tickets / seats for events such as concerts, ball games, various entertainment, corporate events, and the like. Users can view promotional information about the event or interact with it in other ways. The user can view loyalty program content such as points or reserve values associated with the event. The user may be provided access to the event due to, or in connection with, a loyalty program. Users may be given the opportunity to view "bonus" material at the event due to their loyalty program. In embodiments, the user can view auxiliary services and merchandise associated with the event, purchase him or the like. In embodiments, a user may view AR content at an event such as a first down line, a goal marker, an interview with a player / player, and the like. In embodiments, the user may view alternate video feeds such as sideline views, back stage views / video feeds, etc. when the user is in another seat in the arena.

In embodiments, the glasses may be used for commerce in an event environment. By way of example, a user can purchase / reserve tickets, view selected / available seats, and so on. The user can reserve ancillary items such as purchasing a back stage pass, upgrading his / her seat, and the like. In embodiments, the user may purchase event related merchandise such as a jersey, a concert shirt, a poster, and the like. The user can also pay at points such as those associated with a reserve or frequent attendee program. In embodiments, a user may purchase images such as digital "signed" video from an event, a particular portion or an entire event or event, and / or view a souvenir, collection. The user can view additional videos of the player and / or performer or commentary during these events at an additional cost or for free.

In embodiments, the user may enter items and / or data in an event environment into the glasses platform. In various embodiments, a user may enter tickets, enter tickets, enter events, or otherwise, to find his seat. The user can enter promotional materials such as posters and signboards, and use AR enhancements to report and / or interact with him. The user can enter loyalty program information and scan the card for a specific event. Accordingly, you may interact with, associate with, provide data to, activate such accounts, and so forth with respect to the Event. In embodiments, a user may enter network content into glasses via Wi-Fi, Bluetooth, and the like.

In addition, the system may include an interactive head-mounted eyepiece worn by the user, wherein the eyepiece comprises a module for determining whether the eyepiece is close to the event environment, an optical assembly for viewing the event environment through which the user views the event, A processing module for rendering the display of the event contents targeted for the environment on the head mounted eyepiece, and an image processing module for capturing and processing the image of the wearer's environment of the head mounted eyepiece. And an integrated image light source for introducing the content into the optical assembly, wherein the integrated image light source renders the event content as an overlay in an environment viewed by the eyewear wearer, Associate with wealth; Here, the integrated image light source presents contents related to the environment. In embodiments, the image processing module may lock the display element on the recognized feature of the environment, and the content may be presented in relation to the recognized feature in the display. In various embodiments, the rendering of the content may include entering an event environment, pinning the user's eyes to an item at an event, recognizing features in the environment, scanning a user's ticket, Recognizing what is happening at the event, entering a video from the event, or the like. In embodiments, the content may include at least one of a first down line, a playground marker line, a player's display, a player's display of equipment, instant playback, enhanced view, live video, alternative views, , And the like, as well as augmented visual feeds. In various embodiments, the content may include an augmented audio feed that includes any of a player commentary, commentary audio, game sound, enhanced playing sound, player description, live audio, and the like. The user can interact with the content through at least one of eye movement, hand gestures, nudging, and the like. In embodiments, the eyepiece can be used to recognize that a person is present at the event and present social networking content related to the relationship between the wearer and the perceived person. In addition, a user can do at least one of sending and / or receiving a friend request by gesturing as part of the user's body, such as nudging. The system may include a user interface for purchasing at least one of an event item, a picture, and an event view, and a digitally signed collection from the event. In addition, content may be generated in response to at least one of embedded symbols, television content, advertisements, and the like. In embodiments, the content may include at least one of a backstage, a locker room, a dugout, a bullpen, a player's bench, or other enhancement video and audio. In embodiments, recognizing a feature may include automated processing of the image including the feature, pinging the feature with a signal, communicating with the feature, recognizing the feature by processing the position of the feature, Retrieving information about the part, customizing the feature, and / or the like. In addition, the user can specify a feature to retain the overlay content by interacting with the user interface of the eyepiece. In embodiments, the overlay may present the content on or near the recognized feature, and in embodiments, the recognized feature may include a playground, a ball, a score, a scoreboard, a jumbo tron, a screen, An object of the game including at least one of a ball path, a stadium seat, and the like. In embodiments, the recognized feature may be an object of artistic performance comprising at least one of a musician, a musical instrument, a stage, a music rack, an actor, a set, a set props, a curtain, In embodiments, the recognized feature is an object for a kiosk comprising at least one of a doll, a stuffed animal, a concert shirt, a food, a beverage, a hat, clothing, a beach towel, a toy, a sports collection, a concert collection,

In embodiments, a glasses platform may be used in a drinking / eating environment. For example, glasses may be used for content in a drinking / eating environment. In embodiments, the user may use the glasses for guitars to view ratings, reviews, venue locations, and content to review available seat availability to reserve seats and the like. The user can also view menu details and prices, food and beverage details such as comparisons between venues and venues, reviews, nutritional content, manufacturing methods, wine evaluation, automated wine pairing, and more. The user can view the social content (e.g., recognize and identify a person and / or interact with sponsors of the same venue). In embodiments, a user may view loyalty program content, such as a dining point, associated with a user's account and / or a particular venue. The user can use the glasses for the guitar to search for the name and definition of the content by searching, to translate items on the menu. The user can view the video or image of the menu item. In embodiments, the user can view the AR version of the menu. In embodiments, the user can capture the menu image and view it in infinite focus, increase the magnification, adjust the contrast of the menu, adjust the lighting, and so on. In embodiments, a user can view menu items, automatically pair the sections of wine and beverage by rating and price, and so on. Users can access the database of what they ate and liked, and they can see memories of past meals. In embodiments, the user may see items different from what he or she is eating. For example, if a user ordered a chopped salad, the user could see it as a filet mignon.

In embodiments, glasses may be used in commerce in a drinking / dining environment. For example, glasses may be used to search for venues, to make reservations or updates, to view menus, to select items to purchase from items of interest or menus, and to select items to be ordered at venues. Glasses can be used to pay for goods, to share payments, to calculate tips, to pay points, for guitars.

In embodiments, glasses may be used in a drinking / eating environment to enter data / items. In embodiments, the user may enter content via Wi-Fi, Bluetooth, and the like. In embodiments, the user may enter a menu, signage, etc., and use AR enhancement to report and / or interact with him. In embodiments, a user may enter ad content and use AR enhancement to view and / or interact with it. The user can enter items for payment such as credit card / debit card, loyalty payment / settlement, and the like. Such input can be done through local communication or the like. In embodiments, the user may make a payment through face recognition. In embodiments, glasses may be used to recognize a face of an employee, and such payment may be used based on such face recognition. In other embodiments, the face of the user or another person's face can be recognized and can be withdrawn from the account to make a payment based thereon.

In embodiments, the system may include an interactive head-mounted eyepiece worn by a user, wherein the eyepiece comprises a module for determining whether the eyepiece is close to at least one of a dietary environment and a drinking environment, An optical assembly for viewing the environment, a processing module for recognizing the feature of the environment and rendering at least one of the environment related drinking and drinking related contents, an image processing module for capturing and processing the image of the wearer's environment of the head mounted eyepiece, The image processing module may lock the display element on a recognized feature of the environment and an integrated image light source for introducing the content into the optical assembly, wherein the integrated image light source includes at least one of the meal- and alcohol- Rendered as an overlay in the environment, and the content is recognized It is presented in relation to the feature portion. In embodiments, the integrated image light source may present a display of content that is related to the environment, and this relationship with recognized features and content may not exist. In embodiments, the rendering of the content may include entering at least one of a meal environment and a drinking environment, pinning the user's eyes to a menu in the environment, opening a menu, recognizing markers in the environment, Focusing on the signage of the subject, or at least one of the others. In embodiments, the content may include a menu, a menu item comparison, a nutritional value of the menu item, a paired wine and menu item, an image of the menu item, an audio description of the menu item, Enhancement menu content including at least one of increased enlargement of the item, contrast and illumination, and categorization of the menu item based on geographic area, component, evaluation of the item, user's previous eating of the item, and the like. In embodiments, the content may be received as a menu when waiting for the user to be directed to the seat. In embodiments, the user may interact with the content through at least one of eye movement, hand gestures, head nudity, and the like. In embodiments, the user may place an order through the eyepiece. In embodiments, the user may pay a check, bill, or tab through an eyepiece. In embodiments, an eyepiece may be used for social networking and provide at least one of face recognition of others in the environment and the environment. In embodiments, the user may do at least one of sending and / or receiving a friend request by gesturing as part of the user ' s body. The content may include additional information about menu items retrieved from the Internet. In embodiments, the overlay may present the content on or near the recognized feature. In embodiments, the recognized features may be at least one of a poster, a frame, a menu plate, a menu, a beverage container, a food presentation cart, a bar, a table, a window, a wall, In embodiments, recognizing a feature may include automated processing of the image including the feature, pinging the feature with a signal, communicating with the feature, recognizing the feature by processing the position of the feature, Retrieving information about the part, customizing the feature, and / or the like. In embodiments, a user may designate a feature to retain the overlay content by interacting with a user interface of the eyepiece.

In embodiments, a spectacle platform may be used in an outdoor environment. In embodiments, the glasses may be used to interact with or view the content. The user can view navigation information such as the location of the trail, the time to the destination, the process of trailing or trailing the trail, the trail map, and the AR overlay that the user might not otherwise see. The user may be provided with outdoor conditions such as temperature, weather, sow condition, fishing condition, water level, tide condition, and the like. The user may use the glasses for communication such as coordination of position-specific groups, weather alerts, etc., which are related to the outdoor environment. The user can collect information for identifying plants, trees, animals, birds, sounds, birds, and the like. In embodiments, a user can view an object and ask the glasses what is "what it is" so that the user can be presented with content and / or information about the object. In embodiments, the user can obtain safety information, such as whether something is edible, toxic, dangerous, and so on. For example, a user may raise a problem such as "is this a dangerous snake" when viewed with glasses, and the glasses may then be used to provide the user with information about the snake, such as whether the snake has a dangerous poison . In embodiments, a user may use the glasses to identify and / or receive content associated with landmarks that are associated with the outdoor environment. These landmarks can help users explore the environment or learn about the environment. In embodiments, a user may use the glasses to view procedural guidance, such as how to hit a tent, tie a particular knot, traverse a difficult terrain, or otherwise. The user can ask "How do I build this tent? &Quot;, and the user can receive a step-by-step instructions for that. In embodiments, the user may view content related to an ego behavior or state, or an analysis thereof. The user may request an update to the glasses, such as "Do I show signs of dehydration," "I am hypothermia," "I am hypoxic," and so on. Based on the results, the user can change his behavior to prevent a particular outcome or to promote a particular outcome. In embodiments, a user may view social content and environments, experience blogs, etc., about other people's experiences on the trail. In embodiments, a user may be alerted that a particular ski trail is dedicated to a professional, or the user may also be informed of the current status, such as that there are severe ice operations on various parts of the trail.

In embodiments, the user may use the glasses in connection with commerce in an outdoor environment. The user can download related contents related to the environment. By way of example, the user can download data relating to trail maps, fishing maps, fishing, skiing, snowboarding, and the like. The user can set up accommodations, order supplies, rent equipment, set guides, travel, participate in events, obtain permission for fishing, such as hunting permits, and so on . In this situation, the user can interact with the social network through the glasses (e.g., the user can join a training club, participate in a trail or chat with other people in a particular environment, etc.). The user can focus on and / or track the goal-directed achievement. For example, a user can track or highlight the goal of climbing Whitney Mountain, pay attention to goals for the charity "running contest " and do other things. The user can use a blog-based commerce model or the like. In embodiments, a user may use social networking to fund a particular outdoor event through a glasses platform.

In embodiments, the user may enter content, data, etc., in or out of the environment, into the glasses. In embodiments, the user can use the camera in the glasses for scene recognition, and the user can use the GPS in the glasses to provide information related to the specific environment or to search for the specific environment. A user may transmit and receive communications to and from other users in the environment, or may transmit and receive communications related to the environment. The user can input the landmark data, view the landmark of the environment using AR enhancement, and do other things. The user can enter features such as leaves and flowers, take notes associated with them, capture their photographs and / or learn about them in the environment. Users can capture their images to learn more about articles, animals, etc. from the environment, store data related to them, interact with the AR content associated with them, .

In embodiments, the system may include an interactive head mounted eyepiece worn by a user, wherein the eyepiece comprises a module for determining whether the eyepiece is close to the outdoor environment, an optical assembly through which the user views the surrounding outdoor environment, A processing module for rendering the outdoor contents for the outdoor environment of the eyepiece on the head mounted eyepiece, an image processing module for capturing and processing the image of the wearer's environment of the head mounted eyepiece, And an integrated image light source for introducing the content into the optical assembly, wherein the integrated image light source can render the outdoor content as an overlay in an environment viewed by the eyewear wearer The content may be associated with a feature, It presents content that is related to the outdoor environment. In further embodiments, the image processing module may lock the display element on the recognized feature of the environment and the content may be presented in relation to the recognized feature in the display. In embodiments, it may be desirable to enter the outdoor environment, to fix the user's eyes to the article in the environment, to recognize features in the environment, to recognize the presence of a person in the environment, The content may be rendered as a result of at least one of focusing on the environment, focusing on signing in the environment, and the like. In embodiments, the content may include information on overlayed trail information, time to destination, user progress information, landmark information, safety information about the environment, location in the environment for other sources, ≪ RTI ID = 0.0 > and / or < / RTI > In embodiments, the content may include user instructions for the user, which may be at least one of audio, video, images, 3D images, step-by-step instructions, and so on over the object. The user can interact with the content through at least one of eye movement, hand gestures, nudging, and the like. The user can select at least one of the following options: accommodation, ordering supplies, renting equipment, making travel itineraries, obtaining permissions or accreditation for activities, entering comments about the environment, Can be performed. In addition, the content may enhance at least one of camera input, GPS information, landmarks in the environment, and features in an outdoor environment. In embodiments, the eyepiece is used to recognize that a person is present in the environment and to present social networking content related to the relationship between the wearer and the perceived person. In addition, the user may do at least one of sending and / or receiving a friend request by gesturing as part of the user ' s body. In embodiments, the content may be rendered based on an analysis of the user's status. In embodiments, recognizing a feature may include automated processing of the image including the feature, pinging the feature with a signal, communicating with the feature, recognizing the feature by processing the position of the feature, Retrieving information about the part, customizing the feature, and / or the like. In addition, the user can specify a feature to retain the overlay content by interacting with the user interface of the eyepiece. In embodiments, the overlay may present the content on or near the recognized feature. In addition, the recognized feature may be at least one of a plant, a tree, a bush, a trail, a rock, a fence, a pathway, a camping ground, a cabin, a tent, a water transportation mode, a water vehicle, and an animal.

In embodiments, the user may use the glasses in an athletic environment. In embodiments, the user may use the glasses to view, download, or otherwise interact with the content. For example, the user may say, "Do I show signs of dehydration?"; "Is I hypothermia?"; "I am hypoxia?" Or self-behaviors or status analysis can be used by asking glasses for example. In embodiments, the user may view health-oriented content such as club fees and offers, upcoming training sessions, and the like. The user can view training-oriented contents such as guidance and educational contents. For example, you can see how to squat, stretch, use instructions, video, AR, and more. Users can view, review, and update blogs such as personal experience blogs that are related to the athletic environment.

In embodiments, the user may use the glasses in commerce in an athletic environment. By way of example, a user may download a guidance program, such as that associated with a user's guide, a trainer, or other guide, for a fee, or for free. In embodiments, the user may track success and / or progress through the program to completion. In various embodiments, the application program may have an advertisement associated with it to be displayed to the user. In embodiments, the user may use the glasses to purchase and sell auxiliary equipment. For example, a user may purchase a new sneaker for increased arch support for running. In embodiments, the user may use the glasses for a charitable event such as "Running Event" or "Everest Climbing for Charity X ", where the user collects donations and / View or update blog entries about him through the platform.

In embodiments, the user may use the glasses to input information and / or data in a motion environment. In embodiments, a user may enter data for performance tracking, input data via a sensor, and input video and video. By way of example only, a user may record another in a particular activity, and then use the video during his training to perfect his appearance, skills, and the like.

In embodiments, the system may include an interactive head mounted eyepiece worn by a user, wherein the eyepiece comprises a module for determining whether the eyepiece wearer is performing at least one of being in motion or in close proximity to the motion environment, An optical assembly for viewing the peripheral motion environment through the user, a processing module for rendering motion-related contents on the head-mounted eyepiece, an image processing unit for capturing and processing the image of the wearer's environment of the head mounted eyepiece, Module, content into an optical assembly, wherein the integrated image light source can render the motion content as an overlay in an environment that the user views, wherein when the user moves the eyepiece, , Wherein the integrated image light source Present content related to exercise environment. In embodiments, the rendering of the content may include entering the athletic environment, pinning the user's eyes to the article in the environment, automatically recognizing the features in the eyepiece's vision, using the equipment in the athletic environment, Recognizing markers in the environment, focusing on signage in the environment, and the like. In embodiments, the content may include augmented movement content that includes at least one of training-oriented content, club information content, a user's manual for the exercise, information about the upcoming classes, and the like. In embodiments, the content may include at least one of 3D content, audio, visual, video, and textual content. The user can interact with the content through at least one of eye movement, hand gestures, nudging, and the like. In embodiments, the content may include user information including at least one of vital signs heart rate, exercise time, lap time, best set time, past user data, and the like. The content may allow a user to purchase a training session, machine time, additional time at the club, beverage, health bar, and the like. In embodiments, the content may be an advertisement for at least one of upcoming classes, health clubs, discounts on items in a juice bar, equipment sales, and the like. In addition, in embodiments, an eyepiece may be used for social networking, wherein the eyepiece provides at least one of a user's posture of the environment and other people's face recognition in the environment. In addition, the user may do at least one of sending and / or receiving a friend request by gesturing as part of the user ' s body. In embodiments, a user may send and receive friend requests to / from other members, trainers, instructors, and the like. In embodiments, the overlay may present the content on or near the recognized feature. In addition, the recognized features may include a calendar, a wall, a window, a board, a mirror, a treadmill, a comprehensive fitness machine, a bicycle, a bicycle exercise equipment, an oval fitness equipment, a gym equipment, a heavy bag, a track, An area, a coat area, and the like. In embodiments, recognizing a feature may include automated processing of the image including the feature, pinging the feature with a signal, communicating with the feature, recognizing the feature by processing the position of the feature, Retrieving information about the part, customizing the feature, and / or the like. In embodiments, a user may specify a feature to retain the overlay content, such as by interacting with a user interface of the eyepiece.

Another application that may be of interest to users is mobile online games using augmented reality glasses. These games may be computer video games such as those provided by Electronic Arts Mobile, UbiSoft and Activision Blizzard [e.g., World of Warcraft® (WoW)]. Just as a game and entertainment application is played on a home computer (rather than a computer at work), augmented reality glasses can also use game applications. The screen can appear inside the glasses, thus allowing the user to view the game and participate in the game. In addition, controls for playing the game may be provided via a virtual game controller, such as a joystick, control module, or mouse, described elsewhere herein. The game controller may include sensors or other output type elements attached to the user's hand for feedback from the user, e.g., through acceleration, vibration, force, pressure, electric shock, temperature, field sensing, The sensors and actuators may be attached to the user's hands through wraps, rings, pads, gloves, bracelets, and the like. Accordingly, the eyepiece virtual mouse may allow the user to convert the movement of the hand, wrist and / or finger into the movement of the cursor on the eyepiece display, where the "motion" includes slow motion, rapid motion, , Position changes, etc., and may allow the user to work in three dimensions including some or all of the six degrees of freedom without requiring a physical surface.

27, the game application implementation 2700 may use both the Internet and GPS. In one embodiment, the game is downloaded to the user's computer or augmented reality glasses from the customer database via the game provider, perhaps using his web service and the Internet as shown. At the same time, spectacles with communication capabilities also receive and transmit communication and telemetry signals via cellular towers and satellites. Thus, the online game system accesses information on the user's desired game activity as well as the location of the user.

The game can use this information for each player's location. For example, the game may incorporate a feature that uses the location of the player via a GPS location tracker or magnetometer location tracker to give points for reaching the location. When the player reaches a certain position, the game may also send a message (e.g., a clue, or display a scene or an image). The message may, for example, be to go to the next destination, and this message is then provided to the player. May be provided as part of a struggle or obstacle that a scene or image must overcome or as an opportunity to gain game points. As such, in one embodiment, the augmented reality eyepiece or glasses can use the wearer's position to make the computer-based video game vigorous and vibrant.

One method of playing an augmented reality game is shown in Fig. In this method 2800, the user logs in to the website, thereby allowing access to the game. The game is selected. In one example, if a multi-player game is available and desired, the user can participate in the game; Alternatively, the user may create a customized game, perhaps using a special role that the user desires. The game can be scheduled and in some cases the players can select a specific time and place for the game, distribute the direction to where the game will be played, and so on. Later, the players meet and enter the game, where one or more players use augmented reality glasses. Participants then play the game and, if applicable, game results and any statistics (scores of players, game time, etc.) may be stored. When the game starts, the place may be different for different players in the game, one player sends to one place and the other player or players send to another place. The game may then have different scenarios for each player or group of players based on their GPS or magnetometer location. Each player may also receive different messages or images based on its role, its location, or both. Of course, each scenario can then lead to other situations, different interactions, directions to other places, and so on. In a sense, these games mix the realities of the player's position with the games the player is participating in.

The game can be a simple game of the type that is played in the palm of a player's hand, such as a small single player game. As a further alternative, more complex multiplayer games may be played. Games like SkySiege, AR Drone, and Fire Fighter 360 belong to the former category. In addition, a multiplayer game is also easily considered. Because all players have to log in to the game, they can be played by friends who log in to a particular game and designate others or people. The location of the player is also available via GPS or other methods. As previously described, sensors in augmented reality glasses or in game controllers (such as accelerometers, gyroscopes, or even magnetic compasses) can also be used for orientation and game play. One example is the AR Invaders available as an iPhone application from the App Store. Other games can be obtained from other vendors and for non-iPhone type systems (such as Layar and AR Drone from Amsterdam, AR Flying Ace and AR Pursuit, Parrot SA, Paris, France).

In embodiments, the game may also be 3D so that the user may experience a 3D game. For example, when playing a 3D game, the user may see a virtual augmented reality or other environment where the user can control his or her view perspective. The user can turn his head to see various aspects of a virtual environment or other environment. Thereby, when the user turns his head or makes another motion, the user can see the game environment as if it is actually in the game environment. For example, the user's viewpoint may be such that the user is positioned " within the 3D game environment " having at least some control over the viewpoint, wherein the user is able to move his head, You can have a view. In addition, the user can 'walk in' to the game when actually walking forward, and the user can have a viewpoint change as the user moves. In addition, the point of view can also change when the user moves the staring view of his eye or the like. Additional image information can be provided on the side of the user view or the like that can be accessed by turning the head.

In embodiments, the 3D game environment may be projected onto a lens of the glasses or viewed by other means. In addition, the lens may be opaque or transparent. In embodiments, a 3D game image may be associated with and include the user ' s external environment so that the user can turn his head and the 3D image and the external environment coexist. In addition, it is also possible that the 3D image is associated with two or more parts of two or more objects or objects in the external environment, such that the 3D image is visible to the user as if the 3D image is interacting with various aspects or objects of the real environment. 3D game video and external environment can be changed. By way of example, a user can see a 3D game monster crawling up or over a car, where such a building or car is an actual object in the user's environment. In this game, the user can interact with the monster as part of the 3D game experience. The actual environment around the user can be part of the 3D gaming experience. In embodiments where the lens is transparent, the user may interact in a 3D gaming environment while moving around in its real environment. The 3D game may include elements of the user's environment in the game, may be entirely created by the game, or may be a mixture of the two.

In embodiments, the 3D image may be associated with or generated by an augmented reality program, 3D game software, or other means. In embodiments where an augmented reality is used for a 3D game, a 3D image may be displayed or perceived by the user based on the user's location or other data. Such augmented reality applications can provide a user with interacting with such 3D images or images to provide a 3D game environment when using glasses. For example, when the user changes his position, the play in the game may advance and various 3D elements of the game may become inaccessible or inaccessible to the observer. By way of example, various 3D enemies of the user's game character may appear in the game based on the user's actual location. The user may interact with or react to the 3D elements associated with other users playing the game and / or other users playing the game. These elements associated with users may include weapons, messages, money, 3D images of users, and the like. Based on the user's location or other data, the user can meet, view, or by any means join the 3D elements associated with other users and other users. In embodiments, a 3D game may also be provided by software installed in glasses or downloaded to glasses, where the user's location is not used or used.

In embodiments, the lens may be opaque to provide a virtual reality or other virtual 3D gaming experience to the user, where the user enters a game in which the user's move may change the viewpoint of the 3D gaming environment for the user . The user may be provided with various body, head and / or eye movements, a game controller, one or more touch screens, or any other type of display that allows the user to navigate, manipulate, and interact with the 3D environment, Any of the control techniques described in the specification can be used to navigate or explore in a virtual environment.

In various embodiments, a user may interact with one or more wired or wireless controllers, one or more touch screens, through the use of any of the control techniques described herein, via the body, hands, fingers, eyes, or other movement , Etc., can navigate the 3D game environment, interact with it, manipulate it, and experience 3D games.

In embodiments, the internal and external facilities that can be used by the eyepiece include learning a user's behavior of the eyepiece and enabling the learned behavior to be stored in a behavior database Lt; / RTI > For example, the user may be provided with commands from the user, images sensed by the camera, GPS position of the user, sensor input over time, triggered by the user, communication to / from the user, , The music being heard, the direction requested, the recommendations used or provided, etc., may be recorded by the eyepiece. This behavior data may be stored in the behavior database, for example, tagged with a user identifier or autonomously. The eyepiece can collect this data in learning mode, acquisition mode, and others. The eyepiece may utilize past data acquired by the user to inform or inform the user of what the user has done previously, or as an alternative, the eyepiece may allow the user to view some of the eyepiece functions and applications The data may be used to predict if the data may be needed. In this way, the eyepiece can serve as an automated assistant to the user (e.g., when a user typically launches an application and launches an application, accesses a place or enters a building, Turn off GPS, stream music when the user enters the gym, and play guitar). Alternatively, the learned behaviors and / or behaviors of a plurality of eyepiece users may be autonomously stored in a common behavior database, wherein the learned behaviors of a plurality of users are used for individual users based on similar conditions It is possible. For example, a user may be visiting a city and waiting for a train on a platform, and the user's eyepiece may be used to determine what other people did while waiting for the train (e.g., asking for directions, searching for points of interest, Accessing a common behavior database to check for music, listening to music, viewing train schedules, contacting city websites for travel information, connecting to social networking sites for entertainment in the area, etc.) . In this way, the eyepiece can provide the user with automated assistant means having many different user experience advantages. In embodiments, the learned behavior may be used to develop preference profiles for users / users, recommendations, ad targeting, social network contacts, behavior profiles for users or a group of users, and the like.

In one embodiment, the augmented reality eyepiece or glasses may include one or more acoustic sensors 2900 that detect sound. One example is shown in FIG. 29 above. In a sense, an acoustic sensor is similar to a microphone in that it detects sound. Acoustic sensors typically have one or more frequency bandwidths that are more sensitive, and thus the sensor can be selected for the intended application. Acoustic sensors are available from various manufacturers and are available with appropriate transducers and other necessary circuits. Manufacturers include ITT Electronic Systems of Salt Lake City, Utah; Meggitt Sensing Systems, San Juan Capistrano, CA; And National Instruments of Austin, Tex., USA. Suitable microphones include those that include a single microphone as well as an array of microphones or a microphone array.

Acoustic sensors may include using micro electromechanical systems (MEMS) technology. Due to the very fine structure of the MEMS sensor, the sensor is extremely sensitive and typically has a wide range of sensitivity. MEMS sensors are typically fabricated using semiconductor manufacturing techniques. A typical MEMS accelerometer element is a moving beam structure consisting of two sets of fingers. One set being fixed to a solid ground plane on the substrate; The other set is attached to a known mass mounted on a spring movable in response to an applied acceleration. This applied acceleration changes the capacitance between the fixed beam finger and the moving beam finger. The result is a very sensitive sensor. Such sensors are, for example, manufactured by STMicroelectronics of Austin, Texas and Honeywell International of Morristown, New Jersey, USA.

In addition to identification, the sound function of the augmented reality device can also be applied to locate the location of the source of sound. As is known, at least two sounds or acoustic sensors are needed to locate the sound. Acoustic sensors will be equipped with appropriate transducers and signal processing circuitry (such as digital signal processors) to interpret the signal and achieve the desired target. One application for a sound position sensor may be to determine the source of sound from within a emergency location (e.g., a fire building, a car accident, etc.). Each of the emergency workers mounting the embodiments described herein may have one or more acoustic sensors or microphones embedded within the frame. Of course, the sensor may also be worn on a person's clothing, or even attached to a person. In either case, a signal is transmitted to the controller of the augmented reality eyepiece. The eyepiece or glasses may be equipped with GPS technology and may also be equipped with a direction-finding function; Alternatively, if there are two sensors per person, the microcontroller can determine the direction in which the noise is generated.

If there are two or more firefighters or other emergency responders, their location is known from their GPS function. Either of the two, or the fire brigade or headquarters, will then know the location of the two arrestors and the direction to the noise detected by each of the arrestors. The exact point of the source of the noise can then be determined using known techniques and algorithms. For example, Acoustic Vector-Sensor Beamforming and Capone Direction Estimation, M. Hawkes and A. Nehorai, IEEE Transactions on Signal Processing, vol. 46, no. 9, Sept. 1998 (pages 2291 to 2304); Also, Cramer-Rao Bounds for Direction Finding an Acoustic Vector Sensor Under Nonideal Gain-Phase Responses, Noncollocation or Nonorthogonal Orientation, P.K. Tam and K. T. Wong, IEEE Sensors Journal, vol. 9. No. 8, August 2009 (pp. 969-982). The techniques used may include timing differences (differences in arrival times of sensed parameters), acoustic velocity differences, and sound pressure differences. Of course, acoustic sensors typically measure the level of sound pressure (e.g., decibels), and these other parameters can be used in acoustical sensors of the appropriate type, including acoustic emission sensors and ultrasonic sensors or transducers.

Suitable algorithms and all other necessary programming can be stored in a microcontroller of the eyepiece or in a memory accessible by the eyepiece. With more than two arrestors or a number of arrestors, a potent location can be determined subsequently, and the arrestees can try to find the person to rescue. In other applications, facilitators may use these acoustic functions to determine the location of a person of interest for law enforcement purposes. In other applications, a large number of people in operation may be exposed to enemy fire including direct fire (line of sight) or indirect fire (including high angle fire) You can meet. The same techniques described herein can be used to estimate the location of enemy fire. This can be more accurate if there are a few people in the area, especially if people are at least some distance from a larger area. This can be an effective tool for directing counter-battery fire or counter-mortar fire on enemy forces. Direct shooting can also be used if the target is close enough.

One example of using embodiments of an augmented reality eyepiece is shown in Figure 29B. In this example (2900B), a number of soldiers, each equipped with Augmented Reality eyepieces, are on probing and are alert to enemy fire. Sounds detected by their acoustic sensors or microphones can be relayed to a squad vehicle, their sub-summit, or to a remote tactical operations center (CP) or command post (CP), as shown. Alternatively or additionally, the signal may also be transmitted to a mobile device, such as an airborne platform, as shown. Communication between soldiers and additional locations may be facilitated using a local area network or other network. In addition, all transmission signals can be protected by encryption or other protection measures. One or more of the command vehicles, the platoon leader, the mobile platform, the TOC, or the CPS will have an integrated function to combine inputs from a number of soldiers and determine the possible locations of enemy fire. The signal from each soldier will include the position of the soldier from the augmented reality glasses or the GPS function built into the eyepiece. Acoustic sensors in each soldier can indicate possible directions of noise. Using signals from a number of soldiers, the direction and possibly the location of the enemy fire can be determined. Soldiers can then neutralize their position.

In addition to the microphone, the augmented reality eyepiece may be an articulating earphone, detachably attached (1403), or equipped with an audio output jack (1401), as discussed elsewhere herein , Which may be equipped with an earphone. The eyepiece and earphone may be equipped to deliver noise-canceling interference, thus allowing the user to better hear the sound transmitted from the augmented reality eyepiece or glasses' audio-video communication function, The gain control can be characterized. The augmented reality eyepiece loudspeaker or earphone may also be connected to the entire audio and video functions of the device and thus deliver high quality, clean sound from the included communication device. As discussed elsewhere herein, this includes a radio or cellular (smartphone) audio function and also includes supplemental technologies such as Bluetooth ™ functionality or related technology for a wireless personal area network (WPAN), such as IEEE 802.11 . ≪ / RTI >

Other aspects of the enhanced audio functionality include speech recognition and identification capabilities. Speech recognition is about understanding what is being said, whereas identification is about understanding who the speaker is. Voice identification can work with the face recognition capabilities of these devices to more clearly identify the person of interest. As described elsewhere in this document, a camera connected as part of an augmented reality eyepiece can stealthily focus on a desired person, such as a person in a crowd or multiple faces in a crowd. Using a camera and appropriate face recognition software, images of people or people can be taken. The feature portion of the image is divided into any number of measurements and statistics, and the results are compared to a database of known people. Then, the identity can be confirmed. In the same way, voice or voice sampling from a person of interest can be taken. The sample may be displayed or tagged, e.g., at a specified time interval (e.g., a description or number of physical characteristics of a person). The voice samples can be compared to a database of known people, and if the voice of a person is matched, the identity can be verified. In embodiments, a number of people of interest may be selected for biometric identification and the like. Multiple selection can be through the use of cursors, hand gestures, eye movements, and so on. As a result of the multiple selection, information about the selected person can be provided to the user via, for example, display, audio, and the like.

In embodiments where a camera is used for biometric identification of a large number of people in the crowd, the control techniques described herein may be used to select the face or iris for imaging. For example, cursor selection using a hand wearing control device may be used to select multiple faces in a view of the user's surroundings. In another example, a gaze tracker may be used to select which face to select for biometric identification. In another example, the hand held control device may sense a gesture (e.g., indicating a person) that is used to select a person.

In one embodiment, an important characteristic of a particular person's voice can be understood from one sample of the human voice or from multiple samples. Samples are typically decomposed into segments, frames, and subframes. Typically, the important features include the fundamental frequency, energy, formant, speaking rate, etc. of the human voice. These features are analyzed by software that analyzes speech according to a particular equation or algorithm. This field is constantly changing and improving. However, at present, these classifiers are, among others, neural network classifiers, k-classifiers, hidden Markov models, Gaussian mixture models, and pattern matching algorithms And the like.

A general template 3100 for speech recognition and speaker identification is shown in FIG. The first step 3101 is to provide a voice signal. Ideally, we have a known sample from a previous encounter to compare with the signal. The signal is then digitized in step 3102 and is segmented into fragments such as segments, frames, and subframes in step 3103. Then, in step 3104, features and statistics of the speech samples are generated and extracted. Then, in step 3105, a classifier or two or more classifiers are applied to determine a general classification of the samples. Then, at step 3106, a post-processing of the samples may be applied, for example, to compare the samples with known samples for possible matching and identification. Then, in step 3107, the result may be output. The output can be sent to the person requesting the match, and can also be recorded and sent to other people and / or to one or more databases.

In one embodiment, the audio function of the eyepiece includes hearing protection by the associated earphone. The audio processor of the eyepiece can enable automatic noise suppression if, for example, a large noise is detected near the wearer's head. Any of the control techniques described herein may be used with automatic noise suppression.

In one embodiment, the eyepiece may comprise a Nitinol head strap. The head strap can be a thin band of curved metal that can be pulled outwardly from the arm of the eyepiece for securing the eyepiece to the head, or rotated outward and extend outwardly behind the head. In one embodiment, the tip of the Nitinol strap may have a silicon cover, thus capturing the silicon cover and pulling outwardly from the end of the arm. In embodiments, only one arm has a nitinol band and is fixed to the other arm to form a strap. In other embodiments, both arms have a Nitinol band, and both sides are pulled outward to combine to form a strap or to independently hold a portion of the head and secure the eyepiece to the head of the wearer. In embodiments, the eyepiece may have replaceable equipment (e. G., A joint) that attaches the eyepiece to a person's head, wherein the head strap, eye arm, helmet strap, helmet snap connection, May be attached. For example, there may be a joint in the eyepiece near the user's eyewear, where the eyepiece can be attached to the strap, where the strap can be detached and thus the user can move the eyepiece Attached to a helmet, or otherwise. In embodiments, the replaceable equipment for attaching the eyepiece to the user ' s head or to the helmet may comprise a built-in antenna. For example, a Nitinol head strap may have a built-in antenna for the guitar, for example, for a plurality of frequencies, for a particular frequency. In addition, arms, straps, etc. may include RF absorbing foams to help absorb RF energy while the antenna is being used for transmission.

Referring to FIG. 21, the eyepiece may include one or more adjustable wrap around extendable arms 2134. Adjustable wraparound extendable arms 2134 can secure the position of the eyepiece to the user ' s head. One or more of the extendable arms 2134 may be made of shape memory material. In embodiments, one or both of the arms may be comprised of nitinol and / or any shape memory material. In other cases, the end of at least one arm of the wraparound extendable arms 2134 may be covered with silicon. In addition, adjustable wraparound extendable arms 2134 can extend from the end of eyepiece arm 2115. They can be telescopically extended and / or slid outwardly from the end of the eyepiece arm. They can slide outward from the inside of the eyepiece arm 2116 or slide along the outer surface of the eyepiece arm 2116. [ In addition, the extendable arms 2134 can meet and be secured to one another. The extendable arms may also be attached to other portions of the head mounted eyepiece to create a means for securing the eyepiece to the user ' s head. The wraparound extendable arms 2134 may be secured to one another and may be secured to one another, interlocked, connected, magnetically coupled, or otherwise secured by other means to provide secure attachment to the user's head. . In embodiments, the adjustable wraparound extendable arms 2134 may also be independently adjustable to attach to or hold a portion of the user ' s head. Accordingly, the independently adjustable arms can provide the user with an increase in customizability in which a personalized fit fixes the eyepiece to the user ' s head. In addition, in embodiments, at least one of the wraparound extendable arms 2134 can be detached from the head mounted eyepiece. In still other embodiments, wraparound extendable arms 2134 may be an additional feature of the head mounted eyepiece. In such a case, the user may install an extendable, non-extendable or other arm on the head mounted eyepiece. For example, the arms may be sold as a kit or as part of a kit that allows the user to customize the eyepiece to suit his particular preferences. Accordingly, a user can customize that type of material that makes adjustable wraparound extendable arms 2134 by selecting different kits with particular extendable arms that are suitable for his or her preferences. Accordingly, the user can customize his eyepiece for his specific needs and preferences.

In yet other embodiments, an adjustable strap 2142 may be attached to the eyepiece arm such that the adjustable strap 2142 extends around the rear of the user's head, to secure the eyepiece in place. The strap can be adjusted to fit the right foot. The strap may comprise any suitable material, including but not limited to rubber, silicone, plastic, cotton fabrics, and the like.

In one embodiment, the eyepiece may be secured to the user's head by a plurality of other structures, such as a rigid arm, a flexible arm, a gooseneck flex arm, a cable-tension system, and the like. For example, the flexible arm may be comprised of a flexible tubing, such as in a gooseneck configuration, wherein the flexible arm may be bent into a position for adjustment to a given user's pit, where The flexible arm may be reshaped as needed. In other cases, the flexible arm may be used as a cable-tensioning system, such as in a robot finger configuration having a plurality of joints connecting members curved in a curved shape by a pulling force applied to the cable passing through the joint and member . In this case, the cable drive system may implement an articulated ear horn for size adjustment and eyepiece headwear retention. The cable-tension system may have two or more linkages and the cables may be stainless steel, Nitinol-based, electro-actuated, ratcheted, wheel-adjusted, etc. Lt; / RTI >

Embodiments of the cable-tensioning system 17800 are shown in Figures 178, 179a and 179b. In embodiments, the cable-tensioning system includes an ear horn 17802, which comprises two or more joints connecting members that can be curved in a curved shape by a tensile force applied to a cable 17804 passing through the joint and / ). In an upright posture, as shown in Figure 178, the ear horn can be arranged straight along the user's head. Cable 17804 is attached to and tightened through adjuster 17808 thereby placing the adjuster to increase the tension of cable 17804 can be flexed or bent to conform to the shape of the user's head do. By increasing this tension, the horn can become stiffer and / or tighter. By being shaped to fit the head of the user, the ear horn 17802 can be adjusted to fit the particular user's head and / or to hold the glasses firmly on the user's head thereby helping to retain the glasses. In the embodiments, as the tension of the cable 17804 increases, the ear horn becomes stiffer or less loose to fit the user ' s head and as the tension in the cable 17804 relaxes, So that one or both of the horns can be straightened and / or flat folded. In embodiments, adjuster 17808 may be a ratchet, an electrically actuated, a wheel adjustable, a wedge slider, or the like. In embodiments, the pawl slider may be tapered to allow the position of one or more portions of the ear horn and / or eyepiece to be raised or lowered by pushing or pulling the tab or the like inward or outward to provide adjustment And may be a tapered adjustment. In embodiments, ear horn 17804 may be configured as a robot finger configuration and may be shaped as shown in Figure 179b. The adjustable ear horn can provide the advantage of securing the eyepiece to the user ' s head, while providing folding convenience for ease of use, as described herein. In embodiments, the ear horn may provide a wraparound head design wherein the ear horns of the left and right earpiece wrap around the user ' s head and touch or hardly touch at the back of the user ' s head. In embodiments, the ear horns can then be fastened together for additional safety. This fastening can be through a magnet in each ear horn, a hooking mechanism in the ear horn, and the like. In embodiments, the ear horn may partially or wholly surround or follow the contour of the user ' s head and / or may be secured to the rear of the user ' s ear and / And may be fixed to the user's head. In embodiments, the ear horn may be attached to an earpiece of an eyepiece such as earpiece 2104 shown in Fig. The horn may then be permanently or removably attached to the earpiece. In embodiments, the ear horn can include a portion of the earpiece of the eyepiece as shown in Figure 180, or it can include the entire earpiece (not shown). In embodiments, the adjuster 17808 may be positioned proximate the eyepiece, on a portion of the horn, near the ear of the user, or beyond the ear of the user to the end of the horn, or on the ear horn and / or any other portion of the eyepiece . In embodiments, one or both of the following horns may be adjusted as described herein. In embodiments, the ear horn (shown only without the eyepiece itself), as described herein, is configured to surround the user ' s head and / or to follow the contour of the user ' Can be.

In embodiments, switchable attraction between multiple layers in a laminate may be used in the ear horn. For example, the one or more ear horns can include layers within the laminate, and attraction between the layers can come from magnetic, electrostatic, and / or vacuum means. In embodiments, the magnets can be used by rotating the poles so that the layers in the laminate are attracted to each other so that the horns become stiff and the horns are in an attractive or repulsive position that can push each other to loosen. In embodiments where the laminate layers are close to each other, a voltage may be applied to create an electrostatic attraction that can be electrically switched. When manpower is created, then the horn can become stiff. When the voltage is removed or when the electrostatic attraction is switched, the horn can be loosened. In embodiments, by creating a vacuum by applying forces together to the two layers having a spring back at one or more portions of the layers that are coupled to each other and create cavities or voids between the layers, Lt; / RTI > When layers are energized together, the layers can then stiffen the horn. The vacuum seal can then be broken to allow the horn to loosen. In various embodiments, the ear horn can provide for holding the eyepiece more rigidly and / or firmly to the user's head when the earhorn becomes stiff. In embodiments, the ear horn may be partially or entirely surrounding or contoured around the user ' s head and / or following the side of the user ' s head and / or behind the user ' s ear and / And may be fixed to the user's head by being fixed to the back of the head. When the electrostatic voltage, the magnetic attraction and / or the vacuum are adjusted, the horn may become stiff so that the horn can be fixed to the user's head, and then the horn is loosened or loosened And may be flattened in a straight and / or closed position.

In embodiments, the at least one ear horn can include an inner rod and / or string structure, wherein each ear horn further comprises a magnet. The magnet of each ear horn can be connected to the magnet of the horn on the other side, thereby allowing two ear horns to wrap around the user's head. The actuation of the magnets to each other may allow the string and / or inner rod structure to be taut, thereby providing a more rigid fit to the user's head. By connecting the magnets, in embodiments, the inner rods of the horn can then stand upright or become more rigid so that the horn can wrap around the user ' s head and / or the inner string of the ear horn inner string) is tight and the horn can wrap around the user ' s head. In embodiments, the ear horn may partially or wholly surround or follow the contour of the user ' s head and / or may be secured to the rear of the user ' s ear and / And may be fixed to the user's head. When the magnets are not connected, the horn can then be straightened and / or folded flat.

In embodiments, the at least one ear horn can utilize the air pressure in the chamber inside the ear horn, which can stiffen the ear horn. The air pressure can then be increased to stiffen the horn. This stiffening can allow the horn to be adjusted to fit the user ' s head and / or wrap around the user ' s head when the eyepiece is in use. In embodiments, the ear horn may partially or wholly surround or follow the contour of the user ' s head and / or may be secured to the rear of the user ' s ear and / And may be fixed to the user's head. Air pressure can then be reduced to loosen the horn. When the horn is loosened, the horn can then be straightened and / or folded flat. The air pressure can be adjusted before or after writing on or off the user's head. In embodiments, the air pressure can be regulated by a pump in the side frame operated by finger pressure or other means. In embodiments, the pump may be adjusted through a user interface displayed on the glasses or by other means.

In various embodiments, as described herein, the stiffness of the ear horn may be related to thickness in a cubic relationship. By way of example, two unconnected layers may be twice as stiff as a single layer, but if the layers are connected in a single layer, the combined layer that is twice as thick will have an increased stiffness by eight times. As a further example, three single layers are three times stiffer than a single layer, but three interconnected layers will be 27 times stiffer than a single layer.

In embodiments, the at least one ear horn can include an inner portion and an outer portion such that, as shown in Figure 181, the inner portion is formed from one portion of the horn and the outer portion is formed from another portion of the horn Respectively. The inner portion and the outer portion may be formed from the ear horn to form two separate portions or may be formed in different ways from the ear horn, wherein one portion is the outer portion and the other portion is the inner portion. In embodiments, the inner portion may be in contact with the user ' s head, while the outer portion may be in contact with the inner portion. In embodiments, as shown in the embodiment described in Figure 182, the inner and outer portions may be interlocked. The inner and outer portions may include interlocking grooves, teeth or other means for interlocking or securing them together. The top and / or outer portion may include a tab or other protrusion, thereby allowing the user to prevent the inner and outer portions from further locking together. In embodiments, these portions may be curved to fit the user ' s head. In addition, the inner surface can be pushed outward against the outer surface. By interlocking the inner and outer portions, the thickness of these portions can be doubled. Accordingly, by increasing the thickness of the horn portion, the stiffness can be increased. In embodiments, by doubling the thickness of the ear hone, the stiffness can be increased by a factor of eight compared to the single layer. When the outer layer is peeled off, the horn portion can return to the flexible state, whereby the ear horn can be folded flat. In embodiments, the ear horn may be attached by a magnet, clip, hook, or by other means to secure it to the user's head.

In addition, in embodiments, as shown in Figure 183, one or more ear horns can include three portions. In this embodiment, although the ear horn can include an inner portion and an outer portion, as described with reference to Figures 181 and 182, this embodiment also includes an ear horn, as shown in Figure 183, And may include an intermediate portion 18302 to be composed of three parts. The horn may then further comprise one or more buttons, straps, interlocking grooves, teeth, pins or other means to lock the horn portions together. One or more of the portions may include a tab or other protrusion so that the user can cause the inner and outer portions to no longer be locked to each other by causing the teeth or other means to lock the portions to be released. In embodiments, the three unconnected layers can be three times stiffer than a single layer, but when the three layers are locked / connected to each other, the horn can then be stiffened 27 times more than a single layer. When the three parts are not connected to each other or are unlocked, the horn can be flexible and therefore can be straightened and / or flat folded. In addition, while the parts are not locked together, the parts can slide on top of each other, thereby becoming flexible and easier to store when not in use, and when the layers are locked or pinned to each other, You may not be able to slide above each other. The horn portions may then be in a sheath, tube or other structure that constitutes the horn so that the individual portions are not exposed. While two and three part ear horns are described, one skilled in the art will appreciate that in various embodiments, the ear horn can be made up of four or more parts and / or of varying thicknesses.

In various embodiments described herein, the lapping horn can be folded flat. When the ear hone is folded into the closed position, such as when the user is not using the eyepiece, the horn can be straightened and thus folded more flatly, and then the horn wraps around the user's head and / What you can follow may not hinder you from folding flat. In various embodiments described herein, the horn can be flattened by being folded, so that the horn can be flattened so that the folding can be stored in a more flattened configuration. In various embodiments, the ear horn can be straightened when released at the hinge or by other means, thereby allowing the eyepiece to be folded flat. As described herein, in various embodiments, the ear horn can be made less rigid and thereby fold flat.

In embodiments, a leveler pad may be used for one or more ear horns, so that the leveler pad may provide for adjustment to the ear in a different vertical position, or may take into account the different heights of the user's ear or eyes . In embodiments, the pads may be disposed on the horns at the contact points of the user's ear to adjust the eyepiece to fit different positions of the user's ears and / or eyes. In embodiments, the leveler pads may be adjusted by a pawl slider or by various means. The leveler pad may be part of the ear horn or the leveler pad may be attached to the ear horn through a clip, adhesive, friction or other means.

In various embodiments described herein, the eyepiece and ear horn can be fitted with a closed cell foam in one or more areas that will come into contact with the user. The foam can provide comfort to the user while also preventing moisture and sweat from penetrating the foam. In addition, the closed cell foam can provide a non-porous surface to prevent the eyepiece from transferring bacteria, microorganisms and other organisms and to prevent its growth. In the various embodiments described herein, the foam may be antibacterial and / or anti-bacterial and / or may be treated with a material for this purpose.

In one embodiment, the eyepiece may include security features such as M-Shield Security, Secure Content, DSM, Secure Runtime, IPsec, and the like. Other software features may include a user interface, an app, a framework, a BSP, a codec, integration, testing, system validation, and the like.

In one embodiment, the eyepiece material may be selected to enable retouching.

In one embodiment, the eyepiece can access a 3G access point that includes a 3G wireless, 802.11b connection and a Bluetooth connection to enable it to hop data from the device to a 3G supported embodiment of the eyepiece.

The present disclosure also relates to a method and apparatus for capturing biometric data about an individual. The method and apparatus provide wireless capture of an individual's fingerprints, iris patterns, facial structures, and other unique biometric features and then transmit data to the network or directly to the eyepiece. The data collected from an individual can also be compared to previously collected data and used to identify a particular individual.

In embodiments, the eyepiece 100 may be a mobile biometric device such as a biometric flash light 7300, a biometric telephone 5000, a biometric camera, a pocket biometric device 5400, an arm strap biometric device 5600, A mobile biometric device may be associated with the mobile biometric device, for example, to control the device, to display data from the device, to store data, to connect to an external system, For example, to connect to other eyepieces and / or other mobile biometric devices, or to act as a standalone device or otherwise communicate with the eyepiece. Mobile biometric devices can enable military personnel or other non-military personnel to collect or use existing biometrics to create an individual profile. The device may provide for tracking, monitoring and collecting biometric records, including video, voice, posture, face, iris biometrics, and the like. The device may provide geographic location tags for collected data such as time, date, location, data acquisition personnel, environment, and the like. This device records, collects, identifies and confirms face, fingerprints, irises, potential fingerprints, potentials, items contained in pockets, and other identification visible marks and environmental data using, for example, a thin film sensor, , Fingerprints, long notes, scars, marks, tattoos, audio, video, annotations, and the like. The device can read fingerprints in wet or dry conditions. The apparatus may include a camera having IR illumination, UV illumination, etc., which can project dust, smoke, fog, and the like. The camera can support dynamic range expansion, adaptive defocus pixel correction, advanced sharpness enhancement, geometric distortion correction, advanced color management, hardware-based face detection, and video stabilization. In embodiments, the camera output may be transmitted to an eyepiece for presentation to a soldier. The device may accommodate a plurality of other sensors, such as those described herein, including an accelerometer, compass, ambient light, proximity, air pressure and temperature sensors, etc., depending on the requirements. The device may also have a mosaic print sensor, as described herein, that produces a high resolution image of a person's fingerprint, a plurality of simultaneous fingerprints, a long whorl, have. Soldiers can use mobile biometric devices to more easily collect personal information for document and media exploitation (DOMEX). For example, during interviews, enrollment, interrogation, etc., an executive may capture and read identification data or 'items in a pocket' (eg, passports, IDs, cards, personal documents, cell phone books, And others to create a profile of a person of interest, and the profile can be entered into a searchable security database. In embodiments, biometric data may be filed using the most important images and manual input, which allows for partial data capture. Data may be automatically geo-located, time / date stamped, filed with a digital dossier, etc., with a locally assigned or globally unique identifier (GUID) assigned to the network. For example, a facial image can be captured in a scene of IED bombing, a left iris image can be captured in a scene of suicide bombing, a potential fingerprint can be taken from a sniper rifle, Which are acquired from different mobile biometric devices at different times, all of which identify a person of interest from multiple inputs at random vehicle inspection points and the like.

Additional embodiments of the eyepiece may be used to provide biometric data collection and result reporting. The biometric data may be visual biometric data such as face biometric data or iris biometric data, or may be audio biometric data. 39 shows an embodiment for providing biometric data acquisition. The assembly 3900 includes the eyepiece 100 discussed above with respect to FIG. The eyepiece 100 provides an interactive head-mounted eyepiece including an optical assembly. Other eyepieces providing similar functionality may also be used. The eyepiece may also include a global positioning system (GPS) function to enable location information display and reporting.

The optical assembly allows the user to view the surroundings, including those in the vicinity of the wearer. One embodiment of an eyepiece allows a user to physically identify nearby people using both facial and iris images or facial and iris images or audio samples. The eyepiece includes a correction element for correcting the user view of the surrounding environment, and also displays the contents provided to the user through the integrated processor and the image light source. The integrated image light source introduces the user to the content to be displayed to the optical assembly.

The eyepiece also includes an optical sensor that captures biometric data. In one embodiment, the integrated light sensor may include a camera mounted on an eyepiece. This camera is used to capture a person's biomedical image in the vicinity of the user of the eyepiece. The user directs the optical sensor or camera to a nearby person by placing the eyepiece in the proper direction (which can only be done by looking at a person). The user can select whether to capture one or more of a face image, an iris image, or an audio sample.

The biometric data that can be captured by the eyepiece illustrated in Fig. 39 includes a face image for face recognition, an iris image for iris recognition, and an audio sample for voice recognition. The eyepiece 3900 includes a plurality of microphones 3902 as end fire arrays arranged along both left and right eyepieces of the eyepiece. The microphone array 3902 is specifically tuned to enable acquisition of human speech in environments with high levels of ambient noise. The microphone is directional, adjustable, and concealable. Microphone 3902 provides selectable options for improved audio acquisition, including omni-directional operation or directional beam operation. The directional beam motion allows a user to record audio samples from a particular person by adjusting the microphone array in the direction of the target person. An adaptive microphone array can be created that allows the operator to adjust the directivity of the microphone array in three dimensions where the directional beam is adjusted in real time to maximize the signal for unstoped targets or minimize interference noise . Array processing may enable the summation of cardioid elements by analog or digital means, where there may be a transition between omnidirectional array operation and directional array operation. In embodiments, beamforming, array tuning, adaptive array processing (voice source location), etc. may be performed by the onboard processor. In one embodiment, the microphone is capable of 10 dB directional recording.

The inclusion of phased array audio and video tracking for audio and video capture improves audio capture. In an environment with different noise sources, audio tracking allows the continuous capture of audio samples when the target person is moving. In embodiments, it may be possible to provide a better location tracking, for example, to distinguish what it is talking about, to provide better location tracking, to provide better audio tracking, Can be subtracted from the audio track.

To provide power for the display optics and biometric data collection, the eyepiece 3900 also includes a lithium ion battery 3904 that can operate for more than 12 hours on a single charge. In addition, the eyepiece 100 also includes a processor and solid state memory 3906 for processing the captured biometric data. The processor and memory are configurable to function with any software or algorithm used as part of a biometric capture protocol or format (e.g., .wav format).

A further embodiment of eyepiece assembly 3900 provides an integrated communications facility that transmits captured biometric data to a remote facility that stores biometric data in a biometric data database. The biometric data database interprets the captured biometric data, interprets the data, and prepares the content for display on the eyepiece.

In operation, a wearer of an eyepiece for capturing biometric data from an observed person in the vicinity positions itself so that the person appears in the field of view of the eyepiece. When in the home position, the user starts capturing biometric information. Biometric information that can be captured includes iris images, face images, and audio data.

To describe the operation, a wearer of an eyepiece for capturing audio biometric data from an observer in the vicinity of the eyepiece is set so that the person is positioned near the eyepiece, specifically, in the vicinity of the microphone array located on the eyepiece eyepiece bridge . When in the home position, the user starts capturing audio biometric information. This audio biometric information consists of recorded samples of what the target person is talking about. Audio samples may be captured along with visual biometric data such as iris images and face images.

To capture the iris image, the wearer / user observes the desired person and positions the eyepiece so that the photosensor assembly or camera can capture images of the desired person's biological parameters. Once captured, the eyepiece processor and the solid state memory are ready to transfer the captured image to the remote computing facility for further processing.

The remote computing facility receives the transmitted biometric image and compares the transmitted image with previously captured biometric data of the same type. The iris or facial image is compared to the previously collected iris or facial image to determine if the person has previously met and identified.

If a comparison has been made, the remote computing facility transmits to the wearer / user's eyepiece to display a report of the comparison. The report may indicate that the captured biomedical image matches the previously captured image. In this case, the user receives a report containing the identity of the person, along with other identifying information or statistics. Not all of the captured biometric data allow a clear identification of the identity. In such a case, the remote computing facility may provide a report of the findings of the investigation and may ask to possibly collect other types of additional biomaterials to aid in the identification and comparison process. As a further aid to identification, visual biometric data can be supplemented with audio biometric data.

Facial images are captured in a similar way to iris images. Due to the size of the image collected, the field of view needs to be larger. This also allows the user to stand further away from the subject on which face biometric data is being captured.

In operation, the user may have captured a face image for the first time. However, the facial image may be incomplete or inconclusive, because a person may be wearing other embellishments (e.g., hats) that obscure the clothes or facial features. In this case, the remote computing facility may request that different types of biometric capture be used and additional images or data be transmitted. In the case described above, the user may be instructed to acquire an iris image to supplement the captured face image. In other cases, the additional requested data may be an audio sample of a human voice.

FIG. 40 shows that an iris image is captured for iris recognition. This figure shows the focus parameter used for analyzing the image and includes the geographical position of the person in capturing biometric data. Figure 40 also shows a sample report displayed on the eyepiece.

Figure 41 shows capture of multiple types of biometric data (in this case, facial and iris images). This acquisition can be done at the same time or at the request of a remote computing facility if the first type of biometric data results in non-deterministic results.

Fig. 42 shows the electrical configuration of a plurality of microphone arrays included in the eyepiece of the eyepiece of Fig. 39; The endfire microphone array allows for better distinction and better directivity of the signal at greater distances. Signal processing is improved by including a delay in the transmission line of the rear microphone. The use of a dual omnidirectional microphone allows the transition between an omnidirectional microphone and a directional microphone. This enables better direction detection for the desired person's audio capture. Figure 43 shows the directional improvement available by different microphones.

As shown in the upper portion of FIG. 43, a single non-directional microphone may be used. The microphone can be placed at a given distance from the source, and the sound pressure at the microphone or digital audio (DI) will be at the given dB level. Instead of a single microphone, an array of multiple microphones or microphones may be used. For example, for a distance factor of 2, two microphones can be placed two times farther away from the source, and the sound pressure increases by 6 dB. As an alternative, four microphones at a distance factor of 2.7 can be used, and the sound pressure is increased by 8.8 dB. An array may also be used. For example, an 8-microphone array at a distance factor of 4 can have a DI increase of 12 dB, whereas a 12-microphone array at a distance factor of 5 can have a DI increase of 13.2 dB. The graph of Figure 43 shows points that produce the same signal level at the microphone from a given sound pressure level at that point. As shown in FIG. 43, a primary supercardioid microphone may be used at the same distance, and in this example, a 6.2 dB increase, while a secondary multiple microphone may be arranged in a multiple microphone array . Instead of using one standard high quality microphone to capture the audio sample, the eyepiece eyepiece leg cuff portion houses a plurality of microphones of different characteristics. This can be provided, for example, when a user is generating a biometric fingerprint of a person's voice for future capture and comparison. One example of the use of multiple microphones is to use microphones from a cut off cell phone to reproduce the exact electrical and acoustic characteristics of the human voice. This sample is stored in the database for future comparisons. If a human voice is later captured, the previous sample may be compared and reported to the eyepiece user when the acoustic characteristics of the two samples match.

Figure 44 shows the use of an adaptive array to improve audio data acquisition. By modifying existing algorithms for audio processing, an adaptive array can be created that allows the user to adjust the directivity of the antenna in three dimensions. The adaptive array processing allows to find the source of the voice, and thus to associate the captured audio data with a specific person. The array processing allows simple summation of the cardioid elements of the signal to be done digitally or using analog techniques. In normal use, the user must switch the microphone between the omnidirectional pattern and the directional array. The processor allows beamforming, array adjustment and adaptive array processing to be performed on the eyepiece. In embodiments, an audio phased array may be used for audio tracking of a particular person. For example, a user can lock onto an audio signature of a person in the environment (e.g., obtained in real time or from a database of sound signatures) It is possible to track the position of a person without having to keep it or the user needing to move his head. The position of a person can be projected to the user through an eyepiece display. In embodiments, tracking of a person may also be provided via a camera embedded in the eyepiece, where the user will not need to keep an eye contact with a person or move his head and follow it. That is, in the case of audio tracking or visual tracking, the eyepiece can track a person in the local environment without the user having to show physical movements to indicate that tracking is being done, and even when the user moves his or her viewing direction.

In one embodiment, the integrated camera may continue to record the video file, and the integral microphone may continue to record the audio file. The integrated processor of the eyepiece can enable event tagging in long sections of continuous audio or video recording. For example, a passive recording of a day can be tagged whenever an event, conversation, encounter, or other item of interest occurs. Tagging can be accomplished through explicit pushing of the button, sound or physical tapping, hand gesture, or any other control technique described herein. A marker may be located in an audio or video file, or it may be stored in a metadata header. In embodiments, the marker may include GPS coordinates of an event, conversation, encounter, or other item of interest. In other embodiments, the marker may be time synchronized with the GPS log of the day. Other logic-based triggers, such as proximity relationships to other users, devices, locations, etc., can also tag audio or video files. The event tag includes an active event tag that is manually triggered by the user, a passive event tag that is automatically (e.g., via preprogramming, through event profile management facilities, etc.) A location-sensitive tag triggered by the user's location, and so on. Events that trigger event tags can be triggered by sounds, scenes, visual markers (received from a network connection), optical triggers, sound triggers, proximity triggers, time triggers, geo-spatial triggers, Event triggers can be triggered by generating feedback (audio tones, visual indicators, messages, etc.) to the user, storing information (e.g., storing files, documents, entries in lists, audio files, video files, Information transmission can be generated, and so on.

In one embodiment, an eyepiece can be used as SigInt Glasses. Using one or more of the integrated WiFi, 3G, or Bluetooth wireless units, the eyepiece can be used to clearly and passively collect signal intelligence for devices and people in the vicinity of the user. Signal information can be automatically collected, triggered on others when a particular device ID is in proximity, when a particular audio sample is detected, when a specific geographic location is reached.

Various embodiments of tactical glasses may be used to identify the POI's geographical location with visual anatomy (face, iris, gait) at a safe distance and to identify the POI with a robust sparse recognition algorithm for faces and irises Lt; RTI ID = 0.0 > or < / RTI > The glasses include a hands free display in which the bio-computer interface merges the fingerprint and visual biometrics onto one comprehensive display with augmented target highlighting and sees matches and alerts without informing the POI . The glasses may include location awareness such as displaying the current and average speed and path to the destination and the ETA and preloading or recording the trouble spot and ex-filtration route . The glasses are used to achieve a visual separation range between the friendly and the enemy, to know where the ally is, and to monitor the enemy's geographical location and to share his location in real time. ≪ / RTI > The processor associated with the glasses may include OCR translation and voice translation capabilities.

Augmented reality data relating to such things as direction and team member location data, map information of the area, SWIR / CMOS night vision, vehicle S / A for a soldier, etc. is provided to the user via a graphical user interface projected onto the lens, Geo-location to identify geolocations of POIs or targets to more than 500m with location accuracy To provide laser rangefinder, S / A friendly range ring, Domex registration, AR field overlay, and real-time UAV video Tactical glasses can be used in combat. In one embodiment, the laser range finder may be an eye-safe 1.55 micrometer laser range finder.

As described herein, to provide position and orientation accuracy, the eyepiece may utilize GPS and inertial navigation (e.g., using an inertial measurement unit) as described herein. However, the eyepiece may employ additional sensors and associated algorithms to improve position and orientation accuracy, e.g., with a three-axis digital compass, inclinometer, accelerometer, gyroscope, and the like. For example, military operations may require higher positional accuracy than is available from GPS, and therefore other navigation sensors may be used together to improve the positional accuracy of the GPS.

Tactical glasses can feature improved resolution, such as 1280 x 1024 pixels, and can also feature autofocus.

In dismounted occupation engagements, winning low-intensity, low-density, asymmetric combat depends on efficient information management. The tactical eyewear system includes ES2 (all soldiers are sensors) functionality through non-collaborative data recording and intuitive tactical display of a comprehensive picture of situational awareness.

In embodiments, the tactical glasses may include one or more waveguides integrated into the frame. In some embodiments, the total internal reflection lens is attached to ballistic glasses in a monocular or binocular flip-up / flip-down configuration. The tactical glasses may include an omnidirectional earphone for improved listening and protection and a noise-canceling boom microphone for delivering spoken commands.

In some embodiments, the waveguide may have a contrast control. Contrast may be controlled using any of the control techniques described herein, such as gesture control, automatic sensor control, manual control using a spectacle foot mounted controller, and the like.

The tactical glasses may include a non-slip adjustable resilient head strap. The tactical glasses may include a clip-in correction lens.

In some embodiments, the total internal reflection lens is attached to the helmet-mounted device, as in FIG. 74, and may include a day / night VIS / NIR / SWIR CMOS color camera. This device enables the unobtrusive "sight" of the soldier's own weapons, as well as threats, by means of a "perspective" flip-up electro-optical projector image display. The helmet-mounted device shown in FIG. 74A includes a IR / SWIR illuminator 7402, a UV / SWIR illuminator 7404, a visible to SWIR panoramic lens 7408, a visible to SWIR objective (not shown) Processor, or technology described in connection with the eyepiece described herein, or any other sensor, processor, or technique described in connection with the eyepiece described herein For example, integrated IMU, eye-safe laser rangefinder, integrated GPS receiver, compass and inclinometer for position accuracy, point-of-view control to change the viewing angle of the image to match the eye position, electronic image stabilization and real- A library of threats stored remotely for access via a network, etc.). A body-worn wireless computer can interface with the device of Fig. The helmet-mounted device includes a visible to SWIR projector optical system such as an RGB micro projector optical system. Multispectral IR and UV imaging helps identify forged or altered documents. The helmet-mounted device can be controlled by an encrypted wireless UWB wrist or a weapon front grip controller.

In one embodiment, the transparent monitoring window 7410 may be rotated 180 degrees to project an image on the surface for sharing with others.

74B shows a side view of the disassembled device mounted on the helmet. The device may include a fully ambidextrous mount for mounting on the left or right side of the helmet. In some embodiments, two devices may be mounted on each of the left and right sides of the helmet to enable binocular vision. The device or devices can be snap-on to a standard MICH or PRO-TECH helmet mount.

Currently, combatants can not effectively use fielded data devices. Tactical eyewear systems combine lightweight materials and high-speed processors in a low profile form for quick and accurate determination in the field. The modular design of the system allows the device to be interfaced with any fielded computer, allowing it to be effectively installed on an individual, part, or company. The tactical glasses system includes real-time distribution of data. The onboard computer interface allows the operator to view, upload, or compare data in real time. This provides valuable situations and environmental data that can be quickly distributed to the CP (command post) and TOC (tactical operations center) as well as to all networked people.

75A and 75B are front and side views, respectively, of an exemplary embodiment of a living body and situational awareness glasses. This embodiment includes a plurality of field of view sensors 7502 for biometric acquisition, situational awareness and augmented view user interface, a fast locking system including a 3-axis digital compass for location and directional accuracy, gyroscopes, accelerometers and inclinometers, An integrated digital video recorder that stores a library of threats stored on an onboard Mini SD card or remotely loaded via a tactical network, a GPS receiver and IMU, biometric capture and two flash SD cards, real-time electronic image stabilization and real- Eye-safe 1.55 micrometer laser rangefinder (7504), flip-up photochromic lens 7508, noise-canceling boom microphone 7510, and three-axis detachable stereo earphone and augmented listening and And a protection system 7512. For example, a plurality of field of view sensors 7502 may enable a 100 ° x 40 ° FOV, which may be a panoramic SXGA. For example, the sensor may be a VGA sensor that generates a panoramic SXGA view with a VGA sensor, an SXGA sensor, and a stitched 100 DEG x 40 DEG FOV on the display of the glasses. The display may be translucent and has perspective control to change the viewing angle of the image to match the eye position. This embodiment may also include SWI detection to allow the wearer to see a 1064 nm and 1550 nm laser laser designator that is invisible to the enemy and may include an ultra low power 256 bit AES encrypted connection between the glasses, It can feature instantaneous 2x zoom, automatic face tracking, face and iris recording, and recognition with 1m automatic recognition range and GPS geolocation. This embodiment may include a power supply such as a 24-hour continuous 4-AA alkaline, lithium and rechargeable battery box, and its computer and memory expansion slots have a waterproof and dustproof cord. In one embodiment, the glasses include a curved holographic waveguide.

In embodiments, the eyepiece may sense a laser such as that used in battlefield targeting. For example, sensors in an eyepiece can detect laser light in a conventional military laser transmission band such as 1064 nm, 1550 nm, and the like. In this way, the eyepiece can detect whether its position is being aimed, whether another position is being aimed at, the position of a scout using a laser as a collimating assistant, and the like. Furthermore, since the eyepiece can sense laser light directly or reflected, the soldier can not only detect an enemy laser light source that is directed or reflected at his position, but also finds an optical surface (e.g., binoculars) The laser light source itself can be supplied. For example, a soldier scans a battlefield with a laser and monitors it with an eyepiece to see if there is a reflected return of the laser as an enemy's possible position seen through the binoculars. In embodiments, the eyepiece may continually scan the surrounding environment to see if there is laser light and provide feedback and / or motion (e. G., An audible alert to a soldier, a position indicated by a visual indicator on the eyepiece display, etc.) as feedback .

In some embodiments, a pocket camera allows video recording and capture of still pictures, thereby recording environmental data for analysis with a lightweight robust mobile biometric device sized for the operator to enter the pocket . An embodiment may be 2.25 " x 3.5 "x .375 ", capture a face at 10 feet, capture an iris at 3 feet, record voice, pockets, gaits, Visual mark and environmental data can be recorded in EFTS and EBTS compatible format settings compatible with any iris / face algorithm. The device can be any biometric matching software or EFTS / EBTS / NIST / ISO / ITL 1 -2007 Compatibility This device is designed to pre-qualify and capture critical images, including high-definition video chips, 1GHz processor with 533Mhz DSP, GPS chip, active illumination and pre- In some embodiments, the Pocket Bio Cam may not include a biopsy list, and thus may contain all of the echelon (s) Data can be automatically geo-located and date / time stamped. In some embodiments, the device may be configured to operate a Linux SE OS , Meets the MIL-STD-810 environmental standard, and can be waterproofed to a depth of 3 feet (about 1 m).

In one embodiment, the fingerprint collection device may be known as a bio-print device. The bioprint device includes a clear platen with two beveled edges. The platen is illuminated by a bank of LEDs and one or more cameras. A number of cameras are used and densely arranged and oriented at the oblique edge of the platen. A finger or palm is placed on the platen and pressed against the top surface of the platen, where the camera captures a ridge pattern. This image is recorded using FTIR (frustrated total internal reflection). In FTIR, the light exits the platen through a finger or palm ridge pressed against the platen and a gap created by the valley.

Other embodiments are possible. In one embodiment, a plurality of cameras are arranged in inverted " V " In another embodiment, a rectangle is formed and light passing through one side is used, and an array of cameras captures the generated image. The light enters the rectangle through the sides of the rectangle while the camera is directly below the rectangle so that the camera can capture the ridges and valleys illuminated by the light passing through the rectangle.

After the image is captured, software is used to stitch the images from multiple cameras together. A custom FPGA can be used for digital image processing.

Once captured and processed, the images may be streamed to a remote display such as a smart phone, computer, handheld device, or eyepiece, or other device.

The above description provides an overview of the operation of the method and apparatus of the present disclosure. Additional descriptions and discussion of these and other embodiments are provided below.

45 illustrates the configuration and layout of an optics-based fingerprint and long-range system according to an embodiment. The optical array consists of about 60 wafer scale cameras 4502. The optical system based system uses sequential perimeter illumination 4503 and 4504 for high resolution imaging of the wedge and pore that make up the fingerprint or the long pass. This configuration provides a lightweight, extremely robust configuration of low profile. Scratch proof, transparent platen improves durability.

The mosaic fingerprint sensor uses a frustrated total internal reflection (FTIR) optical faceplate that provides an image to an array of wafer scale cameras mounted on a substrate 4505, such as a PCB. This sensor can be scaled to any flat width and length having a depth of about 1/2 ". From a plate small enough to capture only one finger roll fingerprint, to a plate large enough to simultaneously capture both hand fingerprints Size can vary.

The Mosaic fingerprint sensor allows the operator to capture fingerprints and compare the collected data against the onboard database. Data can also be uploaded and downloaded wirelessly. The unit may operate as a stand-alone unit or may be integral with any biometric system.

In operation, the mosaic fingerprint sensor provides high reliability in harsh environments with excessive sunlight. To provide this capability, multiple wafer scale optical sensors can be digitally connected together using pixel subtraction. The resulting image is made to be over 500 dpi (dots per inch). Power is supplied by a battery or by using a USB protocol to parasitically draw power from another power source. The format settings are compatible with EFTS, EBTS NIST, ISO, and ITL 1 -2007.

Figure 46 shows a conventional optical scheme used by other sensors. This method is also based on frustrated total internal reflection (FTIR). In this figure, a fringe contacts the prism and scatters light. The camera captures scattered light. The fringes on the fingers are represented by dark lines, and the fingerprints are represented by bright lines.

Fig. 47 shows a method used by the mosaic sensor 4700. Fig. Mosaic sensors also use FTIR. However, the plate is illuminated from the side and the internal reflection is limited into the plate of the sensor. The fringe of the fingerprint on which the image is being imaged, shown at the top of the figure, contacts the prism and scatters the light, allowing the camera to capture the scattered light. The fringes on the fingers are represented by bright lines, while the bones are represented by dark lines.

Fig. 48 shows the layout of the mosaic sensor 4800. Fig. An array of LEDs is arranged around the periphery of the plate. The camera used to capture the fingerprint image is at the bottom of the plate. An image is captured on this bottom plate, called the capture plane. The acquisition plane is parallel to the sensor plane in which the finger is located. Depending on the size of the active capturing area of the plate, the thickness of the plate, the number of cameras, and the number of LEDs may vary. The thickness of the plate can be reduced by adding a mirror to bend the optical path of the camera to reduce the required thickness. Each camera must be responsible for one inch of space, and several pixels overlap between cameras. This allows the mosaic sensor to achieve 500 ppi. The camera may have a 60 degree field of view, but there may be significant distortion in the image.

Figure 49 shows an embodiment 4900 of a plurality of camera interactions used in camera view and mosaic sensors. Each camera is responsible for a small capture area. This area depends on the distance between the camera field of view and the top surface of the camera and the plate, where alpha is 1/2 of the horizontal field of view of the camera and beta is 1/2 of the vertical field of view of the camera.

As illustrated in FIG. 50, the mosaic sensor may be included in a bio-phone and a tactical computer. The biphone and tactical computer is a complete mobile computer architecture including a dual core processor, a DSP, a 3D graphics accelerator, a 3G-4G Wi-Lan (according to 802.11 a / b / g / n), Bluetooth 3.0, Lt; / RTI > Biophones and tactical computers deliver power equivalent to standard laptops in phone-sized packages.

Figure 50 shows the components of a bio-phone and a tactical computer. The biophone and tactical computer assembly 5000 provides a display screen 5001, a speaker 5002, and a keyboard 5003 that are contained within a case 5004. These elements are visible in front of the biophone and tactical computer assembly 5000. A camera 5005 for iris imaging, a camera 5006 for facial imaging and video recording, and a bio print fingerprint sensor 5009 are located behind the assembly 5000.

To provide secure communication and data transmission, the device includes 256 bit AES encryption, selectable by COTS sensors and software for biometric pre-qualification for POI acquisition. The software is matched and filed by any approved biometric matching software to transmit and receive secure "perishable" voice, video and data communications. In addition, BioPhone supports Windows Mobile, Linux, and Android operating systems.

BioPhone is a 3G-4G capable handheld device for reach back to a web portal and a biometric enabled watch list (BEWL) database. These databases enable in-field comparisons of captured biomedical images and data. The device is designed to fit into a standard LBV or pocket. In embodiments, the biometrics phone and tactical computer are dual core processors, DSPs, 3D graphics accelerators, 3G-4G, Wi-LAN (802.11 a / b / g / n), Bluetooth 3.0 (supported for security and private networks), GPS receiver, WVGA solar readable capacitive touch screen display capable of outputting stereoscopic 3D video, tactile backlit QWERTY keyboard, onboard storage, And the like. ≪ / RTI >

Biophones can search, collect, register, and verify many types of biometric data, including face, iris, 2-finger prints, as well as biographical data. The device also records video, audio, posture, identification marks, and items contained in the pocket. The items in the pocket may include items such as spare change, ID, passport, recharging card, etc., including various small items carried in a pocket, briefcase or purse. Figure 52 shows a typical set of this type of information. Examples of the object 5200 contained in a bag in a group are shown in Fig. Items of a type that may be included are documents such as personal documents and photos 5201, books 5202, notes and paper 5203, and passports 5204.

Biometric phones and tactical computers may include cameras such as high-definition still and video cameras capable of biometric data acquisition and video conferencing. In embodiments, an eyepiece camera and video conferencing functionality may be used in connection with a biometric phone and a tactical computer, as described herein. For example, a camera integrated in an eyepiece can capture an image and deliver the image to a biometric phone and a tactical computer, and vice versa. Data can be exchanged between the eyepiece and the biometric phone, the network connection can be set and shared by either one, and the like can be done. In addition, biometric phones and tactical computers can be housed in a robust, fully armed structure, resistant to a militarized temperature range, waterproof to a depth of, for example, 5 meters, and others.

51 shows an embodiment 5100 of using a bio-phone to capture a potential fingerprint and a long fingerprint. Fingerprints and long passages are captured at 1000 dpi by active illumination from an ultraviolet diode with a scale overlay. Both fingerprints and long passages 5100 can be captured using a biophone.

The data collected by the biophone is automatically geo-located and date and time stamped using the GPS function. Data can be uploaded or downloaded and compared against onboard or networked databases. This data transfer is facilitated by the 3G-4G, Wi-Lan and Bluetooth functions of the device. Data entry may be done by a QWERTY keyboard or other methods that may be provided (stylus or touch screen, etc.). Biometric data is filed using the most important images after collection. Manual input enables partial data capture. 53 shows an interplay 5300 between a digital document image and a biometric list maintained in a database. A biometric list is used to compare the data captured in the field with previously captured data.

Formatting can use EFTS, EBTS NIST, ISO, and ITL 1 -2007 formats to provide compatibility with a range of diverse databases for biometric data.

Specifications for biophones and tactical computers are given below:

Operating temperature: -22 ° C to + 70 ° C

Connectivity I / O: 3G, 4G, WLAN a / b / g / n, Bluetooth 3.0, GPS, FM

Connection output: USB 2.0, HDMI, Ethernet

Physical Dimensions: 6.875 "(H) x 4.875" (W) x 1.2 "(T)

Weight: 1.75 lbs

Processor: Dual-core - 1 GHz processor, 600 MHz DSP, and 30M polygons / second

         3D graphics accelerator

Display: 3,8 "WVGA (800 x 480) sunlight readable,

          Transflective capacitive touchscreen with 3x 1080p high definition screen

          Scalable display output for simultaneous connections

Operating Systems: Windows Mobile, Linux, SE, Android

Storage: 128 GB solid state drive

Additional storage: Dual SD card slots for additional 128 GB storage

Memory: 4 GB RAM

Camera: Three high-definition still and video cameras: face, iris and conference (face of the user)

3D support: can output stereoscopic 3D video

Camera sensor support: sensor dynamic range expansion, adaptive defect pixel correction,

           Advanced clarity enhancement, geometric distortion correction, advanced management,

           Hardware based face detection, video stabilization

Biometrics: onboard optics, 2 fingerprint sensors, face, DOMEX and iris cameras

Sensors: can accommodate the addition of accelerometers, compasses, ambient light, proximity, air pressure and temperature sensors, depending on requirements

Battery: <8 hours, 1400 mAh, charged Li-ion, hot swap battery pack

Power: Various power options for continuous operation

Software features: powerful display processor with face / gesture detection, noise filtering, pixel-correction multi-overlay, rotation and scaling

Audio: Onboard microphone, speakers, and audio / video inputs

Keyboard: Full-haptic QWERTY keyboard with adjustable backlight

Additional devices and kits may also include mosaic sensors and may operate in conjunction with biophones and tactical computers to provide a complete on-site solution to the collected biometric data.

One such device is the pocket biocide illustrated in FIG. The components of the pocket biotic kit 5400 include a GPS antenna 5401, a biometric sensor 5402, and a keyboard 5404, all of which are contained in a case 5403. The specifications of the bio-kit are given below:

Size: 6 "x 3" x 1.5 "

Weight: 2 pounds total

Processor and memory: 1 GHz OMAP processor

       650 MHz core

       3D accelerator for up to 18 M polygons / second

       64 KB L2 cache

       166 MHz at 32-bit FSB

       1 GB internal PoP memory, expandable up to 4 GB NAND

       64 GB solid state hard drive

Display: 75 mm x 50 mm, 640 x 480 (VGA) daylight readable LCD, anti-glare, anti-reflective, anti-scratch screen

Interface: USB 2.0

     10/100/1000 Ethernet

Power: Battery operation: Approximately 5 minutes per registration, 8 hours continuous registration

Built-in function: Mosaic sensor Optical fingerprint reader

      Digital iris camera with active IR illumination

      Digital face and DOMEX camera with flash (visible)

      High-speed synchronous GPS

The features of the bio-phone and the tactical computer can also be provided in a bio-kit that provides a biometric data collection system that is folded into a robust and compact case. Data is collected in biometric standard images and data formats that can be cross-referenced for near-real-time data communication with the Department of Defense Biometric Authoritative Databases.

The pocket biotic kit shown in Fig. 55 can capture potential fingerprints and long passages at 1,000 dpi by active illumination from ultraviolet diodes having a scale overlay. The BioKit has a 32 GB memory storage card that can be interfaced with a combat radio or computer for uploading and downloading data in real-time field conditions. The power is provided by a lithium ion battery. The components of the bio-kit assembly 5500 include a GPS antenna 5501, a bio-print sensor 5502, and a case 5503 having a base bottom 5505.

Biometric data collection is geo-located to monitor and track individual movements. Fingerprints and long passages, iris images, facial images, potential fingerprints and video can be collected using a bio-kit and registered in the database. Algorithms for fingerprints and long passages, iris images, and facial images facilitate this type of data collection. To help capture iris images and potential fingerprint images at the same time, the biokit has IR and UV diodes that actively illuminate the iris or potential fingerprint. In addition, the PocketBioKit is also fully EFTS / EBTS compatible, including ITL 1 -2007 and WSQ. The bio kit meets MIL-STD-810 to operate in extreme environments and uses the Linux operating system.

To capture the image, the bio-kit uses a high dynamic range camera with wave front coding for maximum depth of field to capture the details of the potential fingerprint and iris image Is captured. Once captured, real-time image enhancement software and image stabilization function to improve readability and provide superior visual discrimination.

The BioKit can also record video and store full motion (30 fps) color video on-board "camcorder on chip".

The eyepiece 100 is a biometric data collection system that is folded into a compact and robust case and thus exposes to a mini workstation for fingerprints, iris and face recognition, potential fingerprints, and other biometric data, as described herein, And can interface with a folded biometric registration kit (also called bio kit) 5500. As with other mobile biometric devices, the mobile fold biometric registration kit 5500 may be used as a standalone device or in conjunction with the eyepiece 100, as described herein. In one embodiment, the mobile foldable biometric registration kit can be folded into smaller sizes, such as 6 "x 3" x 1.5 ", and has a weight of two pounds, etc. It is based on a processor, digital signal processor, 3D accelerator, Hard-drive, display (e.g., 75 mm x 50 mm, 640 x 480 (VGA) daylight readable LCD anti-glare, reflective Anti-scratch screen), USB, Ethernet, built-in battery, mosaic optical fingerprint reader, digital iris camera (with active IR illumination, for example), digital face with flash and DOMEX camera, Data can be collected in biometric standard images and data formats that can be cross-referenced for near-real-time data communication with the Department of Defense's vital database. The biometric data and geographic location of a person of interest may be collected to monitor and track wireless data upload / download using a combat radio or computer with a king interface.

In addition to the biocide, a mosaic sensor may be included in the wrist-mounted fingerprint, long-range, geographical location and POI registration device shown in FIG. The eyepiece 100 includes a biometric device 5600, which is a biometric data collection system worn on the wrist or arm of a soldier and folded open for fingerprint, iris recognition, computer and other biometric data, as described herein You can interface. The device may have an integral computer, a keyboard, a solar readable display, a biometric sensor, etc., so that the operator can quickly or remotely store or compare data for collection and identification. For example, an arm strap biometric sensitive platen may be used to scan long passages, fingerprints, and the like. The device can provide geographic location tags for people of interest and collected data by time, date, location, and so on. As with other mobile biometric devices, the biometric device 5600 may be used as a standalone device or in conjunction with the eyepiece 100, as described herein. In one embodiment, the biometric device may be small and lightweight (e.g., the dimensions are 5 "x 2.5" for active fingerprint and long-range sensors and 16 ounces for weight) so that it can comfortably be worn on a soldier's arm. There can be fingerprint and long fingerprint capture algorithms. The device may include a processor, a digital signal processor, a transceiver, a Qwerty keyboard, a large weatherproof pressure-activated fingerprint sensor, a solar readable transflective QVGA color backlit LCD display, an internal power supply, and the like.

In one embodiment, the wrist-mount assembly 5600 includes the following elements in a case 5601: a strap 5602, a setting and on / off button 5603, a protective cover for the sensor 5604, A pressure-driven sensor 5605, and a keyboard and LCD screen 5606.

The fingerprint, the long pass, the geographical location, and the POI registration device include an integrated computer, a QWERTY keyboard, and a display. The display is designed to allow easy manipulation in strong sunlight and uses an LCD screen or LED indicator to notify the operator of successful fingerprint and long-distance capture. The display uses a transflective QVGA color and has a backlit LCD screen to improve readability. The device is compact and lightweight, weighs 16 ounces and is 5 "x 2.5" in size with a mosaic sensor. This small size and weight allows the device to be slid into the LBV pocket or worn on the forearm of the user, as shown in FIG. As with other devices, including mosaic sensors, all POIs are tagged with geo-location information at acquisition.

The size of the sensor screen enables ten fingers, palms, four fingers, and fingertip capture. The sensor includes a large pressure-driven print sensor for fast registration at a rate of 500 dpi in any weather conditions as specified in MIL-STD-810. Software algorithms support both fingerprint and long capture modes and use the Linux operating system for device management. The acquisition is fast because of the 720 MHz processor with 533 MHZ DSP. This processing capability delivers critical images of the correct format to any existing approved system software. In addition, the device is also fully EFTS / EBTS compatible, including ITL 1 -2007 and WSQ.

As with other mosaic sensor devices, communication in wireless mode is possible using a mobile UWB wireless 256 bit AES transceiver. It also provides secure uploading and downloading to / from a biometric database stored off-site.

The power is supplied using a lithium polymer or AA alkaline battery.

The wrist-mounted device described above may also be used in conjunction with other devices, including an augmented reality eyepiece having the data and video display shown in Fig. Assembly 5700 includes the following components: eyepiece 5702 and bioprint sensor device 5700. Augmented Reality eyepieces provide redundant binocular stereoscopic sensors and displays and provide the ability to view them in a variety of lighting conditions from the brightest day in the sun to extremely low light levels at night. The manipulation of the eyepiece is simple by a rotary switch located at the eyepiece of the eyepiece where the user can access data from the forearm computer or sensor or laptop device. The eyepiece also provides omnidirectional earphones for hearing protection and enhanced listening. A noise canceling boom microphone may also be integrated into the eyepiece to provide better transmission of the voice-coded commands.

The eyepiece can communicate wirelessly with biophone sensors and forearm-mounted devices using 256-bit AES-encrypted UWB. It also allows devices to communicate with laptops or combat radios, as well as networked CP, TOC and biometric databases. The eyepiece is ABIS, EBTS, EFTS, and JPEG 2000 compatible.

Similar to the other mosaic sensor devices described above, the eyepiece uses an RF filter array as well as networked GPS to provide a very precise geographic location of the POI.

In operation, the low profile forearm-mounted computer and tactical display incorporate face, iris, fingerprint, long finger, and fingertip acquisition and identification. The device also records video, voice, pose, and other characteristic features. Face and iris tracking is automatic, thereby allowing the device to help recognize non-cooperative POIs. With the transparent display provided by the eyepiece, the operator can also capture biometric data, as well as navigational, target selection, positional or other information superimposed applications and data from sensor images, moving maps, sensors, UAVs You can see the person or other target / POI.

58 shows a further embodiment of a fingerprint, a long text, a geographic location and a POI registration device. It is 16 ounces (about 450 g) and uses a 5 "x 2.5" active fingerprint and long-range capacitance sensor. The sensor can register a thermal fingerprint, a long finger, a four finger press fingerprint, and a finger tip fingerprint at 500 dpi. A 0.6 to 1 GHz processor with a 430 MHz DSP provides fast registration and data capture. This device is compatible with ABIS, EBTS, EFTS, and JPEG 2000, and features networked GPS for highly accurate positioning of people of interest. In addition, the device communicates wirelessly over a 256 bit AES encrypted UWB laptop or combat radio. Database information can also be stored on the device, thereby enabling field comparisons without uploading information. The onboard data can be wirelessly shared with other devices (such as a laptop or combat radio).

A further embodiment of a wrist-mounted bioprint sensor assembly 5800 includes the following elements: a bioprint sensor 5801, a wrist strap 5802, a keyboard 5803, and a combat radio connector interface 5804, .

The data can be stored in the upper device because the device can use the Mil-con data storage cap for increased storage capacity. Data entry is done on a QWERTY keyboard and can be done with gloves on.

The display is a transflective QVGA color backlit LCD display designed to be readable under sunlight. In addition to operation in strong sunlight, the device can be operated in a wide variety of environments, since it meets the requirements of MIL-STD-810 operation in extreme environments.

The above-described mosaic sensor can also be included in the mobile folding biometric registration kit, as shown in Fig. The mobile folding biometric registration kit 5900 is sized to be folded and into a tactical vest pocket and has a dimension of 8 "x 12" x 4 "when unfolded.

Figure 60 illustrates one embodiment (6000) of how an eyepiece and forearm-mounted device can interface to provide a complete system for biometric data collection.

61 provides a system diagram 6100 for a mobile folded biometric registration kit.

Describing the operation, the mobile foldable biometric registration kit allows the user to search, collect, identify, verify and register face, iris, long finger, fingertip, and personal information for a subject, Goods, and other visible identification marks may be recorded. Once collected, the data is automatically geo-located and stamped with date and time. The collected data can be retrieved and compared against an on-board database and a networked database. Wireless data upload / download using a combat radio or laptop computer with a standard networking interface is provided to communicate with a database that is not on the device. Format settings are compatible with EFTS, EBTS, NIST, ISO, and ITL 1 - 2007. Because the device can use any matching and registration software, the pre-verified images can be sent directly to matching software.

Devices and systems, including those described above, provide a comprehensive solution for mobile biometric data collection, identification, and situational awareness. The device can collect fingerprints, long, fingertip, face, iris, voice, and video data for the recognition of non-cooperative person of interest (POI). The video is captured using high speed video to enable capture in unstable situations (e.g., from moving video). The captured information can be easily shared, and additional data can be input through the keyboard. In addition, all data is tagged with date, time, and geographic location. This facilitates the rapid dissemination of information needed for context awareness in potentially hostile environments. Additional data collection is possible by the fact that more people are equipped with the device (hence the concept "all soldiers are sensors"). The sharing is facilitated by the integration of the biometric device with the combat radio and battlefield computer.

In embodiments, the eyepiece may utilize a flexible thin film sensor, such as one integrated into the eyepiece itself, with an external device that interfaces with the eyepiece. The thin film sensor may include a thin multilayered electromechanical arrangement that generates an electrical signal when subjected to sudden contact forces or continuously varying forces. A typical application of electromechanical thin film sensors utilizes both on-off electrical switching sensing and time-resolved sensing of force. The thin film sensor may comprise a switch, a force gauge, etc., wherein the thin film sensor is designed to operate in response to a sudden electrical contact (switching), a gradual change in electrical resistance under the action of a force, Gradual release, and the generation of gradual electromotive force across the conductor when moving in a magnetic field. For example, a flexible thin film sensor can be used in a force-pressure sensor having a micro-force sensitive pixel for a two-dimensional force array sensor. This includes touch screens for computers, smart phones, laptops, MP3-like devices, especially those with military applications; Such as an unmanned aerial vehicle (UAV), a drones, a mobile robot, an exoskeleton-based device, or the like, which controls anything under computer control. Thin film sensors may be useful in security applications, for example, in remote or local sensors that detect the opening or closing of intrusions, devices, doors, windows, equipment, and the like. Thin film sensors may be useful for trip wire detection, such as in electronic devices and radios used in silent remote trip wire detectors. Thin-film sensors can be used in opening and closing detection, such as force sensors that detect strain-stress in vehicle cabins, ship hulls, airplane panels, and the like. Thin-film sensors can be useful as biometric sensors in fingerprinting, palm-printing, fingertip printing, and the like. Thin film sensors can be useful for leak detection, such as detecting leak tanks and storage facilities. Thin film sensors may be useful in medical sensors, such as detecting liquid or blood outside the body. These sensor applications are intended to illustrate, but in no way limit, many applications in which thin film sensors may be used in connection with the control and monitoring of external devices through an eyepiece.

62 shows an embodiment 6200 of a thin film fingerprint and long-range collection device. The device can record four finger presses and rolled fingerprints, long passages, and fingerprints in accordance with NIST standards. Good quality fingerprint images can be captured with wet or dry hands. This device has reduced weight and power consumption compared to other large sensors. In addition, the sensor is self-contained and hot-swappable. The configuration of the sensor can be varied to suit various needs, and the sensor can be manufactured in various shapes and sizes.

63 shows one embodiment 6300 of a finger, palm, and registration data collection device. The device records fingertips, rolls, presses, and long passages. The built-in QWERTY keyboard allows entry of handwriting registration data. All data is tagged with the date, time, and geographic location of the collection, as in the device described above. The embedded database provides onboard matching of potential POIs against the embedded database. Matching can also be performed for other databases via the battlefield network. The device may be integrated with the optical bio-collecting eyepiece described above to support face and iris recognition.

Specifications for fingers, palms, and registration devices are given below:

Weight and size: insert in 16-ounce biceps strap or LBV pocket

     5 "x 2.5" fingerprint / long range sensor

     5.75 "x 2.75" QWERTY keyboard

     3.5 "x 2.25" LCD display

     One-handed motion

Environment: The sensor operates at all weather conditions (-20 ° C to + 70 ° C).

Waterproof: operates without deterioration for 4 hours at 1m

Biometric collection: fingerprint and long collection, identification

      Keyboard and LCD display for registration of POI

     Maintains over 30,000 full template portfolios (2 irises, 10 fingerprints, face images, 35 fields of personal information) for onboard matching of POIs

     Tag all collected biometric data by time, date, and location

     Pressure Capacitance Fingerprint / Long Range Sensor

     30 fps high contrast bitmap image

     1000 dpi

Wireless: Fully interoperable with combat radios, handheld or laptop computers and 256 bit AES encryption

Battery: Dual 2000 mAh lithium polymer battery

      Fast rechargeable battery with over 12 hours, less than 15 seconds

Processing and Memory: 256 MB Flash and 128 MB SDRA support three SD cards, each with up to 32 GB.

     600-1 GHZ ARM Cortex A8 processor

     1 GB RAM

Figures 64 to 66 illustrate the use of an apparatus including a sensor for collecting biometric data. Figure 64 shows an embodiment 6400 of a two step long pass. 65 shows a collection 6500 using fingertip tapping. FIG. 66 shows an embodiment 6600 in which a pressed fingerprint and a rolled fingerprint are collected.

The discussion above relates to a method for collecting biometric data such as fingerprints or long passages using a platen or a touch screen, as shown in Figs. 66 and 62-66. The present disclosure also includes a touchless or non-contact fingerprinting method and system using polarized light. In one embodiment, a fingerprint can be obtained by the person using a polarized light source and retrieving the image of the fingerprint using the polarized light reflected from the two planes. In another embodiment, a fingerprint may be generated by a person using a light source and using multispectral processing (e.g., using two imagers at two different positions with different inputs) Can be acquired. Different inputs may be caused by using different filters or different sensor / imagers. The application of this technique may include biopsies of unknown persons or subjects whose safety of the person performing the examination may be a problem.

In this way, an unknown person or subject may have access to the checkpoint, for example, to be authorized for further travel to his destination. As shown in the system 6700 shown in Fig. 67, the person P and the appropriate body part (e.g., the hand, palm P, or other part) are illuminated by the light source 6701 of polarized light. As will be appreciated by those skilled in the optical arts, the light source of polarized light may simply be a lamp or other illuminating light source having a polarizing filter to emit light polarized in one plane. The light travels to the person in the area defined for the non-contact fingerprinting, so that the polarized light impinges on the fingers of the person P or other body parts. The incident polarized light is then reflected from fingers or other body parts and travels in all directions from the person. Two imagers or cameras 6704 receive the reflected light after the light has passed through the optical elements, such as lens 6702 and polarizing filter 6703. As discussed above with respect to Figure 8f, the camera or imager may be mounted on augmented reality glasses

The light then travels from the palm or finger or fingers of a person of interest to two different polarizing filters 6704a, 6704b and then to an imager or camera 6705. [ The light passing through the polarizing filter may have a 90 ° orientation difference (horizontal and vertical) or other orientation differences (30 °, 45 °, 60 ° or 120 °, etc.). The camera may be a digital camera with a suitable digital imaging sensor to convert the incident light into an appropriate signal. The signal is then processed by an appropriate processing circuit 6706, such as a digital signal processor. The signals can then be combined in a conventional manner by a digital microprocessor 6707, etc., having memory. A digital processor with the appropriate memory is programmed to generate appropriate data for the palm, fingerprint image or other image as desired. The digital data from the imager can then be combined in this process using, for example, techniques of U. S. Patent No. 6,249, 616 and other patents. As discussed earlier in this disclosure, the combined "image" can then be examined against a database to verify the identity of a person. The augmented reality glasses may include such a database in memory or may refer to signal data elsewhere 6708 for comparison and inspection.

A process for acquiring a non-contact type fingerprint, a long fingerprint or other biometric fingerprint is shown in the flowchart of FIG. In one embodiment, a polarized light source of polarized light is provided (6801). In a second step 6802, the person of interest and the selected body part are placed for light illumination. In other embodiments, it may be possible to use incident white light rather than using a light source of polarized light. When the image is ready to be photographed, light is reflected from the person to two cameras or imagers (6803). A polarizing filter is positioned in front of each of the two cameras, so that light received by the camera is polarized 6804 in two different planes (e.g., in horizontal and vertical planes). Each camera then detects the polarized light (6805). The camera or other sensor then converts the incidence of light into signals or data suitable for the preparation of the image (6806). Finally, the images are then combined (6807) to form a very distinct and reliable fingerprint. The result is a very high quality image that can be compared with a digital database to identify people and detect people of interest.

Although digital cameras are used in this non-contact system, it is well known that other imagers such as active pixel imagers, CMOS imagers, imagers imaging at multiple wavelengths, CCD cameras, photodetector arrays, TFT imagers, . It will also be appreciated that although polarized light is used to produce two different images, other variations of the reflected light may also be used. For example, rather than using polarized light, white light can be used, and different filters such as a Bayer filter, a CYGM filter, or a GBE filter can then be applied to the imager. In other embodiments, it may be possible to remove the light source of polarized light and instead use natural or white light rather than a light source of polarized light.

As evidenced by previous systems, the use of touchless or non-contact fingerprinting has been developed some time ago. For example, U.S. Patent Application No. 2002/0106115 used polarized light in a non-contact system, but required a metal coating on the finger of the person being fingerprinted. Later systems such as those described in U.S. Patent No. 7,651,594 and U.S. Patent Application Publication No. 2008/0219522 required contact with a platen or other surface. The non-contact system described herein does not require contact during imaging and does not require prior contact (e.g., placing a coating or reflective coating on a body part of interest). Of course, the position of the imager or camera relative to each other must be known for easier handling.

To illustrate its use, non-contact fingerprint systems can only be used at checkpoints such as entrances, building entrances, roadside checkpoints or other convenient locations. Such a place may be where it is desirable to allow someone's position and refuse entry or even detain a person of other interest. Indeed, the system can use an external light source such as a lamp, if polarized light is used. A camera or other imager used for non-contact imaging may be mounted on the opposite side of a set of augmented reality glasses (for one person). For example, two versions of the camera are shown in FIG. 8F, and two cameras 870 are mounted on the frame 864. In this embodiment, at least software for processing images may be included in the memory of the augmented reality glasses. As an alternative, digital data from the camera / imager can be sent to the nearby data center for proper processing. This processing may include combining digital data to form an image of the fingerprint. This process may also include examining a database of known people to determine whether the subject is a person of interest.

Other non-contact fingerprinting methods use non-contact scanning of fingers and hands to use quantum dot lasers to detect extremely low explosive compounds in parts per billion (ppb) or even parts per trillion (ppt) concentrations as well as drug compounds. For example, in order to prevent contamination from one subject to another, quantum dots or other types of lasers, laser arrays are mounted behind the bio-phone or in the frame of the glasses to detect by very close proximity, rather than by touching Can be. Thus, in addition to being able to collect biometric data relating to irises, fingerprints, faces, and voices, glasses or other accessory devices may also be able to collect explosive or drug contaminant IDs.

Alternatively, as shown in camera 858 of Figure 8f, one camera may be used for each of the two people. In this configuration, two persons may be relatively close together, and therefore their respective images will be reasonably similar to be combined by appropriate software. For example, two cameras 6705 in Fig. 67 may be mounted on two different augmented reality glasses (for example, two soldiers placed at checkpoints). Alternatively, the camera may be mounted on a wall or on a stationary part of the checkpoint itself. The two images can then be combined by a remote processor 6707 (such as a computer system at a building checkpoint) with a memory.

As discussed above, people using augmented reality glasses can always be in contact with each other through at least one of many wireless technologies, especially when all of them are working at checkpoints. Accordingly, data from one camera or from two camera versions can be sent to the data center or another command post for proper processing, followed by matching of longs, fingerprints, iris prints, etc. Check the database. The data center can conveniently be located near the checkpoint. Due to the availability of modern computers and storage devices, the cost of providing multiple data centers and updating software wirelessly will not be a major cost consideration in such systems.

The touchless or non-contact biometric data collection discussed above may be controlled in several ways, such as the control techniques discussed elsewhere in this disclosure. For example, in one embodiment, a user may initiate a data collection session by pressing a touchpad on a pair of glasses or by providing a voice command. In another embodiment, a user may initiate a session using hand movements or gestures, or any of the control techniques described herein. Any of these techniques may cause a menu to be displayed from which the user can select options such as "start data collection session", "end data collection session", or "continue session". When a data collection session is selected, the computer control menu may provide menu selections for the number of cameras, any camera, etc., such as when the user selects a printer. In addition, there may be a mode such as a polarized light mode, a color filter mode, and the like. After each selection, the system can complete the task or, if appropriate, provide other choices. User intervention may also be required, such as turning on a light source or other light source of polarized light, applying a filter or polarizer, and the like.

After a fingerprint, a long text, an iris image, or other desired data is obtained, the menu may then provide a selection as to which database to use for comparison, which device (s) to use for storage, and so on. The touchless or non-contact biometric data acquisition system may be controlled by any of the methods described herein.

Systems and sensors have obvious uses to identify people of potential interest, but there are also clear uses in the battlefield. A fingerprint sensor can be used to invoke a soldier's medical history and provide appropriate treatment under battlefield conditions by providing immediate information on data that is sensitive to allergy, blood type or other time, have. This is especially helpful for patients who may be unconscious at the time of initial treatment and who may not have an identifier.

Additional embodiments of an apparatus for capturing biometric data from an individual may include a server for storing and processing the collected biometric data. The captured biometric data may include a hand image with a plurality of fingers, a long face, a face camera image, an iris image, an audio sample of a person's voice, and a video of a person's attitude or motion. The collected data must be accessible to be useful.

The processing of biometric data can be done locally or remotely from a separate server. Local processing can capture raw video and audio and provide the option of making information available upon request from a computer host via a WiFi or USB link. Alternatively, another local processing method processes the image and then transmits the processed data over the Internet. This local processing includes the steps of finding the fingerprint, evaluating the fingerprint, cropping the face after finding it, evaluating it after finding it, and other similar steps for audio and video data have. Processing data locally requires more complex code, but it offers the advantage of reducing data transmission over the Internet.

A scanner associated with a biometric data acquisition device can use a code compatible with the USB Image Device protocol, which is a commonly used scanner standard. Other embodiments may use different scanner standards, if desired.

When a WiFi network is used to transmit data, the Bio-Print device described further herein may function or look like a web server to the network. Each of various types of images can be used by selecting or clicking a web page link or button from a browser client. This web server function may be part of a bioprint device, and in particular may be included in a microcomputer function.

The web server can be part of a biometric print microcomputer host, allowing the biometric device to create web pages that expose captured data and also provide some control. Additional embodiments of the browser application include capturing high resolution hand prints, face images, iris images, setting camera resolutions, setting acquisition times for audio samples, and also providing web cam, Skype, or similar mechanisms Can be used to provide control to enable streaming connections. This connection can be connected to audio and face cameras.

Additional embodiments provide a browser application program that provides access to captured images and audio through a file transfer protocol (FTP) or other protocol. Another embodiment of the browser application may provide an automatic refresh at a selectable rate to iteratively capture the preview image.

Additional embodiments provide for local processing of biometric data captured using a microcomputer and provide additional control for displaying an evaluation of the captured image to allow the user to evaluate each of the fingerprints found and to capture It also allows you to search for faces and also to search for cropped iris images and allows the user to evaluate each of the iris fingerprints.

Another embodiment provides a USB port that is compatible with the Open Multimedia Application Platform (OMAP3) system. OMAP3 is a proprietary system on a chip for portable multimedia applications. The OMAP3 device port has a Remote Network Driver Interface Specification (RNDIS), a proprietary protocol that can be used at the top of USB. These systems provide the ability for the device to appear as an IP interface when the bioprint device is plugged into the Windows PC USB host port. This IP interface will be the same as via a WiFi (TCP / IP Web server). This makes it possible to move the data away from the microcomputer host and provides a display of the captured fingerprint.

An application on the microcomputer can implement the above by receiving data from the FPGA via the USB bus. Once received, the JPEG content is generated. This content can be written to or written to a server running on a laptop via a socket. Alternatively, the server may receive a socket stream, pop an image, leave the image open in the window, and thus create a new window for each bio-capture. If the microcomputer is running Network File System (NFS), a protocol used by Sun-based systems, or SAMBA, a free software reimplementation that provides file and print services to Windows clients, the captured files can be NFS or SMB System Management Bus), and may be shared and accessed by any client executing a PC communication bus implementation. In this embodiment, the JPEG viewer will display the file. The display client may include a laptop, augmented reality glasses, or a phone running an Android platform.

Additional embodiments provide server-side applications that provide the same services as described above.

An alternative embodiment to the server-side application displays results on the augmented reality glasses.

A further embodiment provides a microcomputer on a removable platform similar to a mass storage or streaming camera. The mobile platform also includes an active USB serial port.

In embodiments, the eyepiece may include an audio and / or visual sensor that captures sound and / or visuals from 360 degrees around the wearer of the eyepiece. This may be from sensors that are mounted on the eyepiece itself or attached to sensors mounted on a vehicle with a wearer. For example, a sound sensor and / or a camera may be mounted external to the vehicle, wherein the sensors are communicatively coupled to the eyepiece to provide a surround sound and / or visual 'view' of the surrounding environment. In addition, the sound system of the eyepiece may provide sound protection, removal, enhancement, etc. to help improve the wearer's listening quality while the wearer is surrounded by external or loud noises. In one example, the wearer may be connected to cameras mounted on the vehicle in which he or she is driving. These cameras can then be in communication with the eyepiece and provide a 360 degree view around the vehicle, such as is provided in a graphical image projected onto a wearer through an eyepiece display.

In one example, referring to Fig. 69, the control aspect of the eyepiece may include a wrist including a receiver and / or a transmitter for interfacing with the eyepiece for messaging and / or for controlling the eyepiece when the user is not wearing the eyepiece And may include a remote device in the form of a clock controller 6902. The wristwatch controller includes a camera, a fingerprint scanner, individual control buttons, a 2D control pad, an LCD screen, a capacitive touchscreen for multi-touch control, a shake motor / piezo bumper to provide tactile feedback, (E.g., provided in the control function area 6904 or on another function portion 6910 of the wristwatch controller 6902), a button with a tactile feel, a Bluetooth, a camera, a fingerprint scanner, an accelerometer, and the like. For example, the wristwatch controller may have a standard wristwatch display 6908, but additionally has the ability to control the eyepiece, e.g., via control functions 6914 in the control function area 6904, Lt; / RTI &gt; The wristwatch controller may display to the user a message (e.g., email, advertisement, calendar reminder, etc.) from the eyepiece and / or otherwise notify (e.g., vibration, audible sound) You can show the contents of the message from the eyepiece. A wobble motor, a piezo bumper, etc., can provide tactile feedback to the touch screen control interface. The wristwatch receiver can provide virtual buttons and clicks to the control function area 6904 user interface and buzz and bump the user's wrists when the message is received and do guitars. The communication link between the eyepiece and the wristwatch receiver may be provided via Bluetooth, WiFi, a cell network, or any other communication interface known in the art. The wristwatch controller may be used for video conferencing (e.g., for recording in the database to store an image of an iris, as described herein), for use in authentication with respect to a stored existing iris image, You can use the built-in camera for iris scanning, photography, video and so on. The wrist watch controller may have a fingerprint scanner as described herein. A wristwatch controller or any other tactile interface described herein may measure a user's pulse through a pulse sensor 6912 (which may be located below the body of the wristwatch in a band) or the like. In embodiments, the eyepiece and other control / tactile interface components may have pulse detection such that pulses from different control interface components are monitored in a synchronized manner for health, activity monitoring, authorization, and the like. For example, both the wristwatch controller and eyepiece may have pulse monitoring, where the eyepiece senses whether they are synchronized, such as when they both match a previously measured profile (e.g., for authentication) can do. Similarly, other biometrics may be used for authentication between a number of control interfaces and eyepieces, such as in fingerprints, iris scans, pulses, health profiles, etc., where the eyepiece may be used by the same person as an interface component Controller) and an eyepiece. By looking at the IR LED view of the skin to see the subsurface pulse, etc., the person's health / health can be judged. In embodiments, multiple device authentication (e.g., a token for Bluetooth handshake) may be performed using sensors, e.g., sensors on both devices (e.g., fingerprints on both devices as a hash for a Bluetooth token) Can be used.

In one embodiment, the wristwatch controller may have a touch screen that may be useful to control the glasses even when not on the user's face (e.g., in a backpack). The transparent lens of the wristwatch may have an OLED display with a switchable mirror attached to the underside of the lens. In other embodiments, the wristwatch controller lens may include an electric mirror or an electronic ink display (E-Ink display). In either case, the lens may cover a standard analog wristwatch mechanism and a transparent lens, including a switchable mirror or an electric mirror or an electronic ink display, may be activated to display the content. A wristwatch can be used for gesture control for an integrated sensor that detects gestures. The wristwatch can be used as an AR marker, so when the camera in the eyeglasses recognizes the wristwatch, the app can be activated. One such app could be using a wristwatch as a physical surface with an overlaid virtual image that makes the wristwatch virtually a touch screen interface.

70A to 70D, an eyepiece can be stored in an eyepiece carrying case including, for example, a charging function, an integral display, and the like. Figure 70a shows one embodiment of a case shown as closed with an integral charging AC plug and a digital display, Figure 70b shows that the case of the same embodiment is open, Figure 70c shows that the case of another embodiment is closed And Fig. 70d shows that the same embodiment is open, wherein the digital display is shown through the cover. In embodiments, the case may be in the case, for example, through an AC connection or via a battery (e.g., a rechargeable lithium-ion battery that is embedded in the carrying case to charge the eyepiece while away from the AC power source) As shown in Fig. Electrical power may be transmitted to the eyepiece via a wired or wireless connection (e.g., via a wireless induction pad configuration between the case and the eyepiece). In embodiments, the case may include a digital display in communication with the eyepiece, e.g., via a Bluetooth radio or the like. The display may provide information about the status of the eyepiece, such as a received message, a battery level indication, a notification, and the like.

71, the eyepiece 7120 may include a pile 7104 which may be fired from a remote control helicopter, which may be dropped by an airplane, which may be inserted into the ground 7118 by a person, (Unattended ground sensor unit) 7102 that is formed as an unattached ground sensor unit (not shown). The ground sensor unit 7102 may include a camera 7108, a controller 7110, a sensor 7112, and the like. The sensor 7112 may include a magnetic sensor, a sound sensor, a vibration sensor, a thermal sensor, a manual IR sensor, a motion detector, a GPS, a real time clock and the like and may provide monitoring at the location of the ground sensor unit 7102 have. The camera 7108 may have a field of view 7114 at both the azimuth and elevation - for example, a full or partial 360 degree camera array at azimuth and +/- 90 degrees at elevation. The ground sensor unit 7102 can capture sensor and image data of the event (s) and transmit it to the eyepiece 7120 via the wireless network connection. In addition, the eyepiece can then transmit the data to an external communication facility 7122 (cell network, satellite network, WiFi network, etc.), to another eyepiece, and so on. In embodiments, terrestrial sensor unit 7102 may relay data from unit to unit (e.g., 7102A through 7102B through 7102C). In addition, the data can then be relayed from the eyepiece 7120A to the eyepiece 7120B and subsequently to the communication facility 7122, as in the backhaul data network. Data collected from the ground sensor unit 7102 or an array of ground sensor units may be shared with a plurality of eyepieces (e.g., between an eyepiece, between a communication facility and an eyepiece, etc.) Can be used and shared in a raw form or in a post processed form (i.e., a graphical display of data through an eyepiece). In embodiments, the ground sensor unit may be inexpensive, disposable, toy-grade, and the like. In embodiments, the terrestrial sensor unit 7102 may provide a backup to a computer file from the eyepiece 7120.

72, the eyepiece includes a transceiver 7228 from an external computing facility 7232, an external application program 7234, an event and / or data feed 7238, an external device 7240, a third party 7242, An input device 7204, a sensing device 7208, a user motion capture device 7210, an internal processing facility 7212, an internal multimedia processing facility, an internal application Control may be provided through equipment located inside and outside the eyepiece, such as initiated from the program 7214, the camera 7218, the sensor 7220, the earpiece 7222, and the projector 7224. [ User application 7248, external device interaction 7250, reception of events and / or data feeds 7252, execution of internal applications 7254, execution of external applications 7258, The command and control mode 7260 of the eyepiece can be started. In embodiments, at least an event and / or data feed, a sense input and / or sense device, a user action capture input and / or output, a user movement and / or action to control and / An application on a platform on which the command can be used to respond to input, a communication and / or connection from an on-platform interface to an external system and / or device, an external device, an external application, There may be a series of steps involved in the execution control, including feedback for the user (such as external devices, external application programs, etc.), or any combination of the two.

In embodiments, the event and / or data feed may be in the form of an email, military communications, timetable notification, security event, safety event, financial event, personal event, input request, One type of environment, a hostile environment, a location, etc., and combinations thereof.

In embodiments, the sensing input and / or sensing device may be a charge-coupled device, a black silicon sensor, an IR sensor, an acoustic sensor, an induction sensor, a motion sensor, A sensor, an opacity sensor, a proximity sensor, an inductive sensor, an eddy current sensor, a passive infrared proximity sensor, a capacitance sensor, a capacitive displacement sensor, a Hall effect sensor, a magnetic sensor, Sensor, thermocouple, thermistor, photoelectric sensor, ultrasonic sensor, infrared laser sensor, inertial motion sensor, MEMS internal motion sensor, ultrasonic 3D motion sensor, accelerometer, inclinometer, force sensor, piezoelectric sensor, rotary encoder, linear encoder, chemical Water sensor, ozone sensor, smoke sensor, heat sensor, magnetometer, carbon dioxide detector, carbon monoxide detector, oxygen sensor, glucose sensor, smoke detector, metal detector, rainwater (Eg, billboards), a landmark detector (eg, a geographic location marker for advertising), a laser rangefinder, a sonar, an altimeter, a GPS, Capacitance, optical response, heart rate sensor, RF / MIR (micropower impulse radio) sensor, and the like, and combinations thereof.

In embodiments, the user motion capture input and / or device may include a head tracking system, a camera, a speech recognition system, a body motion sensor (e.g., kinetic sensor), a pupil gaze detection system, a tongue touch pad, sip-and-puff system, joystick, cursor, mouse, touch screen, touch sensor, finger tracker, 3D / 2D mouse, inertial movement tracking, microphone, wearable sensor set, robot motion detection system, optical movement tracking system, laser A motion detection system, a keyboard, a virtual keyboard, a virtual keyboard on a physical platform, a context determination system, an activity determination system (e.g., on a train, in an airplane, walking, Hand-held display, a hydration system, a trackball, a hand-held camera, a sensor located on the legs of a spectacle, a sensor located on the glasses, Communication may include wireless communication, satellite communication, etc, and combinations thereof.

In embodiments, the user's movements or actions that control or initiate an instruction may be selected from the group consisting of head movement, head waving, head nodding, hair turning, forehead distortion, ear movements, eye movements, eye movements, eye winding, eye blinking, Swinging or sucking with a straw, tongue movement, finger movement, one or more finger movements, finger-pushing, finger bending, finger subtraction, thumb-pushing , Creating a symbol with your finger (s), creating a symbol with your fingers and your thumb, pressing your thumb's finger, dragging and dragging with your fingers, touch and drag, touch and drag with two fingers, wrist movement, wrist- Arm movements, arm stretches, arm movements, arm left turn signals, arm right turn signals, arms on both arms (arms akimbo), arms stretched Leg movement, leg kick, leg stretch, leg curl, jumping jack, body movement walk, run turn left, turn right, turn back, turn around, (Eg, one left out and twirl), a rounding motion in various hand and arm positions, a fingering movement in the hands and arms, a finger movement (eg, (E.g., virtual typing), snapping, hip tapping, shoulder, foot, swipe, hydration (e.g., ASL), etc., and combinations thereof.

In embodiments, the commands and / or control modes and interfaces through which inputs may be reflected may include a graphical user interface (GUI), a hearing command interface, a clickable icon, a searchable list, a virtual reality interface, (E.g., by a persistent virtual keyboard that is locked in place), a predictive and / or learning-based user interface (e.g., a wearer), a semi-opaque display, a 3D navigation interface, a command line, a virtual touch screen, (Such as a hand gesture that starts an application program), a Bluetooth controller, a cursor hold, a virtual display lock, and so on) Head movement around a positioned cursor, etc., and combinations thereof. have.

In embodiments, the applications on the eyepiece that can use and / or respond to commands may be selected from the group consisting of a military application, a weapon control application, a military target selection application, a wargame simulation, a battlefield combat simulator, , Tactical operations applications, mobile phone applications (eg, iPhone apps), information processing, fingerprint capture, face recognition, information display, information delivery, information gathering, iris capture, entertainment, easy access to information on pilots, (Eg, maintenance, assembly, first aid, etc.), blind navigation assistance, and the like, in order to identify the location of the object in 3D, to select the target for the citizen, to select the target for the police, , Communication, music, search, advertisement, video, computer game, video, computer game, e-book, advertisement, shopping, It may include commerce, video conferencing, etc., and combinations thereof.

In embodiments, the communication and / or connection from the eyepiece interface to the external system and / or device may be performed by a microcontroller, a microprocessor, a digital signal processor, a steering wheel control interface, a joystick controller, a motion and sensor resolver, An application programming interface (API), a graphical user interface (GUI), a navigation system controller, a network router, a network controller, an automatic classification system, a payment system, Sensors, and the like.

In embodiments, the external device to be controlled may be any of a variety of devices, including but not limited to, weapons, weapon control systems, communication systems, bomb detection systems, bomb disassembly systems, remote control vehicles, computers (and thus many devices that can be controlled by a computer) (Eg, computer, video, TV screen), video game, war game simulator, mobile game, pointing or tracking device, wireless or sound system, distance meter, audio system, A game platform (such as a game platform that pre-loads the user to recognize and play the user), a vehicle, a game machine, a game machine, a game machine, a game machine, an entertainment system, a computer controlled weapon system, a drones, a robot, an automobile dashboard interface, A storage support device, a payment system, an ATM, a POS system, and the like.

In embodiments, the applications associated with the external device may be selected from the group consisting of a military application, a weapon control application, a military target selection application, a wargame simulation, a battlefield combat simulator, a repair manual application, a tactical operational application, Fingerprint capture, face recognition, iris capture, entertainment, easy access to information on pilots, object location in 3D in the real world, target selection for citizens, target selection for police, (Eg, maintenance, assembly, first aid), blind navigation assistance, music, search, advertising, video, computer games, electronic books, automotive dashboard applications, Electronic commerce, and the like, and combinations thereof.

In embodiments, the feedback to the wearer associated with the external device and the application may include a visual display, a head-up display, a bulls-eye or target tracking display, a tone output or sound alert, a performance or evaluation indicator, , An action completion indication, playback of content, display of information, reporting, data mining, referrals, targeted advertisements, and the like.

In one example, the control aspects of the eyepiece may include head nodding from the soldier as a motion to initiate a silent command (e.g. during battle engagement), a graphical user interface for the reflective mode, and / , Military applications in the eyepiece using commands and / or responding to control inputs, audio system controllers communicating and / or connecting to the external system from the eyepiece interface, and the like. For example, a soldier may be controlling a secure communication device through an eyepiece during combat engagement and may be able to control certain aspects of communication such as channel, frequency, encoding level, etc. with minimal movement without sound to minimize the possibility of being audible or visible May be desired. In this example, a soldier's head nod may be programmed to indicate a change (e.g., a fast nod to the front indicates the beginning of the transmission, a quick nod to the rear indicates the end of the transmission, etc.). In addition, the eyepiece may be projecting to the soldier a graphical user interface for the secure communication device, e.g., showing which channel is active, which alternative channel is available, others in his team currently transmitting, . The nudity of the soldier can then be interpreted as a change command by the processing equipment of the eyepiece, the command being sent to the audio system controller, and the graphical user interface of the communications device showing the change. In addition, a particular nod / body movement can be interpreted as a specific command to be transmitted, so that the eyepiece transmits the preset communication without the need for the soldier to be heard. That is, a soldier can transmit pre-canned communications to his team through body movements (such as, for example, decisions made with the team prior to engagement). In this way, the soldier wearing and using the eyepiece equipment can be connected with and interacting with the externally secured communication device in a completely confidential manner, and is able to interact with his team during silence, even when not invisible to the team, ). In embodiments, the instructions, commands and / or control modes in which inputs may be reflected, and / or other motions or actions that control or initiate an interface, and / or commands and / An application on the platform, an on-platform interface to an external system and a device, or the like can also be applied.

In one example, the control aspects of the eyepiece include movement and position sensors as a sensing input, an augmented reality interface as a command and control interface that the input can be reflected to the soldier, a motion sensor and distance meter of the weapon system as an external device to be controlled, Information collected from the device, feedback to the soldier associated with the external device, and the like. For example, a soldier wearing an eyepiece may be monitoring a military movement in the environment with a motion sensor, and when the motion sensor is triggered, the target, such as a person, vehicle, or the like for additional monitoring and / An augmented reality interface that helps to identify can be projected to the wearer. In addition, the range finder can be used to measure the distance to an object and use the information at the aim (e.g., manually if the soldier is performing a fire operation; or if the weapon system receives information for aiming and the soldier If you provide an order, you can automatically provide feedback to the soldier. In embodiments, the augmented reality interface may include a location of an object on a 2D or 3D projected map, an ID of the target from previously collected information (e.g., stored in the object database, including face recognition, object recognition) Such as coordinates of a target, night vision images of a target, and the like, to a soldier. In embodiments, the triggering of the motion detector may be interpreted as a warning event by the processing equipment of the eyepiece, and the command may be sent to the range finder to identify the position of the object, May be transmitted to the speaker of the eyepiece of the eyepiece to provide an audio alert to the soldier that it has been detected. Audio warnings and visual indicators for soldiers should be paid attention to moving objects, such as when an object is identified as an object of interest to a soldier, e.g. through an accessible database of known combatants, base vehicle types, Can serve as an input to the soldier. For example, a soldier can be in the prairie at night watching the perimeter around the perimeter. In this case, the environment can be dark, and the soldier can be in a state of reduced attentiveness because all environmental conditions can be quiet late at night. The eyepiece can then function as a sentry augmentation device to 'watch' from the soldier's personal point of view (as opposed to any external surveillance equipment at the border crossing). When the eyepiece detects movement, the soldier can be guided to promptly alert, as well as to position, distance measure, identify, etc. the movement. In this way, the soldier can react to avoid personal hazards, to aim the shot towards a locat- ed movement, and to inform potential hazards of course for others. In addition, if a shootout is to be followed, the soldier can have an improved response time as a result of warning from the eyepiece, make better decisions with information about the target, minimize the risk of injury, Do not let them get hurt. In embodiments, other sensing inputs and / or sensing devices, commands and / or control modes and interfaces to which inputs may be reflected, useful external devices to be controlled, external devices, and / or feedback related to external applications are also applied .

In the examples, the eyepiece can enable remote control of the vehicle such as trucks, robots, drones, helicopters, boats, and the like. For example, a soldier wearing an eyepiece may command through an internal communication interface to control the vehicle. Vehicle control may be provided via voice commands, body movements (e.g., a soldier having a motion sensor in interactive communication with the eyepiece, which interfaces with the eyepiece to control the vehicle), a keyboard interface, and the like. In one example, a soldier wearing an eyepiece may provide remote control to a bomb processing robot or vehicle, wherein the command is generated by a soldier through an instruction interface of the eyepiece as described herein. In another example, a soldier may command a plane, such as a remote control drones, a tactical counter-rotating helicopter, or the like. In other words, a soldier can control a remote control plane through a control interface as described herein.

In one example, the control aspect of the eyepiece is a combination of a wearable sensor set as an actuation acquisition input to a soldier, a drone or other robotic device as an external device to be controlled, etc., using a robot control interface as a command and control interface through which inputs may be reflected . &Lt; / RTI &gt; For example, a soldier wearing an eyepiece may control a military drones having a motion sensor input or the like to control the movement of the drones (e.g., via a graphical user interface displayed through an eyepiece) Hand recognition control, voice command input for control of the drones, and the like. In embodiments, control of the drones through the eyepiece may include control of flight, control of onboard illumination sensors (e.g., visible cameras, IR cameras, radar), threat avoidance, and the like. The soldier can direct the drones to his intended targets, imagining the actual battlefield, using a body-mounted sensor and a virtual 2D / 3D projected image, where the flight, camera, Lt; / RTI &gt; In this way, the soldier can maintain a complete visual commitment of the individual on the drones' flight and environment for more intuitive control. The eyepiece may have a robot control interface for managing and adjusting various control inputs from the military wear sensor set and for providing an interface for control of the drone. The drones can then be controlled remotely through physical behavior of the soldier (e.g., via a wireless connection to a military control center for dron control and management). In another similar example, a soldier can control a bomb disassembly robot that can be controlled through a soldier wear sensor set and associated eyepiece robot control interface. For example, a soldier may be provided with a graphical user interface that provides a 2D or 3D view of the environment around the bomb decay robot, where the sensor pack translates the movement of a soldier (e.g., arm, hand, etc.) . In this way, the soldier can provide a remote control interface to the robot to allow for more sensitive control during difficult bomb disassembly processes. In embodiments, other user action acquisition inputs and / or devices, commands and / or control modes and interfaces, input devices to be controlled, etc. may also be applied, as described herein, .

In one example, the control aspects of the eyepiece include an event display for the soldier when the soldier enters the place, a command reflecting the occurrence of an event input and a control mode and / or a predictive learning based user interface as an interface, System, and the like. For example, when an eyepiece is used to indicate a soldier's behavior (e.g., what a soldier typically does when he or she enters a particular environment with a particular weapon control system, such as a wearer turning on the system, arming the system, Displaying a display, and so on). From this learned behavior, the eyepiece can predict what the soldier wants instead of the eyepiece control function. For example, a soldier can be pushed into combat situations and requires immediate use of a weapon control system. In this case, the eyepiece can sense the position and / or the ID of the weapon system when the soldier approaches and, as in the previous use of the weapon system when the eyepiece is in the learning mode, You can configure / enable the weapon system depending on how you normally configure the weapon system when you are there, and command the weapon control system to turn on the weapon system as configured last. In embodiments, the eyepiece may sense the position and / or the identity of the weapon system via a plurality of methods and systems (e.g., via a vision system, a RFID system, a GPS system, etc.) . In embodiments, commanding the weapon control system may include providing a graphical user interface for providing visual information to the soldier for fire-control of the weapon system, providing a selection to the soldier and providing speech recognition for the command An audio-voice command system interface, and automatic activation of functions. In embodiments, there may be a profile associated with such a learned instruction, where the soldier may modify the learned profile and / or modify the profile in the learned profile to help in optimizing automated actions, Preferences can be set. For example, a soldier may have an individual weapon control profile for weapon readiness (ie, while waiting for action at the headquarters) and for active weapon engagement with enemy forces. A soldier may need to modify the profile to adjust according to changing circumstances associated with the use of the weapon system (e.g., changes in the weapon command protocol, ammo type, added functionality of the weapon system, etc.). In embodiments, other events and / or data feeds, commands and / or control modes and interfaces to which input may be reflected, as well as useful external devices to be controlled, etc., as described herein, may also be applied.

In one example, the control aspects of the eyepiece may include an individual responsibility event (such as being placed in an operational area and managing its time) for a soldier as an event and / or data feed, a speech recognition system as a user action capture input device, And a video-based communication as an application program on the eyepiece used for responding to input from the soldier, and the like. For example, a soldier wearing an eyepiece may be projected with a visual indication of a scheduled event for a group video supported communication between commanders. The soldier can then use a voice command for the audible command interface on the eyepiece and a voice command for the group video communication to be initiated to display the contact information for the call. In this manner, the eyepiece can act as a personal assistant to the soldier to present the hands-free command interface to the soldier to cause the scheduled event to occur and to execute the scheduled event. In addition, the eyepiece may provide a visual interface for group video communication, wherein images of the other commanders are projected to the soldier through the eyepiece, where the external camera is connected to the eyepiece via a communication link (e.g., (By means of an external device having a camera, a mirror with an internally integrated camera, or the like), as described. In this way, the eyepiece can provide a fully integrated personal assistant and a telephone / video-based communication platform, and can be used with other traditionally separate electronic devices (e.g., wireless, cellular, videophone, personal computer, calendar, hands- Control interface, etc.). In embodiments, other events and / or data feeds, user action capture inputs and / or devices, commands and / or control modes and interfaces, commands that may be reflected in inputs, as described herein &Lt; / RTI &gt; and / or an application on the platform that is capable of responding to input may also be applied.

In one example, the control aspect of the eyepiece is a security event for a soldier as an event and / or data feed; A camera and a touch screen as a user action capturing input device; Information processing on eyepiece responding to input, fingerprint capture, face recognition application program; A graphical user interface for communication and / or connection between the eyepiece and the external system and device; And a combination of external information processing, fingerprint capture, facial recognition applications and databases for accessing external security facilities and connections, and the like. For example, a soldier may receive a &quot; security event &quot; while a plurality of people are at a security checkpoint and / or at a checkpoint at an identified military checkpoint. In this case, for example, because it does not appear in the security database, it may be necessary to record the biometrics of the individual because of other reasons, because of the suspicious behavior and the profile of the members of the combat unit. The soldier may then use a biometric input device such as a camera for photographing a face and a touch screen for recording a fingerprint, where the biometric input is managed through internal information, processing, fingerprint capture, and face recognition application programs on the eyepiece. In addition, the eyepiece may provide a graphical user interface as a communication link to external information, processing, fingerprint capture, and face recognition application programs, where the graphical user interface may include a data capture interface, external database access, . An eyepiece is an end-to-end security management facility, including monitoring of the person of interest, an input device for acquiring biometric data, displaying input and database information, connection to external security and database applications, etc. Can be provided. For example, a soldier can check people passing through a military checkpoint, and a soldier is commanded to collect face images, such as an iris biofilm, for any person who does not currently exist in the security database and meets the profile. When approaching a soldier, such as people standing in line to pass a checkpoint, a soldier's eyepiece is scanned through a database accessible via a network communication link, And acquires a high-resolution image. A person can be allowed to pass through a checkpoint if they are in the database with an indication that they do not meet the profile (for example, a child) or that they are not considered a threat. A person may not be allowed to pass through the checkpoint, and if the individual is marked as a threat or if the profile is met and is not in the database, it is called on one side. If they need to be entered into the security database, the soldier may use the equipment of the eyepiece directly or in close-up of the face and / or iris of the person, collecting personal information about the person, for example as described herein A person can be handled by an eyepiece that controls an external device that captures an image, records a fingerprint, or performs a guitar. In embodiments, as described herein, other events and / or data feeds that can use commands and / or respond to input, user action capture inputs and / or devices, applications on a platform, Communication or connection from the on-platform interface to external systems and devices, applications to external devices, etc. may also be applied.

In one example, the control aspect of the eyepiece may include a finger movement as a user action to initiate an eyepiece command, a command and control mode in which a user action may be reflected, and / or a clickable icon as an interface, Military control, military movement, informational data feeds, etc.), military application tracking APIs as communication and / or connection from an eyepiece application to an external system, external personal tracking application, feedback to military personnel, have. For example, a system in which a soldier monitors the selection of an application on an eyepiece may be provided to a soldier where monitoring provides monitoring and tracking of the use of the application, and to other applications available to the soldier based on the monitored behavior Providing feedback to the soldier, and so on. During the course of a day, the soldier may, for example, have the ability of a soldier to be presented with clickable icons and to allow the soldier to use a finger motion control implementation (e.g., in this case, Or downloading, via a graphical user interface that can select an icon based on the system (e.g., the system). This selection can then be monitored via the military application tracking API to send the selected or stored selection count to an external personnel tracking application (e.g., sending stored selections over a period of time). The soldier's application selection (in this case, "virtual click") can then be analyzed for optimization of usage, for example, by increasing bandwidth, changing available applications, improving existing applications, have. In addition, the external workforce tracking application can use the analysis to determine what the wearer's preferences are in terms of application usage, and can include recommendations of applications that the wearer may be interested in, preference profiles, The user can transmit the feedback to the wearer in the form of a list of things the user is using. In embodiments, the eyepiece can be used to provide services that enhance the soldier's experience with the eyepiece, such as recommendations for how the soldier can benefit, such as helping guide the use of the eyepiece and its application for military use . For example, a soldier using an eyepiece for the first time may not be able to take full advantage of his functions, such as in use of an augmented reality interface, organic application, mission support, and the like. The eyepiece monitors the utilization of the soldier, compares the utilization to a measure of utilization (eg, stored in an external eyepiece utilization facility), and provides feedback to the soldier to improve the use and associated efficiency of the eyepiece . In embodiments, as described herein, commands, commands and / or control modes in which inputs may be reflected, and / or other user actions or actions that control or initiate an interface and / Communications on or connection to an external system and device from an on-platform interface, applications on external devices, feedback on external devices, and / or external applications may also be applied.

In one example, the control aspects of the eyepiece include IR, heat, power, carbon monoxide, and other sensors as inputs; A microphone as an additional input device; A voice command as an operation for the soldier to initiate the command; A head up display as a command and control interface through which input can be reflected; Instructional applications that provide guidance while reducing the need for a soldier to use his hands, such as in the field of emergency repairs, maintenance, assembly, etc; A visual display that provides feedback to the soldier based on the actions of the soldier and sensor inputs, and the like. For example, a soldier's vehicle may have been damaged in a gunfight, and the soldier (s) could not be immediately transported and stuck. A soldier can display a training guide application, such as running through an eyepiece, to provide a hands-free instruction manual and computer-based expert knowledge access to diagnosing a problem in a vehicle. In addition, the application program may provide a description of unfamiliar procedures (e.g., restoring the basic and temporary functions of the vehicle) to the soldier. The eyepiece can also monitor various sensor inputs (e.g., IR, heat, power, ozone, carbon monoxide, and other sensors) related to the diagnosis and thus the sensor input can be accessed by the training application and / Can be directly accessed. The application also includes a microphone on which a voice command can be received; User guidance information, a head-up display for displaying a 2D or 3D representation of the portion of the vehicle under repair, and the like. In embodiments, the eyepiece may provide the soldier with a hands-free virtual assistant to assist the soldier in diagnosing and repairing the vehicle to recreate a means of transport that allows the soldier to re-engage with the enemy or move to a safe location. In embodiments, as described herein, other sensing inputs and / or sensing devices that control or initiate an instruction, user motion capture inputs and / or devices, user movements or actions, And / or control modes, feedback on applications, external devices and / or external applications on the platform using commands and / or responding to input may also be applied.

In one example, the control aspect of the eyepiece is that the eyepiece enters an 'activity state' such as a 'military engagement' activity mode (eg, a soldier orders the eyepiece to enter a military engagement mode) A combination of detection of proximity to military activities (possibly even predetermined or targeted engagement areas) through instructions (which may be further developed through self-monitoring and learning of the wearer's general engagement missions) . &Lt; / RTI &gt; Continuing with this example, entering an activity state (e.g., a military engagement activity state), such as while driving a vehicle into an enemy or hostile territory, may include an object detector as a sensing input or sensing device, On-camera and / or a pupil gaze detection system as a user, an eye movement as a user movement or action for commanding, a command and control mode in which the input can be reflected, and / or a 3D navigation interface as an interface, An engagement management application embedded in an eyepiece as an application for adjusting a user interface, a navigation system controller for communicating or connecting with an external system or device, a vehicle navigation system as an external device to be controlled and / or interfaced, A military planning and execution facility as an external application processing an action, a target or target tracking system as feedback to the wearer regarding an enemy aiming opportunity in the field during operation, and the like. For example, a soldier can enter hostile environments while driving his vehicle, and an eyepiece that detects the presence of an enemy engagement zone (for example, by looking at a target through GPS, an integrated camera, etc.) Active state &quot; (e.g., enabled and / or authorized by the soldier). The eyepiece can then detect enemy vehicles, enemy accommodation, etc., by means of an object detector that finds an enemy targeting opportunity, for example, through a head-mounted camera. In addition, the eye-gaze detection system on the eyepiece can monitor where a soldier is watching and possibly highlight information about targets (enemy agents, enemy vehicles, enemy vehicles, as well as friendly forces) in the eye of the wearer , In which case friendly and enemy forces are identified and distinguished. For example, for the purpose of changing a target of interest, or for command input (e.g., quick nudging indicating a selection command, downward eye movement indicating a command for additional information, etc.), a soldier's eye movement may also be tracked . The eyepiece may include a 3D navigation interface projection to assist the soldier in providing information related to his surroundings, and a state of military engagement activity (e.g., receiving input from a soldier, providing output to a 3D navigation interface, Interfacing with devices and applications, etc.) can be invoked. The eyepiece may, for example, use a navigation system controller to interface with the vehicle navigation system, and thus incorporate the vehicle navigation system into the military engagement experience. As an alternative, the eyepiece may use its own navigation system, for example, in place of or in order to reinforce the vehicle system, when a soldier is to get off the vehicle and want to be provided with an over-the-ground direction . As part of a military engagement activity, the eyepiece may interface with an external military planning and execution facility to provide, for example, current status, unit usage, weather conditions, friendly locations and forces. In embodiments, a soldier may be provided with feedback associated with an activity state through entering an activity state, and may provide feedback in the form of information associated with the identified target for, for example, a military engagement activity state Receive. In embodiments, other instructions and / or data feeds, sensing inputs and / or sensing devices, user motion capture inputs and / or devices, inputs, Or other user actions or actions that control or initiate the control mode and interface, applications on the platform that are capable of using commands and / or responding to inputs, communication or connection from the on-platform interface to external systems and devices, Feedback relating to an application program, an external device and / or an external application program, etc., may also be applied.

In one example, the control aspects of the eyepiece include receiving secure communications as a triggering event for a soldier, tracking inertial movement as a user motion capture input device, dragging and dropping a finger by a soldier as a user movement or action to control or initiate an instruction And swipe movements, a command that can reflect the input and a searchable list as a control interface, information delivery as a kind of application program on the eyepiece that can use commands and respond to the input, from the interface on the eyepiece for external systems and devices An adjustment system as a communication or connection, an iris acquisition and recognition system as an external application for external systems and devices, and the like. The soldier wearing the eyepiece can receive secure communications, which can be used to initiate an application or operation in the eyepiece, for example, with visual and / or audible alarms to trigger an operating mode of the eyepiece, For others, you can enter the eyepiece as an 'event' for the soldier. A soldier may be able to use a plurality of control mechanisms - e.g., via a hand gesture interface (e.g., via a camera and hand gesture application program embedded in the eyepiece, where the wearer places the email or information in the communication in a file, application program, Drag & drop ', swipe, etc) of the wearer's fingers and hands. The wearer may call the searchable list as part of acting on the communication. The user may communicate information from the secure communication to an external system and device (e.g., an adjustment system for tracking communications and related actions) through an eyepiece application. In embodiments, the eyepiece and / or security access system may require identification verification via biometric ID verification (e.g., fingerprint capture, iris capture recognition, etc.) or others. For example, a soldier may receive a secure communication that is a security alert, where the secure communication comes with a secure link to additional information, where the soldier needs to provide biometrics before being provided access. Once authenticated, the soldier may use the hand gesture in his response and manipulation of the content available through the eyepiece (e.g., manipulating available lists, links, data, images, etc., from the communication and / or via included links) . Providing a soldier with the ability to respond and manipulate content in relation to secure communications can enable a soldier to better interact with messages and content in a way that does not contaminate any non-security environment that he may be present. In embodiments, control and / or control of other events and / or data feeds, user motion capture inputs and / or devices, commands, commands and / or control modes in which inputs may be reflected, Other user actions or actions that initiate, application programs on the platform that can use and / or respond to input, communication or connection from the on-platform interface to external systems and devices, applications to external devices, etc., are also applied .

In one example, the control aspect of the eyepiece may comprise a combination of using an inertial user interface as a user motion capture input device to provide military indication to a soldier through an eyepiece to an external display device. For example, a soldier wearing an eyepiece may wish to provide instructions to a group of other soldiers on the battlefield from a briefing made available through the eyepiece facilities. The soldier has a physical 3D or 2D mouse (e.g., an inertial motion sensor, a MEMS inertial sensor, an ultrasonic 3D motion sensor, an IR, an ultrasonic or capacitive tactile sensor, an accelerometer, etc.) to provide an interface for manipulating the content in the briefing. ), Virtual mice, virtual touch screens, virtual keyboards, and more. The briefing can be viewed and manipulated through an eyepiece, but can also be exported in real time to an external router connected to, for example, an external display device (e.g., a computer monitor, projector, video screen, TV screen, etc.). Accordingly, the eyepiece can provide a way for the soldier to see what he sees through the eyepiece and under the control of the eyepiece through the control facility, thereby allowing the soldier to be associated with the same briefing as is possible through the eyepiece The multimedia contents can be exported to other non-contact eyeglass wearers. In one example, a mission briefing may be provided to a commander in the battlefield, and the commander may use the multimedia and augmented reality resources to brief his team through the eyepiece, as described herein, Obtain the advantages provided by visual resources. In embodiments, other sensing inputs and / or sensing devices, user motion capture inputs and / or devices, commands and / or control modes and interfaces through which inputs may be reflected, on-platform interfaces Communications or connections to external systems and devices, useful external devices to be controlled, external devices, and / or feedback related to external applications may also be applied.

In one example, the control aspects of the eyepiece may include head nodding as a user movement initiating a command, an interface reflecting a graphical user interface and / or a control input to the reflective mode, using an instruction, and / An audio system controller for communicating and / or connecting to an external system or device from an eyepiece interface, and the like. For example, the wearer of the eyepiece may be controlling the audio player through the eyepiece and may wish to change to the next track. The nod of the wearer's head may be programmed to indicate a change in track in this case. In addition, the eyepiece may be projecting a graphical user interface to the audio player (e.g., showing which track is being played) to the wearer. The wearer's nod can then be interpreted as a track change command by the processing equipment of the eyepiece and the command can then be sent to the audio system controller to change the track and the graphical user interface to the audio player will change the track You can show it to the wearer.

In one example, the control aspect of the eyepiece includes a combination of a motion sensor as a sensing input, an instruction that the input can be reflected to the wearer, an augmented reality interface as a control interface, a distance meter as an external device to be controlled and information gathered therefrom . For example, a wearer of an eyepiece may be monitoring movement in the environment with a motion sensor, and an augmented reality interface may be projected to the wearer to help identify the object when the motion sensor is triggered. In addition, it can help identify other sensors, such as a range finder that determines the distance to an object. The augmented reality interface is used to determine the position of an object on a 2D or 3D projected map, the ID of an object from previously collected information (e.g., stored in an object database including face recognition, object recognition), the coordinates of an object, Information about an object such as a night vision image can be provided to the wearer. The triggering of the motion detector can be interpreted as a warning event by the processing equipment of the eyepiece and the command can then be sent to the distance meter to confirm the position of the object and also provide an audio alert to the wearer that a moving object has been detected To the earphone of the eyepiece. The audio alert and visual indicator for the wearer may serve as an input to the wearer who must pay attention to the moving object, such as when the object is identified as an object of interest to the wearer.

In one example, the control aspect of the eyepiece may comprise a combination of a wearable sensor set as a user motion capture input, a command that can receive input and a robot control interface as a control interface, a drone or other robotic device as an external device to be controlled, have. For example, the wearer of the eyepiece may be controlled by a drones, such as a motion sensor input to control the movement of the drones, a hand recognition control for manipulation of the control features of the drones (e.g., via a graphical user interface displayed through the eyepiece) A voice command input for the control of the speaker, and the like. The eyepiece may have a robot control interface for managing and adjusting various control inputs from the sensor set and for providing an interface for control of the drone. The drones can then be remotely controlled via the wearer's actions (e.g., via a wireless connection to the control center for dron control and management, more directly to the drones, to the guitar). In another similar example, a robot (e.g., a bomb disassembly robot) may be controlled via a sensor set and an eyepiece robot control interface. For example, a wearer may be provided with a graphical user interface that provides a 2D or 3D view of the environment around the robot, where the sensor pack is used to transform the movement of a wearer (e.g., an arm, a hand, etc.) to provide. In this manner, the wearer can provide the remote control interface to the robot.

In one example, the control aspect of the eyepiece may include an entry into the event as an event for the eyepiece, a command reflecting the input generation of the event and a control mode and / or a predictive learning based user interface as an interface, an entertainment system as an external device to be controlled, Combinations thereof. For example, the eyepiece can be programmed to learn the behavior of the wearer (e.g., what the wearer typically does when the wearer enters the room with the entertainment system, e.g., the wearer turns on the television, sound system, gaming system, etc.) . From this learned behavior, the eyepiece can predict what the wearer would want instead of the eyepiece control function. For example, when walking into the living room, the eyepiece senses the position, commands the entertainment system to play music through the entertainment system typically when the wearer enters the room, and finally plays the reproduced music. In embodiments, the eyepiece may sense the location through a plurality of methods and systems (e.g., through a visual system, a RFID system, a GPS system, etc.) that recognize the location. Instructing the entertainment system may be through a graphical user interface that provides a selection item to the wearer, an audio-voice command system interface that provides a choice to the wearer and provides speech recognition for the command, and automatic activation of the command. There may be a profile associated with such a learned command where the wearer may modify the learned profile and / or set preferences within the learned profile to help optimize automated actions, have.

In one example, the control aspects of the eyepiece include an event and / or data feed, a voice recognition system as a user motion capture input device, an audible command interface as an instruction to which the input can be reflected, and a control interface, A video conference as an application program on an eyepiece, and the like. For example, a visual indication of a calendar event for a conference call may be projected to the wearer. The user can then use a voice command for the audible command interface on the eyepiece and a voice command for the video conference to be launched to display dial-in information for the call. In this way, the eyepiece can act as a personal assistant to present a hands-free command interface to the wearer to cause certain events to occur and to execute certain events. In addition, the eyepiece can provide a visual interface for video conferencing, where images of others are projected onto the wearer through the eyepiece, where the external camera provides the wearer's video image to the eyepiece via a communication connection . The eyepiece can provide a fully integrated personal assistant and telephone / videoconferencing platform and includes the functionality of other traditionally separate electronic devices (e.g., cell phones, PDAs, calendars, hands-free commands and control interfaces, etc.).

In one example, the control aspect of the eyepiece is a security event as an event and / or data feed; A camera and a touch screen as a user action capturing input device; Information processing on eyepiece responding to input, fingerprint capture, face recognition application program; A graphical user interface for communication and / or connection between the eyepiece and the external system and device; And a combination of external information processing, fingerprint capture, facial recognition applications and databases for accessing external security facilities and connections, and the like. For example, a security official may be handling a &quot; security event &quot; that may be a checkpoint where many people are to be checked and / or authenticated, others should be checked and / or identified, , Where it is necessary to record a person's biometric characteristics (e.g., due to a suspicious behavior, etc.) because they do not appear in the security database. The security person may then use a biometric input device such as a camera for photographing a face and a touch screen for recording a fingerprint, where the biometric input is managed through internal information on the eyepiece, processing, fingerprint capture, and a face recognition application program. In addition, the eyepiece may provide a graphical user interface as a communication link to external information, processing, fingerprint capture, and face recognition application programs, where the graphical user interface may include a data capture interface, external database access, . An eyepiece is an end-to-end security management facility, including monitoring of the person of interest, an input device for acquiring biometric data, displaying input and database information, connection to external security and database applications, etc. Can be provided.

In one example, the control aspects of the eyepiece may include finger movements as user actions to initiate an instruction, commands and control modes in which user actions may be reflected, and / or a clickable icon as an interface, an application program on the eyepiece , Music search, advertising selection, etc.), communication from an eyepiece application to an external system and / or a combination of an advertising tracking API as a connection, an external advertising application, feedback to the user, and the like. For example, a system for monitoring user selection of an application program on an eyepiece may be implemented via an API, and thus monitoring may include services to the placement facility, other applications that may be of interest to the wearer based on the monitored behavior As well as feedback to the wearer about the wearer. During the day, the wearer may be instructed, for example, to select a clickable icon and a wearer's finger movement control implementation (e.g., in this case, to select a clickable icon, Or downloading, via a graphical user interface that can select an icon based on the system (e.g., the system). This selection may then be monitored via the ad tracking API to send the selected or saved number of selections to an external ad application (e.g., sending saved selections over a period of time). The wearer ' s application selection (in this case, &quot; virtual click &quot;) is then used to create ad revenue, e.g., by repositioning the ad to the wearer, selling data to a third party ad equipment, Can be analyzed. In addition, the external advertising application may use the analysis to determine what the wearer's preferences are in terms of application usage, and may include recommendations of applications that the wearer may be interested in, preference profiles, A list of those interested in downloading, and the like can be transmitted to the wearer. In embodiments, the eyepiece can be configured to provide a better viewing experience, e.g., by a recommendation for a download that the wearer may be interested in, by better targeted advertising that the wearer may be interested in, Services, while helping to generate advertising revenue for third parties through external advertising applications.

In one example, the control aspects of the eyepiece include a body movement (e.g., a motion sensor sensing head movement, a camera sensing head movement, a motion sensor sensing body movement) and a user motion capture sensor A user for controlling and / or initiating an instruction (e.g., via gesture control), a touch sensor or audio (e.g., sensing a game device such as a steering wheel, a sword, A virtual reality interface as a control interface, an information display as an application program on an eyepiece capable of responding to an input, a computer game device as an external device to be controlled through a game application program, And reproduction of game contents for wearer and external device and application program And may include a combination of performance, evaluation, score, etc., as feedback to the user. For example, a wearer may play an interactive computer game in connection with an eyepiece (e.g., in a computer, a computer-based game platform, a mobile gaming platform), where the physical movement of the wearer may include, , An infrared sensor, an IR camera, a visible camera, and the like. In this way, rather than using a more conventional control input, such as a handheld game controller, the wearer's body movement can be fed into a computer game. For example, the eyepiece can detect movement of the user's hand or the like through IR and / or visible cameras on the eyepiece and can be processed through onboard or external gesture recognition algorithms. The eyepiece can detect movement of the user's head through a motion sensor on the eyepiece (e.g., to sense the user's jumping, back and forth movement, left and right movement in response to the game). In one embodiment, a gesture can be captured by a camera that is capturing down through the use of a downward facing camera or a bending optics. In other embodiments, the camera may capture a non-line of sight gesture for recognition. For example, motion tracking and room mapping may be performed when a foviated or segmented camera is looking forward, but it may be desirable to have a motion tracking and room mapping function on the side of a user that is not parallel to the central axis of the display, Has a quadrant or hemisphere looking down to capture gestures and user interface control commands by the user's hand. Of course, hand gestures can be tracked, for example, by using IMU sensors, magnetic markers, RF tags, etc. in devices (ring controller, wrist watch, hand grip, etc.), as described herein . These body motion control inputs can then be fed to a virtual reality interface and information display application program on the eyepiece to provide the user with a visual representation of the game environment and can be fed to a computer game platform And can be provided to both the virtual reality interface and the information display of the eyepiece and gaming platform to create an augmented reality game platform via the eyepiece, or the like. In embodiments, a control device or interactive control element used to detect body movement or otherwise display user interaction may be removed from the computer image by a processor associated with the eyepiece. In the event that the sensor does not want to be part of the game, all or a portion of the image of the control device may be removed from the image generated for game play. For example, if the sensor is simply used to detect hand / arm movement, the sensor can be removed from the image generation, but if the sensor or control device is an object (e.g., gum) May be displayed as such in a game play or AR environment. In embodiments, it may be desired to be visible at a location other than where the control device is actually located. For example, a target that a user throws a dart may be displayed at the end of the hole in front of the user, instead of being displayed in association with the dart the user is throwing. As a further example, if the user never fires a dart in an actual gameplay, the dart as a control device can be seen to advance towards the target based on the characteristics of the user's throw. In embodiments, the computer game may be implemented entirely as a local game application on the eyepiece, interfaced to an external game facility local to the wearer, or connected to a networked game facility (e.g., massively multiplayer online game game, MMOG), or a combination of things on the eyepiece and through the game platform, or the like. In the event that the eyepiece is interfacing with and controlling a local external game facility (e.g., a game platform at the wearer's home), the eyepiece application portion of the game execution may provide the wearer with a visual environment and information display, The facility may provide a game application execution. As an alternative, the eyepiece may provide a user motion detection interface and provide that information to the gaming platform, which in turn provides the user with a visual interface of the game. Alternatively, the eyepiece may provide a user motion detection interface, where the information may be provided to both the eyepiece and the gaming platform to create an augmented reality game interface that combines a visual interface and a gaming platform in presenting the game to the user. Lt; / RTI &gt; In embodiments, the application program may reinforce an advertisement or the like on the side of a building or other structure with an object passing through the foreground. When the user drives the car, the camera may notice that the object (e.g. street lamps on the roadside) is moving through the field of view faster than the augmented surface in the background. The display system may subtract a portion of the augmented image to preserve the virtual layering of the content behind the image. This may require rigorous correction of the parallax between the user's eyes, display and camera. In embodiments, this technique may be used to generate a depth map. It will be apparent to those skilled in the art that many different partitioning configurations between the processing provided by the eyepiece and the processing provided by the external facility may be implemented. In addition, as in MMOG, game implementations can be extended to external game facilities over the Internet. Subsequently, the external facility may be part of at least the reproduced content (e.g., locally provided game projection combined with content from external equipment and other players), whether it is local or over the Internet, Etc., it is possible to provide feedback to the wearer. In embodiments, the eyepiece may provide a user environment for a computer game, where the eyepiece interfaces with an external control input and an external processing facility to create a next-generation game platform.

As an alternative to an eyepiece for detecting body movement through a sensor (e.g., a motion sensor, a body motion sensor, a touch sensor) physically connected directly to a wearer, the eyepiece is used to detect the position of the wearer's hands, feet, For example, an active remote sensing system for indirect sensing and interpretation of a wearer's body movement through the use of a 3D active depth sensor utilizing projected IR, sonar, RF, energy, and the like. An active 3D depth sensor can also be used with a visible or IR camera on an eyepiece. The combination of camera and 3D depth sensor can provide 3D motion capture that is processed in the eyepiece to provide advanced gesture recognition. The 3D active depth sensor may include a source (e.g., an IR laser projector) and a receive sensor (e.g., an IR sensor). In embodiments, the camera and the 3D active depth sensor may be oriented downward, sideways, and outward relative to the eyepiece line of sight to enhance system visibility to the user's hands, feet, and the like. In embodiments, one or more cameras (e.g., one facing the front, one detecting the eye movement and one facing back), imaging and eyepiece functions for imaging, as described herein, A plurality of cameras such as one or more cameras for detecting movement of a wearer for controlling an application program, an external device, and the like may be on the eyepiece. In one example, the combination of the depth sensor and the camera may be directed downward to capture the image and movement of the wearer &apos; s hand, where the ophthalmologic processor is capable of moving the hand (e.g., translation and rotation of the hand, ), Calculates the motion of the hand using a motion algorithm, and controls the eyepiece function based on the onboard database of detected motion and function commands as a function of the detected motion do. In embodiments, interpreted hand movements may be used for eyepiece function, control of an eyepiece application, control of an external facility via an eyepiece, input to an external game platform, input to an external virtual reality game, and the like. In embodiments, a camera, a 3D active depth sensor, and associated algorithms may be combined with an array of onboard microphones or microphones to detect both the sound and movement of the surroundings.

The present disclosure may include an interactive head mounted eyepiece worn by a user, wherein the eyepiece has a view of the ambient environment along the line of sight looking forward and a view of the displayed content entering the optical assembly from the integrated image light source And provides an integrated camera with a view of the surroundings along the line of sight looking downward for user gesture recognition through the integrated gesture recognition facility. In embodiments, the gesture recognition facility may identify the movement that the eyepiece interprets as an instruction to the eyepiece. Movement can be hand movements, arm movements, finger movements, foot movements, leg movements, and the like. The integrated camera can see not only the surroundings of the eyes looking forward but also the eyes looking downward for gesture recognition. The integrated camera may have segmented optical elements to simultaneously image a forward-looking view of the eye and a downward-facing view of the eye. In addition, the eyepiece may have an active sensing system for indirect sensing and interpretation of the user &apos; s body movement, wherein the active sensing system provides an active signal through the optical assembly along the eye view looking downward. Active signals can be IR, sonar, RF, and other active signals. The active sensing system may include a 3D active depth sensor for sensing the position of at least one of the user's hands, feet, body, or the like. To further provide user gesture recognition, an active sensing system may be used in conjunction with an integrated camera.

In embodiments, the eyepiece may include dual modes for marker location and tracking. A GPS for the general marker position in the vicinity of the POI can be generated and then another second marker can be generated. The second marker can be generated through sensor reading, image processing, image recognition, user feedback, and the like. This second marker can be used for tracking between acquisitions of GPS readings. In embodiments, a second marker may be used to provide a distance to or from the point of interest. The dual marker system can provide the distance, time and direction between the two points to the user. This point can be a point of interest for travel, transportation, business, commerce, and so on. Dual mode for marker location and tracking can allow users to find items to buy, items to visit, travel destinations, traffic patterns, and more. Transportation items can include your car, train, airport, taxi stand, taxi, subway, etc. Items of commerce may include various items such as, but not limited to, food, entertainment, shopping, clothing, books, services, and the like. In embodiments, the items to be found may be tourist spots, restaurants, parks, streets, and the like. There can be ecosystems of markers ranging from QR codes to a wide range of communication devices (routers and switches) or passive sensors (RFID tags that can be pinged), all of which know what content the backend network should deliver Such as the exact location at which the marker is placed, or any content itself specific to the location of the marker. Two markers can be used to help a single party orientate or triangulate the glasses to provide precise orientation and distance information, which can be more difficult to obtain with any single marker (especially the invisible ones). have. In embodiments, the markers may be processed. The eyepiece can recognize two markers in close proximity / field of view and can act on them at the same time (e.g., for triangulation) or assign a priority to one of them (e.g., in an ad scenario, May have priority over free markers; safety-oriented markers may have priority over ad markers, and others). The marker may come from the glasses, but may also be placed by other glasses or other systems (e.g., a system of advertisers, government parties, etc.), for example.

In embodiments, the system may include an interactive head-mounted eyepiece worn by a user, wherein the eyepiece comprises an optical assembly through which the user views the ambient environment and the displayed content, an integrated image An integrated processor for generating a marker for a point of interest based on the light source and GPS readings and storing the marker in a memory of the eyepiece, wherein the integrated processor generates a second marker in association with the GPS point and a second marker And stores it in the memory. In embodiments, the second marker may be generated through at least one of sensor reading, image processing, image recognition, user feedback on the current position, and the like. The second marker may be used to calculate at least one of the distance, direction, and time to the point of interest. In embodiments, the point of interest may be at least one of a tourist attraction, a restaurant, a park, a street, and the like. In embodiments, the GPS point may be used with the second marker to provide at least one of distance, direction, time, etc. to a commercial item. In embodiments, the GPS point may be used with the second marker to provide at least one of distance, direction, time of way to traffic mode, tracking of traffic mode, and the like. In such embodiments, the mode of transportation may include at least one of a train, a subway, an automobile, and the like. In embodiments of this system, a GPS point may be used to provide tracking of points of interest with the second marker. In embodiments, a GPS point may be used to provide tracking of a commercial item with a second marker. In embodiments, the user feedback may be a verbal input to the eyepiece. The second marker may be generated through various means. By way of example, the second marker may be based on the processing of the still and / or video image captured by the eyepiece to obtain positional information for the subject of the image. In embodiments, the second marker may be based on data obtained from an Internet search, a QR code, a bar code, an object scan, or the like.

In various embodiments, the eyepiece may include an earpiece to provide augmented hearing that allows the user to hear additional audio as well as its surroundings. Such audio may include game content, sports commentary, and the like. In embodiments, the microphone and / or earphone may play audio in a binaurally or otherwise manner. In embodiments, a bone conduction earpiece may be used with the eyepiece. These earpieces allow the user to transmit an audio wave through the skull to the inside, thereby bypassing the user's eardrum. In embodiments, the earpiece may be used with cheekbones immediately preceding the user &apos; s ear and / or with other bones capable of transmitting audio. Accordingly, the user can monitor or hear the audio while recognizing its surroundings. In embodiments, the earpiece may also use a sound laser, whereby the earpiece emits sound waves through the use of a laser. In embodiments, the earpiece may also employ a device that allows the user to experience enhanced volume and / or clarity of the sound generated by the external sound or earpiece. In various embodiments, the eyepiece earpiece may reproduce and / or transmit audio from the radio, wirelessly acquired audio, audio over the Internet, or the like. In embodiments, the eyepiece may also transmit satellite radio audio to the user.

In various embodiments, the eyepiece may include RF shielding for the brain and / or other body parts of the user. In embodiments, any portion of the eyepiece transmitting the electromagnetic field may be shielded by a barrier made of a conductive or magnetic material, or otherwise. In embodiments, the shield may include a sheet metal, a metal screen, a foamed metal, a foam, or the like. The hole in the shield or mesh may be considerably smaller than the wavelength of the radiation or other radiation being blocked. In embodiments, the interior or other portion of the eyepiece and / or eyepiece enclosure may be coated with metallic ink or other material to provide shielding. Such metals may be copper, nickel, etc., which are in the form of very small particulates. These metals can be sprayed onto the enclosure. In further embodiments, such RF shields may be worn by the user to prevent the various frequencies from reaching the user's brain, eyes, or other body parts.

In embodiments, the interactive head-mounted eyepiece worn by the user includes an optical assembly through which a user views the ambient environment and the displayed content therethrough, and an integrated image light source, a wireless element, and a shield for introducing the content into the optical assembly Wherein the radio element is capable of emitting electromagnetic radiation, and the shield shields radiation from being emitted from a portion of the eyepiece. In further embodiments, the shield may be disposed to protect the user from radiation. In addition, the shield may be arranged to protect the user and others from radiation. In embodiments, the shield may shield at least the user's brain, part of the user's brain, other parts of the user's body, and the like. In embodiments, the shield may comprise at least one of a conductive material, a magnetic material, a sheet metal, a metal screen, a mesh, and a foamed metal. In the embodiments described herein, the shield may include apertures that are smaller than the wavelength of the particular radiation, and the apertures may be less than the wavelength of the radiation emitting from the eyepiece. In embodiments, the shield may comprise at least one of metal ink, copper ink, nickel ink, and the like. In embodiments, the shield may coat the interior of the eyepiece. In embodiments, the shield may be disposed on at least one of the arm of the eyepiece, the forehead section of the eyepiece, or the like. In embodiments, the shield may be disposed on at least the forehead section of the eyepiece and the arm of the eyepiece. In embodiments, the shield may be worn by the user.

In one example, the control aspects of the eyepiece include IR, heat, power, ozone, carbon monoxide, etc as inputs; A microphone as an additional input device; A voice command as an operation for the wearer to start the command; A head up display as a command and control interface through which input can be reflected; A usage guide application that provides guidance while reducing the need to use a user's hand in maintenance and assembly; A visual display that provides feedback to the wearer based on wearer &apos; s motion and sensor input, and the like. For example, a mechanic may be the wearer of an eyepiece, where the mechanic uses an eyepiece to help maintain the vehicle. A usage guide application running through the eyepiece can provide a mechanic with hands-free usage instructions and access to computer-based expert knowledge while diagnosing problems in the vehicle. In addition, this application program can provide instructions on how to use procedures that are not familiar to mechanics. The eyepiece can also monitor a variety of sensor inputs (e.g., IR, heat, power, ozone, carbon monoxide, and other sensors) related to diagnostics and safety so that the sensor input is accessible by the use descriptive application and / It can be accessed directly by a mechanic. The application also includes a microphone on which a voice command can be received; A user guidance information, a head-up display for displaying a 2D or 3D representation of the portion of the vehicle being serviced; Feedback on time and cost of repair, and the like. In embodiments, the eyepiece may provide a hands-free virtual assistant to the mechanic to assist the mechanic in diagnosing and repairing the vehicle.

In one example, the control aspect of the eyepiece is that the eyepiece enters an 'activity' mode, such as a 'shopping' activity mode (eg, the user commands the eyepiece to enter store shopping mode) Sensing a proximity to a shopping area (perhaps even a shopping area of interest to the wearer), which may be further developed through self-monitoring in part and by learning the wearer's shopping preferences can do. Continuing with this example, entering an activity state (e.g., a shopping activity state) while driving a vehicle may include an object detector as a sensing input or sensing device, a head-mounted camera as a user motion acquisition input and / An eye movement as a user movement or action for controlling or initiating an instruction, an instruction and control mode in which the input can be reflected, and / or a 3D navigation interface as an interface, an instruction input and an application program for controlling the user interface An electronic commerce application, a navigation system controller communicating or connecting with an external system or device, a vehicle navigation system as an external device to be controlled and / or interfaced, an external application program While standing advertising equipment, operation can be combined with the wearer feedback as a target or a target tracking system with about shopping opportunities within the field. For example, a wearer may enter a shopping area while driving his vehicle, and an eyepiece that detects the presence of a shopping area (e.g., via GPS, looking directly at a target through an integrated camera, etc.) (E. G., Enabled and / or approved by the wearer). The eyepiece can then be detected, for example, by a head-mounted camera, by an object detector that finds shopping opportunities, in front of the billboard, in front of the store, and so on. In addition, the eyepiece eye detection system on the eyepiece can monitor where the wearer is looking, and possibly highlight information about the target (e.g., the sale item or special discount currently available in the store) at the location of the wearer's gaze . For example, the wearer &apos; s eye movement can also be tracked to change a target of interest, or for command input (e.g., quick nudge indicating a selection command, downward eye movement indicative of a command for additional information, etc.) . For example, a user &apos; s iris or retina may be tracked to provide a control input. The eyepiece includes a 3D navigation interface projection to assist the wearer in providing information associated with his or her surroundings, and a shopping activity state (e.g., receiving input from a wearer, providing output to a 3D navigation interface, And interfacing with an application program, etc.). The eyepiece can, for example, utilize a navigation system controller to interface with the vehicle navigation system and thus include the vehicle navigation system in the shopping experience. As an alternative, the eyepiece can use its own navigation system, for example, in lieu of or in order to reinforce the vehicle system, when the wearer wishes to be provided with a direction to walk down from the vehicle. As part of the shopping activity state, the eyepiece may interface with the external advertising facility to provide, for example, current transactions for nearby merchants, special discounts, pop-up advertisements, and the like. External advertising facilities may also be associated with third party advertisers, publishers, merchant support facilities, and may contribute to information provided to the wearer. In embodiments, the wearer can be provided with feedback associated with the activity state through entering the activity state, for example, in the form of information associated with the identified target for the shopping activity state .

In one example, the control aspects of the eyepiece include receiving e-mail as a triggering event, tracking inertial movement as a user motion capture input device, finger drag-and-drop and swipe movement as user movement or action to control or initiate an instruction, A searchable list as a command and control interface that can be reflected, an information delivery as a kind of application program on an eyepiece that can use commands and respond to an input, a communication system from an interface on an eyepiece for an external system and apparatus, An iris acquisition and recognition system as an external application for external systems and devices, and the like. For example, the wearer may receive an invoice via e-mail, and the e-mail may be sent to the user to initiate an application in the eyepiece, for example, with a visual and / or audible alert to trigger an operating mode of the eyepiece, , It can enter the eyepiece as an 'event' for the wearer. The wearer may have a plurality of control mechanisms - e.g., via a hand gesture interface (e.g., via a camera and hand gesture application embedded in the eyepiece, where the wearer drags the information in the email or email to a file, application, And respond to email events via 'drag and drop', swipes, etc. via wearer's fingers and hands. The wearer can call the searchable list of invoices and make other payments. The user can deliver information from the email (e.g., billing information, account number, bill amount, etc.) to an external system and device (such as a payment system for paying bills) via an eyepiece application. In embodiments, the eyepiece and / or payment system may require biometric ID verification (e.g., fingerprint capture, iris capture recognition, etc.), or identification verification via others.

In one example, the control aspect of the eyepiece may include a combination of using an inertial user interface as a user motion capture input device to provide an indication to an external display device through the eyepiece. For example, a wearer may wish to provide instructions to a group of people from available presentations through the equipment of the eyepiece. The wearer may use a physical 3D or 2D mouse (e.g., an inertial motion sensor, a MEMS inertial sensor, an ultrasound 3D motion sensor, an accelerometer, etc.), a virtual mouse, a virtual touch screen, a virtual touch screen, etc. to provide an interface for manipulating content in a presentation You can get help through the use of keyboards. The presentation can be viewed and manipulated through an eyepiece, but can also be exported in real time to an external router connected to, for example, an external display device (e.g., a computer monitor, projector, video screen, TV screen, etc.). Accordingly, the eyepiece can provide a way for other people to see what the wearer sees through the eyepiece and through the control equipment of the eyepiece under control, so that the wearer can view the multimedia content made possible through the eyepiece, Can be exported to.

In one example, the control aspects of the eyepiece may include a combination of using an event / data feed and a sensing input / sensing device, for example, where security events and acoustic sensors can be implemented. A security alert can be sent to the soldier, and an acoustic sensor is used as an input device to monitor the audio content in the surrounding environment, the direction of shooting, and the like. For example, a security alert is broadcast to all military personnel in a particular area and, by warning, the eyepiece monitors the built-in acoustic sensor array, which analyzes the loud sound to identify the type of sound source and the direction To activate the application. In embodiments, other events and / or data feeds, sensing inputs and / or sensing devices, etc., as described herein, may also be applied.

In one example, the control aspect of the eyepiece may include a combination of using an event / data feed and a user motion capture input / device, e.g., for a request for input and use of a camera. A soldier may be in a position of interest, such as when the request is accompanied by an indication of what to shoot, and receives a request for a photo or video from his location. For example, a soldier is at a checkpoint, and at some central headquarters, a person of interest is determined to be able to attempt to cross the checkpoint. The central command may then provide an instruction to the user proximate to the checkpoint to record and upload video and video, and in embodiments, this may be performed automatically, without the need for the soldier to manually turn on the camera. In embodiments, other events and / or data feeds, user action capture inputs, and / or devices, etc., as described herein may also be applied.

In one example, the control aspect of the eyepiece is that of using an event / data feed and user movement or motion to control or initiate commands, such as when a soldier is entering an 'active state' and using a hand gesture for control Combinations thereof. The soldier may be in an active state ready to engage the enemy, and the soldier uses the hand gesture to silently command the eyepiece within the engagement command and control environment. For example, a soldier may suddenly get into a spot by new information received that puts the eyepiece in an enhanced alarm state. In this state, the requirement is that silence may be required, so the eyepiece transitions to hand gesture command mode. In embodiments, other events and / or data feeds, such as those described herein, user actions or actions for controlling or initiating commands, etc., may also be applied.

In one example, the control aspect of the eyepiece includes a combination of an event / data feed and an instruction / control mode and interface that can be reflected in the input, such as entering a type of environment and using a virtual touch screen can do. The soldier can enter the weapon system area and the virtual touch screen becomes available to the wearer for at least a portion of the control of the weapon system. For example, a soldier enters a weapon vehicle, the eyepiece detects that the weapon system is present, and that the soldier is authorized to use the weapon, and presents a virtual fire control interface with a virtual touch screen. In embodiments, other events and / or data feeds, commands and / or control modes and interfaces through which inputs may be reflected, etc., may also be applied, as described herein.

In one example, the control aspect of the eyepiece uses an application on the platform that can use / respond to / input events / data feeds and commands for safety events, for example, with easy access to information about the pilot And the like. A military pilot (or a person in charge of a flight check of an unmanned aircraft) can receive a safety event notification when approaching an aircraft before takeoff, and an application appears to guide the pilot to a preflight check. For example, a drone expert approaches the drones to prepare for launch, and an interactive check procedure is displayed to the soldier by the eyepiece. In addition, the communication channel can be opened to the pilot of the drone, so the pilot is included in the pre-flight check. In embodiments, other events and / or data feeds, such as those described herein, an application on a platform that is capable of using commands and / or responding to input, etc., may also be applied.

In one example, the control aspect of the eyepiece is a combination of event / data feed and use of communication or connection from an on-platform interface to an external system and device (e.g., a soldier entering a location and a graphical user interface (GUI) . &Lt; / RTI &gt; A soldier can enter a location where it needs to interact with an external device, where the external device interfaces through a GUI. For example, a soldier is on a military move, and a soldier is presented with a GUI that opens an interactive interface that instructs the soldier what to do during the different phases of movement. In embodiments, other events and / or data feeds, such as those described herein, communications or connections from an on-platform interface to external systems and devices, etc., may also be applied.

In one example, the control aspects of the eyepiece may include a combination of an event / data feed and using a useful external device to be controlled, such as for the instructions and weapon systems provided. The soldier may be provided with a feed of instructions or instructions, where at least one instruction is about the control of the external weapon system. For example, a soldier may be manipulating a cannon, and the eyepiece not only provides performance and procedural information related to the weapon, but also provides a feed, such as instructions, corrections, etc., associated with the aim. In embodiments, other events and / or data feeds, such as those described herein, useful external devices to be controlled, etc., may also be applied.

In one example, the control aspects of the eyepiece may include a combination of using an event / data feed and an application to a useful external device, such as in a security event / feed and biometric feature capture / recognition. A soldier can be sent a security event notification to capture a particular person's biometric characteristics (fingerprint, iris scan, gait profile) (e.g., via a secure feed) / RTI &gt; and / or &lt; / RTI &gt; through an external biometric application (served from the cloud). In embodiments, other events and / or data feeds, such as those described herein, applications for external devices, etc., may also be applied.

In one example, the control aspects of the eyepiece include a combination of using feedback on an event / data feed and a soldier associated with an external device and an application (e.g., entering an active state and a soldier being provided with a display of information) . &Lt; / RTI &gt; The soldier can place the eyepiece in an active state for military staging, readiness, action, return reporting, etc., and as a feedback on putting the active state in the active state, the soldier receives a display of information about the entered state. For example, a soldier enters a state of gathering for a mission, where the eyepiece brings information (including security equipment, additional training, etc.) from a remote server as part of a mission that a soldier must accomplish during the gathering. In embodiments, other events and / or data feeds, such as those described herein, feedback related to external devices and / or external applications may also be applied.

In one example, the control aspect of the eyepiece may include a combination of a sensing input / sensing device and a user motion acquisition input / device, such as in an inertial motion sensor and a head tracking system. The head movement of the soldier can be traced through the inertial movement sensor (s) in the eyepiece for nudge control of the eyepiece, detection of the direction of view of the eyepiece, and so on. For example, the soldier may be aiming at the weapon system, and the eyepiece senses the heading direction of the soldier's head through the inertial motion sensor (s) to provide a continuous aiming of the weapon. In addition, the weapon system can move continuously in response to the direction of the soldier's gaze, and thus may be ready to continue shooting targets. In embodiments, other sensing inputs and / or sensing devices, user motion acquisition inputs and / or devices, etc., as described herein, may also be applied.

In one example, the control side of the eyepiece includes a combination of a light sensor and a sensing input / sensing device and user motion or motion to control or initiate an instruction, such as in blink, blink, and other motion can do. The state of the soldier's eye can be detected by an optical sensor included in the optical chain of the eyepiece for example using eye movements for control of the eyepiece. For example, a soldier may be aiming at his rifle, where the rifle can be shot through a control command from the eyepiece (in the case of a sniper, the command through the eyepiece is triggered manually by the trigger Errors can be reduced). The soldier can then fire the weapon through a command initiated by a light sensor that detects a predetermined eye movement, such as in a command profile maintained on the eyepiece. In embodiments, other sensing inputs and / or sensing devices, such as those described herein, user motions or actions for controlling or initiating commands, etc., may also be applied.

In one example, the control aspect of the eyepiece may include a combination of a sensing input / sensing device and an instruction / control mode and interface that may be reflective of the input, such as in a proximity sensor and a robot control interface. Proximity sensors integrated into the eyepiece can be used to detect soldier proximity to the robot control interface to enable and enable use of the robot. For example, a soldier walks up to an explosive detection robot, and the robot automatically activates and initializes a configuration for this particular soldier (e.g., configuration for a soldier's preferences). In embodiments, other sensing inputs and / or sensing devices, commands and / or control modes and interfaces, etc., to which inputs may be reflected may also be applied, as described herein.

In one example, the control aspect of the eyepiece includes a combination of using a sensing input / sensing device and an application on a platform that is capable of using and responding to commands, such as in an audio sensor and a music / sound application can do. The audio sensor may be able to initiate and / or adjust the volume, ambient tone, mute, etc., for the music to monitor ambient sounds and to assist in responding to undesirable ambient sounds. For example, a soldier is picked up on a vehicle, and the engine of the vehicle is initially turned off. At this time, the soldier may not have any other mission except to take a break, so the vehicle commences the music to help take a break. When the engine of the vehicle is turned on, the music / sound application program adjusts the volume and / or initiates additional unmute audio to help keep the music input the same as before the engine started. In embodiments, other sensing inputs and / or sensing devices, such as those described herein, an application on a platform that is capable of using commands and / or responding to inputs, etc., may also be applied.

In one example, the control aspect of the eyepiece includes a combination of sensing input / sensing device and using communication or connection from an on-platform interface to an external system and device, such as in a manual IR proximity sensor and an external digital signal processor . The soldier may be monitoring the night scene with a manual IR proximity sensor, which indicates motion, and the eyepiece initiates a connection to an external digital signal processor to help identify the target from the proximity sensor data. In addition, an IR imaging camera can be disclosed to provide additional data to the digital signal processor. In embodiments, other sensing inputs and / or sensing devices, such as those described herein, communication or connection from an on-platform interface to an external system and device, etc. may also be applied.

In one example, the control aspect of the eyepiece may include a combination of a sensing input / sensing device and using a useful external device to be controlled, such as for an acoustic sensor and an inorganic system, wherein the eyepiece worn by the soldier For example, an explosion or a fire, wherein the eyepiece then initiates control of the weapon system for possible action on a target associated with the production of loud sounds. For example, a soldier is on a guard duty, and a shot is heard. The eyepiece can detect the direction of the shooting and guide the soldier to the position where the shooting was done. In embodiments, other sensing inputs and / or sensing devices, such as those described herein, useful external devices to be controlled, etc., may also be applied.

In one example, the control aspect of the eyepiece may include a combination of using a sensing input / sensing device and an application for its useful external device, such as for an external application program for a camera and an instruction. The camera built into the soldier's eyepiece can see a target icon indicating that there is an indication, and the eyepiece has access to an external application for indication. For example, a soldier is moved to a gathering area, and upon entry, the eyepiece camera views the icon, accesses instructions from outside, provides instructions to the soldier about what to do, All steps can be automatic so that instructions are provided without having to. In embodiments, other sensing inputs and / or sensing devices, such as those described herein, applications for external devices, etc., may also be applied.

In one example, the control aspect of the eyepiece may include a combination of sensing input / sensing devices and using feedback to the user related to the external device and application, such as in a visual display from a GPS sensor and a remote application have. The soldier may have a built-in GPS sensor that transmits / streams position coordinates to a remote location facility / application that sends / streams a visual display of the surrounding physical environment to the eyepiece for display. For example, a soldier could always see the surrounding environment through an eyepiece, and a visual display overlay, which allows the soldier to have an augmented reality view of the surrounding environment, even as a change location, Streaming. In embodiments, other sensing inputs and / or sensing devices, such as those described herein, external devices and / or feedback related to external applications may also be applied.

In one example, the control aspects of the eyepiece include a combination of user motion capture input / device and user motion or motion to control or initiate an instruction, such as in a body motion sensor (e.g., motion sensor) and arm motion . A soldier can have a body motion sensor attached to his arm, where the movement of his arm delivers commands. For example, a soldier can have a motion sensor on his arm, his arms move the same in an aircraft landing light system, and thus the lighting device that the assistant usually helps to land on can be bigger and better visible have. In embodiments, other user action acquisition inputs and / or devices, such as those described herein, user actions or actions for controlling or initiating commands, etc., may also be applied.

In one example, the control aspects of the eyepiece may include a combination of a user motion capture input / device and an instruction / control mode and interface that may be reflective of the input, such as a wearable sensor set and predictive learning based user interface have. The soldier may wear a sensor set wherein data from the sensor set is continuously collected and fed to the machine learning facility via a learning based user interface where the soldier accepts, rejects, corrects can do. For example, a soldier can perform the same task on Monday morning, usually in the same physical way, and a machine learning facility can be used to repair certain equipment, fill in certain forms, play specific music, You can set up a learned routine that will be provided to a soldier next Monday morning, such as a meeting with a person, a reminder to do guitar, and so on. In addition, the soldier can modify the outcome of the learning through direct editing of the work, as in the learned behavior profile. In embodiments, other user action acquisition inputs and / or devices, commands and / or control modes and interfaces through which inputs may be reflected, etc., may also be applied, as described herein.

In one example, the control aspect of the eyepiece includes a combination of using a user motion capture input / device and an application on a platform that can use and respond to commands, such as in a finger track camera and video application can do. The soldier can control the direction in which the built-in eyepiece camera captures video through the resident video application. For example, a soldier may be watching a battle scene where a soldier needs to shoot in a different direction (eg, the current engagement point) while gazing in one direction, such as guarding against new developments in an engagement have. In embodiments, other user action acquisition inputs and / or devices, such as those described herein, an application on a platform that is capable of using commands and / or responding to input may also be applied.

In one example, the control aspect of the eyepiece includes a combination of user motion capture input / device and using communication or connection from an on-platform interface to an external system and device, such as a microphone and voice recognition input and a steering wheel control interface . The soldier may change aspects of operating the vehicle through voice commands received via the eyepiece and transmitted to the vehicle's steering wheel control interface (e.g., via wireless communication between the eyepiece and the steering wheel control interface). For example, a soldier is driving a vehicle, and therefore the vehicle has certain operational functions that are ideal for the road. However, vehicles also have different modes under different conditions (e.g., off-road, snowing, muddy, in heavy rain, tracking other vehicles, etc.). In this example, the soldier can change modes through voice commands when the vehicle changes driving conditions. In embodiments, other user action acquisition inputs and / or devices, such as those described herein, communication or connection from an on-platform interface to an external system and device, etc., may also be applied.

In one example, the control aspect of the eyepiece may include a combination of a user motion capture input / device and a useable external device to be controlled, such as a microphone and voice recognition input and an automotive dashboard interface device. The soldier can use voice commands to control various devices associated with the vehicle's dashboard, such as heating and ventilation, radio, music, lighting, trip computers, and the like. For example, a soldier may be driving a vehicle through a rough terrain during a mission, so a soldier can not place the steering wheel in either hand to manually control the vehicle dashboard device. In this example, the soldier can control the vehicle dashboard device through voice control over the eyepiece. Voice commands via the eyepiece may be particularly advantageous, for example, in contrast to voice commands via the dashboard microphone system, because the military vehicle may fall into a very loud acoustic environment and, therefore, using a microphone in the eyepiece, Lt; RTI ID = 0.0 &gt; substantially &lt; / RTI &gt; In embodiments, other user action acquisition inputs and / or devices, such as those described herein, useful external devices to be controlled, etc., may also be applied.

In one example, the control aspect of the eyepiece may include a combination of user motion capture input / device and application to a useful external device, such as for a joystick device and an external entertainment application. The soldier can access the game joystick controller and play the game through an external entertainment application, such as a multiplayer game hosted on a network server. For example, a soldier may experience a downtime during installation and, based thereon, access a joystick device that interfaces with the eyepiece, which in turn accesses an external entertainment application. In embodiments, the soldier may be networked with other military personnel through the network. The soldier may be storing preferences, profiles, etc. associated with gameplay. The external entertainment application program may be adapted to play a game of a soldier in terms of, for example, its placement, the current state of readiness, the required state of readiness, past history, capability level, command location, rating, geographic location, . In embodiments, other user action acquisition inputs and / or devices, applications for external devices, etc., as described herein may also be applied.

In one example, the control aspects of the eyepiece include a combination of user motion capture inputs / devices and feedback to the user associated with external devices and applications, such as in an activity determination system and a tonal output or sound alert . &Lt; / RTI &gt; The soldier can access the activity determination system through the eyepiece to monitor and determine the status of the soldier's activities (extremes, resting, boredom, anxiety, during exercise, etc.) (E.g., pre-set, learned, typical, etc.) limitations, it is possible to provide various types of tone output or sound warnings. For example, a soldier may be monitored for a current health condition during a battle, where the soldier and / or others (e.g., medics, hospital staff, other members of the military's team, command center, etc.) When entering a dangerous level (indicating, for example, that a soldier was injured during battle), an audible signal is provided. As a result, others will be notified of the soldier's injuries and will be able to take care of injuries in a more time-effective manner. In embodiments, other user action acquisition inputs and / or devices, external devices, and / or feedback related to external application programs, such as those described herein, may also be applied.

In one example, the control aspect of the eyepiece includes a combination of user movement or motion for controlling or initiating an instruction, such as a tight fist and searchable list, and using an instruction / control mode and interface in which the input can be reflected can do. The soldier can make the navigable list appear on the ocular display with a gesture such as a tight fist. For example, an eyepiece camera can view a soldier's hand gesture (s), recognize and identify the hand gesture (s), and execute commands in accordance with a predetermined gesture-command database. In embodiments, the hand gesture may include a gesture of a hand, a finger, an arm, a leg, and the like. In embodiments, as described herein, other user actions or actions that control or initiate an instruction, commands and / or control modes and interfaces through which input may be reflected, etc. may also be applied.

In one example, the control aspects of the eyepiece include a combination of user movement or motion to control or initiate an instruction, such as head nodding and information display, and using an application on the platform to use / respond to the input . A soldier can display an information display application program with gestures such as head-to-head, arm movements, leg movements, eye movements, and the like. For example, a soldier may wish to access an application, a database, a network connection, etc. through an eyepiece, and may have access to his or her computer (e.g., a headset, soldier's head, soldier's helmet, The head nod can cause the display application to appear as part of the graphical user interface. In embodiments, other user movements or actions that control or initiate an instruction, such as those described herein, an application on a platform that is capable of using commands and / or responding to input, etc., may also be applied.

In one example, the control aspects of the eyepiece include user motions or actions that control or initiate commands, such as through an eye blinking and an API for external applications, and communications or connections to external systems and devices from the on- Lt; RTI ID = 0.0 &gt; a &lt; / RTI &gt; A soldier can access an external application program by displaying an application program interface, such as blinking of an eye, nodding of a head, motion of an arm or leg, and the like. For example, a soldier can access an external application via an API built into the eyepiece facility and do so with an eye blinking as detected by optical monitoring through an eyepiece optical system. In embodiments, other user motions or actions that control or initiate an instruction, such as those described herein, communication or connection to an external system and device from an on-platform interface, etc., may also be applied.

In one example, the control aspect of the eyepiece may include a combination of user movement or motion to control or initiate an instruction, or to use an external device to be controlled, such as foot tapping accessing an external range finder device . A soldier may have a sensor such as a motion sensor in his shoe that will detect movement of the soldier's foot and the soldier may use a tapping or the like of his foot to use an external range finder device to determine the distance to an object, Of the foot. For example, a soldier may be aiming at a weapon system and may be using both hands in this process. In this case, commanding by the foot operation through the eyepiece allows a "hands free" command. In embodiments, other user movements or actions that control or initiate commands, such as those described herein, useful external devices to be controlled, etc., may also be applied.

In one example, the control aspect of the eyepiece includes a combination of user motion or motion to control or initiate an instruction, and application of the useful external device, such as creating a symbol by hand and communicating an application program can do. A soldier can use hand-formed symbols to trigger information shared through external information delivery applications, such as external information feeds, photo / video sharing applications, text applications, and the like. For example, a soldier uses a hand signal to turn on the built-in camera, shares the video stream with others, stores it, and plays guitar. In embodiments, other user movements or actions that control or initiate an instruction, such as those described herein, an application to an external device, etc., may also be applied.

In one example, the control aspect of the eyepiece may include a combination of user movement or motion to control or initiate the command, such as head-to-head traversal and audible alarm, and feedback to the soldier associated with the external device and application have. The soldier may be wearing an eyepiece equipped with an accelerometer [or a similar sensor that can detect a g-force head sideways], where a soldier may experience a GeForce head sideways at a dangerously high level An audible alarm is heard as feedback to the user, as determined as part of the on-eyepiece or off-eyepiece application. In addition, the output of the accelerometer can be recorded and stored for analysis. For example, a soldier can experience Geforce head traversing from an adjacent explosion, and an eyepiece can sense and record sensor data associated with head traversing. In addition, dangerous level headstroke can trigger automatic operation by an eyepiece, such as sending an alert to another soldier and / or a command center, monitor the health of the soldier from other body-mounted sensors, and / or And may provide the soldier with an audible indication of his potential injury, and so forth. In embodiments, other user movements or actions that control or initiate commands, such as those described herein, feedback related to external devices and / or external application programs, etc., may also be applied.

In one example, the control aspect of the eyepiece may include a command / control mode and interface that can reflect the input, such as a graphical user interface and various application programs resident on the eyepiece, and an interface / Application programs, and the like. The eyepiece can provide the soldier with a graphical user interface and an application program presented for selection. For example, a graphical user interface that provides areas of different application programs (military, personal, civilian, etc.) can be projected to the soldier by the eyepiece. In embodiments, as described herein, other commands and / or control modes and interfaces through which inputs may be reflected, application programs on a platform that are capable of using commands and / or responding to inputs, etc., Can be applied.

In one example, the control aspects of the eyepiece include command / control modes and interfaces that may be reflective of input, such as a 3D navigation eyepiece interface and a navigation system controller interface to an external system, and communication from an on- Or using a connection. The eyepiece can enter the navigation mode and can be connected to an external system via the navigation system controller interface. For example, a soldier may be in a military operation, display a preloaded 3D image of the surrounding terrain through the eyepiece navigation mode, and automatically update the eyepiece for an existing interest object, such as an update, Lt; / RTI &gt; In embodiments, other commands and / or control modes and interfaces from which inputs may be reflected, as well as communications or connections from an on-platform interface to an external system, as described herein, may also be applied.

In one example, the control aspect of the eyepiece may include a combination of an instruction / control mode and interface that may be reflective of the input, such as an augmented reality interface and an external tracking device, and using an external device to be controlled. The soldier's eyepiece may enter an augmented reality mode and interface with an external tracking device to overlay information about the tracked object or person's position with the augmented reality display. For example, the augmented reality mode can include a 3D map, the location of a person determined by an external tracking device can be overlaid on the map, and the traced person shows traces as they move. In embodiments, other commands and / or control modes and interfaces, external devices that are useful to be controlled, etc., may also be applied, as described herein, as well.

In one example, the control aspect of the eyepiece may include a combination of an instruction / control mode and interface that may reflect input, such as a translucent display mode and a simulation application, and an application using an interface and an external device . The eyepiece can be placed in translucent display mode to enhance display of the simulation display application to the soldier. For example, before a soldier is preparing for a mission and before entering the battlefield, the soldier is provided with a simulation of the mission environment and the user does not need to see the actual environment around him during the simulation, . In the embodiments, other commands and / or control modes and interfaces to which inputs may be reflected, as well as applications for external devices, etc., as described herein, may also be applied.

In one example, the control aspects of the eyepiece include a combination of command / control modes and interfaces, where input can be reflected, such as an auditory command interface and tone output feedback, and feedback using the user associated with external devices and applications . &Lt; / RTI &gt; The soldier can place the eyepiece in an audible command interface mode and the eyepiece responds back to the tone output as feedback from the system that the eyepiece is ready to receive audible commands. For example, the auditory command interface may include at least a portion of an audible command interface at an external location (e.g., on a network), and a tone is provided when the entire system is ready to receive audible commands. In embodiments, other commands and / or control modes and interfaces to which inputs may be reflected, as well as feedback related to external devices and / or external application programs may also be applied, as described herein.

In one example, the control aspects of the eyepiece include communications applications and network routers, applications on the platform that can use and respond to commands / commands, and communications or connections from the on-platform interface to external systems and devices , Where the soldier can open a communication application and the eyepiece automatically retrieves the network router for connection to the network infrastructure. For example, a soldier is on the battlefield with his unit, and a new base camp is built. The soldier's eyepiece can be connected to a secure wireless connection once the communication facility is established. In addition, when telecommunications equipment is built, the eyepiece can alert the soldier, even if the soldier has not yet attempted to communicate. In embodiments, other applications on the platform using commands and / or responding to inputs, such as those described herein, communications or connections from the on-platform interface to external systems and devices, etc., are also applied .

In one example, the control aspect of the eyepiece may include a combination of an application on the platform that can use and / or respond to commands, such as a video application and an external camera, and using a useful external device to be controlled . The soldier can interface with the installed camera, for example, for monitoring at the battlefield. For example, a mobile installable camera can be dropped from an aircraft, and the soldier then connects to the camera via an eyepiece video application. In embodiments, other applications on the platform, such as those described herein using commands and / or responding to inputs, useful external devices to be controlled, etc., may also be applied.

In one example, the control aspect of the eyepiece is a combination of using an application on a platform and / or an application on an external device, such as a search application on an eyepiece and an external search application, . &Lt; / RTI &gt; Search applications on the eyepiece can be augmented with external search applications. For example, a soldier can search for the identity of a suspected individual, and when the search on the eyepiece does not find anything, the eyepiece connects to the external search facility. In embodiments, other applications on the platform, such as those described herein, using commands and / or responding to inputs, applications to external devices, etc., may also be applied.

In one example, the control aspect of the eyepiece uses feedback to the soldier involved in applications on the platform and external devices and applications, such as entertainment applications and performance indicator feedback, using commands and responding to input And the like. An entertainment application may require rest, but may be worried in other ways, such as during a recession that requires a break, but is still in a critical state of distress, Can be used as a resting mechanism for a soldier, and performance feedback is designed for a soldier in a given environment. For example, a soldier is in transit and can enter into battle soon. In this example, the entertainment application may be an action-thinking game to increase attention and aggressiveness, where the performance indicator feedback is designed to maximize the desire of the soldier to perform in a fast and efficient manner and to fully consider the problem. In embodiments, other applications, external devices, and / or feedback related to external applications on the platform that are capable of using commands and / or responding to inputs, such as those described herein, may also be applied .

In one example, the control aspect of the eyepiece includes a combination of communicating or connecting from an on-platform interface to an external system and device and using an external device to be controlled, such as a processor interface on an eyepiece to an external facility and an external projector . The eyepiece processor may be connected to an external projector so that others can view content available on the eyepiece. For example, a soldier can be on the battlefield and access content that needs to be shared with other people who are not wearing eyepieces (e.g., those who are not in the army). In this example, a soldier's eyepiece can interface with an external projector and can feed content from the eyepiece to the projector. In embodiments, the projector may be a pocket projector, in a vehicle, in a conference room, at a remote location, or the like. In embodiments, the projector may also be integral to the eyepiece, such that the content can be projected from an integral projector from the outside. In embodiments, other communications or connections from an on-platform interface to an external system and device, such as those described herein, as well as useful external devices to be controlled, etc., may also be applied.

In one example, the control aspects of the eyepiece may include a combination of communicating or connecting from an on-platform interface to an external system and device, such as an audio system controller interface and an external sound system, and using an application to an external device have. The soldier may connect the audio portion of the eyepiece facility (e.g., music, audio playback, audio network file, etc.) to an external sound system. For example, a soldier may patch communications received by an eyepiece to a vehicle sound system so that others can hear it. In embodiments, other communications or connections from an on-platform interface to an external system and device, such as those described herein, an application to an external device, etc., may also be applied.

In one example, the control aspects of the eyepiece include a combination of communication or connection from an on-platform interface to an external system and device, such as a stepper controller interface and status feedback, and feedback to a soldier associated with external devices and applications . The soldier can access and control the mechanism with digital stepper control via the stepper controller interface, where the mechanism provides feedback to the soldier about the status of the mechanism. For example, a soldier who works to remove a street barricade may have a crane mechanism on his vehicle, and a soldier may interface directly with the crane mechanism through an eyepiece. In embodiments, other communications or connections from an on-platform interface to an external system and device, such as those described herein, feedback related to external devices and / or external applications may also be applied.

In one example, the control aspect of the eyepiece may include a combination of an external device to be controlled and an application for that external device, such as a storage assist device and an automatic backup application program. Soldiers on the battlefield can be provided with data storage facilities and associated automatic backup applications. This is particularly true, for example, when the storage facility may be located in a military vehicle, so that data can be backed up from the eyepieces of multiple soldiers to the vehicle and the network link is not available for download to the remote backup site. The storage facility may be associated with a camp, may be associated with some of the soldiers in the battlefield (e.g. within a group), may be located in the soldier itself, or may be other. In embodiments, a local storage facility may upload a backup when a network service connection is made available. In embodiments, other useful external devices to be controlled, such as those described herein, applications to external devices, etc., may also be applied.

In one example, the control aspect of the eyepiece may include a combination of external devices to be controlled and feedback to the soldier associated with the external device and the application program, such as feedback from an external payment system and system. A soldier may have access to a military managed payment system where the system provides feedback to the soldier (e.g., receipts, account balances, account activity, etc.). For example, a soldier can make a payment to a vendor through an eyepiece where the eyepiece and an external payment system exchange data, permissions, funds, and the payment system provides feedback data to the soldier. In embodiments, other useful external devices, external devices, and / or feedback related to external applications to be controlled, such as those described herein, may also be applied.

In one example, the control aspects of the eyepiece include a combination of application to the external device and feedback to the military associated with the application, such as feedback with information display and information display from an external 3D mapping-rendering facility . The soldier can display the 3D mapping information data via the eyepiece, where the mapping facility is able to display the 3D mapping information data on the basis of the geographical area and the geographical area based on the request from the other people in the area, And provides feedback to the soldier, by others, based on the associated changes. For example, a soldier may be receiving 3D map rendering from an external application, where the external application also provides 3D map rendering to at least a second soldier in the same geographic area. The soldier can then receive feedback (e.g., its location, ID information, history of movement, etc.) displayed on the 3D map rendering from external equipment associated with the second soldier. In embodiments, other applications to the external device, such as those described herein, external devices, and / or feedback related to external applications may also be applied.

In embodiments, the eyepiece may provide various types of guidance to the user in response to a medical condition. As a first example, a user may use an eyepiece for training purposes to simulate medical situations that may occur in others, such as in combat, in training, during work / on duty. The simulation can be tailored to professional medical or non-medical personnel.

As an example, a low-level combat soldier can use an eyepiece to view a medical simulation as part of a training module to provide training to respond to medical conditions on the battlefield. The eyepiece can provide an augmented environment where the user sees overlaid injury to other soldiers to simulate what is common or happening on the battlefield. The soldier may then be asked to respond to the presented situation through the user interface. The user may receive step-by-step instructions of the behavioral policy in providing emergency medical care on the battlefield, or the user may perform the action to be corrected later, in response to the situation, until an appropriate response is provided.

Similarly, eyepieces can provide a training environment for professional medical personnel. The eyepiece can present the user with a medical emergency or situation that requires a medical response to train a professional medical staff. The eyepiece can reproduce the normal battle scenarios where the user must master the appropriate response and lifesaving skills.

For example, a user may be presented with an augmented reality of a wounded soldier suffering a gunshot wound on a soldier's body. The professional medical staff can then carry out the actions that they think are appropriate for the situation and can select actions that are deemed appropriate for the situation through the user interface of the eyepiece, Others can do. A user may perform a response through the use of sensors and / or input devices, or may enter his corresponding actions into the user interface via eye movements, hand gestures, and the like. Similarly, the user can select appropriate actions to be presented to him via the user interface through eye movements, hand gestures, and the like. When actions are performed and the user makes a decision about treatment, the user can be presented with additional guidance and instructions based on his performance. For example, if a user presents a soldier who is injured in a chest and the user begins to lift a soldier to a dangerous position, the user may be warned or prompted to change his treatment course. As an alternative, the user can be prompted with the correct actions to execute the appropriate procedure. In addition, the trainee may be presented with an example of a medical chart for a wounded person in a training situation, where the user may have to make his decision based, at least in part, on what is included in the medical chart. In various embodiments, the user's actions and performance may be recorded and / or documented by the eyepiece for further evaluation and indication after the training session is paused or otherwise interrupted.

In embodiments, the eyepiece may provide various types of guidance to the user in response to an actual medical situation at battle. For example, an untrained soldier may receive a step-by-step rescue order for a fellow soldier in medical emergency when the medic is not immediately available. When a fellow soldier is injured, the user can enter the type of injury, the eyepiece can detect the injury, or a combination of these can be done. From there, the user can be provided with lifesaving instructions to treat the wounded. Such an indication can be presented in the form of augmented reality in a step-by-step pointing process to the user. In addition, the eyepiece can provide the user with an augmented visual aids for the location of critical organs near the wound of the wounded person, anatomical overlay of the soldier's body, and the like. In addition, the eyepiece can take a video of the situation, which is then sent back to the medics who are not at the scene or are coming to the scene, so that the medic can guide the untrained user from the battlefield to the appropriate lifesaving technique . In addition, the eyepiece of the wounded person may be transmitted to the soldier's eyepiece, which transmits important information, such as information collected from the integrated type or related sensor of the wounded person, or may be transmitted to the soldier who is treating the wounded person based on information collected from the eyepiece of the wounded person And may be sent directly to a medic at a remote location to provide medical assistance to the wounded.

In other embodiments, when presented with a medical emergency on the battlefield, a trained medic may use the eyepiece to provide a dissection overlay of the soldier's body to better accommodate the current situation. By way of example only, and not by way of limitation of the present disclosure, when the injured person is bleeding due to a gunshot of the leg, the user may be asked to determine whether the artery has been shot, The view can be presented. The user can be presented with an appropriate treatment plan for a given injury through the eyepiece so that each step can be examined as the treatment progresses. These treatment plans can also be presented to the user in augmented reality, video, audio or other formats. The eyepiece can provide a medication plan to the medic in the form of augmented reality instructions in a step-by-step process. In embodiments, the user may also be presented with an augmented reality overlay of the wounded's organs to guide the medic to any procedure so as not to add harm to the military's organ during treatment. In addition, the eyepiece can provide the user with an augmented visual aids for the location of critical organs near the wound of the wounded person, anatomical overlay of the soldier's body, and the like.

In embodiments, an eyepiece may be used to scan a wounded retina to pull up a medical chart of a wounded person on the battlefield. This can alert the medics to possible allergies to treatment or other important problems, which can provide benefits during medical treatment.

In addition, if the wounded person is wearing an eyepiece, the device can provide the medic's glasses with information including the heart rate of the wounded person, blood pressure, breathing stress, and the like. The eyepiece may also help the user to observe the gait of the soldier to determine if a soldier has a head injury and may help the user determine the location of the bleeding or injury. This information may provide the user with information about possible medical care, and in embodiments, a selection of a treatment plan or treatment plan suitable for helping the user to treat the patient may be displayed to the user.

In other embodiments, the eyepiece may allow the user to monitor other symptoms of the patient for a mental health condition check. Similarly, a user can use an eyepiece to provide a patient with a calming treatment, such as providing a patient with eye movements, breathing exercises, etc., that can be examined to determine if the patient is exhibiting rapid eye movement have. In addition, the medic will be provided with information on life signs and health data of the wounded, because this information is collected from the eyepiece of the wounded soldier and transmitted to the eyepiece of the medical officer. This can provide real-time data from the wounded to the medic, for example, by measuring the blood pressure of the wounded person, without having to determine this data by himself.

In various embodiments, the user may be provided with an alarm from the eyepiece that informs the user how far the air or ground structure is from its position on the battlefield. This can provide important information to the medic and alert the medic to what procedure should be attempted if the time available is available in that situation, providing relief to the injured party informing them that a helping hand is coming You can tell him that he may or may need another source of help.

In other embodiments, the user may be alerted of his own vital signs if a problem is detected. For example, a soldier can be alerted if his blood pressure is too high, thereby informing him that he must be treated or, if possible, excluded from combat to restore his blood pressure to a safe level. The user may also be informed of other such personal data, such as pupil size, heart rate, gait changes, etc., to determine if the user is experiencing a medical problem. In other embodiments, the user &apos; s eyepiece may also alert the user's medical condition to medical personnel at other locations to provide a helping hand to the user, whether or not the user needs this help. In addition, general data can be counted from multiple eyepieces to provide the commander with details about his wounded soldier, how many soldiers are in combat, and how many of them have been injured.

In various embodiments, a trained professional medical staff may use the eyepiece in a medical response other than combat. These eyepieces may have uses similar to those described above in or outside the home of a medic, other than in combat situations. In this way, the eyepiece can be used as a means of documenting the medical procedure, assisted augmented reality during a medical procedure, at a military base, or otherwise, performing medical procedures with the remote commander's guidance via video and / or audio Can be provided to the user. This can provide assistance in multiple situations where the medic may require additional assistance. An example of this can happen when the medic is engaged in training exercises, calisthenics outing, military hike, etc. This help may be important to the guitar, when it is the only first aid, when it is a newcomer, when it comes to a new situation.

In some embodiments, the eyepiece may provide user guidance in an environment related to military transport. For example, an eyepiece can be used in such an environment, such as during training, when engaged in combat, during surveillance or rescue mission, when moving equipment, when performing maintenance on an aircraft, and so on. These applications may be suitable for a variety of classes and levels of personnel.

For illustrative purposes, the user can receive audio and visual information via the eyepiece while riding in a transport and entering training exercises. This information can provide the user with details about the training mission, such as battlefield status, weather conditions, mission instructions, map of the area, and so on. The eyepiece can simulate real battle scenarios to prepare the user for battle. The eyepiece can also record user responses and actions through various means. Such data collection may allow a user to receive feedback on his performance. In addition, the eyepiece can then change simulations based on the results obtained during the training exercises, in order to change the simulations while in progress or to change future simulations for the user or various users.

In embodiments, the eyepiece may provide user guidance and / or interaction on the military transport when it is put into combat. The user can receive audio and visual information about the mission while boarding the aircraft. A checklist may be presented to the user to ensure that the user has the appropriate data and equipment for the mission. In addition, instruction manuals for the protection of equipment and proper use of safety equipment may be presented with information about the aircraft (such as emergency exits, oxygen tank position and safety devices). The user may be presented with instructions such as when to rest before the assignment and whether to take prescribed medicine for that purpose. The eyepiece can provide noise cancellation to the user for resting before the mission, and then inform the user when his rest is over and additional mission preparation begins. Additional information may be provided such as a map of the battle area, the number of people in the vehicle and / or battlefield, the weather conditions of the battle area, and the like. The device may provide links to other soldiers so that instructions and battle arrangements may include military interactions where the subordinates listen to the commander and do the guitar. In addition, information about each user may be formatted to suit his particular needs. For example, a commander may receive higher-level or more confidential information that may not need to be provided to a lower-ranking officer.

In embodiments, the user may use the eyepiece on a military transport aircraft that is under surveillance or rescue mission, where the eyepiece is located at a point of interest when flying over areas that may be used to obtain information about potential ground combat areas, Capture and store various images and / or video. The eyepiece can be used to detect the movement of people and vehicles on the ground and thereby detect enemy, structure or allied forces to defeat. The eyepiece can provide the ability to fly above and tag the map or image of the searched area or tag it to give specific color coding to the area that needs to be searched or still needs to be searched.

In embodiments, a user on a military transport carrier may be provided with a user manual and / or a checklist of the equipment to be stocked, a quantity and a location to be moved and a special handling instruction for various equipment. An alert for a vehicle approaching when loading or unloading goods to ensure security may be provided to the user.

For maintenance and safety of military aircraft, the user may be provided with preflight checks for proper functioning of the aircraft. Pilots may be alerted if proper maintenance is not completed prior to the mission. In addition, the aircraft operator may be provided with a graphical overview or list of aircraft history to track the history of aircraft maintenance.

In some embodiments, the eyepiece may provide user guidance in an environment related to a military fighter. For example, in training, in combat, in maintenance, in guitars, eyepieces can be used in this environment. These applications may be suitable for a variety of classes and levels of personnel.

As an example, the user may use the eyepiece for training for military fighter combat. The user may be presented with an augmented reality situation simulating the battle situation in a particular military jet or military aircraft. The user's responses and actions may be recorded and / or analyzed to provide additional information, ratings to the user, and to modify training exercises based on historical data.

In embodiments related to actual combat, the user may be presented with information showing the allied and VIA military aircraft that are in the vicinity of the user and / or accessing the user. The user can be presented with information about enemy aircraft such as maximum speed, operational capability and missile range. In embodiments, the user may receive information regarding the presence of a ground threat and receive alerts regarding the same. The eyepiece may be synchronized to the user's aircraft and / or aircraft instrument and meter so that the pilot can view emergency alerts and additional information about the aircraft that may not normally be displayed in the cockpit. In addition, the eyepiece can display the time of a few seconds to the target area, the time at which the missile fires or escapes from the aircraft, based on incoming threats. The eyepiece can suggest an operation to be performed by the pilot based on the surrounding environment, potential threats, and the like. In embodiments, the eyepiece can detect and display allied aircraft, even when such an aircraft is in stealth mode.

In embodiments, the user may be provided pre-flight checks for the fighter to function properly. By linking to maintenance records, aircraft computers, and so on, the pilot can be alerted if proper routine maintenance is not completed prior to the mission. The eyepiece can allow the pilot to view the history of aircraft maintenance with his diagram and overview.

In some embodiments, the eyepiece may provide user guidance in an environment related to military helicopters. For example, in training, in combat, in maintenance, in guitars, eyepieces can be used in this environment. These applications may be suitable for a variety of classes and levels of personnel.

As an example, the user may use the eyepiece for training in military helicopter operations in combat or in stressful situations. The user may be presented with an augmented reality situation simulating the battle situation on a particular aircraft. The user's responses and actions may be recorded and / or analyzed to provide additional information, ratings to the user, and to modify training exercises based on historical data.

During training and / or combat, the user's eyepiece may be synchronized to the aircraft for alerts regarding important statistics and maintenance of the aircraft. The user can view the program, safety procedures and emergency procedures for the passenger when the passenger boarded the aircraft. This procedure can show, among other things, the location of life saving equipment, in particular how to safely ride the aircraft, how to operate the door to enter or leave the aircraft, and the like. In embodiments, the eyepiece may present the user with the location and / or location of a threat that could jeopardize the helicopter during normal flight. For example, a user may be presented with the location of a low-flying threat such as a drones or other helicopter, and the location of a ground threat. In embodiments, the eyepiece may include a noise canceling earphone and a multi-user user interface to enable communication during flight. In the event of a helicopter crash, the user's eyepiece may transmit position and helicopter information to the commander and rescue team. In addition, the use of night vision of the eyepiece during low-flying missions can allow the user to turn off the high-power helicopter spotlight to detect or locate enemy forces without being detected.

In embodiments, the eyepiece may provide assistance in tracking maintenance of the aircraft and determining whether proper routine maintenance has been performed, as described in various instances herein. In addition, for other aircraft and vehicles mentioned herein, an augmented reality can be used to help maintain the aircraft and work on the aircraft.

In some embodiments, the eyepiece may provide user guidance in an environment related to a military drones aircraft or robot. For example, in reconnaissance, arrest, and rescue missions, in areas that put people at particular risk, and in other such environments, eyepieces can be used.

In embodiments, the eyepiece may provide the user with a video feed relating to the surroundings of the drones. A real time video of secondary information about the various areas of interest can be displayed. Gathering this information can provide soldiers with the number of enemy units in the area, the layout of the building, and other information. In addition, data may be collected and transmitted to the eyepiece from the drone and / or robot to collect information about the location of the person of interest to arrest or rescue. By way of illustration, a user outside a safe premises or bunker may use a drone and / or a robot to return a video or data feed of the location, number and activity of people in the safe premises for the purpose of arresting or rescue.

In embodiments, the use of an eyepiece for a drones and / or robots may allow a commander to collect battlefield data during a mission in order to make a plan change according to the collected data and to provide various instructions of the team . In addition, the eyepiece and associated controls allow the user to install the weapon in the drones and / or robots via the user interface in the eyepiece. The data feeds sent from the drone and / or robot can provide user information about which weapons to install and when to install.

In embodiments, the data collected from the drone and / or the robot may allow the user to reach a potentially dangerous situation. For example, this may allow the user to examine biological spills, bombs, narrow aisles, trenches, etc., to provide context and environmental data to the user without direct risk to the user.

In some embodiments, the eyepiece may provide user guidance in an environment related to the warship being sailed. For example, an eyepiece can be used in this environment, such as during training, when engaged in combat, when performing search and rescue missions, when performing disaster recovery, when performing maintenance, and so on. These applications may be suitable for a variety of classes and levels of personnel.

In embodiments, the eyepiece may be used in training to prepare the user for various techniques for performing his / her task at the vessel. Training can include simulations that test the ability of a user to navigate, control a vessel, and / or perform various tasks while in combat situations. The user's responses and actions may be recorded and / or analyzed to provide additional information, ratings to the user, and to modify training exercises based on historical data.

In embodiments, the eyepiece may provide the user with an augmented reality view of the potential vessel threats that are likely to occur in the near future, thereby allowing the user to view potential vessel threats. Such threats may be indicated by points, graphics or other means. If the eyepiece detects a particular threat, an indication of the preparation for engagement with the enemy may be transmitted to the user via the eyepiece. In addition, the user can view the map or video of the harbor to be anchored and be provided with the enemy location. In embodiments, the eyepiece may allow the user to synchronize with the vessel and / or weaponry to guide the user to use the equipment during battle. The user can be alerted by the eyepiece to see where the international and domestic boundaries are located.

In embodiments where navigation and structure are required, the eyepiece may provide tracking of algae and / or tagging recently discovered waters. In embodiments where algae are tracked, this may provide user information conveying a potential location or a changed location of a person of interest to be rescued. Similarly, an eyepiece can be used in an environment in which the user must examine the surrounding environment. For example, the user may be alerted to significant changes in hydraulic pressure and / or water movement that may signal the impending movement of the mantle and / or of an upcoming disaster. Alerts relating to movement of the mantle, earthquake and / or threat of a tsunami can be transmitted to the user via the eyepiece. Such an alert can be provided by an eyepiece that is synchronized with a device on the ship by tracking ocean movement, algae changes, changes in water pressure, descent or elevation of surrounding water, and the like.

In embodiments in which a warship is deployed for disaster recovery, an eyepiece can be used to detect contaminated areas, predicted speed and depth of movement of contamination, and where contamination is settled. In embodiments, the eyepiece may be useful for detecting parts per million (ppm) of contamination and variations thereof to determine a change in the amount of contamination.

In various embodiments, the eyepiece may provide the user with a program that checks whether the equipment on the vessel and the vessel is functioning properly. In addition, various operators of the ship may be alerted if proper routine maintenance is not completed prior to deployment. In embodiments, the user can also view the maintenance history of the ship with the status of the ship's critical functions.

In embodiments, the eyepiece may provide various types of guidance to the user in the environment of the submarine. For example, in training, in combat, in maintenance, in guitars, eyepieces can be used in this environment. These applications may be suitable for a variety of classes and levels of personnel.

As an example, a user may use an eyepiece for training in combat or in a stressful situation for submarine manipulation. The user may be presented with an augmented reality situation or otherwise simulating the battle situation in a particular submarine. The training program can be based on the user's class so that the user's class determines the type of situation in which it is presented. The user's responses and actions may be recorded and / or analyzed to provide additional information, ratings to the user, and to modify training exercises based on historical data. In embodiments, the eyepiece can also train the user to maintain the submarine, use the submarine, and appropriate safety procedures.

In a combat environment, an eyepiece can be used to provide information to the user about the depth of the user, the enemy and the location of the object, the ally and / or the enemy on the surface. In embodiments, this information may be conveyed to the user via a visual representation, audio, or the like. In various embodiments, the eyepiece may be synchronized with and / or utilized by the apparatus and equipment of the submarine to collect data from a GPS receiver, etc., to collect various information, such as the location of other objects, submarines, The eyepiece can display instructions to the soldier regarding safety procedures, mission details and the presence of enemy forces in the area. In embodiments, the device may communicate with or be in synchronism with the vessel and / or weaponry equipment to guide a soldier in using such equipment and to provide a display associated with the particular equipment. Such displays may include visual and audio data related to the equipment. As a further example, information that may not be displayed (such as the location of enemy forces outside the field of view, the location of the enemy, the country of origin, and the like) by enhancing the visual appearance and / or audio of the user and using potential threats, International boundaries, various threats, etc.) can be used for the periscope.

The eyepiece can also be used to maintain the submarine. This can, for example, provide the user with a pre-journey check that the ship is functioning properly and can indicate whether proper routine maintenance operations have been performed prior to the mission or not. In addition, the user can be provided with detailed history to review the maintenance performed and the like. In embodiments, the eyepiece can also help maintain the submarine by providing an augmented reality or other program that instructs the user when performing such maintenance.

In embodiments, the eyepiece may provide various types of guidance to the user in the environment of the anchorage vessel. For example, in training, in combat, in maintenance, in guitars, eyepieces can be used in this environment. Such applications may be suitable for agents of various grades and levels.

As an example, the user may use the eyepiece for training for the vessel during battle, when attacked, or in stressful situations. The user may be presented with augmented reality situations or otherwise simulating combat situations that may be seen at a particular port and on such a vessel. The training program can show data about the various ports around the world and surrounding terrain data, the number of allied or enemy ships that may be in the port at any given time, and show local refueling stations. The training program can be based on the user's class so that the user's class determines the type of situation in which it is presented. The user's responses and actions may be recorded and / or analyzed to provide additional information, ratings to the user, and to modify training exercises based on historical data. In embodiments, the eyepiece may also be used to maintain and perform mechanical maintenance on the vessel, and to train the user in the use of the vessel and in appropriate safety procedures to be used on the vessel.

In a combat environment, an eyepiece may be used to provide information to a user about the port to which the user is moored or moored. The user may be provided with information on the location of the enemy and / or friendly vessels in the port or other visual representation. In embodiments, a user may obtain alerts of an approaching aircraft and an enemy ship, and the user may use the ship and / or weapon to guide the user &apos; s use of the equipment while providing information and / Can be synchronized to the equipment. Such data may include the amount and efficacy of a particular ammunition. The eyepiece can display instructions to the soldier regarding safety procedures, mission details and the presence of enemy forces in the area. Such displays may include visual and / or audio information.

The eyepiece can also be used to maintain the vessel. For example, it can provide pre-departure checks to the user to ensure that the ship is functioning properly and to indicate whether proper day-to-day maintenance operations have been performed or not completed prior to the mission. In addition, the user can be provided with detailed history to review the maintenance performed and the like. In embodiments, the eyepiece may also help maintain the vessel by providing an augmented reality or other program that instructs the user when performing such maintenance.

In other embodiments, the user may use an eyepiece or other device to obtain biometric information of the persons entering the port. This information can provide the user &apos; s identity and allow the user to know whether the person is a threat or an interested person. In other embodiments, the user may scan an object or container imported into the harbor for potential threats in shipping such as cargo. The user can detect a dangerous substance based on the density or various other information collected by the eyepiece or sensors associated with the device. The eyepiece can scan the document to record information or to determine how the document may have been falsified or altered. This can help the user inspect their personal credentials, which may be associated with a particular cargo to inform the user of potential threats or problems that may be related to cargo, such as incorrect cargo lists, counterfeit documents, It can be used to check documents that are

In embodiments, the eyepiece may provide various types of guidance to the user when using a tank or other ground vehicle. For example, eyepieces can be used in this environment when in training, when engaged in combat, for surveillance, for group transport, for maintenance, and so on. Such applications may be suitable for agents of various grades and levels.

By way of example, the user may use the eyepiece for training in tanks or other ground vehicles when in combat, in an attack, in a stressful situation, or otherwise. The user may be presented with an augmented reality situation or otherwise simulating the battle situation that may be seen in the tank and / or while operating the tank. The training program can test the user for specific equipment and weapon use. The training program can be based on the user's class so that the user's class determines the type of situation in which it is presented. The user's responses and actions may be recorded and / or analyzed to provide additional information, ratings to the user, and to modify training exercises based on historical data. In embodiments, the eyepiece can also train the user in maintenance of the tank, in using the tank, and in appropriate safety procedures to use when in a tank or ground vehicle.

In a combat environment, an eyepiece can be used to provide the user with information and / or visual representation of the location of enemy and friendly vehicles in the foreground. In embodiments, a user may acquire alerts of an approaching aircraft and an enemy vehicle, and the user may interact with a tank and / or weapon to guide the user to use the equipment while providing information and / Can be synchronized to the equipment. Such data may include the amount and efficacy of a particular ammunition. The eyepiece can display to the soldier instructions about safety procedures, mission details and the presence of enemy and friendly forces in the area. Such displays may include visual and audio information. In embodiments, a user may stream a 360 degree view from the surrounding environment outside the tank by using an eyepiece to synchronize with the camera or other device with this view. Video / audio feeds may be provided to users within or outside the tank / vehicle as needed. This can allow the user to monitor the vehicle and stopping threats. The eyepiece can be used to monitor vehicle statistics, such as armor breach, engine condition, etc., and to communicate with the vehicle and with a variety of vehicles, aircraft, vessels, and devices as described herein or otherwise apparent to those skilled in the art . The eyepiece may provide additional GPS for navigation, use of black silicon or other techniques such as those described herein to detect the enemy and to move to an environment such as night vision and non-optimal vision .

In addition, an eyepiece can be used in a tank / ground vehicle environment for surveillance. In embodiments, a user may be able to synchronize to a camera or other device to obtain a 360 degree field of view for acquiring information. Night Vision and / or SWI as described herein may be used for additional information collection when needed. The user can use the eyepiece to probe the environment to acquire a heat signature to detect potential threats and look at soil density to detect road bombs, vehicle markings, various threats, and the like.

In embodiments, an eyepiece may be used to facilitate group transport by tank or other ground vehicle. For example, a user may be provided with a visual, interactive or other checklist of goods and personnel to transport. The user can track and update the inventory of the items to be tracked, such as being in transit. The user can view a map of the surrounding area, scan documents and documents for identification of the personnel, identify and track the items associated with the person in transit, and provide information about the journey / mission of the person in transit You can see and do other things.

The eyepiece can also be used for maintenance of the vehicle. For example, it may provide a pre-departure check to the user to determine if the tank or other vehicle functions properly and to indicate that proper routine maintenance operations have been performed or not completed prior to the mission. In addition, the user can be provided with detailed history to review the maintenance performed and the like. In embodiments, the eyepiece can also help maintain the vehicle by providing an augmented reality or other program that instructs the user when performing such maintenance.

In embodiments, the eyepiece may provide various types of guidance to the user when in an urban or suburban environment. For example, in training, in combat, in surveillance, in guitars, eyepieces can be used in this environment. Such applications may be suitable for agents of various grades and levels.

As an example, a user may use the eyepiece during a battle, in an attacked or stressed situation, when interacting with local people, and in other urban or suburban settings for training. The user may be presented with an augmented reality situation or the like that simulates battle situations that may be seen in such an environment. The training program can test the user for specific equipment and weapon use. The training program can be based on the user's class so that the user's class determines the type of situation in which it is presented. The user's responses and actions may be recorded and / or analyzed to provide additional information, ratings to the user, and to modify training exercises based on historical data. In embodiments, the user can view alternative scenarios of urban and suburban environments, including real building and building layouts and areas of potential battle. The user can be provided with climate and weather information prior to entering the area and can be informed of the number of people in the area at any given time or at the time of day to prepare for possible attacks or other engagements. In addition, the user can be provided with the location of a person at, and around, a building in a given area so that the user is ready before entering the environment.

In urban and suburban environments, an eyepiece or other device may allow the user to explore the area as well. The user can collect the face, iris, voice, and fingerprint and long-range data of the person of interest. The user can scan this data at a distance of 0 to 5 meters, at a greater distance, or right next to the POI, without user detection. In embodiments, the user may be required to view the smoke and / or the destroyed environment, to memorize and record the presence of the vehicle in that area, to record the environmental image for future use in battle plans, You can use the eyepiece for guitar to note the population density of the area, the layout of various buildings and alleys at various times. In addition, the user can collect and receive facts about a particular Aboriginal population that a soldier will be contacting.

The user can also use an eyepiece or other device in an urban / suburban environment while in battle. The device may allow a user to use a geographic location with a laser range finder to locate and kill enemy targets. In embodiments, this may provide a bird's-eye view of the surrounding environment and the building. This can display the enemy forces in the area surrounding the user and identify the locations of people such as enemy or friend or people of the user's team. The user may use an eyepiece or other device to view / hear instructions from the commander through the eyepiece in order to contact with the home, where the indication may appear after viewing or hearing data from the user's environment. In addition, the eyepiece can also allow the user to command others on his team. In embodiments, a user may perform biometric data collection on people in the vicinity, record such information, and / or retrieve information about them for use during combat. The user can link with other military devices to monitor and use various devices carried by the soldier. In embodiments, the eyepiece can alert the user to the edge of the approaching building when it is on the roof, alerting when approaching a ground change or protrusion. The user can view the map overlay of the environment and members of his team and detect nearby signals to alert others to nearby enemies to receive an alert. In various embodiments, the user can use the eyepiece to communicate with other team members to implement the plan. In addition, the user can use the eyepiece to detect enemy forces in dark tunnels and other areas where enemy forces may be located.

Eyepieces can also be used in desert environments. In addition to the general and / or applicable applications mentioned herein in connection with training, combat, survival, surveillance purposes, etc., eyepieces can also be used in a variety of usage scenarios that may be encountered in an environment such as a desert environment. As an example, when entering a battle or training, the user may use the eyepiece to correct the sight lost by sandstorms during combat, surveillance and training. In addition, the eyepiece can simulate the poor visibility of sandstorms and other desert hazards for users in training mode. During combat, the eyepiece can help the user see or detect enemy forces in the presence of sandstorms through various means as previously described. In addition, the user can be alerted to and / or be aware of the difference in sand dust generated by the vehicle and sand caused by the wind, in order to receive an alert for potential enemy access.

In various embodiments, the user may use an eyepiece to detect ground and environmental hazards. For example, a user can use an eyepiece to detect the edges of a sand dune, a sand pit, and the like. The user can also use the eyepiece to detect sand densities and detect various hazards such as ground holes, cliffs, buried devices (such as mines and bombs), and the like. The user may be presented with a map of the desert to see the location of these dangers. In embodiments, the user monitors his vital signs and provides a means to alert the user when he or she is at risk of hitting extreme environmental conditions (daytime heat, nighttime cold, fluctuating temperature, dehydration, etc.) Can be provided. Such alerting and monitoring may be provided graphically and / or via audio information to the user interface displayed on the eyepiece.

In embodiments, the user may be presented with a map of the desert to view the location of his team, and the user may be presented with an audio alert to detect nearby signals or from an audio alert from the earpiece, You can use an eyepiece to acquire alerts for an enemy in a different way. In such embodiments, the user may have an advantage over his enemy because he can have the ability to identify the location of his team and enemy forces in sand storms, buildings, vehicles, and the like. The user can see a map of his location where he can show the area he recently moved to as one color and show the new area as a different color. In this way or through other means, the device may allow the user to be not lost and / or continue to move in the correct direction. In embodiments, the user may be provided with a weather satellite overlay to warn the user of sand storms and dangerous weather.

Eyepieces can also be used in the wilderness environment. In addition to the general and / or applicable applications mentioned herein with respect to training, combat, survival, surveillance purposes, etc., eyepieces can also be used in a variety of usage scenarios that may be encountered in an environment such as a wilderness environment.

As an example, a user may use an eyepiece in training to prepare for being in the wilderness. For example, a user can use an eyepiece to simulate various degrees of a wilderness environment. In embodiments, the user experience may experience very coarse and heavy trees / bushes where dangerous animals are in and around, and in other training environments, the user may be in a situation where there are fewer places to hide the body from the enemy forces.

During combat, the user can use the eyepiece for various purposes. The user can use an eyepiece to detect newly broken twigs and branches to detect the presence of a recent enemy. In addition, the user can use the eyepiece to detect dangerous cliffs, caves, changes in terrain, recently moved / touched dust, and the like. By detecting the presence of recently touched dust that can be detected, for example, by having a density or a heat signal different from the surrounding dust / leaves, or by detecting other means, the user can trap, bomb or other dangerous device Can be alerted to. In the various environments described herein, a user communicates with his team through a user interface or other means such that communication may not be detected by the enemy forces in a quiet and / or closed environment, an open environment vulnerable to eco, You can use an eyepiece to do this. Also, in various embodiments, the user may utilize a night vision as described herein to detect the presence of an enemy. The user can also view an overlay of a trail map and / or a trail map in the eyepiece so that the user can view the path before encountering a possibly dangerous terrain and / or situation where the enemy may be present. In the various environments described herein, the eyepiece can also amplify the user's hearing for the detection of a potential enemy.

In embodiments, the user may utilize the eyepiece in a wilderness environment in a navigational and structural use scenario. For example, a user can use an eyepiece to track a person's trail and to detect the soil / leaves to determine if the soil / leaf has been picked up to find the buried body. The user can view a map of the tagged area to show the area already processed by the Air Force and / or other team member searches to send the user from the already searched area to the uncovered area. In addition, the user can use the eyepiece for night vision for detecting people and / or animals in trees, bushes, forests, and the like. In addition, by using an eyepiece to detect the presence of a newly broken twig, the user can detect the presence or recent presence of a person of interest when monitoring and / or rescue mission. In embodiments, the user can also view an overlay of a trail map and / or a trail map in the eyepiece so that the user can view the path before encountering a potentially hazardous terrain and / or situation.

In still other embodiments, the user may utilize living on the ground and using an eyepiece in the wilderness for survival type situations. As an example, a user may use an eyepiece to track animal presence and movement to hunt for food. In addition, the user can use the eyepiece for detecting soil moisture and for detecting the presence and location of the water supply. In embodiments, the eyepiece may also amplify the user &apos; s hearing to detect potential prey.

Eyepieces can also be used in arctic environments. In addition to the general and / or applicable applications mentioned herein with respect to training, combat, survival, surveillance purposes, etc., eyepieces may also be used in a variety of usage scenarios that may be encountered in an environment such as an Arctic environment. For example, during training, the eyepiece may simulate visual and audio whiteout conditions that a user may encounter in an Arctic environment so that the user can adapt to working under such stress. In addition, the eyepiece can provide the user with a program that simulates various conditions and scenarios due to extreme cold that the user may experience, and the program can track and display data related to the user's predicted heat loss. In addition, the program can be adapted to simulate these conditions that the user will experience with such heat loss. In embodiments, the program may simulate a user not being able to properly control his limbs (which may appear to be a loss of weapon accuracy). In other embodiments, the user may be provided with lifesaving information and instructions on things such as oysters for warmth, and various survival strategies for polar conditions. In yet other embodiments, the eyepiece may be synchronized to the vehicle so that the vehicle responds as if the vehicle is operating in a particular environment (e.g., in an arctic condition and in snow and ice). Accordingly, the vehicle can respond to the user by itself, and the eyepiece can also simulate visual and audio as if the user were in this environment.

In embodiments, the user may use the eyepiece during battle. Soldiers can use the eyepiece to see through the whiteout conditions. The user can pull up overlay maps and / or audio to provide information on building drainage, ground hazards, etc. to allow soldiers to safely move around the environment. The eyepiece can be used to detect an increase or decrease in eye density (e.g., to indicate possible drainage, holes or other hazards, an object buried in the eye, etc.) to inform the user when the mass of the land beneath the eye has changed The user can be alerted. Furthermore, under difficult conditions, the user may be provided with the position of his team member and enemy forces, whether or not the eye interferes with his field of view. The eyepiece can also provide a thermal signal to the user to display the animal and person in the Arctic environment. In embodiments, the user interface in the eyepiece can show the vital signs of a soldier and provide an alert when the soldier is in danger due to extreme environmental conditions around him. In addition, the eyepiece can assist the user in operating the vehicle in snowy conditions by providing an alert from the vehicle to the user regarding transmission slipping, wheel spinning, and the like.

The eyepiece can also be used in a jungle environment. In addition to the general and / or applicable applications mentioned herein with respect to training, combat, survival, surveillance purposes, etc., the eyepiece may also be used in various usage scenarios that may be encountered in an environment such as a jungle environment. For example, an eyepiece can be used to provide users with information about which plants can be eaten in training, which are toxic, and which insects and animals can pose a risk to the user. In embodiments, the eyepiece can simulate various sounds and environments the user may encounter in the jungle, so that the environment is not disturbed during battle. In addition, when in combat or in a real jungle environment, the user is provided with a graphical overlay or other map to help the user track the surrounding area and / or track where the user is and where to go . This may alert the user to allied and enemy forces in the area and may detect movement to alert the user to nearby animals and / or insects. These alerts can help the user to survive by avoiding attacks and finding food. In other embodiments, the user may be able to determine whether the user is safe to eat, which is toxic, etc., for example, to allow the user to compare the creature and / In the form of a graphic overlay, the augmented reality data can be provided. By having information that a particular creature is not a threat to the user, the user may not need to deploy the weapon while in stealth mode or silent mode.

Eyepieces can also be used in connection with special unit missions. In addition to the general and / or applicable applications mentioned herein with respect to training, combat, survival, surveillance purposes, etc., eyepieces may also be used in a variety of usage scenarios that may be encountered in connection with special unit missions. In embodiments, the eyepiece may be particularly useful in stealth missions. For example, the user can communicate with his team completely silently through the user interface, which each member can see in his eyepiece. User sharing information may be navigated through the user interface by eye movements and / or controller devices. Other users may also view the data as the user provides instructions and / or searches the user interface and specific data regarding the information to be communicated. In embodiments, various users may insert questions to be answered by the commander via the user interface. In embodiments, the user may speak or activate other audio that all users can hear through his or her eyepiece or other device. This can allow users at various locations in the battlefield to communicate battle plans, instructions, questions, share information, and allow users to do so without being detected.

In embodiments, the eyepiece may also be used for military fire-fighting. By way of example only, the user may use the eyepiece to perform a simulation of the fire scenario. The device can utilize augmented reality to simulate fire and structural damage to the building over time, and reproduce realistic scenarios in different ways over time. As discussed herein, a training program may monitor user progress and / or change scenarios and training modules based on user behavior. In embodiments, an eyepiece may be used in actual fire fighting. The eyepiece allows the user to view the smoke through various means such as those described herein. The user can view, download, or otherwise access the layout of a fired building, vessel, aircraft, vehicle or structure. In embodiments, the user will have an overview map or other map that displays where each team member is. The eyepiece can monitor user wear or other devices during fire fighting. The user can see his oxygen supply level in his eyepiece and be alerted when he should be out for additional oxygen. The eyepiece can send a notification from the user's device to the command outside the structure to place new personnel into or out of the fire and to provide status updates and alerts of possible firefighter hazards. The user can display his vital signs to determine whether the user is overheating, losing too much oxygen, or the like. In embodiments, an eyepiece may be used to analyze whether a beam is cracking based on beam density, thermal signals, etc. and to inform the user of the structural integrity of the building or other environment. The eyepiece can provide an automatic alarm when structural integrity is compromised.

In embodiments, the eyepiece may also be used for maintenance. For example, the eyepiece may provide the user with a pre-mission and / or pre-use checklist to ensure that the article to be used functions properly. The eyepiece can alert the operator if proper maintenance is not logged in the article database. The eyepiece may provide a virtual maintenance and / or performance history so that the user can determine the safety of the item or the safety of the necessary measures to be taken for safety and / or performance. In embodiments, an eyepiece may be used to perform an augmented reality program or the like for training a user in weapon management and maintenance and for training a mechanic of new and / or advanced equipment. In embodiments, an eyepiece may be used in the maintenance and / or repair of various articles such as weapons, vehicles, aircraft, devices, and the like. The user can use the eyepiece to view the overlay of the visual and / or audio manual of the article that directs the user to maintenance without the need for a mini manual. In embodiments, video, still images, 3D and / or 2D images, animated images, audio, etc. may be used for such maintenance. In embodiments, a user may view overlays and / or video of various images of an article and thus may be able to view the video, how to remove it, in what order, and what parts to add, replace, repair, Show. In embodiments, this maintenance program may be an augmented reality program or the like. In embodiments, the user may use the eyepiece to monitor the functions and / or critical statistics of the machine or device to assist in repair and / or to connect to the machine or device to provide maintenance information. In embodiments, the user can use the eyepiece to suggest the next course of action during maintenance, and the eyepiece can be used to determine if the action will harm the machine, help to repair the machine, Information on how to function and / or function later can be transmitted to the user. In embodiments, the eyepiece may be used for maintenance of any article, machine, vehicle, device, aircraft, etc., as referred to herein or that may be applied or stuck in a different manner in a military environment.

Eyepieces can also be used in environments where the user is somewhat unfamiliar with the language used. By way of example, a soldier may use an eyepiece and / or device to access near real-time translations of what is being said around him. Through the earpiece of the device, the soldier can hear his native translation of what someone is talking to him. In addition, a soldier can record and translate comments made by prisoners and / or other detainees. In embodiments, the soldier may have a user interface that allows translation of the phrase or via ear piece, text image via the user's eyepiece, and other translation to the user. In embodiments, the eyepiece may be used by the linguist to provide supplemental information to a skilled linguist about what is being spoken by people in or near the dialect spoken in a particular area. In embodiments, the linguist may use the eyepiece to record a language sample for further comparison and / or research. Other experts can use the eyepieces to use speech analysis to determine if the speaker is experiencing anger, shame, lies, etc., by monitoring speech patterns, tone, stuttering, and the like. This can provide the speaker's intention to the native speaker even when the speaker and the speaker are speaking different languages.

In embodiments, the eyepiece may allow a user to decipher body language and / or facial expressions or other biometric data from another person. For example, a user can analyze a person's pupil dilation, eye flicker rate, voice inflection, and body movements to determine whether a person is lying, hostile, stressed, or threatened You can use the device to do this. In embodiments, the eyepiece may also collect data, such as data of facial expressions, to detect the user and alert the user if the speaker is lying, untrusted, or hostile or otherwise. In embodiments, the eyepiece may provide an alert to a user when interacting with a resident or other person to warn of a potential non-combat or ordinary citizen or a likely threat to disguise as another person. The user alert can be audio and / or visual and can be displayed on the user interface in the user's eyepiece, overlaid on the user's view, and / or associated with an investigated person in the user's line of sight. Such monitoring may not be detected, as described herein, because the user may use the eyepiece and / or device to collect data from afar or may be performed in close proximity in disguise or individual manner Since it can be performed in a state in which the person in question knows it and / or under his / her consent.

Eyepieces may also be used to handle bombs and other hazardous environments. By way of example, the eyepiece may provide an alert to the user of a change in soil density in the vicinity of the roadside, which may alert the user and / or team to the buried bomb. In embodiments, similar methods may be used in various environments, such as testing the density of the eye to determine if a bomb or other explosive can be found in an arctic environment or the like. In embodiments, the eyepiece may provide density calculations to determine whether the baggage and the transported article have an unexpected density or an out-of-range density for the article being transported. In embodiments, the eyepiece may provide a similar density calculation and provide an alert if the density is found to be of a density that falls within the expected density for a blower, other weapon, and the like. Those skilled in the art will appreciate that explosive detection can be used through chemical sensors and / or means known in the art and can be used by the eyepiece in various embodiments. In embodiments, the eyepiece may be useful in bomb processing. The user may be provided with an augmented reality or other audio and / or visual overlay to receive instructions on how to disassemble a particular type of bomb present. Similar to the maintenance program described above, the user may be provided with instructions to dismantle the bomb. In embodiments, if the bomb type is not known, the user interface may provide the user with instructions for secure processing and possible follow-up actions that can be taken. In embodiments, the user may be alerted to potential bombs nearby, and may be provided with instructions on how to safely escape from the bomb area, how to safely exit the vehicle with the bomb, how close the user can safely approach the bomb, Instructions for safe handling of situations such as how to dismantle the bomb through appropriate instructions to the situation and the user's proficiency. In embodiments, the eyepiece can also provide the user with training in such a hazardous environment.

In embodiments, the eyepiece can detect a variety of other hazards such as biological effluents, chemical effluents, and the like, and can alert the user to dangerous situations. In embodiments, the user may also be provided with various instructions regarding diffusing the situation in and / or under these conditions, arriving in a safe place, and taking care of the safety of others. Although the situation for the bomb is described, it should be seen that the eyepiece can likewise be used to protect it from and against it in various harmful and / or dangerous situations and when encountering such risks and hazards, and / or to provide instructions .

In various embodiments, an eyepiece may be used in a typical body training and training environment. The eyepiece can provide information such as the distance traveled during running, hiking, walking, etc. to the user. The eyepiece can provide the user with information such as the number of exercises performed, consumed calories, and the like. In embodiments, the eyepiece may provide the user with a virtual indication associated with correctly performing a particular motion, and may provide additional motion to the user as and when desired. In addition, the eyepiece can provide a user interface or the like, in which case a body reference point is shown to meet the requirements of the soldier for his particular program. In addition, the eyepiece can provide data relating to the amount and type of exercise that the user needs to perform to meet these requirements. These requirements may be tailored to special unit qualifications, basic training, and so on. In embodiments, a user may train a virtual obstacle during an inspection to prevent the user from installing an actual obstacle, obstruction, or the like.

Although various specific embodiments and use scenarios are described herein, this description is not intended to be limiting. In addition, it should be appreciated that an eyepiece can be used in a variety of situations that are apparent to those skilled in the art. It should also be understood that the applicable application of the eyepiece, as has been discussed for a particular environment, can be applied to a variety of different environments, even if not specifically mentioned.

In embodiments, a user may access a library of information stored on a secure digital (SD) card, mini SD card, other memory, remotely loaded via a tactical network, or otherwise stored and / You can manipulate it in a way. The library may be part of the user's equipment and / or be remotely accessible. The user's equipment may include a DVR or other means for storing the information collected by the user, and the recorded data and / or feed may be transferred to another location as desired. In embodiments, the library may include images of local threats, information and / or images of various people listed as threats, and so on. The library of threats may be stored on an onboard mini SD card or other means. In embodiments, the library may be remotely loaded via a tactical network. In addition, in embodiments, the library of information may include programs and other information useful for maintenance of military vehicles, or the data may be of a variety or of any type of information. In various embodiments, a library of information may be used in the apparatus such that the data is transmitted and / or transmitted to and from the storage medium and the user's device. As an example, data may be transmitted to the user's eyepiece and from a stored library so that the user can view images of a local person of interest. In embodiments, data may be transmitted to and from a library contained in the equipment of a soldier, or may be located remotely, and data may be transmitted to and from the various devices, as described herein . In addition, data may be transmitted between various devices as described herein and various libraries as described above.

In embodiments, military simulation and training may be used. By way of example, game scenarios commonly used for entertainment may be adapted and used for battlefield simulation and training. Various devices such as an eyepiece described herein can be used for this purpose. In these simulations, short-range communications can be used to alert personnel, to present risks, to change strategies and scenarios, and to a variety of other communications. If it is necessary to provide instructions and / or information, this information may be posted to share information. Various scenarios, training modules, etc. may be implemented on the user's equipment. Rather than limiting the use of such training, for example, the user's eyepiece can display the augmented reality battle environment. In embodiments, the user may act and react in this environment as if it were actually in combat. The user can advance or retreat according to his performance. In various embodiments, the user's behavior may be recorded for feedback to be provided based on his performance. In embodiments, the user may be provided feedback regardless of whether his performance has been recorded. In embodiments, the information posted as described above may be password-protected or biometric-protected and / or encrypted and may be immediately available or available after a certain period of time. This information, stored in electronic form, can be updated immediately for any change orders and updates that may be requested.

If it is necessary to provide instructions and / or information, local communication or other means may be used for training and maintenance in the training environment to share and publish information. For example, information may be posted in classrooms, laboratories, maintenance facilities, repair areas, etc., or as needed for such training and instruction. A user's device, such as an eyepiece, described herein may enable such transmission and reception of information. Information can be shared through the augmented reality, where the user enters a particular area and is notified of this information once there. Similarly, as described herein, local communications may be used in maintenance. For example, it can be posted exactly where information is needed (e.g., maintenance facility, repair area, etc., associated with the article to be repaired). More specifically, and not to limit the present disclosure, repair instructions may be posted under the hood of a military vehicle and may be seen by use of a soldier's eyepiece. Similarly, various instruction and training information may be shared with various users in any given training situation, such as training for combat and / or training for military device maintenance. In embodiments, the information posted as described above may be password-protected or biometric-protected and / or encrypted and may be immediately available or available after a certain period of time. This information, stored in electronic form, can be updated immediately for any change orders and updates that may be requested.

In embodiments, the application program applied to the present disclosure may be for face recognition or sparse facial recognition. Such rare face recognition may use one or more face features to preclude the possibility of identifying the person of interest. Rare face recognition can have automatic obstacle masking and error and angle correction. In the embodiments, eyepieces, flashlights, and devices such as those described herein may enable rare face recognition, rather than limiting the present disclosure. This can act like a human eye and quickly eliminate spurious matching or entire profiles using sparse matching at once for all image vectors. This makes it almost impossible for false positives. In addition, it can use multiple images simultaneously to expand the vector space and increase accuracy. It can operate on multiple databases or multiple target images based on availability or operational requirements. In embodiments, the device can identify one or more specific clean features, either manually or automatically, with minimal accuracy loss. As an example, the accuracy can have a wide range and can be at least 87.3% for nose, 93.7% for eye, and 98.3% for mouth and jaw. In addition, angle correction may be used in face reconstruction, and in embodiments, angle correction of up to 45 degrees in face reconstruction may be achieved. This can be further enhanced with 3D video mapping technology. In addition, unascertained area masking and substitution can be used. In embodiments, unspecified area masking and replacement of 97.5% and 93.5%, respectively, can be achieved by sunglasses and scarves. In embodiments, the ideal input image may be 640 x 480. Target images can be reliably matched to less than 10% of the input resolution due to long distances or air obstructions. In addition, as noted above, the specific range may be greater or less than in various embodiments.

In various embodiments, the devices and / or networks described herein may be applied for identification and / or tracking of allied and / or allied forces. In embodiments, face recognition may be used to unambiguously identify friendly and / or friendly forces. In addition, real-time network traces and / or allied and enemy real-time network traces can allow a user to know where his allied and / or friendly members are. In embodiments, there may be visual separation distances between the enemy and the enemy forces identified by various markers and / or means. In addition, the user can identify the enemy's geographic location and share the enemy's location in real time. In addition, allied positions can also be shared in real time. The devices used for such applications may be biocompatible glasses, eyepieces, other devices as described herein, and those known to those skilled in the art.

In embodiments, the devices and / or networks described herein may be applied to medical treatment in diagnosis. By way of example, such a device may allow medical personnel to perform remote diagnostics. In addition, for example, when a combatant arrives at the scene or at a remote location, the medic will immediately call the medical history, allergy, blood type and other time sensitive medical data of the soldier, Devices can be used. In an embodiment, such data may be called through military face recognition, iris recognition, etc., which may be accomplished through an eyepiece or other device as described herein.

In embodiments, users may share various data through various networks and devices such as those described herein. By way of example, a 256-bit AES encrypted video wireless transceiver may share bi-directional video between units and / or with a vehicle's computer. In addition, biometric data collection, registration, identification and verification of persons of potential interest, biometric data of a person of interest, etc., can be remotely shared locally and / or over a wireless network. In addition, this identification and verification of a person of potential interest may be achieved or assisted by data that is remotely shared locally and / or over a wireless network. Biometric systems and devices such as those described herein may also share data over a network. In the embodiments, data may be shared among various devices, persons, vehicles, locations, units, etc., from and / or between them. In embodiments, there may be inter-unit and intra-unit communication and data sharing. Cone type cable, removable SD and / or microSD memory card, Humvee, PSDS2, unshielded twisted-pair cable with 256-bit encryption, Such as, but not limited to, aircraft, WBOTM or other network repeaters, combat radios, mesh network computers, various devices described herein, biophone 3G / 4G network computers, digital documents, tactical operations headquarters a tactical operating center, a command post, a DCSG-A, a BAT server, a group of individuals and / or individuals, any of the eyepieces and / or devices described herein, and / Data may be shared among, among, and / or between them.

In embodiments, a device or other device as described herein may include a viewing pane in which the squad leader and / or team leader is inverted to project the image on any surface for viewing the combat team . A transparent monitoring window or other monitoring window may be rotated 180 degrees or another positive angle in projection mode to share data with the team and / or others. In embodiments, devices including monocular and binocular NVGs (including but not limited to) can interface with all or nearly all of the tactical radios in use and allow the user to view live video, S / A, biometric data, and other data In real time or in other ways. The devices discussed above, such as binocular and monocular devices, may be self-contained VIS, NIR and / or SWIR binocular or monocular devices, and may be small, encrypted wireless-enabled computers Color night vision / night vision and / or digital display. Various data can be shared in real time or near real time via combat radio, mesh network and long-range tactical network. In addition, the data can be composed of digital documents. The data of a person of interest (POI), whether or not such POI breaks are registered, can be configured as a digital document. In embodiments, shared data may be compared, manipulated, and the like done. Although specific devices are mentioned, any device described herein may share information, as described herein and / or as will be appreciated by those skilled in the art.

In embodiments, biometric data, video, and various other types of data may be collected via various devices, methods, and means. For example, fingerprints and other data may be collected from combat, terrorism, and / or weapons and other objects at the scene of a crime. Such collection may be captured by video or other means. A pocket biocam, a flash light as described herein, which is embedded in a still video camera, various other devices or other devices described herein collect video, record, monitor, and collect biometric data And can identify. In embodiments, various devices record and collect data and biometric data relating to faces, fingerprints, potential fingerprints, potentials, irises, voices, pockets, scars, tattoos, , Identify and verify. The data can be geo-located and date / time stamped. The device is capable of capturing EFTS / EBTS compatible critical images to be matched and filed by any biometric matching software. In addition, potential matches to video scans and embedded or remote iris and face databases can be performed. In embodiments, a variety of biometric data may be captured and / or compared to a database and / or composed of digital documents. In embodiments, the imaging and detection system can provide biometric feature scans and enable face tracking and iris recognition of multiple subjects. The subject may be moving at high speed into or out of the crowd and may be immediately identified, and local and / or remote storage and / or analysis may be performed on such images and / or data. In embodiments, the device may perform multi-mode biometrics. For example, the device may collect and identify face and iris, iris and potential fingerprints, various other combinations of biometric data, and the like. In addition, the device can record video, posture, fingerprints, potential fingerprints, long passages, latent lines, etc. and other break marks and / or movements. In various embodiments, biometric data can be filed using the most important images and manual input, which allows for partial data capture. With locally assigned or network assigned GUIDs, the data can be automatically geo-located, time / date stamped, and filed in digital papers. In embodiments, the device may record the entire live-scanned four-finger press and roll fingerprints, press and roll fingerprints, long passages, fingertips, and fingerprints. In embodiments, the operator may monitor the Aboriginal Army and collect POIs and verify with an on-board or remote database. In embodiments, the device may have access to a web portal and a biometric awareness watchlist database and / or may include existing biometric dictionary validation software for POI acquisition. In embodiments, biometric characteristics may be matched and filed by any approved biometric matching software to transmit and receive secure, deciphering voice, video, and data. The device may integrate and / or otherwise analyze the biometric content. In embodiments, biometric data may be collected in biometric standard images and data formats that can be cross-referenced for near-real-time or real-time data communication with US Department of Defense bioprocessing or other databases. In embodiments, the device may utilize algorithms for detection, analysis, and the like in conjunction with fingerprints and long passages, iris, and facial images. In embodiments, the device may simultaneously illuminate the iris or potential fingerprint for a comprehensive solution. In embodiments, the device can use high-speed video to capture critical images in unstable situations and facilitate high-speed distribution of context awareness with intuitive tactical display. Real-time situational awareness can be provided to the headquarters and / or tactical operations center. In embodiments, the device may allow all soldiers to be a sensor, to be observed and reported. The collected data can be tagged with the date, time and geographic location of the collection. In addition, biomedical images can be NIST / ISO compatible, including ITL 1 -2007. In addition, in embodiments, the laser range finder can aid in biopsy and goal setting. The library of threats can be stored on the onboard mini SD card or can be remotely loaded via the tactical network. In embodiments, the device may wirelessly transmit encrypted data between devices by a band transceiver and / or an ultra-wideband transceiver. The device can securely perform onboard matching of the potential POF to the embedded database or over a full network. In addition, the device can use high-speed video to capture critical images under all environmental conditions. Biometric profiles can be uploaded, downloaded and retrieved within seconds. In embodiments, the user can use the device to identify the geographic location of the POI at a safe distance from the visual biometric feature and to clearly identify the POI with a sparse recognition algorithm that is stable to the face, iris, and the like. In embodiments, the user can merge and print visual biometric features on one comprehensive display with enhanced target highlight, and view matching and alerting without alerting the POI. Such displays may be in a variety of devices such as eyepieces, handheld devices, and the like.

In embodiments, a person who is an indigenous person enters through a controlled checkpoint and / or a vehicle stop, and an operator can collect, register, identify and verify POIs from the watch list using low profile face and iris biometric features. In embodiments, biometric collection and identification can be done at the crime scene. For example, an operator can quickly collect biometric data from all potential POIs at a blast site or other crime scene. This data can be stored in collections, geotagging, and digital papers to compare POI against past and future crime scenes. In addition, biometric data can be collected in real time from the POI from home and building searches. Such displayed data may allow the operator to know whether to release, detain, or arrest a potential POI. In other embodiments, low profile data collection and identification may be done in a distance environment or otherwise. The user can be assimilated with the locals, for example, by moving through the marketplace and collecting biometric, geographical location and / or environmental data with minimal visible effects. In addition, biometric data for these may be collected to identify whether the deceased or injured is a POI. In embodiments, the user may identify a known or unknown POI by the face of the deceased or injured, or other persons, iris identification, fingerprint identification, visible identification mark, etc., and maintain the updated digital document with this data.

In embodiments, a laser range finder and / or inclinometer may be used to determine the position of a person of interest and / or a rapid-blowing device, other items of interest, and the like. Various devices described herein may include a digital compass, an inclinometer, and a laser range finder to provide a geographic location, such as a POI, a target, an IED, an item of interest, and the like. The POI and / or geographic location of the item of interest may be transmitted over a network, tactical network, or the like, and such data may be shared among people. In embodiments, the apparatus may allow the optical array and laser range finder to simultaneously identify and measure the geographic location of multiple POIs by continuous observation of groups or crowds in the field in uncontrolled environments have. In addition, in embodiments, the apparatus may include a laser range finder and a designator to simultaneously measure and colorize the target by successive observations of one or more targets. In addition, in embodiments, the device may be a military wearable, handheld or otherwise, and may be used to determine the geographic location of the target by an integrated laser rangefinder, digital compass, inclinometer, and GPS receiver to determine the position of the enemy on the battlefield . &Lt; / RTI &gt; In embodiments, the device may include an integrated digital compass, an inclinometer, a MEMS gyro, and a GPS receiver to record and display the position of the soldier and its line of sight. In addition, the various devices may be integrated with an integrated GPS receiver or other GPS receiver, an IMU, a three-axis digital compass or other compass, a laser rangefinder, a gyroscope, a micro-electro-mechanical system (MEMS )- based gyroscope, An accelerometer and / or an inclinometer. Various devices and methods, such as those described herein, may allow a user to locate enemy and POI on the battlefield and share this information with friends through a network or other means.

In embodiments, the user may be connected to a mesh network or may be networked with communication and geographical locations. In addition, each user can be provided with a pop-up or other location map of all users or proximity users. This can provide the user with information on where the ally is located. As previously described, the enemy's position can be found. It may be provided as an enemy pop-up or other location map that can track the enemy's location and provide information to the user where the ally is located. Positions of friendly and enemy forces can be shared in real time. The user can be provided with a map indicating such a position. Such maps of friendly, enemy positions and / or numbers and combinations thereof may be displayed on the user's eyepiece or other device for viewing.

In embodiments, the devices, methods, and applications may enable hands-free, wireless maintenance and repair with visual and / or audio enhanced instructions. This application may include RFID detection for component location and kitting. In the examples, the user may use a device for augmented reality guidance field repair. Such on-site repairs can be guided by hands-free, radio maintenance and repair instructions. Devices such as eyepieces, projectors, monocular instruments and the like and / or other devices described herein may display images of maintenance and repair procedures. In embodiments, such an image may be still image and / or video, animated, 3D, 2D, and the like. In addition, the user can be provided with voice and / or audio commentary of this procedure. In embodiments, this application may be used in a high-threat environment where it is a safety consideration to work undetected. The augmented reality video and video may be projected or otherwise overlaid on the actual object on which the user is working or within the user's view of the object to provide video, graphics, text, or other indication of the procedure to be performed. In embodiments, a library of programs for various procedures may be downloaded and accessed from a body-worn computer or from a remote device, a database and / or a server, etc., either wired or wirelessly. Such programs may be used for actual maintenance or training purposes.

In embodiments, the apparatus, method, and description herein may provide an inventory tracking system. In embodiments, such a tracking system may enable scanning at distances up to 100 meters to handle more than 1000 simultaneous links at a data rate of 2mb / s. The system can provide audio and / or visual information with commentary on inventory tracking when viewing and / or in the vicinity of the inventory. In embodiments, the device may include an eyepiece, a monocular device, a binocular device, and / or other devices described herein, and inventory tracking may include SWIR, SWIR color, and / or Night Vision technology, Wired wired or wireless computers, wireless UWB security tags, RFID tags, helmet / helmet readers and displays. In embodiments, by way of example only, the user may be provided with visual and / or audio information relating to an inventory such as which item should be discarded, transported, discarded or transported, etc., Lt; / RTI &gt; In addition, this information can be used to highlight or otherwise provide visual identification of the item in question with an indication. Such information may be displayed on the user's eyepiece, projected on the article, displayed on a digital or other display or monitor, or the like. The article in question may be tagged with UWB and / or RFID tags, and / or the augmented reality program may be provided to the user for visualization and / or identification so that the various devices described herein may provide the information necessary for inventory tracking and management. Or instructions.

In various embodiments, SWIR, SWIR color, monocular, night vision, body-worn wireless computers, eyepieces such as those described herein, and / or devices such as those described herein may be used in evolutionary work have. In embodiments, the user may have increased visibility through smoke and may be provided to the user by his device in a map or other map overlayed with the location of various people so that the user can know the location of the firefighter and / Can be displayed. The device can show a real-time display of the location of all firefighters and can provide hot spot detection of areas with temperatures below &lt; RTI ID = 0.0 &gt; 200 C &lt; / RTI &gt; and without triggering false alarms. To help guide the user through the structure and / or environment, a map of the facility is also provided by the device, displayed on the device, projected from the device, and / or displayed on the user's eyes via an augmented reality or other means. Can be overlaid.

Systems and devices such as those described herein may be configurable with any software and / or algorithm to meet mission-related requirements and / or system upgrades.

73, the eyepiece 100 includes a biometric data acquisition sensor for recording an individual's biometric signature (s), as well as a typical handheld flashlight function and the like, Biometric flashlight '(7300). The biometric flash light may be coupled to the eyepiece directly, e.g., via a direct wireless connection from the biometric flashlight to the eyepiece 100, or wirelessly with the biometric flash light, as shown in the embodiment shown in FIG. 73 Through the intermediate transceiver 7302 and from a transceiver to an eyepiece via a wired or wireless interface (e.g., where the transceiver device is worn on a belt, etc.). Although other mobile biometric devices are shown in the drawings without representing the transceiver, those skilled in the art will appreciate that any of the mobile biometric devices may be indirectly connected to the eyepiece 100 via the transceiver 7300, ), Or may be operating independently. Data may be transferred from the biometric flash light to the eyepiece memory, into the memory within the transceiver device, to the removable storage card 7304 that is part of the biometric flash light, and so on. As described herein, a biometric flash light may include an integrated camera and a display. In embodiments, the biometric flash light may be used as a standalone device without an eyepiece, wherein the data is stored internally and the information is provided on the display. In this way, non-military personnel can use biometric flashlights more easily and safely. Biometric flashlights may have a range (e.g., a range of 1 meter, 3 meters, 10 meters, etc.) that capture specific types of biometric data. The camera can provide monochrome or color images. In embodiments, the biometric flashlight can provide a concealed biometric data acquisition flashlight-camera that can quickly geolocate, monitor and collect environmental and biometric data for on-board or remote biometric matching. In an exemplary usage scenario, a soldier may be assigned to a borderline at night. Soldiers may use biometric flashlights, such as seemingly only typical flashlights, but they may be executed as part of a data collection and / or biometric identification process and / or acquire biometric features without the knowledge of the person illuminated by the device do.

Referring now to Figure 76, a 360 ° imager utilizes digital foveated imaging to focus pixels on any given area and deliver a high resolution image of a designated area. Embodiments of the 360 ° imager can feature a continuous 360 ° x 40 ° panoramic FOV with ultra-high-resolution, forbidden view and simultaneous, independent 10x optical zoom. A 360 ° imager may include a dual 5 megapixel sensor and an imaging capability of 30 fps and an image acquisition time of less than 100. The 360 ° imager may include a gyro-stabilized platform with an independently stabilized image sensor. The 360 ° imager can have only one moving part and two imaging sensors, which allows for reduced image processing bandwidth in small optical system designs. The 360 ° imager can also feature low angular resolution and high speed video processing and can be sensor agnostic. The 360 ° imager can be used as a surveillance device in a start-up vehicle with a gyro stabilization platform, at a facility, or at a traffic light or pole, robot, aircraft or other location, which enables permanent surveillance. Multiple users can view the environment imaged by the 360 ° imager independently and simultaneously. For example, an image captured by a 360 ° imager may be displayed on an eyepiece to allow all recipients of data (e.g., all occupants in a combat vehicle) to have real-time 360 ° situational awareness. A panoramic 360 ° imager can recognize people at 100 meters, and a 10x zoom that is patched to read a license plate at 500 meters can be used. The 360 ° imager allows for constant recording of the environment and features an independently controllable, bubbly imager.

FIG. 76A shows an assembled 360 DEG imager, and FIG. 76B shows a view of a 360 DEG imager. The 360 ° imager comprises a capture mirror 7602, an objective lens 7604, a beam splitter 7608, lenses 7610 and 7612, a MEMS mirror 7614, a panoramic sensor 7618, a panoramic image lens 7620, An image sensor 7622, a vibration sensor 7624, and a poisoned image lens 7628. The images collected by the 360 ° imager can be geo-located and time and date stamped. Other sensors, such as thermal image sensors, NIR sensors, SWIR sensors, etc., may be included in the 360 ° imager. The MEMS mirror 7614 is a proprietary mirror prism that uses a single viewpoint hemispherical acquisition system to enable high and uniform resolution. The imager design enables scan accuracy of less than 0.1 °, foveated distortion of less than 1%, 50% MTF at 400 lp / mm, and foveated acquisition of less than 30 milliseconds .

The 360 ° imager may be part of a network with wireless or physical reach back to the TOC or database. For example, a user can use a display with a 360 ° imager driver to view images from a 360 ° imager wirelessly or using a wired connection (such as a mil-con type cable). The display may be a combat walkie-talkie or mesh network computer connected to the headquarters network. Data from a database (such as a Dod institution database) can be accessed by a combat radio or mesh network computer, for example, using a removable memory storage card or through a network connection.

Referring now to Figure 77, multiple simultaneous view cameras may be used for imaging. A feed from multiple simultaneous view cameras may be transmitted to eyepiece 100 or any other suitable display device. In one embodiment, the multiple simultaneous view camera may be a fully articulated, 3- or 4-concurrent view, a SWIR / LWIR imaging and a target designation system that simultaneously enables wide field, mid-field and narrow field views, The sensor is VGA or SXVGA resolution for day or night operation. The lightweight godfather sensor array can be geographically referenced as well as inertially stabilized, thereby enabling highly accurate sensor positioning and target designation by virtue of its NVG-compatible laser pointer function under all conditions. His unique multiple simultaneous field of view allows wide area surveillance in the visible, near-infrared, short-wave infrared and long-wave infrared regions. It also enables high resolution, narrow field of view for more accurate target identification and assignment by point-to-grid coordinates when combined with output from a digital compass, inclinometer and GPS receiver.

In one embodiment of a multiple concurrent view camera, a 256-bit AES-encrypted (e.g., 256-bit AES-encrypted) image with automated POI or multiple POI tracking, face and iris recognition, onboard match and laptop, combat radio, or other networked or mesh- There may be individual, adjustable and simultaneous views such as 30 °, 10 ° and 1 ° with wireless communication via UWB. The camera can be networked to the CP, TOC and biometric databases and includes a high-resolution sensor with 3-axis, gyro-stabilized, high dynamic range to provide the ability to see conditions from the sun to extremely low light . The ID can be created immediately and stored locally or on a remote storage device for analysis. The camera can "look and locate" the exact geographic location of POIs and threats to distances of over 1,000 meters, an integrated 1550nm eye-safe laser rangefinder, networked GPS, three-axis gyros, It is compatible with ABIS, EBTS, EFTS and JPEG 2000, features electronic image enhancement and enhancement electronic stabilization assistant in tracking and recording accelerometer and inclinometer, full motion (30 fps) color video, and supports MIL -STD 810 can be satisfied. The camera is equipped with a gimbaled ball system that integrates mobile non-collaborative biometric collection and identification for isolated biopsy solutions in gates, checkpoints, and facilities, as well as laser distance measurement and POI geographic location. . Multi-mode biometrics include collecting and identifying faces and irises, and recording video, posture, and other break marks or movements. The camera may include the capability of geo-location tagging with time, date, and location for all POI and collected data. The camera facilitates rapid deployment of context awareness to network support units, CPs and TOCs.

In another embodiment of a multiple simultaneous-view camera, the camera includes three separate color VGA SWIR electro-optic modules providing 20 °, 7.5 ° and 2.5 ° simultaneous field of view and a 1 in 1 wide area for accurately imaging the POI and target in a tiny configuration Lt; RTI ID = 0.0 &gt; LWIR &lt; / RTI &gt; The 3-axis gyro-stabilized, high-dynamic-range color VGA SWIR camera provides the ability to view through fog, smoke, and mist without blooming, as well as from dazzling sunlight to extremely low light conditions. The geographic location is acquired by the integration of a MEMS tri-axis gyroscope and a tri-axis accelerometer to reinforce the GPS receiver and magnetometer data. The integrated 1840nm eye-safe, laser range finder and target designator, GPS receiver and IMU provide "seeing" the exact geographic location of POIs and threats up to 3km away. The camera displays full-motion (30 fps) color video and stores it in its "camcorder on chip", allowing it to be used for remote access during flight or for post-op review, Save to drive. Electronic image enhancement and enhancement Electronic stabilization helps in tracking POIs and targets, measuring and specifying geographic location distances. As such, the eyepiece 100 provides an unobstructed "view" of the threat by displaying a feed from multiple simultaneous view cameras. In certain embodiments of the eyepiece 100, the eyepiece 100 may also include a "perspective", flip up / down, electro-optical display mechanism that displays sensor images, movement maps and data, You can provide a view. In one embodiment, the flip up / down, electro-optic display mechanism can be snap-on to the NVG mount of any standard, MICH or PRO-TECH helmet.

77 is a view showing the arrangement of the laser distance meter and designator 7702, the total internal reflection lens 7704, the mounting ring 7708, the total internal reflection lens 7710, the total internal reflection lens 7714, the antireflection beehive ring 7718, 1024 SWIR 380-1600 nm sensor 7720, anti-reflective beehive ring 7722, 1280 x 1024 SWIR 380-1600 nm sensor 7724, anti-reflective beehive ring 7728, and 1280 x 1024 SWIR 380-1600 nm sensor 7730, Lt; RTI ID = 0.0 &gt; view camera. &Lt; / RTI &gt; Other embodiments may include additional TIR lenses, FLIR sensors, and the like.

78, a flight eye is shown. The feed from the flight eye may be transmitted to the eyepiece 100 or any other suitable display device. The flight eye may include a plurality of individual SWIR sensors mounted on a curved imaging array having a plurality of FOVs. Flight Eye is a low profile monitoring and targeting system that enables continuous imaging of the entire field with a single flyover. Each sensor is capable of displaying VGA to SXGA resolution to be. Its modular design allows selective fixed resolution changes from 1 [deg.] To 30 [deg.] To telephoto in any element, to wide angle imaging in any area of the array. The resolution of each SWIR imager is 1280 x 1024 and is sensitive to 380 to 1600 nm. The multiple DSP array boards "stitch" all the images together and auto-subtract duplicate pixels for smooth images. The simultaneous 1064 nm laser target indicator and range finder 7802 can be mounted simultaneously with any imager without interfering with the FOV of the imager.

106, the eyepiece 100 may operate in cooperation with an internal software application 7214 for an eyepiece, which may be developed in connection with an eyepiece application development environment 10604, wherein the eyepiece 100 Projected images on a see-through or translucent lens to include projection equipment suitable for allowing the wearer of the eyepiece to view the displayed image as well as the surrounding environment provided through the software application 7214 . A processor, which may include memory and an operating system (OS) 10624, hosts the software internal application 7214, controls the interface between the eyepiece command and control and the software application, controls the projection facility, .

The eyepiece 100 may include an operating system 10624 running on a multimedia computing facility 7212 hosting a software internal application 7214 wherein the internal application 7214 is a third party Which is developed for the purpose of downloading from the application store 10602, the 3D AR eyepiece app store 10610, the networked third party application server 10612, and the like to the eyepiece 100, Lt; / RTI &gt; The internal application program 7214 is operable, for example, in cooperation with the API 10608, via an input device 7204, an external device 7240, an external computing facility 7232, an instruction and control 10630 facility, And can interact with the eyepiece control process facility 10634. The internal application 7214 may be used in the eyepiece 100 via a network communication connection 10622 such as the Internet, a local area network (LAN), a mesh network with other eyepieces or mobile devices, a satellite communication link, a cellular network, . The internal application 7214 may be purchased through an application store such as the App Store 10602, the 3D AR eyepiece App Store 10610, and the like. An internal application program 7214 such as a software internal application 7214 developed specifically for the eyepiece 100 may be provided through the 3D AR eyepiece store 10610. [

A software developer may be required to modify the base application to create a new 3D application version of the base application to create a new eyepiece application (e.g., a 3D application), an eyepiece application Development environment 10604 may be available. The eyepiece application development environment 10604 allows the developer to provide access to control methods, UI parameters, and other specifications available in the eyepiece, when the completed application is loaded into the eyepiece or otherwise functioned for the eyepiece And may comprise a configured 3D application environment. The eyepiece may include an API (10608) designed to facilitate communication between the completed application and the eyepiece computing system. Within the developer's development environment, application developers can focus on developing applications with specific functionality without worrying about details of how to interact with eyepiece hardware. The API may also make it easier for the developer to modify existing applications to create 3D applications for use in the eyepiece 100. [ In an embodiment, the internal application 7214 may include a client-server configuration, a hybrid client-server configuration (e.g., an internal application 7214) locally on the eyepiece 100, (E.g., running on server 7214), hosting the application program entirely on the server, being downloaded from the server, and so on. With respect to the internal application 7214, a network data storage device 10614 may be provided, for example, in connection with an application server 10612, purchased applications, and the like. In embodiments, the internal application 7214 may include other features to provide market content to the user of the eyepiece 100, for example, in cooperation with the execution of the internal application 7214 to provide sponsored advertising. To-sponsor facility 10618, market 10620, and the like.

In embodiments, software and / or application programs that are used in the eyepiece or complementary to the eyepiece may be developed. The eyepiece application can be developed through an open source platform, a closed source platform, and / or a software development kit. Software development kits for developing applications for eyepieces and software developed therefrom can be open source or closed source. Applications that are compatible with Android, Apple, and other platforms can be developed. An application may be sold by an app store associated with an eyepiece, or therefrom, downloaded from an independent app store, or otherwise.

For example, an integrated processor of the eyepiece may process the content to launch and display at least one software application and the integrated image light source may introduce the content into the optical assembly of the eyepiece. The software application may provide interactive 3D content to the user through interaction with at least one of the control and sensor facilities of the eyepiece.

In embodiments, the eyepiece can be used for a variety of applications. Eyepieces can be used for consumer applications. It is not intended to provide an exhaustive list, but merely to illustrate how the eyepiece can be used in a variety of applications, such as travel applications, training applications, video applications, exercise applications, personal assistant applications, augmented reality applications, , A navigation application, a movie application, a face recognition application, a place identifier application, a person identifier application, a text application, an instant messaging application, an email application, a todo application, a social networking application, or the like It can be used against it. Social networking applications can include applications such as Facebook, Google +, and so on. In embodiments, the eyepiece may also be used for an enterprise application program. It is not intended to provide an exhaustive list, but rather to provide a method and system for providing an e-mail message, such as an eyepiece, for example, in a billing application, a customer relationship management application, a business intelligence application, a human resource management application, a form automation application, Microsoft Office, and so on. In embodiments, the eyepiece may be used for industrial applications. For example, eyepieces can be used for or for advanced product quality planning software, mass production part approval software applications, statistical process control applications, vocational training applications, and the like, rather than providing an exhaustive list.

107, an eyepiece application development environment 10604 may be used for developing an application program that may be provided to an application store 10602, a 3D AR eyepiece application store 10610, and the like. The eyepiece application development environment 10604 may include a user interface 10702, a control method access 10704, and the like. For example, a developer may use menus and dialog boxes within the user interface to access the control scheme 10704 for selection, thus allowing the application developer to select the approach. Developers can generally choose a template method to run an application, but they can have individual controls that can be selected for various functions that may override the template function method at application run time. The developer can also use the user interface 10702 to develop an application program with a control scheme by FOV control, for example, via a field of view (FOV) interface. The FOV interface can provide a way back and forth between the FOV (showing each display) and the FOV showing a single display (for each eye). In embodiments, a 3D application for the eyepiece may be designed within a single display view, since the API 10610 will provide a transformation to determine which display to use for which content, It is possible to select a specific eye display. In embodiments, the developer may manually select and / or view what is to be displayed in each eye, such as through the user interface 10802 or the like.

The eyepiece may have a software stack 10800 as described in FIG. The software stack 10800 may have a head-mounted hardware and software platform layer 10818, an interface-API-wrapper layer 10814 for the platform, a library layer 10812 for development, an application layer 10801, have. The application layer 10801 may in turn comprise a consumer application 10802, an enterprise application 10804, an industrial application 10808, and other similar applications 10810. In addition, hardware 10820 that is associated with the execution or development of internal application program 7214 may also be included in software stack 10800.

In embodiments, the user experience can be optimized by allowing the augmented image to be focused on the surrounding environment and allowing the display to be set to the appropriate brightness given the ambient light and the content being displayed.

In one embodiment, the eyepiece optical assembly may include an electro-optic module (i.e., display) for each eye that conveys the content in a stereoscopic manner. In certain cases, stereoscopic views are not desired. In embodiments, for a particular content, only one display may be turned on, or only one electro-optic module may be included in the optical assembly. In other embodiments, the brightness of each display may be varied such that the brain ignores the darker display. The automatic brightness control of the image light source can control the brightness of the displayed content based on the brightness in the environment. The rate of change in brightness may depend on changes in the environment. The rate of change in brightness can be matched to the adaptation of the eye. Display content may turn off for a period of time after a sudden change in ambient brightness. By darkening the environment, the display content can be dim. As the environment becomes bright, the display content can be brighter.

When going from a bright environment to a dark environment, it takes a certain period of time for a person's eyes to adapt to the darkness. During this period, the eyes have only limited visibility for dark environments. In some situations, such as in security or law enforcement situations, it is important to be able to move from a bright environment to a dark environment and quickly determine what activities or objects are in a dark environment. However, it can take up to 20 minutes for a person's eyes to fully adapt to a dark environment. During this time, the person's vision about the environment drops, which can lead to dangerous situations.

In some cases, a bright light such as a flashlight can be used to illuminate a dark environment. In other cases, it is possible to cover the human eye for a period of time before entering the dark environment to allow the eyes to partially adapt to the dark environment before entering the dark environment. However, support is being made to reduce the time that a person's vision falls during a transition from light to dark when bright light can not be used in a dark environment and it is not feasible to cover the human eye before entering a dark environment A method of providing assisted viewing is needed.

Night vision goggles and binoculars that provide images in a dark environment are known. However, these devices provide a constant brightness image and thus do not allow the user's eyes to adapt to darkness, and therefore the device must be used continuously in dark environments. As a result, these devices do not take advantage of the fact that people can see very well in dark environments after their eyes are fully adapted to darkness.

U.S. Patent No. 8094118 provides a method of adjusting the brightness of a display in response to the brightness of the surrounding environment in order to save power. This method relates to the perceived brightness of the display and not to the adaptation of the user's eye in a transition from a bright environment to a dark environment. In addition, this method does not help the user to see the environment.

Thus, there is a need for a way to help a person move from a bright environment to a dark environment during the period of adaptation to darkness.

The head-mounted display with perspective features provides a clear view of the scene in front of the user, as well as the ability to display the image, where the user sees the combined image of the displayed image overlaid with the perspective view. The present disclosure provides a method of providing a support view of the environment when the user is transitioning from a bright environment to a dark environment. This method uses a camera on the head mounted display to quickly capture the dark conditions and capture conditions so that the image is displayed to the user. The brightness of the displayed image is gradually reduced to allow the user's eyes to adapt to the dark environment.

Data from Hecht and Mandelbaum in Figure 154 (Davis, H., "The Eye" vol. 2, London Academic Press, 1962, Chapter 5 "Dark Adaptation and Night Vision" ) Is a chart of a typical dark adaptation curve for the human eye, wherein the shaded area represents 80% of the group of subjects. In this chart, the curve shows the lowest illuminance that can be observed at a particular time, starting from the bright light environment at time 0 and going straight to the dark environment, where the lowest illuminance that can be observed is the illuminance Of the light, and the person reports a spot visible after a different time in darkness. As can be seen from the curve, the human eye adapts with the passage of time, and thus a spot of lower illuminance can be seen gradually over a period of about 25 minutes. As shown in the chart of FIG. 154, there are actually two mechanisms that contribute to dark adaptation in the human eye. The cone in the eye (also known as photopic vision) adapts more quickly in brighter conditions than a relatively slow adaptive rod (also called scotopic vision). As a result, the time to adapt to moving from a lighter condition to a darker condition can take a considerable amount of time depending on how dark the environment is. During darkness, a person can be close to the blind.

Table 2 provides typical illuminance values (in units: both Lux and Lambert) for normal illumination conditions. The range of illumination for outdoor lighting conditions spans nine digits between bright sunlight and cloudy night without a moon. The illuminance values are also given for indoor lighting conditions for comparison.

Table 2 shows typical lighting levels from the website http://www.engineeringtoolbox.com/light-level-rooms-d_708.html :

Table 3 provides perceived brightness values (in Brils) when the eye changes from one fully illuminated illumination condition to a darker one. The changes in the lighting conditions that are shown are from Table 2 to the illuminance values given as Lambert. An example given at the bottom of Table 3 is an example of Bril, where the perceived brightness of 1 Bril is approximately the brightness provided by the half moon outdoors on a private night or 0.000001 Lambert, where the perceived brightness in the human visual system is illuminated An expression relating to a change in condition is provided in U.S. Patent No. 8094118, which is given below as Equation 3 for reference.

here

In other examples shown in Table 3, it can be easily seen that a change in illumination condition occurs from a number of conditions encountered in real life to a situation that causes perceived darkness. Changes in various lighting conditions and perceived brightness when the change first occurs are shown in Table 3. In many of these examples, the perceived brightness when moving from a bright condition to a darker condition for the first time is well below the perceived brightness provided by the half moon after being completely darkened. Moving from daylight to a warehouse or a dark public place is particularly problematic, and the eyes are essentially nothing for a period of time until they are adapted to new lighting conditions. The disclosure set forth herein provides a method of assisting the human eye while the eye adapts to darker conditions during a transition from light to darker conditions.

Table 3 shows the perceived brightness levels when changing from a bright environment to a dark environment using the illuminance values from Equations 3 and 2.

FIG. 155 is a flow chart of the method of the present invention described in Spillman L., Nowlan AT, Bernholz CD., "Dark Adaptation in the Presence of Waning Background Luminances", Journal of the Optical Society of America, Vol. 2, Feb 1972. &lt; tb &gt;&lt; TABLE &gt; Figure 155 shows that the incremental threshold is measured for logarithmic background luminance that decreases linearly over time. The background changed over seven log units within 3.5 min (), 7 min (), 14 min (), 21 min (), and 3.5 min without pre-exposure (). The arrows indicate the time of background extinction. The normal dark threshold value ( X ) recorded in the absence of any background luminance generally coincides with the curve for the steepest background slope, and is omitted after being unchanged.

The data in Figure 155 shows the lowest detectable level (threshold) of illumination on the illuminated spot by the eye as the human eye is darkened when the illumination condition changes from 0.325 Lambert (some cloudy days) to complete darkness And is based on a measured speed that is gradually reduced. The other curves shown in the chart of FIG. 155 are for the illumination conditions in which the change from bright to dark is performed at a different linear velocity instead of the instant as shown in FIG. The curves on the left side of the chart show a faster adaptation to darkness under conditions where the change from bright to dark is fast. As supported by the data shown in FIG. 154 and shown in FIG. 155, the typical time to darken when moving from light to full dark is about 15 minutes. The data in FIG. 155 shows that the brightness can be changed linearly over a period of 14 minutes with only a small disadvantage in terms of the time to darkness, and the data in FIG. 155 shows that the darkening time is 15 minutes To 19 minutes for a gradual change over 14 minutes. The disclosure herein provides a display of a darkened environment with a gradually decreasing brightness over time to allow the user to adapt to a dark environment while still being provided with an observable image of a dark environment . &Lt; / RTI &gt; This method uses a camera that quickly adapts to dark environments to capture images in dark environments. The captured image is provided to a user on a perspective head-mounted display, wherein the brightness of the image is progressively reduced as time elapses, thereby allowing the user's eyes to darken and allowing the user to view the perspective view of the perspective- You can gradually see the environment.

156 shows a head mounted display device 15600 having a perspective function. The on-board display device 15600 includes a perspective display 15602, one or more cameras 15604 and an electronic device 15608 wherein the electronic device 15608 can be a processor, a battery, a global positioning sensor (GPS) A directional sensor, a data storage device, a wireless communication system, and a user interface.

In one embodiment, a head mounted display device 15600 having at least one camera 15604 or 15610 provides an enhanced view of the dark environment on the perspective display 15602 for a period of time for the user's eyes to adapt to the dark environment . The camera 15604 or 15610 can automatically adjust capture settings such as gain, ISO, resolution, or pixel binning very quickly by the automatic exposure system. In some embodiments, the lens of the camera 15604 or 15610 is changeable to enable improved image capture in a dark environment. The brightness of the image displayed on the perspective display 15602 may be adjusted over time to match the adaptation of the eye and any change in photochromic material that may be associated with the on-board display device 15600 have. In this way, a rapidly changing photochromic material is not required. Photochromic materials having transition times to the order of a few degrees of transparent are well suited to the embodiments of the present disclosure. In either case, the field of view of the displayed image of the environment must match the field of view of the on-board display device 15600 to provide a dark environment display that is easy to interface within the augmented reality mode.

The present disclosure provides a head mounted display device 15600 having one or more cameras 15604 or 15610 wherein the captured image of the scene in front of the user is displayed with a range of brightness over time . The camera 15604 or 15610 and associated autoexposure systems can adapt to changes in the brightness of the environment much faster (typically less than one second) than the user's eyes can adapt. In one embodiment, the camera 15604 or 15610 captures an image of a scene in front of the user, and when the brightness of the scene rapidly changes from light to dark, the captured image of the scene is displayed on the perspective display 15602 to the user Is displayed. Immediately after moving into a dark environment, the user is provided with a bright image of the scene, and the brightness of the displayed image is adjusted so that the brightness of the displayed image is decreased with the lapse of time . FIG. 157 shows a graph of the brightness of a displayed image provided to a user over time, where t1 is a time at which the brightness of the environment changes from bright to dark. The acquisition of the image of the environment can be started at or before time t1. After t1, the brightness of the displayed image is reduced until time t2 when the user's eyes are adapted to the dark environment. After time t2, the brightness of the displayed image is kept constant at a level at which the user can observe the environment in the perspective mode. In other embodiments of the present disclosure, the brightness of the displayed image after time t2 is zero, thus the user only observes the dark environment in the perspective mode. In a further embodiment of the present disclosure, the image content of the displayed image changes from the captured image of the environment in front of the user after t2 to another image or information (augmented reality information, e.g., direction or direction). In another embodiment of the present disclosure, when the environment is darker than a predetermined level, the brightness of the displayed image of the environment is reduced to a level maintained after time t2, thereby providing a version of night vision, Responds to rapid changes in ambient lighting, and also provides longer night vision when conditions are too dark to allow the eye to adapt for the current task. The level of darkness at which nightvision imaging is provided after time t2 may be selected by the user in the mode of operation setting, wherein an operation that requires detection of additional details in the environment may result in a brighter display of the environment Use settings that provide video.

In a preferred embodiment, the brightness of the displayed image of the environment is reduced at a rate corresponding to the rate at which the user's eyes adapt to the dark environment (e.g., from a bright image corresponding to the curves shown in FIG. 155 to a dim image or image 14 minutes transition to none). In this way, while the user's eyes adapt to darkness, the user is temporarily provided with an image of the environment, but the time to adapt to the dark environment does not substantially extend compared to the time to adapt without the displayed image.

In a further embodiment of the present disclosure, the lens of the camera 15604 or 15610 changes when the user enters a dark environment to provide an improved low light image capture function. In this case, cameras 15604 or 15610 or other photoactive detectors within electronic device 15608 detect a change from a bright environment to a dark environment, where the brightness of the environment is controlled by an automatic exposure sensor within electronic device 15608, Is detected by detecting a decrease in the pixel code value from the image sensor in the camera 15604 or 15610. [ The lens of the camera 15604 or 15610 is subsequently changed to increase the light collection capability or to allow the camera 15604 or 15610 to capture the infrared image. Example: By changing to a lens with a lower f #, the light collection ability can be increased. Example: By removing the infrared cut filter in the lens assembly, by moving the lens elements relative to each other to refocus, or by changing one or more of the lens elements to an infrared lens element, the camera 15604 15610 can be enabled to capture infrared images. In another embodiment, the image sensor in the camera 15604 or 15610 is changed to enable infrared image capture.

158 shows a flow chart of the method of the present disclosure. At step 15802, the user moves from a bright environment to a dark environment. At step 15804, camera 15604 (or other photoactive detector in electronic device 15608) detects that the illumination condition in the environment has changed to a dark condition. At step 15808, the capture conditions used by the camera 15604 or 15610 to enable image capture, specifically video image capture, in a dark environment are adjusted by an automatic exposure system. At step 15810, an image of the environment is captured by the camera 15604 or 15610 and displayed on the perspective display 15602 at a first brightness level, wherein the first brightness level of the displayed image is such that the environment is bright Is similar to the brightness perceived by the user in the perspective view of the environment just before changing to dark lighting conditions. Then, at step 15812, the brightness of the displayed image of the environment is reduced over time such that the user's eye can be dark while viewing the image of the environment. The decrease in brightness may be linear over time, or may be non-linear, as shown in FIG. The period in which the brightness of the image is reduced can correspond to the change of the illumination condition in the environment. Depending on how dark the environment is, at step 15812, the brightness of the displayed image may be reduced to zero or maintained at a predetermined level to provide a version of night vision.

Exemplary Scenario 1

As indicated by the data in Table 2, the police working at the daytime (about 1.0 Lambert) break the doors to some dark rooms, similar to the darkness at many restaurants (around 0.0035 Lambert). When the door is opened, the police will recognize that the dark room is 0.000007 Bril or 10000 X darker than the illumination provided by the half moon, as shown in the data in Table 3. In essence, the police will not see anything in the dark room. Based on the curves in FIG. 155, after about 1 minute, the police will be able to see anything in the dark room with 0.0035 Lambert (0.0035 Lambert = 0.54 Logmolambert). This is a dangerous situation, because the eyes of people in dark rooms are already darkened and therefore they will be able to see the police. If the police are wearing a head-mounted display device having a camera and a perspective display as described herein, a dark room image can be presented to the police for about 1.5 minutes during the time the police's eyes are dark . After that time, the police can see the dark room through the perspective display. Perspective display may still be used to transmit instructions or other information to the police while the police are watching the dark room through the perspective display (as in the AR reality imaging system). As such, the head-mounted display device of the present disclosure provides the police with instant visual acuity in a dark room, such as is limited only by the low illumination function of the camera.

As long as the field of view presented on the displayed image closely matches a portion of the field of view of the police, and the video image is live and has only a limited delay between acquisition and display, the police can easily move back and forth in the dark room will be. As time passes, the brightness of the displayed image decreases as the police eyes adapt to the dark room.

To provide video imaging up to partial moonlight level illumination, the camera may be a fairly common digital camera with good low light performance operating in high ISO and binned mode. To provide imaging to a darker level, a short-wave infrared camera or a camera with visible + near-infrared imaging capability (a camera with an infrared cut filter removed) may be used. As indicated by the data shown in Figures 154 and 155, in very dark conditions, it may be necessary to provide the image to the user for a maximum of 25 minutes (at this point the user's eyes will be fully depressed).

Exemplary Scenario 2

Soldiers in illuminated houses (illuminance 0.025 Lambert = 0.40 log Muller Lambert) open the door and walk to the night with a full moon (illuminance 0.00001 Lambert-2 Logmilambert). As can be seen from the numbers in Table 3, the perceived darkness when a soldier first walks to night is 1000000X times darker than night with half moon when the eyes are fully acclimated. 0.000001 Brill full darkness with brightness of Brill to be. The curves in FIG. 155 show that for this illumination change, the soldier's eyes will take about two minutes before the object can be seen in darker conditions. As in the previous example, this can be a dangerous situation, since the soldier essentially does not see anything for two minutes. The present disclosure provides a tofu head mounted display that captures an image of the environment and displays it to a soldier to eliminate periods of invisible sight. In this case, the brightness of the image can be reduced over a period of 3 to 4 minutes, so that the soldier's eyes can be darkened, and at the end of that time, the soldier can see the head-mounted display Can be operated.

Immediate visibility can be provided to the displayed image over the display field of view. Transitions to the perspective view are provided by progressively reducing the brightness of the displayed image as the user's eyes adapt to the dark conditions.

This technique can also be used to compensate for the photochromic lens associated with a head mounted display device, which may not change very quickly to transparent.

In alternate embodiments, the image presented to the user may be captured 2D by a single camera 15610 (in this case, the images presented to both eyes of the user are the same) or 3D captured by stereoscopic camera 15604 (In this case, the image presented to the user's eyes provides a different viewpoint of the scene). As is known to those skilled in the art, the use of a lens with a split pupil, or another such as using a lens with a microlens array to enable bright field imaging Methods are also possible.

The displayed image may also be adjusted to a different color, such as red or green, to help the eye adapt more quickly to darkness, as is often done in night vision binoculars.

In embodiments, the augmented reality eyepiece AR of the present disclosure is configured to determine and / or compensate for the vergence of the user's binocular vision. The bi-directional motion is the simultaneous rotation of the user's two eyes about the vertical axis to move the optical axis of each of the two eyes in opposite directions to acquire or maintain binocular vision. When a person looks at a closer object, the human binocular moves his or her optical axis inward toward the nose (compound movement called convergence). To see a farther object, a person's binocular moves his or her optical axis away from the nose (compound movement called divergence). When a person fixes his gaze to a point at infinity or far away, the eyes of the person are opened until their respective optical axes are essentially parallel to each other. The bi-directional motion works in conjunction with eye accommodation to allow a person to maintain a clear image of the object as the object moves relative to the person. In the situation where a virtual image (i.e., AR image) such as a label or other information is located near the actual image or overlaps with the actual image, or in order to accurately position the virtual image with respect to the actual image, It is important to compensate for the bending motion when overlapping. The methods of this disclosure for bi-directional motion compensation and / or determination are described herein and are collectively referred to as bi-directional motion methods.

The directional motion method involves determining the distance between the object of interest and the user of the AR eyepiece and then using that distance to determine the direction of the vergence angle (i. E., By the intersection of the optic axis of the user & The angle formed). The angular motion angle is then used to determine the correct placement of the AR image relative to the object (which may be in front of, behind, or in alignment with the object). For example, in a first set of directional motion method embodiments, a single autofocus digital camera having an output signal may be mounted to the AR eyepiece at any convenient location (e.g., in the nose pads section or in the vicinity of one of the pair of eye glasses) . The output of the camera is provided to the microprocessor in the AR eyepiece and / or transmitted to the remote processor. In either case, his signal associated with his autofocus function is used to determine the distance to an object that the user can see when the user is looking straight ahead. This distance, together with the dynamic distance of the user's binocular, is used to determine the exact orientation of the virtual image (e.g., label) that may be desired for the diagonal motion and its objects. The distance and / or directional motion angles may also be used to determine the focus level at which the virtual object must be properly observable by the user. Optionally, additional information about the particular user &apos; s biased motion characteristics may be input and stored in a memory associated with the microprocessor and used to adjust the determination of the biased motion.

In a second set of directional motion method embodiments, an electronic distance meter independent of the camera is included in the AR eyepiece at any convenient location (e.g., in the nose section or near one of the eyeglasses legs). In these embodiments, the output of the electronic range finder is used in the same manner as the output of the autofocus camera described in connection with the first set of directional motion method embodiments.

In a third set of directional motion method embodiments, the AR eyepiece includes a plurality of distance measuring devices, which may be autofocus cameras and / or electronic range finders. All of the plurality of devices may be aligned to determine the distance of objects in the same direction or one or more of the devices may be arranged differently from other devices so that information about the distance to various objects is obtainable . The output from one or more of the devices is input and analyzed in the same manner as the output of the autofocus camera described in connection with the first set of bi-directional motion methods.

In a fourth set of directional motion method embodiments, one or more distance measuring devices are used in the manner discussed above. In addition, the AR eyepiece includes at least one eye tracking device configured to track the movement and / or viewing direction of one or both of the user &apos; s two eyes. The output of the eye tracking device may be provided to a microprocessor in the AR eyepiece or may be transmitted to a remote processor. This output is used to determine the direction in which the user is viewing, and is used to determine the bi-directional motion of the user &apos; s binocular eye tracking information from both eyes, if available. This direction and, if available, the directional motion information can then be used, alone or in combination with the directional motion information determined from the distance measuring device, to arrange one or more virtual images related to one or more objects the user may be viewing, Is used to determine the focus level.

In a fifth set of directional motion methods, one or more distance measuring devices are oriented away from the straight direction towards the front of the user of the AR eyepiece. The distance to the object detected by the distance measuring device is used to display the virtual image of the object in the manner described above. The user may recognize the virtual image when the user is looking straight forward, or not, but the user will recognize the virtual image when the user is viewing the direction of the object associated with it.

A calibration sequence may be used for any of the bi-directional motion method embodiments. The calibration sequence may utilize mechanical calibration characteristics, electronic calibration characteristics, or both. During the calibration sequence, the user's dynamic spatial distance can be determined. The user may also be asked to view a series of real or virtual objects having a range of actual or virtual distances (e.g., from near to far), and the bi-directional motion of the binocular is measured mechanically or electronically or both . The information from this calibration sequence can then be used in determining the directional motion, focus adjustment, and / or virtual image placement when the AR eyepiece is in use. The calibration sequence is preferably used when the user wears the AR eyepiece for the first time, but can be used whenever the user feels re-calibration will be helpful. The information correlating the user with what is obtained during the calibration sequence may be obtained, for example, using any of the techniques described in this document, whenever that particular user identifies himself / herself to the AR eyepiece as his user Can be stored for use.

It should be noted that some distance measuring device uses a distance determination method in which the information received from the sensors of the device is mapped to a rectilinear or non-rectilinear lattice. The information from the various sectors of the grid is compared to determine the distance. In the bi-directional motion method embodiments, raw sensor information, mapping information, calculated distances, or any combination thereof may be used in determining the placement and / or focus of a virtual image or images.

It will be appreciated that the bi-directional motion method embodiments include the placement of a virtual image for either one of the user's eyes or both eyes of the user. In some embodiments, one virtual image is provided in the user's left eye, and another virtual image is provided in the user's right eye. This will enable, for example, acquiring information from the other eye for viewing while providing virtual images or images to one eye. When multiple images are placed in front of the user, the placement may be simultaneous, different, or interlaced in time, regardless of whether the images are the same or different, for example, if the images are at a predetermined flicker rate or rates For example, 30, 60, and / or 180 Hz), and there is an image for the left eye when there is no image for the right eye, and vice versa. In some embodiments, the virtual image is visible only to a person's dominant eye, and in other embodiments, the virtual image is visible only to the user's non-dominant eye. In some embodiments using images that are interlaced in time, virtual images of various objects located at various distances from the user are displayed in the manner described above; When the user views the actual image of one object toward the actual image of another object, only the virtual image corresponding to the actual image of the object to be viewed will be shown by the user's brain. For example, by using a focusing mechanism (a piezoelectric actuator attached to the LCoS or a variable focus lens inserted in the optical path) operating at a high rate (e.g., 30 to 60 Hz), one of the same or different virtual images One or more of which may be placed in more than one depth plane for one or both of the user's binoculars.

In some embodiments, the focal length of the virtual image may be adjusted to provide the user with the illusion that the virtual image is at a desired distance. This adjustment is particularly useful when the image is being presented to both eyes of the user and the relative lateral position of the two images is adjusted for the forward movement. This adjustment may be accomplished, for example, by adjusting the length of the optical path to the image display or by using one or more variable lenses (in some embodiments of the present disclosure, e.g., by raising or lowering the LCoS panel) Which can be accomplished by the method described above.

In embodiments, the present disclosure provides a method of providing depth cues to augmented reality virtual objects or virtual information capable of delivering a broad perceived depth to a wide range of people with different eye characteristics. These depth cue method embodiments of the present disclosure use the difference in lateral orientation or mismatch of an augmented reality image provided to the human &apos; s eyes to provide a difference in the direction of movement of a virtual object or virtual information conveying a sense of depth. One advantage of these methods is that the lateral shift of the augmented reality image can be different for that portion, such that the perceived depth is different for different portions of the augmented reality image. In addition, lateral transitions can be performed through image processing of portions of the augmented reality image. The user can experience a full range of perceived depth through this method from a location close enough for a person to focus on infinity regardless of the age of the person.

In order to better understand this depth clipping method embodiment of the present disclosure, a head-mounted display is provided to add, in some aspects of the augmented reality, an image of a virtual object or virtual information associated with a view of the scene the user is viewing It is useful to keep in mind that it is used. To add additional effects to the perception of augmented reality, it is useful to place virtual objects or virtual information at a perceived depth in the scene. As an example, a virtual label, such as the name of a building, may be placed on an object in the scene. The perceived association between the virtual label and the building is improved if the label and the building are perceived by the user to be at the same depth in the scene. The head mounted display with perspective functionality is well suited for providing augmented reality information, such as labels and objects, because it provides the user with a clear view of the environment. However, in order for the augmented reality information to be useful, the information must be easily associated with the object in the environment, and accordingly placement of the augmented reality information on the object in the perspective view is important. If the head mounted display has a camera that can be calibrated according to the perspective view, horizontal and vertical placement of the augmented reality information is relatively simple, but the depth placement is more complex. US Patent No. 6690393 describes a method for placing 2D labels in a 3D virtual world. However, this method is not concerned with a display having a perspective view, in which most of the image that the user sees is not provided digitally and thus does not know the 3D position of the object. U.S. Patent No. 7,907,166 describes a robotic surgery system that uses a stereoscopic viewer in which a telestration graphic is overlaid on a stereoscopic image of a surgical site. However, similar to the method described in U.S. Patent No. 6,690,393, the system uses captured images that are later manipulated to add graphics, and the majority of the images are not provided digitally, It does not solve the unique situation in the unknown perspective display. Another prior art augmented reality method is to adjust the focus of a virtual object or virtual information so that the user is aware of the difference in focus depth that provides a depth cue to the user. The user perceives the associated depth because the user has to re-focus the binocular to view the object in the scene and to view the virtual object or virtual information. However, the range of depths that may be associated with focus is limited by the distance accommodation the user's binocular can make. This distance adaptation can be limited in some people, especially when people are older and the two eyes lose most of his distance adaptation range. In addition, the distance adaptation range differs depending on whether the user is nearsighted or primitive. These factors render the results of using focus cues unreliable for a large group of users with different age and different eye characteristics. Therefore, there is a need to go beyond what is available in the prior art for widely available methods of associating depth information with augmented reality.

Some of the depth cue method embodiments of the present disclosure are described in this and subsequent paragraphs with respect to Figures 109-121. The head mounted display with perspective features provides a clear view of the scene in front of the user, while also providing the ability to display an image, where the user can view a combined view consisting of a perspective view in which the displayed image is overlaid . The method involves displaying a 3D label and other 3D information using a perspective display to help the user interpret the environment surrounding the user. Stereoscopic pairs of 3D labels and other 3D information may be presented in the user's left and right eyes to place 3D labels and other 3D information at different depths in the scene recognized by the user. In this way, 3D labels and other 3D information can be more easily associated with the perspective view and the surrounding environment.

Figure 109 shows a head mounted display device 109100 with a perspective function and is a special version of the augmented reality eyepiece 100 shown in Figure 1 and described throughout this document. The head mounted display device 109100 includes a perspective display 109110, a stereoscopic camera 109120, an electronic device 109130, and a distance measuring device 109140. The electronic device may include one or more of a processor, a battery, a global positioning sensor (GPS), a directional sensor, a data storage device, a wireless communication system, and a user interface.

Figure 110 shows a scene in front of the user as the user sees in the perspective view. The number of objects at different depths in the scene for discussion is shown. In Figure 111, some of the objects in the scene are identified and labeled. However, labels are presented in a two-dimensional (2D) manner by presenting a label to only one eye of the user or by presenting a label to each eye at the same position in the image, do. This type of labeling makes it more difficult to associate labels with objects, especially since they all appear to be at the same perceived depth, especially when there are foreground and background objects.

Label or other information as a three-dimensional (3D) label or other 3D information so that the information is perceived to be at different depths by the user, to make it easier to associate labels or other information with the desired object or aspect of the environment . This can be done by presenting the 3D label or other 3D information in the overlayed image to the user's eyes with the lateral shift of the position between the images overlaid on the perspective image so that the overlaid images have a perceived depth . This lateral shift between images is also known to the person skilled in the art of stereoscopic imaging, which causes the user to change the relative pointing of his or her two eyes to visually align the image, which leads to the perception of depth . An image having a discrepancy is an image of a 3D label or 3D information overlaid on a perspective view of a scene viewed by a user. By providing a 3D label with a large disparity, the user must align the optical axes of his binocular to align the labels in the stereoscopic image to provide a perception of nearby labels to the user. A 3D label with a small discrepancy (or no discrepancy) can be visually aligned with the binocular of the user looking straight ahead, which provides recognition of a 3D label located a long distance away.

112 and 113 show stereoscopic image pairs for a 3D label to be applied to the perspective view shown in FIG. 112 is a 3D label image shown in the left eye of the user, and FIG. 113 is an image of a 3D label displayed in the right eye of the user. Figures 112 and 113 together provide a stereoscopic image pair. In this stereo pair, the lateral arrangement of the 3D label differs between the images shown in Figs. 112 and 113. Fig. Figure 114 provides the overlaid image of Figure 112 and Figure 113. For the sake of clarity in Fig. 114, the 3D label from Fig. 113 is shown in gray while the 3D label from Fig. 112 is shown in black. In the foreground of FIG. 114, the 3D label from FIG. 113 is disposed to the left of the 3D label from FIG. 112 with a relatively large mismatch. In the background of FIG. 114, the 3D label from FIG. 113 is placed on top of it simultaneously with the 3D label from FIG. 112 without inconsistency. In the mid-ground region shown in FIG. 114, the 3D labels from FIGS. 112 and 113 have medium mismatches. This relative disparity of the 3D label presented in the left eye and right eye corresponds to the depth perceived by the user. By choosing a depth for a 3D label that matches the depth of an object in a scene with which the 3D label is associated, it is possible for the user to understand the connection between the 3D label and the object or other aspect of the environment that the user sees in the perspective view . 115 shows a perspective view of a scene with a 3D label showing a discrepancy. However, in real life, the user will change the orientation of the binocular to make the 3D label consistent within each left / right set, which provides the user with a perception of depth. The calculation of the mismatch is known to those skilled in the art. The equation relating disagreement and distance is given by Equation 1:

Z = Tf / d

Where Z is the distance from the stereoscopic camera to the object, T is the separation distance between stereoscopic cameras, f is the focal length of the camera lens, and d is the mismatch distance on the camera sensor between images of the same object in the scene. When the terms are rearranged to solve for mismatch, the above equation becomes:

d = TF / Z

For example, for a 7 mm focal length camera used in conjunction with an image sensor that is separated by 120 mm and has a center-to-center pixel distance of 2.2 micrometers, when a display is compared to another display, Discrepancies in the number of pixels are given in Table 1 for some representative distances (given in meters).

It should be noted that, in the art, sometimes the disparity value for a stereoscopic image is described using a number from negative to positive, where a value of 0 for an object at a distance selected from the observer, Inconsistencies are defined. To account for this transition at point 0, the aforementioned equations must be adjusted. When a discrepancy value is described in this way, discrepancies between nearby objects and distant objects may be the same magnitude, but the signs may be the opposite.

116 shows an example of a stereoscopic image pair captured by the stereoscopic camera 109120 on the head mounted display device 109100. Fig. Since these images are captured from different viewpoints, they will have a mismatch corresponding to the distance from the on-board display device 109100. In FIG. 117, two images from FIG. 116 are overlaid to show discrepancies between images in stereoscopic pairs. This discrepancy corresponds to the discrepancy seen in the 3D labels shown for the objects in Figures 114 and 115. Accordingly, it will be recognized that the 3D label is located at the same depth as the object that is to be associated with the 3D label. Figure 118 shows the 3D label shown by the user as an overlay for a perspective view with the left eye and right eye.

119 is a flow chart of a depth cue method embodiment of the present disclosure; In step 119010, the electronic device 109130 in the on-board display device 109100 determines the GPS position using the GPS for the on-board display device 109100. In optional step 119020, electronic device 109130 uses an electronic compass to determine the orientation of the view. This can be achieved by comparing the GPS position of the head mounted display device 109100 with the database of the GPS position of another object in the head mounted display device 109100 or by connecting to another database using a wireless connection, The view position and the view direction can be determined so that the object in the vicinity and the object in the vicinity can be positioned with respect to the user's field of view. At step 119030, an object of interest is identified with respect to the user's view by analyzing the database stored in the device 109100, or by wirelessly communicating with other devices in cooperation with the electronic device 109130 . In step 119040, the distance to the object of interest is determined by comparing the GPS position of the head mounted display device 109100 with the GPS position of the object of interest. In step 119050, a label on the name or other information about the object of interest is then generated with the mismatch to provide a 3D label to the distance perceived by the user corresponding to the distance to the object of interest. 111 shows an example of a label including name, distance, and description for an object of interest in the user's field of view. At step 119060, a 3D label for the object of interest is displayed in the left and right eyes of the user, with mismatches, to provide a 3D label at the desired depth.

120 is a flow chart for another depth cue method embodiment of the present disclosure, wherein steps similar to those in the steps of FIG. 119 are numbered using the same numbers used in FIG. In step 120140, the distance and direction to an object of interest is determined by the electronic device 109130 on the device, or in cooperation with other wirelessly connected devices, for the user's field of view. In step 120160, in order to provide a 3D label at a desired depth, the 3D label is displayed in the left and right eyes of the user along with the mismatch, and in addition, the 3D label is displayed in the field of view of the user corresponding to the direction of the object of interest Lt; / RTI &gt; FIG. 111 shows an example in which a label for an object of a distant interest is provided in the rearward direction of the user's field of view and in a direction away from the object of interest, in this example the label " " This feature provides a visual clue in 3D information that makes it easy for the user to move to an object of interest. Note that a 3D label can be provided in front of other objects in the field of view of the eye.

121 is a flowchart of another depth cue method embodiment of the present disclosure. In this embodiment, the distance from the scene to the object of interest is determined by a distance measuring device 109140, such as a distance measuring device. At step 121010, one or more images of a scene adjacent to the head mounted display device 109100 are captured using the stereoscopic camera 109120. [ As another alternative, a single camera may be used to capture one or more images of the scene. One or more images of the scene may be of different spectral image types (e.g., the image may be a visible light image, a low extraneous image, an infrared image, or a hyperspectral image). In step 121020, images or images are analyzed to identify one or more objects of interest, wherein the analysis may be performed by electronic device 109130, or the image may be wirelessly transmitted to another device for analysis have. In step 121030, the distance to the object of interest is determined using the distance measuring device 109140. At step 121040, a discrepancy is determined that correlates the distance of the object of interest. At step 121050, a label or other information for an object of interest is determined. At step 121060, a 3D label or other 3D information for the object of interest is displayed.

122 is a flowchart of another depth cue method embodiment of the present disclosure. In this embodiment, the distance to an object in the scene is directly measured using a stereoscopic camera to obtain a depth map of the scene. In step 122010, a stereoscopic camera 109120 is used to capture one or more sets of stereoscopic images of a scene adjacent to the on-board display device 109100. [ One or more sets of stereoscopic images of a scene may be of different spectral image types (e.g., the stereoscopic image may be a visible light image, a low extraneous image, an infrared image, or a hyperspectral image). At step 122020, a set or sets of stereoscopic images are analyzed to identify one or more objects of interest, wherein the analysis may be performed by the electronic device 109130, or a set of stereoscopic images or sets of stereoscopic images As shown in FIG. In step 122030, images in the set or sets of stereoscopic images are compared to determine an inconsistency for one or more objects of interest. In step 122040, a label or the like relating to one or more objects of interest is determined. At step 122050, 3D labels and / or 3D information for one or more objects of interest are displayed.

In embodiments, the present disclosure may provide for display content placement using camera focal length information (e.g., using an integrated camera operating in conjunction with an autofocus determination facility), wherein the real world objects Is extracted by the integrated processor from the autofocus determination facility, where the integrated processor determines the placement position for the content within the field of view of the optical assembly, based on the distance. The view may include two individually controllable views-each of which is aligned with one of the user's eyes so that the user can see the surrounding area and content with the two eyes, and the placement position for the content is two And an arrangement position for each individually controllable field of view. The content may include two independent images, where the two independent images must be individually placed in each of two independently controllable fields of view, wherein the two independent images are in two individually controllable fields of view A 3D image can be formed when displayed to the user. The placement position can be determined by extracting the placement value from the table of placement values corresponding to the distance to the real world object. The integrated processor can calculate the placement position.

In embodiments, the present disclosure may provide display content placement using distance meter information by a range finder that is, for example, integral with the eyepiece and that operates to determine the distance to the real world object in the surrounding environment , Where the integrated processor determines the placement position for the content within the field of view of the optical assembly, based on distance. The view may include two individually controllable views-each of which is aligned with one of the user's eyes so that the user can see the surrounding area and content with the two eyes, and the placement position for the content is two And an arrangement position for each individually controllable field of view. The content may include two independent images, where the two independent images must be individually placed in each of two independently controllable fields of view, wherein the two independent images are in two individually controllable fields of view A 3D image can be formed when displayed to the user. The placement position can be determined by extracting the placement value from the table of placement values corresponding to the distance to the real world object. The integrated processor can calculate the placement position.

In embodiments, the present disclosure is directed to a method and apparatus for displaying a content arrangement using a plurality of distance determination sensors, for example, through the use of a plurality of integral distance determination sensors, which operate to determine distances to real- Where the integrated processor determines the placement position for the content within the field of view of the optical assembly, based on distance. The view may include two individually controllable views-each of which is aligned with one of the user's eyes so that the user can see the surrounding area and content with the two eyes, and the placement position for the content is two And an arrangement position for each individually controllable field of view. The content may include two independent images, where the two independent images must be individually placed in each of two independently controllable fields of view, wherein the two independent images are in two individually controllable fields of view A 3D image can be formed when displayed to the user. The placement position can be determined by extracting the placement value from the table of placement values corresponding to the distance to the real world object. The integrated processor can calculate the placement position. In embodiments, the plurality of integral distance determination sensors may be a camera sensor, a distance meter, or the like.

In embodiments, the present disclosure provides a distance determination sensor and a plurality of integrated sensors (e. G., A plurality of integrated sensors &lt; RTI ID = 0.0 &gt; A user, for example, a camera, a distance meter), and eye tracking information from an eye tracking facility included in connection with the optical assembly of the eyepiece. In embodiments, the present disclosure provides a method and apparatus for locating and positioning an image in a user &apos; s peripheral field of view, using a calibration sequence, using a grating to assist in location and / or calibration, Other facilities related to the placement of the content within the field of view of the optical assembly, such as interlacing the image to the eye, may be utilized.

In embodiments, the present disclosure may provide display content control during movement of an eyepiece, for example, through an integrated motion detection facility configured to detect movement of a head mounted eyepiece when the user is wearing, wherein The integrated processor determines the type of motion and reduces the appearance of the displayed content based on the type of motion. The type of movement can be jitter, fast motion, etc. Reduction in appearance may be removal of the displayed content, reduction of the brightness of the displayed content, reduction of the contrast of the displayed content, change of the focus of the displayed content, and the like.

The NFC enables short-range wireless data exchange between the NFC reader and the passive NFC device, where the NFC reader functions as an initiator of communication (providing power for exchange) and the passive NFC device "Target" (receiving power from the RF field from the NFC reader and providing the data back to the reader). One example of this configuration may be an NFC reader device that reads identification information from a tag such as a clothing tag. It should be noted that NFC is also compatible with radio frequency identification (RFID) technology. The NFC wireless data exchange may also be bi-directional if the two electronic devices include an NFC reader and are very close to each other. An example of this configuration is that two NFC-enabled smartphones exchange information between them (e.g., exchange of electronic business cards), an NFC-enabled smartphone (e.g., for electronic money transfer as in a GOOGLE WALLET mobile payment system) Exchange information with NFC-enabled point-of-service (POS) devices, exchange information between two NFC-enabled mobile game devices, and the like. Applications of NFC technology include electronic money transfer, mobile payment, file sharing, electronic business card exchange, mobile game, social network connection, ticket purchase, boarding check in, POS, coupon collection and / Card, car or hotel key, and the like. NFC technology has an actual reach of about 4 cm (theoretically about 20 cm), so the initiator and the target must be in close proximity in order for the communication to take place.

In one example, a user can store credit card information on his NFC-enabled smartphone, and by bringing his smartphone in close proximity to the POS device (as implemented in the GOOGLE WALLET mobile payment system) Can make electronic cash payments to NFC-enabled POS devices in the network. In this way, the user does not need to take out a real credit card to make a transaction, because the credit card information is read from the smartphone by the POS device via the NFC connection. However, the user still suffers from taking his smartphone out of his pocket or purse, bringing it closer to the POS device, and then reinserting the smartphone.

The present disclosure provides a more convenient and convenient way to enable NFC-enabled wireless trading by providing the user with an NFC watch device such as that being worn on the user's wrist (which can always be conveniently located on other NFC-enabled devices for data exchange) Solution. Although the present disclosure describes embodiments of the NFC 'wristwatch', this is by no means limiting, and those skilled in the art will appreciate that alternatives for implementing the concepts of the present invention, such as those embodied as bracelets, You will be familiar with the implementation of. Embodiments of the NFC wristwatch may include a standalone NFC device, an NFC relay device, etc., wherein the NFC relay device may include an NFC-enabled device (e.g., an NFC-enabled POS device) as well as a second NFC- E.g., a user's smartphone). For example, in the case where the NFC wristwatch serves as a standalone NFC device, the NFC wristwatch may have information to be exchanged (e.g., credit card information). In the case where the NFC wristwatch is functioning as an NFC repeater, the wristwatch may not have the information to be exchanged, and rather the information to be exchanged may be transmitted to other electronic devices (smartphones, mobile computing devices, A personal computer, etc.).

In embodiments, if the NFC wristwatch serves as an NFC relay device, the user may leave his personal device (e.g., a smartphone) in his pocket or purse and simply connect the NFC wristwatch to another NFC-enabled device Where the NFC wristwatch provides communication between the other NFC-enabled device and the user's personal device. For example, a user could be putting an NFC watch on his wrist and put his smartphone with his credit card information in his pocket. When a user accesses an NFC-enabled POS device for electronic payment, the user can now leave his smartphone on and can simply bring his NFC wristwatch to the POS device, And communicates with the user's smartphone via a proximity communication link (e.g., Bluetooth, WiFi, etc.). The NFC wristwatch reads the user's credit card information from the smartphone and sends this data to the POS device. In this configuration, the user never needs to take his smartphone out because his or her NFC wristwatch can be used to make electronic payments while the user keeps all his personal and financial information centrally located on his smartphone. You just have to bring it close to the device.

207, the NFC watch 20702 includes a wrist watch front 20704 for displaying time and date 20708, a function button 20710, a built-in controller for the wristwatch function, and the like It can provide the normal function of a typical wristwatch. However, in addition, the NFC wristwatch can be used for long-range communications (e.g., Bluetooth) to nearby devices, for example, for short range communications to NFC-enabled devices, (E. G., WiFi). &Lt; / RTI &gt; In embodiments, an antenna 20712A-B for short range communication may be used as antenna 20712A in a watch band (e.g., with an NFC loop antenna) [e.g., an NFC 'stamp' antenna ) Antenna 20712B in the wristwatch main body, and may be provided as a guitar. In the case where antenna 20712A is located in the wristband, the user can bring the watchband portion of the NFC wristwatch closer to the NFC-supported device for data exchange during operation. When the antenna 20712B is located in the wristwatch main body, the user can bring the wristwatch main body portion of the NFC wristwatch close to the NFC supported device for data exchange during operation. In embodiments, the wrist watch display 20704 also includes a control interface 20718 that allows the user to enter and / or select information such as credit card number, verification code, data for transfer, etc. for use in an electronic money exchange, Where the control interface 20718 may include a display, a control button, a 2D control pad, a touch screen, and the like.

208, an exemplary usage scenario may include that the user 20802 is wearing an NFC watch 20702A that functions as a standalone NFC device, wherein the NFC communication link 20804A is used to pay for the purchase To the NFC-enabled POS device 20804. The NFC- In this case, the NFC watch 20702A includes payment information to be exchanged with the NFC-enabled POS device 20804. [ In embodiments, the information contained in the NFC watch 20702A may be communicated over a wired or wireless connection to a computing facility (e.g., a mobile computing device, a smart phone, a personal computer), via a control interface 20718, (E. G., A local network connection, a WiFi connection). &Lt; / RTI &gt;

209, an exemplary usage scenario uses an NFC watch 20702B functioning as an NFC relay device in which the user 20902 is in wireless communication 20908A with a smartphone 20908 in the user's pocket , Where information for data exchange is contained in the user's smartphone 20908. [ If you are not using an NFC wrist watch, you will have to take your smartphone out of your pocket and bring it close to an NFC-enabled POS device for data exchange. In accordance with the present invention, a user 20902 may leave the smartphone 20908 in his pocket and simply place the NFC watch 20702B on an NFC-enabled POS device 20804 where the NFC watch 20702B Communication between the smartphone 20908 and the NFC-enabled POS device 20804 via the two communication channels 20804A and 20908A set by the NFC-enabled POS device 20804 and thus by the NFC watch 20702B, It enables the transmission of information. In this configuration, the smartphone 20908 may not need to be NFC-capable because the only communication link required by the smartphone 20908 is an intermediate distance communication link 20908A (e.g., using Bluetooth or the like) For the NFC Wrist Watch 20702B.

In embodiments, the NFC wristwatch may be coupled to the NFC middle range communication link (s) through a non-NFC mid-range communication link (s), such as a personal computer, a mobile computer, a mobile communication device, a navigation device, Devices, home security devices, home automation devices, local network connections, personal network connections, and the like. For example, an NFC wristwatch can communicate with an eyepiece, such as an eyepiece, that includes an optical system that allows for perspective display in which content served from an integral processor can be displayed, wherein the side of the eyepiece is a sensor, , An accessory device, or the like. In embodiments, the NFC wristwatch can interface with the glasses to present transaction details to the glasses. The eyeglass control system can be used as an interface for any interaction required for the transaction.

In embodiments, the NFC wristwatch may include a microcontroller, a memory, an input / output device (e.g., a memory card, a wired connection) independent of the communication link, a computing resource such as a wireless connection to the local network for updating, programmability, Can be provided. For example, the NFC relay device may provide memory to store history of purchase, preferred merchandise, personal profile, sale offer, redemption code, preferred customer ID, reserve information, loyalty program data, and the like.

The methods and systems described herein, particularly various embodiments of the augmented reality eyepiece of the present invention, may be configured to communicate and receive communications via and / or via any electronic communication system or network. Examples of such electronic communication systems and network types and their associated protocols, topologies, network elements and the like include: (1) a wide area network (WAN) - point-to-point (PPP) a leased line using protocols such as high-level data link control, synchronous data link control (SDLC), and digital subscriber line; Circuit switching using protocols such as PPP and ISDN; Frame relay, X.25 (pre-OSI stack), packet over synchronous optical networking / synchronous hierarchy (SONET), multi-protocol-label-switching (MPLS), switched multi-megabit data service ), Packet switching using protocols such as Ethernet (e.g., 10 GB, 100 GB); A cell relay using a protocol such as ATM (asynchronous transfer mode) protocol; And network elements such as routers, switches, hubs, and firewalls; (b) Metropolitan Area Networking (MAN) using protocols such as ATM, fiber distributed network interface (FDDI), SMDS, Metro Ethernet, and distributed-queue dual-bus (DQDB); Using topologies such as star, bus, mesh, ring and tree; And network elements such as routers, switches, hubs, and firewalls; (c) a local area network (LAN) using high-speed serial interface protocols such as Ethernet (e.g., Ethernet, Fast, 1 GB, 10 GB, and 100 GB); Using topologies such as star and tree; And network elements such as routers, switches, hubs, and firewalls; (d) a wired network such as a personal area network (PAN) using technologies such as USB and FireWire; (2) (a) RTT (CDMA), EDGE (GSM), EV-DO (CDMA / TDMA), Flash-OFDM, GPRS, HSPA D and UMTS / 3GSM, LTE Wide area network (WAN) using standards such as 3GPP, UMTS-TDD (UMTS / 3GSM), WIMAX (802.16), satellite, mobile Internet (such as general 3G and general 4G); Using network elements such as base station subsystems, networks and switching subsystems, GPRS core networks; UMTS, TD-SCDMA-UMTS, UMTS, UTRAN-FDD (Universal Mobile Telecommunication System) The equipment interface includes a network layer such as UMTS, radio resource control (radio link control, medium access control), Um interface (physical layer such as GMS or 8PS modulation, data link layer such as LAPDm and radio resources, mobility management, &Lt; / RTI &gt; (b) a Metropolitan Area Network (MAN) using a protocol such as WIMAX (802.16); LAN (Local Area Network) using technologies such as Wi-Fi with modes such as Ad-hoc and Infrastructure, OSI layers such as SCMA / CA, and sub technologies such as OFDM and spread spectrum; And network elements such as routers, switches, hubs, firewalls, access points, base stations, and clients (such as personal computers, laptop computers, Internet protocol telephones, cellular phones and smart phones); (c) a topology such as star, tree, and mesh, and (i) Bluetooth (e.g., using a master, a slave, and a concurrent master / slave) (Legacy pairing and SSPs), a pairing method (a legacy pairing and an SSP), a legacy pairing and an SSP (Eg, Simple Secure Pairing), air interface [license-free ISM band (2.402-2.480 GHz)]], (ii) IrDA (Infrared Data Association (E.g., Tiny Transport Protocol (Tiny TP), Infrared Communications Protocol (IrCOMM), Object Exchange (OBEX), etc.) , (Iii) a wireless USB, (iv) a Z-Wave (e.g., a source-routed mesh network topology, one or more master controller controls Routing and security, and FGS modulation; (v) ZigBee [eg, the physical and media access control layers defined in 802.15.4 and network layers, application layers, Zigbee device objects, A personal area network (PAN) using technologies such as a BAN (Body Area Network), and (vii) Wi-Fi; (3) a short range, such as operating with a peer-to-peer network type at 13.56 MHz, with an ISO / IEC 18000-3 air interface and a passive and / or active communication mode with a data rate of 106 Kbits / s to 424 kbits / Communication (NFC). The methods and systems described herein (and in particular, various embodiments of the augmented reality eyepiece of the present invention) can be used in a wide range of applications including policy management, user management, profile management, business intelligence, incident management, performance management, enterprise- , Multi-platform supported mobile device management (including sub-aspects such as software and SaaS), security management [certificate control (eg, related to email, application, Wi-Fi access, and VPN access) Device locking, device wiping, remote locking, trail auditing / logging, centralized device configuration verification, jailbreak / rooted detection, secure container, and application wrapping), platform support (eg, Android, iOS, Blackberry, Symbian, Windows mobile, and Windows phone) liance management, software management [application downloader, application verification, application update support, application patch support, application store support (e.g., enterprise applications and third party applications)], and hardware / Some aspects of the mobile device network management system, such as device registration (e.g., ownership, staging, registration, user authentication, EULA development and restriction development), external memory blocking, And may be configured to match all of them. The methods and systems described herein (particularly various embodiments of the augmented reality eyepiece of the present invention) may be implemented in software as a service (SaaS), platform as a service (PaaS), and / or infrastructure as a service Communities, or hybrid cloud computing networks or in a cloud computing environment, including, but not limited to, &lt; RTI ID = 0.0 &gt;

The methods and systems described herein may be distributed, in part or in whole, through a machine executing computer software, program code, and / or instructions on the processor. The processor may be part of a server, a cloud server, a client, a network infrastructure, a mobile computing platform, a fixed computing platform, or other computing platform. A processor may be any type of computing or processing device capable of executing program instructions, code, binary instructions, and so on. A processor may be a single processor, a digital processor, a microprocessor or a coprocessor (such as a mathematical coprocessor, a graphics coprocessor, a communications coprocessor, etc.) that can facilitate the execution of program code or program instructions stored thereon, either directly or indirectly. And may include any variations thereof. In addition, the processor may execute multiple programs, threads, and code. Threads can be executed concurrently to improve the performance of the processor and to facilitate simultaneous operations of the application. As an implementation, the methods, program code, program instructions, and so on described herein may be implemented in one or more threads. A thread may spawn another thread that may have assigned a priority associated with it; The processor may execute these threads based on priority or on any other order based on the instructions provided in the program code. A processor may include a memory that stores methods, codes, instructions, and programs as described herein and elsewhere. The processor may be accessed via an interface to a storage medium capable of storing methods, codes, and instructions, as described herein and elsewhere. A storage medium in connection with a processor that stores a method, program, code, program command or other type of instruction that may be executed by a computing or processing device includes, but is not limited to, a CD-ROM, DVD, memory, hard disk, flash drive, , Cache, and the like, but is not limited thereto.

A processor may include one or more cores capable of improving the speed and performance of the multiprocessor. In embodiments, the processor may be a dual core processor, quad core processor, other chip level multiprocessor, etc., which may have two or more independent cores (referred to as D).

The methods and systems described herein may be distributed, in part or in whole, through a server, client, firewall, gateway, hub, router, or other such computer and / or machine running computer software on such networking hardware. A software program may be associated with a server that may include a file server, a print server, a domain server, an Internet server, an intranet server, and other variations (secondary server, host server, distributed server, etc.). The server may include one or more of memory, a processor, a computer readable medium, a storage medium, a port (physical and virtual), a communication device, and an interface capable of accessing other servers, clients, machines, and devices via a wired or wireless medium, . The methods, programs, or codes described herein and elsewhere may be executed by the server. In addition, other devices required to implement the method as described in this application may be considered as part of the infrastructure associated with the server.

The server may provide an interface to other devices, including but not limited to clients, other servers, printers, database servers, print servers, file servers, communication servers, distributed servers, social networks, In addition, such associations and / or connections may facilitate remote execution of programs over the network. Networking of some or all of these devices may facilitate parallel processing of programs or methods at one or more locations without departing from the scope of the present disclosure. In addition, any of the devices connected to the server via the interface may include at least one storage medium capable of storing methods, programs, code, and / or instructions. The central repository may provide program instructions to be executed on different devices. In this implementation, the remote repository may function as a storage medium for program code, instructions and programs.

A software program may be associated with a client that may include a file client, a print client, a domain client, an Internet client, an intranet client, and other variations (ancillary clients, host clients, distributed clients, A client may include one or more of memory, a processor, a computer readable medium, a storage medium, a port (physical and virtual), a communication device, and an interface capable of accessing other clients, servers, machines, and devices via a wired or wireless medium, . The methods, programs, or codes described herein and elsewhere may be executed by a client. In addition, other devices needed to implement the method as described in this application may be considered as part of the infrastructure associated with the client.

The client may provide an interface to other devices, including but not limited to a server, a cloud server, another client, a printer, a database server, a print server, a file server, a communication server, a distributed server, In addition, such associations and / or connections may facilitate remote execution of programs over the network. Networking of some or all of these devices may facilitate parallel processing of programs or methods at one or more locations without departing from the scope of the present disclosure. In addition, any of the devices connected to the client via the interface may include at least one storage medium capable of storing methods, programs, applications, code, and / or instructions. The central repository may provide program instructions to be executed on different devices. In this implementation, the remote repository may function as a storage medium for program code, instructions and programs.

The methods and systems described herein may be deployed, in part or in whole, over a network infrastructure. Network infrastructure may include elements such as computing devices, servers, cloud servers, routers, hubs, firewalls, clients, personal computers, communication devices, routing devices, and other active and passive devices, modules and / . The computing and non-computing device (s) associated with the network infrastructure may include storage media such as flash memory, buffer, stack, RAM, ROM, etc., in addition to other components. The processes, methods, program codes, and instructions described herein and elsewhere may be executed by one or more of the network infrastructure elements.

The methods, program codes, and instructions described herein and elsewhere can be implemented in a cellular network having multiple cells. The cellular network may be a frequency division multiple access (FDMA) network or a code division multiple access (CDMA) network. A cellular network may include a mobile device, a cell site, a base station, a repeater, an antenna, a tower, and the like. The cell network may be GSM, GPRS, 3G, EVDO, mesh or other network type.

The methods, program codes, and instructions described herein and elsewhere may be implemented on or through a mobile device. The mobile device may include a navigation device, a cell phone, a mobile phone, a mobile personal digital assistant (PDA), a laptop, a palmtop, a netbook, a pager, an electronic books reader, These devices may include, in addition to other components, flash memory, a buffer, a storage medium such as RAM, ROM, and the like, and one or more computing devices. The computing device associated with the mobile device may execute program code, methods, and instructions stored thereon. As another alternative, the mobile device may be configured to execute instructions in cooperation with other devices. The mobile device may communicate with a base station that interfaces with the server and is configured to execute program code. The mobile device may communicate via a peer-to-peer network, a mesh network, or other communications network. The program code may be stored on a storage medium associated with the server and executed by a computing device embedded within the server. The base station may include a computing device and a storage medium. The storage device may store program code and instructions that are executed by a computing device associated with the base station.

Computer software, program code, and / or instructions are computer components, devices, and recording media that retain digital data used to compute for some period of time; A semiconductor storage device called a random access memory (RAM); Mass storage devices typically for more permanent storage such as various types of magnetic storage devices such as optical disks, hard disks, tapes, drums, cards and other types; Processor registers, cache memory, volatile memory, non-volatile memory; Optical storage devices such as CD and DVD; Removable media such as flash memory (e.g., a USB stick or key), a floppy disk, a magnetic tape, a paper tape, a punch card, a stand alone RAM disk, a Zip drive, a removable mass storage device, Dynamic memory, static memory, read / write storage, mutable storage, read only, random access, sequential access, location addressable, file addressable, content addressable content addressable, network-attached storage devices, storage area networks, other computer memory such as bar codes, magnetic ink, and the like, and / or may be accessed on machine readable media.

The methods and systems described herein may convert physical and / or intangible items from one state to another. The methods and systems described herein may also convert data representing physical and / or intangible items from one state to another.

The elements described and illustrated throughout this specification, including flow charts and block diagrams, refer to the logical boundaries between elements. However, in accordance with software or hardware engineering practice, it is contemplated that the illustrated elements and their program instructions stored therein may be used as a monolithic software structure, as a standalone software module, or as a module using external routines, code, services, On a machine having a processor that can be executed as any combination of these, and all such implementations may fall within the scope of the present disclosure. Examples of such machines are personal digital assistants (PDAs), laptops, personal computers, mobile phones, other handheld computing devices, medical equipment, wired or wireless communication devices, transducers, chips, calculators, , Electronic devices, devices having artificial intelligence, computing devices, networking equipment, servers, routers, and the like. In addition, elements shown in the flowchart and block diagrams or any other logical component may be implemented on a machine capable of executing program instructions. Thus, although the drawings and description above describe functional aspects of the disclosed system, specific configurations of software implementing these functional aspects should not be inferred from these descriptions, unless explicitly stated otherwise or clear from the context. Similarly, it will be appreciated that the various steps described above may be varied and that the order of the steps may be adapted to the particular application of the techniques described herein. All such variations and modifications are to be regarded as being within the scope of this disclosure. Accordingly, it is to be understood that the order and / or description of the sequences for the various steps may be combined and / or followed by a particular execution of the steps, unless explicitly required by the specific application, They should not be construed as requiring an order.

The methods and / or processes described above and their steps may be implemented in hardware, software, or any combination of hardware and software appropriate for the particular application. The hardware may comprise a general purpose computer and / or a particular computing device or a particular computing device or a particular aspect or component of a particular computing device. The process may be implemented in one or more microprocessors, microcontrollers, embedded microcontrollers, programmable digital signal processors, or other programmable devices, with internal and / or external memory. The process may, in addition or alternatively, be implemented in an application specific integrated circuit (ASIC), a programmable gate array, programmable array logic, or any other device or combination of devices that may be configured to process electronic devices . It will also be appreciated that one or more of the processes may be implemented as computer-executable code that may be executed on a machine-readable medium.

The computer executable code may be stored, compiled, or interpreted to execute on any of the above devices as well as a processor, processor architecture, or a combination of different hardware and software, or any other machine capable of executing program instructions (C ++, etc.), or any other high-level or low-level programming language (including assembly language, hardware description language, and database programming language and techniques) .

As such, in one aspect, each of the methods and combinations described above may be implemented with computer executable code that performs the steps thereof when executed on one or more computing devices. In another aspect, a method may be implemented in a system that implements the steps of the method, distributed across devices in a number of ways, or all functions may be integrated into a dedicated standalone device or other hardware. In other aspects, the means for performing the steps associated with the processes described above may include any of the hardware and / or software described above. All such substitutions and combinations are to be regarded as being within the scope of this disclosure.

While this disclosure has been disclosed in terms of the preferred embodiments shown and described in detail, various modifications and improvements thereto will be readily apparent to those skilled in the art. Accordingly, the spirit and scope of the present disclosure should not be limited by the above examples, but should be accorded the broadest interpretation allowed to do so.

All documents referred to herein are incorporated herein by reference.

Claims (10)

As a system,
An interactive head-mounted eyepiece worn by a user, the eyepiece comprising an optical assembly in which a user views the surrounding environment and the displayed content,
An integrated processor for processing content for display to a user, and
And an integrated image source for introducing the content into the optical assembly,
Wherein the processor is configured to modify the content, wherein the modification is made in response to a sensor input. 2. The system of claim 1, wherein the content is a video image. 3. The method of claim 2, wherein the modification includes adjusting brightness, adjusting color saturation, adjusting a color balance, adjusting a color hue, Adjusting the resolution, adjusting the transparency, adjusting the compression ratio, adjusting the frame rate per second, isolating a portion of the video, stopping the playback of the video, pausing the video, or Wherein the video is at least one of restarting the video. The method of claim 1, wherein the sensor input is selected from the group consisting of a charge coupled device, a black silicon sensor, an IR sensor, an acoustic sensor, an induction sensor, a motion sensor, an optical sensor, an opacity sensor, An inductive sensor, an eddy current sensor, a passive infrared proximity sensor, a radar, a capacitive displacement sensor, a hall-effect sensor, a magnetic sensor, a GPS sensor, a thermal imaging sensor, a thermocouple, an ultrasonic 3D motion sensor, an accelerometer, an inclinometer, a force sensor, a piezoelectric sensor, a rotary encoder, a linear encoder, a thermistor, a photoelectric sensor, an ultrasonic sensor, an infrared laser sensor, A linear encoder, a chemical sensor, an ozone sensor, a smoke sensor, a thermal sensor, a magnetometer, a carbon dioxide detector, a carbon monoxide detector, an oxygen sensor, a glucose sensor, A rain sensor, an altimeter, an activity sensor, an object detector, a marker detector, a laser range finder, a sonar, a capacitive sensor, a heart rate sensor, RF / MIR (micropower impulse radio) sensor. 4. The system of claim 3, wherein the content is discontinued in response to an indication from an accelerometer input that the user's head is moving. 5. The system of claim 4, wherein the audio sensor input is generated by at least one participant in the video conference speaking. 5. The system of claim 4, wherein the visual sensor input is a video image of at least one participant of the videoconference. 5. The system of claim 4, wherein the visual sensor input is a video presentation of a visual presentation. 8. The system of claim 7, wherein the modification is at least one of making the video image more transparent or less transparent in response to an indication from a sensor that the user is moving. As a system,
An interactive head-mounted eyepiece worn by a user, the eyepiece comprising an optical assembly in which a user views the surrounding environment and displayed content,
An integrated processor for processing content for display to a user, and
And an integrated image light source for introducing the content into the optical assembly,
Wherein the processor is configured to modify the content, the modification being made in response to a sensor input,
Further comprising an integrated video image capture facility that provides the content for recording and displaying a view of the environment.

Images