GB2551841A

GB2551841A - Glasses for use in surgical procedures

Info

Publication number: GB2551841A
Application number: GB1611556.0A
Authority: GB
Inventors: Aebi Max; Schneider Marcel; Schaerer Anton; Grossenbacher Jonas; Thomas Andrews Geoffrey
Original assignee: Bern Univ Of Applied Sciences Engineering And Technology (bfh-Ti); Ecce Medical Ltd
Current assignee: Bern Univ Of Applied Sciences Engineering And Technology (bfh-Ti); Ecce Medical Ltd
Priority date: 2016-07-01
Filing date: 2016-07-01
Publication date: 2018-01-03
Also published as: WO2018002674A1; GB201611556D0

Abstract

A pair of glasses 10 for use during surgery, in particular for recording or transmitting the images captured by the glasses. The glasses comprise a frame 12, a pair oflenses 14 held in the frame, and a camera 16 mounted to the frame, and a processor coupled to the camera. The camera is for viewing or recording a surgical procedure being performed. The camera has an adjustable field of view (FOV) in the same direction as a wearers FOV enabling the correct viewing angle to be recorded or viewed by the glasses during the procedure. The FOV may be adjusted mechanically, using a swiveling mount 18 (either by hand or using motors), or digitally, where the image is processed by the processor to view or record the correct FOV of a larger image. The mount may swivel about a vertical or horizontal axis adjusting the corresponding FOV. The mount may be coupled to an actuator such as a rod and a motor to provide the swiveling movement. The camera may be positioned centrally between the lenses on the frame. A second camera also with an adjustable FOV may be employed to produce 3D images.

Description

(71) Applicant(s):

Ecce Medical Limited

Compass House, Vision Park, Histon, Cambridge, CB24 9AD, United Kingdom

Jonas Grossenbacher

Lohhiisli 107a, Wyssachen 4954, Switzerland

1611556.0 (51) [_NT ci_:

G02B 27/01 (2006.01)

01.07.2016 A61B 90/50 (2016.01)

A61B 90/30 (2016.01)

(56) Documents Cited: GB 2496064 A	EP 2437099 A2
DE 019853912A1	US 20160286115	A1
US 20150084850 A1	US 20110193963	A1
(58) Field of Search: INT CLA61B, G02B Other: EPODOC & WPI

Bern University of Applied Sciences, Engineering and Technology (BFH-TI)

Quellgasse 11, Biel 2501, Switzerland (72) Inventor(s):

Max Aebi Marcel Schneider Anton Schaerer Jonas Grossenbacher Geoffrey Thomas Andrews (74) Agent and/or Address for Service:

Marks & Clerk LLP

62/68 Hills Road, CAMBRIDGE, CB2 1LA, United Kingdom (54) Title of the Invention: Glasses for use in surgical procedures

Abstract Title: A pair of glasses mounted with a camera having an adjustable field of view.

(57) A pair of glasses 10 for use during surgery, in particular for recording or transmitting the images captured by the glasses. The glasses comprise a frame 12, a pair oflenses 14 held in the frame, and a camera 16 mounted to the frame, and a processor coupled to the camera. The camera is for viewing or recording a surgical procedure being performed. The camera has an adjustable field of view (FOV) in the same direction as a wearer’s FOV enabling the correct viewing angle to be recorded or viewed by the glasses during the procedure. The FOV may be adjusted mechanically, using a swiveling mount 18 (either by hand or using motors), or digitally, where the image is processed by the processor to view or record the correct FOV of a larger image. The mount may swivel about a vertical or horizontal axis adjusting the corresponding FOV. The mount may be coupled to an actuator such as a rod and a motor to provide the swiveling movement. The camera may be positioned centrally between the lenses on the frame. A second camera also with an adjustable FOV may be employed to produce 3D images.

Figure 1

At least one drawing originally filed was informal and the print reproduced here is taken from a later filed formal copy.

1/3

10 17

Figure 2

2/3

10 17

Figure 4

3/3

10 17

Figure 5

Glasses For Use In Surgical Procedures

FIELD OF THE INVENTION

The present invention relates to a pair of glasses for use during a surgical procedure, in particular, to a pair of glasses for use during surgical procedures that may record or transmit the images captured by the glasses.

BACKGROUNG OF THE INVENTION

Professional video-production of surgical interventions is expensive. Image sections are often covered by hands or tools and there is little or no opportunity to interact with the surgeon.

Known solutions to this include the use of video glasses, for example using the now discontinued Google (RTM) Glass, which comprised a standard definition camera integrated into the side arm of a frame of a pair of glasses. Other solutions involve clipping or arranging camera modules onto a pair of surgical glasses to capture images during a surgical procedure.

However, these solutions are not without their problems. Often there are angular offsets with regards to the field of view of the camera in relation to what the surgeon is seeing, and the glasses can become bulky and heavy, which are not conducive to an optimal environment in which the surgeon can work.

We have therefore appreciated the need for an improved pair of glasses for use in surgical procedures.

SUMMARY OF THE INVENTION

The present invention therefore provides a pair of glasses for use during a surgical procedure, the glasses comprising: a frame; a pair of lenses held in the frame; and a camera mounted to the frame for viewing or recording a surgical procedure being performed, the camera having a field of view in the same direction as a field of view of the wearer of the glasses; and a processor coupled to the camera for processing images received from the camera, wherein the field of view of the camera is adjustable.

Advantageously, these glasses enable the correct view to be recorded and/or transmitted when the user wears them during a procedure such as a surgical procedure.

The camera may be mounted to the frame via a swivelable mount, the swivelable mount being mounted to the frame such that it is swivelable about an axis to adjust the field of view of the camera. The axis may arranged such that swivelling the mount about the axis causes the field of view to be adjusted vertically. Alternatively, or additionally, the axis may be arranged such that swivelling the mount about the axis causes the field of view to be adjusted horizontally.

The mount may be coupled to an actuator to effect the swivel movement of the mount. The actuator may comprise a rod coupled to the mount, wherein rotation of the rod about the axis swivels the mount. Alternatively, the actuator may comprise a motor for swivelling the mount about the axis.

In an alternative embodiment, the camera comprises an angular field of view that is greater than required for viewing or recording, and wherein the processor is configured to adjust the field of view by processing the image generated by the camera to generate an image having the required field of view.

In any of the above embodiments, the camera may be positioned centrally between the lenses on the frame.

The glasses may comprise a second camera, and wherein the viewing or recording of a surgical procedure is done in 3D using the images output by the first and second cameras. Each of the cameras may have an adjustable field of view. Each of the cameras may be located adjacent respective lenses in order to form the 3D image.

Any of the above glasses may also comprise one or more controllable light sources, where the controllable light source may be configured to illuminate a portion of the field of view of the camera to identify an area central to the field of view, or to identify one or more boundaries of the field of view of the camera. Alternatively, or additionally, the controllable light source may be configured to illuminate an object or scene being viewed by the wearer of the glasses.

The glasses may also comprise one or more filters in the optical path of the camera for adjusting the intensity of light from the subject being viewed or recorded by the camera. The filter may be adjustable.

An image generated by the camera may be recorded on a memory device and/or transmitted to a remote receiver.

The glasses may also comprise a microphone for receiving sound in the vicinity of the glasses, the microphone being coupled to the processor and the processor being configured to receive a microphone signal from the microphone. The microphone signal is recorded on a memory device and/or transmitted to a remote receiver.

The glasses may also comprise a speaker adjacent at least one of the user’s ears, the speaker being coupled to the processor, and being configured to receive a signal from the processor and to output sound. The speaker may comprise a bone conduction speaker using a portion of the frame in contact with the user’s head.

The glasses may also a power source, which may be located remote from the frame, connected to the glasses via a cable. The enables the weight of the glasses to be reduced.

The processor may be located remote from the frame, and is connected to the glasses via a cable. This enables the weight of the glasses to be reduced.

The lenses may be removable from the frame. This enables a variety of lenses to be catered for the user.

The glasses, not including any separate battery pack and processor (if located remote from the glasses) weigh less than 10Og, for example 60g. More preferably they weigh less than 50g.

LIST OF FIGURES

The invention will now be described, by example only, with reference to the accompanying figures, in which:

Figure 1 shows a pair of glasses according to the invention;

Figure 2 shows the glasses of figure 1, with portions of the frame removed to show the adjustment mechanism;

Figure 3 shows an embodiment of the adjustment mechanism;

Figure 4 shows a high level overview of the glasses circuit diagram; and

Figure 5 shows an example image captured using the glasses.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In brief, the present invention provides a pair of glasses that may be used during surgical procedures for viewing or recording a surgical procedure being performed. The glasses comprise a frame, a pair of lenses held in the frame, and a camera mounted to the frame, and a processor coupled to the camera. The camera is for viewing or recording a surgical procedure being performed. The camera has a field of view in the same direction as a field of view of the wearer of the glasses. The field of view of the camera is adjustable, which enabled to the correct viewing angle to be recorded or viewed by the glasses during the procedure. The field of view may be adjusted mechanically, using a moveable mount (either by hand or using motors), or digitally, where the image is processed by the processor to view or record the correct field of view of a larger image.

Figure 1 to 3 show various aspects of the mechanical arrangement of an embodiment of the glasses.

The glasses 10 comprise a frame 12, a pair of lenses 14 held in the frame 12, and a camera 16. The camera is mounted to the frame 12 for viewing or recording a surgical procedures being performed. As can be seen, the camera faces forwards of the frame, that is the camera has a field of view in the same direction as a field of view of the wearer of the glasses. The camera 16 is mounted approximately centrally within the frame 12, that is approximately centrally between the two lenses 14, to provide a view of the camera that is as close as possible to what the wearer of the glasses (for example a surgeon) sees whilst wearing the glasses.

The field of view of the camera 16 is adjustable, that is the field of view of the camera 16 is adjustable in a vertical axis and/or a horizontal axis, to enable the correct view of the camera to set such that the images being recorded or viewed are set appropriately for the intended use.

In the embodiment shown in the figures, the camera 16 is mounted to the frame 12 via a swivelable mount 18. The swivelable mount 18 is mounted to the frame such that it is swivelable about an axis (horizontal and/or vertical) to adjust the field of view of the camera 16.

In the embodiment shown, the mount 18 is coupled to an actuator, or adjuster 20 to effect appropriate movement of the mount 18 to obtain the desired field of view. In this embodiment, the actuator is a rod or shaft 24 coupled to the adjuster 20, where rotation of the adjuster 20 about an axis extending longitudinally through the shaft 24 causes the mount 18 to rotate about the same axis. In this case, the movement of the mount 18 would cause the camera to be angled, and thus the field of view (since the angular field of view is fixed) to move in the vertical plane.

Of course, movement in the vertical plane is not the only plane of possible movement. If the mount 18 were to be mounted such that is also rotated about an axis that was perpendicular to the vertical plane, and an appropriate adjuster coupled to the mount 18, then movement of the field of view of the camera 16 in the horizontal plane would also be possible, that is adjustment of the field of view in the left-right horizontal plane.

In the embodiment shown, the shaft 24 is held between friction bearings 26 and a spring 22 applies pressure to the assembly in order to prevent the mount 18 from moving freely. A force sufficient to overcome the friction is required in order to effect movement of the mount 18.

Instead of the user manually adjusting the position of the mount using adjusters 20, movement may be effected using motors coupled to the mount 18. In such an embodiment, the motors may be activated or controlled in response to signals provided by the user, for example by eye movement detection, a switch, button or pedal or other kinds of actuators.

In a second embodiment, the camera may have a much wider angular field of view than that required for the use. In this situation, it may be envisaged that instead of adjusting the physical position of the camera to effect a desired change or adjustment in the field of view of the image, the image itself may be processed in order to provide the desired field of view.

A processor 36 may be coupled to the camera (as will be described in more detail later when discussing the circuit diagram). In this case, the processor may process the image output by the camera to reduce and/or adjust the field of view of the image generated by the camera. The desired field of view, and position of that field of view within the camera’s field of view is selected by the user, and the processor outputs an image that is as required.

There is also a third embodiment of the glasses, in which there are two cameras, preferably with each camera being positioned adjacent (for example above) each of the lenses). In this embodiment, each of the cameras is as described above, and mounted, or connected to a processor, as described above with reference to the first and second embodiments. The purpose of the two camera arrangement is to provide the user with a 3D image output, which also can be used in a virtual reality viewer.

In any of the above embodiments, there may be provided controllable light sources 28. These serve multiple purposes. Firstly, there may be provided light sources 28 for illuminating an area desired by the user. Secondly, the light sources 28 may be used to indicate the location of a particular area or spot such that the user of the glasses may identify the centre of the image being viewed by the camera, or the boundaries of the field of view of the camera.

The different purposes of the light source 28 may be performed by the same light source 28, or there may be a plurality of light sources 28 performing either or both functions.

In some embodiments, where the light source 28 is used to indicate the field of view or position of the camera, the light source may be a laser light source. The laser preferably has a power below 1mW.

In order to combat the very bright environment of the surgical theatre, the glasses may be provided with one or more filters in the optical path to the camera 16. These may be fixed filters, for example neutral density filters, in order to reduce the amount of light reaching the camera 16 (for example to prevent the images appearing over-exposed). The filters, on the other hand, may be graduated filters, or variable filters or controllable filters.

The glasses 10 may also be provided with a microphone 48, for recording sounds in the proximity of the wearer of the glasses.

Likewise, the glasses 10 may also be provided with a speaker arrangement such that the wearer of the glasses may receive instruction from a recording or other person, or listen to other audio such as music. This may be provided via a speaker, or using bone conduction, where the music is channelled into the head of the wearer via contact of the frame 12 with the user’s head.

The glasses may also be configured to allow surgeon’s loupes (magnifying lenses fitted to the front of each lens) to be attached to the front of each glass lens. Preferably, the lenses are removable and interchangeable to suit the needs of the wearer.

We will now move on to discuss the circuit diagram of the glasses, which is shown as a high level in figure 4. Like features have the same reference numeral as above.

In order to keep the weight of the glasses 10 down, the power source 44 and the processor 36 and other components are housed in a housing 32 which is remote from the glasses 10, but connected via cabling.

The glasses 10, as well has comprising the camera 16, microphone 48 and speaker (not shown), and light source 28 also comprises a number of indicators 34 (for example LED lights) that may be used to indicate a status of the glasses. For example, the indicators 34 may comprise a record indicator to indicate that the camera is recording or an error indicator to indicate that the glasses may be in an error state. The light source is controlled via the switch 46, which the wearer my use in order to activate or deactivate the light source 28 as required. A record or start transmitting button may also be provided on the glasses to enable the camera to begin recording or transmitting the viewed image.

An audio/video module 40 connects the glasses 10 (some of which are preferably housed within a housing 30 of the glasses) with the circuits in the separate housing 32, and enables video and audio signals to be passed back and forth between the modules. Preferably the audio/video module 40 is a HDMI extension module, but the invention is not limited to this type of module.

In terms of the separate housing 32, this is used to house the heavier components, such as the power source 44 (for example a rechargeable battery), a sound module 38, for recording audio from the glasses microphone, or generating audio from an audio signal. Indicators 34 are also present in order to display pertinent status information to the user regarding the status of the various modules and the power source. For example, the indicators 34 may indicate that the module is in a ready state, that the image is being recorded or transmitted, or that the module is connected or disconnected from a wireless network. Furthermore, the separate housing 32 houses a memory card module (not shown) for enabling the video and/or audio signals generated by the glasses 10 to be stored in memory for later retrieval. The audio and video may be stored and compressed using known techniques.

Furthermore, the system may be configured to stream the video and/or audio captured by the camera 16 and microphone 48 live or in a delayed fashion to an external receiver (that is, external to the glasses and separate housing; this could be in the same room, or another room in the same building, or another place elsewhere in the world). The system may connect to the internet via a Bluetooth (RTM) or WiFi network and transmit the information to a different receiver. For example, students may watch a live or recorded operation via the glasses.

During live streaming of the audio and/or video, the system may be configured to enable remote spectators to participate via the speaker on the glasses 10. That is, the remote spectator may be able to communicate with the wearer of the glasses.

Preferably, the glasses weigh less than 10Og, more preferably less than 50g. This is achievable because the heavier components are held in the separate housing 32.

The processor may be, for example, a low power processor such as a Raspberry Pi 3. The third version of the open source on board computer was launched in February 2016 and is supplied with WLAN and Bluetooth on board.

The camera 16, which may for example be the Raspberry Pi spy camera, preferably supports Full-HD recordings.

The operating system for the Raspberry Pi is Raspian Jessie Lite. Bash scripts, which allow to execute command line instructions, are used for control the Raspberry. More complex controls may be programmed with C++ or Python.

Figure 5 shows an example image as recorded by a prototype pair of glasses according to the invention.

Surgical methods

Whilst the glasses may be used in a variety of situations, the primary focus of the development of the glasses has been to enable surgeons to record and/or transmit images of surgical procedures being performed from their perspective, without intruding on the surgeon’s environment.

As such, we envisage that the invention would also comprise a method of performing a surgical procedure, where the user would wear a pair of the glasses as described above, begin recording and/or transmitting with the glasses, and perform the procedure. The user may be the surgeon, or another person in the surgical environment. The user may turn the recording and/or transmitting off after the procedure.

During the procedure, if there is a microphone, the audio will also be recorded and/or transmitted along with the video signal. In the case where there is a speaker or bone conducting mechanism to enable the user to hear audio received by the glasses, the user may receive instructions from another party, whether that be a previous recording (for example a set of instructions previously recorded, or music or other audio file). In the situation where the audio is being received from a live source, the source could be in the same room or another room, another building, or even another country and connected via the internet. The speaker and microphone would enable to user to interact live with someone else during the procedure.

Prior to performing the procedure, the user may set the field of view of the camera, for example using the methods described above with reference to the swivelable mount, or the image processing.

The user may use the light source to identify the position of the camera’s view to set the correct field of view. Alternatively (or additionally), the user may use a live feed from the camera (that is, a live signal from the camera fed into a portable device or laptop or computer of TV or other type of viewing device) to adjust the field of view of the camera.

No doubt many other effective alternatives will occur to the skilled person. It will be understood that the invention is not limited to the described embodiments and encompasses modifications apparent to those skilled in the art lying within the scope of the claims appended hereto.

Claims

CLAIMS:

1. A pair of glasses for use during a surgical procedure, the glasses comprising: a frame;

a pair of lenses held in the frame; and a camera mounted to the frame for viewing or recording a surgical procedure being performed, the camera having a field of view in the same direction as a field of view of the wearer of the glasses; and a processor coupled to the camera for processing images received from the camera, wherein the field of view of the camera is adjustable.

2. A pair of glasses according to claim 1, wherein the camera is mounted to the frame via a swivelable mount, the swivelable mount being mounted to the frame such that it is swivelable about an axis to adjust the field of view of the camera.

3. A pair of glasses according to claim 2, wherein the axis is arranged such that swivelling the mount about the axis causes the field of view to be adjusted vertically.

4. A pair of glasses according to claim 2 or 3, wherein the axis is arranged such that swivelling the mount about the axis causes the field of view to be adjusted horizontally.

5. A pair of glasses according to any one of claims 2 to 4, wherein the mount is coupled to an actuator to effect the swivel movement of the mount.

6. A pair of glasses according to claim 5, wherein the actuator comprises a rod coupled to the mount, wherein rotation of the rod about the axis swivels the mount.

7. A pair of glasses according to claim 5, wherein the actuator comprises a motor for swivelling the mount about the axis.

8. A pair of glasses according to claim 1, wherein the camera comprises an angular field of view that is greater than required for viewing or recording, and wherein the processor is configured to adjust the field of view by processing the image generated by the camera to generate an image having the required field of view.

9. A pair of glasses according to any preceding claim, wherein the camera is positioned centrally between the lenses on the frame.

10. A pair of glasses according to any one of claims 1 to 8, comprising a second camera, and wherein the viewing or recording of a surgical procedure is done in 3D using the images output by the first and second cameras.

11. A pair of glasses according to claim 10, wherein each of the cameras has an adjustable field of view.

12. A pair of glasses according to claim 10 or 11, wherein each of the cameras is located adjacent respective lenses in order to form the 3D image.

13. A pair of glasses according to any preceding claim, comprising one or more controllable light sources.

14. A pair of glasses according to claim 13, wherein the controllable light source is configured to illuminate a portion of the field of view of the camera to identify an area central to the field of view, or to identify one or more boundaries of the field of view of the camera.

15. A pair of glasses according to claim 13 or 14, wherein the controllable light source is configured to illuminate an object or scene being viewed by the wearer of the glasses.

16. A pair of glasses according to any preceding claim, comprising one or more filters in the optical path of the camera for adjusting the intensity of light from the subject being viewed or recorded by the camera.

17. A pair of glasses according to claim 16, wherein the filter is adjustable.

18. A pair of glasses according to any preceding claim, wherein an image generated by the camera is recorded on a memory device and/or transmitted to a remote receiver.

19. A pair of glasses according to any preceding claim, comprising a microphone for receiving sound in the vicinity of the glasses, the microphone being coupled to the processor and the processor being configured to receive a microphone signal from the microphone.

20. A pair of glasses according to claim 19, wherein the microphone signal is recorded on a memory device and/or transmitted to a remote receiver.

19. A pair of glasses according to any preceding claim, comprising a speaker adjacent at least one of the user’s ears, the speaker being coupled to the processor, and being configured to receive a signal from the processor and to output sound.

20. A pair of glasses according to claim 19, wherein the speaker comprises a bone conduction speaker using a portion of the frame in contact with the user’s head.

21. A pair of glasses according to any preceding claim, comprising a power source wherein the power source is located remote from the frame, and is connected to the glasses via a cable.

22. A pair of glasses according to any preceding claim, wherein the processor is located remote from the frame, and is connected to the glasses via a cable.

23. A pair of glasses according to any preceding claim, wherein the lenses are removable from the frame.

24. A pair of glasses according to any preceding claim, wherein the glasses weigh less than 10Og, more preferably they weigh less than 50g.

25. A pair of glasses substantially as herein described with reference to the accompanying figures 1 to 4.

Intellectual

Property

Office

Application No: Claims searched: