WO2011029483A1 - Visual categorization system - Google Patents
Visual categorization system Download PDFInfo
- Publication number
- WO2011029483A1 WO2011029483A1 PCT/EP2009/061882 EP2009061882W WO2011029483A1 WO 2011029483 A1 WO2011029483 A1 WO 2011029483A1 EP 2009061882 W EP2009061882 W EP 2009061882W WO 2011029483 A1 WO2011029483 A1 WO 2011029483A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- categorization
- captured image
- sound
- visual
- image
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B23/00—Telescopes, e.g. binoculars; Periscopes; Instruments for viewing the inside of hollow bodies; Viewfinders; Optical aiming or sighting devices
- G02B23/16—Housings; Caps; Mountings; Supports, e.g. with counterweight
- G02B23/18—Housings; Caps; Mountings; Supports, e.g. with counterweight for binocular arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
Definitions
- the present invention relates to a visual categorization system.
- the invention also relates to a corresponding visual categorization method and a computer program product.
- WO 98/26321 provides a solution to this problem, in which an instrument with the possibility of freezing the instrument-produced image is disclosed.
- the frozen image may then be studied without inconvenience, even if the instrument itself is subjected to shaking movements.
- the technique provides the user with the possibility of studying the frozen image in detail under larger magnification.
- the viewer uses a low degree of magnification, which provides the user with an overview of the scene. Thereafter, as soon as the user viewer has found the object of interest and preferably centered it to the centre field of the image, the image can be frozen and studied without being subjected to shaking movements.
- a visual categorization system comprising an image capturing module, a display member viewable by a user, and a control unit connected to the image capturing module and the display member.
- the control unit is configured to acquire an image captured by the image capturing module, provide the captured image to the display member, receive an object categorization input based on an image analysis of the content of the captured image, acquire object categorization information from an object categorization database based on the object categorization input, and provide the object categorization information to the display member to be viewable together with the captured image.
- the present invention is based on the understanding that it may be possible to include additional information corresponding to an object identified within the captured image to be displayed together on a display member.
- Advantages of the invention include for example increased understanding of objects visualized on the display member.
- the image analysis may be automatic, but is preferably semi-automated, e.g. by means of allowing first the captured image to be displayed on the display member, to be viewed by the user, and then providing the user with a selection of possible information corresponding to the object captured on in the image.
- the visual categorization system further comprising a satellite navigation receiver, such as a GPS, GLONASS or Galileo satellite receiver connected to the control unit, and the object categorization information is acquired from a sub-set of the object categorization database based on positional data from the satellite navigation receiver.
- a satellite navigation receiver such as a GPS, GLONASS or Galileo satellite receiver connected to the control unit
- the object categorization information is acquired from a sub-set of the object categorization database based on positional data from the satellite navigation receiver.
- an input organ such as e.g. a jog button
- a direction perpendicular to the image plane of the captured image may be determined using e.g. a gyro and/or a digital compass comprised with the visual categorization system.
- the gyro may be calibrated using information from the satellite navigation receiver and/or the digital compass.
- the visual categorization system may be arranged to comprise a first display screen having an ocular and a second display screen having an ocular, and the control unit may further be configured to provide the captured image to the first display screen and to provide the object categorization information to the second display screen.
- the control unit may further be configured to provide the captured image to the first display screen and to provide the object categorization information to the second display screen.
- the visual categorization system further comprises one or a plurality of sound sensors configured to provide sound data to the control unit
- the control unit may further be configured to categorizing the sound data, and depending on the number of sound sensors determining at least one of a distance, a direction and a visual location in respect of the sound data to the captured image and to provide at least one of the distance, the direction and the visual location to the display member to be viewable together with the captured image.
- further information may be acquired from a sound categorization database e.g. based on the categorized sound data, information from the satellite navigation system, the time of the day and/or the day of the year.
- the visual categorization system may further comprise at least one transducer connected to the control unit and configured to generate a response to at least one of the object categorization information and the sound categorization information.
- the transducer may also be used for transmitting information in the direction of the object, such as in an exemplary scenario of transmitting a calling sound towards birds located in the surrounding area with purpose of acquiring a response from the bird for e.g. allowing further identification of the bird type.
- the visual categorization system may furthermore comprise means configured to allow detachable connection to a binocular, thereby forming a visual observation system.
- the visual categorization system may be integrated with at least one of a digital camera, a mobile phone and a binocular.
- a method for visual categorization of image data comprising the steps of acquiring a captured image, acquiring capturing details for the captured image, the capturing details comprising at least one of the time and the location of capturing of the image, providing the captured image to a display member, receiving an object categorization input based on an image analysis of the content of the captured image, acquiring object categorization information from an object categorization database based on the object categorization input and the capturing details for the captured image, and providing the object categorization information to the display member to be viewable together with the captured image.
- a computer program product comprising a computer readable medium having stored thereon computer program means for causing a control unit to provide a visual categorization method
- the computer program product comprises code for acquiring a captured image, code for acquiring capturing details for the captured image, the capturing details comprising at least one of the time and the location of capturing of the image, code for providing the captured image to a display member, code for receiving an object
- categorization input based on an image analysis of the content of the captured image
- code for acquiring object categorization information from an object categorization database based on the object categorization input and the capturing details for the captured image code for providing the object categorization information to the display member to be viewable together with the captured image.
- the control unit is preferably a micro processor, possibly integrated in e.g. a digital camera or a mobile phone, and the computer readable medium may be one of a removable nonvolatile random access memory, a hard disk drive, a floppy disk, a CD-ROM, a DVD-ROM, a USB memory, an SD memory card, or a similar computer readable medium known in the art.
- Fig. 1 illustrates a visual observation system according to a currently preferred embodiment of the invention
- Fig. 2 shows a flow chart of a method according to an embodiment of the invention
- Fig 3 a illustrates a visual categorization system according to an embodiment of the invention
- Fig. 3b - 3c illustrates examples of a graphical user interface used with the visual categorization system.
- a visual observation system 100 comprising a visual categorization system 102 and a binocular 104.
- the binocular 104 is detachably connected to the visual categorization system 102 using fastening means 106, 108 of the visual categorization system 102.
- the visual categorization system 102 further comprises a camera 110 in the form of for example a CMOS or CCD image sensor, a first 112 and a second 114 display screen physically arranged to allow for a user to "drop" his gaze direction from the binocular to the display screens 112, 114, and a sound sensor 116 all connected to a control unit 118 of the visual categorization system 102.
- a camera 110 in the form of for example a CMOS or CCD image sensor
- a first 112 and a second 114 display screen physically arranged to allow for a user to "drop" his gaze direction from the binocular to the display screens 112, 114
- a sound sensor 116 all connected to a control unit 118 of the visual categorization system 102.
- the control unit 118 may include a microprocessor, a microcontroller, a programmable digital signal processor or another programmable device.
- the control unit 118 may also, or instead, include an application specific integrated circuit (ASIC), a
- control unit 118 includes a programmable device such as the microprocessor or microcontroller mentioned above, the processor may further include computer executable code that controls operation of the programmable device.
- the visual categorization system 102 may additionally comprise a GPS-receiver, a digital compass and a gyro in connection with the control unit 118 and configured to allow for directional and/or positional information that may be used in conjunction with the inventive visual categorization according to the invention as will be discussed further below in relation to Fig. 2.
- the visual categorization system 102 may be equipped with storage means for storing the images/video sequences captured using the camera 110, and be provided with communication means for transmitting captured images and video sequences to an external unit, for example in the form of a portable or stationary computer, such as a laptop, a Personal Digital Assistant (PDA), a desktop, or to an Internet connected
- a portable or stationary computer such as a laptop, a Personal Digital Assistant (PDA), a desktop, or to an Internet connected
- the transmission may be wired or wireless, including for example wired connections like USB, Fire Wire, VGA, or similar, and wireless connections like WLAN, CDMA, GSM, GPRS, 3G mobile communications, 4G mobile communications, Bluetooth, infrared, or similar.
- the visual categorization system 102 may be provided with a transducer (not shown), such as a speaker arranged in communication with the control unit 118 and configured to, as discussed above, provide an audio response to e.g. object and/or sound categorization information provided by the visual categorization system 102.
- the user may observe objects at a distance using the binocular 104.
- input control means such as a button comprised with the visual categorization system 102 for acquiring an image, SI, and capturing details, S2, about the captured image, including for example the direction in which the image was captured, the time of the day and/or the day of the year, and/or the physical coordinates, e.g. GPS coordinates of the position where the image was captured.
- the object may be studied by moving his direction of gaze downwards towards the display screens 112, 114 where the image is displayed, S3. That is, a line of sight of the binocular 104 and the camera 110 are preferably arranged to overlap such that an object visualized in the center when looking into the binocular 104 will appear at the center of the acquired image.
- the acquired image may be studied, either at a corresponding zoom level as is provided by the binocular 104, or the acquired image may be zoomed into such that, depending on the resolution of the camera 110, further details may be studied. Additionally, the camera 110 may also have a broader field of view then what may be provided by the binocular 104, thereby providing for possible detailed studies of parts of the image not initially visual using the binocular 104.
- Objects displayed on the captured image may be identified manually by the user and/or automatically using the control unit 118.
- the object identification is semi-manual, and the user provides an object categorization input, S4, using e.g. a button, to the control unit 118.
- the object categorization input may for example comprise an area of interest within the captured image where e.g. an animal is located, the animal being an object of interest.
- the control unit 118 acquires, S5, corresponding object categorization information from a database, the database possibly being locally accessible on the visual categorization system 102 for example stored onto a memory, or using wired and/or wireless means as discussed above for accessing a remotely located database.
- the object categorization information preferably further depend on the capturing details of the captured image, e.g. the physical coordinates relating to where the image was captured.
- the capturing details allows for sorting of the database such that manual, semi and/or automatic object categorization is improved, both in relation to the possible amount of information that needs to be transmitted i.e. wirelessly to and from the visual categorization system 102 and in relation to the amount of processing power that is necessary for possibly identifying the object of interest.
- the user may identify an object of interest, e.g. a bird.
- the user may have limited knowledge about birds, but may have interest in knowing more about the specific bird displayed within the captured image. In such a case, the user locates the bird within the captured image and provides that information as the object categorization input.
- further information is used, e.g. the capturing details of the image.
- the physical coordinates it may be possible to exclude information from the database as e.g. a specific physical location directly excludes specific bird types from possibly being the bird within the captured image due to geographic knowledge about bird types.
- additional information such as the day of the year, an additional finer exclusion may be possible.
- Further relations may be used for excluding possible information, i.e. in this example bird types.
- the object categorization information may be displayed together with the captured image on the display screens 112, 114. Displaying the object categorization information together with the captured image includes e.g. displaying the object
- categorization information on a first display screen e.g. display screen 112, and the captured image on the second display screen 114.
- Further possibilities include for example sequential "display switching" between the object categorization information and the captured image, or overlaying, possibly using some type of transparency, the object categorization information on top of section of the captured image.
- microphones of the visual categorization system 102 including microphone 116 may be used for determining the direction, the distance to the identified bird and/or for providing further identification possibilities e.g. by providing a frequency analysis of sound captured by the microphones for determining the type of bird.
- the user may be given the option to respond to the identified bird using e.g. a calling sound of the specific bird type.
- the transducer may also be used for transmitting information in the direction of the object, such as in an exemplary scenario of transmitting a calling sound towards birds located in the surrounding area with purpose of acquiring a response from the bird for e.g. allowing further
- the visual categorization system 102 and the binocular 104 are presented as separate parts of the visual observation system 100. However, it should be noted that functionality of visual categorization system 102 and the binocular
- an alternative visual observation system may also be somewhat different in relation to the positioning of the display member, for example by means of providing a "flip-up" display screen on top of the alternative visual observation system.
- a visual categorization system in the form of a mobile device such as a mobile phone 300, where the similar components such as a display screen 302 and a camera 304 are comprised.
- Fig. 3b and 3c provides exemplary user interfaces possible using the specifically configured mobile phone 300.
- a similar configuration of the user interfaces are of course possible also in relation to the visual categorization system 102 of Fig. 1.
- the display screen 302 has been essentially divided into a first 306 and a second 308 portion.
- the first portion 306 is configured for displaying the captured image and the second portion 308 is configured for displaying the further information, i.e. the object categorization information relating to a bird 310 located by the user.
- the further information may as discussed above include different illustrated examples 312, 314, of the bird type and textual information as to e.g. travelling pattern, mating periods, etc.
- the display screen 302 may be arranged such that the further information, e.g. the illustrated example of the bird type 312, is displayed on top of the captured image.
- the further information e.g. the illustrated example of the bird type 312
- the distance and direction 318 and the relative position 320 of the bird in relation to the captured image may also be determined and displayed with the captured image.
- icons including e.g. an icon 322 for activating a sound file to be played to the user (using a speaker or by providing connection of headphones) of the birds general sound may be provided.
- Additional icons may e.g. include an information icon 324 for displaying textual information 316, a video icon 326 for playing a video sequence of the bird on the display screen, and a play icon 328 for activating the speaker for transmitting a respond sound to the bird.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Library & Information Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Astronomy & Astrophysics (AREA)
- Optics & Photonics (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
Abstract
The present invention relates to a visual categorization system, comprising an image capturing module, a display member viewable by a user, and a control unit connected to the image capturing module and the display member. The control unit is configured to acquire an image captured by the image capturing module, provide the captured image to the display member, receive an object categorization input based on an image analysis of the content of the captured image, acquire object categorization information from an object categorization database based on the object categorization input, and provide the object categorization information to the display member to be viewable together with the captured image. The invention also relates to a corresponding visual categorization method and a computer program product.
Description
VISUAL CATEGORIZATION SYSTEM
TECHNICAL FIELD
The present invention relates to a visual categorization system. The invention also relates to a corresponding visual categorization method and a computer program product. BACKGROUND OF THE INVENTION
In using conventional hand-held viewing instruments, the largest useful magnification is restricted by the difficulty experienced by a user holding such handheld instruments sufficiently stable. The slightest shaking movement of the instrument causes the image viewed by the user to shake. In the case of high degrees of magnification, the shaking movements affect the image to such an extent that as to make perception thereof difficult. Hand-held viewing instruments therefore are rarely used for magnification above ten, and a magnification of seven or eight is common for practical purposes. When viewing instruments are used to view objects that appear small against a large background, such as birds or airplanes against the sky, high-magnification instruments may make it difficult to find the objects one wishes to study. High magnification reduces the field of vision.
WO 98/26321 provides a solution to this problem, in which an instrument with the possibility of freezing the instrument-produced image is disclosed. The frozen image may then be studied without inconvenience, even if the instrument itself is subjected to shaking movements. The technique provides the user with the possibility of studying the frozen image in detail under larger magnification. To find the object to be viewed, the viewer uses a low degree of magnification, which provides the user with an overview of the scene. Thereafter, as soon as the user viewer has found the object of interest and preferably centered it to the centre field of the image, the image can be frozen and studied without being subjected to shaking movements.
However, even though the aforementioned prior art provides high functionality, it would be desirable to allow further analysis of the detailed frozen image for extracting further information of interest.
SUMMARY OF THE INVENTION
According to an aspect of the invention, the above is at least partly met by a visual categorization system, comprising an image capturing module, a display member
viewable by a user, and a control unit connected to the image capturing module and the display member. The control unit is configured to acquire an image captured by the image capturing module, provide the captured image to the display member, receive an object categorization input based on an image analysis of the content of the captured image, acquire object categorization information from an object categorization database based on the object categorization input, and provide the object categorization information to the display member to be viewable together with the captured image.
The present invention is based on the understanding that it may be possible to include additional information corresponding to an object identified within the captured image to be displayed together on a display member. Advantages of the invention include for example increased understanding of objects visualized on the display member. The image analysis may be automatic, but is preferably semi-automated, e.g. by means of allowing first the captured image to be displayed on the display member, to be viewed by the user, and then providing the user with a selection of possible information corresponding to the object captured on in the image.
In an embodiment of the invention, the visual categorization system further comprising a satellite navigation receiver, such as a GPS, GLONASS or Galileo satellite receiver connected to the control unit, and the object categorization information is acquired from a sub-set of the object categorization database based on positional data from the satellite navigation receiver. Advantageously, it is possible to allow for the provision of a "smart" selection of information from the object categorization database to be selected by the user, e.g. by means of an input organ (such as e.g. a jog button) for acquiring further information of identified object(s) within the image. Additionally, it may be possible to also include the time of the day, the day of the year, and/or the direction to the identified object for determining the smart selection from the object categorization database. In an embodiment, a direction perpendicular to the image plane of the captured image may be determined using e.g. a gyro and/or a digital compass comprised with the visual categorization system.
Possibly, the gyro may be calibrated using information from the satellite navigation receiver and/or the digital compass.
Advantageously, the visual categorization system may be arranged to comprise a first display screen having an ocular and a second display screen having an ocular, and the control unit may further be configured to provide the captured image to the first display screen and to provide the object categorization information to the second display screen. In using such a configuration there is provided separation of the captured image and
the further information to the user's respective eyes, thereby possibly increasing the user's reception and understanding of the further information. Alternatively it may be possible to divide each of the first and the second display screens into first and second respective portions, and to provide the captured image to the first portion of each of the first and the second display screen, and to provide the object categorization information to the second portion of each of the first and the second display screen.
In a preferred embodiment, the visual categorization system further comprises one or a plurality of sound sensors configured to provide sound data to the control unit, the control unit may further be configured to categorizing the sound data, and depending on the number of sound sensors determining at least one of a distance, a direction and a visual location in respect of the sound data to the captured image and to provide at least one of the distance, the direction and the visual location to the display member to be viewable together with the captured image. In conjunction with this, further information may be acquired from a sound categorization database e.g. based on the categorized sound data, information from the satellite navigation system, the time of the day and/or the day of the year.
Also, the visual categorization system may further comprise at least one transducer connected to the control unit and configured to generate a response to at least one of the object categorization information and the sound categorization information. The transducer may also be used for transmitting information in the direction of the object, such as in an exemplary scenario of transmitting a calling sound towards birds located in the surrounding area with purpose of acquiring a response from the bird for e.g. allowing further identification of the bird type.
The visual categorization system may furthermore comprise means configured to allow detachable connection to a binocular, thereby forming a visual observation system. Alternatively, the visual categorization system may be integrated with at least one of a digital camera, a mobile phone and a binocular.
According to another aspect of the present invention there is provided a method for visual categorization of image data, comprising the steps of acquiring a captured image, acquiring capturing details for the captured image, the capturing details comprising at least one of the time and the location of capturing of the image, providing the captured image to a display member, receiving an object categorization input based on an image analysis of the content of the captured image, acquiring object categorization information from an object categorization database based on the object categorization input and the capturing details for the captured image, and providing the object categorization information to the display
member to be viewable together with the captured image. This aspect of the invention provides similar advantages as discussed above in relation to the previous aspect of the invention.
According to a still further aspect of the present invention there is provided a computer program product comprising a computer readable medium having stored thereon computer program means for causing a control unit to provide a visual categorization method, wherein the computer program product comprises code for acquiring a captured image, code for acquiring capturing details for the captured image, the capturing details comprising at least one of the time and the location of capturing of the image, code for providing the captured image to a display member, code for receiving an object
categorization input based on an image analysis of the content of the captured image, code for acquiring object categorization information from an object categorization database based on the object categorization input and the capturing details for the captured image, and code for providing the object categorization information to the display member to be viewable together with the captured image.
The control unit is preferably a micro processor, possibly integrated in e.g. a digital camera or a mobile phone, and the computer readable medium may be one of a removable nonvolatile random access memory, a hard disk drive, a floppy disk, a CD-ROM, a DVD-ROM, a USB memory, an SD memory card, or a similar computer readable medium known in the art.
Further features of, and advantages with, the present invention will become apparent when studying the appended claims and the following description. The skilled addressee realize that different features of the present invention may be combined to create embodiments other than those described in the following, without departing from the scope of the present invention.
BRIEF DESCRIPTION OF THE DRAWINGS
The various aspects of the invention, including its particular features and advantages, will be readily understood from the following detailed description and the accompanying drawings, in which:
Fig. 1 illustrates a visual observation system according to a currently preferred embodiment of the invention;
Fig. 2 shows a flow chart of a method according to an embodiment of the invention; and
Fig 3 a illustrates a visual categorization system according to an embodiment of the invention and Fig. 3b - 3c illustrates examples of a graphical user interface used with the visual categorization system. DETAILED DESCRIPTION
The present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which currently preferred embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided for thoroughness and completeness, and fully convey the scope of the invention to the skilled addressee. Like reference characters refer to like elements throughout.
Referring now to the drawings and to Fig. 1 in particular, there is depicted a visual observation system 100 according to an embodiment of the invention, comprising a visual categorization system 102 and a binocular 104. The binocular 104 is detachably connected to the visual categorization system 102 using fastening means 106, 108 of the visual categorization system 102. The visual categorization system 102 further comprises a camera 110 in the form of for example a CMOS or CCD image sensor, a first 112 and a second 114 display screen physically arranged to allow for a user to "drop" his gaze direction from the binocular to the display screens 112, 114, and a sound sensor 116 all connected to a control unit 118 of the visual categorization system 102.
The control unit 118 may include a microprocessor, a microcontroller, a programmable digital signal processor or another programmable device. The control unit 118 may also, or instead, include an application specific integrated circuit (ASIC), a
programmable gate array programmable array logic, a programmable logic device, or a digital signal processor. Where the control unit 118 includes a programmable device such as the microprocessor or microcontroller mentioned above, the processor may further include computer executable code that controls operation of the programmable device.
Further to the above mentioned components of the visual categorization system 102, the visual categorization system 102 may additionally comprise a GPS-receiver, a digital compass and a gyro in connection with the control unit 118 and configured to allow for directional and/or positional information that may be used in conjunction with the inventive visual categorization according to the invention as will be discussed further below in relation to Fig. 2.
Additionally, the visual categorization system 102 may be equipped with storage means for storing the images/video sequences captured using the camera 110, and be provided with communication means for transmitting captured images and video sequences to an external unit, for example in the form of a portable or stationary computer, such as a laptop, a Personal Digital Assistant (PDA), a desktop, or to an Internet connected
broadcasting service such as YouTube. The transmission may be wired or wireless, including for example wired connections like USB, Fire Wire, VGA, or similar, and wireless connections like WLAN, CDMA, GSM, GPRS, 3G mobile communications, 4G mobile communications, Bluetooth, infrared, or similar. Furthermore, the visual categorization system 102 may be provided with a transducer (not shown), such as a speaker arranged in communication with the control unit 118 and configured to, as discussed above, provide an audio response to e.g. object and/or sound categorization information provided by the visual categorization system 102.
During operation of the visual observation system 100, with reference in parallel to Fig. 2, the user may observe objects at a distance using the binocular 104. At a point in time when the user observes an object of interest, he may use input control means, such as a button comprised with the visual categorization system 102 for acquiring an image, SI, and capturing details, S2, about the captured image, including for example the direction in which the image was captured, the time of the day and/or the day of the year, and/or the physical coordinates, e.g. GPS coordinates of the position where the image was captured. The object may be studied by moving his direction of gaze downwards towards the display screens 112, 114 where the image is displayed, S3. That is, a line of sight of the binocular 104 and the camera 110 are preferably arranged to overlap such that an object visualized in the center when looking into the binocular 104 will appear at the center of the acquired image.
The acquired image may be studied, either at a corresponding zoom level as is provided by the binocular 104, or the acquired image may be zoomed into such that, depending on the resolution of the camera 110, further details may be studied. Additionally, the camera 110 may also have a broader field of view then what may be provided by the binocular 104, thereby providing for possible detailed studies of parts of the image not initially visual using the binocular 104.
Objects displayed on the captured image may be identified manually by the user and/or automatically using the control unit 118. In an embodiment, the object identification is semi-manual, and the user provides an object categorization input, S4, using
e.g. a button, to the control unit 118. The object categorization input may for example comprise an area of interest within the captured image where e.g. an animal is located, the animal being an object of interest. Based on the object categorization input the control unit 118 acquires, S5, corresponding object categorization information from a database, the database possibly being locally accessible on the visual categorization system 102 for example stored onto a memory, or using wired and/or wireless means as discussed above for accessing a remotely located database. The object categorization information preferably further depend on the capturing details of the captured image, e.g. the physical coordinates relating to where the image was captured.
In a preferred embodiment, the capturing details allows for sorting of the database such that manual, semi and/or automatic object categorization is improved, both in relation to the possible amount of information that needs to be transmitted i.e. wirelessly to and from the visual categorization system 102 and in relation to the amount of processing power that is necessary for possibly identifying the object of interest. For example, in a semi- automatic scenario, the user may identify an object of interest, e.g. a bird. The user may have limited knowledge about birds, but may have interest in knowing more about the specific bird displayed within the captured image. In such a case, the user locates the bird within the captured image and provides that information as the object categorization input. However, in acquiring object categorization information from the database, further information is used, e.g. the capturing details of the image. Making use of e.g. the physical coordinates it may be possible to exclude information from the database as e.g. a specific physical location directly excludes specific bird types from possibly being the bird within the captured image due to geographic knowledge about bird types. By providing additional information, such as the day of the year, an additional finer exclusion may be possible. Further relations may be used for excluding possible information, i.e. in this example bird types.
In the end, a limited amount of information needs to be processed for identifying the specific bird type marked by the user. This possibly limits the necessary bandwidth of transmitted information or the processing power/time needed for identifying the specific bird type. Improvements in relation to both bandwidth and processing power may possibly be seen in a case where parts of the identification process is done using cloud computing, i.e. where at least a part of the identification and the database are performed on a remotely located server.
Following the identification and acquisition of the object categorization information, the object categorization information may be displayed together with the
captured image on the display screens 112, 114. Displaying the object categorization information together with the captured image includes e.g. displaying the object
categorization information on a first display screen, e.g. display screen 112, and the captured image on the second display screen 114. However, it may also be possible to divide each of the display screens 112, 114 into first and second display portions, and provide the object categorization information on a first display portions and the captured image on the second display portions, respectively. Further possibilities include for example sequential "display switching" between the object categorization information and the captured image, or overlaying, possibly using some type of transparency, the object categorization information on top of section of the captured image.
Using the above approach the user is provided with additional information about the located bird, possibly including the type of bird, the sound made by the bird, differences in gender, etc. Furthermore, microphones of the visual categorization system 102, including microphone 116 may be used for determining the direction, the distance to the identified bird and/or for providing further identification possibilities e.g. by providing a frequency analysis of sound captured by the microphones for determining the type of bird. Using the above mentioned transducer/speaker, the user may be given the option to respond to the identified bird using e.g. a calling sound of the specific bird type. The transducer may also be used for transmitting information in the direction of the object, such as in an exemplary scenario of transmitting a calling sound towards birds located in the surrounding area with purpose of acquiring a response from the bird for e.g. allowing further
identification of the bird type.
In the embodiment of Fig. 1, the visual categorization system 102 and the binocular 104 are presented as separate parts of the visual observation system 100. However, it should be noted that functionality of visual categorization system 102 and the binocular
104 may be integrated as one, i.e. forming an alternative combined visual observation system (not shown). Such an alternative visual observation system may also be somewhat different in relation to the positioning of the display member, for example by means of providing a "flip-up" display screen on top of the alternative visual observation system.
As a further alternative, there may in another embodiment of the invention such as shown in Fig. 3a be provided a visual categorization system in the form of a mobile device such as a mobile phone 300, where the similar components such as a display screen 302 and a camera 304 are comprised. Fig. 3b and 3c provides exemplary user interfaces possible using the specifically configured mobile phone 300. A similar configuration of the
user interfaces are of course possible also in relation to the visual categorization system 102 of Fig. 1.
In Fig 3b, the display screen 302 has been essentially divided into a first 306 and a second 308 portion. The first portion 306 is configured for displaying the captured image and the second portion 308 is configured for displaying the further information, i.e. the object categorization information relating to a bird 310 located by the user. The further information may as discussed above include different illustrated examples 312, 314, of the bird type and textual information as to e.g. travelling pattern, mating periods, etc.
Alternatively and as is shown in Fig. 3c, the display screen 302 may be arranged such that the further information, e.g. the illustrated example of the bird type 312, is displayed on top of the captured image. Using one or a plurality of the microphones comprised with the mobile phone 300 (not shown), the distance and direction 318 and the relative position 320 of the bird in relation to the captured image may also be determined and displayed with the captured image. Furthermore, icons including e.g. an icon 322 for activating a sound file to be played to the user (using a speaker or by providing connection of headphones) of the birds general sound may be provided. Additional icons may e.g. include an information icon 324 for displaying textual information 316, a video icon 326 for playing a video sequence of the bird on the display screen, and a play icon 328 for activating the speaker for transmitting a respond sound to the bird.
Even though the invention has been described with reference to specific exemplifying embodiments thereof, many different alterations, modifications and the like will become apparent for those skilled in the art. Variations to the disclosed embodiments can be understood and effected by the skilled addressee in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. For example, even though the description above have been made in relation to different bird types it is possible and within the scope of the invention to use the inventive concept in relation to
providing/displaying further information about e.g. different geographical locations, persons, vehicles (military and/or civilian) or in relation to any other type of objects captured using the camera and/or microphone(s). Furthermore, in the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality.
Claims
1. Visual categorization system, comprising:
- an image capturing module;
- a display member viewable by a user; and
- a control unit connected to the image capturing module and the display member,
wherein the control unit is configured to:
- acquire an image captured by the image capturing module;
- provide the captured image to the display member;
- receive an object categorization input based on an image analysis of the content of the captured image;
- acquire object categorization information from an object categorization database based on the object categorization input, and
- provide the object categorization information to the display member to be viewable together with the captured image.
2. Visual categorization system according to claim 1, further comprising a satellite navigation receiver connected to the control unit, and wherein the object
categorization information is acquired from a sub-set of the object categorization database based on positional data from the satellite navigation receiver.
3. Visual categorization system according to claim 2, further comprising means for determining at least one of the day of the year and the time of the day, and wherein the object categorization information is acquired from a part of the sub-set of the object categorization database based on at least one of the day of the year and the time of the day.
4. Visual categorization system according to any one of the preceding claims, wherein the display member comprises a first display screen having an ocular and a second display screen having an ocular, and the control unit is further configured to provide the captured image to the first display screen and to provide the object categorization information to the second display screen.
5. Visual categorization system according to any one of the preceding claims, wherein the object categorization input is provided by means of an input organ connected to the control unit and configured to be controlled by the user.
6. Visual categorization system according to any one of the preceding claims, further comprising means for determining a direction perpendicular to the image plane of the captured image.
7. Visual categorization system according to any one of the preceding claims, further comprising at least one transducer connected to the control unit and configured to generating a response to the object categorization information.
8. Visual categorization system according to claim 7, wherein the control unit is further configured to provide a calling sound to be transmitted by the transducer.
9. Visual categorization system according to any one of the preceding claims, further comprising a sound sensor, and wherein the control unit is further configured to acquire sound data from the sound sensor, to categorizing the sound data, to acquire sound categorization information from a sound categorization database based on the categorized sound data from the sound sensor, and to provide the sound categorization information to the display member to be viewable together with the captured image.
10. Visual categorization system according to claim 9, comprising a plurality of sound sensors, and wherein the control unit is further configured determine at least one of a distance, a direction and a visual location in respect of the acquired sound data to the captured image and to provide at least one of the distance, the direction and the visual location to the display member to be viewable together with the captured image.
11. Visual categorization system according to claim 9 or 10, wherein the at least one transducer is further configured to generate a response to the sound categorization information.
12. Visual categorization system according to any one of the preceding claims, further comprising means configured to allow detachable connection to a binocular.
13. Visual categorization system according to any one of claims 1 - 12, wherein the visual categorization system is integrated with at least one of a digital camera, a mobile phone and a binocular.
14. Visual observation system, comprising a visual categorization system according to any one of claims 1 - 11 and a binocular.
15. Method for visual categorization of image data, comprising the steps of:
- acquiring a captured image;
- acquiring capturing details for the captured image, the capturing details comprising at least one of the time and the location of capturing of the image;
- providing the captured image to a display member;
- receiving an object categorization input based on an image analysis of the content of the captured image;
- acquiring object categorization information from an object categorization database based on the object categorization input and the capturing details for the captured image, and
- providing the object categorization information to the display member to be viewable together with the captured image.
16. Method according to claim 15, wherein the captured image is provided to a first portion of the display member visible solely to a first eye of a user and the categorization information is provided to a second portion of the display member visible solely to a second eye of the user.
17. Method according to claim 15, wherein the display member comprises a first and a second display screen, the first display screen visible solely to a first eye of a user and the second display screen visible solely to a second eye of the user, and the captured image is provided to a first portion of the first and the second display screens and the categorization information is provided to a second portion of the first and the second display screens.
18. Method according to any one of claims 15 - 16, further comprising the steps of:
- acquiring sound data from a sound sensor; - categorizing the sound data;
- acquiring sound categorization information from a sound categorization database based on the categorized sound data from the sound sensor, and
- providing the sound categorization information to the display member to be viewable together with the captured image.
19. Method according to any one of claims 15 - 17, further comprising the steps of:
- acquiring sound data from a plurality of sound sensors;
- determining at least one of a distance, a direction and a visual location of the acquired sound in respect to the captured image;
- categorizing the sound data;
- acquiring sound categorization information from a sound categorization database based on the categorized sound data from the sound sensor, and
- providing the sound categorization information to the display member to be viewable together with the captured image.
20. Method according to claim 15, further comprising the step of generating a drive signal for a transducer in response to the object categorization information.
21. Method according to claim 18, further comprising the step of generating a drive signal for a transducer in response to the sound categorization information.
22. Computer program product comprising a computer readable medium having stored thereon computer program means for causing a control unit to provide a visual categorization method, wherein the computer program product comprises:
- code for acquiring a captured image;
- code for acquiring capturing details for the captured image, the capturing details comprising at least one of the time and the location of capturing of the image;
- code for providing the captured image to a display member;
- code for receiving an object categorization input based on an image analysis of the content of the captured image; - code for acquiring object categorization information from an object categorization database based on the object categorization input and the capturing details for the captured image, and
- code for providing the object categorization information to the display member to be viewable together with the captured image.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/EP2009/061882 WO2011029483A1 (en) | 2009-09-14 | 2009-09-14 | Visual categorization system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/EP2009/061882 WO2011029483A1 (en) | 2009-09-14 | 2009-09-14 | Visual categorization system |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2011029483A1 true WO2011029483A1 (en) | 2011-03-17 |
Family
ID=41800513
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/EP2009/061882 WO2011029483A1 (en) | 2009-09-14 | 2009-09-14 | Visual categorization system |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2011029483A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2012249155A (en) * | 2011-05-30 | 2012-12-13 | Nikon Corp | Electronic apparatus |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1315102A2 (en) * | 2001-11-21 | 2003-05-28 | Hewlett-Packard Company | A context-aware imaging device |
US20030115215A1 (en) * | 2001-12-18 | 2003-06-19 | Daniel Swarovski | Method and system for watching and tracking birds |
JP2004219520A (en) * | 2003-01-10 | 2004-08-05 | Pentax Corp | Optical equipment with digital camera function, and recording medium for the same |
-
2009
- 2009-09-14 WO PCT/EP2009/061882 patent/WO2011029483A1/en active Application Filing
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1315102A2 (en) * | 2001-11-21 | 2003-05-28 | Hewlett-Packard Company | A context-aware imaging device |
US20030115215A1 (en) * | 2001-12-18 | 2003-06-19 | Daniel Swarovski | Method and system for watching and tracking birds |
JP2004219520A (en) * | 2003-01-10 | 2004-08-05 | Pentax Corp | Optical equipment with digital camera function, and recording medium for the same |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2012249155A (en) * | 2011-05-30 | 2012-12-13 | Nikon Corp | Electronic apparatus |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US12112443B2 (en) | Display control apparatus, display control method, and program | |
US8564710B2 (en) | Photographing apparatus and photographing method for displaying information related to a subject | |
US10181211B2 (en) | Method and apparatus of prompting position of aerial vehicle | |
CN103248790B (en) | server, client terminal and system | |
US8994785B2 (en) | Method for generating video data and image photographing device thereof | |
US20140160129A1 (en) | Information processing apparatus and recording medium | |
EP3229459A1 (en) | Information processing device, information processing method and program | |
EP3163863B1 (en) | Image display device and image display method | |
CN109844600B (en) | Information processing apparatus, information processing method, and program | |
US20170264822A1 (en) | Mounting Device for Portable Multi-Stream Video Recording Device | |
CN107667524A (en) | The method and imaging device that Moving Objects are imaged | |
JP2019114147A (en) | Image processing apparatus, control method for image processing apparatus, and program | |
JP4920660B2 (en) | Apparatus and method for displaying virtual object superimposed on real image taken in the past | |
CN112818240A (en) | Comment information display method, comment information display device, comment information display equipment and computer-readable storage medium | |
WO2011029483A1 (en) | Visual categorization system | |
JP5649842B2 (en) | Information providing apparatus, information providing method, and program | |
JP2016025620A (en) | Image processing system, client system, image processing method, and storage medium | |
CN108600797B (en) | Information processing method and electronic equipment | |
JP6754893B2 (en) | Image generator and image generation method | |
JP6451110B2 (en) | Imaging apparatus, image generation method, and program | |
CN116820229B (en) | XR space display method, XR equipment, electronic equipment and storage medium | |
CN110766129A (en) | Neural network training system and data display method | |
US20240323537A1 (en) | Display terminal, communication system, display method, and recording medium | |
US10904420B2 (en) | Control device and control method for managing a captured image | |
CN107872637B (en) | Image reproducing apparatus, image reproducing method, and recording medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 09782977 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 09782977 Country of ref document: EP Kind code of ref document: A1 |