Nothing Special   »   [go: up one dir, main page]

US11223763B2 - Multi-mode image sensor applicable to image sensing and additional detection - Google Patents

Multi-mode image sensor applicable to image sensing and additional detection Download PDF

Info

Publication number
US11223763B2
US11223763B2 US16/784,257 US202016784257A US11223763B2 US 11223763 B2 US11223763 B2 US 11223763B2 US 202016784257 A US202016784257 A US 202016784257A US 11223763 B2 US11223763 B2 US 11223763B2
Authority
US
United States
Prior art keywords
mode
photodiodes
image sensor
als
output
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US16/784,257
Other versions
US20210250503A1 (en
Inventor
Chih-Yen Yang
Yu-Yu Sung
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Himax Imaging Ltd
Original Assignee
Himax Imaging Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Himax Imaging Ltd filed Critical Himax Imaging Ltd
Priority to US16/784,257 priority Critical patent/US11223763B2/en
Assigned to HIMAX IMAGING LIMITED reassignment HIMAX IMAGING LIMITED ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: SUNG, YU-YU, YANG, CHIH-YEN
Priority to TW109113268A priority patent/TWI728782B/en
Priority to CN202011055529.5A priority patent/CN113259608B/en
Publication of US20210250503A1 publication Critical patent/US20210250503A1/en
Application granted granted Critical
Publication of US11223763B2 publication Critical patent/US11223763B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • H04N5/23245—
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/667—Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J1/00—Photometry, e.g. photographic exposure meter
    • G01J1/42—Photometry, e.g. photographic exposure meter using electric radiation detectors
    • G01J1/4204—Photometry, e.g. photographic exposure meter using electric radiation detectors with determination of ambient light
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/08—Systems determining position data of a target for measuring distance only
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/481—Constructional features, e.g. arrangements of optical elements
    • G01S7/4816—Constructional features, e.g. arrangements of optical elements of receivers alone
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B5/00—Optical elements other than lenses
    • G02B5/20—Filters
    • G02B5/201—Filters in the form of arrays
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B5/00—Optical elements other than lenses
    • G02B5/20—Filters
    • G02B5/208—Filters for use with infrared or ultraviolet radiation, e.g. for separating visible light from infrared and/or ultraviolet radiation
    • H—ELECTRICITY
    • H01—ELECTRIC ELEMENTS
    • H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
    • H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
    • H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
    • H01L27/144—Devices controlled by radiation
    • H01L27/146—Imager structures
    • H01L27/14601—Structural or functional details thereof
    • H01L27/1462—Coatings
    • H01L27/14621—Colour filter arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
    • H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
    • H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
    • H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
    • H04N25/131—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements including elements passing infrared wavelengths
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
    • H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
    • H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
    • H04N25/134—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on three different wavelength filter elements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
    • H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
    • H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
    • H04N25/135—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on four or more different wavelength filter elements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/63—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to dark current
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/68—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to defects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/71—Charge-coupled device [CCD] sensors; Charge-transfer registers specially adapted for CCD sensors
    • H04N25/75—Circuitry for providing, modifying or processing image signals from the pixel array
    • H04N9/0455—

Definitions

  • the present invention relates to sensors, and more particularly, to a multi-mode image sensor, wherein the multi-mode image sensor is applicable to image sensing and additional detection such as ambient light detection and proximity detection.
  • CMOS Complementary Metal-Oxide-Semiconductor
  • a multifunctional mobile phone may need multiple holes on a certain portion of mechanical structure thereof for performing multiple types of optical sensing. More particularly, as one or more materials of the portion of mechanical structure may block the light, each type of the multiple types of optical sensing may need one or more corresponding holes, causing the design and implementation of the portion of mechanical structure to be complicated. In addition, the multiple types of optical sensing may cause high power consumption.
  • a novel architecture to enhance the overall performance of an electronic device without introducing a side effect or in a way that is less likely to introduce a side effect.
  • the multi-mode image sensor may comprise a photodiode array, a hybrid type color filter array that is positioned on the photodiode array, and a mode controller.
  • the photodiode array may comprise a plurality of photodiodes.
  • the hybrid type color filter array may be arranged to perform optical filtering for the photodiode array.
  • the hybrid type color filter array may comprise a plurality of optical filters aligned to the plurality of photodiodes, respectively.
  • the plurality of optical filter may comprise a plurality of types of color filters respectively corresponding to multiple color channels, and a plurality of infrared (IR) band-pass filters corresponding to a predetermined IR band, where each type of the plurality of types of color filters may comprise at least one color filter corresponding to one of the multiple color channels.
  • the mode controller may activate the photodiode array to output photo detection results in any of at least two modes, wherein said at least two modes comprise an image sensing mode and an ambient light sensor (ALS) mode.
  • ALS ambient light sensor
  • the image sensor utilizes all of the plurality of photodiodes to output a set of primary photo detection results, for performing the image sensing; and in the ALS mode, the image sensor utilizes a portion of photodiodes corresponding to the ALS mode within the plurality of photodiodes to output a set of secondary photo detection results, for performing ambient light detection, wherein the additional detection comprises the ambient light detection.
  • the multi-mode image sensor may comprise a photodiode array, a hybrid type color filter array that is positioned on the photodiode array, and a mode controller.
  • the photodiode array may comprise a plurality of photodiodes.
  • the hybrid type color filter array may be arranged to perform optical filtering for the photodiode array.
  • the hybrid type color filter array may comprise a plurality of optical filters aligned to the plurality of photodiodes, respectively.
  • the plurality of optical filter may comprise a plurality of types of color filters respectively corresponding to multiple color channels, and a plurality of IR band-pass filters corresponding to a predetermined IR band, where each type of the plurality of types of color filters may comprise at least one color filter corresponding to one of the multiple color channels.
  • the mode controller may activate the photodiode array to output photo detection results in any of at least two modes, wherein said at least two modes comprise an image sensing mode and a proximity sensor (PS) mode.
  • PS proximity sensor
  • the image sensor utilizes all of the plurality of photodiodes to output a set of primary photo detection results, for performing the image sensing; and in the PS mode, the image sensor utilizes a portion of photodiodes corresponding to the PS mode with the plurality of IR band-pass filters being aligned thereon within the plurality of photodiodes to output a set of secondary photo detection results, for performing proximity detection, wherein the additional detection comprises the proximity detection.
  • the present invention apparatus such as the image sensor applicable to image sensing and additional detection (e.g. ambient light detection and/or proximity detection) can minimize the number of holes for multiple types of optical sensing on mechanical structure of an electronic device such as a multifunctional mobile phone, and can save power and time in one or more of the multiple readout modes.
  • the present invention apparatus can enhance overall performance without introducing any side effect or in a way that is less likely to introduce a side effect.
  • FIG. 1 is a diagram of an image sensor applicable to image sensing and additional detection according to an embodiment of the present invention, where an infrared (IR) light emitting diode (LED) and associated lens modules, holders, etc. are also illustrated for better comprehension.
  • IR infrared
  • LED light emitting diode
  • FIG. 2 illustrates a pixel array of the image sensor shown in FIG. 1 according to an embodiment of the present invention.
  • FIG. 3 illustrates a triple-mode sensor according to an embodiment of the present invention, where the image sensor shown in FIG. 1 may be implemented to be a multi-mode sensor such as the triple-mode sensor.
  • FIG. 4 illustrates an example of an image sensing mode of multiple readout modes of the image sensor shown in FIG. 1 .
  • FIG. 5 illustrates an example of a proximity sensor (PS) mode of the multiple readout modes of the image sensor shown in FIG. 1 .
  • PS proximity sensor
  • FIG. 6 illustrates an example of an ambient light sensor (ALS) mode of the multiple readout modes of the image sensor shown in FIG. 1 .
  • ALS ambient light sensor
  • FIG. 7 illustrates another example of the ALS mode.
  • FIG. 8 illustrates yet another example of the ALS mode.
  • FIG. 9 illustrates a readout circuit of the image sensor shown in FIG. 1 according to an embodiment of the present invention, where an analog-to-digital converter (ADC) is also illustrated for better comprehension.
  • ADC analog-to-digital converter
  • FIG. 1 is a diagram of an image sensor 100 applicable to image sensing and additional detection according to an embodiment of the present invention, where an infrared (IR) light emitting diode (LED) and associated lens modules (labeled “Lens” for brevity), holders (e.g. the structure beside the lens modules as shown in FIG. 1 ), etc. are also illustrated for better comprehension.
  • the additional detection may comprise ambient light detection and proximity detection
  • the image sensor 100 may play the roles of a normal image sensor (e.g. a color image sensor), a proximity sensor (PS), and an ambient light sensor (ALS) in different modes, respectively (labeled “Image sensor & PS & ALS” for brevity).
  • a normal image sensor e.g. a color image sensor
  • PS proximity sensor
  • ALS ambient light sensor
  • the image sensor 100 may comprise a photodiode array 110 equipped with multiple readout modes, and a hybrid type color filter array 120 that is positioned on the photodiode array 110 .
  • the photodiode array 110 may be arranged to perform photo detection
  • the hybrid type color filter array 120 may be arranged to perform optical filtering for the photodiode array 110 .
  • the IR LED may be utilized as an IR emission source for illumination when the image sensor 100 plays the role of the PS, but the present invention is not limited thereto.
  • the present invention apparatus such as the image sensor 100 can minimize the number of holes for multiple types of optical sensing on the mechanical structure of an electronic device (e.g. a multifunctional mobile phone, a tablet, etc.) comprising the image sensor 100 , and can save power and time in one or more of the multiple readout modes.
  • FIG. 2 illustrates a pixel array 200 of the image sensor 100 shown in FIG. 1 according to an embodiment of the present invention, where the pixel array 200 may comprise the photodiode array 110 (labeled “PD array” for brevity) and the hybrid type color filter array 120 .
  • the photodiode array 110 may comprise a plurality of photodiodes (labeled “PD” for brevity), and the hybrid type color filter array 120 may comprise a plurality of optical filters aligned to the plurality of photodiodes, respectively.
  • the plurality of optical filter may comprise a plurality of types of color filters respectively corresponding to multiple color channels, such as red (R), green (G) and blue (B) color filters respectively corresponding to R, G and B color channels (e.g. the R/G/B color filters are labeled “R”, “G” and “B” to indicate their types, respectively), and a plurality of infrared (IR) band-pass filters (labeled “IR” for brevity) corresponding to a predetermined IR band, where each type of the plurality of types of color filters may comprise at least one color filter corresponding to one of the multiple color channels, such as one or more R/G/B color filters corresponding to the R/G/B color channel.
  • the predetermined IR band may represent an IR band around 850 nanometers (nm), but the present invention is not limited thereto.
  • the multiple readout modes may comprise an image sensing mode, an ALS mode, and a PS mode.
  • the image sensor 100 may utilize (e.g. activate) all of the plurality of photodiodes to output a set of primary photo detection results, for performing image sensing;
  • the image sensor 100 may utilize (e.g. activate) a portion of photodiodes corresponding to the ALS mode within the plurality of photodiodes to output a first set of secondary photo detection results, for performing the ambient light detection; and when the photodiode array 110 operates in the PS mode, the image sensor 100 may utilize (e.g.
  • the photodiode array 110 does not operate in any two of the multiple readout modes at the same time, and any two of the multiple readout modes are not activated at the same time.
  • one or more additional modes differing from the image sensing mode such as the ALS mode and the PS mode does not require all of the plurality of photodiodes to output photo detection results.
  • the image sensor 110 may utilize a portion of photodiodes to output the corresponding secondary photo detection results, for performing the additional detection.
  • the portion of photodiodes corresponding to the ALS mode rather than all of the plurality of photodiodes, may be arranged to output the first set of secondary photo detection results, whereby saving power and time.
  • the portion of photodiodes corresponding to the PS mode may be arranged to output the second set of secondary photo detection results, whereby saving power and time. Therefore, the image sensor 100 can save power and time in these readout modes.
  • the arrangement of the hybrid type color filter array 120 may vary.
  • the number of optical filters in any type of optical filters within the plurality of types of color filters and/or the relative locations thereof in the hybrid type color filter array 120 may vary.
  • the number of IR band-pass filters and/or the relative locations thereof in the hybrid type color filter array 120 may vary.
  • the image sensor 100 may select a set of photodiodes with at least one portion (e.g. a portion or all) of the plurality of types of color filters being aligned thereon within the plurality of photodiodes, such as a set of photodiodes differing from the portion of photodiodes corresponding to the PS mode, to be the portion of photodiodes corresponding to the ALS mode, but the present invention is not limited thereto.
  • the image sensor 100 may select a set of photodiodes with at least one portion (e.g.
  • the plurality of IR band-pass filters being aligned thereon within the plurality of photodiodes, such as the portion of photodiodes corresponding to the PS mode, to be the portion of photodiodes corresponding to the ALS mode.
  • FIG. 3 illustrates a triple-mode sensor 300 according to an embodiment of the present invention
  • FIGS. 4-6 illustrate respective examples of the image sensing mode such as a color image mode (referred to as the color mode for brevity), the PS mode, and the ALS mode
  • the image sensor 100 may be implemented to be a multi-mode sensor such as the triple-mode sensor 300 , but the present invention is not limited thereto.
  • the electronic device may comprise the multi-mode sensor such as the triple-mode sensor 300 , and may further comprise a post-processor 301 running one or more application programs (labeled “Application” for brevity), and comprise an IR light source 302 (e.g. the IR illumination module comprising the IR LED and the lens module thereof as shown in FIG.
  • the triple-mode sensor 300 may comprise a mode controller 303 and an in-system programming (ISP) circuit 304 (labeled “ISP” for brevity) running ISP codes of image sensing processing 3041 , the PS processing 3042 and the ALS processing 3043 respectively for the image sensing mode, the PS mode and the ALS mode.
  • the mode controller 303 may request the pixel array 200 and the ISP circuit 304 to operate in any mode of the image sensing mode (such as the color mode), the PS mode, and the ALS mode, and more particularly, may activate the pixel array 200 (e.g. the photodiode array 110 therein) to output corresponding photo detection results in the aforementioned any mode.
  • ISP in-system programming
  • the mode controller 303 may activate the image sensing mode such as the color mode, and activate the associated ISP processing such as the image sensing processing 3041 in the image sensing mode.
  • the mode controller 303 may activate the PS mode, and activate the associated ISP processing such as the PS processing 3042 together with the IR light source 302 in the PS mode.
  • the mode controller 303 may activate the ALS mode, and activate the associated ISP processing such as the ALS processing 3043 in the ALS mode. It is noted that the mode controller 303 may be commanded by a host or the post-processor 301 . In one embodiment, the mode controller 303 may be commanded to activate the PS mode or the ALS mode every predetermined time (e.g., 0.2 seconds (s)).
  • the ISP circuit 304 running the image sensing processing 3041 may perform dark level compensation and/or bad pixel compensation, and more particularly, may perform color de-mosaicking, but the present invention is not limited thereto.
  • the post-processor 301 running the one or more application programs may perform the color de-mosaicking.
  • the ISP circuit 304 may output a detection digital code indicating whether an object is within a predetermined distance in the PS mode and output an illumination average value ILLUMINATION AVG in the ALS mode by processing the second and the first sets of secondary photo detection results respectively, but the present invention is not limited thereto.
  • the ISP circuit 304 running the PS processing 3042 may output a detection signal (e.g. a signal carrying digital code 1 such as a logical value 1) to indicate that there is an object nearby.
  • a detection signal e.g. a signal carrying digital code 1 such as a logical value 1
  • the detection signal may indicate that the object is within the predetermined distance (e.g. 5 cm) corresponding to the predetermined threshold TH IR .
  • the one or more application programs may comprise a backlight control program module for controlling a backlight of a touch-sensitive display panel (e.g. a touch screen) of the electronic device.
  • the post-processor 301 running the backlight control program module may turn off the backlight according to the detection signal (e.g. the signal carrying digital code 1 such as the logical value 1).
  • the post-processor 301 running the backlight control program module may adjust the magnitude (e.g. brightness) of the backlight according to the illumination average value ILLUMINATION AVG generated through the ALS processing 3043 .
  • the post-processor 301 running the backlight control program module may decrease the magnitude (e.g. brightness) of the backlight; and when the ambient light becomes brighter (e.g. the illumination average value ILLUMINATION AVG increases), the post-processor 301 running the backlight control program module may increase the magnitude (e.g. brightness) of the backlight.
  • the image sensor 100 may utilize all of the plurality of photodiodes, such as the R, G and B photodiodes (respectively labeled “R”, “G” and “B” for brevity) respectively having the R, G and B color filters thereon and the IR photodiodes (labeled “IR” for brevity) having the IR band-pass filters thereon, to generate the corresponding pixel outputs to be the set of primary photo detection results.
  • the R, G and B photodiodes respectively having the R, G and B color filters thereon
  • IR photodiodes labeled “IR” for brevity
  • the R, G and B pixel outputs of a pixel may represent a summation of respective photo detection results of two R photodiodes minus two times an average of respective photo detection results of four IR photodiodes (which may be written as “(2R ⁇ 2IR AVG )” or “2(R ⁇ IR AVG )” for brevity), a summation of respective photo detection results of eight G photodiodes minus eight times the average of the respective photo detection results of the four IR photodiodes (which may be written as “(8G ⁇ 8IR AVG )” or “8(G ⁇ IR AVG )” for brevity), and a summation of respective photo detection results of two B photodiodes minus two times the average of the respective photo detection results of the four IR photodiodes (which may be written as “(2B ⁇ 2IR AVG )” or “2(B ⁇ IR AVG )” for brevity), respectively, but the present invention is not
  • the image sensor 100 may utilize the portion of photodiodes corresponding to the PS mode, such as the IR photodiodes, to generate an IR average to be the second set of secondary photo detection results, where the IR LED may be turned on in the PS mode.
  • the image sensor 100 may utilize the portion of photodiodes corresponding to the ALS mode, such as the R, G and B photodiodes, to generate an illumination average to be the first set of secondary photo detection results.
  • FIG. 9 illustrates a readout circuit of the image sensor 100 shown in FIG. 1 according to an embodiment of the present invention, where an analog-to-digital converter (ADC) is also illustrated for better comprehension.
  • the readout circuit may be coupled to at least one portion (e.g.
  • R photodiodes may read the pixel outputs, the IR average, and the illumination average in the image sensing mode (such as the color mode), the PS mode, and the ALS mode as mentioned above, respectively.
  • IR photodiodes may read the pixel outputs, the IR average, and the illumination average in the image sensing mode (such as the color mode), the PS mode, and the ALS mode as mentioned above, respectively.
  • the readout circuit may read respective photo detection results of the one or more R photodiodes and the one or more IR photodiodes to generate a linear combination thereof (e.g.
  • (2R ⁇ 2IR AVG )”) to be the R pixel output
  • read respective photo detection results of the one or more G photodiodes and the one or more IR photodiodes to generate a linear combination thereof e.g. “(8G ⁇ 8IR AVG )” to be the G pixel output
  • read respective photo detection results of the one or more B photodiodes and the one or more IR photodiodes to generate a linear combination thereof e.g. “(2B ⁇ 2IR AVG )”
  • CMOS Image Sensor CIS

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Optics & Photonics (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Electromagnetism (AREA)
  • Toxicology (AREA)
  • Health & Medical Sciences (AREA)
  • Sustainable Development (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Power Engineering (AREA)
  • Condensed Matter Physics & Semiconductors (AREA)
  • Computer Hardware Design (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)
  • Color Television Image Signal Generators (AREA)

Abstract

A multi-mode image sensor applicable to image sensing and additional detection is provided. The multi-mode image sensor includes: a photodiode array, a hybrid type color filter array that is positioned on the photodiode array, and a mode controller. The photodiode array may include a plurality of photodiodes. The hybrid type color filter array may be arranged to perform optical filtering for the photodiode array. The mode controller may activate the photodiode array to output corresponding photo detection results in any of at least two modes. For example, in addition to an image sensing mode, the at least two modes may further include an ambient light sensor (ALS) mode and/or a proximity sensor (PS) mode.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention
The present invention relates to sensors, and more particularly, to a multi-mode image sensor, wherein the multi-mode image sensor is applicable to image sensing and additional detection such as ambient light detection and proximity detection.
2. Description of the Prior Art
Image sensors such as that implemented by way of Complementary Metal-Oxide-Semiconductor (CMOS) sensor device, etc. have been widely used in electronic devices such as multifunctional mobile phones. According to the related art, a multifunctional mobile phone may need multiple holes on a certain portion of mechanical structure thereof for performing multiple types of optical sensing. More particularly, as one or more materials of the portion of mechanical structure may block the light, each type of the multiple types of optical sensing may need one or more corresponding holes, causing the design and implementation of the portion of mechanical structure to be complicated. In addition, the multiple types of optical sensing may cause high power consumption. Hence, there is a need for a novel architecture to enhance the overall performance of an electronic device without introducing a side effect or in a way that is less likely to introduce a side effect.
SUMMARY OF THE INVENTION
It is therefore an objective of the present invention to provide a multi-mode image sensor applicable to image sensing and additional detection such as ambient light detection and proximity detection, in order to solve the above-mentioned problems.
At least one embodiment of the present invention provides a multi-mode image sensor applicable to image sensing and additional detection. The multi-mode image sensor may comprise a photodiode array, a hybrid type color filter array that is positioned on the photodiode array, and a mode controller. The photodiode array may comprise a plurality of photodiodes. The hybrid type color filter array may be arranged to perform optical filtering for the photodiode array. In addition, the hybrid type color filter array may comprise a plurality of optical filters aligned to the plurality of photodiodes, respectively. More particularly, the plurality of optical filter may comprise a plurality of types of color filters respectively corresponding to multiple color channels, and a plurality of infrared (IR) band-pass filters corresponding to a predetermined IR band, where each type of the plurality of types of color filters may comprise at least one color filter corresponding to one of the multiple color channels. Additionally, the mode controller may activate the photodiode array to output photo detection results in any of at least two modes, wherein said at least two modes comprise an image sensing mode and an ambient light sensor (ALS) mode. For example, in the image sensing mode, the image sensor utilizes all of the plurality of photodiodes to output a set of primary photo detection results, for performing the image sensing; and in the ALS mode, the image sensor utilizes a portion of photodiodes corresponding to the ALS mode within the plurality of photodiodes to output a set of secondary photo detection results, for performing ambient light detection, wherein the additional detection comprises the ambient light detection.
At least one embodiment of the present invention provides a multi-mode image sensor applicable to image sensing and additional detection. The multi-mode image sensor may comprise a photodiode array, a hybrid type color filter array that is positioned on the photodiode array, and a mode controller. The photodiode array may comprise a plurality of photodiodes. The hybrid type color filter array may be arranged to perform optical filtering for the photodiode array. In addition, the hybrid type color filter array may comprise a plurality of optical filters aligned to the plurality of photodiodes, respectively. More particularly, the plurality of optical filter may comprise a plurality of types of color filters respectively corresponding to multiple color channels, and a plurality of IR band-pass filters corresponding to a predetermined IR band, where each type of the plurality of types of color filters may comprise at least one color filter corresponding to one of the multiple color channels. Additionally, the mode controller may activate the photodiode array to output photo detection results in any of at least two modes, wherein said at least two modes comprise an image sensing mode and a proximity sensor (PS) mode. For example, in the image sensing mode, the image sensor utilizes all of the plurality of photodiodes to output a set of primary photo detection results, for performing the image sensing; and in the PS mode, the image sensor utilizes a portion of photodiodes corresponding to the PS mode with the plurality of IR band-pass filters being aligned thereon within the plurality of photodiodes to output a set of secondary photo detection results, for performing proximity detection, wherein the additional detection comprises the proximity detection.
The present invention apparatus such as the image sensor applicable to image sensing and additional detection (e.g. ambient light detection and/or proximity detection) can minimize the number of holes for multiple types of optical sensing on mechanical structure of an electronic device such as a multifunctional mobile phone, and can save power and time in one or more of the multiple readout modes. In comparison with the related art, the present invention apparatus can enhance overall performance without introducing any side effect or in a way that is less likely to introduce a side effect.
These and other objectives of the present invention will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiment that is illustrated in the various figures and drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a diagram of an image sensor applicable to image sensing and additional detection according to an embodiment of the present invention, where an infrared (IR) light emitting diode (LED) and associated lens modules, holders, etc. are also illustrated for better comprehension.
FIG. 2 illustrates a pixel array of the image sensor shown in FIG. 1 according to an embodiment of the present invention.
FIG. 3 illustrates a triple-mode sensor according to an embodiment of the present invention, where the image sensor shown in FIG. 1 may be implemented to be a multi-mode sensor such as the triple-mode sensor.
FIG. 4 illustrates an example of an image sensing mode of multiple readout modes of the image sensor shown in FIG. 1.
FIG. 5 illustrates an example of a proximity sensor (PS) mode of the multiple readout modes of the image sensor shown in FIG. 1.
FIG. 6 illustrates an example of an ambient light sensor (ALS) mode of the multiple readout modes of the image sensor shown in FIG. 1.
FIG. 7 illustrates another example of the ALS mode.
FIG. 8 illustrates yet another example of the ALS mode.
FIG. 9 illustrates a readout circuit of the image sensor shown in FIG. 1 according to an embodiment of the present invention, where an analog-to-digital converter (ADC) is also illustrated for better comprehension.
 DETAILED DESCRIPTION
FIG. 1 is a diagram of an image sensor 100 applicable to image sensing and additional detection according to an embodiment of the present invention, where an infrared (IR) light emitting diode (LED) and associated lens modules (labeled “Lens” for brevity), holders (e.g. the structure beside the lens modules as shown in FIG. 1), etc. are also illustrated for better comprehension. As the additional detection may comprise ambient light detection and proximity detection, the image sensor 100 may play the roles of a normal image sensor (e.g. a color image sensor), a proximity sensor (PS), and an ambient light sensor (ALS) in different modes, respectively (labeled “Image sensor & PS & ALS” for brevity). As shown in FIG. 1, the image sensor 100 may comprise a photodiode array 110 equipped with multiple readout modes, and a hybrid type color filter array 120 that is positioned on the photodiode array 110. The photodiode array 110 may be arranged to perform photo detection, and the hybrid type color filter array 120 may be arranged to perform optical filtering for the photodiode array 110. In addition, the IR LED may be utilized as an IR emission source for illumination when the image sensor 100 plays the role of the PS, but the present invention is not limited thereto. Based on the architecture shown in FIG. 1, the present invention apparatus such as the image sensor 100 can minimize the number of holes for multiple types of optical sensing on the mechanical structure of an electronic device (e.g. a multifunctional mobile phone, a tablet, etc.) comprising the image sensor 100, and can save power and time in one or more of the multiple readout modes.
FIG. 2 illustrates a pixel array 200 of the image sensor 100 shown in FIG. 1 according to an embodiment of the present invention, where the pixel array 200 may comprise the photodiode array 110 (labeled “PD array” for brevity) and the hybrid type color filter array 120. The photodiode array 110 may comprise a plurality of photodiodes (labeled “PD” for brevity), and the hybrid type color filter array 120 may comprise a plurality of optical filters aligned to the plurality of photodiodes, respectively. More particularly, the plurality of optical filter may comprise a plurality of types of color filters respectively corresponding to multiple color channels, such as red (R), green (G) and blue (B) color filters respectively corresponding to R, G and B color channels (e.g. the R/G/B color filters are labeled “R”, “G” and “B” to indicate their types, respectively), and a plurality of infrared (IR) band-pass filters (labeled “IR” for brevity) corresponding to a predetermined IR band, where each type of the plurality of types of color filters may comprise at least one color filter corresponding to one of the multiple color channels, such as one or more R/G/B color filters corresponding to the R/G/B color channel. For example, the predetermined IR band may represent an IR band around 850 nanometers (nm), but the present invention is not limited thereto.
The multiple readout modes may comprise an image sensing mode, an ALS mode, and a PS mode. For example, when the photodiode array 110 operates in the image sensing mode, the image sensor 100 may utilize (e.g. activate) all of the plurality of photodiodes to output a set of primary photo detection results, for performing image sensing; when the photodiode array 110 operates in the ALS mode, the image sensor 100 may utilize (e.g. activate) a portion of photodiodes corresponding to the ALS mode within the plurality of photodiodes to output a first set of secondary photo detection results, for performing the ambient light detection; and when the photodiode array 110 operates in the PS mode, the image sensor 100 may utilize (e.g. activate) a portion of photodiodes corresponding to the PS mode with the plurality of IR band-pass filters being aligned thereon within the plurality of photodiodes to output a second set of secondary photo detection results, for performing the proximity detection. In addition, the photodiode array 110 does not operate in any two of the multiple readout modes at the same time, and any two of the multiple readout modes are not activated at the same time.
In comparison with the image sensing mode, one or more additional modes differing from the image sensing mode, such as the ALS mode and the PS mode does not require all of the plurality of photodiodes to output photo detection results. When the photodiode array 110 operates in any of the one or more additional modes, the image sensor 110 may utilize a portion of photodiodes to output the corresponding secondary photo detection results, for performing the additional detection. For example, when the photodiode array 110 operates in the ALS mode, the portion of photodiodes corresponding to the ALS mode, rather than all of the plurality of photodiodes, may be arranged to output the first set of secondary photo detection results, whereby saving power and time. For another example, when the photodiode array 110 operates in the PS mode, the portion of photodiodes corresponding to the PS mode, rather than all of the plurality of photodiodes, may be arranged to output the second set of secondary photo detection results, whereby saving power and time. Therefore, the image sensor 100 can save power and time in these readout modes.
According to some embodiments, the arrangement of the hybrid type color filter array 120 may vary. For example, the number of optical filters in any type of optical filters within the plurality of types of color filters and/or the relative locations thereof in the hybrid type color filter array 120 may vary. For another example, the number of IR band-pass filters and/or the relative locations thereof in the hybrid type color filter array 120 may vary.
According to some embodiments, the image sensor 100 may select a set of photodiodes with at least one portion (e.g. a portion or all) of the plurality of types of color filters being aligned thereon within the plurality of photodiodes, such as a set of photodiodes differing from the portion of photodiodes corresponding to the PS mode, to be the portion of photodiodes corresponding to the ALS mode, but the present invention is not limited thereto. According to some embodiments, the image sensor 100 may select a set of photodiodes with at least one portion (e.g. a portion or all) of the plurality of IR band-pass filters being aligned thereon within the plurality of photodiodes, such as the portion of photodiodes corresponding to the PS mode, to be the portion of photodiodes corresponding to the ALS mode.
FIG. 3 illustrates a triple-mode sensor 300 according to an embodiment of the present invention, and FIGS. 4-6 illustrate respective examples of the image sensing mode such as a color image mode (referred to as the color mode for brevity), the PS mode, and the ALS mode, where the image sensor 100 may be implemented to be a multi-mode sensor such as the triple-mode sensor 300, but the present invention is not limited thereto. The electronic device may comprise the multi-mode sensor such as the triple-mode sensor 300, and may further comprise a post-processor 301 running one or more application programs (labeled “Application” for brevity), and comprise an IR light source 302 (e.g. the IR illumination module comprising the IR LED and the lens module thereof as shown in FIG. 1). In addition, the triple-mode sensor 300 may comprise a mode controller 303 and an in-system programming (ISP) circuit 304 (labeled “ISP” for brevity) running ISP codes of image sensing processing 3041, the PS processing 3042 and the ALS processing 3043 respectively for the image sensing mode, the PS mode and the ALS mode. Additionally, the mode controller 303 may request the pixel array 200 and the ISP circuit 304 to operate in any mode of the image sensing mode (such as the color mode), the PS mode, and the ALS mode, and more particularly, may activate the pixel array 200 (e.g. the photodiode array 110 therein) to output corresponding photo detection results in the aforementioned any mode. For example, the mode controller 303 may activate the image sensing mode such as the color mode, and activate the associated ISP processing such as the image sensing processing 3041 in the image sensing mode. For another example, the mode controller 303 may activate the PS mode, and activate the associated ISP processing such as the PS processing 3042 together with the IR light source 302 in the PS mode. For yet another example, the mode controller 303 may activate the ALS mode, and activate the associated ISP processing such as the ALS processing 3043 in the ALS mode. It is noted that the mode controller 303 may be commanded by a host or the post-processor 301. In one embodiment, the mode controller 303 may be commanded to activate the PS mode or the ALS mode every predetermined time (e.g., 0.2 seconds (s)).
Regarding the image sensing mode, the ISP circuit 304 running the image sensing processing 3041 may perform dark level compensation and/or bad pixel compensation, and more particularly, may perform color de-mosaicking, but the present invention is not limited thereto. For example, the post-processor 301 running the one or more application programs may perform the color de-mosaicking. In one embodiment, the ISP circuit 304 may output a detection digital code indicating whether an object is within a predetermined distance in the PS mode and output an illumination average value ILLUMINATIONAVG in the ALS mode by processing the second and the first sets of secondary photo detection results respectively, but the present invention is not limited thereto.
Regarding the PS mode, if an IR average value IRAVG_LED_ON is greater than a predetermined threshold THIR, the ISP circuit 304 running the PS processing 3042 may output a detection signal (e.g. a signal carrying digital code 1 such as a logical value 1) to indicate that there is an object nearby. For example, the detection signal may indicate that the object is within the predetermined distance (e.g. 5 cm) corresponding to the predetermined threshold THIR. In addition, the one or more application programs may comprise a backlight control program module for controlling a backlight of a touch-sensitive display panel (e.g. a touch screen) of the electronic device. The post-processor 301 running the backlight control program module may turn off the backlight according to the detection signal (e.g. the signal carrying digital code 1 such as the logical value 1). Regarding the ALS mode, the post-processor 301 running the backlight control program module may adjust the magnitude (e.g. brightness) of the backlight according to the illumination average value ILLUMINATIONAVG generated through the ALS processing 3043. For example, when the ambient light becomes dimmer (e.g. the illumination average value ILLUMINATIONAVG decreases), the post-processor 301 running the backlight control program module may decrease the magnitude (e.g. brightness) of the backlight; and when the ambient light becomes brighter (e.g. the illumination average value ILLUMINATIONAVG increases), the post-processor 301 running the backlight control program module may increase the magnitude (e.g. brightness) of the backlight.
As shown in FIG. 4, in the image sensing mode such as the color mode, the image sensor 100 may utilize all of the plurality of photodiodes, such as the R, G and B photodiodes (respectively labeled “R”, “G” and “B” for brevity) respectively having the R, G and B color filters thereon and the IR photodiodes (labeled “IR” for brevity) having the IR band-pass filters thereon, to generate the corresponding pixel outputs to be the set of primary photo detection results. For example, the R, G and B pixel outputs of a pixel may represent a summation of respective photo detection results of two R photodiodes minus two times an average of respective photo detection results of four IR photodiodes (which may be written as “(2R−2IRAVG)” or “2(R−IRAVG)” for brevity), a summation of respective photo detection results of eight G photodiodes minus eight times the average of the respective photo detection results of the four IR photodiodes (which may be written as “(8G−8IRAVG)” or “8(G−IRAVG)” for brevity), and a summation of respective photo detection results of two B photodiodes minus two times the average of the respective photo detection results of the four IR photodiodes (which may be written as “(2B−2IRAVG)” or “2(B−IRAVG)” for brevity), respectively, but the present invention is not limited thereto.
As shown in FIG. 5, in the PS mode, the image sensor 100 may utilize the portion of photodiodes corresponding to the PS mode, such as the IR photodiodes, to generate an IR average to be the second set of secondary photo detection results, where the IR LED may be turned on in the PS mode. For example, the IR average may represent an average of respective photo detection results of four IR photodiodes with the IR LED being turned on (which may be written as “IRAVG_LED_ON=4IR/4” for brevity), but the present invention is not limited thereto. As shown in FIG. 6, in the ALS mode, the image sensor 100 may utilize the portion of photodiodes corresponding to the ALS mode, such as the R, G and B photodiodes, to generate an illumination average to be the first set of secondary photo detection results. For example, the illumination average may represent an average of respective photo detection results of two R photodiodes, eight G photodiodes, and two B photodiodes (which may be written as “ILLUMINATIONAVG=(2R+8G+2B)/12” for brevity), but the present invention is not limited thereto. For another example, as shown in FIG. 7, the illumination average may represent an average of respective photo detection results of one R photodiode, four G photodiodes, and one B photodiode (which may be written as “ILLUMINATIONAVG=(R+4G+B)/6” for brevity). For yet another example, as shown in FIG. 8, the image sensor 100 may utilize the portion of photodiodes corresponding to the PS mode, such as the IR photodiodes, to generate the illumination average to be the first set of secondary photo detection results, where the illumination average may represent an average of respective photo detection results of four IR photodiodes (which may be written as “ILLUMINATIONAVG=4IR/4” for brevity).
FIG. 9 illustrates a readout circuit of the image sensor 100 shown in FIG. 1 according to an embodiment of the present invention, where an analog-to-digital converter (ADC) is also illustrated for better comprehension. The readout circuit may be coupled to at least one portion (e.g. a portion or all) of one or more R photodiodes, one or more G photodiodes, one or more B photodiodes and one or more IR photodiodes (labeled “R”, “G”, “B”, and “IR” for brevity) in any of the multiple readout modes, and more particularly, may read the pixel outputs, the IR average, and the illumination average in the image sensing mode (such as the color mode), the PS mode, and the ALS mode as mentioned above, respectively. Taking the ALS mode as an example, the readout circuit may read respective photo detection results of the one or more R photodiodes, the one or more G photodiodes, and the one or more B photodiodes to generate the average thereof to be the illumination average (e.g. “ILLUMINATIONAVG=(2R+8G+2B)/12” or “ILLUMINATIONAVG=(R+4G+B)/6”), but the present invention is not limited thereto. For another example, the readout circuit may read respective photo detection results of the one or more IR photodiodes (e.g. some IR photodiodes) to generate the average thereof to be the illumination average (e.g. “ILLUMINATIONAVG=4IR/4”). For yet another example, regarding the PS mode, the readout circuit may read respective photo detection results of the one or more IR photodiodes (e.g. some IR photodiodes) to generate the average thereof to be the IR average (e.g. “IRAVG_LED_ON=4IR/4”). In some examples, regarding the image sensing mode such as the color mode, the readout circuit may read respective photo detection results of the one or more R photodiodes and the one or more IR photodiodes to generate a linear combination thereof (e.g. “(2R−2IRAVG)”) to be the R pixel output, read respective photo detection results of the one or more G photodiodes and the one or more IR photodiodes to generate a linear combination thereof (e.g. “(8G−8IRAVG)”) to be the G pixel output, and read respective photo detection results of the one or more B photodiodes and the one or more IR photodiodes to generate a linear combination thereof (e.g. “(2B−2IRAVG)”) to be the B pixel outputs. For brevity, similar descriptions for this embodiment are not repeated in detail here.
Please note that the method for reading from a photodiode is known in the CMOS Image Sensor (CIS) art, and therefore, some implementation details of the readout circuit may be omitted here for brevity.
Those skilled in the art will readily observe that numerous modifications and alterations of the device and method may be made while retaining the teachings of the invention. Accordingly, the above disclosure should be construed as limited only by the metes and bounds of the appended claims.

Claims (12)

What is claimed is:
1. A multi-mode image sensor, applicable to image sensing and additional detection, the multi-mode image sensor comprising:
a photodiode array, wherein the photodiode array comprises a plurality of photodiodes;
a hybrid type color filter array, positioned on the photodiode array, arranged to perform optical filtering for the photodiode array, wherein the hybrid type color filter array comprises:
a plurality of optical filters, aligned to the plurality of photodiodes, respectively, the plurality of optical filter comprising:
a plurality of types of color filters respectively corresponding to multiple color channels, wherein each type of the plurality of types of color filters comprises at least one color filter corresponding to one of the multiple color channels; and
a plurality of infrared (IR) band-pass filters corresponding to a predetermined IR band; and
a mode controller, activating the photodiode array to output photo detection results in any of at least two modes, wherein said at least two modes comprise an image sensing mode and an ambient light sensor (ALS) mode;
wherein:
in the image sensing mode, the image sensor utilizes all of the plurality of photodiodes to output a set of primary photo detection results, for performing the image sensing; and
in the ALS mode, the image sensor utilizes a portion of photodiodes corresponding to the ALS mode within the plurality of photodiodes to output a set of secondary photo detection results, for performing ambient light detection, wherein the additional detection comprises the ambient light detection.
2. The multi-mode image sensor of claim 1, wherein in the ALS mode, the portion of photodiodes corresponding to the ALS mode, rather than all of the plurality of photodiodes, is arranged to output the set of secondary photo detection results, whereby saving power and time.
3. The multi-mode image sensor of claim 1, wherein the mode controller selects a set of photodiodes with at least one portion of the plurality of types of color filters being aligned thereon within the plurality of photodiodes to be the portion of photodiodes corresponding to the ALS mode.
4. The multi-mode image sensor of claim 1, wherein the mode controller selects a set of photodiodes with at least one portion of the plurality of IR band-pass filters being aligned thereon within the plurality of photodiodes to be the portion of photodiodes corresponding to the ALS mode.
5. The multi-mode image sensor of claim 1, wherein said at least two modes further comprise a proximity sensor (PS) mode; and the mode controller utilizes a portion of photodiodes corresponding to the PS mode with the plurality of IR band-pass filters being aligned thereon within the plurality of photodiodes to output another set of secondary photo detection results, for performing proximity detection, wherein the additional detection further comprises the proximity detection.
6. The multi-mode image sensor of claim 5, wherein the mode controller selects a set of photodiodes differing from the portion of photodiodes corresponding to the PS mode to be the portion of photodiodes corresponding to the ALS mode.
7. The multi-mode image sensor of claim 5, wherein the mode controller selects the portion of photodiodes corresponding to the PS mode to be the portion of photodiodes corresponding to the ALS mode.
8. The multi-mode image sensor of claim 5, further comprising:
an in-system programming (ISP) circuit, arranged to output a detection digital code indicating whether an object is within a predetermined distance in the PS mode and output an illumination average value in the ALS mode by processing the sets of secondary photo detection results respectively.
9. The multi-mode image sensor of claim 1, further comprising:
an in-system programming (ISP) circuit, arranged to output an illumination average value in the ALS mode by processing the set of secondary photo detection results.
10. A multi-mode image sensor, applicable to image sensing and additional detection, the multi-mode image sensor comprising:
a photodiode array, wherein the photodiode array comprises a plurality of photodiodes
a hybrid type color filter array, positioned on the photodiode array, arranged to perform optical filtering for the photodiode array, wherein the hybrid type color filter array comprises:
a plurality of optical filters, aligned to the plurality of photodiodes, respectively, the plurality of optical filter comprising:
a plurality of types of color filters respectively corresponding to multiple color channels, wherein each type of the plurality of types of color filters comprises at least one color filter corresponding to one of the multiple color channels; and
a plurality of infrared (IR) band-pass filters corresponding to a predetermined IR band; and
a mode controller, activating the photodiode array to output photo detection results in any of at least two modes, wherein said at least two modes comprise an image sensing mode and a proximity sensor (PS) mode;
wherein:
in the image sensing mode, the image sensor utilizes all of the plurality of photodiodes to output a set of primary photo detection results, for performing the image sensing; and
in the PS mode, the image sensor utilizes a portion of photodiodes corresponding to the PS mode with the plurality of IR band-pass filters being aligned thereon within the plurality of photodiodes to output a set of secondary photo detection results, for performing proximity detection, wherein the additional detection comprises the proximity detection.
11. The multi-mode image sensor of claim 10, wherein in the PS mode, the portion of photodiodes corresponding to the PS mode, rather than all of the plurality of photodiodes, is arranged to output the set of secondary photo detection results, whereby saving power and time.
12. The multi-mode image sensor of claim 10, further comprising:
an in-system programming (ISP) circuit, arranged to output a detection digital code indicating whether an object is within a predetermined distance in the PS mode by processing the set of secondary photo detection results.
US16/784,257 2020-02-07 2020-02-07 Multi-mode image sensor applicable to image sensing and additional detection Active 2040-08-30 US11223763B2 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US16/784,257 US11223763B2 (en) 2020-02-07 2020-02-07 Multi-mode image sensor applicable to image sensing and additional detection
TW109113268A TWI728782B (en) 2020-02-07 2020-04-21 Multi-mode image sensor applicable to image sensing and additional detection
CN202011055529.5A CN113259608B (en) 2020-02-07 2020-09-30 Multi-mode image sensor suitable for image sensing and additional detection

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US16/784,257 US11223763B2 (en) 2020-02-07 2020-02-07 Multi-mode image sensor applicable to image sensing and additional detection

Publications (2)

Publication Number Publication Date
US20210250503A1 US20210250503A1 (en) 2021-08-12
US11223763B2 true US11223763B2 (en) 2022-01-11

Family

ID=77036716

Family Applications (1)

Application Number Title Priority Date Filing Date
US16/784,257 Active 2040-08-30 US11223763B2 (en) 2020-02-07 2020-02-07 Multi-mode image sensor applicable to image sensing and additional detection

Country Status (3)

Country Link
US (1) US11223763B2 (en)
CN (1) CN113259608B (en)
TW (1) TWI728782B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11785176B1 (en) * 2020-02-28 2023-10-10 Apple Inc. Ambient light sensor-based localization

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009046268A1 (en) 2007-10-04 2009-04-09 Magna Electronics Combined rgb and ir imaging sensor
TW201239387A (en) 2011-03-25 2012-10-01 Pixart Imaging Inc Optical sensor capable of detecting IR light and visible light simultaneously
US8717464B2 (en) 2011-02-09 2014-05-06 Blackberry Limited Increased low light sensitivity for image sensors by combining quantum dot sensitivity to visible and infrared light
US8743176B2 (en) * 2009-05-20 2014-06-03 Advanced Scientific Concepts, Inc. 3-dimensional hybrid camera and production system
US9257461B2 (en) 2013-09-10 2016-02-09 Samsung Electronics Co., Ltd. Image device including dynamic vision sensor, ambient light sensor and proximity sensor function
US10082576B2 (en) 2013-09-27 2018-09-25 Magnachip Semiconductor, Ltd. Optical sensor sensing illuminance and proximity
US11138428B2 (en) * 2016-04-07 2021-10-05 Tobii Ab Image sensor for vision based human computer interaction

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7071456B2 (en) * 2004-03-30 2006-07-04 Poplin Dwight D Camera module with ambient light detection
JP2009232351A (en) * 2008-03-25 2009-10-08 Seiko Epson Corp Image pickup device and color filter array
US8345132B2 (en) * 2010-07-23 2013-01-01 Omnivision Technologies, Inc. Image sensor with dual element color filter array and three channel color output
WO2012057558A2 (en) * 2010-10-29 2012-05-03 삼성전자주식회사 Filter for selective transmission of visible rays and infrared rays using an electrical signal
US8735795B2 (en) * 2012-01-20 2014-05-27 Omnivision Technologies, Inc. Image sensor with integrated ambient light detection
US9143704B2 (en) * 2012-01-20 2015-09-22 Htc Corporation Image capturing device and method thereof
TWI497041B (en) * 2013-02-18 2015-08-21 Eminent Electronic Technology Corp Ltd Optical sensor apparatus and image sensor apparatus
US9952092B2 (en) * 2015-10-30 2018-04-24 Avago Technologies General Ip (Singapore) Pte. Ltd. Matrix arrangement of photodetector for use as a proximity sensor and ambient light sensor
TWI658410B (en) * 2016-09-07 2019-05-01 李美燕 Optical imaging system with variable light field for biometrics application
US10347678B2 (en) * 2017-11-16 2019-07-09 Visera Technologies Company Limited Image sensor with shifted microlens array
KR20190103778A (en) * 2018-02-28 2019-09-05 빌리브마이크론(주) Ultra micro complex sensor with multiple environment acknoledge

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009046268A1 (en) 2007-10-04 2009-04-09 Magna Electronics Combined rgb and ir imaging sensor
US8743176B2 (en) * 2009-05-20 2014-06-03 Advanced Scientific Concepts, Inc. 3-dimensional hybrid camera and production system
US8717464B2 (en) 2011-02-09 2014-05-06 Blackberry Limited Increased low light sensitivity for image sensors by combining quantum dot sensitivity to visible and infrared light
TW201239387A (en) 2011-03-25 2012-10-01 Pixart Imaging Inc Optical sensor capable of detecting IR light and visible light simultaneously
US9257461B2 (en) 2013-09-10 2016-02-09 Samsung Electronics Co., Ltd. Image device including dynamic vision sensor, ambient light sensor and proximity sensor function
US10082576B2 (en) 2013-09-27 2018-09-25 Magnachip Semiconductor, Ltd. Optical sensor sensing illuminance and proximity
US11138428B2 (en) * 2016-04-07 2021-10-05 Tobii Ab Image sensor for vision based human computer interaction

Also Published As

Publication number Publication date
CN113259608B (en) 2024-06-28
CN113259608A (en) 2021-08-13
TW202131504A (en) 2021-08-16
TWI728782B (en) 2021-05-21
US20210250503A1 (en) 2021-08-12

Similar Documents

Publication Publication Date Title
US9001220B2 (en) Image sensor chip, method of obtaining image data based on a color sensor pixel and a motion sensor pixel in an image sensor chip, and system including the same
US9230310B2 (en) Imaging systems and methods for location-specific image flare mitigation
US9513724B2 (en) Device and method for adjusting object illumination
CN107580185B (en) Image sensor, operating method and imaging device
US8896733B2 (en) Imaging system with automatic conversion gain selection
US11134195B2 (en) Image sensor, camera module and electronic device
KR101262745B1 (en) Image sensor and photographing apparatus having the same
US20140168372A1 (en) Sensing apparatus and sensing method for generating three-dimensional image information
EP2565602B1 (en) Device and method for adjusting object illumination
JP2008167452A (en) Image sensor with temperature sensor and driving method and system thereof
US11223763B2 (en) Multi-mode image sensor applicable to image sensing and additional detection
US20120313908A1 (en) Device and method of adjusting electromagnetic radiation transmitted by an optical element
TW201427418A (en) Sensing apparatus and sensing method
US11150768B2 (en) Position detection device, display device, and position detection method
US11333959B2 (en) Position detection device, display device, and position detection method
KR20110072213A (en) Image correction method
KR20050039919A (en) Apparatus for sensing illumination by using image sensor and method thereof
JP2015072204A (en) Monitoring camera and monitoring system
KR20100131189A (en) Image sensor and method for measuring illumination using thereof
KR101429510B1 (en) Apparatus for protecting image saturation
CN106325987B (en) Control method and electronic equipment
KR101185573B1 (en) Image sensor and photographing apparatus having the same
JP2013024755A (en) Photosensor
KR101162264B1 (en) Image sensor and photographing apparatus having the same
JP2009055118A (en) Receiving system of visible light communication

Legal Events

Date Code Title Description
AS Assignment

Owner name: HIMAX IMAGING LIMITED, TAIWAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:YANG, CHIH-YEN;SUNG, YU-YU;REEL/FRAME:051747/0169

Effective date: 20200114

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: APPLICATION DISPATCHED FROM PREEXAM, NOT YET DOCKETED

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE