US10367949B2 - Echo canceller device and voice telecommunications device - Google Patents
Echo canceller device and voice telecommunications device Download PDFInfo
- Publication number
- US10367949B2 US10367949B2 US15/775,201 US201515775201A US10367949B2 US 10367949 B2 US10367949 B2 US 10367949B2 US 201515775201 A US201515775201 A US 201515775201A US 10367949 B2 US10367949 B2 US 10367949B2
- Authority
- US
- United States
- Prior art keywords
- echo
- filter
- voice
- false
- coefficient
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M9/00—Arrangements for interconnection not involving centralised switching
- H04M9/08—Two-way loud-speaking telephone systems with means for conditioning the signal, e.g. for suppressing echoes for one or both directions of traffic
- H04M9/082—Two-way loud-speaking telephone systems with means for conditioning the signal, e.g. for suppressing echoes for one or both directions of traffic using echo cancellers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B3/00—Line transmission systems
- H04B3/02—Details
- H04B3/20—Reducing echo effects or singing; Opening or closing transmitting path; Conditioning for transmission in one direction or the other
- H04B3/23—Reducing echo effects or singing; Opening or closing transmitting path; Conditioning for transmission in one direction or the other using a replica of transmitted signal in the time domain, e.g. echo cancellers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/60—Substation equipment, e.g. for use by subscribers including speech amplifiers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M9/00—Arrangements for interconnection not involving centralised switching
- H04M9/08—Two-way loud-speaking telephone systems with means for conditioning the signal, e.g. for suppressing echoes for one or both directions of traffic
- H04M9/087—Two-way loud-speaking telephone systems with means for conditioning the signal, e.g. for suppressing echoes for one or both directions of traffic using different frequency bands for transmitting and receiving paths ; using phase shifting arrangements
Definitions
- the present invention relates to an echo-cancellation technique for reducing acoustic echo components included in an acoustic signal detected by a microphone, and more particularly to an echo-cancellation technique for reducing an acoustic echo component by using an adaptive filter.
- a sound e.g., a sender's voice or background noise
- a microphone of one of the voice telecommunications devices is played back by a speaker of the other voice telecommunications device after being transmitted via an electrical communication network to the other voice telecommunications device on the other end of the communication.
- the played-back sound goes around from the speaker directly or indirectly to the microphone and is collected, the played-back sound is returned to the voice telecommunications device which is the transmission source.
- Such a played-back sound returned to the voice telecommunications device which is the transmission source is called the “acoustic echo.”
- an echo canceller which employs an adaptive filter is used widely.
- an adaptive filter successively estimates, i.e., learns the characteristics of a transmission path (also referred to as an “echo path” hereafter) via which an acoustic echo travels from a speaker to a microphone, and generates a false echo signal (also referred to as an “echo replica”). Then, by removing the false echo signal from the voice-transmission signal to be transmitted to the voice telecommunications device which is the transmission source, the acoustic echo component included in the voice-transmission signal can be reduced. Therefore, it can be said that the performance of the adaptive filter determines the echo cancellation performance of the echo canceller.
- a cause of an impediment to such the learning of the echo path characteristics which is carried out by the adaptive filter there can be considered mixing of a disturbance signal into an acoustic signal detected by the microphone.
- a near end speaker's voice or background noise which is mixed into an acoustic echo is a disturbance signal, and is an impediment to the learning of the echo path characteristics.
- a double talk state in which a near end speaker's voice and an acoustic echo are mixed into the microphone becomes a problem as a cause of an impediment to the learning of the echo path characteristics.
- an echo canceller including an adaptive filter and a double talk detecting circuit for determining the presence or absence of occurrence of a double talk.
- This type of echo canceller is disclosed by, for example, Nonpatent Literature 1 mentioned below.
- An acoustic echo canceller disclosed by Nonpatent Literature 1 includes a double talk detecting circuit that monitors the amount of echo cancellation as a detection parameter, and that, when the amount of echo cancellation becomes equal to or larger than a predetermined amount, determines that a double talk has occurred and stops updating of an adaptive filter coefficient.
- an echo canceller that in order to improve the robustness against a disturbance signal of the learning of the echo path characteristics, includes two types of filters is also known.
- This type of echo canceller is disclosed by, for example, Nonpatent Literature 2 as listed below.
- the echo canceller disclosed by Nonpatent Literature 2 includes a background (BG) filter that learns the echo path characteristics, and a foreground (FG) filter in which a result of the learning carried out by the BG filter is incorporated over a certain period of time, and uses the FG filter for echo cancellation.
- BG background
- FG foreground
- Patent Literature 1 Japanese Patent Application Publication No. 1994(H06)-338827.
- Patent Literature 1 Japanese Patent Application Publication No. 1994(H06)-338827 (for example, see FIG. 1 and paragraphs [0014] to [0020])
- Non-Patent Literature 1 Fujii Kensaku, Ohga Juro, “Double-Talk Detection Method with Detecting Echo Path Fluctuation,” IEICE Transactions (A), vol. J78-A, no. 3, pp. 314-322, March 1995.
- Non-Patent Literature 2 K. Ochiai, T. Araseki, and T. Ogihara, “Echo canceller with two echo path models,” IEEE Transactions on Communications, vol. COM-25, no. 6, pp. 589-595, June 1977.
- a conventional echo canceller including such a double talk detecting circuit as above, there is a case in which the amount of echo cancellation extremely decreases when an adaptive filter carries out erroneous learning as a result of erroneous determination carried out by the double talk generating circuit. Further, in a conventional echo canceller including such two types of filters as above, there is a case in which the robustness against a disturbance signal cannot be maintained when both the two types of filters carry out erroneous learning as a result of the occurrence of a double talk. Therefore, the above-mentioned conventional echo cancellers do not necessarily provide echo cancellation performance which is stable under an environment in which a double talk occurs.
- an echo canceller device which includes: an adaptive filtering unit configured to perform a filtering operation using a filter-coefficient group on a sequence of reception signals input thereto, and to adaptively update the filter-coefficient group on the basis of a voice-transmission signal input thereto from a sound collector; a false echo calculator configured to acquire current and previous filter-coefficient groups which have been used by the adaptive filtering unit, and to calculate false echo signals by performing filtering operations using the current and previous filter-coefficient groups, respectively, on the sequence of reception signals; an evaluation value calculator configured to calculate evaluated values of an echo cancellation quantity corresponding to the respective false echo signals on the basis of the voice-transmission signal and the false echo signals; a false echo selector configured to select, as an estimated echo component, a false echo signal from among the false echo signals on the basis of the evaluated values of the echo cancellation quantity; and a signal output unit configured to output a residual signal indicating a difference between the estimated echo component and the voice-transmission signal.
- an echo canceller device which includes: an adaptive filtering unit configured to perform a filtering operation using a filter-coefficient group on a sequence of reception signals input thereto, and to adaptively update the filter-coefficient group on the basis of a voice-transmission signal input thereto from a sound collector; a false echo calculator configured to acquire current and previous filter-coefficient groups which have been used by the adaptive filtering unit, and to calculate false echo signals by performing filtering operations using the filter-coefficient groups, respectively, on the sequence of reception signals; an evaluation value calculator configured to calculate evaluated values of an echo cancellation quantity corresponding to the respective false echo signals on the basis of the voice-transmission signal and the false echo signals; a filter selector configured to select a new filter-coefficient group from among the current and previous filter-coefficient groups on the basis of the evaluated values of the echo cancellation quantity; a foreground filter configured to generate an estimated echo component by performing a filtering operation using the new filter-coefficient group on the sequence of reception signals; and
- a voice telecommunications device which includes: a communication function unit configured to communicate with another voice telecommunications device via an electrical communication network; and the echo canceller device according to the above first or second aspect.
- the erroneous learning can be corrected and good echo-cancellation performance can be maintained.
- FIG. 1 is a diagram showing the schematic configuration of a communication system according to Embodiment 1 of the present invention
- FIG. 2 is a block diagram showing the schematic configuration of an echo canceller of Embodiment 1;
- FIG. 3 is a graph showing an example of evaluated values of an echo cancellation quantity according to Embodiment 1;
- FIG. 4 is a flow chart showing an example of the procedure of echo cancellation processing according to Embodiment 1;
- FIG. 5 is a diagram showing an example of the hardware configuration of the echo canceller of Embodiment 1;
- FIG. 6 is a diagram showing another example of the hardware configuration of the echo canceller of Embodiment 1;
- FIG. 7 is a block diagram showing the schematic configuration of an echo canceller of Embodiment 2 of the present invention.
- FIG. 8 is a flow chart showing an example of the procedure of echo cancellation processing according to Embodiment 2.
- FIG. 1 is a diagram showing the schematic configuration of a communication system including two voice telecommunications devices 1 A and 1 B each having an echo canceller 10 of Embodiment 1 according to the present invention. As shown in FIG. 1 , the two voice telecommunications devices 1 A and 1 B are connected to each other via a communication line network NW, and have the same configuration.
- Each of the voice telecommunications devices 1 A and 1 B is connected to both a sound collector MK including a microphone, and a speaker SP that outputs a played-back sound which is an acoustic wave, and includes a communication function unit 11 that communicates with a voice telecommunications device on the other end of the communication, and an echo canceller 10 that reduces an acoustic echo component in a voice-transmission signal to be transmitted to the voice telecommunications device on the other end of the communication.
- the sound collector MK When the sound collector MK is acoustically coupled to the speaker SP, more specifically, when an acoustic wave outputted by the speaker SP propagates through a medium such as air, a fluid, or a solid, and goes around to the sound collector MK and is detected, the sound collector MK can detect the acoustic wave as an acoustic echo.
- the communication line network NW for example, a wide area network such as a telephone network, a mobile communication network, or the Internet, or a small-scale communication network such as a LAN (Local Area Network) is provided.
- FIG. 2 is a block diagram showing the schematic configuration of the echo canceller 10 of Embodiment 1.
- the echo canceller 10 includes a signal input unit S in that samples a acoustic-sensor signal inputted thereto from the sound collector MK, to output a voice-transmission signal y(n), a line side signal output unit S out that outputs a residual signal e(n) which is the voice-transmission signal in which an acoustic echo component is reduced to the communication function unit 11 , a line side signal input unit R in to which a reception signal x(n) received by the communication function unit 11 is inputted, and a signal output unit R out that outputs the reception signal x(n) to the speaker SP. All of the voice-transmission signal y(n), the reception signal x(n), and the residual signal e(n) are discrete time signals, and n is an integer specifying a discrete sampling time T n .
- the echo canceller 10 also includes an adaptive filtering unit 20 that performs a filtering operation using a filter-coefficient group w(n) in a time domain on a sequence of inputted reception signals x(n) (also referred to as a “reception signal sequence” hereafter).
- This adaptive filtering unit 20 also has a function of adaptively updating the filter-coefficient group w(n) dependently on the voice-transmission signal y(n), in accordance with a well-known adaptation algorithm such as an LMS (Least Mean Square) method or an NLMS (Normalized Least Mean Square) method.
- the adaptive filtering unit 20 is configured so as to include an adaptive filter 21 that performs a convolution operation on the reception signal sequence and the filter-coefficient group w(n), a subtractor 22 that subtracts a filter output d(n) of this adaptive filter 21 from the voice-transmission signal y(n) to generate an error signal ⁇ (n), and an adaptation algorithm unit (AAL) 23 that adaptively updates the filter-coefficient group w(n) dependently on this error signal ⁇ (n), as shown in FIG. 2 .
- AAL adaptation algorithm unit
- the reception signal sequence X(n) can be expressed by an N-dimensional vector given by the following equation (1).
- X ( n ) [ x ( n ), x ( n ⁇ 1), . . . , x ( n ⁇ N+ 1)] T (1)
- N is an integer equal to or larger than 3
- T shows a transposition that transforms a row vector with one row and N columns into a column vector with N rows and one column.
- the filter-coefficient group w(n) used in the convolution operation can be expressed by an N-dimensional vector given by the following equation (2).
- w ( n ) [ w 0 ( n ), w 1 ( n ), . . . , w N ⁇ 1 ( n )] T (2)
- the adaptive filter 21 can calculate the filter output d(n) by performing a convolution operation on the reception signal sequence X(n) and the filter-coefficient group w(n) in accordance with the following equation (3).
- d ( n ) ⁇ w ( n ) ⁇ T X ( n ) (3)
- the subtractor 22 calculates the error signal ⁇ (n) by subtracting the filter output d(n) from the voice-transmission signal y(n), as shown in the following equation.
- ⁇ ( n ) y ( n ) ⁇ w ( n ) ⁇ T X ( n ) (4)
- the adaptation algorithm unit 23 successively corrects the filter-coefficient group w(n) in such a way that the size of the error signal ⁇ (n) is minimized under a predetermined condition.
- the adaptation algorithm unit 23 can derive a new filter-coefficient group w(n+1) in accordance with the following equation (5).
- ⁇ (n ⁇ 1) is a coefficient at a sampling time T n ⁇ 1 , and, for example, is given by the following equation (6).
- ⁇ ⁇ ( n - 1 ) ⁇ N ⁇ ⁇ ⁇ x + ⁇ ⁇ ⁇ ⁇ X ⁇ ( n - 1 ) ⁇ T ⁇ X ⁇ ( n - 1 ) + ⁇ ( 6 )
- ⁇ is a step size for adjusting the amount of update of the filter-coefficient group
- ⁇ is a very small number preventing the denominator of the middle expression of the equation (6) from becoming zero.
- N is equal to the filter length of the adaptive filter 21
- ⁇ x is a variance of the reception signal.
- N ⁇ x can be approximated by the square of the norm of a reception signal sequence X(n ⁇ 1):
- 2 ( ⁇ X(n ⁇ 1) ⁇ T ⁇ X(n ⁇ 1))
- the adaptation algorithm unit 23 then updates the filter-coefficient group w(n) by replacing the current-time filter-coefficient group w(n) in the adaptive filter 21 with the new filter-coefficient group w(n+1). Because the filter-coefficient group w(n) indicates an estimated amount of the characteristics of an echo path extending from the speaker SP to the sound collector MK, the successive updating of the filter-coefficient group w(n) means that successive estimation of the echo path characteristics, i.e., learning of the echo path characteristics is carried out.
- the adaptation algorithm is not limited to an LMS method and an NLMS method.
- the configuration of the adaptive filtering unit 20 can be modified suitably in such a way that the adaptive filtering unit operates in accordance with another adaptation algorithm such as an affine projection method or an RLS (Recursive Least Square) method.
- the adaptive filtering unit 20 according to the present embodiment has a function of successively updating the filter-coefficient group w(n) in a time domain while performing an adaptive filtering operation in a time domain, the present invention is not limited to this example.
- an adaptive filter configuration for successively updating a filter-coefficient group in a frequency domain while performing an adaptive filtering operation in a frequency domain can be used.
- the echo canceller 10 includes an echo estimator 30 that stabilizes the learning carried out by the above-mentioned adaptive filtering unit 20 , and that calculates an estimated echo component d (p) (n).
- This echo estimator 30 is configured so as to include a filter coefficient buffer 31 , a false echo calculator 32 , a false echo buffer 33 , an evaluation value calculator 34 , and a buffer controller 35 .
- the filter coefficient buffer 31 Every time the filter-coefficient group in the adaptive filter 21 is updated, the filter coefficient buffer 31 temporarily stores the updated filter-coefficient group supplied thereto from the adaptive filter 21 . Further, the filter coefficient buffer 31 temporarily stores a fixed number of the current-time and previous-time filter-coefficient groups w (1) (n), w (2) (n), . . . , and w (M) (n) (M is an integer equal to or larger than 3) which have been used by the adaptive filtering unit 20 . It is assumed in this specification that “current-time” means the latest sampling time, and “previous-time” means a sampling time prior to the latest sampling time.
- a relationship between the current and previous filter-coefficient groups w (1) (n), w (2) (n), . . . , and w (M) (n) stored in the filter coefficient buffer 31 , and the filter-coefficient groups used by the adaptive filtering unit 20 is, for example, as shown in the following equation (7).
- [ w (1) ( n ), w (2) ( n ), . . . , w (M) ( n )] [ w ( n ), w ( n ⁇ 1), . . . , w ( n ⁇ M+ 1)] (7)
- the k-th filter-coefficient group w (k) (n) stored in the filter coefficient buffer 31 is the filter-coefficient group w(n ⁇ k+1) at the sampling time T n ⁇ k+l .
- the filter coefficient buffer 31 deletes the oldest filter-coefficient group from the stored filter-coefficient groups, to update the filter-coefficient groups, as shown in the following equation (8).
- [ w (1) ( n+ 1), w (2) ( n+ 1), . . . , w (M) ( n+ 1)] [ w ( n+ 1), . . . , w ( n ⁇ M+ 2)] (8)
- the sampling times T n and T n ⁇ 1 of the current and previous filter-coefficient groups w(n) and w(n ⁇ 1) stored in the filter coefficient buffer 31 be successive as shown in the above equation (7), the present invention is not limited to this example.
- the current and previous filter-coefficient groups at non-successive sampling times can be alternatively stored in the filter coefficient buffer 31 , and can be used.
- the false echo calculator 32 calculates M false echo signals d (1) (n) to d (M) (n) by performing a filtering operation using each of the filter-coefficient groups w (1) (n) to w (M) (n) read from the filter coefficient buffer 31 on the reception signal sequence X(n) M times.
- the k-th false echo signal d (k) (n) is calculated by performing a convolution operation using the k-th filter-coefficient group w (k) (n) in accordance with the following equation (9).
- d (k) ( n ) ⁇ w (k) ( n ) ⁇ T X ( n ) (9)
- the false echo buffer 33 temporarily stores the false echo signals d (1) (n) to d (M) (n) which are supplied thereto from the false echo calculator 32 .
- the evaluation value calculator 34 calculates M evaluated values dEV 1 (n) to dEV M (n) of an echo cancellation quantity on the basis of both the false echo signals d (1) (n) to d (m) (n) read from the false echo buffer 33 , and the current voice-transmission signal y(n).
- the k-th evaluated value dEV k (n) of the echo cancellation quantity is calculated as the square of the ratio of the voice-transmission signal y(n) to the difference between the voice-transmission signal y(n) and the k-th false echo signal d (k) (n).
- EV k ( n ) ⁇ y ( n ) ⁇ 2 / ⁇ y ( n ) ⁇ d (k) ( n ) ⁇ 2 (10)
- the evaluation value calculator 34 further calculates an evaluated value dEV k (n) of the echo cancellation quantity expressed in decibels, as shown below as equation (11), and supplies the evaluated value dEV k (n) of the echo cancellation quantity to the buffer controller 35 .
- the buffer controller 35 has a filter selector 35 f and a false echo selector 35 e .
- the false echo selector 35 e selects, as the estimated echo component d (p) (n), one optimal false echo signal from the M false echo signals d (1) (n) to d (M) (n) on the basis of the M evaluated values dEV 1 (n) to dEV M (n) of the echo cancellation quantity supplied thereto from the evaluation value calculator 34 , and supplies a control signal Ce showing the selection result to the false echo buffer 33 .
- the false echo buffer 33 supplies the estimated echo component d (p) (n) to the subtractor 25 in accordance with this control signal Ce.
- the subtractor 25 receives both the voice-transmission signal y(n) and the estimated echo component d (p) (n), and subtracts the estimated echo component d (p) (n) from the voice-transmission signal y(n) to generate a residual signal e(n), as shown in the following equation (12).
- e ( n ) y ( n ) ⁇ d (p) ( n ) (12)
- This residual signal e(n) is outputted to the communication function unit 11 by the line side signal output unit S out .
- FIG. 3 is a graph showing an example of evaluated values dEV 1 to dEV 5 of an echo cancellation quantity expressed in decibels.
- the false echo selector 35 e can select, as the estimated echo component d (p) (n), a false echo signal corresponding to the largest evaluated value dEV 3 of the echo cancellation quantity among the evaluated values dEV 1 to dEV 5 of the echo cancellation quantity.
- the filter selector 35 f selects a filter-coefficient group w (p) (n) from the filter-coefficient groups w (1) (n) to w (M) (n) stored in the filter coefficient buffer 31 on the basis of the M values dEV 1 (n) to dEV M (n) of the echo cancellation quantity.
- the filter selector 35 f can select, as a new filter-coefficient group, a filter-coefficient group w (p) (n) corresponding to the largest evaluated value dEV p (n) of the echo cancellation quantity among the evaluated values dEV 1 (n) to dEV M (n) of the echo cancellation quantity, from the filter-coefficient groups w (1) (n) to w (M) (n).
- the filter selector 35 f then supplies a control signal Cf showing the selection result to the filter coefficient buffer 31 .
- the filter coefficient buffer 31 supplies the new filter-coefficient group w (p) (n) to the adaptive filter 21 in accordance with this control signal Cf. Accordingly, the adaptive filter 21 updates the filter-coefficient group w(n) currently set thereto by replacing the filter-coefficient group w(n) with the new filter-coefficient group w (p) (n). Therefore, even when the degree of accuracy of the estimation of the echo path characteristics decreases in the adaptive filtering unit 20 , and erroneous learning of the echo path characteristics occurs, the filter-coefficient group w(n) can be replaced by a previous filter-coefficient group which has been used when the learning could be carried out normally.
- an advantage of providing an improvement in the stabilization of the learning of the echo path characteristics which is carried out by the adaptive filtering unit 20 is provided. Further, the robustness against a disturbance signal, such as a near end speaker's voice or background noise, which is mixed into the acoustic echo is also improved.
- FIG. 4 is a flow chart schematically showing an example of the procedure of the echo cancellation processing carried out by the echo canceller 10 .
- the adaptive filtering unit 20 stands by until a reception signal sequence X(n) is inputted at a sampling time T n (when NO in step ST 1 ).
- the adaptive filtering unit 20 performs a filtering operation as mentioned above to update the filter-coefficient group (step ST 2 ).
- the updated filter-coefficient group w(n) is supplied to the filter coefficient buffer 31 .
- the filter coefficient buffer 31 updates the filter-coefficient groups stored therein (step ST 3 ).
- the false echo calculator 32 sets the filter-coefficient group number k to “1” (step ST 4 ), and performs a filtering operation using the k-th filter-coefficient group w (k) (n) read from the filter coefficient buffer 31 , to generate a false echo signal d (k) (n) (step ST 5 ).
- This false echo signal d (k) (n) is stored in the false echo buffer 33 .
- the evaluation value calculator 34 calculates an evaluated value dEV k (n) of the echo cancellation quantity on the basis of both the false echo signal d (k) (n) and the current voice-transmission signal y(n) (step ST 6 ).
- This evaluated value dEV k (n) of the echo cancellation quantity is supplied to the buffer controller 35 .
- the evaluation value calculator 34 increments the number k by 1 (step ST 7 ), and, when the number k is equal to or smaller than a maximum M (when YES in step ST 8 ), returns the processing procedure to the step ST 5 .
- the processing procedure shifts to step ST 9 .
- the M evaluated values dEV 1 (n) to dEV M (n) of the echo cancellation quantity are calculated in this order (steps ST 4 to ST 8 ).
- the M evaluated values dEV 1 (n) to dEV M (n) of the echo cancellation quantity can be concurrently calculated in parallel.
- the false echo selector 35 e selects, as the estimated echo component d (p) (n), one optimal false echo signal from the M false echo signals d (1) (n) to d (M) (n) on the basis of the evaluated values dEV 1 (n) to dEV M (n) of the echo cancellation quantity (step ST 9 ).
- a control signal Ce showing this selection result is supplied to the false echo buffer 33 .
- the filter selector 35 f selects a new filter-coefficient group w (p) (n) from the filter-coefficient groups w (1) (n) to w (M) (n) stored in the filter coefficient buffer 31 on the basis of the evaluated values dEV 1 (n) to dEV M (n) of the echo cancellation quantity (step ST 10 ).
- a control signal Cf showing this selection result is supplied to the filter coefficient buffer 31 .
- the filter coefficient buffer 31 replaces the filter-coefficient group w(n) in the adaptive filtering unit 20 with the new filter-coefficient group w (p) (n) by supplying the new filter-coefficient group w (p) (n) specified by the control signal Cf to the adaptive filter 21 (step ST 11 ).
- the false echo buffer 33 causes the subtractor 25 to generate a residual signal e(n) by supplying the estimated echo component d (p) (n) specified by the control signal Ce to the subtractor 25 (step ST 12 ). After that, the line side signal output unit S out outputs the residual signal e(n) to the communication function unit 11 (step ST 13 ).
- steps ST 9 , ST 10 , ST 11 , and ST 12 do not have to be processed in this order.
- the steps ST 9 and ST 10 can be performed simultaneously in parallel, and the steps ST 11 and ST 12 can be performed simultaneously in parallel.
- the step ST 11 can be performed before the step ST 10 is performed.
- the hardware configuration of the above-mentioned echo canceller 10 can be implemented by, for example, a computer, such as a workstation or a mainframe, in which a CPU (Central Processing Unit) is included.
- a computer such as a workstation or a mainframe, in which a CPU (Central Processing Unit) is included.
- the hardware configuration of the above-mentioned echo canceller 10 can be implemented by an LSI (Large Scale Integrated circuit) such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Gate Array).
- LSI Large Scale Integrated circuit
- DSP Digital Signal Processor
- ASIC Application Specific Integrated Circuit
- FPGA Field-Programmable Gate Array
- FIG. 5 is a block diagram showing an example of the hardware configuration of the echo canceller 10 which is configured using an LSI such as a DSP, an ASIC, or an FPGA.
- the echo canceller 10 is comprised of a signal processing circuit 50 , an audio input/output unit 51 , an input/output unit 52 on the communication-line side, a recording medium 53 , and a signal path 54 such as a bus.
- the audio input/output unit 51 is an interface circuit that implements the functions of the signal input unit S in and the signal output unit R out
- the input/output unit 52 on the communication-line side is an interface circuit that implements the functions of the line side signal output unit S out and the line side signal input unit R in .
- the functions of the adaptive filtering unit 20 , the echo estimator 30 , and the subtractor 25 can be implemented by the signal processing circuit 50 and the recording medium 53 .
- the recording medium 53 can be used as the filter coefficient buffer 31 and the false echo buffer 33 .
- a volatile memory such as an SDRAM (Synchronous DRAM), an HDD (hard disk drive), or an SSD (solid-state drive) can be used.
- FIG. 6 is a block diagram showing an example of the hardware configuration of the echo canceller 10 which is configured using a computer.
- the echo canceller 10 is comprised of a processor 60 in which a CPU 60 c is included, a RAM (Random Access Memory) 61 , a ROM (Read Only Memory) 62 , an audio input/output unit 63 , an input/output unit 64 on the communication-line side, a recording medium 65 , and a signal path 66 such as a bus.
- the audio input/output unit 63 is an interface circuit that implements the functions of the signal input unit S in and the signal output unit R out
- the input/output unit 64 on the communication-line side is an interface circuit that implements the functions of the line side signal output unit S out and the line side signal input unit R in .
- the functions of the adaptive filtering unit 20 , the echo estimator 30 , and the subtractor 25 can be implemented by the processor 60 and the recording medium 65 .
- the recording medium 65 can be used as the filter coefficient buffer 31 and the false echo buffer 33 .
- the processor 60 can carry out the same signal processing as that carried out by the adaptive filtering unit 20 , the echo estimator 30 , and the subtractor 25 by using the RAM 61 as a memory for operations, and operating in accordance with a computer program read from the ROM 62 .
- the recording medium 65 for example, a volatile memory such as an SDRAM, an HDD, or an SSD can be used.
- the echo canceller 10 of Embodiment 1 can generate a residual signal e(n) by selectively using an estimated echo component d (p) (n) from the false echo signals stored in the false echo buffer 33 even when the accuracy of estimation of the echo path characteristics degrades in the adaptive filtering unit 20 and erroneous learning of the echo path characteristics occurs. Therefore, echo cancellation performance which is stable under an environment in which a double talk occurs can be achieved. Further, the filter-coefficient group w(n) in the adaptive filtering unit 20 can also be replaced by a previous filter-coefficient group which has been used when the learning could be carried out normally. As a result, the learning of the echo path characteristics which is carried out by the adaptive filtering unit 20 can be more stabilized compared with conventional techniques.
- the robustness against a disturbance signal such as a near end speaker's voice or background noise, which is mixed into the acoustic echo is improved.
- a disturbance signal such as a near end speaker's voice or background noise
- the echo canceller 10 can maintain echo cancellation performance which is stable even under an environment in which a double talk or an echo-path variation occurs.
- FIG. 7 is a block diagram showing the schematic configuration of an echo canceller 10 A of Embodiment 2.
- a voice telecommunications device can be configured by a combination of this echo canceller 10 A and a communication function unit 11 shown in FIG. 1 , like in the case of the echo canceller 10 of Embodiment 1 described above.
- the echo canceller 10 A of Embodiment 2 includes a signal input unit S in , a signal output unit R out , a line side signal output unit S out , and a line side signal input unit R in , and also includes an adaptive filtering unit 20 and a subtractor 25 , like the echo canceller 10 according to Embodiment 1 described above.
- the echo canceller 10 A includes an echo estimator 30 A.
- This echo estimator 30 A has a filter coefficient buffer 31 , a false echo calculator 32 , and an evaluation value calculator 34 , like the echo estimator 30 according to Embodiment 1 described above, and further has a buffer controller 35 A including a filter selector 35 f , and an FG (foreground) filter 36 .
- the FG filter 36 generates an estimated echo component d FG (n) by performing a filtering operation using a filter-coefficient group w FG (n) on a reception signal sequence X(n).
- the FG filter 36 can generate an estimated echo component d FG (n) by performing a convolution operation on the reception signal sequence X(n) and the filter-coefficient group w FG (n) in accordance with the following equation (13).
- d FG ( n ) ⁇ w FG ( n ) ⁇ T X ( n ) (13)
- the subtractor 25 subtracts the estimated echo component d FG (n) from a voice-transmission signal y(n) to generate a residual signal e(n). This residual signal e(n) is outputted to the communication function unit 11 by the line side signal output unit S out .
- FIG. 8 is a flow chart schematically showing an example of the procedure of the echo cancellation processing carried out by the echo canceller 10 A. Because the details of operations of steps ST 1 to ST 11 of FIG. 8 are the same as those of the steps ST 1 to ST 11 of FIG. 3 according to Embodiment 1 described above, the explanation of the details of the operations will be omitted hereafter.
- the FG filter 36 in step ST 20 , generates an estimated echo component d FG (n) by performing a filtering operation using a new filter-coefficient group w (p) (n) acquired from the filter coefficient buffer 31 (step ST 20 ).
- the FG filter 36 can then generate an estimated echo component d FG (n) by performing a convolution operation on the updated filter-coefficient group w FG (n) and the reception signal sequence X(n).
- the FG filter 36 can generate a combination filter-coefficient group by performing a linear combination of the new filter-coefficient group w (p) (n) and the previous filter-coefficient group w FG (n ⁇ 1) which has been used by the FG filter 36 , and replace the previous filter-coefficient group w FG (p) (n ⁇ 1) with the combination filter-coefficient group w (p) (n).
- the FG filter 36 can then generate an estimated echo component d FG (n) by performing a filtering operation using this combination filter-coefficient group w FG (n).
- the combination filter-coefficient group w FG (n) can be calculated in accordance with the following equation (14).
- w FG ( n ) (1 ⁇ ) ⁇ w FG ( n ⁇ 1)+ ⁇ w (p) ( n ) (14)
- the weighting factor ⁇ is called an oblivion coefficient. It is preferable to set the oblivion coefficient ⁇ to a value close to zero. For example, it is desirable that the oblivion coefficient is larger than zero and falls within a range smaller than 0.1. Because the filter-coefficient group of the adaptive filtering unit 20 is incorporated in the FG filter 36 over a certain period of time by using such the combination filter-coefficient group as above, an improvement in the robustness against a double talk can be provided.
- the subtractor 25 subtracts the estimated echo component d FG (n) from the voice-transmission signal y(n) to generate a residual signal e(n) (step ST 21 ), and the line side signal output unit S out outputs the residual signal e(n) to the communication function unit 11 (step ST 22 ).
- step ST 23 when the echo cancellation processing is continued (when YES in step ST 23 ), the processing procedure returns to the step ST 1 . In contrast, when the echo cancellation processing is not continued (when NO in step ST 23 ), the echo cancellation processing is ended.
- the hardware configuration of the echo canceller 10 A can be implemented by, for example, a computer, such as a workstation or a mainframe, in which a CPU is included.
- the hardware configuration of the echo canceller 10 A can be implemented by an LSI such as a DSP, an ASIC, or an FPGA.
- the hardware configuration of the echo canceller 10 A can be alternatively implemented by the configuration shown in FIG. 5 or 6 , like in the case of Embodiment 1.
- an optimal filter-coefficient group w (p) (n) is selected from the current and previous filter-coefficient groups w (1) (n) to w (M) (n) which have been used by the adaptive filtering unit 20 , and this filter-coefficient group w (p) (n) is incorporated in the filter-coefficient group w FG (n) of the FG filter 36 .
- the FG filter 36 generates an estimated echo component d FG (n) by using this filter-coefficient group w FG (n).
- the filter-coefficient group w(n) in the adaptive filtering unit 20 can also be replaced by a previous filter-coefficient group which has been used when the learning could be carried out normally. Therefore, the robustness against a disturbance signal can be more improved compared with conventional techniques. Therefore, stable echo cancellation performance can be achieved.
- the filter coefficient buffer 31 according to any of Embodiments 1 and 2 is configured so as to store three or more filter-coefficient groups w (1) (n) to w (M) (n).
- the present invention is not limited to this example, and the filter coefficient buffer can be configured so as to store two filter-coefficient groups w (1) (n) and w (2) (n).
- the evaluation value calculator 34 calculates two evaluated values dEV 1 (n) and dEV 2 (n) of the echo cancellation quantity.
- Embodiments 1 and 2 as described above can be made, a change can be made in an arbitrary component according to any of the above-mentioned embodiments, or an arbitrary component according to any of the above-mentioned embodiments can be omitted.
- the echo canceller device and the voice telecommunications device according to the present invention can be used for, for example, a fixed-line phone, a mobile phone with a loud speaking function, a handsfree communication system, and a video conferencing system.
- 1 A, 1 B voice telecommunications devices; 10 , 10 A: echo cancellers; 11 : communication function unit; 20 : adaptive filtering unit; 21 : adaptive filter; 22 : subtractor; 23 : adaptation algorithm unit (AAL); 25 : subtractor; 30 , 30 A: echo estimators; 31 : filter coefficient buffer; 32 : false echo calculator; 33 : false echo buffer; 34 : evaluation value calculator; 35 , 35 A: buffer controller; 35 e : false echo selector; 35 f : filter selector; 36 : FG (foreground) filter; 50 : signal processing circuit; 51 : audio input/output unit; 52 : input/output unit on the communication-line side; 53 : recording medium; 54 : signal path; 60 : processor; 60 c : CPU; 61 : RAM; 62 : ROM; 63 : audio input/output unit; 64 : line side input output unit; 65 : recording medium; 66 : signal path; MK: sound collector; SP:
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Telephone Function (AREA)
- Cable Transmission Systems, Equalization Of Radio And Reduction Of Echo (AREA)
- Circuit For Audible Band Transducer (AREA)
Abstract
Description
X(n)=[x(n),x(n−1), . . . , x(n−N+1)]T (1)
w(n)=[w 0(n),w 1(n), . . . ,w N−1(n)]T (2)
d(n)={w(n)}T X(n) (3)
δ(n)=y(n)−{w(n)}T X(n) (4)
w(n+1)=w(n)+μ(n−1)·e(n−1)·X(n−1) (5)
[w (1)(n),w (2)(n), . . . ,w (M)(n)]=[w(n),w(n−1), . . . ,w(n−M+1)] (7)
[w (1)(n+1),w (2)(n+1), . . . ,w (M)(n+1)]=[w(n+1), . . . ,w(n−M+2)] (8)
d (k)(n)={w (k)(n)}T X(n) (9)
EVk(n)={y(n)}2 /{y(n)−d (k)(n)}2 (10)
e(n)=y(n)−d (p)(n) (12)
d FG(n)={w FG(n)}T X(n) (13)
w FG(n)=(1−γ)·w FG(n−1)+γ·w (p)(n) (14)
Claims (8)
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2015/082109 WO2017085760A1 (en) | 2015-11-16 | 2015-11-16 | Echo canceler and communication device |
Publications (2)
Publication Number | Publication Date |
---|---|
US20180343345A1 US20180343345A1 (en) | 2018-11-29 |
US10367949B2 true US10367949B2 (en) | 2019-07-30 |
Family
ID=58718546
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/775,201 Expired - Fee Related US10367949B2 (en) | 2015-11-16 | 2015-11-16 | Echo canceller device and voice telecommunications device |
Country Status (4)
Country | Link |
---|---|
US (1) | US10367949B2 (en) |
JP (1) | JP6279172B2 (en) |
DE (1) | DE112015007019B4 (en) |
WO (1) | WO2017085760A1 (en) |
Families Citing this family (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10367949B2 (en) * | 2015-11-16 | 2019-07-30 | Mitsubishi Electric Corporation | Echo canceller device and voice telecommunications device |
US10200540B1 (en) * | 2017-08-03 | 2019-02-05 | Bose Corporation | Efficient reutilization of acoustic echo canceler channels |
US10542153B2 (en) | 2017-08-03 | 2020-01-21 | Bose Corporation | Multi-channel residual echo suppression |
US10594869B2 (en) * | 2017-08-03 | 2020-03-17 | Bose Corporation | Mitigating impact of double talk for residual echo suppressors |
EP3692704B1 (en) | 2017-10-03 | 2023-09-06 | Bose Corporation | Spatial double-talk detector |
CN107910014B (en) * | 2017-11-23 | 2021-11-09 | 苏州科达科技股份有限公司 | Echo cancellation test method, device and test equipment |
US10964305B2 (en) | 2019-05-20 | 2021-03-30 | Bose Corporation | Mitigating impact of double talk for residual echo suppressors |
CN114760389B (en) * | 2022-06-16 | 2022-09-02 | 腾讯科技(深圳)有限公司 | Voice communication method and device, computer storage medium and electronic equipment |
Citations (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH06338827A (en) | 1993-05-28 | 1994-12-06 | Matsushita Electric Ind Co Ltd | Echo controller |
US5592548A (en) * | 1995-05-31 | 1997-01-07 | Qualcomm Incorporated | System and method for avoiding false convergence in the presence of tones in a time-domain echo cancellation process |
US5859914A (en) * | 1996-07-23 | 1999-01-12 | Nec Corporation | Acoustic echo canceler |
US5905717A (en) * | 1995-11-29 | 1999-05-18 | Nec Corporation | Echo canceller for a packet signal |
US6078567A (en) * | 1994-11-10 | 2000-06-20 | British Telecommunications Plc | Echo cancellation using cross-correlation of buffered receive and transmit sample segments to determine cancelling filter coefficients |
US6181753B1 (en) * | 1997-04-30 | 2001-01-30 | Oki Electric Industry Co., Ltd. | Echo/noise canceler with delay compensation |
US6185300B1 (en) * | 1996-12-31 | 2001-02-06 | Ericsson Inc. | Echo canceler for use in communications system |
US6192126B1 (en) * | 1996-11-27 | 2001-02-20 | Nokia Mobile Phones Ltd. | Double talk detector, method for double talk detection and device incorporating such a detector |
JP2008124914A (en) | 2006-11-14 | 2008-05-29 | Nippon Telegr & Teleph Corp <Ntt> | Echo cancelling apparatus, method and program, and recording medium therefor |
JP2009159274A (en) | 2007-12-26 | 2009-07-16 | Toshiba Corp | Echo suppression processing apparatus |
US20100191527A1 (en) * | 2007-10-12 | 2010-07-29 | Fujitsu Limited | Echo suppressing system, echo suppressing method, recording medium, echo suppressor, sound output device, audio system, navigation system and mobile object |
US20140112467A1 (en) * | 2012-10-23 | 2014-04-24 | Interactive Intelligence, Inc. | System and Method for Acoustic Echo Cancellation |
US20150078567A1 (en) * | 2013-09-18 | 2015-03-19 | Imagination Technologies Limited | Varying Adaptive Filter Step Size in Acoustic Echo Cancellation |
US20180343345A1 (en) * | 2015-11-16 | 2018-11-29 | Mitsubishi Electric Corporation | Echo canceller device and voice telecommunications device |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2643852B2 (en) * | 1994-08-31 | 1997-08-20 | 日本電気株式会社 | Echo canceller |
JPH08256089A (en) * | 1995-03-17 | 1996-10-01 | Toshiba Corp | Echo canceler |
JPH10229354A (en) * | 1997-02-14 | 1998-08-25 | Fujitsu Ltd | Echo controller |
JP3640576B2 (en) * | 1999-09-07 | 2005-04-20 | 沖電気工業株式会社 | Echo canceller and operation method thereof |
JP2004266726A (en) * | 2003-03-04 | 2004-09-24 | Denso Corp | Echo canceller, method for calculating filter coefficient for echo canceling processing, and computer program |
US8761385B2 (en) * | 2004-11-08 | 2014-06-24 | Nec Corporation | Signal processing method, signal processing device, and signal processing program |
JP4834046B2 (en) * | 2008-08-27 | 2011-12-07 | 日本電信電話株式会社 | Echo erasing device, echo erasing method, echo erasing program, recording medium |
US8879720B2 (en) | 2013-03-17 | 2014-11-04 | Revolabs, Inc. | Acoustic echo cancellation using a variable length adaptive filter |
-
2015
- 2015-11-16 US US15/775,201 patent/US10367949B2/en not_active Expired - Fee Related
- 2015-11-16 WO PCT/JP2015/082109 patent/WO2017085760A1/en active Application Filing
- 2015-11-16 DE DE112015007019.2T patent/DE112015007019B4/en not_active Expired - Fee Related
- 2015-11-16 JP JP2017551405A patent/JP6279172B2/en not_active Expired - Fee Related
Patent Citations (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5463618A (en) | 1993-05-28 | 1995-10-31 | Matsushita Electric Industrial Co., Ltd. | Echo canceller |
JPH06338827A (en) | 1993-05-28 | 1994-12-06 | Matsushita Electric Ind Co Ltd | Echo controller |
US6078567A (en) * | 1994-11-10 | 2000-06-20 | British Telecommunications Plc | Echo cancellation using cross-correlation of buffered receive and transmit sample segments to determine cancelling filter coefficients |
US5592548A (en) * | 1995-05-31 | 1997-01-07 | Qualcomm Incorporated | System and method for avoiding false convergence in the presence of tones in a time-domain echo cancellation process |
US5905717A (en) * | 1995-11-29 | 1999-05-18 | Nec Corporation | Echo canceller for a packet signal |
US5859914A (en) * | 1996-07-23 | 1999-01-12 | Nec Corporation | Acoustic echo canceler |
US6192126B1 (en) * | 1996-11-27 | 2001-02-20 | Nokia Mobile Phones Ltd. | Double talk detector, method for double talk detection and device incorporating such a detector |
US6185300B1 (en) * | 1996-12-31 | 2001-02-06 | Ericsson Inc. | Echo canceler for use in communications system |
US6181753B1 (en) * | 1997-04-30 | 2001-01-30 | Oki Electric Industry Co., Ltd. | Echo/noise canceler with delay compensation |
JP2008124914A (en) | 2006-11-14 | 2008-05-29 | Nippon Telegr & Teleph Corp <Ntt> | Echo cancelling apparatus, method and program, and recording medium therefor |
US20100191527A1 (en) * | 2007-10-12 | 2010-07-29 | Fujitsu Limited | Echo suppressing system, echo suppressing method, recording medium, echo suppressor, sound output device, audio system, navigation system and mobile object |
JP2009159274A (en) | 2007-12-26 | 2009-07-16 | Toshiba Corp | Echo suppression processing apparatus |
US20140112467A1 (en) * | 2012-10-23 | 2014-04-24 | Interactive Intelligence, Inc. | System and Method for Acoustic Echo Cancellation |
US20150078567A1 (en) * | 2013-09-18 | 2015-03-19 | Imagination Technologies Limited | Varying Adaptive Filter Step Size in Acoustic Echo Cancellation |
US20180343345A1 (en) * | 2015-11-16 | 2018-11-29 | Mitsubishi Electric Corporation | Echo canceller device and voice telecommunications device |
Non-Patent Citations (3)
Title |
---|
Fujii et al., "Double-Talk Detection Method with Detecting Echo Path Fluctuation," IEICE Transactions (A), vol. J78-A, No. 3, Mar. 1995, pp. 314-322. |
International Search Report for PCT/JP2015/082109 (PCT/ISA/210) dated Feb. 9, 2016. |
Ochiai et al., "Echo Canceler with Two Echo Path Models", IEEE Transacitons on Communications, vol. Com-25, No. 6, Jun. 1977, pp. 589-595. |
Also Published As
Publication number | Publication date |
---|---|
JP6279172B2 (en) | 2018-02-14 |
DE112015007019B4 (en) | 2019-07-25 |
US20180343345A1 (en) | 2018-11-29 |
JPWO2017085760A1 (en) | 2018-04-26 |
DE112015007019T5 (en) | 2018-07-12 |
WO2017085760A1 (en) | 2017-05-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10367949B2 (en) | Echo canceller device and voice telecommunications device | |
US7856097B2 (en) | Echo canceling apparatus, telephone set using the same, and echo canceling method | |
CN105577961B (en) | Automatic tuning of gain controller | |
US8139760B2 (en) | Estimating delay of an echo path in a communication system | |
JP5049277B2 (en) | Method and system for clear signal acquisition | |
KR101250124B1 (en) | Apparatus and Method for Computing Control Information for an Echo Suppression Filter and Apparatus and Method for Computing a Delay Value | |
US8254588B2 (en) | System and method for providing step size control for subband affine projection filters for echo cancellation applications | |
CN108134863B (en) | Improved double-end detection device and detection method based on double statistics | |
US9191519B2 (en) | Echo suppressor using past echo path characteristics for updating | |
US8300802B2 (en) | Adaptive filter for use in echo reduction | |
SE511073C2 (en) | Methods and apparatus for echo estimation and suppression in telephone systems | |
JP2009105666A (en) | Loudspeaker call device | |
US20170310360A1 (en) | Echo removal device, echo removal method, and non-transitory storage medium | |
US10498389B2 (en) | Echo canceller device and voice telecommunications device | |
US20020039414A1 (en) | Acoustic echo canceler and handsfree telephone set | |
Paleologu et al. | A Kalman filter with individual control factors for echo cancellation | |
US7856087B2 (en) | Circuit method and system for transmitting information | |
JP2017098861A (en) | Echo canceller and echo cancellation method | |
JP2006279191A (en) | Echo canceller | |
US10999444B2 (en) | Acoustic echo cancellation device, acoustic echo cancellation method and non-transitory computer readable recording medium recording acoustic echo cancellation program | |
Chandra et al. | Performance Evaluation of Adaptive Algorithms for Monophonic Acoustic Echo Cancellation: A Technical | |
KR20220157475A (en) | Echo Residual Suppression | |
Hamidia et al. | A new structure for acoustic echo cancellation in double-talk scenario using auxiliary filter | |
JP6180689B1 (en) | Echo canceller apparatus, echo cancellation method, and echo cancellation program | |
JP2006148375A (en) | Echo cancellation method, echo canceller, and telephone repeater |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
AS | Assignment |
Owner name: MITSUBISHI ELECTRIC CORPORATION, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:AWANO, TOMOHARU;FURUTA, SATORU;ISHII, JUN;REEL/FRAME:046137/0351 Effective date: 20180320 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT RECEIVED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20230730 |