JP7304627B2 - Answering machine judgment device, method and program - Google Patents

Description

The present invention relates to an answering machine determination device, method, and program for determining whether or not a caller's response was an answering machine when a call was made automatically using automatic voice.

When an incoming call to the telephone is not off-hooked by the user for a certain period of time, the telephone automatically answers the telephone, plays a predetermined preset answering machine message, and then prompts the caller to leave a message, thereby prompting the caller to leave a message. 2. Description of the Related Art Telephones with an answering machine function capable of recording a message and a function capable of setting an answering machine service through a communication network have become widespread.

However, when the answering machine answers, the caller can leave his or her own message for the other party, but in addition to being charged for the call, the original purpose of the call by informing the other party of the desired business is lost. The problem was that it was not achievable.

In order to solve this problem, in Japanese Unexamined Patent Application Publication No. 2002-100003, a telephone device on the caller side measures the answering machine response time of the telephone of the called party, stores it together with the telephone number of the called party, and stores the answering time with the telephone number of the called party. There is disclosed a technique for preventing useless payment of call charges by automatically disconnecting a call just before the stored answering machine response time.

JP 2017-11623 A

However, in the technique of Patent Document 1, it is possible to prevent wasteful payment of call charges by automatically disconnecting from the second call. cannot be prevented. Also, in the first place, when the calling party makes an automatic call by a computer program or the like and makes an automatic voice call, there is a problem that it is not possible to distinguish whether the response given by the called party is a human voice or an answering machine message. there were.

When the called party's phone goes off-hook, if it is not possible to determine whether it was an answering machine or whether the matter conveyed by automated voice was not answered despite the fact that a human answered the call, call immediately. I can't decide whether to fix it or call back later. For this reason, there is a problem that the automatic voice telephone repeats call operations unnecessarily, such as immediately calling back many times even though the caller is not at home.

Therefore, in the present invention, after the call is completed, the call data is acquired from the automatic voice telephone, at least the speaker overlap feature amount and the clustering feature amount are extracted from the call data, and the decision unit generated by machine learning To prevent useless calling operation after that and to take an appropriate response by judging whether or not a response is made by an answering machine.

According to the present invention, there are provided call data acquisition means for acquiring phone call data, speaker overlap feature quantity extraction means for calculating the degree of duplication of both speakers of the call data, and clustering feature quantity calculation of the call data. Using the clustering feature extracting means, the speaker duplication feature, the clustering feature, and the presence or absence of a response by an answering machine as teacher data, it is determined whether or not the answer is a response by an answering machine generated by machine learning. determining means, wherein the determining means determines whether or not the call data is a response by an answering machine based on the overlapping speaker feature amount and the clustering feature amount extracted from the call data. A response determination device can be provided.

Furthermore, the answering machine response determination device has a call time feature amount extraction means for extracting a call time feature amount of the call data, and the determination means further uses the call time feature amount as teacher data.

Also, the call duration feature amount is the energy statistic of the call voice.

The answering machine response determination device according to the present invention further includes a voice recognition means for converting call data into text data and a response detection means for detecting a response by the answering machine from the text data by machine learning and calculating a text feature amount. text feature amount extraction means, and the determination means further uses the text feature amount as teacher data.

Furthermore, machine learning is performed using machine synthesized speech data, human speech data, and a Gaussian mixture histogram as teacher data, and the probability that the speech of the answering side in the call data is machine synthesized speech data is calculated as machine synthesized speech data. Provided is an answering machine response determination device, further comprising a machine-synthesized sound feature quantity generating means for generating as a feature quantity, wherein the determination means further uses the machine-synthesized sound feature quantity as teacher data.

The answering machine response determination method according to the present invention includes a call data acquisition step of acquiring call data from telephone calls, a speaker overlap feature amount extraction step of calculating the degree of redundancy of both speakers in the call data, and clustering of the call data. Generated by machine learning using the clustering feature amount extraction step of calculating the feature amount, the call duration feature amount, the speaker overlapping feature amount, the clustering feature amount, and the presence or absence of the answering machine as teacher data. and a judgment step of judging whether or not the call is answered by an answering machine.

Further, an answering machine response determination program to be executed by a computer as an answering machine response determination apparatus of the present invention includes a call data acquisition step of acquiring call data from a telephone call, A duplicate feature quantity extraction step, a clustering feature quantity extraction step of calculating a clustering feature quantity of call data, a call duration feature quantity, a speaker overlap feature quantity, a clustering feature quantity, and the presence or absence of an answering machine response are supervised. Provided is an answering machine response judgment program for executing a judgment step of judging whether or not a response is made by an answering machine by a judging unit generated by machine learning using data as data.

According to the present invention, it is possible to obtain a predetermined feature value from the call data and determine whether or not the callee's telephone was answered by an answering machine. If you are called, wait a few hours and try to call again during business hours if there is a high possibility that you are at home or at a store, and even if a human answers, the call will be made by an automated voice. If it is determined that the call was immediately disconnected because of a

FIG. 1 is a block diagram showing an example of a hardware configuration diagram of an answering machine response determining apparatus according to the present invention. FIG. 2 is a functional block diagram of the answering machine response determination system 1 according to the first embodiment of the present invention. FIG. 3 is a functional block diagram of answering machine response determination system 1 according to the second embodiment of the present invention. FIG. 4 is a graph showing the number-of-seconds distribution in the case of human responses. FIG. 5 is a graph showing the distribution of the number of seconds according to the machine response of the answering machine. FIG. 6 is a flow chart showing a process of generating, by machine learning, a classifier for determining whether or not an answering machine has answered on the called party side using teacher data.

EMBODIMENT OF THE INVENTION Hereinafter, the form for implementing this invention is demonstrated, referring drawings. In the present specification and drawings, constituent elements having substantially the same functions and configurations are denoted by the same reference numerals, and redundant explanations are omitted.

FIG. 1 is a block diagram showing an example of a hardware configuration diagram of an answering machine response determining apparatus according to the present invention. The hardware configuration of answering machine response determination device 10, which is a computer device shown in FIG. 1, can be realized mainly by a computer device. The answering machine response determination device 10 extracts various feature values from the call data received from the automatic voice telephone 20, and executes an answering machine response determination program for determining answering machine response, thereby determining whether or not answering machine response. make a judgment.

The answering machine response determination device 10 has an answering machine response determination unit that is generated by machine learning using call data, various feature values, and the presence or absence of answering machine responses as teacher data, and automatically generates new call data. When received from the voice telephone 20, various feature quantities are extracted and it is determined whether or not the other party's response was an answering machine response.

As shown in FIG. 1, the computer forming the answering machine response determination device 10 includes a CPU 11, a communication interface 12, a ROM 13, a RAM 14, a hard disk drive 15, an input/output interface 16, a display unit 17 connected to the input/output interface 16, A pointing device 18 and a keyboard 19 are connected to the bus. Also, an external storage device 20 such as a USB memory can be connected to the input/output interface 16 .

The display unit 17 is, for example, a display device such as a liquid crystal display. The pointing device 18 is, for example, a mouse or trackball.

When a series of processes are executed by a program, for example, a call data acquisition unit, a speaker overlap feature amount extraction unit, a clustering feature amount extraction unit, a call time feature amount extraction unit, a text feature amount extraction unit, and a machine synthesized sound feature The amount extractor and the determiner are stored in the ROM 13 or the hard disk drive 15 as an answering machine response determination program, and are executed by the CPU 11 to perform various functions. Installation by connecting an external storage device 20 such as a USB memory in which an answering machine response determination program is stored to the input/output interface 16, installation of the answering machine response determination program from the network 12 to a computer, and installation of the apparatus main body , such as the ROM 13 in which an answering machine response determination program is recorded.

FIG. 2 is a functional block diagram of the answering machine response determination system 1 according to the first embodiment of the present invention. The answering machine response determination system 1 comprises an automated voice telephone 20 and an answering machine response determination device 10 . The automatic voice telephone 20 is a telephone capable of making a call without human presence by automatically calling a predetermined telephone number and making the other party hear a predetermined message in an artificially generated voice. For example, a telephone reservation message necessary for making a reservation call to a restaurant is registered in advance in the automatic voice telephone 20, and the telephone number of the restaurant to be reserved can be obtained by appropriately combining the desired reservation date and time, the number of people, etc. with the telephone reservation message. is read out, a call is made automatically, the generated message for telephone reservation is read out by artificial voice, and the other party is asked whether the reservation is possible. The automatic voice telephone 20 may be configured by installing a computer program for making a call by referring to a telephone number stored in a database in a computer such as a server, and may be configured by a plurality of computer devices. may

The answering machine response determination device 10 receives call data recorded by the automatic voice telephone 20 via a communication network such as the Internet, extracts various feature values from the call data, and performs machine learning using teacher data. It is a device that determines whether or not the other party has responded with an answering machine message based on the determination unit.

Although the automatic voice telephone 20 and the answering machine response determination device 10 are illustrated here as separate devices, the present invention is not limited to this, and the automatic voice telephone 20 and the answering machine response determination device 10 are configured as one device. may Further, each function constituting the automatic voice telephone 20 and the answering machine response determination device 10 may be configured as an independent device.

The answering machine answering device 10 has a call data acquisition unit 101 , a speaker overlapping feature amount extraction unit 111 , a clustering feature amount extraction unit 113 and a determination unit 121 .

A call data acquisition unit 101 receives call data of a call made automatically by the automatic voice telephone 20 without relying on human hands, from the automatic voice telephone 20, and acquires the call data. The data format of the call data is limited to a specific data format, as long as it is recorded in two channels, i.e., the voice of the caller and the voice of the called party. do not have.

The speaker duplication feature quantity extraction unit 111 calculates the degree of duplication of both speakers in the call data. Specifically, since the call data is recorded on the caller side and the called party side in dual channels, the speaker overlap feature amount extraction unit 111 detects a section in which the speaker is speaking on both channels, degree is determined. The speaker overlapping feature amount extraction unit 111 has, for example, a voice activity detection (VAD), extracts speaking intervals in both the channel of the caller and the called party, and calculates the degree of overlap. do. For example, a linear model of MFCC (Mel-frequency cepstrum coefficients) may be used as the voice activity detector. The speaker overlap feature quantity extraction unit 111 performs the following calculation to calculate the overlap time (overlap) of the speaking section of the caller and the called party. Shello is the period during which the caller is speaking, Srestaurant is the period during which the called party is speaking, and |S| represents the length of the period in seconds.

The clustering feature quantity extraction unit 113 calculates a clustering feature quantity from the voice data of the channel of only the receiver side among the call data. The clustering feature amount extraction unit 113 performs clustering so as to classify responses by a machine such as an answering machine, human responses, and characteristic voice patterns of each response. Specifically, a feature value called BoAW (Bag of Audio Words) is calculated. The clustering feature quantity extraction unit 113 first calculates the MFCC feature quantity for the received call data.

Specifically, one frame is set to 0.025 seconds and shifted by 0.01 seconds to generate 100 frames per second. Generate a clustering by fitting the -means clustering. k-means generates a clustering by minimizing the following equation. x is the point in each frame, Si is the set of points contained in the i-th cluster, μi is the center of that cluster, and S is the set of all clusters.

For example, if 100 pieces of call data for 10 seconds are prepared, 10×100×100=100000 points are present, and k-means clustering is applied to these points to generate 100 clusterings. Next, the MFCC feature amount of call data for answering machine response determination is calculated, and the distance to which clustering the MFCC feature amount of each frame is close is calculated. By assigning the frame to the closest clustering, a clustering histogram is generated and clustering features are extracted. That is, the clustering feature quantity extraction unit 113 extracts the clustering feature quantity by generating clustering from the MFCC feature quantity, which is the speech feature quantity, for the voice data of the called party and further generating a clustering histogram.

The determination unit 121 is generated by machine learning using various feature values extracted from call data and the presence/absence of a response to an answering machine as teacher data. By inputting the overlapping speaker feature amount and the clustering feature amount extracted from the call data for which it is desired to determine whether or not the response was made by an answering machine to the determination unit 121 generated by machine learning, the determination unit 121 determines the call data , it is determined whether or not there is a response by answering machine.

The determination unit 121 is composed of a classifier that performs binary classification upon input of a feature quantity extracted from target call data. Here, the response is classified into two values: machine response, which is the answering machine response, and human response. As a classifier, for example, logistic regression, random forest, SVM (support vector machine), etc. are used, and any of them may be used. In the case of more than that, you may use SVM etc. separately.

For example, logistic regression shows the best performance when there is little teacher data, and the following formula classifies the machine response C1, which is the answering machine response, from the input feature value x under the weight w parameter. Calculate probabilities.
P(C1|x;w)=σ( ^wTx + _w0 )

Also, when using a random forest as a classifier, the calculation is based on the individual decision tree results f _k .

When the amount of data reaches a certain amount or more, for example, 10,000 voice data or more, the SVM can perform classification using higher performance as a classifier by replacing logistic regression. By using kernel tricks, it is possible to achieve nonlinear classification ability by drawing hyperplanes in higher dimensions than originally intended. The interface is defined by the following formula. (zi, yi) are the feature quantity and label of the existing i-th teacher data, and w and b are weights to be learned.

FIG. 3 is a functional block diagram of answering machine response determination system 1 according to the second embodiment of the present invention. In the second embodiment, feature amounts other than the overlapping speaker feature amount and the clustering feature amount are also extracted and used to determine whether or not the response was made by an answering machine. Note that detailed descriptions of the same configurations as those of the first embodiment will be omitted.

The answering machine response determination system 1 comprises an automated voice telephone 20 and an answering machine response determination device 10 . The automatic voice telephone 20 is a telephone capable of making a call without human presence by automatically making a call to a predetermined telephone number and making the other party hear an artificially generated voice of a predetermined message. The answering machine response determination device 10 receives call data recorded by the automatic voice telephone 20 via a communication network such as the Internet, extracts various feature values from the call data, and performs machine learning using teacher data. It is a device that determines whether or not the other party has responded with an answering machine message based on the determination unit.

In this embodiment, the answering machine response determination device 10 includes a call data acquisition unit 101, a speaker overlap feature extraction unit 111, a clustering feature extraction unit 113, a call duration feature extraction unit 115, and a text feature extraction unit. 117 , a machine synthesized sound feature amount extraction unit 119 , and a determination unit 121 .

A call data acquisition unit 101 receives call data of a call made automatically by the automatic voice telephone 20 without relying on human hands, from the automatic voice telephone 20, and acquires the call data. In addition, the speaker duplication feature amount extraction unit 111 calculates the degree of duplication of both speakers in the call data. The clustering feature quantity extraction unit 113 calculates a clustering feature quantity from the voice data of the channel of only the receiver side among the call data. Since the overlapping speaker feature quantity extraction unit 111 and the clustering feature quantity extraction unit 113 have the same configurations as those in the first embodiment, detailed description thereof will be omitted here.

The call time feature amount extraction unit 115 extracts the call time feature amount of the call data. For example, the call time itself in the call data may be used as the feature quantity. In another example, the square of the call duration may be used as the feature quantity of the call duration. Since the recording of a predetermined template is often used for the call time when answering the answering machine, the call ends at the same time. Therefore, by using the square of the call duration as a feature quantity, a quadratic function is expressed to capture the peak in the case of an answering machine response.

そして、ナイキスト周波数までのパワースペクトルを足し合わせることで、エネルギー統計量を算出する。

Further, the call duration feature quantity extraction unit 115 may extract the energy statistic of each frame of speech as the call duration feature quantity. In this case, the noise is measured over the entire call duration by calculating the energy statistic for each frame of speech. A high energy statistic indicates that there is a lot of noise in the background, and a low energy statistic indicates that there is not much noise in the background. Specifically, the following formula is calculated as an energy statistic. Let x(t) be the audio signal, divide it into frames of 0.025 rows, and apply a window function w(t) to convert the signal X[t, f] by short-time Fourier transform. t is time and f is frequency.

Then, the energy statistic is calculated by summing the power spectra up to the Nyquist frequency.

Note that the call duration feature quantity extraction unit 115 may extract only the energy statistic and the call duration feature quantity. You may In addition, you may extract two feature-values, an energy statistic and call time.

The text feature amount extraction unit 117 has a speech recognition unit 1171 that converts the text data into text data, and a response detection unit 1172 that detects a response to an answering machine by machine learning from the text data and calculates a text feature amount. The voice recognition unit 1171 performs voice recognition on the voice data of the called party in the acquired call data, and converts it into text.

The response detection unit 1172 detects whether or not there is a typical text message of machine response by an answering machine from the text data generated by voice recognition by the voice recognition unit 1171 . For example, it detects messages that are frequently used in answering messages by answering machines, such as "I'm out of the office right now," "Please leave a message," and "It's out of business hours." Specifically, the response detection unit 1172 applies a Japanese pre-trained language model such as BERT (Bidirectional Encoder Representations from Transformer) or XLNet to perform response detection.

The machine-synthesized sound feature quantity extraction unit 119 extracts a feature quantity for determining whether the speech of the called party is machine-synthesized sound. A large amount of speech and artificially synthesized speech are collected, put into an LSTM (Long Short-Term Memory), and trained to classify human speech and synthetic speech. In addition, the histogram of the mixed Gaussian distribution of the speech data is calculated, the histogram for human speech and the histogram for synthesized speech are calculated, and these are used as training data in a neural network for machine learning to create a machine synthesized sound classifier. generate it.

When the call data is acquired, the machine synthesized sound feature amount extraction unit 119 extracts the voice data of the called party's channel, calculates the histogram of the mixed Gaussian distribution of the voice data, and machine-learned the machine synthesized sound classifier. is made to calculate the probability of whether it is a synthetic sound or not, and the result is extracted as a machine-synthesized sound feature amount.

The determination unit 121 is generated by machine learning using various feature values extracted from call data and the presence/absence of a response to an answering machine as teacher data. By inputting the feature amount extracted from the call data for which it is desired to determine whether or not the response was by an answering machine to the determination unit generated by machine learning, the determination unit 121 determines whether or not there was a response by an answering machine in the call data. make a judgment as to whether In the present embodiment, a speaker overlap feature amount, clustering feature amount, call duration feature amount, text feature amount, and machine synthesized speech feature amount are input and determination is performed.

Note that in the second embodiment, the five feature amounts of overlapping speaker feature amount, clustering feature amount, call duration feature amount, text feature amount, and machine synthesized speech feature amount are input. A configuration may be adopted in which determination is performed by inputting one of the synthesized speech feature amounts and the other three feature amounts. Further, it may be configured such that determination is performed by inputting three feature amounts, namely, the overlapping speaker feature amount, the clustering feature amount, and the call duration feature amount. Processing can be speeded up by using only these three feature amounts.

FIG. 4 is a graph showing the number-of-seconds distribution in the case of human response. As can be seen from the figure, when a caller answers the call, the call ends in a short number of seconds. This is probably because, when receiving a call with an automatic answerer, the called party tends to hang up immediately when the other party recognizes that it is an automatic answerer (machine).

FIG. 5 is a graph showing the distribution of the number of seconds according to the machine response of the answering machine. It can be seen that the call duration peaks at 50 to 60 seconds. In the case of machine answering by answering machine, it takes a certain number of seconds because a standard message is played. When a call is made with an automatic answering machine, if the called party uses an answering machine to answer the call, there is time to record the message after the answering message, and after the predetermined recording time, the called party hangs up, so the time peak is 50 ~60 seconds.

FIG. 6 is a flow chart showing a process of generating, by machine learning, a classifier for determining whether or not an answering machine has answered on the called party side using teacher data. First, the call data acquisition unit 101 acquires call data and response result data indicating whether or not the called party answers the answering machine (step S601). Since the purpose here is to allow the classifier to perform machine learning, the call data and the response result data are acquired as teacher data.

Next, each feature quantity extraction unit extracts each feature quantity from the call data (step S602). In the first embodiment, the overlapping speaker feature amount extraction unit 111 and the clustering feature amount extraction unit 113 extract the overlapping speaker feature amount and the clustering feature amount from the call data. In the second embodiment, the speaker overlapping feature quantity extraction unit 111, the clustering feature quantity extraction unit 113, the call time feature quantity extraction unit 115, the text feature quantity extraction unit 117, and the machine synthesized speech feature quantity extraction unit 119 are We extract speaker overlapping features, clustering features, call duration features, text features, and machine synthesized speech features.

Next, a classifier used in the determination unit is generated by machine learning (step S603). Machine learning is performed using various feature values extracted from call data and the presence or absence of a response by an answering machine as teacher data to generate a classifier that determines whether the response was by an answering machine. Machine learning methods include logistic regression, random forest method, and support vector machine, any of which may be used. In the first embodiment, overlapping speaker feature amount, clustering feature amount, and in the second embodiment, overlapping speaker feature amount, clustering feature amount, call duration feature amount, text feature amount, and It is a machine-synthesized sound feature quantity. A classifier generated from the teacher data determines whether or not the call data to be determined was a response to an answering machine.

1 answering machine response determination system 10 answering machine response determination device 20 automatic voice telephone

Claims (7)

call data acquisition means for acquiring call data by telephone;
speaker duplication feature extracting means for calculating the degree of duplication of both speakers of the call data;
a clustering feature extraction means for calculating a clustering feature of the call data;
a determining means for determining whether or not a response is generated by machine learning, using the overlapping speaker feature amount, the clustering feature amount, and the presence or absence of a response to an answering machine as teacher data;
has
The determination means determines whether or not the call data is an answering machine response based on the speaker duplication feature extracted from the call data and the clustering feature. 2. The answering machine response determination device according to claim 1, further comprising a call time feature amount extracting means for extracting a call time feature amount of said call data,
The determination means is an answering machine response determination device that further uses a characteristic amount of call time as teacher data. 3. An answering machine response determination apparatus according to claim 2, wherein said call time feature quantity is an energy statistic of call voice. The answering machine response determination device according to any one of claims 1 to 3,
Further, text feature extracting means includes speech recognition means for converting the call data into text data and response detection means for detecting a response to an answering machine from the text data by machine learning and calculating a text feature. death,
The determination means is an answering machine response determination device that further uses the text feature quantity as teacher data. The answering machine response determination device according to any one of claims 1 to 4,
Furthermore, machine learning is performed using machine synthesized speech data, human speech data, and a Gaussian mixture histogram as teacher data, and the probability that the speech of the answering side in the call data is machine synthesized speech data is calculated as machine synthesized speech data. further comprising machine-synthesized sound feature quantity generating means for generating as a feature quantity,
The determination means is an answering machine response determination device that further uses the machine-synthesized sound feature quantity as teacher data. An answering machine response determination method comprising:
a call data acquisition step of acquiring call data by telephone;
a speaker duplication feature extracting step of calculating the degree of duplication of both speakers of the call data;
a clustering feature extraction step of calculating a clustering feature of the call data ;
Using the speaker duplication feature amount, the clustering feature amount, and the presence or absence of a response to an answering machine as teacher data, a determination unit generated by machine learning determines whether or not there is a response to an answering machine. a determination step;
answering machine response determination method. An answering machine response determination program executed by a computer as an answering machine response determination device,
a call data acquisition step of acquiring call data by telephone;
a speaker duplication feature extracting step of calculating the degree of duplication of both speakers of the call data;
a clustering feature extraction step of calculating a clustering feature of the call data ;
Using the speaker duplication feature amount, the clustering feature amount, and the presence or absence of a response to an answering machine as teacher data, a determination unit generated by machine learning determines whether or not there is a response to an answering machine. a determination step;
An answering machine response judgment program for executing

Images