CN111881251A

CN111881251A - AI telephone sales test method, device, electronic equipment and storage medium

Info

Publication number: CN111881251A
Application number: CN202010719764.1A
Authority: CN
Inventors: 宫雪
Original assignee: Ping An Technology Shenzhen Co Ltd
Current assignee: Ping An Technology Shenzhen Co Ltd
Priority date: 2020-07-23
Filing date: 2020-07-23
Publication date: 2020-11-03
Anticipated expiration: 2040-07-23
Also published as: CN111881251B; WO2021135531A1

Abstract

The application relates to artificial intelligence and provides an AI telephone sales test method, an AI telephone sales test device, electronic equipment and a storage medium. The method is capable of processing a dialoging table, generating a human answer dictionary, processing an interaction table, generating a dialog dictionary, generating a plurality of dictionaries as test cases, improving the speed of data reading and the comprehensiveness of test data, reading an initial dialog from the AI dialog of the dialog dictionary, calling an AI telemarketing model, traversing the dialog dictionary from the initial dialog to obtain human answer categories, and then traverse to all possible human answer categories, to fully test the AI telemarketing model, querying a human answer corresponding to the human answer category in a human answer dictionary, feeding back the human answer to the AI telemarketing model, outputting an AI feedback answer corresponding to the human answer, according to the human answer category, the expected answer is inquired in the dialogue dictionary to verify the AI feedback answer, so that the automatic test of the AI telephone sales model is realized, and the test efficiency is improved.

Description

AI telephone sales test method, device, electronic equipment and storage medium

Technical Field

The invention relates to the technical field of artificial intelligence, in particular to an AI customer service model testing method, an AI customer service model testing device, electronic equipment and a storage medium.

Background

At present, for the project of AI (Artificial Intelligence) intelligent telephone sales, manual testing is usually adopted, so that the testing time is long, and the scene coverage is not comprehensive enough.

And the AI model can automatically enter different flows according to the answers of the user, the part has a plurality of logic branches, coverage by adopting a manual mode is easy to miss, and some logic branches can be ignored.

Therefore, when testing the accuracy of the answer of the AI smartphone sales model to the user's question in the prior art, not only the testing speed is slow, but also the testing is not comprehensive enough, and when the logic changes, the whole testing scheme needs to be changed again for the changed logic, which is time-consuming and labor-consuming.

Disclosure of Invention

In view of the above, it is necessary to provide an AI telemarketing test method, apparatus, electronic device and storage medium, which can generate a plurality of dictionaries as test cases, improve the speed of data reading and the comprehensiveness of test data, and traverse to all possible human answer categories, so as to comprehensively test an AI telemarketing model, further implement automatic testing of the AI telemarketing model, and improve the testing efficiency.

An AI telemarketing test method, the AI telemarketing test method comprising:

acquiring an original data table, wherein the original data table comprises a dialect table and an interactive table;

processing the dialoging table and generating a human answer dictionary, and processing the interaction table and generating a dialog dictionary;

reading an originating utterance from an AI utterance of the dialog dictionary;

calling an AI telemarketing model, and traversing the dialogue dictionary from the initial dialogue to obtain human answer categories;

querying the human answer dictionary for human answers corresponding to the human answer categories;

feeding back the human answers to the AI telemarketing model, and outputting AI feedback answers corresponding to the human answers;

inquiring in the dialog dictionary according to the human answer category to obtain an expected answer;

verifying the AI feedback response based on the expected response.

According to a preferred embodiment of the present invention, said processing said dialogical table and generating a human answer dictionary comprises:

opening the dialect table by adopting an open _ workbook function, and determining a first index corresponding to the human answer category and a second index corresponding to the human answer;

reading each record in the dialect table based on the first index, and determining the read data as a Key value;

reading each record in the dialect table based on the second index, and determining the read data as a Value;

determining the corresponding relation of the first index and the second index in the dialect table;

and inserting the Key Value and the Value into a binary tree node according to the corresponding relation to generate the human answer dictionary.

According to the preferred embodiment of the present invention, the AI telemarketing test method further comprises:

responding to an updating instruction of the human answer dictionary, and acquiring a current Key Value and a current Value;

querying the human answer dictionary for the current Key value;

when the current Key Value does not exist in the human answer dictionary, creating a node in the human answer dictionary and inserting the current Key Value and the current Value into the created node; or

When the current Key Value exists in the human answer dictionary, acquiring a Value array corresponding to the current Key Value, and inserting the current Value into the Value array.

According to a preferred embodiment of the present invention, the processing the interaction table and generating the dialog dictionary comprises:

reading the human answer category and the expected answer from the interactive table to construct an AI answer dictionary, wherein a Key Value of the AI answer dictionary is the human answer category, and a Value of the AI answer dictionary is the expected answer;

reading the AI dialog from the interactive table;

and constructing the dialog dictionary according to the AI dialogs and the AI answer dictionary, wherein the Key Value of the dialog dictionary is the AI dialogs, and the Value of the dialog dictionary is the AI answer dictionary.

According to a preferred embodiment of the present invention, the invoking of the AI telemarketing model to traverse the dialog dictionary from the originating dialogs to obtain the human answer categories comprises:

sequentially traversing the Key values of the dialogue dictionary by preset logic from the beginning of the dialogue, stopping the traversal until the ending of the dialogue is traversed, and acquiring elements in an AI answer dictionary corresponding to the traversed Key values;

and determining a Key value in the acquired element as the human answer category.

According to a preferred embodiment of the present invention, said querying in said dialog dictionary according to said human answer category to obtain an expected answer comprises:

determining elements of the human answer category that correspond in the AI answer dictionary;

taking the Value in the determined element as the expected answer.

According to a preferred embodiment of the present invention, said verifying said AI feedback response based on said expected response comprises:

establishing a plurality of data groups by using the expected answer and the AI feedback answer, wherein each data group comprises one sub expected answer in the expected answer and one sub AI feedback answer corresponding to the sub expected answer in the AI feedback answer;

calculating the similarity between the sub expected response and the sub AI feedback response in each data group;

for each data group, when the similarity is greater than or equal to the configuration similarity, determining that the sub AI feedback response in the data group passes verification;

determining a first number of validated data sets, and determining a total number of all data sets;

and dividing the first quantity by the total quantity to obtain the accuracy of the AI telemarketing model.

An AI telemarketing testing device, the AI telemarketing testing device comprising:

the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring an original data table, and the original data table comprises a dialect table and an interactive table;

a generating unit for processing the dialoging table and generating a human answer dictionary, and processing the interactive table and generating a dialog dictionary;

a reading unit for reading an initial utterance from an AI utterance of the dialog dictionary;

the calling unit is used for calling an AI (artificial intelligence) telephone sales model and traversing the dialogue dictionary from the initial dialogue to obtain a human answer category;

a querying unit configured to query the human answer dictionary for human answers corresponding to the human answer categories;

the feedback unit is used for feeding back the human answers to the AI telemarketing model and outputting AI feedback answers corresponding to the human answers;

the query unit is further used for querying in the dialog dictionary according to the human answer categories to obtain expected answers;

a verification unit for verifying the AI feedback response based on the expected response.

According to a preferred embodiment of the present invention, the generating unit processes the dialogical table and generates the human answer dictionary includes:

According to a preferred embodiment of the present invention, the obtaining unit is further configured to obtain a current Key Value and a current Value in response to an update instruction to the human answer dictionary;

the query unit is further configured to query the current Key value in the human answer dictionary;

the AI telemarketing test apparatus further includes:

a creating unit, configured to create a node in the human answer dictionary and insert the current Key Value and the current Value into the created node when the current Key Value does not exist in the human answer dictionary; or

The obtaining unit is further configured to, when the current Key Value exists in the human answer dictionary, obtain a Value array corresponding to the current Key Value, and insert the current Value into the Value array.

According to a preferred embodiment of the present invention, the generating unit processes the interaction table and generates the dialog dictionary includes:

reading the AI dialog from the interactive table;

According to the preferred embodiment of the present invention, the AI dialect of the dialog dictionary includes a termination dialect, and the call unit is specifically configured to:

According to a preferred embodiment of the present invention, the querying unit queries the dialog dictionary according to the human answer category, and obtaining the expected answer includes:

taking the Value in the determined element as the expected answer.

According to a preferred embodiment of the present invention, the verification unit is specifically configured to:

An electronic device, the electronic device comprising:

a memory storing at least one instruction; and

and a processor executing the instructions stored in the memory to implement the AI telemarketing testing method.

A computer-readable storage medium having stored therein at least one instruction, the at least one instruction being executable by a processor in an electronic device to implement the AI telemarketing testing method.

According to the technical scheme, the method can obtain the original data table which comprises the dialoging table and the interaction table, process the dialoging table and generate the human answer dictionary, process the interaction table and generate the dialogue dictionary, further generate a plurality of dictionaries to be used as test cases, improve the data reading speed and the test data comprehensiveness, read the initial dialoging from the AI dialoging of the dialogue dictionary, call the AI telephone sales model, traverse the dialogue dictionary from the initial dialoging to obtain the human answer categories, further traverse all possible human answer categories so as to comprehensively test the AI telephone sales model, inquire the human answers corresponding to the human answer categories in the human answer dictionary, feed the human answers back to the AI telephone sales model, and output the AI feedback responses corresponding to the human answers, and inquiring in the dialog dictionary according to the human answer category to obtain an expected answer, verifying the AI feedback answer based on the expected answer, realizing automatic test of an AI telephone sales model, and improving the test efficiency.

Drawings

FIG. 1 is a flow chart of the preferred embodiment of the AI telemarketing test method of the present invention.

Fig. 2 is a functional block diagram of a preferred embodiment of the AI telemarketing test apparatus of the present invention.

Fig. 3 is a schematic structural diagram of an electronic device implementing the AI telemarketing test method according to the preferred embodiment of the invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in detail with reference to the accompanying drawings and specific embodiments.

Fig. 1 is a flow chart of the AI telemarketing test method according to the preferred embodiment of the present invention. The order of the steps in the flow chart may be changed and some steps may be omitted according to different needs.

The AI telemarketing test method is applied to one or more electronic devices, which are devices capable of automatically performing numerical calculation and/or information processing according to preset or stored instructions, and the hardware thereof includes, but is not limited to, a microprocessor, an Application Specific Integrated Circuit (ASIC), a Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), an embedded device, and the like.

The electronic device may be any electronic product capable of performing human-computer interaction with a user, for example, a Personal computer, a tablet computer, a smart phone, a Personal Digital Assistant (PDA), a game machine, an interactive Internet Protocol Television (IPTV), an intelligent wearable device, and the like.

The electronic device may also include a network device and/or a user device. The network device includes, but is not limited to, a single network server, a server group consisting of a plurality of network servers, or a cloud computing (cloud computing) based cloud consisting of a large number of hosts or network servers.

The Network where the electronic device is located includes, but is not limited to, the internet, a wide area Network, a metropolitan area Network, a local area Network, a Virtual Private Network (VPN), and the like.

S10, obtaining a raw data table, wherein the raw data table comprises a dialect table and an interactive table.

The original data table may be configured in advance, or may be called from a database, which is not limited in the present invention.

As shown in table 1 below, is the interaction table.

AI dialect	Human answer categories	Expected answer
			A	A1	A
A	A2	B
			A	A3	C
A	A4	D
			B	B1	A
B	B2	B
			B	B3	C
B	B5	D
			C	C3	B
C	C4	C
			C	C5	C
C	C6	D
			C	C7	D
D

TABLE 1

Wherein, AI (Artificial Intelligence) speech means speech output by an AI robot.

The human answer category refers to a conversational category of possible human answers to the AI conversation.

The desired answer refers to reply content that the AI should output after acquiring an answer of a human.

Specifically, a/B/C/D represents the classification of the AI dialogs, such as a being an open field white class, D being a hang-up class, etc.

A1/B1/C2 and the like represent categories to which answers given to the AI speaker belong. For example: a1 indicates rejection, and C1 indicates rejection again.

As shown in table 2 below, is the dialogical table.

Human answer categories	Human responses
		A1	A11；A12；A13
B2	B11；B12；B13
		C3	C11；C12:C13

TABLE 2

Wherein the human answer categories A1/B1/C1 have the same meaning as Table 1.

The human answers represent the true possible answers of humans, for example: when a1 indicates a decline, a11 may be "i do not need a loan," a22 may be "i are now busy," etc.

S11, processing the dialoging table and generating a human answer dictionary, and processing the interaction table and generating a dialog dictionary.

In at least one embodiment of the invention, the processing the dialogical table and generating the human answer dictionary comprises:

For example: when a dictionary is generated by using table 2, the dictionary includes three elements, the Key values of the three elements are a1, B2 and C3 respectively, and the corresponding Value values are arrays and are [ a11 respectively; a12; a13, [ B11; b12; b13, [ C11; c12: C13 ].

The Value is in an array form, elements in the array are stored in the memory continuously, time complexity can be constant time when the elements are read, and the reading speed is higher.

Firstly, the data structure of the dictionary is in an array mode, the reading speed is higher, the rapid search of data can be assisted, and the testing speed is further improved.

In addition, the binary tree nodes have logarithm searching efficiency and linked list characteristics, the advantages of the linked list and the ordered array can be utilized, and the advantages of the ordered array and the linked list can be combined, so that the data processing speed can be further improved by introducing the binary tree nodes, and the testing speed is further improved.

Further, the method further comprises:

querying the human answer dictionary for the current Key value;

Different from the prior art that the whole test scheme needs to be changed again aiming at the changed logic, the time and labor are wasted, the dictionary is only updated when the logic is changed, the whole test scheme does not need to be updated, and the maintenance cost is low.

In at least one embodiment of the invention, the processing the interaction table and generating the dialog dictionary comprises:

reading the AI dialog from the interactive table;

Through the embodiment, a plurality of dictionaries can be generated as the test cases, and the data reading speed and the comprehensiveness of the test data are improved.

S12, reading the initial dialog from the AI dialog of the dialog dictionary.

For example: the initial utterance may be the first utterance when an AI electricity-pin is incoming.

And S13, calling an AI telemarketing model, traversing the dialogue dictionary from the initial dialogue to obtain human answer categories.

In at least one embodiment of the present invention, the AI dialogs of the dialog dictionary include a termination dialogs, the invoking the AI telemarketing model, traversing the dialog dictionary from the initiation dialogs, and the obtaining the human answer categories include:

For example: the termination technique may be the last sentence when the phone hangs up.

Through the implementation mode, all possible human answer categories can be traversed based on the dictionary, so that the AI telemarketing model can be comprehensively tested, and due to the fact that the logic branches are completely covered, omission does not occur, and the test result is more reliable.

S14, querying the human answer dictionary for human answers corresponding to the human answer categories.

For example: when the human answer category represents a rejection, the corresponding human answer may be "i do not need a loan" or "i are now busy", etc.

And S15, feeding back the human answer to the AI telemarketing model, and outputting an AI feedback answer corresponding to the human answer.

Wherein the AI feedback response refers to a response output by the AI telemarketing model.

And S16, inquiring in the dialog dictionary according to the human answer categories to obtain expected answers.

In at least one embodiment of the present invention, said querying in said dialog dictionary according to said human answer categories, resulting in desired answers comprises:

taking the Value in the determined element as the expected answer.

Through the implementation mode, the expected answer can be quickly inquired based on the attribute of the dictionary, and the follow-up test is convenient.

S17, verifying the AI feedback response based on the expected response.

In at least one embodiment of the invention, the verifying the AI feedback response based on the expected response comprises:

Through the embodiment, automatic testing can be performed, and testing time is effectively reduced.

Fig. 2 is a functional block diagram of the AI telemarketing test apparatus according to the preferred embodiment of the present invention. The AI telemarketing testing apparatus 11 includes an acquisition unit 110, a generation unit 111, a reading unit 112, a calling unit 113, an inquiry unit 114, a feedback unit 115, a verification unit 116, and a creation unit 117. The module/unit referred to in the present invention refers to a series of computer program segments that can be executed by the processor 13 and that can perform a fixed function, and that are stored in the memory 12. In the present embodiment, the functions of the modules/units will be described in detail in the following embodiments.

The obtaining unit 110 obtains a raw data table, which includes a dialect table and an interaction table.

As shown in table 1 below, is the interaction table.

TABLE 1

As shown in table 2 below, is the dialogical table.

TABLE 2

Wherein the human answer categories A1/B1/C1 have the same meaning as Table 1.

The generation unit 111 processes the dialoging table and generates a human answer dictionary, and processes the interaction table and generates a dialog dictionary.

In at least one embodiment of the present invention, the generating unit 111 processes the dialogical table and generates the human answer dictionary comprises:

Further, the obtaining unit 110 obtains a current Key Value and a current Value in response to an update instruction to the human answer dictionary;

query unit 114 queries the human answer dictionary for the current Key value;

when the current Key Value does not exist in the human answer dictionary, creating unit 117 creates a node in the human answer dictionary and inserts the current Key Value and the current Value into the created node;

or when the current Key Value exists in the human answer dictionary, the obtaining unit 110 obtains a Value array corresponding to the current Key Value, and inserts the current Value into the Value array.

In at least one embodiment of the present invention, the generating unit 111 processes the interaction table and generates the dialog dictionary includes:

reading the AI dialog from the interactive table;

The reading unit 112 reads the originating utterance from the AI utterance of the dialog dictionary.

The calling unit 113 calls an AI telemarketing model to traverse the dialog dictionary from the originating dialogs to obtain human answer categories.

In at least one embodiment of the present invention, the AI dialogs of the dialog dictionary include a termination dialogs, the invoking unit 113 invokes an AI telemarketing model, traversing the dialog dictionary from the initiation dialogs, and obtaining the human answer categories includes:

The querying unit 114 queries the human answer dictionary for human answers corresponding to the human answer categories.

The feedback unit 115 feeds back the human answer to the AI telemarketing model, and outputs an AI feedback answer corresponding to the human answer.

The query unit 114 performs a query in the dialog dictionary according to the human answer category to obtain an expected answer.

In at least one embodiment of the present invention, the querying unit 114 queries the dialog dictionary according to the human answer categories, and obtaining the expected answer comprises:

taking the Value in the determined element as the expected answer.

The verification unit 116 verifies the AI feedback response based on the expected response.

In at least one embodiment of the present invention, the verifying unit 116 verifying the AI feedback response based on the expected response comprises:

Fig. 3 is a schematic structural diagram of an electronic device according to a preferred embodiment of the AI telemarketing test method of the present invention.

The electronic device 1 may comprise a memory 12, a processor 13 and a bus, and may further comprise a computer program, such as an AI telemarketing test program, stored in the memory 12 and executable on the processor 13.

It will be understood by those skilled in the art that the schematic diagram is merely an example of the electronic device 1, and does not constitute a limitation to the electronic device 1, the electronic device 1 may have a bus-type structure or a star-type structure, the electronic device 1 may further include more or less hardware or software than those shown in the figures, or different component arrangements, for example, the electronic device 1 may further include an input and output device, a network access device, and the like.

It should be noted that the electronic device 1 is only an example, and other existing or future electronic products, such as those that can be adapted to the present invention, should also be included in the scope of the present invention, and are included herein by reference.

The memory 12 includes at least one type of readable storage medium, which includes flash memory, removable hard disks, multimedia cards, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disks, optical disks, etc. The memory 12 may in some embodiments be an internal storage unit of the electronic device 1, for example a removable hard disk of the electronic device 1. The memory 12 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the electronic device 1. Further, the memory 12 may also include both an internal storage unit and an external storage device of the electronic device 1. The memory 12 may be used not only to store application software installed in the electronic device 1 and various types of data such as codes of an AI telemarketing test program, etc., but also to temporarily store data that has been output or is to be output.

The processor 13 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be composed of a plurality of integrated circuits packaged with the same or different functions, including one or more Central Processing Units (CPUs), microprocessors, digital Processing chips, graphics processors, and combinations of various control chips. The processor 13 is a Control Unit (Control Unit) of the electronic device 1, connects various components of the electronic device 1 by various interfaces and lines, and executes various functions and processes data of the electronic device 1 by running or executing programs or modules (e.g., executing AI telemarketing test programs, etc.) stored in the memory 12 and calling data stored in the memory 12.

The processor 13 executes an operating system of the electronic device 1 and various installed application programs. The processor 13 executes the application program to implement the steps in the various embodiments of the AI telemarketing testing method described above, such as the steps shown in fig. 1: s10, S11, S12, S13, S14, S15, S16, and S17.

Alternatively, the processor 13, when executing the computer program, implements the functions of the modules/units in the above device embodiments, for example:

reading an originating utterance from an AI utterance of the dialog dictionary;

verifying the AI feedback response based on the expected response.

Illustratively, the computer program may be divided into one or more modules/units, which are stored in the memory 12 and executed by the processor 13 to accomplish the present invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program in the electronic device 1. For example, the computer program may be divided into an acquisition unit 110, a generation unit 111, a reading unit 112, a calling unit 113, an inquiry unit 114, a feedback unit 115, a verification unit 116, a creation unit 117.

The integrated unit implemented in the form of a software functional module may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a computer device, or a network device) or a processor (processor) to execute parts of the AI telemarketing test method according to various embodiments of the present invention.

The integrated modules/units of the electronic device 1 may be stored in a computer-readable storage medium if they are implemented in the form of software functional units and sold or used as separate products. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented.

Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM).

The bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one arrow is shown in FIG. 3, but this does not indicate only one bus or one type of bus. The bus is arranged to enable connection communication between the memory 12 and at least one processor 13 or the like.

Although not shown, the electronic device 1 may further include a power supply (such as a battery) for supplying power to each component, and preferably, the power supply may be logically connected to the at least one processor 13 through a power management device, so as to implement functions of charge management, discharge management, power consumption management, and the like through the power management device. The power supply may also include any component of one or more dc or ac power sources, recharging devices, power failure detection circuitry, power converters or inverters, power status indicators, and the like. The electronic device 1 may further include various sensors, a bluetooth module, a Wi-Fi module, and the like, which are not described herein again.

Further, the electronic device 1 may further include a network interface, and optionally, the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a bluetooth interface, etc.), which are generally used for establishing a communication connection between the electronic device 1 and other electronic devices.

Optionally, the electronic device 1 may further comprise a user interface, which may be a Display (Display), an input unit (such as a Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the electronic device 1 and for displaying a visualized user interface, among other things.

It is to be understood that the described embodiments are for purposes of illustration only and that the scope of the appended claims is not limited to such structures.

Fig. 3 only shows the electronic device 1 with components 12-13, and it will be understood by a person skilled in the art that the structure shown in fig. 3 does not constitute a limitation of the electronic device 1, and may comprise fewer or more components than shown, or a combination of certain components, or a different arrangement of components.

With reference to fig. 1, the memory 12 of the electronic device 1 stores a plurality of instructions to implement an AI telemarketing test method, and the processor 13 executes the plurality of instructions to implement:

reading an originating utterance from an AI utterance of the dialog dictionary;

verifying the AI feedback response based on the expected response.

Specifically, the processor 13 may refer to the description of the relevant steps in the embodiment corresponding to fig. 1 for a specific implementation method of the instruction, which is not described herein again.

In the embodiments provided in the present invention, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.

The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.

In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.

It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.

The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.

Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the system claims may also be implemented by one unit or means in software or hardware. The terms second, etc. are used to denote names, but not any particular order.

Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An AI telemarketing test method, the AI telemarketing test method comprising:

reading an originating utterance from an AI utterance of the dialog dictionary;

verifying the AI feedback response based on the expected response.

2. The AI telemarketing testing method of claim 1, wherein the processing the dialogs table and generating a human answer dictionary comprises:

3. The AI telemarketing testing method of claim 2, further comprising:

querying the human answer dictionary for the current Key value;

4. The AI telemarketing testing method of claim 1, wherein the processing the interaction table and generating a dialog dictionary comprises:

reading the AI dialog from the interactive table;

5. The AI telemarketing testing method of claim 1, wherein the AI dialogs of the dialog dictionary include a termination dialogs, the invoking the AI telemarketing model, traversing the dialog dictionary from the initiation dialogs, and deriving the human answer categories comprises:

6. The AI telemarketing testing method of claim 1, wherein said querying in the dialog dictionary according to the human answer categories to obtain desired answers comprises:

taking the Value in the determined element as the expected answer.

7. The AI telemarketing testing method of claim 1, wherein the validating the AI feedback response based on the expected response comprises:

8. An AI telemarketing test apparatus, comprising:

9. An electronic device, characterized in that the electronic device comprises:

a memory storing at least one instruction; and

a processor that executes instructions stored in the memory to implement the AI telemarketing testing method of any of claims 1-7.

10. A computer-readable storage medium characterized by: the computer-readable storage medium has stored therein at least one instruction that is executed by a processor in an electronic device to implement the AI telemarketing testing method of any of claims 1-7.