Nothing Special   »   [go: up one dir, main page]

WO2020078300A1 - 一种终端投屏的控制方法和终端 - Google Patents

一种终端投屏的控制方法和终端 Download PDF

Info

Publication number
WO2020078300A1
WO2020078300A1 PCT/CN2019/110926 CN2019110926W WO2020078300A1 WO 2020078300 A1 WO2020078300 A1 WO 2020078300A1 CN 2019110926 W CN2019110926 W CN 2019110926W WO 2020078300 A1 WO2020078300 A1 WO 2020078300A1
Authority
WO
WIPO (PCT)
Prior art keywords
terminal
voice
result
application program
voice data
Prior art date
Application number
PCT/CN2019/110926
Other languages
English (en)
French (fr)
Inventor
夏少华
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to US17/285,563 priority Critical patent/US20210398527A1/en
Publication of WO2020078300A1 publication Critical patent/WO2020078300A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1423—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/18—Speech classification or search using natural language modelling
    • G10L15/1815—Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/18—Speech classification or search using natural language modelling
    • G10L15/1822—Parsing for meaning understanding
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/223—Execution procedure of a spoken command
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/225—Feedback of the input speech

Definitions

  • the present application relates to the field of communication technologies, and in particular, to a method and terminal for controlling a screen cast by a terminal.
  • a mobile screen projection method is adopted, that is, a large screen is connected to the mobile terminal, and the user can operate an application of the mobile terminal, and the mobile terminal is connected to the large screen to display the user's operation content, thereby realizing a large screen-based Content sharing.
  • the user is required to hold the terminal, or connect the mouse and keyboard to the terminal to control the application program. Since the prior art requires the user to manually control the terminal to display the application on the large screen, the user's hands cannot be freed, and the application processing efficiency in the scenario where the terminal is connected to the large screen is reduced.
  • Embodiments of the present application provide a method and a terminal control method for terminal projection, which are used to improve application processing efficiency in a scenario where the terminal is connected to a large screen.
  • an embodiment of the present application provides a method for controlling a screen cast by a terminal.
  • the method is applied to a terminal, and the terminal is connected to a display device.
  • the method includes: the terminal collecting first voice data; the terminal Perform voice recognition processing on the first voice data; the terminal controls the display device to display content associated with the first voice data according to the result of the voice recognition processing.
  • a terminal is connected to a display device, the terminal collects first voice data, and then the terminal performs voice recognition processing on the first voice data to generate a result of voice recognition processing, and then the terminal controls according to the result of the voice recognition processing
  • the application program of the terminal and finally the terminal displays the control process of the application program on the display device.
  • the user can directly issue a voice command to the terminal through voice communication, and the terminal can collect the first voice data sent by the user.
  • the terminal can control the application program according to the result of the voice recognition processing, so that the execution of the application program
  • the process can display the control process on the display device connected to the terminal device, without requiring the user to manually operate the terminal, thus improving the application processing efficiency in the scenario where the terminal is connected to a large screen.
  • the terminal controlling the display device to display the content associated with the first voice data according to the result of the voice recognition processing includes: the terminal recognizing the voice recognition An application program interface corresponding to the processed result; the terminal controls the application program through the application program interface and displays related content on the display device.
  • the terminal recognizes the application program that the user needs to control according to the result of the voice recognition process.
  • the terminal recognizes the application program interface corresponding to the result of the voice recognition process.
  • Different application programs are configured with different application program interfaces.
  • the terminal recognizes After the application program interface, the terminal can determine the application program that the user needs to control through the application program interface.
  • the terminal recognizing the application program interface corresponding to the result of the speech recognition process includes: the terminal performing semantic analysis on the result of the speech recognition process to generate a semantic analysis Results; the terminal extracts instructions from the semantic analysis results; the terminal recognizes the application program interface according to the instructions.
  • the result of the speech recognition processing generated by the terminal may be text information.
  • the terminal performs semantic analysis on the text information to generate a semantic analysis result.
  • the terminal extracts instructions from the semantic analysis result.
  • the terminal generates instructions according to a preset instruction format.
  • the terminal recognizes the application program interface according to the extracted instruction.
  • a semantic parsing function can be configured in the terminal, that is, the terminal can learn and understand the semantic content represented by a piece of text, and finally convert it into commands and parameters that can be recognized by the machine.
  • the terminal recognizing the application program interface corresponding to the result of the voice recognition process includes: the terminal sending the result of the voice recognition process to a cloud server
  • the cloud server performs semantic analysis on the result of the voice recognition processing; the terminal receives the analysis result fed back by the cloud server after semantic analysis; the terminal recognizes the application program interface according to the analysis result.
  • the result of the speech recognition processing generated by the terminal may be text information, and the terminal establishes a communication connection with the cloud server.
  • the terminal may send the text information to the cloud server, and the cloud server performs semantic analysis on the text information.
  • the cloud server performs semantic analysis After completion, the instruction is generated, the cloud server sends the instruction, the terminal can receive the analysis result fed back by the cloud server after semantic analysis, and finally the terminal recognizes the application program interface according to the extracted instruction.
  • the method further includes: the terminal acquiring A feedback result of the application program; the terminal converts the feedback result into second voice data and plays the second voice data; or, the terminal displays the feedback result on the display device.
  • the application program may also generate a feedback result, and the feedback result may indicate that the application program successfully responds to the user's voice command, or may indicate that the application program fails to respond to the voice command.
  • the terminal can convert the feedback result into second voice data and play the second voice data, for example, a player is configured in the terminal, and the terminal can play the second voice data through the player, so that the user The second voice data can be heard.
  • the terminal can also display the feedback result on the display device, so that the user can determine whether the voice command execution is successful or failed from the terminal connected display device.
  • the terminal collecting the first voice data includes: the terminal calling up a voice assistant in a wake-up-free manner, and the voice assistant voices the first voice data collection.
  • a voice assistant can be configured in the terminal, and voice collection can be performed through the voice assistant.
  • the terminal can use a wake-up word-free way to call up the voice assistant, which is relative to the voice assistant No need to open the application of voice assistant first, the user can directly say something to the terminal, the terminal can automatically call up the voice assistant and execute voice commands.
  • an embodiment of the present application provides a terminal connected to a display device.
  • the terminal includes: a voice collector and a processor; the processor and the voice collector communicate with each other;
  • the voice collector is used to collect first voice data;
  • the processor is used to perform voice recognition processing on the first voice data; according to the result of the voice recognition processing, the display device and the first A content associated with voice data.
  • the processor is further configured to recognize an application program interface corresponding to the result of the voice recognition process; control the application program through the application program interface, and The related content is displayed on the display device.
  • the processor is further configured to call a management service function module through the application program interface; and control the application program through the management service function module.
  • the processor is further configured to perform semantic analysis on the result of the speech recognition processing to generate a semantic analysis result; extract an instruction from the semantic analysis result; according to the The instruction identifies the application program interface.
  • the processor is further configured to send the result of the voice recognition processing to a cloud server, and the cloud server performs semantic analysis on the result of the voice recognition processing; receiving An analysis result fed back by the cloud server after semantic analysis; identifying the application program interface according to the analysis result.
  • the terminal further includes: a player connected to the processor; the processor is further configured to control display according to the result of the voice recognition process After displaying the content associated with the first voice data on the device, obtain the feedback result of the application program; convert the feedback result into second voice data, and control the player to play the second voice data; or To control the display device to display the feedback result.
  • the processor is also used to call up a voice assistant in a wake-up-free manner; the voice collector is used to control the voice assistant under the control of the voice assistant A voice data for voice collection.
  • the component modules of the terminal may also perform the steps described in the foregoing first aspect and various possible implementations. For details, see the foregoing description of the first aspect and various possible implementations .
  • an embodiment of the present application further provides a terminal, the terminal is connected to a display device, and the terminal includes:
  • Collection module used to collect the first voice data
  • a voice recognition module configured to perform voice recognition processing on the first voice data
  • the display module is configured to control the display device to display the content associated with the first voice data according to the result of the voice recognition process.
  • the display module includes: an interface recognition unit for recognizing an application program interface corresponding to the result of the voice recognition process; a control unit for The application program interface controls the application program and displays related content on the display device.
  • the interface recognition unit is configured to perform semantic analysis on the result of the speech recognition process to generate a semantic analysis result; extract instructions from the semantic analysis result; The instruction identifies the application program interface.
  • the interface recognition unit is configured to send the result of the voice recognition process to a cloud server, and the cloud server performs semantic analysis on the result of the voice recognition process Receiving the analysis result fed back by the cloud server after semantic analysis; identifying the application program interface according to the analysis result.
  • the terminal further includes: an acquisition module and a playback module, wherein the acquisition module is used by the display module to control display according to the result of the voice recognition processing After displaying the content associated with the first voice data on the device, obtain the feedback result of the application; the playback module is configured to convert the feedback result into second voice data and play the second voice Data; or, the display module is also used to display the feedback result on the display device.
  • the collection module is further configured to call up a voice assistant in a wake-up-free manner, and the voice assistant performs voice collection on the first voice data.
  • an embodiment of the present application provides a computer-readable storage medium having instructions stored therein, which when executed on a computer, causes the computer to execute the method described in the first aspect above.
  • an embodiment of the present application provides a computer program product containing instructions, which, when run on a computer, causes the computer to execute the method described in the first aspect above.
  • an embodiment of the present application provides a communication device.
  • the communication device may include an entity such as a terminal or a chip.
  • the communication device includes: a processor and a memory; the memory is used to store instructions; and the processor is used to execute The instructions in the memory cause the communication device to execute the method as described in any one of the first aspects.
  • the present application provides a chip system that includes a processor for supporting a terminal to implement the functions involved in the above aspects, for example, sending or processing data and / or information involved in the above method.
  • the chip system further includes a memory, which is used to store necessary program instructions and data of the terminal.
  • the chip system may be composed of chips, and may also include chips and other discrete devices.
  • FIG. 1 is a schematic structural diagram of a communication system to which a method for controlling a screen projection provided by an embodiment of the present application is applied;
  • FIG. 2 is a schematic flowchart of a flow of a method for controlling a screen projection of a terminal provided by an embodiment of the present application
  • FIG. 3 is a schematic diagram of an implementation architecture for terminal screen control of a document application provided by an embodiment of the present application
  • FIG. 4 is a schematic flowchart of voice control of a document application program provided by an embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of a terminal according to an embodiment of the present application.
  • 6-a is a schematic structural diagram of another terminal structure provided by an embodiment of the present application.
  • 6-b is a schematic structural diagram of a display module according to an embodiment of the present application.
  • 6-c is a schematic structural diagram of another terminal composition provided by an embodiment of this application.
  • FIG. 7 is a schematic structural diagram of another terminal structure provided by an embodiment of the present application.
  • Embodiments of the present application provide a method and a terminal control method for terminal projection, which are used to improve application processing efficiency in a scenario where the terminal is connected to a large screen.
  • the communication system includes a terminal, and the terminal is connected to a display device.
  • the display device may be a large display device.
  • the terminal can be connected to the display device in a wired or wireless manner, for example, the terminal is connected to the display device through a high definition multimedia interface (HDMI), or the terminal is connected to the display device through a type-c interface.
  • HDMI high definition multimedia interface
  • the terminal is also called user equipment (user equipment (UE), mobile station (MS), mobile terminal (MT), etc.), which is a device that provides voice and / or data connectivity to users Or, a chip installed in the device, for example, a handheld device, a vehicle-mounted device, etc., which have wireless connection permission.
  • UE user equipment
  • MS mobile station
  • MT mobile terminal
  • terminals are: mobile phones, tablets, laptops, PDAs, mobile Internet devices (MID), wearable devices, virtual reality (VR) devices, and augmented reality (augmented reality, AR) equipment, wireless terminals in industrial control, wireless terminals in self-driving (self-driving), wireless terminals in remote surgery (remote medical), and smart grids (smart grids) Wireless terminals, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, etc.
  • the terminal provided by the embodiment of the present application only needs to be connected to a display device to execute the method for controlling the screen projection of the terminal provided by the embodiment of the present application.
  • the embodiment of the present application proposes a method for controlling the screen projection of a terminal.
  • the method is applied to a terminal, and the terminal is connected to a display device. Please refer to FIG. 2. :
  • the terminal collects first voice data.
  • a user can operate an application through a terminal, and the type of the application is not limited.
  • the application may be a document application, a game application, or an audio-video application.
  • the application is displayed on the terminal connected display device.
  • the voice control method is used, that is, the user issues a voice command.
  • the terminal has a built-in voice collector, and the terminal uses the voice collector to collect the user's voice.
  • Voice commands For example, the terminal collects the first voice data within a period of time.
  • the terminal screen projection control process of the first voice data is used as an example for description. Other voice data collected by the terminal may also be controlled by the terminal screen projection according to the processing process of the first voice data, which is only described here. .
  • the terminal collecting the first voice data includes:
  • the terminal calls up the voice assistant in a wake-up-free way, and the voice assistant collects the first voice data by voice.
  • a voice assistant can be configured in the terminal, and voice collection can be performed through the voice assistant.
  • the terminal can use a wake-up word-free way to call up the voice assistant, which is relative to the voice assistant No need to open the application of voice assistant first, the user can directly say something to the terminal, the terminal can automatically call up the voice assistant and execute voice commands.
  • the terminal performs voice recognition processing on the first voice data.
  • the terminal after the terminal collects the first voice data, the terminal performs voice recognition processing on the first voice data to recognize the text information corresponding to the first voice data, and the result of the voice recognition processing generated by the terminal This text information can be included.
  • the terminal may perform speech recognition processing on the first speech data through a natural speech understanding (NLU) tool, where speech recognition refers to letting the machine transform the first speech data into a recognition and understanding process Corresponding to the process of text information, the result of the speech recognition process generated by the terminal can be used to control the application of the terminal.
  • NLU natural speech understanding
  • the terminal controls the display device to display the content associated with the first voice data according to the result of the voice recognition process.
  • the terminal may use the result of the voice recognition processing to control the application program, and the terminal may directly use the result of the voice recognition processing as a command to control the application
  • the terminal may also obtain an instruction corresponding to the result of the voice recognition process, and control the application program according to the instruction.
  • the way to control the application depends on the result of the voice recognition process generated by the terminal. Taking the application program as the document application program as an example, if the user issues a voice command to open the document A, the terminal can control the document application program to open the document A.
  • step 203 the terminal controls the display device to display the content associated with the first voice data according to the result of the voice recognition process, including:
  • the terminal recognizes the application program interface corresponding to the result of the voice recognition process
  • the terminal controls the application program through the application program interface and displays related content on the display device.
  • the terminal recognizes the application program that the user needs to control according to the result of the voice recognition process. For example, the terminal recognizes the application program interface corresponding to the result of the voice recognition process. Different application programs are configured with different application program interfaces. After the terminal recognizes the application program interface, the terminal can determine the user needs to control through the application program interface application.
  • a management service function module can be set in the terminal, and the application program can be controlled through the management service function module.
  • the management service function module may specifically be a personal computer (PC) management service module.
  • the management service module recognizes the application program interface, and controls the application program that the user needs to control through the application program interface.
  • the terminal identifying the application program interface corresponding to the result of the voice recognition process includes:
  • the terminal performs semantic analysis on the result of speech recognition processing to generate a semantic analysis result
  • the terminal extracts instructions from the semantic analysis results
  • the terminal recognizes the application program interface according to the instruction.
  • the result of the speech recognition processing generated by the terminal may be text information.
  • the terminal performs semantic analysis on the text information to generate a semantic analysis result.
  • the terminal extracts instructions from the semantic analysis result. For example, the terminal generates instructions according to a preset instruction format. Finally, the terminal recognizes the application program interface according to the extracted instruction.
  • a semantic parsing function can be configured in the terminal, that is, the terminal can learn and understand the semantic content represented by a piece of text, and finally convert it into commands and parameters that can be recognized by the machine.
  • the terminal recognizes the application program interface corresponding to the result of the voice recognition process, including:
  • the terminal sends the result of the voice recognition process to the cloud server, and the cloud server performs semantic analysis on the result of the voice recognition process;
  • the terminal receives the analysis result fed back by the cloud server after semantic analysis
  • the terminal recognizes the application program interface according to the analysis result
  • the result of the voice recognition processing generated by the terminal may be text information, and the terminal establishes a communication connection with the cloud server.
  • the terminal may send the text information to the cloud server, and the cloud server performs semantic analysis on the text information.
  • the instruction is generated, the cloud server sends the instruction, the terminal can receive the analysis result fed back by the cloud server after semantic analysis, and finally the terminal recognizes the application program interface according to the extracted instruction.
  • the content associated with the first voice data is displayed on the display device according to the result of the voice recognition process.
  • the terminal controls the application
  • the terminal generates the content associated with the first voice data.
  • the control process of displaying the application on the display device connected to the terminal Since the user uses voice to issue the voice command of the application, the user does not need to hold the terminal for touch operation or use the mouse and keyboard to operate the application.
  • step 203 after the terminal controls the display device to display the content associated with the first voice data according to the result of the voice recognition process, the method for controlling the screen projection of the terminal provided in the embodiments of the present application performs the above steps
  • the terminal can also perform the following steps:
  • the terminal obtains the feedback result of the application program
  • the terminal converts the feedback result into second voice data and plays the second voice data; or,
  • the terminal displays the feedback result on the display device.
  • the application program when the terminal executes the application program, the application program may also generate a feedback result, and the feedback result may indicate that the application program successfully responds to the user's voice command, or may indicate that the application program fails to respond to the voice command.
  • the description is as follows. Taking the application as the document application as an example, if the user issues a voice command to open the document A, the terminal can control the document application to open the document A, and the document application can generate a feedback result according to the execution status of the document A. The feedback result may be that document A is successfully opened or failed.
  • the terminal can convert the feedback result into second voice data and play the second voice data, for example, a player is configured in the terminal, and the terminal can play the second voice data through the player, so that the user The second voice data can be heard.
  • the terminal can also display the feedback result on the display device, so that the user can determine whether the voice command execution is successful or failed from the terminal connected display device.
  • the application may also generate a feedback result only when the execution fails, and prompt the user of the execution failure, and the application may not generate the feedback result when the execution is successful, thereby reducing the terminal Disturb the user.
  • the terminal is connected to a display device, the terminal collects first voice data, and then the terminal performs voice recognition processing on the first voice data to generate a result of voice recognition processing, and then the terminal recognizes The result of the processing controls the application of the terminal, and finally the terminal displays the control process of the application on the display device.
  • the user can directly issue a voice command to the terminal through voice communication, and the terminal can collect the first voice data sent by the user.
  • the terminal can control the application program according to the result of the voice recognition processing, so that the execution of the application program
  • the process can display the control process on the terminal device connected to the display device without requiring the user to manually operate the terminal, thus improving the application processing efficiency in the scenario where the terminal is connected to a large screen.
  • the terminal is connected to a large screen (referred to as a large screen for short).
  • the terminal first performs speech recognition. After the user issues an instruction, the terminal converts the collected user's voice into text, and then the terminal sends the text to the cloud server for semantic analysis, that is, the cloud server parses the recognized text and converts it into a machine. Recognizable instructions and parameters.
  • the terminal finally executes the commands, that is, the terminal can execute the recognized various commands on the large screen according to the instructions and parameters.
  • the execution of various commands on the large screen means that the user feels that the application is operated on the large screen, but in actual execution, the application is still running on the terminal, just projecting the control process of the terminal on the large screen, and What is displayed on the large screen is different from the terminal, that is, the terminal executes a heterogeneous mode.
  • FIG. 3 it is a schematic diagram of an implementation architecture for terminal screen control of a document application provided by an embodiment of the present application.
  • the document application program may be a WPS document or a DOC document.
  • the lecturer is explaining the document (for example, PPT), using the mobile phone to project the screen, and the mobile phone is in a heterogeneous mode. If the lecturer is far from the mobile phone, then the mouse click method in the prior art is Cannot control applications on the big screen.
  • the lecturer can control the document application program by voice.
  • Step 1 The instructor can send a pre-trained "free wake word” command to the mobile phone to call up the voice assistant. For example, by sending the voice of " ⁇ ⁇ ⁇ ⁇ " to the mobile phone, you can call up the voice assistant and enter the listening state.
  • the voice assistant will record, and the remaining process is performed by the voice control module.
  • the role of the voice assistant is to convert the collected user voice data into text.
  • the voice assistant After receiving a command, the voice assistant sends the recorded data to the NLU module to recognize the voice and turn it into text information. Then the voice assistant will send the text information to the semantic parsing module of the cloud server. For example, the voice assistant sends the command corpus to the cloud server. The cloud server parses the text. After the cloud server parses the text, it forms the commands and parameters that the mobile phone can recognize and sends the command semantics. Give voice assistant. Then the voice assistant sends it to the phone. The mobile phone executes the corresponding command, WPS is turned on, the mobile phone is connected to the display or the TV displays the operation process of the mobile phone projecting the past document application. Next, the phone sends feedback of the command to the voice assistant. Finally, the voice assistant broadcasts feedback to the lecturer.
  • WPS is turned on
  • the mobile phone is connected to the display or the TV displays the operation process of the mobile phone projecting the past document application.
  • the phone sends feedback of the command to the voice assistant.
  • the voice assistant broadcasts feedback to
  • the instructor can continue to speak the following commands to give a complete PPT explanation.
  • the instructor can issue the following voice commands: "Open second document”, “Play”, “Next page”, “Previous page”, “Exit”, “Close”.
  • the lecturer can also say “maximize”, “minimize”, “full screen”, etc., to control the windows of WPS or other applications accordingly.
  • the system architecture consists of the following typical modules:
  • the voice assistant can receive user voice input, and then perform speech recognition through NLU into text, and then send it to the cloud server for semantic recognition. After being recognized by the cloud server, it is sent to the PC management service module (such as PC Service) of the mobile phone through the voice assistant on the mobile phone.
  • the PC Service is a newly added system service in the mobile phone, and is the server that manages the projection of heterogeneous modes on the mobile phone.
  • the voice assistant can also broadcast feedback of the execution results sent by the PC Service.
  • the cloud server parses the text to form commands and parameters that the PC Service can recognize.
  • the window management system in the mobile phone controls the window size.
  • the window management system may include: a dynamic management service module (ActivityManagerService), and may also include a window management service (WindowManagerService) module, for example, a dynamic management service module is used to control the window size, such as Maximize, minimize, full screen, close, etc.
  • ActivityManagerService and WindowManagerService are android applications and window management modules on mobile phones.
  • PC Service calls the application programming interfaces (application programming interface, API) of these two services to control the window.
  • the PC, Service, ActivityManagerService, and WindowManagerService are all in the Android system service, and the PC Service can call ActivityManagerService and WindowManagerService.
  • PC Service maps all commands, and then selects the interface of the appropriate object module to run. According to the result of command execution, form feedback to voice assistant.
  • the window is maximized and minimized, these are what the ActivityManagerService and WindowManagerService can do, then the PC Service calls their API.
  • PC Service and WPS module are required to cooperate.
  • PC Service sends a command to WPS module, and then they execute and notify the result after execution.
  • the application may be a document application (for example, a WPS application), a game application, or an audio and video application.
  • FIG. 4 it is a schematic flowchart of voice control of a document application program provided by an embodiment of the present application.
  • the user may need to free his hands when using the large screen for a period of time. It is expected that through voice communication, in this embodiment of the present application, the user can directly issue commands to the mobile phone, execute instructions on the large screen, and make appropriate feedback when necessary.
  • the user wants to open a PPT file for browsing, and then close it after browsing.
  • the user can send a series of commands on the mobile phone.
  • the voice assistant in the mobile phone converts the voice command into text, and then sends it to the cloud server.
  • the cloud server generates the formatted commands and parameters after semantic analysis, and then sends it to the PC management service module of the mobile phone. And parameters are sent to the window management system of the mobile phone.
  • the window management system maximizes and minimizes the control of applications such as documents.
  • the window management system can also generate execution results and send them to the PC management service module.
  • Voice assistant broadcast feedback by voice assistant.
  • this command can open the voice assistant on the mobile phone, the mobile phone opens the voice assistant through the wake-up-free word, and automatically enters the listening state.
  • the user needs to open the office application on the large screen, and the user issues the following voice command: open WPS, then the mobile phone opens WPS on the large screen and enters the document list.
  • the user issues the following voice command: when the second document is opened, the mobile phone opens the second PPT on the list.
  • the user issues the following voice command: play, then the PPT on the large screen of the mobile phone enters the play state.
  • the mobile phone will turn the PPT to the next page. For example, if the user needs to look back at the previous page, the user issues the following voice command: Previous page, the mobile phone will turn the PPT to the previous page. For example, if the user needs to end the playback, the user issues the following voice command: Exit, the mobile phone returns the PPT to the unplayed state. For example, if the user needs to close the PPT, the user issues the following voice command: close WPS, the mobile phone closes the WPS application.
  • the large screen can be controlled by voice for mobile office.
  • FIG. 5 is a schematic structural diagram of a terminal in an embodiment of the present application.
  • the terminal is connected to a display device.
  • the terminal 500 may include: a voice collector 501, a processor 502; the processor 502 and the The voice collector 501 communicates with each other;
  • the voice collector 501 is used to collect first voice data
  • the processor 502 is configured to perform voice recognition processing on the first voice data; control the display device to display content associated with the first voice data according to the result of the voice recognition processing.
  • the processor 502 is further used to identify an application program interface corresponding to the result of the speech recognition process; control the application program through the application program interface, and Related content is displayed on the display device.
  • the processor 502 is further configured to call a management service function module through the application program interface; and control the application program through the management service function module.
  • the processor 502 is further configured to perform semantic analysis on the result of the speech recognition process to generate a semantic analysis result; extract an instruction from the semantic analysis result; according to the instruction The application program interface is identified.
  • the processor 502 is further configured to send the result of the speech recognition process to a cloud server, and the cloud server performs semantic analysis on the result of the speech recognition process; An analysis result fed back by the cloud server after semantic analysis; identifying the application program interface according to the analysis result.
  • the terminal 500 further includes: a player 503, and the player 503 is connected to the processor 502;
  • the processor 502 is further configured to obtain the feedback result of the application program after displaying the control process of the application program on the display device; convert the feedback result into second voice data to control the playback Device 503 plays the second voice data; or, controls the display device to display the feedback result.
  • the processor 502 is also used to call up the voice assistant in a wake-up-free manner
  • the voice collector 501 is configured to perform voice collection on the first voice data under the control of the voice assistant.
  • a terminal is connected to a display device, the terminal collects first voice data, and then the terminal performs voice recognition processing on the first voice data to generate a result of voice recognition processing, and then the terminal controls according to the result of the voice recognition processing
  • the application program of the terminal and finally the terminal displays the control process of the application program on the display device.
  • the user can directly issue a voice command to the terminal through voice communication, and the terminal can collect the first voice data sent by the user.
  • the terminal can control the application program according to the result of the voice recognition processing, so that the execution of the application program
  • the process can display the control process on the terminal device connected to the display device without requiring the user to manually operate the terminal, thus improving the application processing efficiency in the scenario where the terminal is connected to a large screen.
  • an embodiment of the present application further provides a terminal 600.
  • the terminal 600 is connected to a display device.
  • the terminal 600 includes:
  • the voice recognition module 602 is configured to perform voice recognition processing on the first voice data
  • the display module 603 is configured to control the display device to display the content associated with the first voice data according to the result of the voice recognition process.
  • the display module 603 includes:
  • the interface recognition unit 6031 is configured to recognize an application program interface corresponding to the result of the voice recognition process
  • the control unit 6032 is configured to control the application program through the application program interface and display related content on the display device.
  • the interface recognition unit 6031 is configured to perform semantic analysis on the result of the speech recognition process to generate a semantic analysis result; extract an instruction from the semantic analysis result; according to the instruction The application program interface is identified.
  • the interface recognition unit 6031 is configured to send the result of the voice recognition process to a cloud server, and the cloud server performs semantic analysis on the result of the voice recognition process; An analysis result fed back by the cloud server after semantic analysis; identifying the application program interface according to the analysis result.
  • the terminal 600 further includes: an obtaining module 604 and a playing module 605, where,
  • the obtaining module 604 is configured to obtain the feedback result of the application program after the display module 603 displays the control process of the application program on the display device;
  • the playback module 605 is configured to convert the feedback result into second voice data and play the second voice data; or,
  • the display module 603 is also used to display the feedback result on the display device.
  • An embodiment of the present application further provides a computer storage medium, wherein the computer storage medium stores a program, and the program executes some or all of the steps described in the foregoing method embodiments.
  • the terminal may include: a processor 131 (for example, a CPU), a memory 132, a transmitter 134, and a receiver 133; a transmitter 134 and a receiver 133 is coupled to the processor 131, and the processor 131 controls the transmission action of the transmitter 134 and the reception action of the receiver 133.
  • the memory 132 may include a high-speed RAM memory, or may also include a non-volatile memory NVM, for example, at least one magnetic disk memory, and various instructions may be stored in the memory 132 for performing various processing functions and implementing the method of the embodiments of the present application step.
  • the terminal involved in the embodiment of the present application may further include one or more of a power supply 135, a communication bus 136, and a communication port 137.
  • the receiver 133 and the transmitter 134 may be integrated in the transceiver of the terminal, or may be separate receiving and transmitting antennas on the terminal.
  • the communication bus 136 is used to realize the communication connection between the elements.
  • the above communication port 137 is used to implement connection communication between the terminal and other peripheral devices.
  • the above memory 132 is used to store computer executable program code, and the program code includes instructions; when the processor 131 executes the instruction, the instruction causes the processor 131 to perform the processing action of the terminal in the above method embodiment, so The implement 134 performs the sending action of the terminal in the above method embodiment, and its implementation principle and technical effect are similar, and will not be repeated here.
  • the chip when the terminal is a chip, the chip includes: a processing unit and a communication unit, the processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin, or Circuit etc.
  • the processing unit can execute the computer execution instructions stored in the storage unit, so that the chip in the terminal executes the wireless communication method of any one of the above-mentioned first aspects.
  • the storage unit is a storage unit in the chip, such as a register, a cache, etc.
  • the storage unit may also be a storage unit in the terminal outside the chip, such as a read-only memory (read -only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (random access memory, RAM), etc.
  • ROM read -only memory
  • RAM random access memory
  • the processor mentioned in any of the above can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more for controlling the above
  • the first aspect is an integrated circuit in which the program of the wireless communication method is executed.
  • the device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be The physical unit can be located in one place or can be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.
  • the technical solution of the present application can be embodied in the form of a software product in essence or a part that contributes to the existing technology, and the computer software product is stored in a readable storage medium, such as a computer floppy disk , U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., including several instructions to enable a computer device (which may be a personal computer, server, or network device, etc.) to perform the methods described in various embodiments of the present application .
  • a computer device which may be a personal computer, server, or network device, etc.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be from a website site, computer, server or data center Transmit to another website, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.).
  • wired such as coaxial cable, optical fiber, digital subscriber line (DSL)
  • wireless such as infrared, wireless, microwave, etc.
  • the computer-readable storage medium may be any available medium that can be stored by a computer or a data storage device including a server, a data center, and the like integrated with one or more available media.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, Solid State Disk (SSD)), or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

一种终端（500）投屏的控制方法和终端，用于提高在终端（500）连接大屏幕场景下的应用程序处理效率。一种终端（500）投屏的控制方法，该方法应用于终端（500），终端（500）与显示设备连接，方法包括：终端（500）采集第一语音数据（201）；终端（500）对第一语音数据进行语音识别处理（202）；终端（500）根据语音识别处理的结果控制显示设备上显示与第一语音数据关联的内容（203）。

Description

一种终端投屏的控制方法和终端
本申请要求于2018年10月16日提交中国国家知识产权局，申请号为CN201811204521.3、发明名称为“一种终端投屏的控制方法和终端”的中国专利申请，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及通信技术领域，尤其涉及一种终端投屏的控制方法和终端。
背景技术
目前移动终端已经广泛应用，用户经常会遇到一些场景，例如想把移动终端里的照片、视频、文档等内容分享给身边的朋友。现有技术常用的方式是打开移动终端上的小屏幕来分享给别的用户，从用户体验上讲，移动终端这块“小屏幕”绝对不能令人满意。
现有技术中出现了采用移动终端投屏的方式，即为移动终端连接一个大屏幕，用户可以操作移动终端的应用程序，通过移动终端连接大屏幕来显示用户的操作内容，从而实现基于大屏幕的内容分享。
现有技术中，需要用户手持终端，或者为终端外接鼠标、键盘来对应用程序进行控制。由于现有技术中需要用户手动的控制终端在大屏幕上显示应用程序，因此无法解放用户的双手，降低了在终端连接大屏幕场景下的应用程序处理效率。
发明内容
本申请实施例提供了一种终端投屏的控制方法和终端，用于提高在终端连接大屏幕场景下的应用程序处理效率。
为解决上述技术问题，本申请实施例提供以下技术方案：
第一方面，本申请实施例提供一种终端投屏的控制方法，所述方法应用于终端，所述终端与显示设备连接，所述方法包括：所述终端采集第一语音数据；所述终端对所述第一语音数据进行语音识别处理；所述终端根据所述语音识别处理的结果控制显示设备上显示与所述第一语音数据关联的内容。
在本申请实施例中，终端与显示设备连接，该终端采集第一语音数据，然后终端对第一语音数据进行语音识别处理，生成语音识别处理的结果，接下来终端根据语音识别处理的结果控制终端的应用程序，最后终端在显示设备上显示应用程序的控制过程。本申请实施例中用户可以通过语音交流的方式，直接向终端下达语音的命令，终端就可以采集用户发出的第一语音数据，终端可以根据语音识别处理的结果控制应用程序，从而应用程序的执行过程可以在终端设备连接的显示设备上显示控制过程，不需要用户来手动的操作终端，因此提高了在终端连接大屏幕场景下的应用程序处理效率。
在第一方面的一种可能实现中，所述终端根据所述语音识别处理的结果控制显示设备上显示与所述第一语音数据关联的内容，包括：所述终端识别出与所述语音识别处理的结果对应的应用程序接口；所述终端通过所述应用程序接口控制应用程序，并在所述显示设备上显示相关内容。其中，终端根据语音识别处理的结果来识别用户需要控制的应用程序，例如终端识别出与语音识别处理的结果对应的应用程序接口，不同的应用程序配置有不同的应用程序接口，终端在识别出应用程序接口之后，终端通过应用程序接口就可以确定出用户需要控制的应用程序。
在第一方面的一种可能实现中，所述终端识别出与所述语音识别处理的结果对应的应用程序接口，包括：所述终端对所述语音识别处理的结果进行语义解析，生成语义解析结果；所述终端从所述语义解析结果中提取出指令；所述终端根据所述指令识别出所述应用程序接口。其中，终端生成的语音识别处理的结果可以是文本信息，终端对该文本信息进行语义解析，生成语义解析结果，终端从语义解析结果中提取出指令，例如终端根据预置的指令格式生成指令，最后终端根据所提取出的指令来识别出应用程序接口。本申请实施例中，终端中可以配置语义解析功能，即终端可以学习与理解一段文本所表示的语义内容，最终转换成机器能识别的命令和参数。
在第一方面的一种可能实现中，所述终端识别出与所述语音识别处理的结果对应的应用程序接口，包括：所述终端将所述语音识别处理的结果发送给云端服务器，由所述云端服务器对所述语音识别处理的结果进行语义解析；所述终端接收所述云端服务器进行语义解析后反馈的解析结果；所述终端根据所述解析结果识别出所述应用程序接口。其中，终端生成的语音识别处理的结果可以是文本信息，终端与云端服务器建立有通信连接，例如终端可以将文本信息发送给云端服务器，由云端服务器对文本信息进行语义解析，云端服务器在语义解析完成之后生成指令，云端服务器发送指令，终端可以接收云端服务器进行语义解析后反馈的解析结果，最后终端根据所提取出的指令来识别出应用程序接口。
在第一方面的一种可能实现中，所述终端根据所述语音识别处理的结果控制显示设备上显示与所述第一语音数据关联的内容之后，所述方法还包括：所述终端获取所述应用程序的反馈结果；所述终端将所述反馈结果转换成第二语音数据，并播放所述第二语音数据；或者，所述终端在所述显示设备上显示所述反馈结果。其中，终端在执行应用程序时，应用程序还可以生成反馈结果，该反馈结果可以指示应用程序成功响应用户的语音命令，也可以指示应用程序响应该语音命令失败。终端在获取到该反馈结果之后，终端可以将反馈结果转换成第二语音数据，并播放第二语音数据，例如终端内配置有播放器，终端可以通过该播放器播放第二语音数据，从而用户可以听到该第二语音数据。终端除了使用语音的方式播放反馈结果对应的第二语音数据之外，终端还可以在显示设备上显示反馈结果，使得用户可以从终端连接显示设备上确定出语音命令执行成功或者失败。
在第一方面的一种可能实现中，所述终端采集第一语音数据，包括:所述终 端通过免唤醒词的方式调起语音助手，由所述语音助手对所述第一语音数据进行语音采集。其中，终端中可以配置语音助手，通过该语音助手可以进行语音采集，为了提高语音采集的效率，终端可以使用免唤醒词的方式来调起语音助手，免唤醒词是相对于语音助手而言的，不需要先打开语音助手这个应用，用户可以直接对着终端说某句话，终端就可以自动的调起语音助手，并执行语音命令。
第二方面，本申请实施例提供一种终端，所述终端与显示设备连接，所述终端包括：语音采集器、处理器；所述处理器和所述语音采集器之间进行相互的通信；所述语音采集器，用于采集第一语音数据；所述处理器，用于对所述第一语音数据进行语音识别处理；根据所述语音识别处理的结果控制显示设备上显示与所述第一语音数据关联的内容。
在第二方面的一种可能实现中，所述处理器，还用于识别出与所述语音识别处理的结果对应的应用程序接口；通过所述应用程序接口控制所述应用程序，并在所述显示设备上显示相关内容。
在第二方面的一种可能实现中，所述处理器，还用于通过所述应用程序接口调用管理服务功能模块；通过所述管理服务功能模块对所述应用程序进行控制。
在第二方面的一种可能实现中，所述处理器，还用于对所述语音识别处理的结果进行语义解析，生成语义解析结果；从所述语义解析结果中提取出指令；根据所述指令识别出所述应用程序接口。
在第二方面的一种可能实现中，所述处理器，还用于将所述语音识别处理的结果发送给云端服务器，由所述云端服务器对所述语音识别处理的结果进行语义解析；接收所述云端服务器进行语义解析后反馈的解析结果；根据所述解析结果识别出所述应用程序接口。
在第二方面的一种可能实现中，所述终端还包括：播放器，所述播放器和所述处理器相连接；所述处理器，还用于根据所述语音识别处理的结果控制显示设备上显示与所述第一语音数据关联的内容之后，获取所述应用程序的反馈结果；将所述反馈结果转换成第二语音数据，控制所述播放器播放所述第二语音数据；或者，控制所述显示设备显示所述反馈结果。
在第二方面的一种可能实现中，所述处理器，还用于通过免唤醒词的方式调起语音助手；所述语音采集器，用于在所述语音助手的控制下对所述第一语音数据进行语音采集。
在本申请的第二方面中，终端的组成模块还可以执行前述第一方面以及各种可能的实现方式中所描述的步骤，详见前述对第一方面以及各种可能的实现方式中的说明。
第三方面，本申请实施例还提供一种终端，所述终端与显示设备连接，所述终端包括：
采集模块，用于采集第一语音数据；
语音识别模块，用于对所述第一语音数据进行语音识别处理；
显示模块，用于根据所述语音识别处理的结果控制显示设备上显示与所述 第一语音数据关联的内容。
在第三方面的一种可能的实现方式中，所述显示模块，包括：接口识别单元，用于识别出与所述语音识别处理的结果对应的应用程序接口；控制单元，用于通过所述应用程序接口控制应用程序，并在所述显示设备上显示相关内容。
在第三方面的一种可能的实现方式中，所述接口识别单元，用于对所述语音识别处理的结果进行语义解析，生成语义解析结果；从所述语义解析结果中提取出指令；根据所述指令识别出所述应用程序接口。
在第三方面的一种可能的实现方式中，所述接口识别单元，用于将所述语音识别处理的结果发送给云端服务器，由所述云端服务器对所述语音识别处理的结果进行语义解析；接收所述云端服务器进行语义解析后反馈的解析结果；根据所述解析结果识别出所述应用程序接口。
在第三方面的一种可能的实现方式中，所述终端，还包括：获取模块和播放模块，其中，所述获取模块，用于所述显示模块在根据所述语音识别处理的结果控制显示设备上显示与所述第一语音数据关联的内容之后，获取所述应用程序的反馈结果；所述播放模块，用于将所述反馈结果转换成第二语音数据，并播放所述第二语音数据；或者，所述显示模块，还用于在所述显示设备上显示所述反馈结果。
在第三方面的一种可能的实现方式中，所述采集模块，还用于通过免唤醒词的方式调起语音助手，由所述语音助手对所述第一语音数据进行语音采集。
第三方面，本申请实施例提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有指令，当其在计算机上运行时，使得计算机执行上述第一方面所述的方法。
第四方面，本申请实施例提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述第一方面所述的方法。
第五方面，本申请实施例提供一种通信装置，该通信装置可以包括终端或者芯片等实体，所述通信装置包括：处理器、存储器；所述存储器用于存储指令；所述处理器用于执行所述存储器中的所述指令，使得所述通信装置执行如前述第一方面中任一项所述的方法。
第六方面，本申请提供了一种芯片系统，该芯片系统包括处理器，用于支持终端实现上述方面中所涉及的功能，例如，发送或处理上述方法中所涉及的数据和/或信息。在一种可能的设计中，所述芯片系统还包括存储器，所述存储器，用于保存终端必要的程序指令和数据。该芯片系统，可以由芯片构成，也可以包括芯片和其他分立器件。
附图说明
图1为本申请实施例提供的一种终端投屏的控制方法应用的通信系统的组成结构示意图；
图2为本申请实施例提供的一种终端投屏的控制方法的流程方框示意图；
图3为本申请实施例提供的对文档应用程序进行终端投屏控制的实现架构示意图；
图4为本申请实施例提供的对文档应用程序进行语音控制的流程示意图；
图5为本申请实施例提供的一种终端的组成结构示意图；
图6-a为本申请实施例提供的另一种终端的组成结构示意图；
图6-b为本申请实施例提供的一种显示模块的组成结构示意图；
图6-c为本申请实施例提供的另一种终端的组成结构示意图；
图7为本申请实施例提供的另一种终端的组成结构示意图。
具体实施方式
本申请实施例提供了一种终端投屏的控制方法和终端，用于提高在终端连接大屏幕场景下的应用程序处理效率。
下面结合附图，对本申请的实施例进行描述。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的术语在适当情况下可以互换，这仅仅是描述本申请的实施例中对相同属性的对象在描述时所采用的区分方式。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，以便包含一系列单元的过程、方法、系统、产品或设备不必限于那些单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它单元。
本申请实施例的技术方案可以应用于各种数据处理的通信系统，如图1所示，该通信系统中包括有终端，该终端与显示设备连接，该显示设备可以是一个用于显示的大屏幕，终端可以通过有线或者无线的方式连接该显示设备，例如终端通过高清晰度多媒体接口(high definition multimedia interface，HDMI)和显示设备连接，或者终端通过type-c接口和显示设备连接。其中，终端又称之为用户设备(user equipment，UE)、移动台(mobile station，MS)、移动终端(mobile terminal，MT)等，是一种向用户提供语音和/或数据连通性的设备，或，设置于该设备内的芯片，例如，具有无线连接功允许的手持式设备、车载设备等。目前，一些终端的举例为：手机(mobile phone)、平板电脑、笔记本电脑、掌上电脑、移动互联网设备(mobile internet device，MID)、可穿戴设备，虚拟现实(virtual reality，VR)设备、增强现实(augmented reality，AR)设备、工业控制(industrial control)中的无线终端、无人驾驶(self driving)中的无线终端、远程手术(remote medical surgery)中的无线终端、智能电网(smart grid)中的无线终端、运输安全(transportation safety)中的无线终端、智慧城市(smart city)中的无线终端、智慧家庭(smart home)中的无线终端等。本申请实施例提供的终端只需要连接一显示设备，就可以执行本申请实施例提供的终端投屏的控制方法。
本申请实施例提出终端投屏的控制方法，该方法应用于终端，该终端与显示设备连接，请参阅图2所示，为本申请实施例提供的终端投屏的控制方法，主要包括如下步骤：
201、终端采集第一语音数据。
在本申请实施例中，用户可以通过终端来操作应用程序，应用程序的类型 不做限定，例如该应用程序可以是文档应用程序，也可以是游戏应用程序，或者音视频应用程序等。在该终端连接显示设备上显示该应用程序，当用户需要操作该应用程序时采用语音控制的方式，即用户发出语音命令，终端中内置有语音采集器，终端通过该语音采集器来采集用户发出的语音命令。例如终端在一段时间内采集到第一语音数据。后续实施例中以对第一语音数据的终端投屏控制过程为例说明，终端采集到的其它语音数据，也可以按照该第一语音数据的处理过程进行终端投屏控制，此处仅作说明。
在本申请的一些实施例中，步骤201终端采集第一语音数据，包括:
终端通过免唤醒词的方式调起语音助手，由语音助手对第一语音数据进行语音采集。
其中，终端中可以配置语音助手，通过该语音助手可以进行语音采集，为了提高语音采集的效率，终端可以使用免唤醒词的方式来调起语音助手，免唤醒词是相对于语音助手而言的，不需要先打开语音助手这个应用，用户可以直接对着终端说某句话，终端就可以自动的调起语音助手，并执行语音命令。
202、终端对第一语音数据进行语音识别处理。
在本申请实施例中，终端在采集到第一语音数据之后，终端对该第一语音数据进行语音识别处理，以识别出该第一语音数据对应的文本信息，终端生成的语音识别处理的结果可以包括该文本信息。
本申请实施例中，终端可以通过自然语音理解(natural language understanding，NLU)工具对第一语音数据进行语音识别处理，其中，语音识别是指让机器通过识别和理解过程把第一语音数据转变为相应的文本信息的过程，终端生成的语音识别处理的结果可以用于控制终端的应用程序。
203、终端根据语音识别处理的结果控制显示设备上显示与第一语音数据关联的内容。
在本申请实施例中，终端在对第一语音数据进行语音识别处理之后，终端可以使用该语音识别处理的结果进行应用程序的控制，终端可以直接以该语音识别处理的结果作为命令来控制应用程序，终端也可以获取语音识别处理的结果对应的指令，根据该指令来控制应用程序。其中，对应用程序进行控制的方式取决于终端生成的语音识别处理的结果。以应用程序为文档应用程序为例，若用户发出打开文档A的语音命令，则终端可以控制文档应用程序打开该文档A。
在本申请的一些实施例中，步骤203终端根据语音识别处理的结果控制显示设备上显示与第一语音数据关联的内容，包括：
终端识别出与语音识别处理的结果对应的应用程序接口；
终端通过应用程序接口控制应用程序，并在显示设备上显示相关内容。
其中，终端根据语音识别处理的结果来识别用户需要控制的应用程序。例如终端识别出与语音识别处理的结果对应的应用程序接口，不同的应用程序配置有不同的应用程序接口，终端在识别出应用程序接口之后，终端通过应用程序接口就可以确定出用户需要控制的应用程序。在实际应用中，终端中可以设置管理服务功能模块，通过管理服务功能模块对所述应用程序进行控制，例如 该管理服务功能模块具体可以是个人电脑(personal computer，PC)管理服务模块，通过PC管理服务模块来识别出应用程序接口，通过应用程序接口来控制用户需要控制的应用程序。
进一步的，在本申请的一些实施例中，终端识别出与语音识别处理的结果对应的应用程序接口，包括：
终端对语音识别处理的结果进行语义解析，生成语义解析结果；
终端从语义解析结果中提取出指令；
终端根据指令识别出应用程序接口。
其中，终端生成的语音识别处理的结果可以是文本信息，终端对该文本信息进行语义解析，生成语义解析结果，终端从语义解析结果中提取出指令，例如终端根据预置的指令格式生成指令，最后终端根据所提取出的指令来识别出应用程序接口。本申请实施例中，终端中可以配置语义解析功能，即终端可以学习与理解一段文本所表示的语义内容，最终转换成机器能识别的命令和参数。
在本申请的另一些实施例中，终端识别出与语音识别处理的结果对应的应用程序接口，包括：
终端将语音识别处理的结果发送给云端服务器，由云端服务器对语音识别处理的结果进行语义解析；
终端接收云端服务器进行语义解析后反馈的解析结果；
终端根据解析结果识别出应用程序接口
其中，终端生成的语音识别处理的结果可以是文本信息，终端与云端服务器建立有通信连接，例如终端可以将文本信息发送给云端服务器，由云端服务器对文本信息进行语义解析，云端服务器在语义解析完成之后生成指令，云端服务器发送指令，终端可以接收云端服务器进行语义解析后反馈的解析结果，最后终端根据所提取出的指令来识别出应用程序接口。
在本申请实施例中，根据语音识别处理的结果控制显示设备上显示与第一语音数据关联的内容，终端在控制应用程序时，终端生成与第一语音数据关联的内容，基于该相关内容在终端连接的显示设备上显示应用程序的控制过程，由于用户使用语音来下发应用程序的语音命令，因此用户不需要手持终端进行触摸操作，也不需要使用鼠标键盘来操作应用程序，提高了在终端连接大屏幕场景下的应用程序处理效率。
在本申请的一些实施例中，步骤203终端根据语音识别处理的结果控制显示设备上显示与第一语音数据关联的内容之后，本申请实施例提供的终端投屏的控制方法除了执行前述步骤之外，终端还可以执行如下步骤：
终端获取应用程序的反馈结果；
终端将反馈结果转换成第二语音数据，并播放第二语音数据；或者，
终端在显示设备上显示反馈结果。
其中，终端在执行应用程序时，应用程序还可以生成反馈结果，该反馈结果可以指示应用程序成功响应用户的语音命令，也可以指示应用程序响应该语音命令失败。举说明如下，以应用程序为文档应用程序为例，若用户发出打开文档A的语音命令，则终端可以控制文档应用程序打开该文档A，文档应用程 序可以根据文档A的执行情况生成反馈结果，该反馈结果可以是文档A打开成功或者打开失败。终端在获取到该反馈结果之后，终端可以将反馈结果转换成第二语音数据，并播放第二语音数据，例如终端内配置有播放器，终端可以通过该播放器播放第二语音数据，从而用户可以听到该第二语音数据。终端除了使用语音的方式播放反馈结果对应的第二语音数据之外，终端还可以在显示设备上显示反馈结果，使得用户可以从终端连接显示设备上确定出语音命令执行成功或者失败。
在本申请的一些实施例中，应用程序还可以只在执行失败的情况下生成反馈结果，并向用户提示执行失败，而在执行成功的情况下应用程序可以不生成反馈结果，从而减少了终端对用户的打扰。
通过前述实施例对本申请的举例说明可知，终端与显示设备连接，该终端采集第一语音数据，然后终端对第一语音数据进行语音识别处理，生成语音识别处理的结果，接下来终端根据语音识别处理的结果控制终端的应用程序，最后终端在显示设备上显示应用程序的控制过程。本申请实施例中用户可以通过语音交流的方式，直接向终端下达语音的命令，终端就可以采集用户发出的第一语音数据，终端可以根据语音识别处理的结果控制应用程序，从而应用程序的执行过程可以在终端设备连接显示设备上显示控制过程，不需要用户来手动的操作终端，因此提高了在终端连接大屏幕场景下的应用程序处理效率。
为便于更好的理解和实施本申请实施例的上述方案，下面举例相应的应用场景来进行具体说明。
本申请实施例提供的终端投屏的控制方法中，终端连接有一个大屏幕(简称为大屏)。终端首先进行语音识别，用户发出指令后终端把采集到用户的声音转换成文本，然后终端发送文本给云端服务器，由云端服务器进行语义解析，即云端服务器将识别到的文本进行解析，转换成机器能识别的指令和参数。终端最后进行命令的执行，即终端可以根据指令和参数，在大屏上执行识别后的各种命令。其中，在大屏上执行各种命令是指用户感觉是在大屏上操作应用程序，但实际执行中，应用程序还是运行在终端上，只是将这终端的控制过程投射到大屏上，而且大屏上显示的东西与终端不一样，即终端执行的是异源模式。
如图3所示，为本申请实施例提供的对文档应用程序进行终端投屏控制的实现架构示意图。以应用程序为文档应用程序、终端为手机为例，该文档应用程序可以是WPS文档，也可以是DOC文档。在一种实际应用场景中，讲师在进行文档(例如PPT)讲解，使用手机进行投屏，并且手机处于异源模式，如果讲师离手机较远，那么按照现有技术中采用鼠标点击的方式是无法控制大屏上的应用程序。本申请实施例中，讲师可以通过语音的方式来控制文档应用程序。
采用本申请实施例的操作流程如下：
步骤1、讲师可以向手机发出事先训练好的“免唤醒词”命令调起语音助手，比如通过对手机发送“小艺小艺”的语音，就可以调起语音助手并进入聆听状态。
步骤2、讲师说“打开WPS”。
语音助手会录音，剩下的过程由语音控制模块执行，语音助手的作用就是 将采集的用户语音数据转化成文本。
例如，语音助手接收到命令之后，会把录音数据发给NLU模块，对语音进行识别，成为文本信息。然后语音助手会把文本信息发给云端服务器的语义解析模块，例如语音助手发送命令语料给云端服务器，云端服务器解析文本，云端服务器解析出文本后，形成手机能识别的指令和参数，发送命令语义给语音助手。然后语音助手发给手机。手机执行对应的命令，WPS被打开，手机连接显示器或者电视上显示手机投射过去文档应用程序的操作过程。接下来手机向语音助手发送命令的反馈。最后语音助手播报反馈给讲师。
接下来，讲师可以继续说下面的命令，进行一次完整的PPT讲解。例如，讲师可以发出如下语音命令：“打开第二个文档”、“播放”、“下一页”、“上一页”、“退出”、“关闭”。当然，讲师还可以说“最大化”、“最小化”、“全屏”等，对WPS或者其它应用的窗口进行相应控制。
接下来对本申请实施例提供的系统架构进行说明，以Android系统为例。该系统架构由如下典型模块组成：
首先对语音助手进行说明，该语音助手可以接收用户语音输入，然后通过NLU进行语音识别成文本，再发给云端服务器进行语义识别。云端服务器识别后通过手机上的语音助手发给手机的PC管理服务模块(例如PC Service)执行，PC Service是手机中新加的一个系统服务，是手机上的管理异源模式投射的服务端。语音助手还可以播报由PC Service发送执行的结果的反馈。
云端服务器解析文本，形成PC Service可以识别的命令和参数。
手机中的窗口管理系统对窗口大小进行控制，例如，窗口管理系统可以包括：动态管理服务模块(ActivityManagerService)，还可以包括窗口管理服务(WindowManagerService)模块，例如使用动态管理服务模块控制窗口大小，如最大化、最小化、全屏、关闭等。其中，ActivityManagerService和WindowManagerService都是手机上的android的应用、窗口管理模块。PC Service调用这两个服务的应用程序编程接口(application programming interface，API)对窗口进行控制。
接下来对管理服务的实现过程进行说明，PC Service、ActivityManagerService、WindowManagerService都在Android系统服务里，PC Service可以调用ActivityManagerService、WindowManagerService。PC Service对于所有命令进行映射，然后选择合适对象模块的接口运行。根据命令执行的结果，形成反馈发给语音助手。举例说明如下，窗口最大化、最小化，这些是ActivityManagerService、WindowManagerService能做的，那么PC Service就调用它们的API。但是打开WPS中的第N个文档，那就需要PC Service和WPS模块进行合作，PC Service给WPS模块发命令，然后它们执行，执行完通知结果。
最后说明本申请实施例中的应用程序，该应用程序可以是文档应用程序(例如WPS应用程序)，或者游戏应用程序，或者音视频应用程序等。
如图4所示，为本申请实施例提供的对文档应用程序进行语音控制的流程示意图。用户在使用一段时间大屏时，可能需要解放双手，期望通过语音交流 的方式，本申请实施例中用户可以直接向手机下达命令，在大屏上执行指令，必要的时候进行适当的反馈。比如说：用户要打开一个PPT文档进行浏览，浏览后关闭。用户可以手机发送一系列命令。手机中的语音助手将语音命令转换成文本，然后发给云端服务器，云端服务器在进行语义解析后生成格式化的命令和参数，然后发送给手机的PC管理服务模块，然后PC管理服务模块将命令和参数发送给手机的窗口管理系统，窗口管理系统对文档等应用程序进行最大化、最小化等控制，窗口管理系统还可以生成执行结果发送给PC管理服务模块，PC管理服务模块发送执行结果给语音助手，由语音助手进行播报反馈。
例如，该命令可以在手机上打开语音助手，手机通过免唤醒词打开语音助手，并自动进入聆听状态。例如用户需要在大屏上打开办公应用，用户发出如下语音命令：打开WPS，则手机在大屏上打开WPS，进入文档列表。例如用户需要打开文档列表某个PPT文档，用户发出如下语音命令：打开第二个文档，则手机打开列表上第二个PPT。例如用户需要播放PPT，用户发出如下语音命令：播放，则手机大屏上的PPT进入播放状态。例如用户需要进入下一页，用户发出如下语音命令：下一页，则手机将PPT翻到下一页。例如用户需要回看上一页，用户发出如下的语音命令：上一页，则手机将PPT翻到上一页。例如用户需要结束播放，用户发出如下的语音命令：退出，则手机将PPT返回到未播放状态。例如用户需要关闭PPT，用户发出如下的语音命令：关闭WPS，则手机将WPS应用关闭。
通过本申请实施例提供的终端投屏的控制方法，可以通过语音控制大屏进行移动办公。
需要说明的是，对于前述的各方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本申请并不受所描述的动作顺序的限制，因为依据本申请，某些步骤可以采用其他顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作和模块并不一定是本申请所必须的。
为便于更好的实施本申请实施例的上述方案，下面还提供用于实施上述方案的相关装置。
请参阅如图5所示，为本申请实施例中终端的组成结构示意图，所述终端与显示设备连接，终端500可以包括：语音采集器501、处理器502；所述处理器502和所述语音采集器501之间进行相互的通信；
所述语音采集器501，用于采集第一语音数据；
所述处理器502，用于对所述第一语音数据进行语音识别处理；根据所述语音识别处理的结果控制显示设备上显示与所述第一语音数据关联的内容。
在本申请的一些实施例中，所述处理器502，还用于识别出与所述语音识别处理的结果对应的应用程序接口；通过所述应用程序接口控制所述应用程序，并在所述显示设备上显示相关内容。
在本申请的一些实施例中，所述处理器502，还用于通过所述应用程序接口调用管理服务功能模块；通过所述管理服务功能模块对所述应用程序进行控制。
在本申请的一些实施例中，所述处理器502，还用于对所述语音识别处理的 结果进行语义解析，生成语义解析结果；从所述语义解析结果中提取出指令；根据所述指令识别出所述应用程序接口。
在本申请的一些实施例中，所述处理器502，还用于将所述语音识别处理的结果发送给云端服务器，由所述云端服务器对所述语音识别处理的结果进行语义解析；接收所述云端服务器进行语义解析后反馈的解析结果；根据所述解析结果识别出所述应用程序接口。
在本申请的一些实施例中，如图5所示，所述终端500还包括：播放器503，所述播放器503和所述处理器502相连接；
所述处理器502，还用于在所述显示设备上显示所述应用程序的控制过程之后，获取所述应用程序的反馈结果；将所述反馈结果转换成第二语音数据，控制所述播放器503播放所述第二语音数据；或者，控制所述显示设备显示所述反馈结果。
在本申请的一些实施例中，所述处理器502，还用于通过免唤醒词的方式调起语音助手；
所述语音采集器501，用于在所述语音助手的控制下对所述第一语音数据进行语音采集。
在本申请实施例中，终端与显示设备连接，该终端采集第一语音数据，然后终端对第一语音数据进行语音识别处理，生成语音识别处理的结果，接下来终端根据语音识别处理的结果控制终端的应用程序，最后终端在显示设备上显示应用程序的控制过程。本申请实施例中用户可以通过语音交流的方式，直接向终端下达语音的命令，终端就可以采集用户发出的第一语音数据，终端可以根据语音识别处理的结果控制应用程序，从而应用程序的执行过程可以在终端设备连接显示设备上显示控制过程，不需要用户来手动的操作终端，因此提高了在终端连接大屏幕场景下的应用程序处理效率。
如图6-a所示，本申请实施例还提供一种终端600，所述终端600与显示设备连接，所述终端600包括：
采集模块601，用于采集第一语音数据；
语音识别模块602，用于对所述第一语音数据进行语音识别处理；
显示模块603，用于根据所述语音识别处理的结果控制显示设备上显示与所述第一语音数据关联的内容。
在本申请的一些实施例中，如图6-b所示，所述显示模块603，包括：
接口识别单元6031，用于识别出与所述语音识别处理的结果对应的应用程序接口；
控制单元6032，用于通过所述应用程序接口控制所述应用程序，并在所述显示设备上显示相关内容。
在本申请的一些实施例中，所述接口识别单元6031，用于对所述语音识别处理的结果进行语义解析，生成语义解析结果；从所述语义解析结果中提取出指令；根据所述指令识别出所述应用程序接口。
在本申请的一些实施例中，所述接口识别单元6031，用于将所述语音识别处理的结果发送给云端服务器，由所述云端服务器对所述语音识别处理的结果 进行语义解析；接收所述云端服务器进行语义解析后反馈的解析结果；根据所述解析结果识别出所述应用程序接口。
在本申请的一些实施例中，如图6-c所示，所述终端600，还包括：获取模块604和播放模块605，其中，
所述获取模块604，用于所述显示模块603在所述显示设备上显示所述应用程序的控制过程之后，获取所述应用程序的反馈结果；
所述播放模块605，用于将所述反馈结果转换成第二语音数据，并播放所述第二语音数据；或者，
所述显示模块603，还用于在所述显示设备上显示所述反馈结果。
需要说明的是，上述装置各模块/单元之间的信息交互、执行过程等内容，由于与本申请方法实施例基于同一构思，其带来的技术效果与本申请方法实施例相同，具体内容可参见本申请前述所示的方法实施例中的叙述，此处不再赘述。
本申请实施例还提供一种计算机存储介质，其中，该计算机存储介质存储有程序，该程序执行包括上述方法实施例中记载的部分或全部步骤。
如图7所示，为本申请实施例的又一种终端的结构示意图，该终端可以包括：处理器131(例如CPU)、存储器132、发送器134和接收器133；发送器134和接收器133耦合至处理器131，处理器131控制发送器134的发送动作和接收器133的接收动作。存储器132可能包含高速RAM存储器，也可能还包括非易失性存储器NVM，例如至少一个磁盘存储器，存储器132中可以存储各种指令，以用于完成各种处理功能以及实现本申请实施例的方法步骤。可选的，本申请实施例涉及的终端还可以包括：电源135、通信总线136以及通信端口137中的一个或多个。接收器133和发送器134可以集成在终端的收发器中，也可以为终端上分别独立的收、发天线。通信总线136用于实现元件之间的通信连接。上述通信端口137用于实现终端与其他外设之间进行连接通信。
在本申请实施例中，上述存储器132用于存储计算机可执行程序代码，程序代码包括指令；当处理器131执行指令时，指令使处理器131执行上述方法实施例中终端的处理动作，使发送器134执行上述方法实施例中终端的发送动作，其实现原理和技术效果类似，在此不再赘述。
在另一种可能的设计中，当该终端为芯片时，芯片包括：处理单元和通信单元，所述处理单元例如可以是处理器，所述通信单元例如可以是输入/输出接口、管脚或电路等。该处理单元可执行存储单元存储的计算机执行指令，以使该终端内的芯片执行上述第一方面任意一项的无线通信方法。可选地，所述存储单元为所述芯片内的存储单元，如寄存器、缓存等，所述存储单元还可以是所述终端内的位于所述芯片外部的存储单元，如只读存储器(read-only memory，ROM)或可存储静态信息和指令的其他类型的静态存储设备，随机存取存储器(random access memory，RAM)等。
其中，上述任一处提到的处理器，可以是一个通用中央处理器(CPU)，微处理器，特定应用集成电路(application-specific integrated circuit，ASIC)，或一个或多个用于控制上述第一方面无线通信方法的程序执行的集成电路。
另外需说明的是，以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。另外，本申请提供的装置实施例附图中，模块之间的连接关系表示它们之间具有通信连接，具体可以实现为一条或多条通信总线或信号线。
通过以上的实施方式的描述，所属领域的技术人员可以清楚地了解到本申请可借助软件加必需的通用硬件的方式来实现，当然也可以通过专用硬件包括专用集成电路、专用CPU、专用存储器、专用元器件等来实现。一般情况下，凡由计算机程序完成的功能都可以很容易地用相应的硬件来实现，而且，用来实现同一功能的具体硬件结构也可以是多种多样的，例如模拟电路、数字电路或专用电路等。但是，对本申请而言更多情况下软件程序实现是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在可读取的存储介质中，如计算机的软盘、U盘、移动硬盘、ROM、RAM、磁碟或者光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述的方法。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。
所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存储的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘(Solid State Disk，SSD))等。

Claims (15)

  1. 一种终端投屏的控制方法，其特征在于，所述方法应用于终端，所述终端与显示设备连接，所述方法包括：
    所述终端采集第一语音数据；
    所述终端对所述第一语音数据进行语音识别处理；
    所述终端根据所述语音识别处理的结果控制显示设备上显示与所述第一语音数据关联的内容。
  2. 根据权利要求1所述的方法，其特征在于，所述终端根据所述语音识别处理的结果控制显示设备上显示与所述第一语音数据关联的内容，包括：
    所述终端识别出与所述语音识别处理的结果对应的应用程序接口；
    所述终端通过所述应用程序接口控制应用程序，并在所述显示设备上显示相关内容。
  3. 根据权利要求2所述的方法，其特征在于，所述终端识别出与所述语音识别处理的结果对应的应用程序接口，包括：
    所述终端对所述语音识别处理的结果进行语义解析，生成语义解析结果；
    所述终端根据所述语义解析结果识别出所述应用程序接口。
  4. 根据权利要求2所述的方法，其特征在于，所述终端识别出与所述语音识别处理的结果对应的应用程序接口，包括：
    所述终端将所述语音识别处理的结果发送给云端服务器，由所述云端服务器对所述语音识别处理的结果进行语义解析；
    所述终端接收所述云端服务器进行语义解析后反馈的解析结果；
    所述终端根据所述解析结果识别出所述应用程序接口。
  5. 根据权利要求1至4中任一项所述的方法，其特征在于，所述终端根据所述语音识别处理的结果控制显示设备上显示与所述第一语音数据关联的内容之后，所述方法还包括：
    所述终端获取所述应用程序的反馈结果；
    所述终端将所述反馈结果转换成第二语音数据，并播放所述第二语音数据；或者，
    所述终端在所述显示设备上显示所述反馈结果。
  6. 根据权利要求1至5中任一项所述的方法，其特征在于，所述终端采集第一语音数据，包括:
    所述终端通过免唤醒词的方式调起语音助手，由所述语音助手对所述第一语音数据进行语音采集。
  7. 一种终端，其特征在于，所述终端与显示设备连接，所述终端包括：语音采集器、处理器；所述处理器和所述语音采集器之间进行相互的通信；
    所述语音采集器，用于采集第一语音数据；
    所述处理器，用于对所述第一语音数据进行语音识别处理；根据所述语音识别处理的结果控制显示设备上显示与所述第一语音数据关联的内容。
  8. 根据权利要求7所述的终端，其特征在于，所述处理器，还用于识别出 与所述语音识别处理的结果对应的应用程序接口；通过所述应用程序接口控制所述应用程序，并在所述显示设备上显示相关内容。
  9. 根据权利要求8所述的终端，其特征在于，所述处理器，还用于通过所述应用程序接口调用管理服务功能模块；通过所述管理服务功能模块对所述应用程序进行控制。
  10. 根据权利要求8或9所述的终端，其特征在于，所述处理器，还用于对所述语音识别处理的结果进行语义解析，生成语义解析结果；根据所述语义解析结果识别出所述应用程序接口。
  11. 根据权利要求8或9所述的终端，其特征在于，所述处理器，还用于将所述语音识别处理的结果发送给云端服务器，由所述云端服务器对所述语音识别处理的结果进行语义解析；接收所述云端服务器进行语义解析后反馈的解析结果；根据所述解析结果识别出所述应用程序接口。
  12. 根据权利要求7至11中任一项所述的终端，其特征在于，所述终端还包括：播放器，所述播放器和所述处理器相连接；
    所述处理器，还用于根据所述语音识别处理的结果控制显示设备上显示与所述第一语音数据关联的内容之后，获取所述应用程序的反馈结果；将所述反馈结果转换成第二语音数据，控制所述播放器播放所述第二语音数据；或者，控制所述显示设备显示所述反馈结果。
  13. 根据权利要求7至12中任一项所述的终端，其特征在于，所述处理器，还用于通过免唤醒词的方式调起语音助手；
    所述语音采集器，用于在所述语音助手的控制下对所述第一语音数据进行语音采集。
  14. 一种计算机可读存储介质，包括指令，当其在计算机上运行时，使得计算机执行如权利要求1-6任意一项所述的方法。
  15. 一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行如权利要求1-6任意一项所述的方法。
PCT/CN2019/110926 2018-10-16 2019-10-14 一种终端投屏的控制方法和终端 WO2020078300A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/285,563 US20210398527A1 (en) 2018-10-16 2019-10-14 Terminal screen projection control method and terminal

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811204521.3A CN109448709A (zh) 2018-10-16 2018-10-16 一种终端投屏的控制方法和终端
CN201811204521.3 2018-10-16

Publications (1)

Publication Number Publication Date
WO2020078300A1 true WO2020078300A1 (zh) 2020-04-23

Family

ID=65546682

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/110926 WO2020078300A1 (zh) 2018-10-16 2019-10-14 一种终端投屏的控制方法和终端

Country Status (3)

Country Link
US (1) US20210398527A1 (zh)
CN (1) CN109448709A (zh)
WO (1) WO2020078300A1 (zh)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109448709A (zh) * 2018-10-16 2019-03-08 华为技术有限公司 一种终端投屏的控制方法和终端
CN110060678B (zh) * 2019-04-16 2021-09-14 深圳欧博思智能科技有限公司 一种基于智能设备的虚拟角色控制方法及智能设备
CN110310638A (zh) * 2019-06-26 2019-10-08 芋头科技（杭州）有限公司 投屏方法、装置、电子设备和计算机可读存储介质
CN112351315B (zh) * 2019-08-07 2022-08-19 厦门强力巨彩光电科技有限公司 无线投屏方法以及led显示器
CN113129202B (zh) * 2020-01-10 2023-05-09 华为技术有限公司 数据传输方法、装置及数据处理系统、存储介质
CN111399789B (zh) * 2020-02-20 2021-11-19 华为技术有限公司 界面布局方法、装置及系统
CN111341315B (zh) * 2020-03-06 2023-08-04 腾讯科技（深圳）有限公司 语音控制方法、装置、计算机设备和存储介质
CN111524516A (zh) * 2020-04-30 2020-08-11 青岛海信网络科技股份有限公司 一种基于语音交互的控制方法、服务器及显示设备
CN114513527B (zh) * 2020-10-28 2023-06-06 华为技术有限公司 信息处理方法、终端设备及分布式网络
CN112331202B (zh) * 2020-11-04 2024-03-01 北京奇艺世纪科技有限公司 一种语音投屏方法及装置、电子设备和计算机可读存储介质
CN114090166A (zh) * 2021-11-29 2022-02-25 云知声智能科技股份有限公司 一种交互的方法和装置

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030076240A1 (en) * 2001-10-23 2003-04-24 Yu Seok Bae Remote control system for home appliances and method thereof
CN106653011A (zh) * 2016-09-12 2017-05-10 努比亚技术有限公司 一种语音控制方法、装置及终端
CN106847284A (zh) * 2017-03-09 2017-06-13 深圳市八圈科技有限公司 电子设备、计算机可读存储介质及语音交互方法
CN106993211A (zh) * 2017-03-24 2017-07-28 百度在线网络技术（北京）有限公司 基于人工智能的网络电视控制方法及装置
CN108538291A (zh) * 2018-04-11 2018-09-14 百度在线网络技术（北京）有限公司 语音控制方法、终端设备、云端服务器及系统
CN108597511A (zh) * 2018-04-28 2018-09-28 深圳市敢为特种设备物联网技术有限公司 基于物联网的信息展示方法、控制终端及可读存储介质
CN109448709A (zh) * 2018-10-16 2019-03-08 华为技术有限公司 一种终端投屏的控制方法和终端

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4230487B2 (ja) * 1997-10-07 2009-02-25 雅信 鯨田 Ｗｅｂページ連動型の複数連携型表示システム
US9542956B1 (en) * 2012-01-09 2017-01-10 Interactive Voice, Inc. Systems and methods for responding to human spoken audio
KR101944414B1 (ko) * 2012-06-04 2019-01-31 삼성전자주식회사 음성 인식 서비스를 제공하기 위한 방법 및 그 전자 장치
KR101330671B1 (ko) * 2012-09-28 2013-11-15 삼성전자주식회사 전자장치, 서버 및 그 제어방법
WO2014144949A2 (en) * 2013-03-15 2014-09-18 Apple Inc. Training an at least partial voice command system
US9431008B2 (en) * 2013-05-29 2016-08-30 Nuance Communications, Inc. Multiple parallel dialogs in smart phone applications
JP5955299B2 (ja) * 2013-11-08 2016-07-20 株式会社ソニー・インタラクティブエンタテインメント 表示制御装置、表示制御方法、プログラム及び情報記憶媒体
KR102261552B1 (ko) * 2014-06-30 2021-06-07 삼성전자주식회사 음성 명령어 제공 방법 및 이를 지원하는 전자 장치
US9767794B2 (en) * 2014-08-11 2017-09-19 Nuance Communications, Inc. Dialog flow management in hierarchical task dialogs
US9996310B1 (en) * 2016-09-15 2018-06-12 Amazon Technologies, Inc. Content prioritization for a display array
CN107978316A (zh) * 2017-11-15 2018-05-01 西安蜂语信息科技有限公司 控制终端的方法及装置
CN108012169B (zh) * 2017-11-30 2019-02-01 百度在线网络技术（北京）有限公司 一种语音交互投屏方法、装置和服务器
CN108520743B (zh) * 2018-02-02 2021-01-22 百度在线网络技术（北京）有限公司 智能设备的语音控制方法、智能设备及计算机可读介质
CN109117233A (zh) * 2018-08-22 2019-01-01 百度在线网络技术（北京）有限公司 用于处理信息的方法和装置

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030076240A1 (en) * 2001-10-23 2003-04-24 Yu Seok Bae Remote control system for home appliances and method thereof
CN106653011A (zh) * 2016-09-12 2017-05-10 努比亚技术有限公司 一种语音控制方法、装置及终端
CN106847284A (zh) * 2017-03-09 2017-06-13 深圳市八圈科技有限公司 电子设备、计算机可读存储介质及语音交互方法
CN106993211A (zh) * 2017-03-24 2017-07-28 百度在线网络技术（北京）有限公司 基于人工智能的网络电视控制方法及装置
CN108538291A (zh) * 2018-04-11 2018-09-14 百度在线网络技术（北京）有限公司 语音控制方法、终端设备、云端服务器及系统
CN108597511A (zh) * 2018-04-28 2018-09-28 深圳市敢为特种设备物联网技术有限公司 基于物联网的信息展示方法、控制终端及可读存储介质
CN109448709A (zh) * 2018-10-16 2019-03-08 华为技术有限公司 一种终端投屏的控制方法和终端

Also Published As

Publication number Publication date
CN109448709A (zh) 2019-03-08
US20210398527A1 (en) 2021-12-23

Similar Documents

Publication Publication Date Title
WO2020078300A1 (zh) 一种终端投屏的控制方法和终端
JP6952184B2 (ja) ビューに基づく音声インタラクション方法、装置、サーバ、端末及び媒体
JP6713034B2 (ja) スマートテレビの音声インタラクティブフィードバック方法、システム及びコンピュータプログラム
CN109658932B (zh) 一种设备控制方法、装置、设备及介质
US10311877B2 (en) Performing tasks and returning audio and visual answers based on voice command
CN109240107B (zh) 一种电器设备的控制方法、装置、电器设备和介质
JP6681450B2 (ja) 情報処理方法および装置
JP2019046468A (ja) インターフェイススマートインタラクティブ制御方法、装置、システム及びプログラム
WO2018133307A1 (zh) 一种实现语音控制的方法和终端
JP7551773B2 (ja) インタラクション記録生成方法、装置、デバイス及び媒体
CN110992955A (zh) 一种智能设备的语音操作方法、装置、设备及存储介质
JP2023515392A (ja) 情報処理方法、システム、装置、電子機器及び記憶媒体
US20190172461A1 (en) Electronic apparatus and method for controlling same
CN110379406A (zh) 语音评论转换方法、系统、介质和电子设备
CN111539217B (zh) 一种用于自然语言内容标题消歧的方法、设备和系统
CN111580766B (zh) 一种信息显示方法、装置和信息显示系统
CN103260065A (zh) 一种基于Android系统的机顶盒语音控制方法
JP6944920B2 (ja) スマートインタラクティブの処理方法、装置、設備及びコンピュータ記憶媒体
CA3191097A1 (en) Providing transfer and configuration of web conferencing between consumer devices
CN112615906A (zh) 一种庭审控制方法及控制系统、设备及介质
JP2019091448A (ja) 設備の発現方法、装置、設備及びプログラム
WO2019015089A1 (zh) 一种全局菜单的控制方法、装置、设备和存储介质
US11556694B1 (en) Predictive aspect formatting
US20210149965A1 (en) Digital assistant output attribute modification
CN110225364A (zh) 一种视频处理方法、装置、终端、服务器及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19873500

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19873500

Country of ref document: EP

Kind code of ref document: A1