CN112115887B - Monitoring method, vehicle-mounted terminal and computer storage medium - Google Patents
Monitoring method, vehicle-mounted terminal and computer storage medium Download PDFInfo
- Publication number
- CN112115887B CN112115887B CN202011003215.0A CN202011003215A CN112115887B CN 112115887 B CN112115887 B CN 112115887B CN 202011003215 A CN202011003215 A CN 202011003215A CN 112115887 B CN112115887 B CN 112115887B
- Authority
- CN
- China
- Prior art keywords
- monitoring
- monitoring task
- target
- vehicle
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 187
- 238000000034 method Methods 0.000 title claims abstract description 45
- 230000000007 visual effect Effects 0.000 claims description 44
- 238000004590 computer program Methods 0.000 claims description 7
- 230000003993 interaction Effects 0.000 abstract description 8
- 230000007613 environmental effect Effects 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 6
- 230000002093 peripheral effect Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000012790 confirmation Methods 0.000 description 3
- 238000010295 mobile communication Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 241001465754 Metazoa Species 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000006855 networking Effects 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 241001061106 Sargocentron rubrum Species 0.000 description 1
- 230000002730 additional effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 229920000642 polymer Polymers 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C5/00—Registering or indicating the working of vehicles
- G07C5/08—Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
- G07C5/0841—Registering performance data
- G07C5/085—Registering performance data using electronic data carriers
- G07C5/0866—Registering performance data using electronic data carriers the electronic data carrier being a digital video recorder in combination with video camera
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Alarm Systems (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention discloses a monitoring method, which is applied to a vehicle-mounted terminal and comprises the following steps: receiving voice information; identifying a monitoring task contained in the voice information, wherein the monitoring task comprises a target feature, a motion feature and a result feedback mode; and if the monitoring task is executed, starting the data acquisition equipment to execute the monitoring task. The invention can realize the intelligent monitoring of the vehicle on the movable target or the environmental characteristic through voice interaction, and improves the user experience.
Description
Technical Field
The present invention relates to the field of monitoring, and in particular, to a monitoring method, a vehicle-mounted terminal, and a computer storage medium.
Background
In the using process of the vehicle, the driving environment is generally monitored through a driving recorder. The automobile data recorder records video images and sound of the whole automobile running process, and can provide evidence for traffic accidents. However, this simple image recording method lacks interaction with the user, and the operation experience is general. Meanwhile, the recording mode cannot monitor targets interested by users in a targeted manner, the recorded content is redundant and lacks of emphasis, so that the users are very troublesome in extracting corresponding recorded fragments according to needs, and the users need to watch videos for a long time afterwards.
Disclosure of Invention
The invention aims to provide a monitoring method, a vehicle-mounted terminal and a computer storage medium, which can realize intelligent monitoring through voice interaction and improve user experience.
In order to solve the above technical problems, the present application provides a monitoring method, applied to a vehicle-mounted terminal, including:
receiving voice information;
identifying a monitoring task contained in the voice information, wherein the monitoring task comprises a target feature, a motion feature and a result feedback mode;
and if the monitoring task is executed, starting the data acquisition equipment to execute the monitoring task.
The identifying the monitoring task contained in the voice information comprises the following steps:
extracting a monitoring task contained in the voice information, wherein the monitoring task comprises a target feature, a motion feature and a result feedback mode;
judging whether the extracted monitoring task is complete;
if the monitoring task is incomplete, prompting to adjust the monitoring task;
if so, judging whether the monitoring task is capable of executing.
Wherein the determining whether the monitoring task is executed includes:
judging whether the device has the capability of collecting the target characteristics and the motion characteristics, the capability of carrying out visual analysis on the target characteristics and the capability of executing the result feedback mode;
if yes, the monitoring task can be executed.
Wherein the determining whether the target feature is capable of visual analysis comprises:
when the target features do not contain the preset targets, judging whether the capability of identifying the image information and other feature information in the target features is provided according to the existing visual analysis model;
if not, acquiring a visual analysis model for analyzing the corresponding characteristics from the server;
and if the acquisition is successful, determining that the target feature has the capability of performing visual analysis on the target feature.
Wherein, judge whether possess the ability to carry out visual analysis to the target feature, still include:
if the acquisition fails, determining that the capability of performing visual analysis on the target features is not available;
prompting to adjust the target feature and displaying the optional existing target feature on the screen.
Wherein the determining whether the target feature is capable of visual analysis comprises:
when the target features comprise preset targets, searching image information for identifying the preset targets;
searching image information corresponding to the preset target in preset image information stored in the vehicle-mounted terminal and/or the server;
if the image information corresponding to the preset target is not found, prompting supplementary image information;
if the image information corresponding to the preset target is found, the step of judging whether the image information and other characteristic information in the target characteristics are capable of being identified according to the existing visual analysis model is entered.
Wherein determining whether the capability of collecting the target feature and the motion feature is provided includes:
judging whether the motion characteristics are in the acquisition range of the acquisition equipment or not;
if not, prompting to adjust the motion characteristics, and displaying optional existing motion characteristics on a screen.
The data acquisition device is a camera, and the executing the monitoring task includes:
starting the camera to collect images and/or videos;
performing visual analysis on the acquired images and/or videos, and judging whether the acquired images and/or videos are matched with target features and/or motion features of the monitoring task;
and if so, feeding back a monitoring result according to a result feedback mode of the monitoring task.
The result feedback mode comprises pushing at least one of information, video and/or photographing tracking, flashing light reminding, voice reminding, loudspeaker reminding, recording and vehicle ignition to the mobile terminal and/or the vehicle-mounted terminal.
Before the monitoring task is executed, the method further comprises the following steps:
generating key information of the monitoring task;
outputting voice and/or displaying key information of the monitoring task on a screen;
and after receiving the instruction for confirming the execution of the monitoring task, executing the monitoring task.
The application also provides a vehicle-mounted terminal, which comprises:
at least one processor;
at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, which when executed by the at least one processor, cause the apparatus to perform the monitoring method as described above.
The present application also provides a computer storage medium having computer program instructions stored thereon; the computer program instructions, when executed by a processor, implement the monitoring method as described above.
The monitoring method, the vehicle-mounted terminal and the computer storage medium comprise the following steps: receiving voice information; identifying a monitoring task contained in the voice information, wherein the monitoring task comprises a target feature, a motion feature and a result feedback mode; and if the monitoring task executing capability is provided, starting the data acquisition equipment to execute the monitoring task. The invention can realize the intelligent monitoring of the vehicle on the movable target or the environmental characteristic through voice interaction, and improves the user experience.
The foregoing description is only an overview of the technical solutions of the present application, and may be implemented according to the content of the specification, so that the foregoing and other objects, features and advantages of the present application can be more clearly understood, and the following detailed description of the preferred embodiments will be given with reference to the accompanying drawings.
Drawings
FIG. 1 is an application environment diagram of a monitoring method according to an embodiment of the present invention;
FIG. 2 is a flow chart of a monitoring method according to an embodiment of the invention;
FIG. 3 is a specific flow chart of a monitoring method according to an embodiment of the invention;
fig. 4 is one of schematic structural diagrams of an in-vehicle terminal shown according to an embodiment of the present invention;
fig. 5 is a second schematic structural view of the in-vehicle terminal shown in fig. 4.
Detailed Description
Further advantages and effects of the present application will be readily apparent to those skilled in the art from the present disclosure, by describing the embodiments of the present application with specific examples.
In the following description, reference is made to the accompanying drawings, which describe several embodiments of the present application. It is to be understood that other embodiments may be utilized and that mechanical, structural, electrical, and operational changes may be made without departing from the spirit and scope of the present application. The following detailed description is not to be taken in a limiting sense, and the scope of embodiments of the present application is defined only by the claims of the issued patent. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.
Although the terms first, second, etc. may be used herein to describe various elements in some examples, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element.
Furthermore, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It will be further understood that the terms "comprises," "comprising," "includes," and/or "including" specify the presence of stated features, steps, operations, elements, components, items, categories, and/or groups, but do not preclude the presence, presence or addition of one or more other features, steps, operations, elements, components, items, categories, and/or groups. The terms "or" and/or "as used herein are to be construed as inclusive, or meaning any one or any combination. Thus, "A, B or C" or "A, B and/or C" means "any of the following: a, A is as follows; b, a step of preparing a composite material; c, performing operation; a and B; a and C; b and C; A. b and C). An exception to this definition will occur only when a combination of elements, functions, steps or operations are in some way inherently mutually exclusive.
Fig. 1 is a schematic view of an application environment of a monitoring method according to an embodiment of the present invention, including a vehicle 11, a vehicle-mounted terminal 12, and a camera 13. It should be understood that the number of vehicles 11, servers 12 and cameras 13 in fig. 1 and their mounting locations are merely illustrative. There may be any number of vehicles 11, servers 12 and cameras 13 associated as desired for implementation. The vehicle-mounted terminal is a device mounted on a console of an automobile and used for information interaction between a person and the automobile, and a medium for providing a communication link between the vehicle-mounted terminal 12 and the camera 13 through a network, wherein the network can be a mobile communication network, for example, a wireless fidelity network, a long term evolution network, a 4G network or a 5G network. The in-vehicle terminal 12 receives voice information sent by a microphone from a user, and the camera 13 may be provided with a microphone, may be connected to a microphone in the vehicle or a microphone of a mobile phone, or may be a combination of various modes. The vehicle-mounted terminal 12 analyzes and identifies the monitoring task through voice interaction and confirms the content of the monitoring task through a user, and then feeds back the monitoring result according to a result feedback mode after the monitoring target is found according to the corresponding monitoring task, so that intelligent monitoring is realized, and timeliness and effectiveness of feedback to the monitoring target can be improved.
Fig. 2 is a flow chart of a monitoring method according to an embodiment of the present invention. As shown in fig. 2, a monitoring method provided in an embodiment of the present invention includes:
step 201: receiving voice information;
step 202: identifying a monitoring task contained in the voice information, wherein the monitoring task comprises a target feature, a motion feature and a result feedback mode;
step 203: and if the monitoring task executing capability is provided, starting the data acquisition equipment to execute the monitoring task.
In this embodiment, after the user issues the voice information, the voice information may be collected by the vehicle-mounted microphone, the camera with the microphone or the mobile phone microphone, and the like, and sent to the vehicle-mounted terminal. The vehicle-mounted terminal analyzes target characteristics, motion characteristics and result feedback modes of the monitoring task contained in the voice information through voice recognition, semantic recognition, keyword matching and other methods. And evaluating whether the monitoring task has the capability of executing the monitoring task according to the specific content of the monitoring task. If the monitoring task is capable of executing the monitoring task, key information of the task, such as target characteristics, motion characteristics and result feedback modes, is repeated to the user, and when the user feels that the vehicle-mounted terminal correctly understands the monitoring task, language information is continuously sent to confirm the execution instruction. The vehicle-mounted terminal receiving the task can acquire the surrounding environment information of the vehicle and conduct intelligent analysis by utilizing the camera, the sensor and the computing capability of the computer of the vehicle, and the analysis result is presented to the user through the Internet of vehicles. The invention combines the data acquisition capability of the vehicle and the operation capability of the vehicle-mounted terminal to complete video analysis and execute monitoring tasks, thereby enabling a user to acquire the monitoring content acquired by the vehicle in time in any networking scene.
In this embodiment, step 202, when identifying a monitoring task included in voice information, the vehicle-mounted terminal first analyzes a target feature, a motion feature and a result feedback mode of the monitoring task through methods such as voice identification, semantic identification, keyword matching, and the like. For example, when a monitoring task is delivered in voice, a prefix "monitoring task" may be added to identify the start of the monitoring task, a prefix "target" may be added to identify the content of the target feature, a prefix "location" may be added to identify the content of the motion feature, and a prefix "feedback" may be added to identify the content of the result feedback mode. Specific content is entered after a particular prefix, e.g., the user uses "location: door right front occurrence ", or" position: the current orientation position of the camera appears ", or" position: the expression "the position of the left driving door and window of the vehicle appears" expresses the specific content of the movement characteristics in the monitoring task. After the monitoring task contained in the voice information is extracted, judging whether the extracted monitoring task is complete, namely whether the monitoring task comprises target characteristics, motion characteristics and a result feedback mode. If the monitoring task is incomplete, prompting the user to adjust the monitoring task to supplement or replace the monitoring task, recommending alternative monitoring tasks according to the collected part of monitoring task content, and directly selecting the monitoring task at the vehicle-mounted terminal or the mobile phone terminal by the user. If the monitoring task is complete, judging whether the monitoring task has the capability of executing the monitoring task or not.
In this embodiment, when judging whether the capability of executing the monitoring task is provided, it may be judged whether the capability of collecting the target feature and the motion feature, the capability of performing the visual analysis on the target feature, and the capability of executing the result feedback mode are provided at the same time. If the above conditions are met, it is determined that the monitoring task is capable of being executed, step 203 may be executed continuously, and the data acquisition device is turned on to execute the monitoring task.
Specifically, in judging whether or not the capability of performing visual analysis on the target feature is provided, the target feature includes a target, a target appearance, a target age, a target kind, and the like. If the target feature contains a preset target, searching image information for identifying the preset target. For example, when the preset target is "wife", the system searches for image information named "wife", or searches for image information of registered user "wife", etc. Image information corresponding to a preset target can be searched in an image information base stored in the vehicle-mounted terminal and/or the server; if the image information corresponding to the preset target is not found, a notification prompting the supplementary image information can be sent to the user; if the image information corresponding to the preset target is found, the method enters a step of judging whether the image information and other characteristic information in the target characteristics are capable of being identified according to the existing visual analysis model. If the existing visual analysis model does not have the capability of identifying the image information and other characteristic information in the target characteristics, acquiring a visual analysis model for analyzing the corresponding characteristics from a server; if the acquisition is successful, the capability of performing visual analysis on the target characteristics is determined. And downloading and searching the visual model to the cloud for the unrecognizable target features. If the task visual model is lacking in the vehicle, the task visual model can be inquired and downloaded to the cloud, and a user confirmation or payment downloading link can be added according to the product requirement in the process. Whether the existing vision model preset in the vehicle is adequate for vision analysis tasks or not is checked, for example, the target feature is to monitor a customer wearing specifically, the vehicle-mounted terminal vision model is required to have clothing analysis and sex analysis model capability, the specific person is accurately monitored, the face recognition capability is required to be accurately monitored, the age analysis capability based on the face requirement of the middle-aged person and the child is monitored and distinguished, and the animal matching model capability is required to be monitored. If the visual model required by the monitoring target can not be downloaded, the user can be prompted by voice that the current monitoring task can not be completed, the monitoring target is recommended to be replaced, other relevant identifiable target characteristics are provided for display on the screen of the vehicle-mounted terminal, and the user can conveniently select and adjust.
When judging whether the capability of collecting the target features and the motion features is provided, judging whether the motion features are in the collecting range of the collecting equipment, namely, confirming whether the position of the target and the spatial range of the motion features are in the range of the camera supporting identification, such as 'the front of a vehicle door', or 'the current direction position of the camera', or 'the left driving door and window position of the vehicle', wherein the motion features can be from left to right, or from right to left, the limbs touch a certain position of the vehicle, approach the vehicle firstly and then keep away, and the like. If the motion characteristics are beyond the acquirable range of the camera, prompting to adjust the motion characteristics, and displaying optional existing motion characteristics on a screen.
When judging whether the capability of executing the result feedback mode is provided, searching whether the result feedback mode of the current monitoring task exists in the existing result feedback mode, if so, judging that the capability of executing the result feedback mode exists, and if not, providing a user to adjust the result feedback mode or adding a new result feedback mode into the system. The result feedback mode comprises at least one of pushing information, video and/or photographing tracking, flashing light reminding, voice reminding, loudspeaker reminding, recording and vehicle ignition to the mobile terminal and/or the vehicle-mounted terminal. The result feedback mode can be through the APP notice of the Internet of vehicles, or telephone notice or short message notice, or in-vehicle reminding. Or intelligent reminding: the user reminds in the car preferentially, and the user is not in the car and the APP is online and then reminds through the APP, and the user APP account is not online and looks ahead preferentially and reminds through WeChat, short message or telephone.
In an embodiment, when the acquisition device is a camera, the camera can be started to acquire images and/or videos by executing a monitoring task; performing visual analysis on the acquired images and/or videos, and judging whether the acquired images and/or videos are matched with target features and/or motion features of a monitoring task; if so, feeding back a monitoring result according to a result feedback mode of the monitoring task.
In this embodiment, step 203, before executing the monitoring task, further includes: generating key information of a monitoring task; the key information of the monitoring task is repeated through voice and/or displayed on a screen; and after receiving the instruction for confirming the execution of the monitoring task, executing the monitoring task. The vehicle-mounted terminal combines the target characteristics, appearance characteristics, movement characteristics, matched actions and additional actions of the task into sentences, reads the voice for a user to listen, and acquires a command for starting to execute or cancel the voice of the user.
The monitoring method comprises the following steps: receiving voice information; identifying a monitoring task contained in the voice information, wherein the monitoring task comprises a target feature, a motion feature and a result feedback mode; and if the monitoring task executing capability is provided, starting the data acquisition equipment to execute the monitoring task. The invention can realize intelligent monitoring through voice interaction and improve user experience.
Fig. 3 is a specific flowchart of a monitoring method according to an embodiment of the present invention. As shown in fig. 3, before receiving the voice information to perform the monitoring task, the user may select a desired visual analysis model, such as a visual recognition model for a person, an animal, a vehicle, etc., at the in-vehicle terminal interface or the car-train APP interface. Then downloaded and stored locally for convenient use at any time. The user can then issue the monitoring task through voice, and the system analyzes the voice information keywords to identify the specific content of the monitoring task. And then confirming whether a corresponding visual model exists or not according to the monitoring task, and setting according to a result feedback mode of the monitoring task. And sending the tidied monitoring task content to a user for confirmation, and after a user confirmation instruction is obtained, starting the data acquisition equipment to acquire, for example, starting a camera and shooting according to the movement characteristic adjustment angle or position of the target, and continuously acquiring the video stream. Meanwhile, the video stream is analyzed, if the target is matched with the content of the monitoring task, a corresponding result feedback mode is executed, for example, a message is pushed to the user, a photo, a video or a text description of the monitoring task can be pushed through the car-link APP, and the real-time performance of feedback can be ensured by directly sending a short message or making a call to the user. In the task execution process, a user can cancel the task being executed through a vehicle-mounted terminal screen operation interface or a vehicle networking APP.
In light of the foregoing, the present invention is particularly applicable to the following scenarios.
Scene one
The user takes the child to drive back to the rural old home for the year, the parents are busy in households, worry that the child leaves the sight range, the vehicle can be stopped in the courtyard, the front camera faces the home, and the voice is given to monitor tasks: ' love car, add monitoring task, target: child, appearance position: and (5) feeding back the vehicle right ahead: after discovery, the phone notifies me and then starts video tracking, voice prompt: "dad does not allow you to go outside playing to catch up home". The vehicle-mounted terminal recognizes the voice and judges whether the task can be accepted after carrying out semantic analysis, if the task can be accepted, the vehicle-mounted terminal replies' good, i successfully accepts the task, and before driving next time or remotely removing the task, i can monitor the front and capture the target: the child, immediately after the target moves too much, calls you and speaks "dad does not allow you to go outside to play to get home" with a horn. And then carrying out camera view movement tracking shooting on the target.
Scene two
The user's vehicle is often scratched or advertised outside, and the user gives the vehicle a task of voice when leaving: ' love car, add monitoring task, target: monitor the taxi and paste the advertiser, the appearance position: contact vehicle body, feedback: the photo is tracked and timely informed. The vehicle-mounted terminal recognizes the voice and performs semantic analysis and then judges whether the task can be accepted or not, if the task is accepted, the vehicle-mounted terminal replies' well, the task is accepted by me, and when the vehicle-mounted terminal approaches a vehicle and has physical contact, a target stranger can automatically shoot and send a photo to inform the user through the internet of vehicles APP. And then starting the camera to monitor.
Scene three
The user stops the car at a certain client such as a certain restaurant gate, and the user does not determine when the client can come out of the restaurant, so that a monitoring task is given to the car, the car is loved, the monitoring task is added, and the aim is that: customer, red coat, black skirt, long hair, position: restaurant gate, feedback: and finding out a suspected target, and prompting me by voice. The vehicle replaces the user to execute the monitoring task until the client is found to go out to prompt the user by voice, so that the time and energy of the user are greatly saved.
The embodiment of the invention fully utilizes the camera and visual analysis capability of the vehicle, helps a user monitor the interested target, records and informs the user in time, increases the intelligent analysis process compared with the traditional video monitoring, does not need to reprocess and arrange the video according to the specific target, can interact with the voice of the user, and utilizes the vehicle to construct a mobile monitoring analysis tool.
As shown in fig. 4, the present application further provides a vehicle-mounted terminal 10, applied to a vehicle, including: a processor 102 and a memory 106 storing a computer program, which when executed by the processor 102, implements a monitoring method comprising:
receiving voice information;
identifying a monitoring task contained in the voice information, wherein the monitoring task comprises a target feature, a motion feature and a result feedback mode;
and if the monitoring task executing capability is provided, starting the data acquisition equipment to execute the monitoring task.
In one embodiment, the step of the processor 102 performing the step of identifying the monitoring task contained in the voice information includes:
extracting a monitoring task contained in the voice information, wherein the monitoring task comprises a target feature, a motion feature and a result feedback mode;
judging whether the extracted monitoring task is complete;
if the monitoring task is incomplete, prompting to adjust the monitoring task;
if so, executing the step of judging whether the monitoring task is capable of being executed.
In one embodiment, the processor 102 performs the steps of determining whether it is capable of performing a monitoring task, including:
judging whether the device has the capability of collecting target features and motion features, the capability of carrying out visual analysis on the target features and the capability of executing a result feedback mode;
if yes, the monitoring task executing capability is provided.
In one embodiment, the target feature comprises at least one of a target, a target appearance, a target age, and a target category, and the processor 102 performs the step of determining whether the target feature is capable of visual analysis, including:
when the target characteristics comprise preset targets, searching image information for identifying the preset targets;
judging whether the device has the capability of identifying the image information and other characteristic information in the target characteristics according to the existing visual analysis model;
if not, acquiring a visual analysis model for analyzing the corresponding characteristics from the server;
if the acquisition is successful, the capability of performing visual analysis on the target characteristics is determined.
In one embodiment, the processor 102 performs the step of determining whether the capability to perform visual analysis of the target feature is available, further comprising:
if the acquisition fails, determining that the capability of performing visual analysis on the target characteristics is not available;
prompting to adjust the target feature and displaying the optional existing target feature on the screen.
In one embodiment, the processor 102 performs the step of looking up image information for identifying a preset target, including:
searching image information corresponding to a preset target in preset image information stored in the vehicle-mounted terminal and/or the server;
if the image information corresponding to the preset target is not found, prompting the supplementary image information;
if the image information corresponding to the preset target is found, the method enters a step of judging whether the image information and other characteristic information in the target characteristics are capable of being identified according to the existing visual analysis model.
In one embodiment, the processor 102 performs the steps of determining whether the capability to collect the target feature and the motion feature is available, including:
judging whether the motion characteristics are in the acquisition range of the acquisition equipment;
if not, prompting to adjust the motion characteristics, and displaying optional existing motion characteristics on a screen.
In one embodiment, the acquisition device is a camera and the processor 102 performs the steps of performing a monitoring task, including:
starting a camera, and collecting images and/or videos;
performing visual analysis on the acquired images and/or videos, and judging whether the acquired images and/or videos are matched with target features and/or motion features of a monitoring task;
if so, feeding back a monitoring result according to a result feedback mode of the monitoring task.
In an embodiment, the result feedback means comprises pushing at least one of a message, video and/or photo tracking, flashing light alert, voice alert, speaker alert, recording and vehicle ignition to the mobile terminal and/or vehicle terminal.
In one embodiment, before the step of performing the monitoring task, the processor 102 further includes:
generating key information of a monitoring task;
the key information of the monitoring task is repeated through voice and/or displayed on a screen;
and after receiving the instruction for confirming the execution of the monitoring task, executing the monitoring task.
The specific process of executing the above steps by the processor 102 in this embodiment is described in detail in the related description of the embodiments of the monitoring method, which is not repeated here.
Referring to fig. 5, in actual implementation, the vehicle terminal 10 includes a memory 102, a memory controller 104, one or more (only one is shown in the figure) processors 106, a peripheral interface 108, a radio frequency module 150, a positioning module 112, a camera module 114, an audio module 116, a screen 118, and a key module 160. These components communicate with each other via one or more communication buses/signal lines 122.
It is to be understood that the configuration shown in fig. 5 is merely illustrative, and the in-vehicle terminal 10 may further include more or less components than those shown in fig. 5, or have a different configuration from that shown in fig. 5. The components shown in fig. 5 may be implemented in hardware, software, or a combination thereof.
The memory 102 may be used to store software programs and modules, such as program instructions/modules corresponding to the monitoring method in the embodiments of the present application, and the processor 106 executes the software programs and modules stored in the memory controller 104 to perform various functional applications and data processing, i.e., implement the monitoring method described above.
Memory 102 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 102 may further include memory remotely located relative to the processor 106, which may be connected to the in-vehicle terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof. Access to the memory 102 by the processor 106 and possibly other components may be under the control of the memory controller 104.
The peripheral interface 108 couples various input/output devices to the CPU and memory 102. The processor 106 executes various software, instructions, and the like within the memory 102 to perform various functions of the in-vehicle terminal 10 and to perform data processing.
In some embodiments, the peripheral interface 108, the processor 106, and the memory controller 104 may be implemented in a single chip. In other examples, they may be implemented by separate chips.
The radio frequency module 150 is configured to receive and transmit electromagnetic waves, and to implement mutual conversion between the electromagnetic waves and the electrical signals, so as to communicate with a communication network or other devices. The radio frequency module 150 may include various existing circuit elements for performing these functions, such as an antenna, a radio frequency transceiver, a digital signal processor, an encryption/decryption chip, a Subscriber Identity Module (SIM) card, memory, and the like. The rf module 150 may communicate with various networks such as the internet, intranets, wireless networks, or other devices via wireless networks. The positioning module 112 is configured to obtain a current position of the in-vehicle terminal 10. Examples of the positioning module 112 include, but are not limited to, global satellite positioning system (GPS), wireless local area network or mobile communication network based positioning technology.
The camera module 114 is used to take pictures or videos. The photographed photo or video may be stored into the memory 102 and may be transmitted through the radio frequency module 150.
The audio module 116 provides an audio interface to the user, which may include one or more microphones, one or more speakers, and audio circuitry. The audio circuit receives sound data from the peripheral interface 108, converts the sound data to electrical information, and transmits the electrical information to the speaker. The speaker converts electrical energy into sound waves that can be heard by the human ear. The audio circuit also receives electrical information from the microphone, converts the electrical signal into sound data, and transmits the sound data to the peripheral interface 108 for further processing. The audio data may be retrieved from the memory 102 or through the radio frequency module 150. In addition, audio data may also be stored in the memory 102 or transmitted through the radio frequency module 150. In some examples, the audio module 116 may also include a headphone jack for providing an audio interface to headphones or other devices.
The screen 118 provides an output interface between the in-vehicle terminal 10 and the user. Specifically, screen 118 displays video output to the user, the content of which may include text, graphics, video, and any combination thereof. Some output results correspond to some user interface objects. It is understood that the screen 118 may also include a touch screen. The touch screen provides one output and input interface between the in-vehicle terminal 10 and the user at the same time. In addition to displaying video output to users, the touch screen also receives user inputs, such as user clicks, swipes, and the like, in order for the user interface object to respond to these user inputs. The technique of detecting user input may be based on resistive, capacitive, or any other possible touch detection technique. Specific examples of touch screen display units include, but are not limited to, liquid crystal displays or light emitting polymer displays.
The key module 160 also provides an interface for a user to input to the in-vehicle terminal 10, and the user can cause the in-vehicle terminal 10 to perform different functions by pressing different keys.
The vehicle-mounted terminal of the embodiment receives voice information; identifying a monitoring task contained in the voice information, wherein the monitoring task comprises a target feature, a motion feature and a result feedback mode; and if the monitoring task executing capability is provided, starting the data acquisition equipment to execute the monitoring task. The invention can realize intelligent monitoring through voice interaction and improve user experience.
The present application also provides a computer storage medium having computer program instructions stored thereon; the computer program instructions, when executed by a processor, implement the monitoring method as described in the above embodiments.
In actual implementation, the computer storage medium is applied to the vehicle-mounted terminal shown in fig. 4 or fig. 5, so as to realize intelligent monitoring. The foregoing embodiments are merely illustrative of the principles of the present application and their effectiveness, and are not intended to limit the application. Modifications and variations may be made to the above-described embodiments by those of ordinary skill in the art without departing from the spirit and scope of the present application. Accordingly, it is intended that all equivalent modifications and variations which may be accomplished by persons skilled in the art without departing from the spirit and technical spirit of the disclosure be covered by the claims of this application.
Claims (9)
1. A monitoring method is applied to a vehicle-mounted terminal and is characterized in that,
comprising the following steps:
receiving voice information;
extracting a monitoring task contained in the voice information, wherein the monitoring task comprises a target feature, a motion feature and a result feedback mode;
judging whether the extracted monitoring task is complete;
if the monitoring task is incomplete, prompting to adjust the monitoring task;
if the monitoring task is complete, judging whether the monitoring task is capable of executing or not;
if the monitoring task is executed, starting the data acquisition equipment to execute the monitoring task;
wherein the determining whether the monitoring task is executed includes:
judging whether the device has the capability of collecting the target characteristics and the motion characteristics, the capability of carrying out visual analysis on the target characteristics and the capability of executing the result feedback mode;
if yes, the monitoring task is executed;
wherein determining whether the capability of performing visual analysis on the target feature is provided comprises:
when the target features comprise preset targets, searching image information for identifying the preset targets;
searching image information corresponding to the preset target in preset image information stored in the vehicle-mounted terminal and/or the server;
if the image information corresponding to the preset target is not found, prompting supplementary image information;
if the image information corresponding to the preset target is found, a step of judging whether the image information and other characteristic information in the target characteristics are capable of being identified according to the existing visual analysis model is performed.
2. The method of monitoring as set forth in claim 1, wherein,
the determining whether the target feature is capable of visual analysis includes:
when the target features do not contain the preset targets, judging whether the capability of identifying the image information and other feature information in the target features is provided according to the existing visual analysis model;
if not, acquiring a visual analysis model for analyzing the corresponding characteristics from the server;
and if the acquisition is successful, determining that the target feature has the capability of performing visual analysis on the target feature.
3. The method of monitoring as set forth in claim 2, wherein,
the determining whether the target feature is capable of performing visual analysis, further comprises:
if the acquisition fails, determining that the capability of performing visual analysis on the target features is not available;
prompting to adjust the target feature and displaying the optional existing target feature on the screen.
4. The method of monitoring as set forth in claim 1, wherein,
the determining whether the target feature and the motion feature are acquired includes:
judging whether the motion characteristics are in the acquisition range of the acquisition equipment or not;
if not, prompting to adjust the motion characteristics, and displaying optional existing motion characteristics on a screen.
5. The method of monitoring as set forth in claim 1, wherein,
the data acquisition device is a camera, and the executing the monitoring task includes:
starting the camera to collect images and/or videos;
performing visual analysis on the acquired images and/or videos, and judging whether the acquired images and/or videos are matched with target features and/or motion features of the monitoring task;
and if so, feeding back a monitoring result according to a result feedback mode of the monitoring task.
6. The method of monitoring as set forth in claim 1, wherein,
the result feedback mode comprises pushing at least one of a message, video and/or photographing tracking, flashing light reminding, voice reminding, loudspeaker reminding, recording and vehicle ignition to the mobile terminal and/or the vehicle-mounted terminal.
7. The method of monitoring as set forth in claim 1, wherein,
before the monitoring task is executed, the method further comprises the following steps:
generating key information of the monitoring task;
outputting voice and/or displaying key information of the monitoring task on a screen;
and after receiving the instruction for confirming the execution of the monitoring task, executing the monitoring task.
8. A vehicle-mounted terminal is characterized in that,
comprising the following steps:
at least one processor;
at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, which when executed by the at least one processor, cause the in-vehicle terminal to perform the monitoring method of any one of claims 1 to 7.
9. A computer storage medium, characterized in that,
the computer storage medium has stored thereon computer program instructions; the computer program instructions, when executed by a processor, implement the monitoring method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011003215.0A CN112115887B (en) | 2020-09-22 | 2020-09-22 | Monitoring method, vehicle-mounted terminal and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011003215.0A CN112115887B (en) | 2020-09-22 | 2020-09-22 | Monitoring method, vehicle-mounted terminal and computer storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112115887A CN112115887A (en) | 2020-12-22 |
| CN112115887B true CN112115887B (en) | 2024-03-29 |
Family
ID=73800952
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011003215.0A Active CN112115887B (en) | 2020-09-22 | 2020-09-22 | Monitoring method, vehicle-mounted terminal and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112115887B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106992009A (en) * | 2017-05-03 | 2017-07-28 | 深圳车盒子科技有限公司 | Vehicle-mounted voice exchange method, system and computer-readable recording medium |
| WO2017166646A1 (en) * | 2016-03-29 | 2017-10-05 | 乐视控股（北京）有限公司 | In-vehicle voice control method and device, and apparatus |
| CN107977183A (en) * | 2017-11-16 | 2018-05-01 | 百度在线网络技术（北京）有限公司 | voice interactive method, device and equipment |
| CN110009900A (en) * | 2019-03-12 | 2019-07-12 | 浙江吉利汽车研究院有限公司 | A kind of vehicle monitoring method and system |
| CN110070889A (en) * | 2019-03-15 | 2019-07-30 | 深圳壹账通智能科技有限公司 | Vehicle monitoring method, device and storage medium, server |
| CN111013138A (en) * | 2019-12-11 | 2020-04-17 | 苏州思必驰信息科技有限公司 | Voice control method and device, electronic equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9918006B2 (en) * | 2016-05-20 | 2018-03-13 | International Business Machines Corporation | Device, system and method for cognitive image capture |
-
2020
- 2020-09-22 CN CN202011003215.0A patent/CN112115887B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017166646A1 (en) * | 2016-03-29 | 2017-10-05 | 乐视控股（北京）有限公司 | In-vehicle voice control method and device, and apparatus |
| CN106992009A (en) * | 2017-05-03 | 2017-07-28 | 深圳车盒子科技有限公司 | Vehicle-mounted voice exchange method, system and computer-readable recording medium |
| CN107977183A (en) * | 2017-11-16 | 2018-05-01 | 百度在线网络技术（北京）有限公司 | voice interactive method, device and equipment |
| CN110009900A (en) * | 2019-03-12 | 2019-07-12 | 浙江吉利汽车研究院有限公司 | A kind of vehicle monitoring method and system |
| CN110070889A (en) * | 2019-03-15 | 2019-07-30 | 深圳壹账通智能科技有限公司 | Vehicle monitoring method, device and storage medium, server |
| CN111013138A (en) * | 2019-12-11 | 2020-04-17 | 苏州思必驰信息科技有限公司 | Voice control method and device, electronic equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| Speech recognition interactive system for vehicle;Chee Yang Loh; Kai Lung Boey; Kai Sze Hong;2017 IEEE 13th International Colloquium on Signal Processing & its Applications (CSPA);全文 * |
| 简易分析判断语音识别智能车的设计;黄恒一;丁佳乐;黎顺;康郢芮;付三丽;;物联网技术(第05期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112115887A (en) | 2020-12-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101852284B1 (en) | Alarming method and device | |
| CN104867324B (en) | The method and relevant apparatus of passenger loading are identified in the service of calling a taxi | |
| CN115103106A (en) | Control method, electronic equipment, computer readable storage medium and chip | |
| CN106407984B (en) | Target object identification method and device | |
| US20180184038A1 (en) | Method and system for creating virtual message onto a moving object and searching the same | |
| CN108182626B (en) | Service pushing method, information acquisition terminal and computer readable storage medium | |
| CN105956518A (en) | Face identification method, device and system | |
| CN106128478B (en) | Voice broadcast method and device | |
| CN111432245B (en) | Multimedia information playing control method, device, equipment and storage medium | |
| CN111339938A (en) | Information interaction method, device, equipment and storage medium | |
| CN109032491A (en) | Data processing method, device and mobile terminal | |
| CN115312061A (en) | Voice question-answer method and device in driving scene and vehicle-mounted terminal | |
| CN105447476A (en) | Driving early warning method and device | |
| KR20180000603A (en) | Mobile terminal and method for controlling the same | |
| CN111950425A (en) | Object acquisition method, device, client, server, system and storage medium | |
| US10606886B2 (en) | Method and system for remote management of virtual message for a moving object | |
| CN110784523B (en) | Target object information pushing method and device | |
| CN114419572B (en) | Multi-radar target detection method and device, electronic equipment and storage medium | |
| CN112533070A (en) | Video sound and picture adjusting method, terminal and computer readable storage medium | |
| CN106559293B (en) | Method, device and system for communicating with network access equipment and network access equipment | |
| CN110956483A (en) | Advertisement information pushing method, system, server and vehicle | |
| CN112115887B (en) | Monitoring method, vehicle-mounted terminal and computer storage medium | |
| CN113987326A (en) | Resource recommendation method and device, computer equipment and medium | |
| CN105426904A (en) | Photo processing method, apparatus and device | |
| KR20150045465A (en) | Method for provisioning a person with information associated with an event |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |