CN116228052A

CN116228052A - Cargo state monitoring method, device, terminal equipment and storage medium

Info

Publication number: CN116228052A
Application number: CN202111451791.6A
Authority: CN
Inventors: 连自锋
Original assignee: SF Technology Co Ltd
Current assignee: SF Technology Co Ltd
Priority date: 2021-12-01
Filing date: 2021-12-01
Publication date: 2023-06-06

Abstract

The application provides a cargo state monitoring method, a device, terminal equipment and a storage medium, wherein the method comprises the following steps: extracting features of the acquired cargo transportation scene image to obtain a single-channel feature image; carrying out optical flow information calculation on the single-channel characteristic image to obtain an optical flow information image; and monitoring the motion state of the goods by combining the goods transportation scene image, the single-channel characteristic image and the optical flow information image. According to the cargo state monitoring method, feature extraction can be carried out on the images of cargoes in the transportation state, and the optical flow information images are combined on the basis of the extracted related feature images such as the cargo motion state, so that the problem of sensitivity of the optical flow information images is solved, and the monitoring precision of the cargo transportation state is improved.

Description

Cargo state monitoring method, device, terminal equipment and storage medium

Technical Field

The application relates to the field of logistics, in particular to a cargo state monitoring method, a device, terminal equipment and a storage medium.

Background

In the express and logistics industry, the loading and unloading events of the transfer are required to be monitored, so that the loading and unloading starting time and the loading and unloading ending time of each container can be accurately obtained, time consumption of each link of logistics is counted, and the operation efficiency is improved.

In the prior art, by calculating optical flow information for consecutive frames of pictures in a video sequence, movement information of each pixel of an image in the horizontal and vertical directions can be obtained.

Prior patents use optical flow information as one of the basis for determining events. However, the optical flow information is very sensitive to pixel changes, and small changes such as brightness changes, object shadows, shooting jitter and the like can seriously affect the calculation of an optical flow diagram, so that the accuracy of event monitoring is affected.

Therefore, on the basis of using the optical flow information, how to solve the problem that the monitoring accuracy is low because the optical flow information is sensitive is a problem to be solved in the field.

Disclosure of Invention

The application provides a cargo state monitoring method which can effectively identify the motion state of cargoes in a transportation scene.

In a first aspect, the present application provides a cargo state monitoring method, the method comprising:

extracting features of the acquired cargo transportation scene image to obtain a single-channel feature image;

performing optical flow information calculation on the single-channel characteristic image to obtain an optical flow information image;

and monitoring the motion state of the goods by combining the goods transportation scene image, the single-channel characteristic image and the optical flow information image.

In some embodiments of the present application, the feature extraction is performed on the acquired image of the cargo transportation scene to obtain a single-channel feature image, including:

and sending the cargo transportation scene image into a preset feature extraction network to obtain the single-channel feature image, wherein the feature extraction network is a single-channel convolution layer, and the single-channel convolution layer is a single-channel convolution layer obtained after the processing of the multi-channel convolution layer.

In some embodiments of the present application, the calculating optical flow information of the single-channel feature image to obtain an optical flow information image includes:

calculating horizontal motion information of each pixel in the single-channel characteristic image to obtain a horizontal optical flow information image;

and calculating the vertical motion information of each pixel in the single-channel characteristic image to obtain a vertical optical flow information image.

In some embodiments of the present application, the monitoring the motion state of the cargo in combination with the cargo transportation scene image, the single-channel feature image, and the optical flow information image includes:

carrying out gray scale processing on the cargo transportation scene image to obtain a gray scale image;

superposing the gray level image, the single-channel characteristic image, the horizontal optical flow information image and the vertical optical flow information image into a four-channel image;

and sending the four-channel image into a preset cargo state monitoring model to monitor the motion state of the cargo.

In some embodiments of the present application, before performing feature extraction on the acquired image of the cargo transportation scene to obtain a single-channel feature image, the method further includes:

acquiring a cargo transportation scene video;

and taking frames of the cargo transportation scene video one by one to obtain the cargo transportation scene image.

In some embodiments of the present application, before the monitoring of the motion state of the cargo in combination with the cargo transportation scene image, the single-channel feature image, and the optical flow information image, the method further comprises:

and carrying out cargo state classification training on a preset initial cargo state monitoring model to obtain the cargo state monitoring model.

In some embodiments of the present application, the training of cargo state classification is performed on a preset initial cargo state monitoring model to obtain the cargo state monitoring model, including:

creating a plurality of different cargo movement state tags;

acquiring a plurality of groups of marked goods moving images, wherein one group of marked goods moving images corresponds to one goods moving state label, and one group of marked goods moving images and one corresponding goods moving state label are one sample training group;

and sending the sample training group into the initial cargo state monitoring model for training to obtain the cargo state monitoring model.

In a second aspect, the present application also provides a cargo state monitoring device, the device comprising:

the feature extraction module is used for extracting features of the acquired cargo transportation scene image to obtain a single-channel feature image;

the optical flow calculation module is used for carrying out optical flow information calculation on the single-channel characteristic image to obtain an optical flow information image;

and the cargo monitoring module is used for combining the cargo transportation scene image, the single-channel characteristic image and the optical flow information image to monitor the motion state of the cargo.

In some embodiments of the present application, the feature extraction module is specifically configured to:

In some embodiments of the present application, the optical flow calculation module is specifically configured to:

In some embodiments of the present application, the cargo monitoring module is specifically configured to:

In a third aspect, the present application also provides a terminal device comprising a processor, a memory and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the steps in any of the cargo state monitoring methods.

In a fourth aspect, the present application also provides a computer readable storage medium having stored thereon a computer program for execution by a processor to perform the steps in the cargo state monitoring method of any one of the claims.

According to the cargo state monitoring method, feature extraction can be carried out on the images of cargoes in the transportation state, and the optical flow information images are combined on the basis of the extracted related feature images such as the cargo motion state, so that the problem of sensitivity of the optical flow information images is solved, and the monitoring precision of the cargo transportation state is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic illustration of a scenario of a cargo state monitoring system provided in an embodiment of the present application;

FIG. 2 is a flow chart of one embodiment of a cargo state monitoring method according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a functional module of a cargo state monitoring device according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of a terminal device in an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.

In the description of the present application, it should be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or an implicit indication of the number of technical features being indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present application, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.

In this application, the term "exemplary" is used to mean "serving as an example, instance, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to make and use the application. In the following description, details are set forth for purposes of explanation. It will be apparent to one of ordinary skill in the art that the present application may be practiced without these specific details. In other instances, well-known structures and processes have not been shown in detail to avoid obscuring the description of the present application with unnecessary detail. Thus, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

The application provides a cargo state monitoring method, a cargo state monitoring device, terminal equipment and a storage medium, and the detailed description is given below.

First, some concepts presented in this application are presented:

optical flow image information: an apparent motion (apparent motion) of such an image luminance pattern is optical flow. Optical flow expresses the change of an image and can be used by an observer to determine the movement of an object, since it contains information about the movement of the object. The definition of optical flow can be extended to an optical flow field, which refers to a two-dimensional (2D) instantaneous velocity field formed by all pixels in an image, wherein the two-dimensional velocity vector is the projection of a three-dimensional velocity vector of a visible point in the scene onto the imaging surface. The optical flow contains not only the motion information of the observed object but also rich information about the three-dimensional structure of the scene. The study of optical flow is an important part of the field of computer vision and related research. Because optical flow plays an important role in computer vision, it has very important applications in object segmentation, recognition, tracking, robotic navigation, shape information retrieval, and the like.

Referring to fig. 1, fig. 1 is a schematic diagram of a scenario of a cargo state monitoring method according to an embodiment of the present application, where the cargo state monitoring system may include a terminal device 100 and a storage device 200, and the storage device 200 may transmit data to the terminal device 100. As in the terminal device 100 of fig. 1, the image data of the goods in transit stored in the storage device 200 may be acquired to perform the goods state monitoring method in the present application.

In the embodiment of the present application, the terminal device 100 includes, but is not limited to, a desktop computer, a portable computer, a web server, a palm computer (Personal Digital Assistant, PDA), a tablet computer, a wireless terminal device, an embedded device, and the like.

In embodiments of the present application, communication between the terminal device 100 and the storage device 200 may be implemented by any communication means, including, but not limited to, mobile communication based on the third generation partnership project (3rd Generation Partnership Project,3GPP), long term evolution (Long Term Evolution, LTE), worldwide interoperability for microwave access (Worldwide Interoperability for Microwave Access, wiMAX), or computer network communication based on the TCP/IP protocol family (TCP/IP Protocol Suite, TCP/IP), user datagram protocol (User Datagram Protocol, UDP), etc.

It should be noted that, the schematic view of the cargo state monitoring system shown in fig. 1 is only an example, and the cargo state monitoring system and the scene described in the embodiments of the present application are for more clearly describing the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application, and those skilled in the art can know that, with the evolution of the cargo state monitoring system and the appearance of a new service scenario, the technical solutions provided in the embodiments of the present application are equally applicable to similar technical problems.

As shown in fig. 2, fig. 2 is a flowchart of one embodiment of a cargo state monitoring method according to an embodiment of the present application, where the cargo state monitoring method may include the following steps 201 to 203:

201. and extracting the characteristics of the acquired cargo transportation scene image to obtain a single-channel characteristic image.

Firstly, in the link that the goods are transported, in order to determine the current transportation state of the goods, the specific transportation state of the goods needs to be determined based on the movement condition of the goods. For example: a moving state or a stationary state. The method for acquiring the images of the cargo transportation scene can be to continuously photograph the cargo in the transportation process through a camera installed in the transportation scene, so that the acquired images of the cargo transportation scene are acquired. After the related transportation state images are acquired, the position change condition of the goods in the continuous transportation state images is performed, so that the moving image characteristics of the current goods are analyzed. For example: it can be obtained from the picture information that the target cargo is currently on the hand of the porter, and at the same time, it can be obtained from the image that the object is moving along with the movement of the porter, so that it can be obtained that the target cargo is in a delivery state on the porter. Of course, various cargo transportation conditions may be included, and are not limited in this regard.

And secondly, acquiring a single-channel feature image of the cargo transportation scene image, and setting the feature dimension of a corresponding feature extraction network or a feature extraction module to be 1 in an initial state, wherein the acquired feature image can be the single-channel feature image.

In order to better implement the embodiments of the present application, in an embodiment of the present application, feature extraction is performed on an acquired image of a cargo transportation scene, to obtain a single-channel feature image, including:

and sending the cargo transportation scene image into a preset feature extraction network to obtain a single-channel feature image, wherein the feature extraction network is a single-channel convolution layer, and the single-channel convolution layer is a single-channel convolution layer obtained after the processing of the multi-channel convolution layer.

If the feature extraction dimension is set to 1 in the feature extraction module according to the above embodiment, although a single-channel feature image can be obtained, the feature dimension is only 1, the obtained feature information is less, and sometimes the whole feature information of the cargo cannot be completely reflected.

Therefore, the single-channel characteristic image of the cargo transportation scene image can be acquired by improving the neural network. The neural network basically comprises at least feature information of an image obtained through a convolution layer, then the feature information is identified according to the related feature information, and finally a corresponding classification result is output, wherein the convolution layer is usually a plurality of layers, and the feature information with different dimensions is obtained through different convolution layers. However, in the embodiment of the present application, in order to obtain a single-channel feature image, the convolution layer with a larger number of channels obviously cannot output a single-channel feature image, so that the last convolution layer of the convolution layers needs to be replaced by a channel number changed to be 1 convolution layer, so that the finally obtained feature image is a single-channel feature image, and the features extracted by the previous convolution layers can be naturally retained. The purpose of obtaining the single-channel feature image is to filter out other noise when calculating the optical flow information image, for the following reasons.

202. And carrying out optical flow information calculation on the single-channel characteristic image to obtain an optical flow information image.

As is available from the above description of optical flow information, optical flow information is information that can express object motion information. In general, optical flow information is calculated by analyzing optical flow information of an object based on an image acquired by an imaging device. However, in practical situations, when the image capturing apparatus captures an associated image, a large amount of noise may be included in the captured image. For example: the imaging device is subject to unavoidable shake when taking a photograph. Therefore, if a transportation scene image with high noise is used, the calculated optical flow information is inaccurate, and thus the motion state of goods cannot be effectively monitored.

The mode of calculating the optical flow information image may be a TVL1 optical flow algorithm, and the obtained continuous cargo image is input into the algorithm to obtain an optical flow calculation result. It should be noted that the TVL1 optical flow algorithm is a conventional algorithm for calculating an optical flow image according to a conventional image, and is only used to obtain optical flow image information, but of course, optical flow image information may be obtained by other means, which is not limited herein.

In order to better implement the embodiments of the present application, in an embodiment of the present application, optical flow information calculation is performed on a single-channel feature image to obtain an optical flow information image, including:

calculating horizontal motion information of each pixel in the single-channel characteristic image to obtain a horizontal optical flow information image; and calculating the vertical motion information of each pixel in the single-channel characteristic image to obtain a vertical optical flow information image.

In order to make the obtained optical flow information more detailed, the optical flow image information obtained on the basis of the TVL1 optical flow algorithm can be decomposed into two optical flow image information, wherein one optical flow image information is horizontal channel optical flow image information, and the other optical flow image information is vertical channel optical flow image information, and the two optical flow image information can respectively show the motion information of each pixel point in the image on the horizontal position and the vertical position. Because the pixel points corresponding to the goods are also in the optical flow image information, the pixel points corresponding to the goods are determined, and the motion characteristics of the goods can be extracted through the horizontal and vertical motion information of the pixel points corresponding to the goods. For example, in a transportation scenario, the cargo box near the aircraft door moves in the right direction, and thus the cargo-corresponding region pixel value is brighter in the horizontal direction. Since there is substantially no movement in the vertical direction, there is no apparent regularity of the cargo correspondence area in the vertical direction, with only some disturbance information. Thus, the specific motion trail of the goods can be obtained in a refined way.

203. And monitoring the motion state of the goods by combining the goods transportation scene image, the single-channel characteristic image and the optical flow information image.

The optical flow information can reflect the motion trail of the goods, so that the motion state of the goods can be reflected to a certain extent, if the goods are at rest, the motion state of the goods is at rest, and the optical flow information cannot reflect the motion trail of the goods. Therefore, when special situations occur, the optical flow information cannot analyze the motion state of the cargo, and a specific cargo transportation scene image, a single-channel characteristic image and an optical flow information image are required to be combined together to monitor the motion state of the cargo. For example: according to the transportation scene image, some information of the goods in the transportation scene can be obtained, for example, the current transportation scene is air transportation, and a specific transportation mode of the goods can be obtained at the moment; at the moment, goods are analyzed and identified from the transportation scene through the feature extraction image; then, verification is performed through the optical flow image information. For example: when the goods and the belt surface of the conveyor belt are relatively static, but the displacement distance between the goods and the cabin is shortened, the moving state of the goods at the moment can be obtained to be the state of being transported by the conveyor belt; when the goods are not stationary with the belt surface of the conveyor belt but the displacement distance from the cabin is shortened, the moving state of the goods can be obtained as the moving state of the goods manually carried, and the like.

In order to better implement the embodiments of the present application, in an embodiment of the present application, in combination with a cargo transportation scene image, a single-channel feature image, and an optical flow information image, monitoring a motion state of cargo includes:

carrying out gray scale processing on the cargo transportation scene image to obtain a gray scale image; superposing the gray level image, the single-channel characteristic image, the horizontal optical flow information image and the vertical optical flow information image into a four-channel image; and sending the four-channel image into a preset cargo state monitoring model to monitor the motion state of the cargo.

According to the above embodiment, the images are analyzed one by one to obtain the motion state of the goods, but this requires storing the images separately, and if the images are fused into a four-channel image, only one storage space is required for storing. Meanwhile, the omitted process of integrating the images after analyzing each type one by one is performed. Therefore, after being fused into a four-channel image, the four-channel image is directly sent into a corresponding neural network model, so that the corresponding cargo transportation state can be directly obtained, and the monitoring efficiency can be improved.

It should be noted that, when the single-channel feature image, the horizontal optical flow information image, and the vertical optical flow information image are acquired, the resolution of these images is also the resolution of one fixed pixel, and specifically, when these corresponding images are acquired, the pixel resolution of these images may be changed in the output stage. In this case, however, the object of gradation processing of the cargo-moving scene image is to scale the gradation image to an image of the same fixed pixel resolution after the gradation processing, and fusion superimposition is easier. Of course, the gray image conversion is performed, and meanwhile, some noise in the original image can be removed.

In order to better implement the embodiments of the present application, in an embodiment of the present application, before performing feature extraction on an acquired image of a cargo transportation scene to obtain a single-channel feature image, the method further includes:

acquiring a cargo transportation scene video; and taking frames of the cargo transportation scene video one by one to obtain a cargo transportation scene image.

The purpose of taking frames of the cargo transportation scene video one by one is to obtain cargo transportation scene images with continuous frames, so that when optical flow information images are calculated, specific motion rules of cargoes can be obtained through better analysis according to the images with continuous frames.

The video frames can be ordered according to the time sequence in a mode of taking frames one by one, and the video frames are ordered according to the time from front to back, so that the obtained continuous cargo transportation images are the same as the direction of the time axis.

In order to better implement the embodiments of the present application, in an embodiment of the present application, in combination with the cargo transportation scene image, the single-channel feature image, and the optical flow information image, before monitoring the motion state of the cargo, the method further includes:

and carrying out cargo state classification training on a preset initial cargo state monitoring model to obtain a cargo state monitoring model.

Because, in the embodiment of the application, the motion state of the goods can be classified and analyzed through the corresponding monitoring neural network model. If the corresponding neural network model is not trained, the accuracy of the finally obtained classification result is poor, and the specific training link is not limited.

Therefore, specifically, the cargo state classification training is performed on the preset initial cargo state monitoring model to obtain the cargo state monitoring model, which includes:

creating a plurality of different cargo movement state tags; acquiring a plurality of groups of marked goods moving images, wherein one group of marked goods moving images corresponds to one goods moving state label, and one group of marked goods moving images and one corresponding goods moving state label are one sample training group; and sending the sample training group into an initial cargo state monitoring model for training to obtain the cargo state monitoring model.

Assuming that the air-borne transportation scenario is taken as a case, different cargo movement state labels are created at this time, and labels such as "no event", "loading", "unloading" and the like can be created. Because, the air traffic scene may include the following objects, for example: conveyors, conveyor belts, cargo, porters, transport vehicles, and the like. It is known that after a cargo is checked by a check-in place, the cargo is sent to the vicinity of a transport vehicle along with the transport vehicle, the transport vehicle and the transport vehicle have a conveyor belt, the approximate track of the cargo can be divided into a transport worker who carries the cargo from the transport vehicle to the conveyor belt, the conveyor belt transports the cargo to a transport worker at the other end, and then the transport worker carries the cargo to an airplane. Assuming that the conveyor belt includes two ends, a and B, the a end is closer to the conveyor than the B end is to the conveyor. Therefore, when the goods are on the transport vehicle, the goods are not moved at the moment, the goods are stationary relative to the transport vehicle at the moment, and if the monitoring model monitors that the goods are on the transport vehicle, namely the goods are not transported, the goods are in an event-free state; when the goods are carried down by the carrier and sent to the end A of the conveyor belt, the end A of the conveyor belt is closer to the transport vehicle than the end B of the conveyor belt, so that the carrier can be obtained to carry the goods from the transport vehicle to the conveyor belt, and when the goods monitoring model monitors the stage, namely the goods are unloaded and are in an unloading state; when the goods are on the conveyor belt, the end A of the conveyor belt is moved to the end B of the conveyor belt, and the end B of the conveyor belt is closer to the conveyor than the end A of the conveyor belt at the moment, so that a carrier can be obtained to carry the goods to the conveyor.

After the corresponding labels are determined, binding the training images corresponding to the corresponding labels with the labels, and sending the training images into the corresponding models one by one for training. For example, binding a corresponding image group of the "no event" tag with the "no event" tag, and sending the group into the monitoring model for training so that the model can learn the characteristic state of the goods when the "no event" occurs; then binding a corresponding image group of the 'loading' label with the 'loading' label, and sending the group into a monitoring model for training, so that the model can learn the characteristic state of goods during 'loading'; and finally, binding a corresponding image group of the unloading label with the unloading label, and sending the image group into a monitoring model for training, so that the model can learn the characteristic state of the goods during unloading. After the model can sufficiently identify the corresponding cargo state, for example, when the identification rate of the model reaches more than 90%, the model can be considered to be trained.

In order to better implement the cargo state monitoring method in the embodiments of the present application, above the cargo state monitoring method, a cargo state monitoring device is further provided in the embodiments of the present application, as shown in fig. 3, an apparatus 300 includes:

the feature extraction module 301 is configured to perform feature extraction on the acquired cargo transportation scene image, so as to obtain a single-channel feature image.

The optical flow calculation module 302 is configured to perform optical flow information calculation on the single-channel feature image to obtain an optical flow information image.

The cargo monitoring module 303 is configured to monitor a motion state of cargo in combination with the cargo transportation scene image, the single-channel feature image, and the optical flow information image.

According to the cargo state monitoring device, the feature extraction module 301 can be used for extracting features of images of cargoes in a transportation state, the optical flow calculation module 302 is combined on the basis of the extracted related feature images such as the motion state of the cargoes, the corresponding optical flow information images are calculated, and finally the motion state of the cargoes is monitored by combining the optical flow information images through the cargo monitoring module 303, so that the problem of sensitivity of the optical flow information images is solved, and the monitoring precision of the cargo transportation state is improved.

In some embodiments of the present application, the feature extraction module 301 is specifically configured to:

In some embodiments of the present application, the optical flow calculation module 302 is specifically configured to:

In some embodiments of the present application, the cargo monitoring module 303 is specifically configured to:

The embodiment of the application also provides a terminal device, which comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to realize the steps in the cargo state monitoring method of any one of the embodiments of the application. The terminal device integrates any cargo state monitoring method provided by the embodiment of the present application, as shown in fig. 4, which shows a schematic structural diagram of the terminal device according to the embodiment of the present application, specifically:

the terminal device may include one or more processing cores 'processors 401, one or more computer-readable storage media's memory 402, power supply 403, and input unit 404, among other components. It will be appreciated by those skilled in the art that the terminal device structure shown in fig. 4 is not limiting of the terminal device and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components. Wherein:

the processor 401 is a control center of the terminal device, connects respective parts of the entire terminal device using various interfaces and lines, and performs various functions of the terminal device and processes data by running or executing software programs and/or modules stored in the memory 402 and calling data stored in the memory 402, thereby performing overall monitoring of the terminal device. Optionally, processor 401 may include one or more processing cores; the processor 401 may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, preferably, the processor 401 may integrate an application processor, which primarily handles operating systems, user interfaces, applications, etc., with a modem processor, which primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 401.

The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and data processing by executing the software programs and modules stored in the memory 402. The memory 402 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data created according to the use of the terminal device, etc. In addition, memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

The terminal device further comprises a power supply 403 for supplying power to the various components, preferably the power supply 403 may be logically connected to the processor 401 by a power management system, so that functions of managing charging, discharging, power consumption management, etc. are achieved by the power management system. The power supply 403 may also include one or more of any of a direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.

The terminal device may further comprise an input unit 404, which input unit 404 may be used for receiving input digital or character information and generating keyboard, mouse, joystick, optical or trackball signal inputs in connection with user settings and function control.

Although not shown, the terminal device may further include a display unit or the like, which is not described herein. In this embodiment, the processor 401 in the terminal device loads executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 executes the application programs stored in the memory 402, so as to implement various functions, for example:

carrying out optical flow information calculation on the single-channel characteristic image to obtain an optical flow information image;

To this end, embodiments of the present application provide a computer readable storage medium, which may include: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like. On which a computer program is stored, the computer program being loaded by a processor for performing the steps of any of the cargo state monitoring methods provided by the embodiments of the present application. For example, the loading of the computer program by the processor may perform the steps of:

In the foregoing embodiments, the descriptions of the embodiments are focused on, and the portions of one embodiment that are not described in detail in the foregoing embodiments may be referred to in the foregoing detailed description of other embodiments, which are not described herein again.

In the implementation, each unit or structure may be implemented as an independent entity, or may be implemented as the same entity or several entities in any combination, and the implementation of each unit or structure may be referred to the foregoing method embodiments and will not be repeated herein.

The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.

The foregoing has described in detail a method and apparatus for monitoring cargo state provided by embodiments of the present application, and specific examples have been applied herein to illustrate principles and embodiments of the present application, where the foregoing examples are provided to assist in understanding the method and core ideas of the present application; meanwhile, as those skilled in the art will vary in the specific embodiments and application scope according to the ideas of the present application, the contents of the present specification should not be construed as limiting the present application in summary.

Claims

1. A method of monitoring cargo state, the method comprising:

2. The method for monitoring cargo state according to claim 1, wherein the feature extraction of the acquired cargo transportation scene image to obtain a single-channel feature image comprises:

3. The cargo state monitoring method according to claim 1, wherein the calculating optical flow information for the single-channel feature image to obtain an optical flow information image includes:

4. The cargo state monitoring method of claim 3, wherein the monitoring the motion state of the cargo in combination with the cargo transportation scene image, the single-channel feature image, and the optical flow information image comprises:

5. The cargo state monitoring method of claim 1, wherein before feature extraction is performed on the acquired cargo transportation scene image to obtain a single channel feature image, the method further comprises:

acquiring a cargo transportation scene video;

6. The cargo state monitoring method of claim 1, wherein the combining the cargo transportation scene image, the single-channel feature image, and the optical flow information image, prior to monitoring the motion state of the cargo, further comprises:

7. The method for monitoring cargo state according to claim 6, wherein the training for classifying cargo states on the preset initial cargo state monitoring model to obtain the cargo state monitoring model comprises:

creating a plurality of different cargo movement state tags;

8. A cargo state monitoring device, the device comprising:

9. A terminal device, characterized in that it comprises a processor, a memory and a computer program stored in the memory and executable on the processor, the processor executing the computer program to carry out the steps in the cargo state monitoring method according to any one of claims 1 to 7.

10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which is executed by a processor to implement the steps in the cargo state monitoring method of any of claims 1 to 7.