CN117708071B - Processing method and device for coal mine equipment operation parameters based on big data - Google Patents
Processing method and device for coal mine equipment operation parameters based on big data Download PDFInfo
- Publication number
- CN117708071B CN117708071B CN202410169628.8A CN202410169628A CN117708071B CN 117708071 B CN117708071 B CN 117708071B CN 202410169628 A CN202410169628 A CN 202410169628A CN 117708071 B CN117708071 B CN 117708071B
- Authority
- CN
- China
- Prior art keywords
- data
- unit
- compression
- operation data
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000003245 coal Substances 0.000 title claims abstract description 30
- 238000003672 processing method Methods 0.000 title claims abstract description 6
- 238000007906 compression Methods 0.000 claims abstract description 136
- 230000006835 compression Effects 0.000 claims abstract description 135
- 238000010977 unit operation Methods 0.000 claims abstract description 59
- 238000012545 processing Methods 0.000 claims abstract description 44
- 238000000034 method Methods 0.000 claims abstract description 42
- 230000005540 biological transmission Effects 0.000 claims abstract description 14
- 230000002159 abnormal effect Effects 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 16
- 238000004891 communication Methods 0.000 claims description 9
- 238000012163 sequencing technique Methods 0.000 claims description 5
- 238000005265 energy consumption Methods 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 4
- 238000013144 data compression Methods 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
- G06F16/1744—Redundancy elimination performed by the file system using compression, e.g. sparse files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/02—Agriculture; Fishing; Forestry; Mining
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Marine Sciences & Fisheries (AREA)
- Human Resources & Organizations (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- General Health & Medical Sciences (AREA)
- Tourism & Hospitality (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Animal Husbandry (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mining & Mineral Resources (AREA)
- General Business, Economics & Management (AREA)
- Strategic Management (AREA)
- Agronomy & Crop Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
- Arrangements For Transmission Of Measured Signals (AREA)
Abstract
The application provides a processing method and device for coal mine equipment operation parameters based on big data, a computer readable medium and electronic equipment. The method comprises the following steps: acquiring operation data of equipment, and storing the operation data based on the data type of the operation data; determining a compression unit and corresponding unit operation data and a data label based on the data quantity and the data value of the operation data; compressing unit operation data corresponding to each compression unit to generate unit compression data, and combining the unit compression data to generate compression data; the compressed data is transmitted to a server. According to the technical scheme, in the aspect of processing the operation data of the coal mine equipment, the operation data is compressed, so that the compression of the operation data can be effectively improved while the accuracy of the data is ensured, the volume and the energy consumption of the processing equipment can be effectively reduced, the pressure of a network bandwidth and a server is greatly reduced, and the data transmission efficiency and the data processing efficiency are improved.
Description
Technical Field
The application relates to the technical field of computers, in particular to a method and a device for processing coal mine equipment operation parameters based on big data, a computer readable medium and electronic equipment.
Background
With the development and popularization of underground coal mine services, real-time data acquisition and monitoring of underground coal mine equipment are required. Due to the wide variety of coal mine equipment, the operating parameters of the equipment in the operation process or the collected environmental data are wide in variety and large in data size. In current downhole operation data transmission, data is typically received and processed in full disc. The large amount of data occupies more storage space and computing resources in the processes of transmission, processing and the like, can cause larger stress on a transmission network, and has errors caused by untimely data transmission. Thus causing a problem of low data processing efficiency.
Disclosure of Invention
The embodiment of the application provides a processing method and device for coal mine equipment operation parameters based on big data, a computer readable medium and electronic equipment, and further solves the problem of low data processing efficiency at least to a certain extent.
Other features and advantages of the application will be apparent from the following detailed description, or may be learned by the practice of the application.
According to one aspect of the application, there is provided a method for processing coal mine equipment operation parameters based on big data, comprising: acquiring operation data of equipment; storing the operation data based on the data type of the operation data; determining a compression unit and corresponding unit operation data and a data label based on the data quantity and the data value of the operation data; the data tag is used for representing a unit identifier corresponding to the compression unit; compressing unit operation data corresponding to each compression unit to generate unit compression data, and combining the unit compression data to generate compression data; and transmitting the compressed data to a server.
In the present application, based on the foregoing scheme, the acquiring operation data of the device includes: acquiring an equipment identifier of the equipment; determining the equipment type according to the equipment identifier; determining the acquisition frequency of operation data of equipment based on a preset relation between the type of the equipment and the acquisition frequency; and acquiring the operation data of the equipment according to the acquisition frequency.
In the present application, based on the foregoing scheme, the storing the operation data based on the data type of the operation data includes: detecting the data type of the operation data when the operation data is acquired; comparing the storage type in the database with the data type to determine a target storage position corresponding to the operation data; and storing the operation data based on the target storage position.
In the present application, based on the foregoing scheme, the compression unit and the corresponding unit operation data and data tag are determined based on the data amount and the data value of the operation data; the data tag is used for representing the unit identifier corresponding to the compression unit, and then further comprises: and generating a data tag of the unit operation data corresponding to the compression unit based on the data value of the unit operation data.
In the present application, based on the foregoing aspect, the compressing unit operation data corresponding to each compression unit to generate unit compressed data includes: sequencing according to the generation time of the operation data to obtain head data and tail data in the unit operation data; on a connecting line formed by head data and tail data, determining target data with the largest distance from the connecting line and the distance between the target data and the connecting line; if the distance is smaller than or equal to a set threshold value, deleting the target data; if the distance is greater than a set threshold, carrying out polling by using the connection lines between the target data and the head data and the tail data respectively; and circulating until all units in the compression unit run data are polled and processed, and generating unit compressed data.
In the present application, based on the foregoing scheme, after the storing the operation data, the method further includes: detecting abnormal data in the operation data; and independently storing the abnormal data and removing the abnormal data from the operation data.
In the present application, based on the foregoing scheme, the transmitting the compressed data to a server includes: generating compression information based on the data amount of the operation data and the data amount of the compression data; and transmitting the compressed data communication compressed information to a server.
According to one aspect of the application, there is provided a processing apparatus for coal mine equipment operating parameters based on big data, comprising:
an acquisition unit configured to acquire operation data of the device;
The storage unit is used for storing the operation data based on the data type of the operation data;
A data unit for determining a compression unit and corresponding unit operation data and data tags based on the data amount and the data value of the operation data; the data tag is used for representing a unit identifier corresponding to the compression unit;
The compression unit is used for compressing the unit operation data corresponding to each compression unit to generate unit compression data, and combining the unit compression data to generate compression data;
And the transmission unit is used for transmitting the compressed data to a server.
In the present application, based on the foregoing scheme, the acquiring operation data of the device includes: acquiring an equipment identifier of the equipment; determining the equipment type according to the equipment identifier; determining the acquisition frequency of operation data of equipment based on a preset relation between the type of the equipment and the acquisition frequency; and acquiring the operation data of the equipment according to the acquisition frequency.
In the present application, based on the foregoing scheme, the storing the operation data based on the data type of the operation data includes: detecting the data type of the operation data when the operation data is acquired; comparing the storage type in the database with the data type to determine a target storage position corresponding to the operation data; and storing the operation data based on the target storage position.
In the present application, based on the foregoing scheme, the compression unit and the corresponding unit operation data and data tag are determined based on the data amount and the data value of the operation data; the data tag is used for representing the unit identifier corresponding to the compression unit, and then further comprises: generating a data tag of the unit operation data corresponding to the compression unit based on the data value of the unit operation data
In the present application, based on the foregoing aspect, the compressing unit operation data corresponding to each compression unit to generate unit compressed data includes: sequencing according to the generation time of the operation data to obtain head data and tail data in the unit operation data; on a connecting line formed by head data and tail data, determining target data with the largest distance from the connecting line and the distance between the target data and the connecting line; if the distance is smaller than or equal to a set threshold value, deleting the target data; if the distance is greater than a set threshold, carrying out polling by using the connection lines between the target data and the head data and the tail data respectively; and circulating until all units in the compression unit run data are polled and processed, and generating unit compressed data.
In the present application, based on the foregoing scheme, after the storing the operation data, the method further includes: detecting abnormal data in the operation data; and independently storing the abnormal data and removing the abnormal data from the operation data.
In the present application, based on the foregoing scheme, the transmitting the compressed data to a server includes: generating compression information based on the data amount of the operation data and the data amount of the compression data; and transmitting the compressed data communication compressed information to a server.
According to one aspect of the present application there is provided a computer readable medium having stored thereon a computer program which when executed by a processor implements a method of processing coal mine equipment operating parameters based on big data as described in the above embodiments.
According to an aspect of the present application, there is provided an electronic apparatus including: one or more processors; and the storage device is used for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are enabled to realize the processing method of the coal mine equipment operation parameters based on big data in the embodiment.
According to one aspect of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium and executes the computer instructions to cause the computer device to perform the method of processing coal mine equipment operating parameters based on big data provided in the various alternative implementations described above.
In the technical scheme of the application, the operation data of the equipment are acquired, and the operation data are stored based on the data type of the operation data; determining a compression unit and corresponding unit operation data and a data label based on the data quantity and the data value of the operation data; the data tag is used for representing a unit identifier corresponding to the compression unit; compressing unit operation data corresponding to each compression unit to generate unit compression data, and combining the unit compression data to generate compression data; and transmitting the compressed data to a server. According to the technical scheme, in the aspect of processing the operation data of the coal mine equipment, the operation data is compressed, so that the compression of the operation data can be effectively improved while the accuracy of the data is ensured, the volume and the energy consumption of the processing equipment can be effectively reduced, the pressure of a network bandwidth and a server is greatly reduced, and the data transmission efficiency and the data processing efficiency are improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application as claimed.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application. It is evident that the drawings in the following description are only some embodiments of the present application and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art.
FIG. 1 schematically illustrates a flow chart of a method of processing coal mine equipment operating parameters based on big data in one embodiment of the application.
FIG. 2 schematically illustrates a flow chart for generating compressed data in one embodiment of the application.
FIG. 3 schematically illustrates a schematic diagram of a processing apparatus for coal mine equipment operating parameters based on big data in one embodiment of the present application.
Fig. 4 shows a schematic diagram of a computer system suitable for use in implementing an embodiment of the application.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the application may be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the application.
The block diagrams depicted in the figures are merely functional entities and do not necessarily correspond to physically separate entities. That is, the functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
The flow diagrams depicted in the figures are exemplary only, and do not necessarily include all of the elements and operations/steps, nor must they be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the order of actual execution may be changed according to actual situations.
The implementation details of the technical scheme of the application are explained in detail as follows:
FIG. 1 illustrates a flow chart of a method of processing coal mine equipment operating parameters based on big data in accordance with one embodiment of the present application. Referring to fig. 1, the method for processing the operation parameters of the coal mine equipment based on the big data at least comprises steps S110 to S150, and is described in detail as follows:
in step S110, operation data of the device is acquired.
In one embodiment of the application, the operational data of the device is obtained prior to performing the compression of the operational data.
The operation data in this embodiment includes operation data of various devices in the coal mine production process.
In one embodiment of the application, obtaining operational data of a device includes:
Acquiring an equipment identifier of the equipment;
determining the equipment type according to the equipment identifier;
determining the acquisition frequency of operation data of equipment based on a preset relation between the type of the equipment and the acquisition frequency;
And acquiring the operation data of the equipment according to the acquisition frequency.
In an embodiment of the present application, an equipment identifier of an equipment to be acquired data is acquired first, and then an equipment type to which the equipment belongs is determined based on the equipment identifier. And then determining the acquisition frequency corresponding to the operation data of the equipment based on a preset relation between the equipment type and the acquisition frequency. To obtain operational data of the device based on the acquisition frequency.
Optionally, the operation data in the scheme can be collected through a sensing device, an infrared device and the device itself.
In step S120, the operation data is stored based on the data type of the operation data.
In one embodiment of the present application, after the operation data is acquired, the operation data is stored for a subsequent compression process according to the data type of the operation data.
In one embodiment of the present application, storing the operation data based on a data type of the operation data includes:
detecting the data type of the operation data when the operation data is acquired;
comparing the storage type in the database with the data type to determine a target storage position corresponding to the operation data;
And storing the operation data based on the target storage position.
In an embodiment of the application, corresponding storage positions are preset for each data type, so that the data can be stored, extracted and applied conveniently. Specifically, when operation data is acquired, the data type of the operation data is detected. And then obtaining the storage type in the database, comparing the storage type with the data type of the operation data, and determining a target storage position corresponding to the data type. The operational data is then stored based on the target storage location.
Specifically, in this embodiment, when determining the target storage location corresponding to the operation data, the character string corresponding to the data type of the operation data is based onCharacter string corresponding to storage type in databaseMatching and determining related parameters/>, between the twoThe method comprises the following steps:
Where i denotes the character order and k denotes the number of characters. When the data type corresponding to the minimum relevant parameter is determined, namely the running data corresponds to the storage type in the database, and then the storage position corresponding to the storage type in the memory is acquired, namely the target storage position.
Further, in the technical scheme of the present application, after storing the operation data based on the data type of the operation data, the method further includes:
detecting abnormal data in the operation data;
and independently storing the abnormal data and removing the abnormal data from the operation data.
In an embodiment of the present application, before data compression, abnormal data in the operation data is detected. The abnormal data is independently stored and removed from the operation data, so that the subsequent data compression is facilitated, and compression errors caused by the abnormal data are prevented. The accuracy of data compression is improved through the mode.
In step S130, determining a compression unit and corresponding unit operation data and a data tag thereof based on the data amount and the data value of the operation data; the data tag is used for representing a unit identifier corresponding to the compression unit.
In one embodiment of the present application, the compression of the operation data is based on the data amount and the data value, and the compressed data is generated by compressing the operation data based on the data amount and the data value of the operation data.
In one embodiment of the application, a compression unit and corresponding unit operation data and data labels are determined based on the data amount and data value of the operation data; the data tag is used for representing a unit identifier corresponding to the compression unit, and comprises:
s210, determining a compression unit based on the data amount of the operation data;
s220, acquiring unit operation data from the operation data based on the compression unit;
S230, generating a data tag of the unit operation data corresponding to the compression unit based on the data value of the unit operation data.
In an embodiment of the present application, the compression efficiency is low due to the numerous data volumes of the operation data. Therefore, in this embodiment, the data size of the operation data is split according to the data size of the operation dataDetermination of compression Unit/>The method comprises the following steps:
Wherein, Representing the compression factor. The compression units are calculated through the above process and are used for representing the data volume of a compressed batch of data, and all the compression units form the data volume of the operation data. After the unit operation data corresponding to the compression unit is acquired, the unit operation data is taken as a compression unit, and the data in the unit operation data are synchronously and parallelly compressed to generate the corresponding unit compression data. And then combining all the unit compressed data to generate compressed data.
In an embodiment of the present application, after obtaining unit operation data from the operation data based on the compression unit, the method further includes: and generating a data tag of the unit operation data corresponding to the compression unit based on the data value of the unit operation data. Specifically, according to the data value of the data in one compression unit, the average value of the data value is obtained and is used as the data label of the unit operation data corresponding to the compression unit, and the data label corresponding to the compression unit is used for representing the uniqueness of the data.
In step S140, the unit operation data corresponding to each compression unit is compressed to generate unit compressed data, and the unit compressed data are combined to generate compressed data.
In an embodiment of the present application, unit operation data corresponding to each compression unit is compressed to generate unit compressed data; and merging the unit compressed data corresponding to each compression unit to generate the compressed data.
The unit operation data corresponding to each compression unit is compressed to generate unit compression data, which comprises the following steps:
Sequencing according to the generation time of the operation data to obtain head data and tail data in the unit operation data;
on a connecting line formed by head data and tail data, determining target data with the largest distance from the connecting line and the distance between the target data and the connecting line;
if the distance is smaller than or equal to a set threshold value, deleting the target data;
If the distance is greater than a set threshold, carrying out polling by using the connection lines between the target data and the head data and the tail data respectively;
And circulating until all units in the compression unit run data are polled and processed, and generating unit compressed data.
In one embodiment of the application, the collected operation data is generated into a scatter diagram according to the independent variable sequence of time, position and the like. For example, the abscissa is the generation time of the operation data, and the ordinate is the numerical value of the operation data.
The stored operation data is compressed, wherein it should be noted that, in the process of compression, due to the difference of the preset thresholds, the compression is performed for a plurality of times to reach the preset threshold, and the compression is stopped after reaching the preset compression rate.
Optionally, a maximum sampling interval between two sampling points is set, so that data omission caused by overlarge sampling interval is avoided.
And acquiring operation data stored in the time period, wherein the first and the last points A, B in the acquired operation data form a motion track which is called an AB line. And calculating a point C with the largest distance from the straight line segment on the curve, and calculating the distance d between the point C and the AB. And comparing the distance of the d value with a preset threshold value, and if the distance is smaller than the preset threshold value, taking the straight line as an approximation of the curve, and finishing the processing of the section of curve. The three points ABC obtained in this way are the final data after compression.
If the distance between the points d is larger than the threshold value, the curve is divided into two sections AC and BC by the point C, and the two sections of the curve are respectively processed by the steps.
When all curves reach the threshold value, representing that the processing of the running data of the road section is finished, sequentially connecting all the dividing points to form broken lines, and taking the broken lines as compressed data corresponding to the original curve of the running data. Each dot is original data, and lines connected by ABC three points are compressed data obtained after algorithm calculation. The points connected on this line are the final compressed data.
In step S150, the compressed data is transmitted to a server.
After the whole processing of the operation data is completed, the calculated compressed data can be transmitted to the target server through the transmission module of the equipment. After all the data are processed and the data are successfully transmitted to the target server, the data can be deleted from the device, the data quantity stored by the device is reduced, errors of invalid data on subsequent data are avoided, the reliability of the data is improved, and meanwhile, the requirement on storage performance is reduced.
In one embodiment of the present application, compression information is generated based on the data amount of the operation data, the data amount of the compression data; and transmitting the compressed data communication compressed information to a server.
In one embodiment of the present application, compressed information is generated based on the data amount of the operation data and the data amount of the compressed data to transmit the compressed data to the server together with the compressed information. To inform the server of the amount of data after the current compression and the amount of data before the compression, facilitating the subsequent data processing and management.
The scheme has the advantages that the compression rate of data can be adjusted as required, and the processing logic is efficient and rapid. Meanwhile, as only the data and the threshold value of each point position are required to be stored, the processing capacity of the method to the equipment is greatly reduced, the volume and the energy consumption of the processing equipment can be effectively reduced, the pressure to the network bandwidth and the server can be greatly reduced by the processed data, and the data transmission efficiency of a user is greatly improved. Therefore, in the aspect of processing the current underground operation data, the operation data is compressed, so that the demands on the aspects of compression, transmission, processing and the like of the operation data can be effectively improved while the accuracy of the data is ensured, the data volume is greatly reduced, the data can be transmitted and analyzed, and the data is greatly improved.
In the technical scheme of the application, the operation data of the equipment are acquired, and the operation data are stored based on the data type of the operation data; determining a compression unit and corresponding unit operation data and a data label based on the data quantity and the data value of the operation data; the data tag is used for representing a unit identifier corresponding to the compression unit; compressing unit operation data corresponding to each compression unit to generate unit compression data, and combining the unit compression data to generate compression data; and transmitting the compressed data to a server. According to the technical scheme, in the aspect of processing the operation data of the coal mine equipment, the operation data is compressed, so that the compression of the operation data can be effectively improved while the accuracy of the data is ensured, the volume and the energy consumption of the processing equipment can be effectively reduced, the pressure of a network bandwidth and a server is greatly reduced, and the data transmission efficiency and the data processing efficiency are improved.
The following describes an embodiment of the apparatus of the present application, which may be used to perform the method for processing the operation parameters of the coal mine equipment based on big data in the above embodiment of the present application. It will be appreciated that the apparatus may be a computer program (including program code) running in a computer device, for example the apparatus being an application software; the device can be used for executing corresponding steps in the method provided by the embodiment of the application. For details not disclosed in the embodiment of the apparatus of the present application, please refer to the embodiment of the method for processing the operation parameters of the coal mine equipment based on big data.
FIG. 3 illustrates a block diagram of a processing device for coal mine equipment operating parameters based on big data in accordance with one embodiment of the present application.
Referring to fig. 3, a processing apparatus for coal mine equipment operation parameters based on big data according to an embodiment of the present application includes:
an acquiring unit 310, configured to acquire operation data of the device;
A storage unit 320, configured to store the operation data based on a data type of the operation data;
a data unit 330 for determining a compression unit and corresponding unit operation data and data tags based on the data amount and data value of the operation data; the data tag is used for representing a unit identifier corresponding to the compression unit;
A compression unit 340, configured to compress unit operation data corresponding to each compression unit to generate unit compression data, and combine the unit compression data to generate compression data;
and a transmission unit 350 for transmitting the compressed data to a server.
In the present application, based on the foregoing scheme, the acquiring operation data of the device includes: acquiring an equipment identifier of the equipment; determining the equipment type according to the equipment identifier; determining the acquisition frequency of operation data of equipment based on a preset relation between the type of the equipment and the acquisition frequency; and acquiring the operation data of the equipment according to the acquisition frequency.
In the present application, based on the foregoing scheme, the storing the operation data based on the data type of the operation data includes: detecting the data type of the operation data when the operation data is acquired; comparing the storage type in the database with the data type to determine a target storage position corresponding to the operation data; and storing the operation data based on the target storage position.
In the present application, based on the foregoing scheme, the compression unit and the corresponding unit operation data and data tag are determined based on the data amount and the data value of the operation data; the data tag is used for representing the unit identifier corresponding to the compression unit, and then further comprises: generating a data tag of the unit operation data corresponding to the compression unit based on the data value of the unit operation data
In the present application, based on the foregoing aspect, the compressing unit operation data corresponding to each compression unit to generate unit compressed data includes: sequencing according to the generation time of the operation data to obtain head data and tail data in the unit operation data; on a connecting line formed by head data and tail data, determining target data with the largest distance from the connecting line and the distance between the target data and the connecting line; if the distance is smaller than or equal to a set threshold value, deleting the target data; if the distance is greater than a set threshold, carrying out polling by using the connection lines between the target data and the head data and the tail data respectively; and circulating until all units in the compression unit run data are polled and processed, and generating unit compressed data.
In the present application, based on the foregoing scheme, after the storing the operation data, the method further includes: detecting abnormal data in the operation data; and independently storing the abnormal data and removing the abnormal data from the operation data.
In the present application, based on the foregoing scheme, the transmitting the compressed data to a server includes: generating compression information based on the data amount of the operation data and the data amount of the compression data; and transmitting the compressed data communication compressed information to a server.
In the technical scheme of the application, the operation data of the equipment are acquired, and the operation data are stored based on the data type of the operation data; determining a compression unit and corresponding unit operation data and a data label based on the data quantity and the data value of the operation data; the data tag is used for representing a unit identifier corresponding to the compression unit; compressing unit operation data corresponding to each compression unit to generate unit compression data, and combining the unit compression data to generate compression data; and transmitting the compressed data to a server. According to the technical scheme, in the aspect of processing the operation data of the coal mine equipment, the operation data is compressed, so that the compression of the operation data can be effectively improved while the accuracy of the data is ensured, the volume and the energy consumption of the processing equipment can be effectively reduced, the pressure of a network bandwidth and a server is greatly reduced, and the data transmission efficiency and the data processing efficiency are improved.
Fig. 4 shows a schematic diagram of a computer system suitable for use in implementing an embodiment of the application.
It should be noted that, the computer system 400 of the electronic device shown in the drawings is only an example, and should not impose any limitation on the functions and the application scope of the embodiments of the present application.
Among them, the computer system 400 includes a central processing unit (Central Processing Unit, CPU) 401 that can perform various appropriate actions and processes, such as performing the methods described in the above embodiments, according to a program stored in a Read-Only Memory (ROM) 402 or a program loaded from a storage section 408 into a random access Memory (Random Access Memory, RAM) 403. In the RAM 403, various programs and data required for the system operation are also stored. The CPU 401, ROM 402, and RAM 403 are connected to each other by a bus 404. An Input/Output (I/O) interface 405 is also connected to bus 404.
The following components are connected to the I/O interface 405: an input section 406 including a keyboard, a mouse, and the like; an output portion 407 including a Cathode Ray Tube (CRT), a Liquid crystal display (Liquid CRYSTAL DISPLAY, LCD), and a speaker, etc.; a storage section 408 including a hard disk or the like; and a communication section 409 including a network interface card such as a LAN (Local Area Network ) card, a modem, or the like. The communication section 409 performs communication processing via a network such as the internet. The drive 410 is also connected to the I/O interface 405 as needed. A removable medium 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed on the drive 410 as needed, so that a computer program read therefrom is installed into the storage section 408 as needed.
In particular, according to embodiments of the present application, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising a computer program for performing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 409 and/or installed from the removable medium 411. When executed by a Central Processing Unit (CPU) 401, performs the various functions defined in the system of the present application.
It should be noted that, the computer readable medium shown in the embodiments of the present application may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-Only Memory (ROM), an erasable programmable read-Only Memory (Erasable Programmable Read Only Memory, EPROM), a flash Memory, an optical fiber, a portable compact disc read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present application, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with a computer-readable computer program embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. A computer program embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. Where each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present application may be implemented by software, or may be implemented by hardware, and the described units may also be provided in a processor. Wherein the names of the units do not constitute a limitation of the units themselves in some cases.
According to one aspect of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions are read from the computer-readable storage medium by a processor of a computer device, and executed by the processor, cause the computer device to perform the methods provided in the various alternative implementations described above.
As another aspect, the present application also provides a computer-readable medium that may be contained in the electronic device described in the above embodiment; or may exist alone without being incorporated into the electronic device. The computer-readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to implement the methods described in the above embodiments.
It should be noted that although in the above detailed description several modules or units of a device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functions of two or more modules or units described above may be embodied in one module or unit in accordance with embodiments of the application. Conversely, the features and functions of one module or unit described above may be further divided into a plurality of modules or units to be embodied.
From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present application may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.) or on a network, and includes several instructions to cause a computing device (may be a personal computer, a server, a touch terminal, or a network device, etc.) to perform the method according to the embodiments of the present application.
Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains.
It is to be understood that the application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.
Claims (9)
1. The processing method of the coal mine equipment operation parameters based on the big data is characterized by comprising the following steps:
Acquiring operation data of equipment;
storing the operation data based on the data type of the operation data;
determining a compression unit and corresponding unit operation data and a data label based on the data quantity and the data value of the operation data; the data tag is used for representing a unit identifier corresponding to the compression unit;
Compressing unit operation data corresponding to each compression unit to generate unit compression data, and combining the unit compression data to generate compression data;
Transmitting the compressed data to a server;
wherein determining the compression unit and the corresponding unit operation data and the data label based on the data amount and the data value of the operation data comprises:
Data volume based on the operational data Determination of compression Unit/>The method comprises the following steps:
Wherein, Representing the compression factor;
Acquiring unit operation data from the operation data based on the compression unit;
and generating a data tag of the unit operation data corresponding to the compression unit based on the average value of the data values of the unit operation data.
2. The method of claim 1, wherein obtaining operational data of the device comprises:
Acquiring an equipment identifier of the equipment;
determining the equipment type according to the equipment identifier;
determining the acquisition frequency of operation data of equipment based on a preset relation between the type of the equipment and the acquisition frequency;
And acquiring the operation data of the equipment according to the acquisition frequency.
3. The method of claim 1, wherein storing the operational data based on a data type of the operational data comprises:
detecting the data type of the operation data when the operation data is acquired;
comparing the storage type in the database with the data type to determine a target storage position corresponding to the operation data;
And storing the operation data based on the target storage position.
4. The method of claim 1, wherein compressing the unit operation data corresponding to each of the compression units to generate unit compressed data comprises:
Sequencing according to the generation time of the operation data to obtain head data and tail data in the unit operation data;
on a connecting line formed by head data and tail data, determining target data with the largest distance from the connecting line and the distance between the target data and the connecting line;
if the distance is smaller than or equal to a set threshold value, deleting the target data;
If the distance is greater than a set threshold, carrying out polling by using the connection lines between the target data and the head data and the tail data respectively;
And circulating until all units in the compression unit run data are polled and processed, and generating unit compressed data.
5. The method of claim 1, wherein after storing the operational data based on the data type of the operational data, further comprising:
detecting abnormal data in the operation data;
and independently storing the abnormal data and removing the abnormal data from the operation data.
6. The method of claim 1, wherein transmitting the compressed data to a server comprises:
generating compression information based on the data amount of the operation data and the data amount of the compression data;
and transmitting the compressed data communication compressed information to a server.
7. A processing apparatus for coal mine equipment operation parameters based on big data, comprising:
an acquisition unit configured to acquire operation data of the device;
The storage unit is used for storing the operation data based on the data type of the operation data;
A data unit for determining a compression unit and corresponding unit operation data and data tags based on the data amount and the data value of the operation data; the data tag is used for representing a unit identifier corresponding to the compression unit;
The compression unit is used for compressing the unit operation data corresponding to each compression unit to generate unit compression data, and combining the unit compression data to generate compression data;
a transmission unit for transmitting the compressed data to a server;
wherein determining the compression unit and the corresponding unit operation data and the data label based on the data amount and the data value of the operation data comprises:
Data volume based on the operational data Determination of compression Unit/>The method comprises the following steps:
Wherein, Representing the compression factor;
Acquiring unit operation data from the operation data based on the compression unit;
and generating a data tag of the unit operation data corresponding to the compression unit based on the average value of the data values of the unit operation data.
8. A computer readable medium having stored thereon a computer program which when executed by a processor implements a method of processing operational parameters of a big data based coal mine equipment as claimed in any of claims 1 to 6.
9. An electronic device, comprising:
one or more processors;
Storage means for storing one or more programs which when executed by the one or more processors cause the one or more processors to implement the method of processing coal mine equipment operating parameters based on big data as claimed in any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410169628.8A CN117708071B (en) | 2024-02-06 | 2024-02-06 | Processing method and device for coal mine equipment operation parameters based on big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410169628.8A CN117708071B (en) | 2024-02-06 | 2024-02-06 | Processing method and device for coal mine equipment operation parameters based on big data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117708071A CN117708071A (en) | 2024-03-15 |
| CN117708071B true CN117708071B (en) | 2024-05-28 |
Family
ID=90162939
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410169628.8A Active CN117708071B (en) | 2024-02-06 | 2024-02-06 | Processing method and device for coal mine equipment operation parameters based on big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117708071B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE10029455A1 (en) * | 1999-10-17 | 2001-07-19 | Mitsubishi Internat Gmbh | Method and a device for transmitting operating data of an operating unit and a method for providing operating data of one or more operating units |
| JP2006174331A (en) * | 2004-12-20 | 2006-06-29 | Fuji Xerox Co Ltd | Image compression apparatus, image compression method, and program therefor |
| CN106713926A (en) * | 2016-12-28 | 2017-05-24 | 北京普及芯科技有限公司 | Compression storage method and device for video data |
| CN113157655A (en) * | 2020-01-22 | 2021-07-23 | 阿里巴巴集团控股有限公司 | Data compression method, data decompression method, data compression device, data decompression device, electronic equipment and storage medium |
| CN113312325A (en) * | 2021-05-24 | 2021-08-27 | 康键信息技术（深圳）有限公司 | Track data transmission method, device, equipment and storage medium |
| CN114298679A (en) * | 2021-12-28 | 2022-04-08 | 深圳市玄羽科技有限公司 | Industrial AI platform based on big data, application method, medium and electronic equipment thereof |
| CN116644031A (en) * | 2023-07-27 | 2023-08-25 | 北京联创高科信息技术有限公司 | Method and system for unified standardization of coal mine water damage data in different formats |
| CN116827965A (en) * | 2023-08-25 | 2023-09-29 | 苔花科迈(西安)信息技术有限公司 | Coal mine underground offline scene data storage and synchronization method based on cloud platform |
| CN116862703A (en) * | 2023-07-18 | 2023-10-10 | 中国安全生产科学研究院 | Multi-port non-coal mine safety monitoring information control system and method |
-
2024
- 2024-02-06 CN CN202410169628.8A patent/CN117708071B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE10029455A1 (en) * | 1999-10-17 | 2001-07-19 | Mitsubishi Internat Gmbh | Method and a device for transmitting operating data of an operating unit and a method for providing operating data of one or more operating units |
| JP2006174331A (en) * | 2004-12-20 | 2006-06-29 | Fuji Xerox Co Ltd | Image compression apparatus, image compression method, and program therefor |
| CN106713926A (en) * | 2016-12-28 | 2017-05-24 | 北京普及芯科技有限公司 | Compression storage method and device for video data |
| CN113157655A (en) * | 2020-01-22 | 2021-07-23 | 阿里巴巴集团控股有限公司 | Data compression method, data decompression method, data compression device, data decompression device, electronic equipment and storage medium |
| CN113312325A (en) * | 2021-05-24 | 2021-08-27 | 康键信息技术（深圳）有限公司 | Track data transmission method, device, equipment and storage medium |
| CN114298679A (en) * | 2021-12-28 | 2022-04-08 | 深圳市玄羽科技有限公司 | Industrial AI platform based on big data, application method, medium and electronic equipment thereof |
| CN116862703A (en) * | 2023-07-18 | 2023-10-10 | 中国安全生产科学研究院 | Multi-port non-coal mine safety monitoring information control system and method |
| CN116644031A (en) * | 2023-07-27 | 2023-08-25 | 北京联创高科信息技术有限公司 | Method and system for unified standardization of coal mine water damage data in different formats |
| CN116827965A (en) * | 2023-08-25 | 2023-09-29 | 苔花科迈(西安)信息技术有限公司 | Coal mine underground offline scene data storage and synchronization method based on cloud platform |
Non-Patent Citations (2)
| Title |
|---|
| mproving ISO 11783 file transfers into mobile farm equipments using on-the-fly data compression;Iglesias, N 等;《COMPUTERS AND ELECTRONICS IN AGRICULTURE》;20150107;101-108 * |
| 大型煤炭企业总部集成监控系统设计;陈汉章;王继生;吴浩;;工矿自动化;20181120(12);93-97 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117708071A (en) | 2024-03-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115122155B (en) | Machine tool remote diagnosis method and system based on industrial internet big data | |
| CN109597810B (en) | Task segmentation method, device, medium and electronic equipment | |
| CN116893912B (en) | Inter-core communication method, system, device, equipment and medium for vehicle-mounted software | |
| CN112162859A (en) | Data processing method and device, computer readable medium and electronic equipment | |
| CN112101543A (en) | Neural network model determination method and device, electronic equipment and readable storage medium | |
| CN113868214A (en) | Log storage method and device, electronic equipment and computer readable medium | |
| CN117788589B (en) | Method and device for reinforcing and packaging semiconductor chip | |
| CN104376584B (en) | A kind of method of data compression, computer system and device | |
| CN113204695B (en) | Website identification method and device | |
| CN117971487A (en) | High-performance operator generation method, device, equipment and storage medium | |
| CN113904943A (en) | Account detection method and device, electronic equipment and storage medium | |
| CN117708071B (en) | Processing method and device for coal mine equipment operation parameters based on big data | |
| CN115186738B (en) | Model training method, device and storage medium | |
| CN116761020A (en) | Video processing method, device, equipment and medium | |
| CN114491171A (en) | Data processing method, system, medium and electronic device based on industrial Internet of things | |
| CN114490547A (en) | Data compression method, device, equipment and medium | |
| CN115330140A (en) | Building risk prediction method based on data mining and prediction system thereof | |
| CN114912111A (en) | Cloud host virus detection method, device and system | |
| CN113556575A (en) | Method, apparatus, device, medium and product for compressing data | |
| CN115828977B (en) | Industrial Internet label coding method, device, medium and electronic equipment | |
| CN110647519B (en) | Method and device for predicting missing attribute value in test sample | |
| CN116341689B (en) | Training method and device for machine learning model, electronic equipment and storage medium | |
| CN114302425B (en) | Equipment network distribution method and device, storage medium and electronic equipment | |
| CN117708755B (en) | Ecological environment-based data processing method and device | |
| CN115482422B (en) | Training method of deep learning model, image processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |