3.1. System Architecture
The system overview of the integrated monitoring platform proposed in this research is shown in
Figure 3.
In this system, sensor devices attached to industrial machines are used to collect data on the industrial machines being manufactured. The collected data are sent to the gateway using LoRa communication, a type of Low Power Wide Area (LPWA), after edge processing at the sensor device. The gateway sends the packets received from the sensor device to the web server by Hypertext Transfer Protocol (HTTP) POST. The web server visualises the POSTed data in the form of a graph, linking it to the sensor device information. The operator can check the status of the equipment from the graph, and if any abnormality is found, the operator can provide feedback by maintaining the machine or changing the production conditions.
Here, the data sent by the sensor device to the gateway are not the data as measured, but data that have been processed by edge processing. This edge processing calculates features for estimating the state of industrial machines from the data obtained from the sensors. To determine the content of edge processing, the measured data themselves must be analysed once. It is not suitable for the sensor device to send all the measured data to the gateway due to the communication capacity limitation of LoRa communication. Therefore, it is necessary to reduce the amount of data by adding some data processing to the measured data as edge processing in the sensor device and then transmitting the data. Based on the above, the operational phase of the integrated platform is defined in this system by dividing it into three phases: sensor data collection phase, sensor data analysis phase and industrial machine monitoring phase.
In the sensor data collection phase, external sensor devices are used to collect data. This is because SMEs in the manufacturing industry tend to have legacy equipment, and legacy industrial machines without network functionality are still in use. The amount of data acquired per unit of time depends on the sensor used and the sampling rate. As these different data cannot be handled in one format, three data formats are defined for this system: Comma Separated Values (CSV) files, sensor data files and radio packets.
A CSV file is a format in which data are recorded in comma-separated lines, where one line is the data for one measurement. Because a single line can have multiple columns separated by commas, multiple sensor data can be stored together in a single file.
A sensor data file is a file in which the data from a single measurement, such as sound data, image data or video data, are written in one file according to the respective file protocol. It is not possible to include data from several sensors in one file, such as a CSV file.
Wireless packets are packets sent by the sensor device to the gateway via LoRa communication. In the LoRa communication used in this system, the payload of a packet transmitted at a time should be limited to about 50 [B] to ensure stable communication. For the transmission frequency, the transmission interval should be at least one second, because high-frequency transmission from sensor devices will affect the communication quality in an environment with multiple sensor devices. Due to the above requirements, it is difficult for wireless packets to continue transmitting the data measured by the sensors as they are. Therefore, wireless packets are used to transmit the results of processing measured data. However, if the data acquisition intervals using sensors are spaced, e.g., once every minute or once every ten minutes, it is possible to transmit the measurement data as they are.
Of these data formats, CSV files and sensor data files are stored on the Secure Digital (SD) card installed in the sensor device. Therefore, to retrieve the data, the operator needs to take out the SD card and upload it to the system. This is a procedure that places a burden on the operator, but since LoRa communication is used in the industrial machine monitoring phase, this is not an operational problem.
The sensor data analysis phase uses the data obtained by the sensor data collection phase to analyse whether any abnormalities or signs of failure appear in industrial machines. Here, not only the simple size of the sensor data is taken into account, but also the cycles in the manufacturing process. A cycle is the unit of measurement from the beginning to the end of a process when the same product is manufactured or processed repeatedly. By analysing the data in consideration of cycles, it is possible to analyse differences in the data from the point of view of the manufacturing process.
The industrial machine monitoring phase collects the valid feature values obtained from the sensor data analysis phase from the sensor devices via LoRa communication and displays the transition in the web application. The feature values of the sensors attached to industrial machines are automatically collected by LPWA communication, and the values can be immediately checked.
3.2. Integrated Monitoring Platform Web Application
In this research, Django 4.2 [
12] is used as the framework for the web application. It is a framework that uses Python 3.12 as its programming language and is characterised by its database-based development. It allows for the easy connection of databases and web pages by defining a data model. It comes standard with a function to manage user accounts, and it is possible to control who is accessing the site by using password authentication and limiting the range of pages that can be accessed by that user for security purposes. The reason for using a Django for the integrated monitoring platform proposed in this research is that the operation of the application does not depend on the terminal environment or operating system. Since this system is expected to be used by workers in SMEs in the manufacturing industry in factories and offices, a Web application that can be accessed and used from a variety of terminals is suitable. Another advantage is that there is no need to install the application on the terminal, and the system can be used simply by accessing it from a Web browser, eliminating the need to build an environment on the terminal.
Besides, a web server is built to run the web application. The web server consists of nginx 1.26.1 [
13], a web server application, Django, a web application framework, and MySQL 8.4 [
14], a relational database management system. As Django is a web application and does not have a function to process HTTP requests at high speed on a web server, nginx, which is open source and is characterised by fast request processing, is used as the web server application. In addition, MySQL is used to build a relational database based on Django’s data model. MySQL supports robust transaction features, offering a strong mechanism for maintaining data consistency. It is also highly compatible with various operating systems, making it excellent in terms of interoperability and compatibility. Additionally, since MySQL is open-source, it can be operated at a lower cost compared to other database management systems. This makes it accessible for SMEs, which can implement it without the need for significant investment.
In the integrated monitoring platform proposed in this research, external sensor devices are attached to industrial machines to collect data. In this case, it is assumed that several industrial machines that manufacture the same products are installed in a factory, and sensor devices with the same configuration are installed on each of them. Therefore, it is necessary to control which sensor device is attached to which industrial machine and which part of the industrial machine the sensor is attached to. The Entity Relationship (ER) diagram of the database designed to realise these requirements is shown in
Figure 4.
The sensor_base_info manages the sensors used in a sensor device. By managing these on the platform, data can be compared between different sensors. The operator registers the sensor model number and physical quantity as information on the sensors used in the sensor device. This makes it possible to obtain data from a device equipped with a certain sensor or to obtain information on what kind of analysis has been carried out with a similar sensor in the past.
The device_base_info manages the configuration of measurement devices with sensors installed on industrial machines and consists of the device model number and sensor installation information. This sensor installation information consists of a combination of the basic sensor information and the sensor number for the device. The reason for the sensor number is to distinguish between two or more sensors that are mounted on a device. The operator registers sensor installation information as the number of sensors installed in the device. By selecting the basic device information, a list of devices with that device configuration can be displayed. This enables data analysis focused on the data measured in devices with similar configurations.
The machine manages the industrial machines that are actually installed in the factory. The machine information consists of machine identification (ID), machine name, installation location and category. The machine ID is an identifier to uniquely identify a machine in the system. The machine name is set to the model number of the machine or an identification name in the company. The installation location is information that manages the installation location of industrial machines and sets the information on which floor and location in which factory the machine is installed. By selecting a machine, the sensors installed on the machine can be listed. This makes it easy to compare and analyse multiple sensors installed on a machine.
The device manages the measurement devices actually installed based on the registered basic device information. The device information consists of a device ID, device name, basic device information, additional information and gateway. The device ID is an identifier to uniquely identify a device in the system. The device name is a name for workers to identify devices associated with the same basic device information. In the device basic information, the device configuration of the device to be registered is selected. In the additional information, points that are specific to the device and that should be shared can be entered.
The sensor manages the sensors possessed by the actually installed devices. The sensor information consists of the sensor mounting information and the device combination. The sensor information consists of the sensor mounting information and the combination of devices. This sensor information is used to manage the locations where the sensors are installed.
The mount manages where on the industrial machine the sensors of the measurement device that are actually installed are located. The installation information consists of the target industrial machine, the sensors installed, the installation point indicating where the sensor was installed on the machine, the installation start date and time, and the installation end date and time. This makes it easy to manage where the sensors of each device are installed in an environment where several devices of the same configuration are installed. The installation information also includes the installation end date and time, so that the history of past installation information can be traced back.
The device_data_csv manages the CSV files of the measurement data stored on the SD card of the measurement device. The purpose of this function is to enable easy management of the conditions under which the CSV files generated when using multiple devices of the same configuration were measured. Even when only one device is used, this function facilitates data analysis during the trial-and-error phase of data acquisition, when considering which part of an industrial machine to install a sensor.
The sensor_installation_info manages the individual data measured by sensors. The target data here are data such as sound data, image data and video data, which do not contain multiple sensor data in one file. These data cannot be registered in the database as they are because the file size is expected to be large. The system uploads sensor data files to a folder on the server and registers the path to the files in the database. This makes it possible to download sensor data later and analyse them under specific measurement conditions.
The gateways manage data aggregation gateways used during data collection. The gateways described here are devices that receive data sent from multiple measurement devices using wired or wireless communication, aggregate them and send them to the server. When sending data from the gateway to the server, the HTTP request is first sent using a TCP connection. The gateway first establishes the TCP connection by using the server’s IP address and port number. The server is waiting on the specified port, and once the TCP connection is established, it receives the HTTP request. The HTTP POST request is sent from the gateway to the server via the TCP/IP network. This request contains header information and the data to be sent. Therefore, it is necessary to interpret the payload of the received HTTP POST packet and convert it into the device’s data In this system, the format of the packet body part of the HTTP POST is not handled by the operator, but by the system implementation. In other words, the system does not support a gateway with a different HTTP POST packet body part format from the assumed gateway. This is because there is no common data format for gateways. However, it is possible to support this by additionally defining which part of the HTTP POST packet body part is the transmission data on the server side. As the format of the transmitted data depends on the implementation of the measurement device, the definition of the format of the transmitted data can be edited by the operator of this system.
In this research, a LoRa gateway, ES920GWX2 manufactured by EASEL Inc. (Kanagawa, Japan) [
15], is used as the gateway. This LoRa gateway receives data sent by the company’s LoRa module ES920LR2 and sends the data to the server via HTTP POST using an Ethernet connection or LTE communication using a SIM card. The body of the packet sent by the LoRa gateway to the server is a character string and has the following format.
where gwid is the ID of the ES920GWX2, ch is the radio channel, sf is the spreading factor, rssi is the received signal strength, panid is the network address and id is the ID of the source module. This format combines information on the source and communication with the transmitted data using the ampersand as a delimiter. Therefore, when data in this format are received by a server, the strings below the sixth ampersand can be treated as transmission data. However, as the extracted transmission data may contain multiple data, it is necessary to cast these further. Here, the first to fourth characters of the payload are interpreted as the device_id, the fifth to seventeenth characters as the average of the norm during uniform motion from the left end to the right end (left_to_right_norm_mean) and the eighteenth to thirtieth characters as the average of the norm during uniform motion from the right end to the left end (right_to_left_norm_mean).
3.3. External Attachable Sensor Device
The integrated monitoring platform proposed in this research uses sensor devices that can be attached externally to industrial machines during the sensor data collection phase and the industrial machine monitoring phase. This section describes the configuration and operation of these devices.
First, the configuration and operation of the SD card-compatible accelerometer device used in the sensor data collection phase of the proposed system are described. The appearance of the SD card-compatible accelerometer device is shown in
Figure 5.
The main element of this device is a microcontroller board called Grand Central M4 Express featuring the SAMD51 (hereafter referred to as ‘Grand Central’) [
16], manufactured by Adafruit Industries (New York, NY, USA). The reason for selecting this microcontroller board was the richness of its interfaces. Grand Central has 62 General Purpose Input Output (GPIO) ports. Among them, 16 ports can be used for analogue input and support many communication standards such as hardware Serial Peripheral Interface (SPI), Inter-Integrated Circuit (I2C) and Universal Asynchronous Receiver Transmitter (UART). This makes it possible to control a large number of sensors and acquire data from a single microcontroller board.
As an acceleration sensor, an ICM-20948 [
17] from TDK Corporation (Tokyo, Japan) is used. This sensor is a nine-axis inertial measurement unit with not only a three-axis accelerometer but also a three-axis gyro sensor and a three-axis magnetometer. As this device only measures acceleration, only the three-axis accelerometer is enabled. The measurable range of acceleration can be selected from four levels, ±2 g, ±4 g, ±8 g and ±16 g, and can be set according to the machine to be measured. The device is also equipped with a First-in First-out (FIFO) for temporarily storing sensor data, making it suitable for applications where sensor data are read out at regular intervals. In the implementation of this device, only the accelerometer is installed in the industrial machine to be measured, while the microcontroller board is installed in a remote location. Therefore, the acceleration sensor is connected to the microcontroller board by extending it with shielded conductors. I2C is used for communication with the microcontroller.
A module with a PCF8523, manufactured by Adafruit Industries (New York, NY, USA) is used as the real-time clock (RTC). The purpose of installing a RTC is to record the measurement time of the data. When a CSV file containing measurement data is saved on an SD card, the measurement time can be recorded in the file so that the data can be checked against the data of which product was being manufactured using the production record data. This makes it possible to compare data from the same product production. As with the acceleration sensor, I2C is used for communication with the microcontroller.
The power supply to the device is via an Alternating Current (AC) adapter from a 100 V AC power supply. Grand Central is equipped with a 5.5 mm/2.1 mm centre plus Direct Current (DC) connector and can accept input voltages from 6 V to 12 V. Simply connecting the power supply to the device is not enough to check whether the sensor device itself has started working properly. A reset button with a Light Emitting Diode (LED) is, therefore, used. The LED inside the button is illuminated when the operation starts using the GPIO pins of Grand Central to inform the operator that the device is working properly. Conversely, if the connection to the sensor fails or the SD card recognition fails, the LED blinks to inform the operator that there is a fault with the device. This button is connected to the reset pin of Grand Central and can be reset by pressing the button.
Here, the firmware of the accelerometer device that supports SD cards is described. The device starts up and begins operating when connected to a power source via an AC adapter. After start-up, the RTC, SD card and accelerometer are initialised in the setup function in Arduino.
The initialisation of the RTC requires the correct time to be set. The time when the firmware is written to the computer is used for this time. However, if this time is set every time the sensor device is reset, the time of the RTC reverts to the time at which the firmware was written when the device is reset. Therefore, Grand Central uses Electrically Erasable Programmable Read-Only Memory (EEPROM), which is a non-volatile memory. A flag is stored on the EEPROM to determine whether or not the device has been started up for the first time, and this flag is set after the device has been started up for the first time. This ensures that the RTC time is set only at the first start-up when the firmware is written.
In the initialisation of SD, the file number in the SD card is acquired. The file number is the serial number of the CSV file created in the sensor device, and the file name of each CSV is set to this file number to prevent duplicate file names. This value is reset to 0 when the firmware is written and is incremented by 1 when the CSV file is saved. However, if the file number is managed as a variable in the firmware, it returns to 0 when the power is turned off or reset, so it is also stored in the non-volatile memory EEPROM to make it persistent.
In the initialisation of the accelerometer, various parameters are set. The parameters to be set are the range of acceleration measured by the accelerometer, the digital low-pass filter, the sampling rate and the FIFO mode. In this research, the acceleration measurement range is set to ±2 g, the cut-off frequency of the low-pass filter to 246 [Hz], the sampling rate to 562.5 [Hz] and the FIFO mode to stream mode, which overwrites the FIFO when it overflows. If the initialisation process for these modules fails, the LED on the reset button blinks.
The flow of operations of this accelerometer device after the initialisation process is completed is shown below.
The acceleration device repeats these operations for each measurement time per file. When creating a CSV file, the CSV file number obtained in the initialisation process is used as the file name to create the CSV file. It also acquires the time from the RTC and sets it as the file creation time. This allows the creation time of the CSV file to be checked. Write one line each for device ID, measurement start time and sampling interval as header information in the created CSV file. This allows the web application to retrieve information on which device the CSV file is from, when the measurement start time of the CSV file is, and how long the sampling interval is. Finally, ‘x, y, z’ is written as the CSV column labels. Here, these labels correspond to the x- y- and z-axes of the triaxial acceleration.
Next, data extraction from the accelerometer’s FIFO and measurement data writing to the CSV file are repeated alternately. In data extraction from the accelerometer’s FIFO, the number of data in the FIFO is first obtained. When data are extracted from the FIFO here, it is necessary to extract the data one by one, but the extracted data are not written to the CSV file one by one and are instead written to a variable array in the microcontroller memory once. The data are inserted once into a variable array in the microcontroller memory. This is because data are added to the FIFO at any time in the accelerometer while writing to the CSV file, and the FIFO may overflow due to variations in the writing time to the SD card. By writing the measurement data to the CSV file after all the data in the FIFO have been extracted once, the data can be stored in the FIFO while writing to the CSV file. The acceleration data extracted to the variable array are written to the CSV file in comma-delimited order in the x- y- and z-axis. After the measurement time per file has elapsed since the CSV file was created, the CSV file is saved. At this time, the next file number is written to the EEPROM.
Next, the configuration and operation of the LPWA communication-compatible accelerometer device used in the industrial machine monitoring phase are described. The appearance of the accelerometer device supporting LPWA communication is shown in
Figure 6.
As the main element of this device, a microcontroller board equipped with an ATSAME53J20A, manufactured by Microchip Technology Inc. (Chandler, AZ, USA) [
18] is used. In addition, an ES920LR2 [
19] from EASEL Corporation (Kanagawa, Japan) [
20] is mounted as the LoRa module that transmits the features calculated by edge processing.
The same ICM-20948 as the accelerometer device in the sensor data collection phase is used as the accelerometer. Power is supplied to the device via the micro USB port.
The firmware of the accelerometer device supporting LPWA communication is described here. The device starts up and starts operating when it is connected to a power source via a microUSB port. After start-up, the LoRa module and accelerometer are first initialised in Ardunio’s setup function. The initialisation process of the LoRa module sets the parameters required for LoRa communication. The parameters to be set include those related to the transmission method and those related to the network. The parameters related to the transmission method include the communication bandwidth, spread ratio, radio channel, existence of acks and the number of retransmissions. The communication bandwidth and radio channel number must be standardised between the LoRa modules that want to communicate. In this system, the bandwidth is set to 500 kHz, which is the maximum value. This is to shorten the communication time and reduce communication failures due to noise during transmission. Increasing the bandwidth increases the communication speed, but there is a trade-off in that the communication range is reduced. Therefore, a smaller bandwidth is necessary in an environment with many obstacles. The spread factor is another parameter that is a trade-off between communication distance and communication speed and is set here to 7, which is the default value of the LoRa module. For communication over a wider area, a higher spreading factor should be set. The radio channel is set to 4. The ack function confirms that the data sent in the communication between LoRa modules have been received by means of an ack packet. In this system, this function is enabled. This ensures that the packets sent by the sensor device reach the gateway. The retransmission frequency parameter can be set to 5 so that up to five retransmissions are performed if no acks are received. These settings reduce the number of data collection failures due to temporary poor signal conditions. The ones related to the network are PANID, OWNID and DSTID. PANID is the address of the network itself, expressed as a four-digit hexadecimal number, and must be standardised between LoRa modules that want to communicate. The OWNID is the address of the LoRa module itself, represented by a four-digit hexadecimal number, and must be unique within the same network. The DSTID is the OWNID of the destination LoRa module, expressed as a four-digit hexadecimal number. In this system, the OWNID of the LoRa module installed in the gateway is set. This allows a single gateway to aggregate data from multiple sensor devices. The initialisation process for the accelerometer is the same as for the accelerometer device in the data collection phase using CSV.
After the initialisation process in this acceleration sensor device is completed, the operating cycle of the industrial machine is detected, the feature values are calculated from the measured data and the feature values are transmitted.