CN110809037B - An IoT dermoscopic system based on deep multi-features - Google Patents

Abstract

The invention discloses an Internet of Things dermoscopy system based on deep multi-features, including a handheld dermoscopy terminal, a dermoscopy server, an online expert system and a user application module. The handheld dermoscopy terminal realizes wireless communication through a built-in 4G IoT card , so that the use of the equipment is not limited by the region and location, and the remote high-performance server is called for computer-aided medical diagnosis; the skin disease model based on the deep multi-features in the dermoscopy server realizes a more versatile by combining the deep features and the multi-features selected manually. skin disease detection method. The invention can solve the problems that skin patients are inconvenient to go to the hospital for examination and cannot be equipped with high-performance dermoscopic equipment in remote areas where the hospital is located, improve the effectiveness of computer-aided medical diagnosis, and integrate medical resources in different regions.

Description

An IoT dermoscopic system based on deep multi-features

technical field

The invention relates to the technical field of medical image processing, in particular to an Internet of Things dermoscopy system based on deep multi-features.

Background technique

With the development of deep learning in the field of medical image processing, hospitals and scientific research institutions have gradually begun to use computer technology to assist medical diagnosis. In the diagnosis of skin diseases, there have been a lot of researches on intelligent recognition based on dermoscopic images. Due to the limitation of dermoscopy equipment, some remote clinics cannot be equipped with dermoscopy equipment for dermoscopy, and it is difficult for patients with limited mobility to go to the hospital for examination, which limits the application of dermoscopy technology.

In view of the problem of dermoscopic equipment, there are two types of mainstream equipment at this stage. One is small hand-held dermatoscope, which is small in size and easy to carry. ; The other type is a large-scale medical dermoscope, which includes a computer, a main console, a handheld collector, a display and other modules, which are not convenient for large-scale movement and can only be used in professional hospitals. Therefore, there is a need to propose a portable dermoscopy system that can perform computer-aided diagnosis through a remote high-performance server.

In addition, facing the problem of skin lesion classification, the current classification is mainly based on two categories of features: multivariate features selected by hand and deep features extracted by deep learning. The hand-selected multi-features are extracted based on a large number of experiments and research summaries of experts. In the case of a large amount of prior knowledge, appropriate features can be selected to achieve the classification effect; the deep features extracted by deep learning are obtained through The deep network is trained with a large amount of data, and the most suitable features are filtered out after a large number of operations, so it is suitable for scenarios with a large amount of data. For complex practical application scenarios, a single type of feature cannot deal with many situations, and a method with higher compatibility needs to be proposed for identification and classification.

In summary, IoT technology can be used to allow handheld dermoscopy devices to access and use a remote high-performance server to assist in diagnosis, and at the same time, a more general dermoscopy detection technology can be provided in the server by combining manual features and deep features.

SUMMARY OF THE INVENTION

The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and proposes an IoT dermoscopy system based on deep multi-features, which connects a handheld dermoscopy terminal and a dermoscopy server through the IoT technology, and is both portable and computer-aided diagnosis. The advantage is that it solves the problem that the single feature classification algorithm has a limited application range and cannot adapt to most production environments, so that the computer-aided diagnosis technology is more widely used.

In order to achieve the above purpose, the technical solution provided by the present invention is: a kind of Internet of things dermoscopy system based on deep multi-features, including:

The hand-held dermoscopic terminal is used to collect dermoscopic images of skin patients for medical staff. It is connected to the dermoscopic server and online expert system through the Internet of Things communication module, which can provide medical staff with confirmation of medical treatment, dermoscopic image collection, and dermatoscopy data. Operations of uploading, online expert calling, and viewing diagnosis results;

Dermoscopy server, built by a server with a memory of not less than 32GB, a video memory of not less than 12GB, and a number of CPU cores of not less than 8 cores, used to store and manage the dermoscopy images collected by the handheld dermoscopy terminal data, the dermoscopy server can preprocess the dermoscopy image data, perform skin disease detection on the preprocessed dermoscopy image data, or perform skin disease model training on the preprocessed dermoscopy image data, and prepare the dermoscopy image data for hand-held The dermoscopy terminal provides an API interface for calling;

Online expert system for managing dermatology expert information, responding to online expert calls initiated by handheld dermoscopy terminals, providing a web interface for managing diagnostic procedures, collecting and managing local dermoscopy databases;

User application module for patients to manage personal medical records, make appointments for on-site examinations, query hospital information and view diagnosis results.

Further, the handheld dermoscopy terminal includes a dermoscopy image acquisition unit, a system application module, an Internet of Things communication module, and a touch screen, wherein:

The dermoscopic image acquisition unit is used for dermoscopic image acquisition, and is composed of an optical magnifying element, a camera, and a digital image acquisition card. The optical magnification element is used to amplify the skin surface, and the camera and the digital image acquisition card are used to collect digital images. forming a dermoscopic image;

The system application module is composed of an operating system and application software, and the operating system is used to provide the calling interface of the underlying hardware, including the dermoscopy image acquisition unit, the Internet of Things communication module, and the calling interface of the touch screen; the application software provides advanced applications based on the underlying hardware. It has the ability to control the process of dermoscopy image acquisition, use the Internet of Things communication module to call the API interface module of the dermoscopy server, and initiate online expert calls;

The Internet of Things communication module includes a 4G Internet of Things card and an Internet of Things communication module. The 4G Internet of Things card enables the networking function, connects to the wireless network through the Internet of Things communication module, and uses the wireless private network technology to connect to the dermoscopy server and online experts. System, the use of wireless private network technology can ensure that data transmission will not be illegally intercepted and stolen, and at the same time ensure the stability of transmission;

The touch screen is responsible for providing instruction input and display, and medical personnel can perform operations such as confirmation of medical treatment, collection of dermoscopy images, upload of dermoscopy data, online expert calls, and viewing of diagnosis results through the touch screen according to the prompt information.

Further, the dermoscopy server includes a skin disease model based on deep multi-features, an image preprocessing module, a skin disease detection module, a skin disease model training module, an API interface module, and a database management module, wherein:

The skin disease model based on deep multi-features is built by python language and is a model based on deep neural network combined with multi-features selected manually. The skin disease model includes two parts: model structure and model parameters, and the model structure includes two inputs. , the first is a three-channel image with a fixed size, and the second is a multivariate feature selected manually, including LBP map features and grayscale co-occurrence matrix features; based on a deep neural network, the skin disease model can mine deep features through training, combined with The manually selected multivariate features make the model parameters converge quickly, and obtain usable prediction results when there are few dermoscopic images, and obtain accurate prediction results when there are many dermoscopic images. After sub-convolution, activation, and pooling, the depth features are obtained, and the depth features are fused with the LBP map features in the input hand-selected multi-feature features, and then three convolution, activation, and pooling are performed. The gray-scale co-occurrence matrix features are fused to obtain deep multivariate features, and the deep multivariate features are classified by the SVM classifier to obtain the output of the skin disease model based on the deep multivariate features; model parameters need to pass the skin disease model training module. After training, it is used for skin disease detection by the skin disease detection module;

The image preprocessing module is responsible for preprocessing the dermoscopy images collected by the handheld dermoscopy terminal. Since the dermoscopy images collected by the handheld dermoscopy terminal are not suitable for direct skin disease detection, they need to be normalized to be suitable for computer processing. The fixed size three-channel image is extracted, and its LBP map features and gray level co-occurrence matrix features are extracted to form hand-selected multivariate features;

The skin disease detection module is responsible for inputting the fixed-size three-channel image preprocessed by the image preprocessing module and the manually selected multivariate features into the skin disease model based on deep The output of the characteristic skin disease model is used as the prediction result, and the prediction result is saved;

The skin disease model training module is responsible for inputting the fixed-size three-channel image and the manually selected multivariate features obtained after preprocessing by the image preprocessing module into the skin disease model based on deep multivariate features, and performing skin disease model training. Model parameters of the skin disease model with deep multi-featured features, and save the model parameters;

The API interface module is responsible for managing and providing API interfaces, and provides services to the handheld dermoscopy terminal and the online expert system in the form of Web Service. The handheld dermoscopy terminal and the online expert system can upload dermoscopy data and activate skin through the API interface module. Disease detection and start skin disease model training, the API interface module can also record the number and time of each user calling each API interface;

The database management module is used to store and manage the operation records of the image preprocessing module, store and manage the operation records of the skin disease detection module, store and manage the operation records of the skin disease model training module, and store and manage the data collected by the handheld dermoscopy terminal. It stores and manages the fixed-size three-channel images and hand-selected multivariate features after preprocessing by the image preprocessing module.

Further, the online expert system includes an expert management module, a diagnosis process management module, and a skin disease database, wherein:

The expert management module manages the dermatology expert information database, realizes the dermatology expert real-name system, records the diagnosis results of the dermatology expert, receives the online expert call initiated by the handheld dermatoscope terminal, and pushes the online expert call to the dermatology expert information database. The dermatologist, the dermatologist responds to the online expert call in the form of online order grabbing, and records the number of times each dermatologist responds to the online expert call as a reference for the performance of the dermatologist;

The diagnostic process management module formulates the process of dermatological diagnosis. First, the skin patient makes an online appointment or goes to the hospital for skin disease detection to generate a visit number. During the visit, the skin patient performs skin disease detection according to the visit number. After collecting the dermoscopy image, initiate an online expert call and connect to the dermoscopy server to upload the dermoscopy data. If there is a response from a dermatology expert, make an online diagnosis based on the prediction results of the skin disease detection module of the dermoscopy server, and obtain the diagnosis result; If there is no response from the dermatologist within the specified time, the prediction result of the dermatology detection module will be used as the initial diagnosis result. After the dermatologist in the dermatologist information database has confirmed the initial diagnosis result, the diagnosis result corresponding to the visit number will be updated, and the diagnosis result will be updated. Push to the user application module;

The dermatological database is used to record the consultation number, the dermatoscope image and the association between the dermatologist and the diagnosis result, and the dermoscopic image and the diagnosis result are regularly uploaded to the dermoscopic data, uploaded to the dermoscopic server, and passed through the dermoscopic server. The database management module updates the dermoscopy images, and uses the API interface module of the dermoscopy server to initiate dermatology model training.

Further, the user application module is a mobile terminal application program, including personal medical record management, appointment visit inspection, hospital information query, and diagnosis result push, wherein:

The personal medical record management is used to view the diagnostic results of historical skin disease detection for skin patients, and to provide historical reference opinions for skin disease experts;

The appointment visit inspection is used to reserve a medical staff for home inspection of skin diseases, which is beneficial for skin patients who are inconvenient to move or go to the hospital in person to contact the medical staff for timely medical treatment;

The hospital information query provides skin patients with the guidance of nearby hospitals, the function of checking the relevant equipment in the hospital, and checking the information of dermatologists;

The diagnosis result push is responsible for pushing the diagnosis results of skin disease detection and related treatment opinions to the skin patient.

Compared with the prior art, the present invention has the following advantages and beneficial effects:

1. The use of Internet of Things technology ensures the portability of terminal equipment while using high-performance servers for computing. The use of wireless private network technology can ensure that data transmission will not be illegally intercepted and stolen, while ensuring the stability of transmission.

2. Use a dedicated dermoscopy server for skin disease detection and skin disease model training, and provide services to handheld dermoscopy terminals and online expert systems in the form of WebService, which can quickly complete training and detection without affecting user experience. Providing timely feedback can effectively reduce the computing pressure of the terminal device and improve the user experience of the handheld dermoscopy terminal.

3. The skin disease model used in the present invention is a model based on a deep neural network and combined with manually selected multiple features. The skin disease model can mine deep features through training, combined with the manually selected multiple features, so that the model parameters quickly converge. When there are few dermoscopic images, usable prediction results can be obtained, and when there are many dermoscopic images, accurate prediction results can be obtained, which can be applied to most production environments without changing the model, which facilitates the rapid deployment of the system and effectively reduces construction costs.

Description of drawings

FIG. 1 is a schematic diagram of the relationship of each module of the Internet of Things dermoscopy system in the embodiment.

FIG. 2 is a schematic diagram of the calling process of each module of the Internet of Things dermoscopy system in the embodiment.

FIG. 3 is a schematic diagram of a skin disease model based on deep multivariate features.

Figure 4 is a flow chart of a skin disease diagnosis.

Detailed ways

The present invention will be further described below in conjunction with specific embodiments.

As shown in Figures 1 and 2, the IoT dermoscopy system based on deep multi-features provided by this embodiment is an overall solution including hardware and software. The hospital headquarters deploys a device with a memory of not less than 32 GB and a lot of video memory. A high-performance server with less than 12GB and no less than 8 CPU cores. Each hospital branch deploys an application server, and configures several handheld dermoscopy terminals according to the scale of the hospital. The network calls the API interface of the dermoscopy server and the online expert system in the form of WebService, and the dermoscopy server performs skin disease detection by calling the Python program.

Specifically, the IoT dermoscopy system includes:

The hand-held dermoscopic terminal is used to collect dermoscopic images of skin patients for medical staff. It is connected to the dermoscopic server and online expert system through the Internet of Things communication module, which can provide medical staff with confirmation of medical treatment, dermoscopic image collection, and dermatoscopy data. Operations of uploading, online expert calling, and viewing diagnosis results;

Dermoscopy server, a server with 32GB memory, 12GB video memory, 8-core CPU, 256GB system hard disk, and 2TB data hard disk is configured in the hospital headquarters, which is used to store and manage the dermoscopy image data obtained by the handheld dermoscopy terminal for dermoscopy image collection. , the dermoscopy server can preprocess the dermoscopy image data, perform skin disease detection on the preprocessed dermoscopy image data, or perform dermatology model training on the preprocessed dermoscopy image data, and provide the hand-held dermoscopy terminal Provide the API interface for calling;

Online expert system, a server with 32GB memory, 4-core CPU, 256GB system hard disk, and 1TB data hard disk is configured in each hospital branch, which is used to manage the information of dermatologists, respond to the online expert call initiated by the handheld dermatoscope terminal, and provide web Interface for managing diagnostic procedures, collecting and managing local dermoscopy databases;

The user application module provides services to skin patients in the form of Android applications, which are used for patients to manage personal medical records, make appointments for on-site examinations, query hospital information and view diagnosis results.

The handheld dermoscopy terminal includes a dermoscopy image acquisition unit, a system application module, an Internet of Things communication module, and a touch screen, wherein:

The dermoscopic image acquisition unit is composed of an optical magnifying element, a 2-megapixel CMOS camera, and an electronic acquisition card. The skin surface is enlarged by the optical magnifying element and collected as a digital image by the camera and the digital image acquisition card to form a dermoscopy image. ;

The system application module is composed of an operating system and application software, and the operating system of the handheld dermoscopy terminal adopts the Linux kernel, which is used to provide the calling interface of the underlying hardware, including the calling interface of the dermoscopy image acquisition unit, the Internet of Things communication module, and the touch screen; The application software provides advanced application capabilities based on the underlying hardware, including the process control of dermoscopy image acquisition, the use of the Internet of Things communication module to call the API interface module of the dermoscopy server, and the initiation of online expert calls;

The Internet of Things communication module includes a 4G Internet of Things card and an Internet of Things communication module. The Internet of Things card adopts the China Mobile Internet of Things card, and a monthly package of 6G traffic is opened. The traffic of the Internet of Things card in the same hospital branch is shared, and the handheld dermoscope terminal passes through. The IoT communication module is connected to the wireless network, and is connected to the dermoscopy server and the online expert system by using the wireless private network technology. Using the wireless private network technology can ensure that the data transmission will not be illegally intercepted and stolen, and at the same time ensure the stability of the transmission;

The touch screen is responsible for providing instruction input and display, and medical personnel can perform operations such as confirmation of medical treatment, collection of dermoscopy images, upload of dermoscopy data, online expert calls, and viewing of diagnosis results through the touch screen according to the prompt information.

The dermoscopy detection server module includes a skin disease model based on deep multi-features, an image preprocessing module, a skin disease detection module, a skin disease model training module, an API interface module, and a database management module, wherein:

The skin disease model based on deep multi-features is built by python language. It is a model based on deep neural network and combined with multi-features selected manually. The skin disease model includes two parts: model structure and model parameters. The model structure is shown in Figure 3. It contains two inputs, the first is a fixed-size three-channel image, and the second is a multivariate feature selected by hand, including LBP map features and gray-scale co-occurrence matrix features; based on deep neural networks, the skin disease model can pass The training and mining of deep features, combined with the multi-features selected by hand, make the model parameters converge quickly, obtain usable prediction results when there are few dermoscopic images, and obtain accurate prediction results when there are many dermoscopic images; the skin disease model has a fixed size for the input. After the three-channel image is convolved, activated, and pooled four times, the depth feature is obtained, and the depth feature is fused with the LBP map feature in the input hand-selected multi-feature feature, and then three convolution, activation, and pooling are performed. It is fused with the gray-level co-occurrence matrix features in the manually selected multi-features to obtain deep multi-features, and the deep multi-features are classified by the SVM classifier to obtain the output of the skin disease model based on the deep multi-features; the model parameters need to pass the skin disease model. The training module is obtained by training the skin disease model, which is used for skin disease detection by the skin disease detection module;

The image preprocessing module is responsible for preprocessing the dermoscopy images collected by the handheld dermoscopy terminal. Since the dermoscopy images collected by the handheld dermoscopy terminal are not suitable for direct use in skin disease detection, they need to be normalized to a length of 150 pixels. , a three-channel image with a width of 150 pixels, and extract its LBP map features and gray level co-occurrence matrix features to form hand-selected multivariate features;

The skin disease detection module is responsible for inputting the fixed-size three-channel image preprocessed by the image preprocessing module and the manually selected multivariate features into the skin disease model based on deep The output of the characteristic skin disease model is used as the prediction result, and the prediction result is saved;

The skin disease model training module is responsible for inputting the fixed-size three-channel image and the manually selected multivariate features obtained after preprocessing by the image preprocessing module into the skin disease model based on deep multivariate features, and performing skin disease model training. Model parameters of the skin disease model with deep multi-featured features, and save the model parameters;

The API interface module is responsible for managing and providing API interfaces, and provides services to the handheld dermoscopy terminal and the online expert system in the form of Web Service. The handheld dermoscopy terminal and the online expert system can upload dermoscopy data and activate skin through the API interface module. Disease detection and start skin disease model training, the API interface module can also record the number and time of each user calling each API interface;

The database management module uses the Oracle database to store and maintain the operation records of the image preprocessing module, the operation records of the skin disease detection module, and the operation records of the skin disease model training module; The fixed-size three-channel image and the manually selected multivariate features obtained after preprocessing by the image preprocessing module are stored in the data hard disk, and the index information is stored and managed using the Oracle database.

The online expert module includes an expert management module, a diagnosis process management module, and a skin disease database, wherein:

The expert management module manages the dermatology expert information database, realizes the dermatology expert real-name system, records the diagnosis results of the dermatology expert, receives the online expert call initiated by the handheld dermatoscope terminal, and pushes the online expert call to the dermatology expert information database. The dermatologist, the dermatologist responds to the online expert call in the form of online order grabbing, and records the number of times each dermatologist responds to the online expert call as a reference for the performance of the dermatologist;

The diagnostic process management module formulates the process of dermatological diagnosis, as shown in Figure 4, firstly, the skin patient generates a visit number when making an online reservation through the user application module or when he goes to the hospital for skin disease detection, and the skin patient is based on the visit number when visiting a doctor. For skin disease detection, the medical staff collects dermoscopic images through the handheld dermoscopic terminal, initiates an online expert call to the online expert system, and connects to the dermoscopic server to upload dermoscopic data. If there is no response from a dermatologist in a short period of time, the prediction result of the dermatology detection module will be used as the initial diagnosis result, and the dermatologist in the dermatologist information database will wait for the initial diagnosis. After the result is confirmed, update the diagnosis result corresponding to the visit number, and push the diagnosis result to the user application module;

The dermatology database uses the Oracle database to record the consultation number, dermoscopic images, and the relationship between dermatologists and diagnosis results. The system defaults to upload dermoscopic images and diagnosis results to the dermatoscope server every month. The administrator of the department can manually upload the dermoscopy data to the dermoscopy server at any time. The dermatologists at the hospital headquarters verify the reliability of the newly uploaded data through the database management module of the dermoscopy server every month, and update the dermoscopy images. Use the API interface module of the dermoscopy server to start the training of the skin disease model.

The user application module is an Android application program written in Java language, and the skin patient uses the user application module through a mobile phone, including personal medical record management, appointment visit inspection, hospital information query, and diagnosis result push, wherein:

The personal medical record management is used to view the diagnostic results of historical skin disease detection for skin patients, and to provide skin disease experts with historical reference opinions when skin patients perform skin disease examinations;

The said appointment visit is used to make an appointment for medical personnel to come to the hospital for skin disease detection. The nearby hospital will receive and arrange medical personnel to feed back the second confirmation appointment to the skin patient. personnel to seek medical treatment in a timely manner;

The hospital information query provides skin patients with the guidance of nearby hospitals, the function of checking the relevant equipment in the hospital, and checking the information of dermatologists;

The diagnosis result push is responsible for pushing the diagnosis results of skin disease detection and related treatment opinions to the skin patient.

The above-mentioned embodiments are only preferred embodiments of the present invention, and are not intended to limit the scope of implementation of the present invention. Therefore, any changes made according to the shape and principle of the present invention should be included within the protection scope of the present invention.

Claims (4)

1. a kind of Internet of things dermoscopy system based on deep multi-feature, is characterized in that, comprises: The hand-held dermoscopic terminal is used to collect dermoscopic images of skin patients for medical staff. It is connected to the dermoscopic server and online expert system through the Internet of Things communication module, which can provide medical staff with confirmation of medical treatment, dermoscopic image collection, and dermatoscopy data. Operations of uploading, online expert calling, and viewing diagnosis results; Dermoscopy server, built by a server with a memory of not less than 32GB, a video memory of not less than 12GB, and a number of CPU cores of not less than 8 cores, used to store and manage the dermoscopy images collected by the handheld dermoscopy terminal data, the dermoscopy server can preprocess the dermoscopy image data, perform skin disease detection on the preprocessed dermoscopy image data, or perform skin disease model training on the preprocessed dermoscopy image data, and prepare the dermoscopy image data for hand-held The dermoscopy terminal provides an API interface for calling; Online expert system for managing dermatology expert information, responding to online expert calls initiated by handheld dermoscopy terminals, providing a web interface for managing diagnostic procedures, collecting and managing local dermoscopy databases; User application module for patients to manage personal medical records, make appointments for on-site examinations, query hospital information and view diagnostic results; The dermoscopy server includes a skin disease model based on deep multi-features, an image preprocessing module, a skin disease detection module, a skin disease model training module, an API interface module, and a database management module, wherein: The skin disease model based on deep multi-features is built by python language and is a model based on deep neural network combined with multi-features selected manually. The skin disease model includes two parts: model structure and model parameters, and the model structure includes two inputs. , the first is a three-channel image with a fixed size, and the second is a multivariate feature selected manually, including LBP map features and grayscale co-occurrence matrix features; based on a deep neural network, the skin disease model can mine deep features through training, combined with The manually selected multivariate features make the model parameters converge quickly, and obtain usable prediction results when there are few dermoscopic images, and obtain accurate prediction results when there are many dermoscopic images. After sub-convolution, activation, and pooling, the depth features are obtained, and the depth features are fused with the LBP map features in the input hand-selected multi-feature features, and then three convolution, activation, and pooling are performed. The gray-scale co-occurrence matrix features are fused to obtain deep multivariate features, and the deep multivariate features are classified by the SVM classifier to obtain the output of the skin disease model based on the deep multivariate features; model parameters need to pass the skin disease model training module. After training, it is used for skin disease detection by the skin disease detection module; The image preprocessing module is responsible for preprocessing the dermoscopy images collected by the handheld dermoscopy terminal. Since the dermoscopy images collected by the handheld dermoscopy terminal are not suitable for direct skin disease detection, they need to be normalized to be suitable for computer processing. The fixed size three-channel image is extracted, and its LBP map features and gray level co-occurrence matrix features are extracted to form hand-selected multivariate features; The skin disease detection module is responsible for inputting the fixed-size three-channel image preprocessed by the image preprocessing module and the manually selected multivariate features into the skin disease model based on deep The output of the characteristic skin disease model is used as the prediction result, and the prediction result is saved; The skin disease model training module is responsible for inputting the fixed-size three-channel image and the manually selected multivariate features obtained after preprocessing by the image preprocessing module into the skin disease model based on deep multivariate features, and performing skin disease model training. Model parameters of the skin disease model with deep multi-featured features, and save the model parameters; The API interface module is responsible for managing and providing API interfaces, and provides services to the handheld dermoscopy terminal and the online expert system in the form of Web Service. The handheld dermoscopy terminal and the online expert system can upload dermoscopy data and activate skin through the API interface module. Disease detection and start skin disease model training, the API interface module can also record the number and time of each user calling each API interface; The database management module is used to store and manage the operation records of the image preprocessing module, store and manage the operation records of the skin disease detection module, store and manage the operation records of the skin disease model training module, and store and manage the data collected by the handheld dermoscopy terminal. It stores and manages the fixed-size three-channel images and hand-selected multivariate features after preprocessing by the image preprocessing module. 2. The Internet of Things dermoscopy system according to claim 1, wherein the handheld dermoscopy terminal comprises a dermoscopy image acquisition unit, a system application module, an internet of things communication module, and a touch screen, in: The dermoscopic image acquisition unit is used for dermoscopic image acquisition, and is composed of an optical magnifying element, a camera, and a digital image acquisition card. The optical magnification element is used to amplify the skin surface, and the camera and the digital image acquisition card are used to collect digital images. forming a dermoscopic image; The system application module is composed of an operating system and application software, and the operating system is used to provide the calling interface of the underlying hardware, including the dermoscopy image acquisition unit, the Internet of Things communication module, and the calling interface of the touch screen; the application software provides advanced applications based on the underlying hardware. It has the ability to control the process of dermoscopy image acquisition, use the Internet of Things communication module to call the API interface module of the dermoscopy server, and initiate online expert calls; The Internet of Things communication module includes a 4G Internet of Things card and an Internet of Things communication module. The 4G Internet of Things card enables the networking function, connects to the wireless network through the Internet of Things communication module, and uses the wireless private network technology to connect to the dermoscopy server and online experts. System, the use of wireless private network technology can ensure that data transmission will not be illegally intercepted and stolen, and at the same time ensure the stability of transmission; The touch screen is responsible for providing instruction input and display, and medical personnel can perform operations such as confirmation of medical treatment, collection of dermoscopy images, upload of dermoscopy data, online expert calls, and viewing of diagnosis results through the touch screen according to the prompt information. 3. a kind of internet of things dermoscopy system based on deep multi-features according to claim 1, is characterized in that: described online expert system comprises expert management module, diagnosis process management module, skin disease database, wherein: The expert management module manages the dermatology expert information database, realizes the dermatology expert real-name system, records the diagnosis results of the dermatology expert, receives the online expert call initiated by the handheld dermatoscope terminal, and pushes the online expert call to the dermatology expert information database. The dermatologist, the dermatologist responds to the online expert call in the form of online order grabbing, and records the number of times each dermatologist responds to the online expert call as a reference for the performance of the dermatologist; The diagnostic process management module formulates the process of dermatological diagnosis. First, the skin patient makes an online appointment or goes to the hospital for skin disease detection to generate a visit number. During the visit, the skin patient performs skin disease detection according to the visit number. After collecting the dermoscopy image, initiate an online expert call and connect to the dermoscopy server to upload the dermoscopy data. If there is a response from a dermatology expert, make an online diagnosis based on the prediction results of the skin disease detection module of the dermoscopy server, and obtain the diagnosis result; If there is no response from the dermatologist within the specified time, the prediction result of the dermatology detection module will be used as the initial diagnosis result. After the dermatologist in the dermatologist information database has confirmed the initial diagnosis result, the diagnosis result corresponding to the visit number will be updated, and the diagnosis result will be updated. Push to the user application module; The dermatological database is used to record the consultation number, the dermatoscope image and the association between the dermatologist and the diagnosis result, and the dermoscopic image and the diagnosis result are regularly uploaded to the dermoscopic data, uploaded to the dermoscopic server, and passed through the dermoscopic server. The database management module updates the dermoscopy images, and uses the API interface module of the dermoscopy server to initiate dermatology model training. 4. a kind of internet of things dermoscopy system based on deep multi-features according to claim 1, is characterized in that: described user application module is mobile phone terminal application program, comprises personal medical record management, appointment visit inspection, hospital information inquiry, Diagnostic result push, including: The personal medical record management is used to view the diagnostic results of historical skin disease detection for skin patients, and to provide historical reference opinions for skin disease experts; The appointment visit inspection is used to reserve a medical staff for home inspection of skin diseases, which is beneficial for skin patients who are inconvenient to move or go to the hospital in person to contact the medical staff for timely medical treatment; The hospital information query provides skin patients with the guidance of nearby hospitals, the function of checking the relevant equipment in the hospital, and checking the information of dermatologists; The diagnosis result push is responsible for pushing the diagnosis results of skin disease detection and related treatment opinions to the skin patient.

Images