research-article

Design of a Novel Neural Network Compression Method for Tiny Machine Learning

Authors:

Kunqin YinAuthors Info & Claims

EITCE '21: Proceedings of the 2021 5th International Conference on Electronic Information Technology and Computer Engineering

Pages 647 - 651

https://doi.org/10.1145/3501409.3501526

Published: 31 December 2021 Publication History

Get Access

Abstract

Traditional IoT processing data is sent from local devices to the cloud for processing, which has disadvantages such as low privacy, high latency, and low energy efficiency. These drawbacks can be effectively remedied by deploying the model for processing on devices at the "edge" of the cloud. In order to realize the data being processed directly on the cloud "edge" devices, the original machine learning algorithms need to be improved. One of the important steps is neural network compression.

In this paper, a neural network compression method for Tiny Machine Learning (TinyML) is proposed. The neural network is compressed by training a conventional neural network and then performing group convolution, pruning and asymmetric ternary quantization. In the next step, a model transformation is performed using TFLite to deploy it on embedded devices. With this novel neural network compression method, the size of the model can be greatly compressed with guaranteed accuracy. Consequently, the traditional machine learning is upgraded to TinyML, and finally a TinyML-based fall monitoring system for the elderly is built.

References

[1]

C.j.abate, Wo Mu. The future of machine learning_Daniel_Situnayake interview[J]. Microcontroller and Embedded Systems Applications, 2021, 21(6): 1--3, 6.

Google Scholar

[2]

Wang L, Zhao YH, Yang GSH, et al. A review of deep neural network model compression techniques for embedded applications[J]. Journal of Beijing Jiaotong University, 2017, 41(6): 34--41.

Google Scholar

[3]

Jin Lilei, Yang Wenzhu, Wang Sile, et al. A hybrid pruning method for compression of convolutional neural networks[J]. Small Microcomputer Systems, 2018, 39(12): 2596--2601.

Google Scholar

[4]

Su Loach. Research and application of lightweight target detection algorithm based on deep learning[D]. South China University of Technology, 2020.

Google Scholar

[5]

Ding J. Research on model compression and forward acceleration techniques for embedded deep neural networks [D]. University of Science and Technology of China, 2018.

Google Scholar

[6]

Dehghanian M, Mosadegh MSM. Ternary Weighted Function and Beurling Ternary Banach Algebra L(1)(omega) (s) [J]. Abstract and Applied Analysis, 2011.

Google Scholar

[7]

Iandola FN, Han S, Moskewicz MW, et al. Squeezenet: Alexnet-level Accuracy with 50x Fewer Parameters and <0.5mb Model Size[J], 2016.

Google Scholar

[8]

Li D, Wang X, Kong D. Deeprebirth: Accelerating Deep Neural Network Execution on Mobile Devices[J], 2017.

Google Scholar

[9]

Zhang X, Zhou X, Lin M, et al. Shufflenet: an Extremely Efficient Convolutional Neural Network for Mobile Devices[J], 2017.

Google Scholar

[10]

Stauffer C, Grimson W E L. Adaptive background mixture models for real-time tracking[C]//Proceedings. 1999 IEEE computer society conference on computer vision and pattern recognition (Cat. No PR00149). IEEE, 1999, 2: 246--252.

Google Scholar

[11]

Barnich O, Van Droogenbroeck M. ViBe: a powerful random technique to estimate the background in video sequences[C]//2009 IEEE international conference on acoustics, speech and signal processing. IEEE, 2009: 945--948.

Google Scholar

[12]

Glorot x, Bengio-Y. Understanding the difficulty of training deep feedforwardneural networks [C]. Proceedings of the Thirteenth International Conference onArtificial Interlligence and Statistics, 2010, 249--256.

Google Scholar

[13]

Chollet F. Xception: Deep Learning with Depthwise Separable Convolutions[C]//2017 {ieee} Conference on Computer Vision and Pattern Recognition ({cvpr}), [S.l.]: Institute of Electrical and Electronics Engineers Inc. 2017.

Google Scholar

[14]

Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift[C]//International conference on machine learning. PMLR, 2015: 448--456.

Google Scholar

Cited By

View all

Dantas PSabino da Silva WCordeiro LCarvalho C(2024)A comprehensive review of model compression techniques in machine learningApplied Intelligence10.1007/s10489-024-05747-w54:22(11804-11844)Online publication date: 2-Sep-2024
https://dl.acm.org/doi/10.1007/s10489-024-05747-w
Fragkou ELygnos VKatsaros D(2022)Transfer Learning for Convolutional Neural Networks in Tiny Deep Learning EnvironmentsProceedings of the 26th Pan-Hellenic Conference on Informatics10.1145/3575879.3575984(145-150)Online publication date: 25-Nov-2022
https://dl.acm.org/doi/10.1145/3575879.3575984

Index Terms

Design of a Novel Neural Network Compression Method for Tiny Machine Learning
1. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing systems and tools
2. Theory of computation
  1. Models of computation
    1. Interactive computation

Recommendations

A novel neural network parallel adder
IWANN'13: Proceedings of the 12th international conference on Artificial Neural Networks: advances in computational intelligence - Volume Part I

Addition is the most commonly used arithmetic operation and is the speed-limiting element in the core of arithmetic logic unit (ALU) in a microprocessor. Perceptron of feedforward neural networks, inspired by the threshold logic unit neuron model of ...
Novel Recurrent Neural Network for Time-Varying Problems Solving [Research Frontier]

By following the inspirational work of McCulloch and Pitts [1], lots of neural networks have been proposed, developed and studied for scientific research and engineering applications [2][18]. For instance, one classical neural network is Hopfield neural ...
A one-layer recurrent neural network for support vector machine learning

This paper presents a one-layer recurrent neural network for support vector machine (SVM) learning in pattern classification and regression. The SVM learning problem is first converted into an equivalent formulation, and then a one-layer recurrent ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

EITCE '21: Proceedings of the 2021 5th International Conference on Electronic Information Technology and Computer Engineering

October 2021

1723 pages

ISBN:9781450384322

DOI:10.1145/3501409

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 December 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

EITCE 2021

EITCE 2021: 2021 5th International Conference on Electronic Information Technology and Computer Engineering

October 22 - 24, 2021

Xiamen, China

Acceptance Rates

EITCE '21 Paper Acceptance Rate 294 of 531 submissions, 55%;

Overall Acceptance Rate 508 of 972 submissions, 52%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
83
Total Downloads

Downloads (Last 12 months)25
Downloads (Last 6 weeks)3

Reflects downloads up to 18 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Dantas PSabino da Silva WCordeiro LCarvalho C(2024)A comprehensive review of model compression techniques in machine learningApplied Intelligence10.1007/s10489-024-05747-w54:22(11804-11844)Online publication date: 2-Sep-2024
https://dl.acm.org/doi/10.1007/s10489-024-05747-w
Fragkou ELygnos VKatsaros D(2022)Transfer Learning for Convolutional Neural Networks in Tiny Deep Learning EnvironmentsProceedings of the 26th Pan-Hellenic Conference on Informatics10.1145/3575879.3575984(145-150)Online publication date: 25-Nov-2022
https://dl.acm.org/doi/10.1145/3575879.3575984

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

A novel neural network parallel adder

Novel Recurrent Neural Network for Time-Varying Problems Solving [Research Frontier]

A one-layer recurrent neural network for support vector machine learning