Beyond Base-2 Logarithmic Number Systems (WiP Paper)
Pages 141 - 145
Abstract
Logarithmic number systems (LNS) reduce hardware complexity for multiplication and division in embedded systems, at the cost of more complicated addition and subtraction. Existing LNS typically use base-2, meaning that representable numbers are some (often fractional) power of two. We argue that other bases should be considered. The base of the LNS determines the distribution of values and may reduce representation errors when converting inputs to LNS in domain-specific embedded hardware accelerators. Further, LNS addition and subtraction are normally implemented with lookup tables whose properties may be a function of the base.
We show that other bases can lower both representation and addition and subtraction error. We consider the case of 8-bit LNS, when converting from 8-bit floating point (FP). We find that base-1.984 significantly reduces the conversion error. Where we can scale values, base-1.851 reduces the error to just 0.011 units of least precision (ULP). A suitable base can also reduce average arithmetic errors. For example, base-1.802 LNS has an average error of 0.242 ULP and 0.212 ULP as compared to 0.243 ULP and 0.226 ULP for addition and subtraction, respectively, for base-2.
Supplementary Material
Presentation Video
- Download
- 17.37 MB
References
[1]
Syed Asad Alam and Oscar Gustafsson. 2014. Design of Finite Word Length Linear-Phase FIR Filters in the Logarithmic Number System Domain. VLSI Des., Vol. 2014 (2014), 14.
[2]
M. Arnold, E. Chester, J. Cowles, and C. Johnson. 2019. Optimizing Mitchell's Method for Approximate Logarithmic Addition via Base Selection with Application to Back-Propagation. In Proc. IEEE Nordic Circuits Syst. Conf. (NorCAS): NORCHIP and Int. Symp. of Syst.-on-Chip (SoC). 1--6.
[3]
M. G. Arnold. 2002. Reduced power consumption for MPEG decoding with LNS. In Proc. IEEE Int. Applicat.-Specific Syst. Arch. Processors Conf. 65--67.
[4]
M. G. Arnold, T. A. Bailey, J. R. Cowles, and M. D. Winkel. 1992. Applying features of the IEEE 754 to sign/logarithm arithmetic. IEEE Trans. Comput., Vol. 41, 8 (Aug. 1992), 1040--1050.
[5]
C. Basetas, I. Kouretas, and V. Paliouras. 2007. Low-Power Digital Filtering Based on the Logarithmic Number System. In Proc. Int. Workshop Power Timing Modeling Optimization Simulation. Vol. 4644. 546--555.
[6]
E. Chung and J. Fowers et al. 2018. Serving DNNs in Real Time at Datacenter Scale with Project Brainwave. IEEE Micro, Vol. 38, 2 (March 2018), 8--20.
[7]
M. Courbariaux, Y. Bengio, and J. P. David. 2014. Training deep neural networks with low precision multiplications. arxiv: 1412.7024
[8]
Itay Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio. 2017. Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations. ACM J. Mach. Learn. Res., Vol. 18, 1 (Jan. 2017), 6869--6898.
[9]
K. Johansson, O. Gustafsson, and L. Wanhammar. 2008. Implementation of elementary functions for logarithmic number systems. IET Comput. Digit. Tech., Vol. 2, 4 (June 2008), 295--304.
[10]
N. G. Kingsbury and P. J. W. Rayner. 1971. Digital Filtering using Logarithmic Arithmetic. Electron. Lett., Vol. 7, 2 (Jan. 1971), 56--58.
[11]
I. Kouretas, Ch. Basetas, and V. Paliouras. 2013. Low-Power Logarithmic Number System Addition/Subtraction and Their Impact on Digital Filters. IEEE Trans. Comput., Vol. 62, 11 (2013), 2196--2209.
[12]
E. H. Lee, D. Miyashita, E. Chai, B. Murmann, and S. S. Wong. 2017. LogNet: Energy-efficient neural networks using logarithmic computation. In Proc. IEEE Int. Conf. Acoust. Speech Signal Process. 5900--5904.
[13]
Y. C. Lim. 1990. Design of discrete-coefficient-value linear phase FIR filters with optimum normalized peak ripple magnitude. IEEE Trans. Circuits Syst., Vol. 37, 12 (1990), 1480--1486.
[14]
A.M. Mansour, A.M. El-Sawy, M.S. Aziz, and A. T. Sayed. 2015. A new hardware implementation of base 2 logarithm for FPGA. Int. J. Signal Proc. Syst., Vol. 3, 2 (2015), 171--181.
[15]
Daisuke Miyashita, Edward H. Lee, and Boris Murmann. [n. d.]. Convolutional neural networks using logarithmic data representation. arxiv: cs.NE/1603.01025v2
[16]
V. Paliouras and T. Stouraitis. 2000. Logarithmic number system for low-power arithmetic. In Proc. Int. Workshop Power Timing Modeling Optimization Simulation, Vol. 1918. 285--294.
[17]
M. Shah, S. Arunachalam, S. Wang, D. Blaauw, D. Sylvester, H. S. Kim, J. s. Seo, and C. Chakrabarti. 2018. A fixed-point neural network architecture for speech applications on resource constrained hardware. J. Signal Process. Syst., Vol. 90, 5 (May 2018), 727--741.
[18]
Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arxiv: 1409.1556
[19]
T. Stouraitis and V. Paliouras. 2002. Considering the alternatives in low-power design. IEEE Circuits Syst. Mag., Vol. 17, 4 (Aug. 2002), 22--29.
[20]
X. Sun, J. Choi, C. Y Chen, N. Wang, S. Venkataramani, V. Srinivasan, X. Cui, W. Zhang, and K. Gopalakrishnan. 2019. Hybrid 8-bit Floating Point (HFP8) Training and Inference for Deep Neural Networks. In Proc. Advances in Neural Information Processing Syst. 4901--4910.
[21]
E.E. Swartzlander Jr. and A.G. Alexopoulos. 1975. The Sign/Logarithm Number System. IEEE Trans. Comput., Vol. C-24, 12 (Dec. 1975), 1238--1242.
[22]
J. N. Coleman et al. 2008. The European Logarithmic Microprocesor. IEEE Trans. Comput., Vol. 57, 4 (April 2008), 532--546.
[23]
V. Vanhoucke, A. Senior, and M. Z. Mao. 2011. Improving the speed of neural networks on CPUs. In Proc. Deep Learning and Unsupervised Feature Learning Workshop, NIPS.
[24]
W. Wolf and M. Kandemir. 2003. Memory system optimization of embedded software. Proc. IEEE, Vol. 91, 1 (Jan. 2003), 165--182.
Index Terms
- Beyond Base-2 Logarithmic Number Systems (WiP Paper)
Recommendations
Low-precision Logarithmic Number Systems: Beyond Base-2
Logarithmic number systems (LNS) are used to represent real numbers in many applications using a constant base raised to a fixed-point exponent making its distribution exponential. This greatly simplifies hardware multiply, divide, and square root. LNS ...
A complex-number multiplier using radix-4 digits
ARITH '95: Proceedings of the 12th Symposium on Computer ArithmeticThis paper describes the design of a 16/spl times/16 complex-number multiplier developed as part of the arithmetic datapath of a complex-number digital signal processor. The complex-number multiplier internally uses binary signed digits for fast ...
Towards a Quaternion Complex Logarithmic Number System
ARITH '11: Proceedings of the 2011 IEEE 20th Symposium on Computer ArithmeticThe well-known generalization of real to complex arithmetic (two reals) extends further to more obscure quaternion arithmetic (four reals), which has applications in signal processing, aerospace, graphics and virtual reality. Quaternion multiplication ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
June 2020
163 pages
Copyright © 2020 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 16 June 2020
Check for updates
Author Tags
Qualifiers
- Short-paper
Funding Sources
- Science Foundation Ireland
- H2020 Marie Sk?odowska-Curie Actions
Conference
LCTES '20: 21st ACM SIGPLAN/SIGBED Conference on Languages, Compilers, and Tools for Embedded Systems
June 16, 2020
London, United Kingdom
Acceptance Rates
Overall Acceptance Rate 116 of 438 submissions, 26%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 366Total Downloads
- Downloads (Last 12 months)86
- Downloads (Last 6 weeks)27
Reflects downloads up to 19 Nov 2024
Other Metrics
Citations
Cited By
View allView Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in