With the increasing real-time constraints being put by the automotive scenario on the use of Deep Neural Networks (DNNs), there is the need to review information representation. A very challenging path is to employ an encoding that allows a fast processing and hardware-friendly representation of information. Among the proposed alternatives to the IEEE 754 standard regarding floating point representation of real numbers, the recently introduced Posit format has been theoretically proven to be really promising in satisfying the mentioned requirements. However, with the absence of proper hardware support for this novel type this evaluation can be conducted only through a software emulation. While waiting for the widespread availability of the Posit Processing Units (the equivalent of the FPUs), we can already exploit the Posit representation and the currently available ALU to speed-up DNNs, by manipulation the low-level bit string representations of posits. As a first step, in this paper we present new arithmetic properties of the Posit number system with a focus on the configuration with 0 exponent bits. In particular, we propose a new class of Posit operators called L1 operators, which consists in fast and approximated versions of existing arithmetic operations or functions (e.g. hyperbolic tangent (TANH) and extended linear unit (ELU)) only using integer arithmetic. These operators introduce very interesting properties and results: (i) faster evaluation than the exact counterpart with a negligible accuracy degradation; (ii) enabling an efficient ALU emulation of a number of Posits operations; (iii) enabling the possibility to vectorize operations n Posits, using existing ALU vectorized operations (such as the Scalable Vector Extension of ARM CPUs or Advanced Vector Extensions on Intel CPUs). As a second step, we test the proposed activation function on Posit-based DNNs, showing how 16-bit down to 10-bit Posits represent an exact replacement for 32-bit floats, while 8-bit Posits could be an interesting alternative to 32-bit Floats since their performance are a bit lower, but their high speed and low storage properties are very appealing (leading to a lower band-width demanding and more cache friendly code). Finally we point out how small posits (i.e., up to 14 bit long) are very interesting, in the meanwhile PPUs become widespread, since posit operations can be tabulated in a very efficient way.

Keywords: {Alternative Representations to Float Numbers; Posit arithmetic; Neural Network activation functions; Sensors signal processing; Deep Neural Networks (DNN) for Sensors Data Analysis}

Link: https://www.mdpi.com/1424-8220/20/5/1515