Lost variable accuracy in Simulation mode
Posted: Sun Oct 10, 2010 11:17 am
Testing the accuracy of the Sinus and Cosinus values, see elsewhere, I found some accuracy loss of floating point variables ONLY IN THE SIMULATION mode.
A macro can return a floating point variable. If used the accuracy *) will be changed to 0.000001 (absolute). Or said it in a different way, the return floating point value of a macro will be rounded to 6 numbers after the decimal point. This happens only in the simulation mode, not when a microcontroller is used.
Be prepared!
Jan Lichtenbelt
*) Note. I do not know the accuray of floating point variables. Floating point variables use 4 bytes. As far as I know, the last byte is used for the exponential part and sign, the other 3 bytes are used for the number after the decimal point. That means the accuracy of floating point variables is 2^24.
A macro can return a floating point variable. If used the accuracy *) will be changed to 0.000001 (absolute). Or said it in a different way, the return floating point value of a macro will be rounded to 6 numbers after the decimal point. This happens only in the simulation mode, not when a microcontroller is used.
Be prepared!
Jan Lichtenbelt
*) Note. I do not know the accuray of floating point variables. Floating point variables use 4 bytes. As far as I know, the last byte is used for the exponential part and sign, the other 3 bytes are used for the number after the decimal point. That means the accuracy of floating point variables is 2^24.