Home/Support/Support Forum/Loss precision on floating point calculation
New and improved user forum site coming soon
Welcome to Digi Forum, where you can ask questions and receive answers from other members of the community.

Loss precision on floating point calculation

0 votes
I am using RCM6760 with Dynamic C 10.72B. I have created the following test code:

void main()
float tempOnTime;
float tempPeriod;
float tempOffTime;
float tempDutyCycle;

tempOnTime = 0.001;
tempOffTime = 0.0;
tempPeriod = tempOnTime + tempOffTime;
tempDutyCycle = tempOnTime / tempPeriod;
tempDutyCycle = tempDutyCycle * 100;
printf("%.6f\n", tempDutyCycle);

The printf() gave 99.999992 which I expect to have the value of 100. The Watches show the tempDutyCycle is "float 100". Can you suggest a way to avoid such precision lost?

asked Sep 27, 2017 in Rabbit Software by andyfung New to the Community (3 points)

Please log in or register to answer this question.

1 Answer

0 votes
How much precision do you require? 99.999992 is only 0.000008% off from 100.

If it isn't precise enough, you'll need to switch to fixed-point math where you use integral units. For example, if 0.001 is seconds, switch to doing your math with milliseconds or microseconds and integer math.

In general, avoid float unless absolutely necessary for your algorithm. And with integer math, do all of your multiplication first (ensuring that you won't overflow a 32-bit value) and then do division (which will lose any fractional results).
tempOnTime = 1; tempOffTime = 0 tempPeriod = tempOnTime + tempOffTime; tempDutyCycle = (100 * tempOnTime) / tempPeriod;
answered Sep 28, 2017 by TomCollins Veteran of the Digi Community (2,382 points)