The following statement has a potential problem.
It uses the ** operator, which returns a float value. Having one operand with a float value means that the entire right-hand-side expression returns a float value. When a float value is assigned to a decimal value (a non-float value), the result is truncated rather than rounded. RPG uses binary floating point, which can't hold every decimal value exactly. So a value that should be say 1.234 might have a floating pointvalue of 1.23999999, which would give 1.233 instead of 1.234 in the result.
One way to solve the rounding problem is to use EVAL(H):
Or, what I would do, calculate the power-of-10 separately. Define "precision" big enough to hold the maximum power of 10 you might need.
Code:
numVeld = intVeld / (10 ** dsFFD(ix).decPos);
It uses the ** operator, which returns a float value. Having one operand with a float value means that the entire right-hand-side expression returns a float value. When a float value is assigned to a decimal value (a non-float value), the result is truncated rather than rounded. RPG uses binary floating point, which can't hold every decimal value exactly. So a value that should be say 1.234 might have a floating pointvalue of 1.23999999, which would give 1.233 instead of 1.234 in the result.
One way to solve the rounding problem is to use EVAL(H):
Code:
eval(h) numVeld = intVeld / (10 ** dsFFD(ix).decPos);
Or, what I would do, calculate the power-of-10 separately. Define "precision" big enough to hold the maximum power of 10 you might need.
Code:
EVAL(H) precision = 10 ** dsFFD(ix).decPos; numVeld = intVeld / precision;
Comment