Caves Travel Diving Graphics Mizar Texts Cuisine Lemkov Contact Map RSS Polski
Trybiks' Dive Texts Programming Rounding and precision on the 8087 FPU YAC Software












Rhino Mocks



UI Testing




Rounding and precision on the 8087 FPU
I, as a user, can understand various defects in a data analysis application, except those that lead to one of: loss of data / work and incorrect results of calculations. Thus, when developing YAC Data Analyzer (YDA) I obviously added many different tests that concentrated mainly on those aspect of the program - that checked the correctness of the many data handling and computation routines used there.

While I was developing the program on a single computer, everything seemed to be working fine. But after I moved the testing programs to a virtual machine (VirtualBox), I started to get various differences between the reference results (calculated and saved on the previous computer) and the newly generated results. The differences weren't big - usually on the last digit of the precision of double numbers. Still, there were differences, so something was being done incorrectly.

A short digression on the calculations that YDA does: these are pretty standard analyses of data from marketing surveys plus more complex analyses, such as media plan optimizations. This includes the weighting of data - each respondent is assigned a weight so that results can be correctly estimated to the whole population. These weights are floating point numbers not "far" from the value of 1 (usually in the range of 0.5 to 2.0, although sometimes we get larger values). Also, these values are calculated based on the structure of the sample and the structure of the whole population (so that after weighting we get, for instance, the same fraction of males and females in the sample as is in the whole population), so these are not simple numbers such as 1/2 or 1/4, but numbers with most digits significant.

At first, I attributed the differences to the virtual machine, though this was more from laziness (and heaps of other work) than from actually thinking the problem through. However, the problem kept nagging me, so after a short while I revisited the issue.

As almost always, Google is you friend here: not knowing where the problem was coming from, I started searching for Delphi related FPU problems with precision and rounding. This is when I found this text that leads to this FPU tutorial.

So, the problem is with the control word of the FPU being initialized differently on different computers and on virtual machines.

There are two fields that are important here:
  • RC (Rounding Control)
  • PC (Precision Control)
To control these fields, the Jcl8087 unit from the Jedi Component Library may be used. There, we have the following functions:
  • Get/Set8087Rounding
  • Get/Set8087Precision
with the values (respectively):
  • rcNearestOrEven, rcDownInfinity, rcUpInfinity, rcChopOrTruncate
  • pcSingle, pcReserved, pcDouble, pcExtended
The meanings of these values are pretty well described on the pages linked to above. The only problem was whether my reference results were the "correct" ones or the newly generated ones, or were these both incorrect because of message up control word values.

First, note the defaults:
  • RC - rcNearestOrEven (standard handling of rounding in finances and statistics)
  • PC - pcExtended (highest)
Ok, let's set these defaults and see what the results are. Fortunately, it turned out that the reference results matched exactly the ones generated now. Next, I checked the settings on the VM - it turns out that some application or startup code on the virtual machine (and my Dell laptop too, it turns out) sets the control word to the correct rounding, but pcDouble precision...

Take a look at the following code:
    vd1, vd2: double;
    // Reset the control word to the defaults:
    vd1 := 1.5;
    vd2 := -1.5;
    WriteLn(Format('%d %d', [Round(vd1), Round(vd2)]));
    WriteLn(Format('%d %d', [Round(vd1), Round(vd2)]));
    WriteLn(Format('%d %d', [Round(vd1), Round(vd2)]));
    WriteLn(Format('%d %d', [Round(vd1), Round(vd2)]));
    vd1 := 1 / 15;
    vd2 := 1 / 7;
    WriteLn(Format('%g', [vd1 + vd2]));
    WriteLn(Format('%g', [vd1 + vd2]));
    WriteLn(Format('%g', [vd1 + vd2]));
and the output:
  2 -2
  1 -2
  2 -1
  1 -1
Note that even simple calculations can lead to different results...

Anyway, to solve the problem of different results of FPU calculations of different computers make sure that the FPU control word is initialized correctly. To be on the safe side of things, I decided not only to run this setup code at the start of my application, bit to run it every time FPU calculations are done in the program - in case some application or third-party module changes the control word to some non-default value.


No comments yet...


Add a comment (fields with an asterisk are required)
Name / nick *
Mail (will remain hidden) *
Your website
Comment (no tags) *
Enter the text displayed below *





Related pages

TFS - The underlying connection was closed: an unexpected error occurred on a receive.

WCF - The underlying connection was closed: an unexpected error occurred on a receive.

Delphi interfaces... again

Saving / restoring window placements in .NET

Checking "Dangling" Event Handlers in Delphi Forms

Meaningful identifiers

Public fields vs. properties

Drag-n-drop files onto the application window

Intraweb and MaxConnections

A Case for FreeAndNIL

Intraweb as an Apache DSO module

"Device not supported" in Intraweb

Automated GUI Testing

Random()'s Determinism

SessionTimeout in Intraweb

Using TChart with Intraweb

Unknown driver: MySQL

TIdMessage's CharSet

Software Guarantees

Automated Testing of Window Forms

TChart - Missing Labels in Axes

Memory Leaks and Connection Explosions in DBExpress

Controlling Conditional Defines and Compilation Switches

Detecting Memory Leaks with DUnit

last_insert_id() and DBExpress

Registering Extensions

DBExpress and Thread Safety

Forms as Frames

Checking Dangling Pointers vs. the New Memory Manager

Accessing Protected Members

Objects, interfaces, and memory management in Delphi