Home     |     .Net Programming    |     cSharp Home    |     Sql Server Home    |     Javascript / Client Side Development     |     Ajax Programming

Ruby on Rails Development     |     Perl Programming     |     C Programming Language     |     C++ Programming     |     IT Jobs

Python Programming Language     |     Laptop Suggestions?    |     TCL Scripting     |     Fortran Programming     |     Scheme Programming Language


 
 
Cervo Technologies
The Right Source to Outsource

MS Dynamics CRM 3.0

Fortran Programming Language

Commutative law in addition in XL Fortran


To group members with more expertise,

I have been experiencing somewhat strange problems
in doing a simple sum of double precision real variables
in my program.

The following is a part of the program that simply adds
four double precision numbers. The program has been
compiled using XL Fortran 9.1 on IBM e-server without
using any compilation options

      integer, parameter :: r8 = selected_real_kind(12)
      real(r8) :: a,b,c,d,e,f

      a =  0.378662378978238886E-11_r8
      b =  0.103523081746141482E-10_r8
      c =  0.880061250413881054E-11_r8
      d = -0.713532278734606307E-10_r8

      e = a+b+c+d
      f = a+d+b+c

In this calculation, "e" and "f" become as follows:
e = -0.484136834049252831E-10,
f  = -0.484136834049252895E-10.

The last two digits are different between "e" and "f".

Have you ever experience this problem?
Is this problem related to the extended number of significant
digit in XL Fortran? How can I remove this behavior without
using compilation option like "-qfloat=ieee". I think it's better
not to use "-qfloat=ieee" because my program should be
inserted as a part of much bigger program (Global atmospheric
model) that is not compiled with such an option.

<insun.s@gmail.com> wrote in message

news:1178819071.009867.158770@e65g2000hsc.googlegroups.com...

> In this calculation, "e" and "f" become as follows:
> e = -0.484136834049252831E-10,
> f  = -0.484136834049252895E-10.

How did you determine these two values? Are you perhaps printing more digits
than are actually available in IBM's double precision?

Regards,

Mike Metcalf

Hello,

My reaction is: so?

You have 16 significant figures that agree. What did you expect to happen?

cheers,

paulv

--
Paul van Delst             Ride lots.
CIMSS @ NOAA/NCEP/EMC               Eddy Merckx

On May 10, 1:57 pm, "Michael Metcalf" <michaelmetc@compuserve.com>
wrote:

> <insun.s@gmail.com> wrote in message

> news:1178819071.009867.158770@e65g2000hsc.googlegroups.com...

> > In this calculation, "e" and "f" become as follows:
> > e = -0.484136834049252831E-10,
> > f  = -0.484136834049252895E-10.

> How did you determine these two values? Are you perhaps printing more digits
> than are actually available in IBM's double precision?

> Regards,

> Mike Metcalf

I just printed out the two variables "e" and "f" like this:

    print *,e
    print *,f

I think there are 3 additional significant digits in XL Fortran
compared with IEEE double precision variable.

On May 10, 2:04 pm, Paul van Delst <Paul.vanDe@noaa.gov> wrote:

Hi,

I think the commutative law in addition should hold in this
calculation.
The commutative law can't be generally held in computation?

<insun.s@gmail.com> wrote in message

news:1178821052.385930.236900@e65g2000hsc.googlegroups.com...
> Hi,

> I think the commutative law in addition should hold in this
> calculation.
> The commutative law can't be generally held in computation?

That's a big subject, but think about

      A + a - B
and
      A - B + a

where A, B >>>> a
and     A ~ B
and A, B, a > 0

Regards,

Mike Metcalf

In article <1178821052.385930.236@e65g2000hsc.googlegroups.com>,
        "insun.s@gmail.com" <insun.s@gmail.com> writes:

> I think the commutative law in addition should hold in this
> calculation.
> The commutative law can't be generally held in computation?

Google "Goldberg floating point".
Read document, then read it again.

--
Steve
http://troutmask.apl.washington.edu/~kargl/

On May 10, 8:17 pm, "insun.s@gmail.com" <insun.s@gmail.com>
wrote:

commutativity is usually preserved in FP addition. However,
associativity is not; what you see here is violation of associativity.
If you write
a+b+c+d, most compiler evaluate this as ((a+b)+c)+d, while the second
expression gives ((a+d)+b)+c - clearly a completely different way of
adding the four numbers.
Note, however, that even if you write
a+(b+c)+d, it is NOT guaranteed that the compiler will evaluate (b+c)
first; according to the Fortran standard the compiler is free to
transform an expression in any mathematically equivalent way. Most
compilers do such rearrangements when
optimization is enabled.

I believe it is guaranteed by Fortran standard, although several
compilers require special options to adhere to this.  Fortran standards,
since f66, have had specific lists of which re-associations are
permitted.   Compilers sometimes get confused between Fortran and
pre-standard C, where these parentheses could be ignored, and standard
C, which is stricter yet than Fortran.

Well, given your surprise at the result of your example, what you *think* should happen
and what actually *is* happening are apparently not congruent.

> The commutative law can't be generally held in computation?

Well, it's not my area of expertise (just experience) so I'm not going to make
generalisations. Let me just say that examples abound showing how the commutative and
associative laws sometimes *don't* hold - but I don't think your example above is one of
those. I guess it depends on what one accepts as being "equal" in any particular situation.

But, regardless, if your algorithms depend on accuracy beyond 16 significant figures, the
(sometimes not so) subtle aspects of finite precision arithmetic are probably the least of
your worries. :o)

cheers,

paulv

--
Paul van Delst             Ride lots.
CIMSS @ NOAA/NCEP/EMC               Eddy Merckx

highegg wrote:

...

> Note, however, that even if you write
> a+(b+c)+d, it is NOT guaranteed that the compiler will evaluate (b+c)
> first; according to the Fortran standard the compiler is free to
> transform an expression in any mathematically equivalent way. Most
> compilers do such rearrangements when
> optimization is enabled.

The Fortran standard requires that the integrity of parenthesis
not be violated.  So, in the above, (b+c) will be evaluated
prior to adding the result to A or to D.  Now, the "mathematically
equivalent" rule *can* be applied to the rest of the expression.
Your expression could be done as:

   register1 = b+c
   register2 = a+d
   answer = register1+register2

Or, it could be done as:

   register1 = b+c
   register2 = a+register1
   answer = register2+d

And so on.  You can control the compiler's choices by using
parenthesis judiciously.  The expression (a+(b+c))+d must be
done in the second way I gave above.  Sure, (b+c) might be
done as (c+b), but most floating-point arithmetic commutes
(the original Cray had a non-commutative multiply and it was
so unpopular that later versions has a slightly less accurate
multiply that *was* commutative).

--
J. Giles

"I conclude that there are two ways of constructing a software
design: One way is to make it so simple that there are obviously
no deficiencies and the other way is to make it so complicated
that there are no obvious deficiencies."   --  C. A. R. Hoare

insun.s@gmail.com wrote:

(snip)

> I think the commutative law in addition should hold in this
> calculation.
> The commutative law can't be generally held in computation?

I believe that some Cray hardware didn't support commutative
multiplication, but most processors do.  Considering
some recent discussions, that assumes no side effects in
evaluating the parts of an expression.

Floating point addition is often not associative, though.

-- glen

On May 10, 2:56 pm, Paul van Delst <Paul.vanDe@noaa.gov> wrote:

Problem is that effects of such subtle errors on global
atmospheric model grow significantly after long term
integration of the model.

I feel I need to explain in more detail how problems
regarding such associativity (not commutativity) can
happen in the model.

Currently, I'm developing a program for tracing particles in
MPI-parallelized global atmospheric model.

Let's assume 4 particles in two MPI-decomposed domains
(domain 1 and 2). Initially, two particles are located in domain 1,
and the other two in domain 2. After a few model steps,
the 4 particles arrive in one grid box located in domain 1.

In case we want to add the values regarding the 4 particles that
arrived in the grid box, I can have different results in the sum
of the 4 numbers depending on how many processes are used
in the model run.

If we used a serial model, the four number will be added in one
process. However, as long as two particles come from the other
domain, I think the order of values used in addition may not be
the same as that in case of serial program.

Would you be happier if I said to change your WRITE statements from

     print *,e
     print *,f

to

     WRITE(*,'("e = ",e22.15)') e
     WRITE(*,'("f = ",e22.15)') f

Then, the "results" should agree "exactly":

     e = -0.484136834049252E-10
     f = -0.484136834049252E-10

How's that? :o)

Seriously, though, if you're adding (presumably lots of) numbers in an effectively random
order, you really do need to understand the limitations of floating point arithmetic and
how it impacts your algorithm.

cheers,

paulv

--
Paul van Delst             Ride lots.
CIMSS @ NOAA/NCEP/EMC               Eddy Merckx

insun.s@gmail.com wrote:

(snip)

> Problem is that effects of such subtle errors on global
> atmospheric model grow significantly after long term
> integration of the model.

Integration usually averages out such errors in the
long term.  It is differentiation, subtracting almost
equal values, that causes increasing errors.

> I feel I need to explain in more detail how problems
> regarding such associativity (not commutativity) can
> happen in the model.

Not knowing the particular details of your model, I would
say that the effect you are describing is a natural part
of the system.  Small changes in the initial state might
result in larger changes in the later or final state.
If you have an atmospheric model that depends on measurements
of physical quantities to more than 16 digits of precision,
I would say that you need a new model.   If you consider
the uncertainty in the initial state, and the propagation
of that uncertainty through the model, I believe that
the errors you are seeing will be smaller than the
natural uncertainty in the system.

-- glen

highegg wrote:

...

> Note, however, that even if you write
> a+(b+c)+d, it is NOT guaranteed that the compiler will evaluate (b+c)
> first; according to the Fortran standard the compiler is free to
> transform an expression in any mathematically equivalent way. Most
> compilers do such rearrangements when
> optimization is enabled.

The Fortran standard requires that the integrity of parenthesis
not be violated.  So, in the above, (b+c) will be evaluated
prior to adding the result to A or to D.  Now, the "mathematically
equivalent" rule *can* be applied to the rest of the expression.
Your expression could be done as:

   register1 = b+c
   register2 = a+d
   answer = register1+register2

Or, it could be done as:

   register1 = b+c
   register2 = a+register1
   answer = register2+d

And so on.  You can control the compiler's choices by using
parenthesis judiciously.  The expression (a+(b+c))+d must be
done in the second way I gave above.  Sure, (b+c) might be
done as (c+b), but most floating-point arithmetic commutes
(the original Cray had a non-commutative multiply and it was
so unpopular that later versions has a slightly less accurate
multiply that *was* commutative).

Note: I sent this message earlier, but it never showed up.  now
it's likely to appear twice I suppose.  Sorry if it does.

--
J. Giles

"I conclude that there are two ways of constructing a software
design: One way is to make it so simple that there are obviously
no deficiencies and the other way is to make it so complicated
that there are no obvious deficiencies."   --  C. A. R. Hoare

On May 10, 12:32 pm, "insun.s@gmail.com" <insun.s@gmail.com>
wrote:

> Problem is that effects of such subtle errors on global
> atmospheric model grow significantly after long term
> integration of the model.

Yes, this is certainly possible, especially in chaotic
processes like models of atmospheric transport.

Suppose that the "true" state vector is x, and you
have some process, f, that produces a forecast f(x).
Let X be a floating-point approximation of x, and
let F(X) be the result that you get from estimating
f(x) in floating-point arithmetic.  You want to know
whether F(X) is close to f(x) in some suitable sense.
If the answer is yes, then F(X) is said to "shadow"
f(x).

There is a fairly extensive literature on this topic.
In general, it isn't possible to shadow processes like
models of atmospheric transport for very long, because
the resulting dynamical system has lots of Lyapunov
exponents that fluctuate around 0, so there' s no
hyperbolic structure that lets you control the arithmetic
error.  (Email me if you want some references.)

I have been told that the best numerical weather models
fail to shadow the atmosphere after a couple of hours.
This is one reason (of many) why weather forecasting
is not an exact science.

--Eric

Paul van Delst wrote:

> Seriously, though, if you're adding (presumably lots of) numbers in an effectively random
> order, you really do need to understand the limitations of floating point arithmetic and
> how it impacts your algorithm.

A lesson you seem to have missed, or you would have noted that
undefining noisy digits would be a prerequisite towards numeric bliss.
As it stands, your crackpot suggestion was nothing but filtered noise.

Likewise, why didn't you enlighten the (dutch") polynomial specialist
that vanishing digits jinxed his mundane polynomial evaluation at the
root, and that algebraic expressions [of polynomial form] follow the
same computational laws.

 "error analysis for brute force polynomial eval"

sureal <sur@state.com> wrote:

[yet another random childish insult]

Time for an addition to my killfile, I suppose. Overdue, actually.

--
Richard Maine                    | Good judgement comes from experience;
email: last name at domain . net | experience comes from bad judgement.
domain: summertriangle           |  -- Mark Twain

On May 10, 3:11 pm, "James Giles" <jamesgi@worldnet.att.net> wrote:

> (the original Cray had a non-commutative multiply and it was
> so unpopular that later versions has a slightly less accurate
> multiply that *was* commutative).

IIRC, the first two Cray 1's had the problem.  Again IIRC, the fix was
to
sort the operands, which would not affect the accuracy.

Bob Corbett

robert.corb@sun.com wrote:
> On May 10, 3:11 pm, "James Giles" <jamesgi@worldnet.att.net> wrote:

>> (the original Cray had a non-commutative multiply and it was
>> so unpopular that later versions has a slightly less accurate
>> multiply that *was* commutative).

> IIRC, the first two Cray 1's had the problem.  Again IIRC, the fix was
> to sort the operands, which would not affect the accuracy.

Could be.  Still I seem to remember there was some sort of price
to pay.  Maybe it was a clock slower.  With pipelining that would
have been a minor price.  Anyone know for sure?

--
J. Giles

"I conclude that there are two ways of constructing a software
design: One way is to make it so simple that there are obviously
no deficiencies and the other way is to make it so complicated
that there are no obvious deficiencies."   --  C. A. R. Hoare

On May 11, 3:41 am, "Michael Metcalf" <michaelmetc@compuserve.com>
wrote:

In this case, however, the order of magnitude of 4 numbers is similar
to
each other.

By the way, if I test the program I mentioned above in Linux machine
using
Intel fortran (or g95), and if I use the following numbers of which
the last
three digits are removed (or rounded up),

     a =  0.378662378978239E-11_r8
      b =  0.103523081746141E-10_r8
      c =  0.880061250413881E-11_r8
      d = -0.713532278734606E-10_r8

Results do not depend on the order of adding.....
This is why I suspect the extended significant digits of IBM XL
Fortran,
besides issues regarding the "associativity" of floating point
operation.

Richard Maine wrote:
| sureal <sur@state.com> wrote:

| [yet another random childish insult]
|
| Time for an addition to my killfile, I suppose. Overdue, actually.

Actually, I'm pleasantly surprised how clf has been persistent in
applying DFTT rule to this pal. He's kind of getting it slow
though.

--
 Jugoslav
___________
www.xeffort.com

Please reply to the newsgroup.
You can find my real e-mail on my home page above.

Yes.  They're off by a bit more than a factor of 4 -- that is, 2**2.
Thus, one would expect to find variation in the last one or two bits.

And, indeed, the differences you're complaining of are about the size
one would expect from changing the last one or two bits.

> By the way, if I test the program I mentioned above in Linux machine
> using
> Intel fortran (or g95), and if I use the following numbers of which
> the last
> three digits are removed (or rounded up),

>      a =  0.378662378978239E-11_r8
>       b =  0.103523081746141E-10_r8
>       c =  0.880061250413881E-11_r8
>       d = -0.713532278734606E-10_r8

> Results do not depend on the order of adding.....

That's probably because on i686, intermediate results are usually put
into extra-precision (80-bit) registers, so you don't get any signficant
rounding until the result is stored in memory.

- Brooks

--
The "bmoses-nospam" address is valid; no unmunging needed.

Add to del.icio.us | Digg this | Stumble it | Powered by Megasolutions Inc