Icon View Thread

The following is the text of the current message along with any replies.
Messages 11 to 15 of 15 total
Thread Runtime dependent number of float value WHERE conditions
Wed, Jun 6 2012 5:00 AMPermanent Link

Roy Lambert

NLH Associates

Team Elevate Team Elevate

John


Agreed, however, I feel the best solution would be to multiply both sides by a scaling factor and truncate so you're comparing integers

Roy Lambert [Team Elevate]
Wed, Jun 6 2012 5:30 AMPermanent Link

John Hay

Roy
>
> Agreed, however, I feel the best solution would be to multiply both sides by a scaling factor and truncate so you're
comparing integers
>

Makes sense, or use ROUND function

John

Thu, Jun 7 2012 3:37 PMPermanent Link

Janusz Cyran

Roy,

Thank you for your in-depth reply. BTW how CAST(floatfield AS DECIMAL(x,y)) works regarding rounding problem?

Regards,
JC




Roy Lambert wrote:

Janusz


Even if I was right you may still have problems. Float equality tests are bad news. You would need to guarantee whatever produced the two floats was working exactly the same, and storing them the same and that FloatToStr was producing what you expected.

If you're only interested in a match to a specific number of decimal places then you'd be better off using CAST(floatfield AS DECIMAL(x,y)) and using the Delphi Format function to format the float from the other table or CASTing that value as well.

If you have problems let us know a bit more about what you're actually trying to achieve and we should be able to help.

Roy Lambert [Team Elevate]
Fri, Jun 8 2012 4:02 AMPermanent Link

Roy Lambert

NLH Associates

Team Elevate Team Elevate

Janusz

>Thank you for your in-depth reply. BTW how CAST(floatfield AS DECIMAL(x,y)) works regarding rounding problem?

To some degree it will depend on how Tim has implemented things ie is it a simple truncate, bankers rounding or what. I don't know. This is why in my reply to John Hay I said that I'd actually prefer to multiply by a scaling factor, truncate and compare as integer. Such an approach, whilst not totally foolproof, puts you more in control.

Roy Lambert [Team Elevate]
Wed, Jun 20 2012 2:05 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Roy,

<< To some degree it will depend on how Tim has implemented things ie is it
a simple truncate, bankers rounding or what. I don't know. This is why in my
reply to John Hay I said that I'd actually prefer to multiply by a scaling
factor, truncate and compare as integer. Such an approach, whilst not
totally foolproof, puts you more in control. >>

The rounding for DECIMAL/NUMERIC works using the "round half up" method.

Tim Young
Elevate Software
www.elevatesoft.com
« Previous PagePage 2 of 2
Jump to Page:  1 2
Image