View previous topic :: View next topic |
Author |
Message |
Redouane Master Cheater Reputation: 3
Joined: 05 Sep 2013 Posts: 363 Location: Algeria
|
Posted: Sat May 16, 2015 12:41 pm Post subject: Lua number precision |
|
|
This is not related to Cheat Engine,I hope it's the right section to post it
I noticed that in many cases,repeated calculations on Lua numbers mess their precision.
Example:
f(x) = (1/2)x^2*(3-2ln(x)) + 1;
This code finds a range to 10^(-2) where the solution of f(x)=0 fits.
Code: | local ln = math.log;
local function f(x)
return 0.5*x^2*(3-2*ln(x)) + 1;
end
-- f(1)>0
for i = 0.01,50,0.01 do
if f(i) < 0 then print(']'..i-0.01 ..';'..i..'[');
break;
end
end |
Result is unexpected,it prints ']4.6899999999999;4.6999999999999[',while it should give ']4.69;4.7[',what's the reason for this?
Using Lua 5.3,on the Lua website (demo)
Last edited by Redouane on Sat May 16, 2015 2:19 pm; edited 2 times in total |
|
Back to top |
|
|
Zanzer I post too much Reputation: 126
Joined: 09 Jun 2013 Posts: 3278
|
Posted: Sat May 16, 2015 1:06 pm Post subject: |
|
|
http://en.wikipedia.org/wiki/Round-off_error
It's just what happens when you use decimals in many programming languages.
One of the first things you learn when dealing with currency is to convert it into cents and never use decimal dollars.
Code: | print(string.format("%1.18f", 0.1 + 0.2)) |
|
|
Back to top |
|
|
Redouane Master Cheater Reputation: 3
Joined: 05 Sep 2013 Posts: 363 Location: Algeria
|
Posted: Sat May 16, 2015 3:50 pm Post subject: |
|
|
Zanzer wrote: | http://en.wikipedia.org/wiki/Round-off_error
It's just what happens when you use decimals in many programming languages.
One of the first things you learn when dealing with currency is to convert it into cents and never use decimal dollars.
Code: | print(string.format("%1.18f", 0.1 + 0.2)) |
|
Thanks for the response.
|
|
Back to top |
|
|
atom0s Moderator Reputation: 198
Joined: 25 Jan 2006 Posts: 8516 Location: 127.0.0.1
|
Posted: Sun May 17, 2015 1:43 am Post subject: |
|
|
How the library is compiled and with what compiler plays a factor into how the floating point numbers will be handled by the system. Lua, by default, treats all numbers (integer and decimal) as doubles. (8 bytes, floating point)
Depending on the compiler and settings used, the calculations and rounding can be more exact or more 'lazy'.
_________________
- Retired. |
|
Back to top |
|
|
Redouane Master Cheater Reputation: 3
Joined: 05 Sep 2013 Posts: 363 Location: Algeria
|
Posted: Sun May 17, 2015 6:40 am Post subject: |
|
|
atom0s wrote: | Lua, by default, treats all numbers (integer and decimal) as doubles. (8 bytes, floating point) |
Since its Lua 5.3 , integers are represented as 64bit ints (by default) ,no?
|
|
Back to top |
|
|
Dark Byte Site Admin Reputation: 457
Joined: 09 May 2003 Posts: 25262 Location: The netherlands
|
Posted: Sun May 17, 2015 7:12 am Post subject: |
|
|
yes, but since your result contains a decimal seperator it can't be handled as an integer, but as a float
_________________
Do not ask me about online cheats. I don't know any and wont help finding them.
Like my help? Join me on Patreon so i can keep helping |
|
Back to top |
|
|
|