Cheat Engine Forum Index Cheat Engine
The Official Site of Cheat Engine
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 


Lua number precision

 
Post new topic   Reply to topic    Cheat Engine Forum Index -> Cheat Engine Lua Scripting
View previous topic :: View next topic  
Author Message
Redouane
Master Cheater
Reputation: 3

Joined: 05 Sep 2013
Posts: 363
Location: Algeria

PostPosted: Sat May 16, 2015 12:41 pm    Post subject: Lua number precision Reply with quote

This is not related to Cheat Engine,I hope it's the right section to post it

I noticed that in many cases,repeated calculations on Lua numbers mess their precision.

Example:
f(x) = (1/2)x^2*(3-2ln(x)) + 1;
This code finds a range to 10^(-2) where the solution of f(x)=0 fits.
Code:
local ln = math.log;
local function f(x)
   return 0.5*x^2*(3-2*ln(x)) + 1;
end
-- f(1)>0
for i = 0.01,50,0.01 do
   if f(i) < 0 then print(']'..i-0.01 ..';'..i..'[');
   break;
   end
end

Result is unexpected,it prints ']4.6899999999999;4.6999999999999[',while it should give ']4.69;4.7[',what's the reason for this?

Using Lua 5.3,on the Lua website (demo)


Last edited by Redouane on Sat May 16, 2015 2:19 pm; edited 2 times in total
Back to top
View user's profile Send private message
Zanzer
I post too much
Reputation: 126

Joined: 09 Jun 2013
Posts: 3278

PostPosted: Sat May 16, 2015 1:06 pm    Post subject: Reply with quote

http://en.wikipedia.org/wiki/Round-off_error
It's just what happens when you use decimals in many programming languages.
One of the first things you learn when dealing with currency is to convert it into cents and never use decimal dollars.
Code:
print(string.format("%1.18f", 0.1 + 0.2))
Back to top
View user's profile Send private message
Redouane
Master Cheater
Reputation: 3

Joined: 05 Sep 2013
Posts: 363
Location: Algeria

PostPosted: Sat May 16, 2015 3:50 pm    Post subject: Reply with quote

Zanzer wrote:
http://en.wikipedia.org/wiki/Round-off_error
It's just what happens when you use decimals in many programming languages.
One of the first things you learn when dealing with currency is to convert it into cents and never use decimal dollars.
Code:
print(string.format("%1.18f", 0.1 + 0.2))


Thanks for the response.
Back to top
View user's profile Send private message
atom0s
Moderator
Reputation: 198

Joined: 25 Jan 2006
Posts: 8516
Location: 127.0.0.1

PostPosted: Sun May 17, 2015 1:43 am    Post subject: Reply with quote

How the library is compiled and with what compiler plays a factor into how the floating point numbers will be handled by the system. Lua, by default, treats all numbers (integer and decimal) as doubles. (8 bytes, floating point)

Depending on the compiler and settings used, the calculations and rounding can be more exact or more 'lazy'.

_________________
- Retired.
Back to top
View user's profile Send private message Visit poster's website
Redouane
Master Cheater
Reputation: 3

Joined: 05 Sep 2013
Posts: 363
Location: Algeria

PostPosted: Sun May 17, 2015 6:40 am    Post subject: Reply with quote

atom0s wrote:
Lua, by default, treats all numbers (integer and decimal) as doubles. (8 bytes, floating point)


Since its Lua 5.3 , integers are represented as 64bit ints (by default) ,no?
Back to top
View user's profile Send private message
Dark Byte
Site Admin
Reputation: 457

Joined: 09 May 2003
Posts: 25262
Location: The netherlands

PostPosted: Sun May 17, 2015 7:12 am    Post subject: Reply with quote

yes, but since your result contains a decimal seperator it can't be handled as an integer, but as a float
_________________
Do not ask me about online cheats. I don't know any and wont help finding them.

Like my help? Join me on Patreon so i can keep helping
Back to top
View user's profile Send private message MSN Messenger
Display posts from previous:   
Post new topic   Reply to topic    Cheat Engine Forum Index -> Cheat Engine Lua Scripting All times are GMT - 6 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group

CE Wiki   IRC (#CEF)   Twitter
Third party websites