Hi,
I was haunting one strange problem with my compounds, that worked fine, but for some isolated combination of value everything was broken.
It was a strange ride, but i reduced the problem to the simplest form and i am asking you if you find it correct or not.
I am doing an integer division. there is not a direct node for it, so i have a compound that converts integer to scalar and uses "divide by scalar" .the result is later
used to return both division and "modulo".
the problem is the conversion int->scalar. because of the precision for some numbers A div A is 0 as you can see on the attached screenshot.
i was expecting precision problems for bigger numbers....
so,what is the most elegant way to compute integer division (with a remainder - if it is correct english term for it) ?
"divide by scalar" and precision problems
- xsisupport
- Posts: 713
- Joined: 09 Jun 2009, 11:02
- Location: Montreal Canada
- Contact:
Re: "divide by scalar" and precision problems
You're always going to get the floor when you let the nodes do the implicit conversion to integer.
Work with scalars and then at the end decide whether to round or floor.
Here's an example Integer Division compound by Guillaume Laforge:
http://screencast.com/t/he5KwoDElvC
Work with scalars and then at the end decide whether to round or floor.
Here's an example Integer Division compound by Guillaume Laforge:
http://screencast.com/t/he5KwoDElvC
Who is online
Users browsing this forum: No registered users and 24 guests