Location: BlogsDevelopments
Posted by: TM Central	9/18/2007 7:15 AM
It is a common fallacy that computers "think" in numbers - they do not. Many others believe they process binary - 1s or 0s - and this would be correct. But have you ever stopped to think how then they "convert" those numbers to binary to story and process mathematics? While this is a huge subject and one more suited for a Computer Science class than a blog, we got on this subject in depth when modifying a client's source code. You see, the previous programmer was using several numeric types interchangeably and when the exact same inputted numbers were compared they did not "match", and since this was a financial application it was more than a bit critical that they balance! Long story short, many of the various numeric datatypes look at and apply different memory reservations and processing to the same number - the most common of these is how each one handles fractions (i.e., non-whole numbers) since they must be ultimately represented to the processor as a binary. For any junior programmers or "script-kiddies" out there, simply ALWAYS USE THE SAME NUMERIC DATATYPE FOR THE SAME PROCESSING! Lots of great stuff out there on how these datatypes, but here are a few links for .NET that get more into how .NET's particular datatypes are handled: http://www.yoda.arachsys.com/csharp/floatingpoint.html http://www.yoda.arachsys.com/csharp/decimal.html
Permalink |  Trackback