Floating Point Hacks

Completely useless, but fun nevertheless.

_{Equations for a "fast" method.}

Table of Contents

• Floating Point Hacks
  • Usage
    • For Contributors
    • Pow
    • Exp
    • Log
    • Geometric Mean
  • Justification
    • Prelude
    • Arbitrary Powers
    • Exp
      • Differentiating the l2f and f2l functions.
      • A Tale of Two Functions
      • Exp, redux.
    • Log
    • Geometric Mean

This repository contains a set of procedures to compute numerical methods in the vein of the fast inverse root method. In particular, we will generate code that

1. Computes rational powers () to an arbitrary precision.
2. Computes irrational powers () to within 10% relative error.
3. Computes to within 10% relative error.
4. Computes to within 10% relative error.
5. Computes the geometric mean of an std::array quickly to within 10% error.

Additionally, we will do so using mostly just integer arithmetic.

Usage

You can use everything in floathacks by including hacks.h

#include <floathacks/hacks.h>
using namespace floathacks; // Comment this out if you don't want your top-level namespace to be polluted

For Contributors

This document is compiled from READOTHER.md by readme2tex. Make sure that you pip install readme2tex. You can run

python -m readme2tex --output README.md --branch svgs

to recompile these docs.

Pow

To generate an estimation for , where is any floating point number, you can run

float approximate_root = fpow<FLOAT(0.12345)>::estimate(x);

Since estimates of pow can be refined into better iterates (as long as c is "rational enough"), you can also compute a more exact result via

float root = pow<FLOAT(0.12345), n>(x);

where n is the number of newton iterations to perform. The code generated by this template will unroll itself, so it's relatively efficient.

However, the optimized code does not let you use it as a constexpr or where the exponent is not constant. In those cases, you can use consts::fpow(x, c) and consts::pow(x, c, iterations = 2) instead:

float root = consts::pow(x, 0.12345, n);

Note that the compiler isn't able to deduce the optimal constants in these cases, so you'll incur additional penalties computing the constants of the method.

Exp

You can also compute an approximation of with

float guess = fexp(x);

Unfortunately, since there are no refinement methods available for exponentials, we can't do much with this result if it's too coarse for your needs. In addition, due to overflow, this method breaks down when x approaches 90.

Log

Similarly, you can also compute an approximation of with

float guess = flog(x);

Again, as is with the case of fexp, there are no refinement methods available for logarithms either.

All of the f*** methods above have bounded relative errors of at most 10%. The refined pow method can be made to give arbitrary precision by increasing the number of refinement iterations. Each refinement iteration takes time proportional to the number of digits in the floating point representation of the exponent. Note that since floats are finite, this is bounded above by 32 (and more tightly, 23).

Geometric Mean

You can compute the geometric mean () of a std::array<float, n> with

float guess = fgmean<3>({ 1, 2, 3 });

This can be refined, but you typically do not care about the absolute precision of a mean-like statistic. To refine this, you can run Newton's method on . As far as I am aware, this is also an original method.

Justification

Prelude

The key ingredient of these types of methods is the pair of transformations and .

• takes a IEEE 754 single precision floating point number and outputs its "machine" representation. In essence, it acts like

    unsigned long f2l(float x) {
      union {float fl; unsigned long lg;} lens = { x };
      return lens.lg;
    }
  

• takes an unsigned long representing a float and returns a IEEE 754 single precision floating point number. It acts like

    float l2f(unsigned long z) {
      union {float fl; unsigned long lg;} lens = { z };
      return lens.fl;
    }
  

So for example, the fast inverse root method:

  union {float fl; unsigned long lg;} lens = { x };
  lens.lg = 0x5f3759df - lens.lg / 2;
  float y = lens.fl;

can be equivalently expressed as

In a similar vein, a fast inverse cube-root method is presented at the start of this page.

We will justify this in the next section.

Arbitrary Powers

We can approximate any using just integer arithmetic on the machine representation of . To do so, compute

where . In general, any value of bias, as long as it is reasonably small, will work. At bias = 0, the method computes a value whose error is completely positive. Therefore, by increasing the bias, we can shift some of the error down into the negative plane and halve the error.

As seen in the fast-inverse-root method, a bias of -0x5c416 tend to work well for pretty much every case that I've tried, as long as we tack on at least one Newton refinement stage at the end. It works well without refinement as well, but an even bias of -0x5c000 works even better.

Why does this work? See these slides for the derivation. In particular, the fast inverse square-root is a subclass of this method.

Exp

We can approximate up to using a similar set of bit tricks. I'll first give its equation, and then give its derivations. As far as I am aware, these are original. However, since there are no refinement methods for the computation of , there is practically no reason to ever resort to this approximation unless you're okay with 10% error.

Here, is the machine epsilon for single precision, and it is computed by .

To give a derivation of this equation, we'll need to borrow a few mathematical tools from analysis. In particular, while l2f and f2l have many discontinuities ( of them to be exact), it is mostly smooth. This carries over to its "rate-of-change" as well, so we will just pretend that it has mostly smooth derivatives everywhere.

Differentiating the l2f and f2l functions.

Consider the function

where the equality is a consequence of the chain-rule, assuming that f2l is differentiable at the particular value of . Now, this raises an interesting question: What does it mean to take a derivative of ?

Well, it's not all that mysterious. The derivative of f2l is just the rate at which a number's IEEE 754 machine representation changes as we make small perturbations to a number. Unfortunately, while it might be easy to compute this derivative as a numerical approximation, we still don't have an approximate form for algebraic manipulation.

While might be difficult to construct, we can fair much better with its sibling, . Now, the derivative is the rate that a float will change given that we make small perturbations to its machine representation. However, since its machine representations are all bit-vectors, it doesn't make sense to take a derivative here since we can't make these perturbations arbitrarily small. The smallest change we can make is to either add or subtract one. However, if we just accept our fate, then we can define the "derivative" as the finite difference

where

Here, equality holds when is a perfect power of (including fractions of the form ).

Therefore,

From here, we also have

A Tale of Two Functions

Given , antidifferentiating both sides gives

Similarly, since satisfies , we have

This makes sense, since we'd like these two functions to be inverses of each other.

Exp, redux.

Consider

which suggests that .

Since we would like , we can impose the boundary condition

which gives . However, while this method gives bounded relative error, in its unbiased form this is pretty off the mark for general purposes (it approximates some other ). Instead, we can add in an unbiased form:

where, empirically, gives a good approximation. Notice that the we've chosen is close to , which is what we need to transform to . In particular, for all , the , , and relative error is always below 10%.

Log

In a similar spirit, we can use the approximation

to derive

Imposing a boundary condition at gives , so we should expect

However, this actually computes some other logarithm , and we'll have to, again, unbias this term

where the term came from the fact that the base computation approximates the 2-logarithm. Empirically, I've found that a bias of works well. In particular, for all , the , , and relative error is always below 10%.

Geometric Mean

There's a straightforward derivation of the geometric mean. Consider the approximations of and , we can refine them as

Therefore, a bit of algebra will show that

which reduces to the equation for the geometric mean.

Notice that we just add a series of integers, followed by an integer divide, which is pretty efficient.

For more information on how the constant () is derived for the cube-root, visit http://www.bullshitmath.lol/.

Equations rendered with readme2tex.