Develop Reference

C# and SIMD: High and low speedups. What is happening?

Introduction of the problem
I am trying to speed up the intersection code of a (2d) ray tracer that I am writing. I am using C# and the System.Numerics library to bring the speed of SIMD instructions.
The problem is that I am getting strange results, with over-the-roof speedups and rather low speedups. My question is, why is one over-the-roof whereas the other is rather low?
Context:
The RayPack struct is a series of (different) rays, packed in Vectors of System.Numerics.
The BoundingBoxPack and CirclePack struct is a single bb / circle, packed in vectors of System.Numerics.
The CPU used is an i7-4710HQ (Haswell), with SSE 4.2, AVX(2), and FMA(3) instructions.
Running in release mode (64 bit). The project runs in .Net Framework 472. No additional options set.
Attempts
I've already tried looking up whether some operations may or may not be properly supported (Take note: these are for c++. https://fgiesen.wordpress.com/2016/04/03/sse-mind-the-gap/ or http://sci.tuomastonteri.fi/programming/sse), but it seems that is not the case because the laptop I work on supports SSE 4.2.
In the current code, the following changes are applied:
Using more proper instructions (packed min, for example).
Not using the float * vector instruction (causes a lot of additional operations, see the assembly of the original).
Code ... snippets?
Apologies for the large amount of code, but I am not sure how we can discuss this concretely without this amount of code.
Code of Ray -> BoundingBox
public bool CollidesWith(Ray ray, out float t)
{
// https://gamedev.stackexchange.com/questions/18436/most-efficient-aabb-vs-ray-collision-algorithms
// r.dir is unit direction vector of ray
float dirfracx = 1.0f / ray.direction.X;
float dirfracy = 1.0f / ray.direction.Y;
// lb is the corner of AABB with minimal coordinates - left bottom, rt is maximal corner
// r.org is origin of ray
float t1 = (this.rx.min - ray.origin.X) * dirfracx;
float t2 = (this.rx.max - ray.origin.X) * dirfracx;
float t3 = (this.ry.min - ray.origin.Y) * dirfracy;
float t4 = (this.ry.max - ray.origin.Y) * dirfracy;
float tmin = Math.Max(Math.Min(t1, t2), Math.Min(t3, t4));
float tmax = Math.Min(Math.Max(t1, t2), Math.Max(t3, t4));
// if tmax < 0, ray (line) is intersecting AABB, but the whole AABB is behind us
if (tmax < 0)
{
t = tmax;
return false;
}
// if tmin > tmax, ray doesn't intersect AABB
if (tmin > tmax)
{
t = tmax;
return false;
}
t = tmin;
return true;
}
Code of RayPack -> BoundingBoxPack
public Vector<int> CollidesWith(ref RayPack ray, out Vector<float> t)
{
// ------------------------------------------------------- \\
// compute the collision. \\
Vector<float> dirfracx = Constants.ones / ray.direction.x;
Vector<float> dirfracy = Constants.ones / ray.direction.y;
Vector<float> t1 = (this.rx.min - ray.origin.x) * dirfracx;
Vector<float> t2 = (this.rx.max - ray.origin.x) * dirfracx;
Vector<float> t3 = (this.ry.min - ray.origin.y) * dirfracy;
Vector<float> t4 = (this.ry.max - ray.origin.y) * dirfracy;
Vector<float> tmin = Vector.Max(Vector.Min(t1, t2), Vector.Min(t3, t4));
Vector<float> tmax = Vector.Min(Vector.Max(t1, t2), Vector.Max(t3, t4));
Vector<int> lessThanZeroMask = Vector.GreaterThan(tmax, Constants.zeros);
Vector<int> greaterMask = Vector.LessThan(tmin, tmax);
Vector<int> combinedMask = Vector.BitwiseOr(lessThanZeroMask, greaterMask);
// ------------------------------------------------------- \\
// Keep track of the t's that collided. \\
t = Vector.ConditionalSelect(combinedMask, tmin, Constants.floatMax);
return combinedMask;
}
Code of Ray -> Circle
public bool Intersect(Circle other)
{
// Step 0: Work everything out on paper!
// Step 1: Gather all the relevant data.
float ox = this.origin.X;
float dx = this.direction.X;
float oy = this.origin.Y;
float dy = this.direction.Y;
float x0 = other.origin.X;
float y0 = other.origin.Y;
float cr = other.radius;
// Step 2: compute the substitutions.
float p = ox - x0;
float q = oy - y0;
float r = 2 * p * dx;
float s = 2 * q * dy;
// Step 3: compute the substitutions, check if there is a collision.
float a = dx * dx + dy * dy;
float b = r + s;
float c = p * p + q * q - cr * cr;
float DSqrt = b * b - 4 * a * c;
// no collision possible! Commented out to make the benchmark more fair
//if (DSqrt < 0)
//{ return false; }
// Step 4: compute the substitutions.
float D = (float)Math.Sqrt(DSqrt);
float t0 = (-b + D) / (2 * a);
float t1 = (-b - D) / (2 * a);
float ti = Math.Min(t0, t1);
if(ti > 0 && ti < t)
{
t = ti;
return true;
}
return false;
}
Code of RayPack -> CirclePack
Original, unedited, code can be found at: https://pastebin.com/87nYgZrv
public Vector<int> Intersect(CirclePack other)
{
// ------------------------------------------------------- \\
// Put all the data on the stack. \\
Vector<float> zeros = Constants.zeros;
Vector<float> twos = Constants.twos;
Vector<float> fours = Constants.fours;
// ------------------------------------------------------- \\
// Compute whether the ray collides with the circle. This \\
// is stored in the 'mask' vector. \\
Vector<float> p = this.origin.x - other.origin.x; ;
Vector<float> q = this.origin.y - other.origin.y;
Vector<float> r = twos * p * this.direction.x;
Vector<float> s = twos * q * this.direction.y; ;
Vector<float> a = this.direction.x * this.direction.x + this.direction.y * this.direction.y;
Vector<float> b = r + s;
Vector<float> c = p * p + q * q - other.radius * other.radius;
Vector<float> DSqrt = b * b - fours * a * c;
Vector<int> maskCollision = Vector.GreaterThan(DSqrt, zeros);
// Commented out to make the benchmark more fair.
//if (Vector.Dot(maskCollision, maskCollision) == 0)
//{ return maskCollision; }
// ------------------------------------------------------- \\
// Update t if and only if there is a collision. Take \\
// note of the conditional where we compute t. \\
Vector<float> D = Vector.SquareRoot(DSqrt);
Vector<float> bMinus = Vector.Negate(b);
Vector<float> twoA = twos * a;
Vector<float> t0 = (bMinus + D) / twoA;
Vector<float> t1 = (bMinus - D) / twoA;
Vector<float> tm = Vector.ConditionalSelect(Vector.LessThan(t1, t0), t1, t0);
Vector<int> maskBiggerThanZero = Vector.GreaterThan(tm, zeros);
Vector<int> maskSmallerThanT = Vector.LessThan(tm, this.t);
Vector<int> mask = Vector.BitwiseAnd(
maskCollision,
Vector.BitwiseAnd(
maskBiggerThanZero,
maskSmallerThanT)
);
this.t = Vector.ConditionalSelect(
mask, // the bit mask that allows us to choose.
tm, // the smallest of the t's.
t); // if the bit mask is false (0), then we get our original t.
return mask;
}
Assembly code
These can be found on pastebin. Take note that there is some boilerplate assembly from the benchmark tool. You need to look at the function calls.
BoundingBox(Pack): https://pastebin.com/RYMQdZMh
Circle(Pack) Tweaked: https://pastebin.com/YZHjc1vY
Circle(Pack) Original: https://pastebin.com/87nYgZrv
Benchmarking
I've been benchmarking the situation with BenchmarkDotNet.
Results for Circle / CirclePack (updated):
Method
Mean
Error
StdDev
Intersection
9.710 ms
0.0540 ms
0.0505 ms
IntersectionPacked
3.296 ms
0.0055 ms
0.0051 ms
Results for BoundingBox / BoundingBoxPacked:
Method
Mean
Error
StdDev
Intersection
24.269 ms
0.2663 ms
0.2491 ms
IntersectionPacked
1.152 ms
0.0229 ms
0.0264 ms
Due to AVX, a speedup of roughly 6x-8x is expected. The speedup of the boundingbox is significant, whereas the speedup of the circle is rather low.
Revisiting the question at the top: Why is one speedup over-the-roof and the other rather low? And how can the lower of the two (CirclePack) become faster?
Edit(s) with regard to Peter Cordes (comments)
Made the benchmark more fair: the single ray version does not early-branch-out as soon as the ray can no longer collide. Now the speedup is roughly 2.5x.
Added the assembly code as a separate header.
With regard to the square root: This does have impact, but not as much as it seems. Removing the vector square root reduces the total time with about 0.3ms. The single ray code now always performs the square root too.
Question about FMA (Fused Multiply Addition) in C#. I think it does for scalars (see Can C# make use of fused multiply-add?), but I haven't found a similar operation within the System.Numerics.Vector struct.
About a C# instruction for packed min: Yes it does. Silly me. I even used it already.

I am not going to try to answer the question about SIMD speedup, but provide some detailed comments on poor coding in the scalar version that carried over to the vector version, in a way that doesn't fit in an SO comment.
This code in Intersect(Circle) is just absurd:
// Step 3: compute the substitutions, check if there is a collision.
float a = dx * dx + dy * dy;
sum of squares -> a is guaranteed non-negative
float b = r + s;
float c = p * p + q * q - cr * cr;
float DSqrt = b * b - 4 * a * c;
This isn't the square root of D, it's the square of D.
// no collision possible! Commented out to make the benchmark more fair
//if (DSqrt < 0)
//{ return false; }
// Step 4: compute the substitutions.
float D = (float)Math.Sqrt(DSqrt);
sqrt has a limited domain. Avoiding the call for negative input doesn't just save the average cost of the square root, it prevents you from having to handle NaNs which are very, very slow.
Also, D is non-negative, since Math.Sqrt returns either the positive branch or NaN.
float t0 = (-b + D) / (2 * a);
float t1 = (-b - D) / (2 * a);
The difference between these two is t0 - t1 = D / a. The ratio of two non-negative variables is also non-negative. Therefore t1 is never larger.
float ti = Math.Min(t0, t1);
This call always selects t1. Computing t0 and testing which is larger is a waste.
if(ti > 0 && ti < t)
{
t = ti;
return true;
}
Recalling that ti is always t1, and a is non-negative, the first test is equivalent to -b - D > 0 or b < -D.
In the SIMD version, Vector.SquareRoot documentation does not describe the behavior when inputs are negative. And Vector.LessThan does not describe the behavior when inputs are NaN.

Finding the true anomaly from state vectors

I'm attempting to convert from state vectors (position and speed) into Kepler elements, however I'm running into problems where a negative velocity or position will give me wrong results when trying to calculate true anomaly.
Here are the different ways I'm trying to calculate the True Anomaly:
/// <summary>
/// https://en.wikipedia.org/wiki/True_anomaly#From_state_vectors
/// </summary>
public static double TrueAnomaly(Vector4 eccentVector, Vector4 position, Vector4 velocity)
{
var dotEccPos = Vector4.Dot(eccentVector, position);
var talen = eccentVector.Length() * position.Length();
talen = dotEccPos / talen;
talen = GMath.Clamp(talen, -1, 1);
var trueAnomoly = Math.Acos(talen);
if (Vector4.Dot(position, velocity) < 0)
trueAnomoly = Math.PI * 2 - trueAnomoly;
return trueAnomoly;
}
//sgp = standard gravitational parameter
public static double TrueAnomaly(double sgp, Vector4 position, Vector4 velocity)
{
var H = Vector4.Cross(position, velocity).Length();
var R = position.Length();
var q = Vector4.Dot(position, velocity); // dot product of r*v
var TAx = H * H / (R * sgp) - 1;
var TAy = H * q / (R * sgp);
var TA = Math.Atan2(TAy, TAx);
return TA;
}
public static double TrueAnomalyFromEccentricAnomaly(double eccentricity, double eccentricAnomaly)
{
var x = Math.Sqrt(1 - Math.Pow(eccentricity, 2)) * Math.Sin(eccentricAnomaly);
var y = Math.Cos(eccentricAnomaly) - eccentricity;
return Math.Atan2(x, y);
}
public static double TrueAnomalyFromEccentricAnomaly2(double eccentricity, double eccentricAnomaly)
{
var x = Math.Cos(eccentricAnomaly) - eccentricity;
var y = 1 - eccentricity * Math.Cos(eccentricAnomaly);
return Math.Acos(x / y);
}
Edit: another way of doing it which Spectre pointed out:
public static double TrueAnomaly(Vector4 position, double loP)
{
return Math.Atan2(position.Y, position.X) - loP;
}
Positions are all relative to the parent body.
These functions all agree if position.x, position.y and velocity.y are all positive.
How do I fix these so that I get a consistent results when position and velocity are negitive?
Just to clarify: My angles appear to be sort of correct, just pointing in the wrong quadrant depending on the position and or velocity vectors.
Yeah so I was wrong, the above all do return the correct values after all.
So I found an edge case where most of the above calculations fail.
Given position and velocity:
pos = new Vector4() { X = -0.208994076275941, Y = 0.955838328099748 };
vel = new Vector4() { X = -2.1678187689294E-07, Y = -7.93096769486992E-08 };
I get some odd results, ie ~ -31.1 degrees, when I think it should return ` 31.1 (non negative). one of them returns ~ 328.8.
However testing with this position and velocity the results apear to be ok:
pos = new Vector4() { X = -0.25, Y = 0.25 };
vel = new Vector4() { X = Distance.KmToAU(-25), Y = Distance.KmToAU(-25) };
See my answer for extra code on how I'm testing and the math I'm using for some of the other variables.
I'm going around in circles on this one. this is a result of a bug in my existing code that shows up under some conditions but not others.
I guess the real question now is WHY am I getting different results with position/velocity above that don't match to my expectations or each other?

Assuming 2D case... I am doing this differently:
compute radius of semi axises and rotation
so you need to remember whole orbit and find 2 most distant points on it that is major axis a. The minor axis b usually is 90 deg from major axis but to be sure just fins 2 perpendicularly most distant points on your orbit to major axis. So now you got both semi axises. The initial rotation is computed from the major axis by atan2.
compute true anomaly E
so if center is x0,y0 (intersection of a,b or center point of both) initial rotation is ang0 (angle of a) and your point on orbit is x,y then:
E = atan2(y-y0,x-x0) - ang0
However in order to match Newton/D'Alembert physics to Kepler orbital parameters you need to boost the integration precision like I did here:
Is it possible to make realistic n-body solar system simulation in matter of size and mass?
see the [Edit3] Improving Newton D'ALembert integration precision even more in there.
For more info and equations see:
Solving Kepler's equation
[Edit1] so you want to compute V I see it like this:
As you got your coordinates relative to parent you can assume they are already in focal point centered so no need for x0,y0 anymore. Of coarse if you want high precision and have more than 2 bodies (focal mass + object + proximity object(s) like moons) then the parent mass will no longer be in focal point of orbit but close to it ... and to remedy you need to use real focal point position so x0,y0 again... So how to do it:
compute center point (cx,cy) and a,b semi axises
so its the same as in previous text.
compute focal point (x0,y0) in orbit axis aligned coordinates
simple:
x0 = cx + sqrt( a^2 + b^2 );
y0 = cy;
initial angle ang0 of a
let xa,ya be the intersection of orbit and major axis a on the side with bigger speeds (near parent object focus). Then:
ang0 = atan2( ya-cy , xa-cx );
and finally the V fore any of yours x,y
V = atan2( y-y0 , x-x0 ) - ang0;

Ok so on further testing it appears my original calcs do all return the correct values, however when I was looking at the outputs I was not taking the LoP into account and basically not recognizing that 180 is essentially the same angle as -180.
(I was also looking at the output in radians and just didn't see what should have been obvious)
Long story short, I have a bug I thought was in this area of the code and got lost in the weeds.
Seems I was wrong above. see OP for edge case.
Here's some code I used to test these,
I used variations of the following inputs:
pos = new Vector4() { X = 0.25, Y = 0.25 };
vel = new Vector4() { X = Distance.KmToAU(-25), Y = Distance.KmToAU(25) };
And tested them with the following
double parentMass = 1.989e30;
double objMass = 2.2e+15;
double sgp = GameConstants.Science.GravitationalConstant * (parentMass + objMass) / 3.347928976e33;
Vector4 ev = OrbitMath.EccentricityVector(sgp, pos, vel);
double e = ev.Length();
double specificOrbitalEnergy = Math.Pow(vel.Length(), 2) * 0.5 - sgp / pos.Length();
double a = -sgp / (2 * specificOrbitalEnergy);
double ae = e * a;
double aop = Math.Atan2(ev.Y, ev.X);
double eccentricAnomaly = OrbitMath.GetEccentricAnomalyFromStateVectors(pos, a, ae, aop);
double aopD = Angle.ToDegrees(aop);
double directAngle = Math.Atan2(pos.Y, pos.X);
var θ1 = OrbitMath.TrueAnomaly(sgp, pos, vel);
var θ2 = OrbitMath.TrueAnomaly(ev, pos, vel);
var θ3 = OrbitMath.TrueAnomalyFromEccentricAnomaly(e, eccentricAnomaly);
var θ4 = OrbitMath.TrueAnomalyFromEccentricAnomaly2(e, eccentricAnomaly);
var θ5 = OrbitMath.TrueAnomaly(pos, aop);
double angleΔ = 0.0000001; //this is the "acceptable" amount of error, really only the TrueAnomalyFromEccentricAnomaly() calcs needed this.
Assert.AreEqual(0, Angle.DifferenceBetweenRadians(directAngle, aop - θ1), angleΔ);
Assert.AreEqual(0, Angle.DifferenceBetweenRadians(directAngle, aop - θ2), angleΔ);
Assert.AreEqual(0, Angle.DifferenceBetweenRadians(directAngle, aop - θ3), angleΔ);
Assert.AreEqual(0, Angle.DifferenceBetweenRadians(directAngle, aop - θ4), angleΔ);
Assert.AreEqual(0, Angle.DifferenceBetweenRadians(directAngle, aop - θ5), angleΔ);
and the following to compare the angles:
public static double DifferenceBetweenRadians(double a1, double a2)
{
return Math.PI - Math.Abs(Math.Abs(a1 - a2) - Math.PI);
}
And eccentricity Vector found thus:
public static Vector4 EccentricityVector(double sgp, Vector4 position, Vector4 velocity)
{
Vector4 angularMomentum = Vector4.Cross(position, velocity);
Vector4 foo1 = Vector4.Cross(velocity, angularMomentum) / sgp;
var foo2 = position / position.Length();
return foo1 - foo2;
}
And EccentricAnomaly:
public static double GetEccentricAnomalyFromStateVectors(Vector4 position, double a, double linierEccentricity, double aop)
{
var x = (position.X * Math.Cos(-aop)) - (position.Y * Math.Sin(-aop));
x = linierEccentricity + x;
double foo = GMath.Clamp(x / a, -1, 1); //because sometimes we were getting a floating point error that resulted in numbers infinatly smaller than -1
return Math.Acos(foo);
}
Thanks to Futurogogist and Spektre for their help.

I am assuming you are working in two dimensions?
Two dimensional vectors of position p and velocity v. The constant K is the the product of the gravitational constant and the mass of the gravity generating body. Calculate the eccentricity vector
eccVector = (dot(v, v)*p - dot(v, p)*v) / K - p / sqrt(dot(p, p));
eccentricity = sqrt(dot(eccVector, eccVector));
eccVector = eccVector / eccentricity;
b = { - eccVector.y, eccVector.x}; //unit vector perpendicular to eccVector
r = sqrt(dot(p, p));
cos_TA = dot(p, eccVector) / r; \\ cosine of true anomaly
sin_TA = dot(p, b) / r; \\ sine of true anomaly
if (sin_TA >= 0) {
trueAnomaly = arccos(cos_TA);
}
else if (sin_TA < 0){
trueAnomaly = 2*pi - arccos(cos_TA);
}

Diagonal sweep and orthogonal distance of a point to diagonal

I have a problem where I'm required to find the maximum number of points that are less than or equal to a given distance D to a line drawn in a two-dimensional Euclidean plane. To solve this I wrote the algorithms that would compute a possible maximum if the line was orthogonal to either the x-axis or the y-axis. My problem is when only a diagonal line would yield the maximum number of points.
Given the constraints that both x and y have a minimum value of -1000000 and maximum of 1000000. I wrote the following algorithm to try and find out the maximum. I don't seem to be getting the right answer. Could someone please guide me on where I am going wrong. I've tried drawing a regression line as well but that used vertical distance which did not work for my purposes. Maybe I'm going all wrong and this problem can be solved as an optimization problem. Anyways' any help with a descent explanation is much appreciated.
// diagonal sweep
for (int degree = 1; degree < 180; degree++) if (degree % 90 != 0)
{
int k = 1, degrees = degree;
double x1 = -1000000, x2 = 1000000;
if (degree > 90 && degree < 180)
{
degrees = 180 - degrees;
k = -1;
}
//slope
double m1 = Math.Tan(Math.PI * degrees * k / 180.0);
//Point A
Point A = new Point(x1, m1 * x1);
//Point B
Point B = new Point(x2, m1 * x2);
for (int i = 0; i < x.Length; i++)
{
//Point P = household that needs power
Point P = new Point(x[i], y[i]);
double normalLength = Math.Sqrt((B.X - A.X) * (B.X - A.X) + (B.Y - A.Y) * (B.Y - A.Y));
double segmentLength = 1d * Math.Abs((P.X - A.X) * (B.Y - A.Y) - (P.Y - A.Y) * (B.X - A.X)) / normalLength;
if (segmentLength <= D)
tempCnt++;
}
maxConnections = Math.Max(maxConnections, tempCnt);
tempCnt = 0;
}
return maxConnections;

If you want to define this problem as an optimization problem, you should do it as follows, but it doesn't seem to me this optimization problem is solveable efficiently as is.
maximize: x_1 + x_2 + ... + x_n + 0*a + 0*b + 0*c
s.t.
x_i * d(p_i, line(a,b,c))/ MAX_DISTANCE <= 1
x_i is in {0,1}
Explanation:
x_i are inclusion variables - can get a value of 0 / 1 , and it indicates if the point p_i is in the required distance from the line.
a,b,c are the parameters for the line: ax + by + c = 0
The idea is to maximize the sum of included points, such that each included point is in the desired range. This is represented by the constraint, if x_i=0 - there is no restriction on the point p_i, as the constraint is always satisfied. Otherwise, x_i=1, and you need the distance from the line (let it be d) satisfy 1* d/MAX_DISTANCE <= 1 - which is exactly what you want.
Though I don't think there is an optimal efficient solution to this optimization problem, you might want to try some heuristical solutions for this optiization - such as Genetic Algorithms or Hill Climbing
As a side note, my gut says this problem is NP-Complete, though I have no proof for it yet - and will update this part of the answer if I (or someone else) can come up with a reduction/polynomial solution.

Storing 2 Single Floats in One

I've read several links discussing storing 2 or 3 floats in one float. Here's an example:
Storing two float values in a single float variable
and another:
http://uncommoncode.wordpress.com/2012/11/07/float-packing-in-shaders-encoding-multiple-components-in-one-float/
and yet another:
decode rgb value to single float without bit-shift in glsl
I've seen others but all of them use the same principle. If you want to encode x and y, they multiply y by some factor and then add x to it. Well this makes since on paper, but I don't understand how in the world it can work when stored to a floating value. Floating values only have 7 significant digits. If you add a big number and a small number, the small number is just truncated and lost. The precision only shows the value of the big number.
Since everyone seems to prescribe the same method, I tried it myself and it did exactly what I thought it would do. When I decoded the numbers, the number that wasn't multiplied turned out as 0.0. It was completely lost in the encoded float.
Here's an example of some MaxScript I tried to test it:
cp = 256.0 * 256.0
scaleFac = 16777215
for i = 1 to 20 do (
for j = 1 to 20 do (
x = (i as float / 20.01f) as float;
y = (j as float / 20.01f) as float;
xScaled = x * scaleFac;
yScaled = y * scaleFac;
f = (xScaled + yScaled * cp) as float
print ("x[" + xScaled as string + "] y[" + yScaled as string + "]" + " e[" + f as string + "]")
dy = floor(f / cp)
dx = (f - dy * cp)
print ("x[" + dx as string + "] y[" + dy as string + "]" + " e[" + f as string + "]")
)
)
dx is 0.0 everytime. Can anyone shed some light on this? NOTE: It doesn't matter whether I make cp = 128, 256, 512 or whatever. It still gives me the same types of results.

This method works for storing two integers. You're effectively converting your floating point numbers to large integers by multiplying by scaleFac, which is good, but it would be better to make it explicit with int(). Then you need to make sure of two things: cp is greater than the largest number you're working with (scaleFac), and the square of cp is small enough to fit into a floating point number without truncation (about 7 digits for a single precision float).

Here is a working code in C to pack two floats into one float and unpack them.
You should change scaleFactor and cp parameters as according to your possible value ranges (yourBiggestNumber * scaleFactor < cp). It is a precision battle. Try printing a few results to find good values for your case. The example below allows floats in [0 to 1) range.
#include <math.h>
/* yourBiggestNumber * scaleFactor < cp */
double scaleFactor = 65530.0;
double cp = 256.0 * 256.0;
/* packs given two floats into one float */
float pack_float(float x, float y) {
int x1 = (int) (x * scaleFactor);
int y1 = (int) (y * scaleFactor);
float f = (y1 * cp) + x1;
return f;
}
/* unpacks given float to two floats */
int unpack_float(float f, float* x, float* y){
double dy = floor(f / cp);
double dx = f - (dy * cp);
*y = (float) (dy / scaleFactor);
*x = (float) (dx / scaleFactor);
return 0;
}

It will work only if your individual floats are small enough to be packed into one float location.
So you can pack 2 numbers by "dividing" this into two to store 2 numbers that can be represented by half the space.

Here's the code I use for packing and unpacking floats. It works by packing first float (0..1) into the first four bytes of a 8-bit (0..256) number, and the next float into the remaining 4 bits. The resulting numbers have 16 possible combinations each (2^4). In some cases this is good enough:
private float PackFloatsInto8Bits( float v1, float v2 )
{
var a = Mathf.Round( v1 * 15f );
var b = Mathf.Round( v2 * 15f );
var bitShiftVector = new Vector2( 1f/( 255f/16f ), 1f/255f );
return Vector2.Dot( new Vector2( a, b ), bitShiftVector );
}
private Vector2 UnpackFloatsFrom8Bits( float input )
{
float temp = input * 15.9375f;
float a = Mathf.Floor(temp) / 15.0f;
float b = Frac( temp ) * 1.0667f;
return new Vector2(a, b);
}

How to calculate turning direction

I have a three lat-lon coordinates that make up two line segment A to B to C. I also found a function that can return north-bearing of a line segment A-B or B-C in -180 to 180 manner. However, I'm having trouble to determine when a car reaches from A to B, should it turn right or left to continue to C.

EDIT: Previous answer was wrong. now this is the correct
public Direction GetDirection(Point a, Point b, Point c)
{
double theta1 = GetAngle(a, b);
double theta2 = GetAngle(b, c);
double delta = NormalizeAngle(theta2 - theta1);
if ( delta == 0 )
return Direction.Straight;
else if ( delta == Math.PI )
return Direction.Backwards;
else if ( delta < Math.PI )
return Direction.Left;
else return Direction.Right;
}
private Double GetAngle(Point p1, Point p2)
{
Double angleFromXAxis = Math.Atan ((p2.Y - p1.Y ) / (p2.X - p1.X ) ); // where y = m * x + K
return p2.X - p1.X < 0 ? m + Math.PI : m ); // The will go to the correct Quadrant
}
private Double NormalizeAngle(Double angle)
{
return angle < 0 ? angle + 2 * Math.PI : angle; //This will make sure angle is [0..2PI]
}

Edited to fix the over 180 issue, also now supports U-Turns.
const int THRESHOLD = 0;
Direction TurnLeftOrRight(Point A, Point B, Point C)
{
int angle = ToAngle(B,C) - ToAngle(A,B);
if((angle > THRESHOLD && angle < 180 - THREASHOLD) || angle < -180 - THREASHOLD)
return Direction.Right;
else if ((angle < 0 - THREASHOLD && angle > -180 + THREASHOLD) || angle > 180 + THREASHOLD)
return Direction.Left;
else if (angle >= 0 - THREASHOLD && angle <= THREASHOLD)
return Direction.Straight
else
return Direction.UTurn;
}
You could also do tolerances between left right and strait just change the first angle > 0 to angle > 45 and the second one to angle < -45

I think your life will be simpler if you use the vector cross product.
Although strictly speaking cross product is defined only for 3D vectors, for 2D vectors p=(px,py) and q=(qx,qy), you can think of their cross product p×q as as pxqy - pyqx. This latter number will be positive if p is clockwise from q, and negative if p is counter-clockwise from q. It will be zero if p and q are parallel - i.e. point in the same or opposite directions.
In your case, you are using (lat,lon). The equivalent in (x,y) coordinates is (-lon, lat), so if you have two vectors (lat1,lon1) and (lat2,lon2), you want to calculate (-lon1, lat1)×(-lon2, lat2), which comes out to lat1*lon2-lon1*lat2.
If this number is zero, you can use the dot product to tell whether the direction is straight or a U-turn.
So your code could look like this, assuming Points and Vectors are written in (lat, lon) form (the code would be a little different if they are in x and y):
public Direction GetTurnDirection(Point A, Point B, Point C)
{
Vector v1 = B - A ;
Vector v2 = C - B ;
double cross = v1.lat*v2.lon - v1.lon*v2.lat ;
if (cross > 0) { return Direction.Left ; }
if (cross < 0) { return Direction.Right ; }
double dot = v1.lat*v2.lat + v1.lon*v2.lon ;
if (dot > 0) { return Direction.Straight ; }
return Direction.UTurn ;
}

If AB is the bearing of B from A, and BC that of C from B, the
turn angle is remainder( BC-AB, 360.0); (assuming degrees). If this
is positive the turn is to the right. In your example remainder( BC-AB, 360.0)
is remainder(271,360) = -89.