Related
When ever I input these coordinates I get a wrong output.
static double ReadCoordinateFromConsole(double lat1, double lon1, double
lat2, double lon2)
{
var R = 6371; // Radius of the earth in km
var dLat = deg2rad(lat2 - lat1);
var dLon = deg2rad(lon2 - lon1);
var a =
Math.Sin(dLat / 2) * Math.Sin(dLat / 2) +
Math.Cos(deg2rad(lat1)) * Math.Cos(deg2rad(lat2)) *
Math.Sin(dLon / 2) * Math.Sin(dLon / 2);
var c = 2 * Math.Atan2(Math.Sqrt(a), Math.Sqrt(1 - a));
var d = R * c; // Distance in km
return d;
}
static double deg2rad(double deg)
{
return deg * (Math.PI / 180);
}
Then in my function where I input some coordinates. 41.507483 -99.436554 38.504048 -98.315949. These coordinates should equal about 347 but instead, I get the output 7022,88 which is wrong and I have no clue as to why.
static double ReadDoubleFromConsole(string msg)
while (true)
{
Console.Write(msg);
string test = Console.ReadLine();
string[] words = test.Split(' ');
bool inputContainsNumber = Regex.IsMatch(words[0], #"^-*[0-9,\.]+$");
bool inputContainsNumber2 = Regex.IsMatch(words[1], #"^-*[0-9,\.]+$");
bool inputContainsNumber3 = Regex.IsMatch(words[2], #"^-*[0-9,\.]+$");
bool inputContainsNumber4 = Regex.IsMatch(words[3], #"^-*[0-9,\.]+$");
if(inputContainsNumber && inputContainsNumber2 && inputContainsNumber3
&& inputContainsNumber4)
{
double test1 = double.Parse(words[0]);
double test2 = double.Parse(words[1]);
double test3 = double.Parse(words[2]);
double test4 = double.Parse(words[3]);
double test5 = ReadCoordinateFromConsole(test1, test2, test3,
test4);
return test5;
}
Console.WriteLine("hmm, doesn't look correct - try again");
}
}
As you've been shown by the other answer your algorithm is correct - your parsing of the user input is wrong!
Here's the demonstration: http://rextester.com/IEEA93176
And the reason is, that in the culture that rextester runs in the , is used as a decimal separator - and I assume its the same in your environment. When you use double.parse("41.1234") in a culture expecting a , you get the value 411234 not 41.1234.
One fix is to force the culture
System.Threading.Thread.CurrentThread.CurrentCulture = new CultureInfo("en-GB");
Live demo(working): http://rextester.com/KALRN89806
There's nothing wrong with your implementation of the Haversine formula - it generates the correct output.
The following prints 347.328348039426:
using System;
namespace ConsoleApp1
{
class Program
{
static void Main()
{
Console.WriteLine(ReadCoordinateFromConsole(41.507483, -99.436554, 38.504048, -98.315949));
}
static double ReadCoordinateFromConsole(double lat1, double lon1, double
lat2, double lon2)
{
var R = 6371; // Radius of the earth in km
var dLat = deg2rad(lat2 - lat1);
var dLon = deg2rad(lon2 - lon1);
var a =
Math.Sin(dLat / 2) * Math.Sin(dLat / 2) +
Math.Cos(deg2rad(lat1)) * Math.Cos(deg2rad(lat2)) *
Math.Sin(dLon / 2) * Math.Sin(dLon / 2);
var c = 2 * Math.Atan2(Math.Sqrt(a), Math.Sqrt(1 - a));
var d = R * c; // Distance in km
return d;
}
static double deg2rad(double deg)
{
return deg * (Math.PI / 180);
}
}
}
Therefore you must look elsewhere for your error. I suspect you're not feeding the correct values to the method - I recommend that you single-step your code in the debugger to determine what's happening.
Incidentally, is there any reason that you're not using the .Net GeoCoordinate class to calculate this? For example:
using System;
using System.Device.Location;
namespace ConsoleApp1
{
class Program
{
static void Main()
{
var a = new GeoCoordinate(41.507483, -99.436554);
var b = new GeoCoordinate(38.504048, -98.315949);
Console.WriteLine(a.GetDistanceTo(b)/1000.0);
}
}
}
This prints 347.628192006498.
I'm working on a Midpoint Riemann Sum program, and it finds the integral of a randomly generated function called f.
Here's what wrote:
public static double FindIntegral (double start, double end, function f)
{
double sum = 0;
double stepsize = 1E-2;
int numSteps = (int)((end - start) / stepsize);
for (int i = 0; i < numSteps; i++)
{
sum += f(start + (stepsize * (i + 0.5)));
}
return sum * stepsize;
}
The function returns numbers that are too low (I have a reliable checking mechanism).
I put in x^3 for f, and I got the right answer. I tried a couple of more integrable functions and got a good answer. But somehow once I put in f it doesn't work.
I got the math formula for "Riemann Midpoint Sum" from here.
My implementation below seems to get the right answer (using the example function on the page). I used a class because 1) I could make the algorithm work specifying either the step size or the number of rectangles (I preferred the latter) and 2) I didn't see any reason to hard-code either into the algorithm.
As it turns out your code seemed to work just fine (see below); Make sure the code you have here in your question is what you're executing and make sure your expected result is accurate and that you're supplying good inputs (i.e. you don't have start and end backwards or the wrong function f or something). In other words what you provided in your question looks fine. Note double is approximate in C# (floating point arithmetic, in general) so to compare equality you can't use == unless you want exact if you're using unit tests or something.
public class Program
{
public static void Main()
{
function f = x => 50 / (10 + x * x);
// 9.41404285216233
Console.Out.WriteLine(new RiemannMidpointSum(6).FindIntegral(1, 4, f));
// 9.41654853716462
Console.Out.WriteLine(new RiemannMidpointSum(1E-2).FindIntegral(1, 4, f));
// 9.41654853716462
Console.Out.WriteLine(Program.FindIntegral(1, 4, f));
}
// This is your function.
public static double FindIntegral (double start, double end, function f)
{
double sum = 0;
double stepsize = 1E-2;
int numSteps = (int)((end - start) / stepsize);
for (int i = 0; i < numSteps; i++)
{
sum += f(start + (stepsize * (i + 0.5)));
}
return sum * stepsize;
}
}
public delegate double function(double d);
public class RiemannMidpointSum
{
private int? _numberOfRectangles;
private double? _widthPerRectangle;
public RiemannMidpointSum(int numberOfRectangles)
{
// TODO: Handle non-positive input.
this._numberOfRectangles = numberOfRectangles;
}
public RiemannMidpointSum(double widthPerRectangle)
{
// TODO: Handle non-positive input.
this._widthPerRectangle = widthPerRectangle;
}
public double FindIntegral(double a, double b, function f)
{
var totalWidth = b - a;
var widthPerRectangle = this._widthPerRectangle ?? (totalWidth / this._numberOfRectangles.Value);
var numberOfRectangles = this._numberOfRectangles ?? ((int)Math.Round(totalWidth / this._widthPerRectangle.Value, 0));
double sum = 0;
foreach (var i in Enumerable.Range(0, numberOfRectangles))
{
var rectangleMidpointX = a + widthPerRectangle * i + widthPerRectangle / 2;
var rectangleHeightY = f(rectangleMidpointX);
var rectangleArea = widthPerRectangle * rectangleHeightY;
sum += rectangleArea;
}
return sum;
}
}
I want to convert NEW ZEALAND MAP GRID coordinates i.e. Northing and Easting into WGS84 coordinates i.e. Latitude and Longitude.
I have searched on internet but there is no proper explanation of how to do this, or an online calculator to do this.
My final goal is to write a program in C# or JAVA , which will convert NZMG coordinates into WGS84 coordinates.
Please note: This implementation does not convert with pinpoint accuracy.
Here is a c# implementation I put together for converting from NZMG to NZGD1949 and then to datum shift from NZGD1949 to NZGD2000. NZGD2000 coordinates are apparently compatible with WGS84 source: http://www.linz.govt.nz/data/geodetic-system/coordinate-conversion/geodetic-datum-conversions/wgs84-nzgd2000. I have added a 'useStaticCorrection' parameter to the datum shift method because I found my results were consistently off by approximately 40 meters in latitude - this varies depending on the input coordinates. This may be from a mistake in the logic but, as far as i can tell, it appears all correct (as by the sources referenced).
Sources:
Constants and coefficients
http://www.linz.govt.nz/data/geodetic-system/coordinate-conversion/projection-conversions/new-zealand-map-grid
Datum info.
http://www.linz.govt.nz/regulatory/25000
NZMG -> NZGD1949 formula
http://www.linz.govt.nz/data/geodetic-system/coordinate-conversion/projection-conversions/new-zealand-map-grid
NZ1949 -> NZGD2000 differences
http://www.linz.govt.nz/data/geodetic-system/coordinate-conversion/geodetic-datum-conversions/nzgd1949-nzgd2000/three-seven
Equivilent of a Three parameter shift.
http://sas2.elte.hu/tg/eesti_datums_egs9.htm
//Lat / Long fudge amounts http://www.linz.govt.nz/data/geodetic-system/coordinate-conversion/geodetic-datum-conversions/datum-transformation-examples
public class Pair<T, U>
{
public Pair()
{
}
public Pair(T first, U second)
{
this.First = first;
this.Second = second;
}
public T First { get; set; }
public U Second { get; set; }
};
using System.Numerics;
public class NZMGConverter
{
//Constants and coefficients
//Source: http://www.linz.govt.nz/data/geodetic-system/coordinate-conversion/projection-conversions/new-zealand-map-grid
private static Double N0 = 6023150;
private static Double E0 = 2510000;
private static Double NZGD1949a = 6378388;
private static Double NZMGLatOrigin = -41;
private static Double NZMGLongOrigin = 173;
//some of these are unused but are included for completeness. (unused arrays are for converting from wgs84 / nzgd2000 -> nzmg).
private static Double[] A = new Double[] { 0.6399175073, -0.1358797613, 0.0632944090, -0.0252685300, 0.0117879000, -0.0055161000, 0.0026906000, -0.0013330000, 0.0006700000, -0.0003400000 };
private static Complex[] B = new Complex[] { new Complex(0.7557853228, 0.0), new Complex(0.249204646, 0.003371507), new Complex(-0.001541739, 0.04105856), new Complex(-0.10162907, 0.01727609), new Complex(-0.26623489, -0.36249218), new Complex(-0.6870983, -1.1651967) };
private static Complex[] C = new Complex[] { new Complex(1.3231270439, 0.0), new Complex(-0.577245789, -0.007809598), new Complex(0.508307513, -0.112208952), new Complex(-0.15094762, 0.18200602), new Complex(1.01418179, 1.64497696), new Complex(1.9660549, 2.5127645) };
private static Double[] D = new Double[] { 1.5627014243, 0.5185406398, -0.0333309800, -0.1052906000, -0.0368594000, 0.0073170000, 0.0122000000, 0.0039400000, -0.0013000000 };
//Datum info.
//Source: http://www.linz.govt.nz/regulatory/25000
private static Double NZGD1949f = 0.003367003;
//Double NZGD1949InverseFlattening = 297;
private static Double NZGD1949e2 = (2 * 0.003367003) - Math.Pow(0.003367003, 2);
private static Double NZGD2000a = 6378137;
private static Double NZGD2000f = 0.003352811;
//Double NZGD2000InverseFlattening = 298.2572221;
private static Double NZGD2000e2 = (2 * 0.003352811) - Math.Pow(0.003352811, 2);
//NZ1949 -> NZGD2000 differences
//Source: http://www.linz.govt.nz/data/geodetic-system/coordinate-conversion/geodetic-datum-conversions/nzgd1949-nzgd2000/three-seven
private static Double differenceX = 54.4;
private static Double differenceY = -20.1;
private static Double differenceZ = 183.1;
//Source: http://www.linz.govt.nz/data/geodetic-system/coordinate-conversion/projection-conversions/new-zealand-map-grid
public static Pair<Double, Double> convertToNZGD1949(Double nzmgX, Double nzmgY)
{
Pair<Double, Double> result = null;
Complex z = new Complex((nzmgY - N0) / NZGD1949a, (nzmgX - E0) / NZGD1949a);
Complex theta0 = new Complex();
for (int i = 0; i < C.Length; ++i) { theta0 += Complex.Multiply(C[i], Complex.Pow(z, i + 1)); }
Double deltaLatWtheta0 = 0;
for (int i = 0; i < D.Length; ++i) { deltaLatWtheta0 += D[i] * Math.Pow(theta0.Real, (i + 1)); }
result = new Pair<double, double>();
result.First = NZMGLatOrigin + (Math.Pow(10, 5) / 3600) * deltaLatWtheta0; //NZGD1949 lat
result.Second = NZMGLongOrigin + 180 / Math.PI * theta0.Imaginary; //NZGD1949 long
return result;
}
//Equivilent of a Three parameter shift.
//Source: http://sas2.elte.hu/tg/eesti_datums_egs9.htm
public static Pair<Double, Double> datumShiftNZGD1949toNZGD2000(Double nzgd1949Lat, Double nzgd1949Long, Boolean useStaticCorrection)
{
Pair<Double, Double> result = null;
Double latInRaidans = nzgd1949Lat * (Math.PI / 180);
Double lngInRaidans = nzgd1949Long * (Math.PI / 180);
Double n = NZGD1949a / Math.Sqrt((1 - NZGD1949e2 * Math.Pow(Math.Sin(latInRaidans), 2)) * (3 / 2));
Double m = NZGD1949a * ((1 - NZGD1949e2) / (1 - NZGD1949e2 * Math.Pow(Math.Sin(latInRaidans), 2) * (3 / 2)));
Double difF = NZGD2000f - NZGD1949f;
Double difA = NZGD2000a - NZGD1949a;
Double changeInLat = (-1 * differenceX * Math.Sin(latInRaidans) * Math.Cos(lngInRaidans)
- differenceY * Math.Sin(latInRaidans) * Math.Sin(lngInRaidans)
+ differenceZ * Math.Cos(latInRaidans) + (NZGD1949a * difF + NZGD1949f * difA) * Math.Sin(2 * latInRaidans))
/ (m * Math.Sin(1));
Double changeInLong = (-1 * differenceX * Math.Sin(lngInRaidans) + differenceY * Math.Cos(lngInRaidans)) / (n * Math.Cos(latInRaidans) * Math.Sin(1));
result = new Pair<Double, Double>();
result.First = (latInRaidans + changeInLat) * (180 / Math.PI);
result.Second = (lngInRaidans + changeInLong) * (180 / Math.PI);
if (useStaticCorrection)
{
//Difference found by 3-parameter test values # http://www.linz.govt.nz/data/geodetic-system/coordinate-conversion/geodetic-datum-conversions/datum-transformation-examples
Double staticLatCorrection = 0.00037432;
Double staticLongCorrection = -0.00003213;
result.First -= staticLatCorrection;
result.Second += staticLongCorrection;
}
return result;
}
}
NZMG to Latitude/Longitude using Swift. Thanks to DavidG's post above. Fixed 40m latitude error.
import Foundation
import Darwin
class NZMGConverter {
private let easting: Double!
private let northing: Double!
private var z: Complex! = nil
private let N0: Double = 6023150; //false Northing
private let E0: Double = 2510000; //false Easting
private let NZMGLatOrigin: Double = -41;
private let NZMGLongOrigin: Double = 173;
private let c: [Complex] = [Complex(real: 1.3231270439, imag: 0.0), Complex(real: -0.577245789, imag: -0.007809598), Complex(real: 0.508307513, imag: -0.112208952), Complex(real: -0.15094762, imag: 0.18200602), Complex(real: 1.01418179, imag: 1.64497696), Complex(real: 1.9660549, imag: 2.5127645)]
private let b: [Complex] = [Complex(real: 0.7557853228, imag: 0.0), Complex(real: 0.249204646, imag: 0.003371507), Complex(real: -0.001541739, imag: 0.04105856), Complex(real: -0.10162907, imag: 0.01727609), Complex(real: -0.26623489, imag: -0.36249218), Complex(real: -0.6870983, imag: -1.1651967)]
private let d: [Double] = [1.5627014243, 0.5185406398, -0.0333309800, -0.1052906000, -0.0368594000, 0.0073170000, 0.0122000000, 0.0039400000, -0.0013000000]
//Datum info.
private let NZGD1949f: Double = 0.003367003;
//Double NZGD1949InverseFlattening = 297;
private let NZGD1949e2: Double = (2 * 0.003367003) - pow(0.003367003, 2);
private let NZGD1949a: Double = 6378388; //Semi-major axis
private let NZGD2000a: Double = 6378137;
private let NZGD2000f: Double = 0.003352811;
//Double NZGD2000InverseFlattening = 298.2572221;
private let NZGD2000e2: Double = (2 * 0.003352811) - pow(0.003352811, 2);
//NZ1949 -> NZGD2000 differences
private let differenceX: Double = 54.4;
private let differenceY: Double = -20.1;
private let differenceZ: Double = 183.1;
init(easting: Double, northing: Double) {
self.easting = easting
self.northing = northing
}
func nzmgToNZGD1949() -> (latitude: Double, longitude: Double) {
z = Complex(real: (northing - N0) / NZGD1949a, imag: (easting - E0) / NZGD1949a)
let theta0 = thetaZero(z)
let theta: Complex = thetaSuccessive(theta0, i: 0)
let latlong = calcLatLong(theta)
print(latlong)
return datumShift(latlong.0, longitude: latlong.1)
}
func thetaZero(z: Complex) -> Complex {
var theta0 = Complex()
for var i = 0; i < c.count; i++ {
theta0 = theta0 + (c[i] * (z ^ (i + 1)))
//print(theta0)
}
return theta0
}
func thetaSuccessive(theta0: Complex, i: Int) -> Complex {
var numerator: Complex = Complex()
var denominator: Complex = Complex()
for var n = 1; n < b.count; n++ {
numerator = numerator + (Complex(real: n) * b[n] * (theta0 ^ (n + 1)))
}
numerator = z + numerator
for var n = 0; n < b.count; n++ {
denominator = denominator + (Complex(real: n + 1) * b[n] * (theta0 ^ n))
}
let theta = numerator / denominator
if i == 2 {
return theta
} else {
return thetaSuccessive(theta, i: i + 1)
}
}
func calcLatLong(theta: Complex) -> (latitude: Double, longitude: Double) {
let deltaPsi = theta.real
let deltaLambda = theta.imag
var deltaPhi: Double = 0
for var n = 0; n < d.count; n++ {
deltaPhi += d[n] * pow(deltaPsi, Double(n + 1))
}
let lat: Double = NZMGLatOrigin + ((pow(10, 5) / 3600) * deltaPhi)
let long: Double = NZMGLongOrigin + ((180 / M_PI) * deltaLambda)
return (latitude: lat, longitude: long)
}
func datumShift(latitude: Double, longitude: Double) -> (latitude: Double, longitude: Double) {
let latInRadians = latitude * (M_PI / 180)
let lngInRadians = longitude * (M_PI / 180)
let v = NZGD1949a / (sqrt(1 - (NZGD1949e2 * pow(sin(latInRadians), 2))))
var x_cart = v * cos(latInRadians) * cos(lngInRadians)
var y_cart = v * cos(latInRadians) * sin(lngInRadians)
var z_cart = (v * (1 - NZGD1949e2)) * sin(latInRadians)
x_cart += differenceX
y_cart += differenceY
z_cart += differenceZ
print(x_cart)
print(y_cart)
print(z_cart)
let p = sqrt(pow(x_cart, 2) + pow(y_cart, 2))
let r = sqrt(pow(p, 2) + pow(z_cart, 2))
let mu = atan((z_cart / p) * ((1 - NZGD2000f) + ((NZGD2000e2 * NZGD2000a) / r)))
let num = (z_cart * (1 - NZGD2000f)) + (NZGD2000e2 * NZGD2000a * pow(sin(mu), 3))
let denom = (1 - NZGD2000f) * (p - (NZGD2000e2 * NZGD2000a * pow(cos(mu), 3)))
var lat = atan(num / denom)
var long = atan(y_cart / x_cart)
print(long)
lat = (180 / M_PI) * lat
long = 180 + (180 / M_PI) * long
return (latitude: lat, longitude: long)
}
}
}
Proj4j provides this functionality:
It is one of the few universal coordinate conversions libs.
http://trac.osgeo.org/proj4j/
Then you have to find the proj4 string that defines the new zeal and datum description:
Depending if you need th eold or the new grid, you need the new zealand datum shift grids,+
which are included in proj4.
As a first step find exactly the name of the datum that you need:
NZGD49 or NZGD2000 ?
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed last year.
Improve this question
Do you know of a .net library to perform a LOESS/LOWESS regression? (preferably free/open source)
Port from java to c#
public class LoessInterpolator
{
public static double DEFAULT_BANDWIDTH = 0.3;
public static int DEFAULT_ROBUSTNESS_ITERS = 2;
/**
* The bandwidth parameter: when computing the loess fit at
* a particular point, this fraction of source points closest
* to the current point is taken into account for computing
* a least-squares regression.
*
* A sensible value is usually 0.25 to 0.5.
*/
private double bandwidth;
/**
* The number of robustness iterations parameter: this many
* robustness iterations are done.
*
* A sensible value is usually 0 (just the initial fit without any
* robustness iterations) to 4.
*/
private int robustnessIters;
public LoessInterpolator()
{
this.bandwidth = DEFAULT_BANDWIDTH;
this.robustnessIters = DEFAULT_ROBUSTNESS_ITERS;
}
public LoessInterpolator(double bandwidth, int robustnessIters)
{
if (bandwidth < 0 || bandwidth > 1)
{
throw new ApplicationException(string.Format("bandwidth must be in the interval [0,1], but got {0}", bandwidth));
}
this.bandwidth = bandwidth;
if (robustnessIters < 0)
{
throw new ApplicationException(string.Format("the number of robustness iterations must be non-negative, but got {0}", robustnessIters));
}
this.robustnessIters = robustnessIters;
}
/**
* Compute a loess fit on the data at the original abscissae.
*
* #param xval the arguments for the interpolation points
* #param yval the values for the interpolation points
* #return values of the loess fit at corresponding original abscissae
* #throws MathException if some of the following conditions are false:
* <ul>
* <li> Arguments and values are of the same size that is greater than zero</li>
* <li> The arguments are in a strictly increasing order</li>
* <li> All arguments and values are finite real numbers</li>
* </ul>
*/
public double[] smooth(double[] xval, double[] yval)
{
if (xval.Length != yval.Length)
{
throw new ApplicationException(string.Format("Loess expects the abscissa and ordinate arrays to be of the same size, but got {0} abscisssae and {1} ordinatae", xval.Length, yval.Length));
}
int n = xval.Length;
if (n == 0)
{
throw new ApplicationException("Loess expects at least 1 point");
}
checkAllFiniteReal(xval, true);
checkAllFiniteReal(yval, false);
checkStrictlyIncreasing(xval);
if (n == 1)
{
return new double[] { yval[0] };
}
if (n == 2)
{
return new double[] { yval[0], yval[1] };
}
int bandwidthInPoints = (int)(bandwidth * n);
if (bandwidthInPoints < 2)
{
throw new ApplicationException(string.Format("the bandwidth must be large enough to accomodate at least 2 points. There are {0} " +
" data points, and bandwidth must be at least {1} but it is only {2}",
n, 2.0 / n, bandwidth
));
}
double[] res = new double[n];
double[] residuals = new double[n];
double[] sortedResiduals = new double[n];
double[] robustnessWeights = new double[n];
// Do an initial fit and 'robustnessIters' robustness iterations.
// This is equivalent to doing 'robustnessIters+1' robustness iterations
// starting with all robustness weights set to 1.
for (int i = 0; i < robustnessWeights.Length; i++) robustnessWeights[i] = 1;
for (int iter = 0; iter <= robustnessIters; ++iter)
{
int[] bandwidthInterval = { 0, bandwidthInPoints - 1 };
// At each x, compute a local weighted linear regression
for (int i = 0; i < n; ++i)
{
double x = xval[i];
// Find out the interval of source points on which
// a regression is to be made.
if (i > 0)
{
updateBandwidthInterval(xval, i, bandwidthInterval);
}
int ileft = bandwidthInterval[0];
int iright = bandwidthInterval[1];
// Compute the point of the bandwidth interval that is
// farthest from x
int edge;
if (xval[i] - xval[ileft] > xval[iright] - xval[i])
{
edge = ileft;
}
else
{
edge = iright;
}
// Compute a least-squares linear fit weighted by
// the product of robustness weights and the tricube
// weight function.
// See http://en.wikipedia.org/wiki/Linear_regression
// (section "Univariate linear case")
// and http://en.wikipedia.org/wiki/Weighted_least_squares
// (section "Weighted least squares")
double sumWeights = 0;
double sumX = 0, sumXSquared = 0, sumY = 0, sumXY = 0;
double denom = Math.Abs(1.0 / (xval[edge] - x));
for (int k = ileft; k <= iright; ++k)
{
double xk = xval[k];
double yk = yval[k];
double dist;
if (k < i)
{
dist = (x - xk);
}
else
{
dist = (xk - x);
}
double w = tricube(dist * denom) * robustnessWeights[k];
double xkw = xk * w;
sumWeights += w;
sumX += xkw;
sumXSquared += xk * xkw;
sumY += yk * w;
sumXY += yk * xkw;
}
double meanX = sumX / sumWeights;
double meanY = sumY / sumWeights;
double meanXY = sumXY / sumWeights;
double meanXSquared = sumXSquared / sumWeights;
double beta;
if (meanXSquared == meanX * meanX)
{
beta = 0;
}
else
{
beta = (meanXY - meanX * meanY) / (meanXSquared - meanX * meanX);
}
double alpha = meanY - beta * meanX;
res[i] = beta * x + alpha;
residuals[i] = Math.Abs(yval[i] - res[i]);
}
// No need to recompute the robustness weights at the last
// iteration, they won't be needed anymore
if (iter == robustnessIters)
{
break;
}
// Recompute the robustness weights.
// Find the median residual.
// An arraycopy and a sort are completely tractable here,
// because the preceding loop is a lot more expensive
System.Array.Copy(residuals, sortedResiduals, n);
//System.arraycopy(residuals, 0, sortedResiduals, 0, n);
Array.Sort<double>(sortedResiduals);
double medianResidual = sortedResiduals[n / 2];
if (medianResidual == 0)
{
break;
}
for (int i = 0; i < n; ++i)
{
double arg = residuals[i] / (6 * medianResidual);
robustnessWeights[i] = (arg >= 1) ? 0 : Math.Pow(1 - arg * arg, 2);
}
}
return res;
}
/**
* Given an index interval into xval that embraces a certain number of
* points closest to xval[i-1], update the interval so that it embraces
* the same number of points closest to xval[i]
*
* #param xval arguments array
* #param i the index around which the new interval should be computed
* #param bandwidthInterval a two-element array {left, right} such that: <p/>
* <tt>(left==0 or xval[i] - xval[left-1] > xval[right] - xval[i])</tt>
* <p/> and also <p/>
* <tt>(right==xval.length-1 or xval[right+1] - xval[i] > xval[i] - xval[left])</tt>.
* The array will be updated.
*/
private static void updateBandwidthInterval(double[] xval, int i, int[] bandwidthInterval)
{
int left = bandwidthInterval[0];
int right = bandwidthInterval[1];
// The right edge should be adjusted if the next point to the right
// is closer to xval[i] than the leftmost point of the current interval
int nextRight = nextNonzero(weights, right);
if (nextRight < xval.Length && xval[nextRight] - xval[i] < xval[i] - xval[left])
{
int nextLeft = nextNonzero(weights, bandwidthInterval[0]);
bandwidthInterval[0] = nextLeft;
bandwidthInterval[1] = nextRight;
}
}
/**
* Compute the
* tricube
* weight function
*
* #param x the argument
* #return (1-|x|^3)^3
*/
private static double tricube(double x)
{
double tmp = Math.abs(x);
tmp = 1 - tmp * tmp * tmp;
return tmp * tmp * tmp;
}
/**
* Check that all elements of an array are finite real numbers.
*
* #param values the values array
* #param isAbscissae if true, elements are abscissae otherwise they are ordinatae
* #throws MathException if one of the values is not
* a finite real number
*/
private static void checkAllFiniteReal(double[] values, bool isAbscissae)
{
for (int i = 0; i < values.Length; i++)
{
double x = values[i];
if (Double.IsInfinity(x) || Double.IsNaN(x))
{
string pattern = isAbscissae ?
"all abscissae must be finite real numbers, but {0}-th is {1}" :
"all ordinatae must be finite real numbers, but {0}-th is {1}";
throw new ApplicationException(string.Format(pattern, i, x));
}
}
}
/**
* Check that elements of the abscissae array are in a strictly
* increasing order.
*
* #param xval the abscissae array
* #throws MathException if the abscissae array
* is not in a strictly increasing order
*/
private static void checkStrictlyIncreasing(double[] xval)
{
for (int i = 0; i < xval.Length; ++i)
{
if (i >= 1 && xval[i - 1] >= xval[i])
{
throw new ApplicationException(string.Format(
"the abscissae array must be sorted in a strictly " +
"increasing order, but the {0}-th element is {1} " +
"whereas {2}-th is {3}",
i - 1, xval[i - 1], i, xval[i]));
}
}
}
}
Since I'm unable to comment on other people's posts (new user), and people seem to think I should do that with this instead of editing the above answer, I'm simply going to write it as an answer even though I know this is better as a comment.
The updateBandwidthInterval method in the above answer forgets to check the left side as written in the method comment. This can give NaN issues for sumWeights. The below should fix that. I encountered this when doing a c++ implementation based on the above.
/**
* Given an index interval into xval that embraces a certain number of
* points closest to xval[i-1], update the interval so that it embraces
* the same number of points closest to xval[i]
*
* #param xval arguments array
* #param i the index around which the new interval should be computed
* #param bandwidthInterval a two-element array {left, right} such that: <p/>
* <tt>(left==0 or xval[i] - xval[left-1] > xval[right] - xval[i])</tt>
* <p/> and also <p/>
* <tt>(right==xval.length-1 or xval[right+1] - xval[i] > xval[i] - xval[left])</tt>.
* The array will be updated.
*/
private static void updateBandwidthInterval(double[] xval, int i, int[] bandwidthInterval)
{
int left = bandwidthInterval[0];
int right = bandwidthInterval[1];
// The edges should be adjusted if the previous point to the
// left is closer to x than the current point to the right or
// if the next point to the right is closer
// to x than the leftmost point of the current interval
if (left != 0 &&
xval[i] - xval[left - 1] < xval[right] - xval[i])
{
bandwidthInterval[0]++;
bandwidthInterval[1]++;
}
else if (right < xval.Length - 1 &&
xval[right + 1] - xval[i] < xval[i] - xval[left])
{
bandwidthInterval[0]++;
bandwidthInterval[1]++;
}
}
Hope someone after 5 years find this useful. This is the original code posted by Tutcugil but with the missing methods and updated.
using System;
using System.Linq;
namespace StockCorrelation
{
public class LoessInterpolator
{
public static double DEFAULT_BANDWIDTH = 0.3;
public static int DEFAULT_ROBUSTNESS_ITERS = 2;
/**
* The bandwidth parameter: when computing the loess fit at
* a particular point, this fraction of source points closest
* to the current point is taken into account for computing
* a least-squares regression.
*
* A sensible value is usually 0.25 to 0.5.
*/
private double bandwidth;
/**
* The number of robustness iterations parameter: this many
* robustness iterations are done.
*
* A sensible value is usually 0 (just the initial fit without any
* robustness iterations) to 4.
*/
private int robustnessIters;
public LoessInterpolator()
{
this.bandwidth = DEFAULT_BANDWIDTH;
this.robustnessIters = DEFAULT_ROBUSTNESS_ITERS;
}
public LoessInterpolator(double bandwidth, int robustnessIters)
{
if (bandwidth < 0 || bandwidth > 1)
{
throw new ApplicationException(string.Format("bandwidth must be in the interval [0,1], but got {0}", bandwidth));
}
this.bandwidth = bandwidth;
if (robustnessIters < 0)
{
throw new ApplicationException(string.Format("the number of robustness iterations must be non-negative, but got {0}", robustnessIters));
}
this.robustnessIters = robustnessIters;
}
/**
* Compute a loess fit on the data at the original abscissae.
*
* #param xval the arguments for the interpolation points
* #param yval the values for the interpolation points
* #return values of the loess fit at corresponding original abscissae
* #throws MathException if some of the following conditions are false:
* <ul>
* <li> Arguments and values are of the same size that is greater than zero</li>
* <li> The arguments are in a strictly increasing order</li>
* <li> All arguments and values are finite real numbers</li>
* </ul>
*/
public double[] smooth(double[] xval, double[] yval, double[] weights)
{
if (xval.Length != yval.Length)
{
throw new ApplicationException(string.Format("Loess expects the abscissa and ordinate arrays to be of the same size, but got {0} abscisssae and {1} ordinatae", xval.Length, yval.Length));
}
int n = xval.Length;
if (n == 0)
{
throw new ApplicationException("Loess expects at least 1 point");
}
checkAllFiniteReal(xval, true);
checkAllFiniteReal(yval, false);
checkStrictlyIncreasing(xval);
if (n == 1)
{
return new double[] { yval[0] };
}
if (n == 2)
{
return new double[] { yval[0], yval[1] };
}
int bandwidthInPoints = (int)(bandwidth * n);
if (bandwidthInPoints < 2)
{
throw new ApplicationException(string.Format("the bandwidth must be large enough to accomodate at least 2 points. There are {0} " +
" data points, and bandwidth must be at least {1} but it is only {2}",
n, 2.0 / n, bandwidth
));
}
double[] res = new double[n];
double[] residuals = new double[n];
double[] sortedResiduals = new double[n];
double[] robustnessWeights = new double[n];
// Do an initial fit and 'robustnessIters' robustness iterations.
// This is equivalent to doing 'robustnessIters+1' robustness iterations
// starting with all robustness weights set to 1.
for (int i = 0; i < robustnessWeights.Length; i++) robustnessWeights[i] = 1;
for (int iter = 0; iter <= robustnessIters; ++iter)
{
int[] bandwidthInterval = { 0, bandwidthInPoints - 1 };
// At each x, compute a local weighted linear regression
for (int i = 0; i < n; ++i)
{
double x = xval[i];
// Find out the interval of source points on which
// a regression is to be made.
if (i > 0)
{
updateBandwidthInterval(xval, weights, i, bandwidthInterval);
}
int ileft = bandwidthInterval[0];
int iright = bandwidthInterval[1];
// Compute the point of the bandwidth interval that is
// farthest from x
int edge;
if (xval[i] - xval[ileft] > xval[iright] - xval[i])
{
edge = ileft;
}
else
{
edge = iright;
}
// Compute a least-squares linear fit weighted by
// the product of robustness weights and the tricube
// weight function.
// See http://en.wikipedia.org/wiki/Linear_regression
// (section "Univariate linear case")
// and http://en.wikipedia.org/wiki/Weighted_least_squares
// (section "Weighted least squares")
double sumWeights = 0;
double sumX = 0, sumXSquared = 0, sumY = 0, sumXY = 0;
double denom = Math.Abs(1.0 / (xval[edge] - x));
for (int k = ileft; k <= iright; ++k)
{
double xk = xval[k];
double yk = yval[k];
double dist;
if (k < i)
{
dist = (x - xk);
}
else
{
dist = (xk - x);
}
double w = tricube(dist * denom) * robustnessWeights[k];
double xkw = xk * w;
sumWeights += w;
sumX += xkw;
sumXSquared += xk * xkw;
sumY += yk * w;
sumXY += yk * xkw;
}
double meanX = sumX / sumWeights;
double meanY = sumY / sumWeights;
double meanXY = sumXY / sumWeights;
double meanXSquared = sumXSquared / sumWeights;
double beta;
if (meanXSquared == meanX * meanX)
{
beta = 0;
}
else
{
beta = (meanXY - meanX * meanY) / (meanXSquared - meanX * meanX);
}
double alpha = meanY - beta * meanX;
res[i] = beta * x + alpha;
residuals[i] = Math.Abs(yval[i] - res[i]);
}
// No need to recompute the robustness weights at the last
// iteration, they won't be needed anymore
if (iter == robustnessIters)
{
break;
}
// Recompute the robustness weights.
// Find the median residual.
// An arraycopy and a sort are completely tractable here,
// because the preceding loop is a lot more expensive
System.Array.Copy(residuals, sortedResiduals, n);
//System.arraycopy(residuals, 0, sortedResiduals, 0, n);
Array.Sort<double>(sortedResiduals);
double medianResidual = sortedResiduals[n / 2];
if (medianResidual == 0)
{
break;
}
for (int i = 0; i < n; ++i)
{
double arg = residuals[i] / (6 * medianResidual);
robustnessWeights[i] = (arg >= 1) ? 0 : Math.Pow(1 - arg * arg, 2);
}
}
return res;
}
public double[] smooth(double[] xval, double[] yval)
{
if (xval.Length != yval.Length)
{
throw new Exception($"xval and yval len are different");
}
double[] unitWeights = Enumerable.Repeat(1.0, xval.Length).ToArray();
return smooth(xval, yval, unitWeights);
}
/**
* Given an index interval into xval that embraces a certain number of
* points closest to xval[i-1], update the interval so that it embraces
* the same number of points closest to xval[i]
*
* #param xval arguments array
* #param i the index around which the new interval should be computed
* #param bandwidthInterval a two-element array {left, right} such that: <p/>
* <tt>(left==0 or xval[i] - xval[left-1] > xval[right] - xval[i])</tt>
* <p/> and also <p/>
* <tt>(right==xval.length-1 or xval[right+1] - xval[i] > xval[i] - xval[left])</tt>.
* The array will be updated.
*/
private static void updateBandwidthInterval(double[] xval, double[] weights,
int i,
int[] bandwidthInterval)
{
int left = bandwidthInterval[0];
int right = bandwidthInterval[1];
// The right edge should be adjusted if the next point to the right
// is closer to xval[i] than the leftmost point of the current interval
int nextRight = nextNonzero(weights, right);
if (nextRight < xval.Length && xval[nextRight] - xval[i] < xval[i] - xval[left])
{
int nextLeft = nextNonzero(weights, bandwidthInterval[0]);
bandwidthInterval[0] = nextLeft;
bandwidthInterval[1] = nextRight;
}
}
private static int nextNonzero(double[] weights, int i)
{
int j = i + 1;
while (j < weights.Length && weights[j] == 0)
{
++j;
}
return j;
}
/**
* Compute the
* tricube
* weight function
*
* #param x the argument
* #return (1-|x|^3)^3
*/
private static double tricube(double x)
{
double tmp = Math.Abs(x);
tmp = 1 - tmp * tmp * tmp;
return tmp * tmp * tmp;
}
/**
* Check that all elements of an array are finite real numbers.
*
* #param values the values array
* #param isAbscissae if true, elements are abscissae otherwise they are ordinatae
* #throws MathException if one of the values is not
* a finite real number
*/
private static void checkAllFiniteReal(double[] values, bool isAbscissae)
{
for (int i = 0; i < values.Length; i++)
{
double x = values[i];
if (Double.IsInfinity(x) || Double.IsNaN(x))
{
string pattern = isAbscissae ?
"all abscissae must be finite real numbers, but {0}-th is {1}" :
"all ordinatae must be finite real numbers, but {0}-th is {1}";
throw new ApplicationException(string.Format(pattern, i, x));
}
}
}
/**
* Check that elements of the abscissae array are in a strictly
* increasing order.
*
* #param xval the abscissae array
* #throws MathException if the abscissae array
* is not in a strictly increasing order
*/
private static void checkStrictlyIncreasing(double[] xval)
{
for (int i = 0; i < xval.Length; ++i)
{
if (i >= 1 && xval[i - 1] >= xval[i])
{
throw new ApplicationException(string.Format(
"the abscissae array must be sorted in a strictly " +
"increasing order, but the {0}-th element is {1} " +
"whereas {2}-th is {3}",
i - 1, xval[i - 1], i, xval[i]));
}
}
}
}
}
The source data :
static double[] felix = new double[] { 0.003027523, 0.002012256, -0.001369238, -0.001737660, -0.001647287,
0.000275154, 0.002017238, 0.001372621, 0.000274148, -0.000913576, 0.001920263, 0.001186456, -0.000364631,
0.000638337, 0.000182266, -0.001275626, -0.000821093, 0.001186998, -0.000455996, -0.000547445, -0.000182582,
-0.000547845, 0.001279006, 0.000456204, 0.000000000, -0.001550388, 0.001552795, 0.000729594, -0.000455664,
-0.002188184, 0.000639620, 0.000091316, 0.001552228, -0.001002826, 0.000182515, -0.000091241, -0.000821243,
-0.002009132, 0.000000000, 0.000823572, 0.001920088, -0.001368863, 0.000000000, 0.002101800, 0.001094291,
0.001639643, 0.002637323, 0.000000000, -0.000172336, -0.000462665, -0.000136141 };
The variance function:
public static double Variance(double[] x)
{
if (x.Length == 0)
return 0;
double sumX = 0;
double sumXsquared = 0;
double varianceX = 0;
int dataLength = x.Length;
for (int i = 0; i < dataLength; i++)
{
sumX += x[i];
sumXsquared += x[i] * x[i];
}
varianceX = (sumXsquared / dataLength) - ((sumX / dataLength) * (sumX / dataLength));
return varianceX;
}
Excel and some online calculator says the variance is 1.56562E-06
While my function gives me 1.53492394804015E-06. I begin to doubt if the C# has accuracy problem or what. Is there anyone have this kind of problem before?
What you are seeing is the difference between sample variance and population variance and nothing to do with floating point precision or the accuracy of C#'s floating point implementation.
You are calculating population variance. Excel and that web site are calculating sample variance.
Var and VarP are distinct calculations and you do need to be careful about which one you are using. (unfortunately people often refer to them as if they are interchangeable when they are not. The same is true for standard deviation)
Sample variance for your data is 1.56562E-06, population variance is 1.53492394804015E-06.
From some code posted on codeproject awhile back:
Variance in a sample
public static double Variance(this IEnumerable<double> source)
{
double avg = source.Average();
double d = source.Aggregate(0.0, (total, next) => total += Math.Pow(next - avg, 2));
return d / (source.Count() - 1);
}
Variance in a population
public static double VarianceP(this IEnumerable<double> source)
{
double avg = source.Average();
double d = source.Aggregate(0.0, (total, next) => total += Math.Pow(next - avg, 2));
return d / source.Count();
}
Here's an alternate implementation, that is sometimes better-behaved, numerically:
mean = Average(data);
double sum2 = 0.0, sumc = 0.0;
for (int i = 0; i < data.Count; i++)
{
double dev = data[i] - mean;
sum2 += dev * dev;
sumc += dev;
}
return (sum2 - sumc * sumc / data.Count) / data.Count;