Develop Reference

RSI vs Wilder's RSI Calculation Problems

I am having trouble getting a smoothed RSI. The below picture is from freestockcharts.com. The calculation uses this code.
public static double CalculateRsi(IEnumerable<double> closePrices)
{
var prices = closePrices as double[] ?? closePrices.ToArray();
double sumGain = 0;
double sumLoss = 0;
for (int i = 1; i < prices.Length; i++)
{
var difference = prices[i] - prices[i - 1];
if (difference >= 0)
{
sumGain += difference;
}
else
{
sumLoss -= difference;
}
}
if (sumGain == 0) return 0;
if (Math.Abs(sumLoss) < Tolerance) return 100;
var relativeStrength = sumGain / sumLoss;
return 100.0 - (100.0 / (1 + relativeStrength));
}
https://stackoverflow.com/questions/...th-index-using-some-programming-language-js-c
This seems to be the pure RSI with no smoothing. How does a smoothed RSI get calculated? I have tried changing it to fit the definitions of the these two sites however the output was not correct. It was barely smoothed.
(I don't have enough rep to post links)
tc2000 -> Indicators -> RSI_and_Wilder_s_RSI (Wilder's smoothing = Previous MA value + (1/n periods * (Close - Previous MA)))
priceactionlab -> wilders-cutlers-and-harris-relative-strength-index (RS = EMA(Gain(n), n)/EMA(Loss(n), n))
Can someone actually do the calculation with some sample data?
Wilder's RSI vs RSI

In order to calculate the RSI, you need a period to calculate it with. As noted on Wikipedia, 14 is used quite often.
So the calculation steps would be as follows:
Period 1 - 13, RSI = 0
Period 14:
AverageGain = TotalGain / PeriodCount;
AverageLoss = TotalLoss / PeriodCount;
RS = AverageGain / AverageLoss;
RSI = 100 - 100 / (1 + RS);
Period 15 - to period (N):
if (Period(N)Change > 0
AverageGain(N) = ((AverageGain(N - 1) * (PeriodCount - 1)) + Period(N)Change) / PeriodCount;
else
AverageGain(N) = (AverageGain(N - 1) * (PeriodCount - 1)) / PeriodCount;
if (this.Change < 0)
AverageLoss(N) = ((AverageLoss(N - 1) * (PeriodCount - 1)) + Math.Abs(Period(N)Change)) / PeriodCount;
else
AverageLoss(N) = (AverageLoss(N - 1) * (PeriodCount - 1)) / PeriodCount;
RS = AverageGain / AverageLoss;
RSI = 100 - (100 / (1 + RS));
Thereafter, to smooth the values, you need to apply a moving average of a certain period to your RSI values. To do that, traverse your RSI values from the last index to the first and calculate your average for the current period based on the preceding x smoothing periods.
Once done, just reverse the list of values to get the expected order:
List<double> SmoothedRSI(IEnumerable<double> rsiValues, int smoothingPeriod)
{
if (rsiValues.Count() <= smoothingPeriod)
throw new Exception("Smoothing period too large or too few RSI values passed.");
List<double> results = new List<double>();
List<double> reversedRSIValues = rsiValues.Reverse().ToList();
for (int i = 1; i < reversedRSIValues.Count() - smoothingPeriod - 1; i++)
results.Add(reversedRSIValues.Subset(i, i + smoothingPeriod).Average());
return results.Reverse().ToList();
}
The Subset method is just a simple extension method as follows:
public static List<double> Subset(this List<double> values, int start, int end)
{
List<double> results = new List<double>();
for (int i = start; i <= end; i++)
results.Add(values[i]);
return results;
}
Disclaimer, I did not test the code, but it should give you an idea of how the smoothing is applied.

You can't get accurate values without buffers / global variables to store data.
This is a smoothed indicator, meaning it doesn't only use 14 bars but ALL THE BARS:
Here's a step by step article with working and verified source codes generating exactly the same values if prices and number of available bars are the same, of course (you only need to load the price data from your source):
Tested and verified:
using System;
using System.Data;
using System.Globalization;
namespace YourNameSpace
{
class PriceEngine
{
public static DataTable data;
public static double[] positiveChanges;
public static double[] negativeChanges;
public static double[] averageGain;
public static double[] averageLoss;
public static double[] rsi;
public static double CalculateDifference(double current_price, double previous_price)
{
return current_price - previous_price;
}
public static double CalculatePositiveChange(double difference)
{
return difference > 0 ? difference : 0;
}
public static double CalculateNegativeChange(double difference)
{
return difference < 0 ? difference * -1 : 0;
}
public static void CalculateRSI(int rsi_period, int price_index = 5)
{
for(int i = 0; i < PriceEngine.data.Rows.Count; i++)
{
double current_difference = 0.0;
if (i > 0)
{
double previous_close = Convert.ToDouble(PriceEngine.data.Rows[i-1].Field<string>(price_index));
double current_close = Convert.ToDouble(PriceEngine.data.Rows[i].Field<string>(price_index));
current_difference = CalculateDifference(current_close, previous_close);
}
PriceEngine.positiveChanges[i] = CalculatePositiveChange(current_difference);
PriceEngine.negativeChanges[i] = CalculateNegativeChange(current_difference);
if(i == Math.Max(1,rsi_period))
{
double gain_sum = 0.0;
double loss_sum = 0.0;
for(int x = Math.Max(1,rsi_period); x > 0; x--)
{
gain_sum += PriceEngine.positiveChanges[x];
loss_sum += PriceEngine.negativeChanges[x];
}
PriceEngine.averageGain[i] = gain_sum / Math.Max(1,rsi_period);
PriceEngine.averageLoss[i] = loss_sum / Math.Max(1,rsi_period);
}else if (i > Math.Max(1,rsi_period))
{
PriceEngine.averageGain[i] = ( PriceEngine.averageGain[i-1]*(rsi_period-1) + PriceEngine.positiveChanges[i]) / Math.Max(1, rsi_period);
PriceEngine.averageLoss[i] = ( PriceEngine.averageLoss[i-1]*(rsi_period-1) + PriceEngine.negativeChanges[i]) / Math.Max(1, rsi_period);
PriceEngine.rsi[i] = PriceEngine.averageLoss[i] == 0 ? 100 : PriceEngine.averageGain[i] == 0 ? 0 : Math.Round(100 - (100 / (1 + PriceEngine.averageGain[i] / PriceEngine.averageLoss[i])), 5);
}
}
}
public static void Launch()
{
PriceEngine.data = new DataTable();
//load {date, time, open, high, low, close} values in PriceEngine.data (6th column (index #5) = close price) here
positiveChanges = new double[PriceEngine.data.Rows.Count];
negativeChanges = new double[PriceEngine.data.Rows.Count];
averageGain = new double[PriceEngine.data.Rows.Count];
averageLoss = new double[PriceEngine.data.Rows.Count];
rsi = new double[PriceEngine.data.Rows.Count];
CalculateRSI(14);
}
}
}
For detailed step-by-step instructions, I wrote a lengthy article, you can check it here: https://turmanauli.medium.com/a-step-by-step-guide-for-calculating-reliable-rsi-values-programmatically-a6a604a06b77
P.S. functions only work for simple indicators (Simple Moving Average), even Exponential Moving Average, Average True Range absolutely require global variables to store previous values.

Weird glitch when creating sine audio signal

I had a look at the wave format today and created a little wave generator. I create a sine sound like this:
public static Wave GetSine(double length, double hz)
{
int bitRate = 44100;
int l = (int)(bitRate * length);
double f = 1.0 / bitRate;
Int16[] data = new Int16[l];
for (int i = 0; i < l; i++)
{
data[i] = (Int16)(Math.Sin(hz * i * f * Math.PI * 2) * Int16.MaxValue);
}
return new Wave(false, Wave.MakeInt16WaveData(data));
}
MakeInt16WaveData looks like this:
public static byte[] MakeInt16WaveData(Int16[] ints)
{
int s = sizeof(Int16);
byte[] buf = new byte[s * ints.Length];
for(int i = 0; i < ints.Length; i++)
{
Buffer.BlockCopy(BitConverter.GetBytes(ints[i]), 0, buf, i * s, s);
}
return buf;
}
This works as expected! Now I wanted to swoop from one frequency to another like this:
public static Wave GetSineSwoop(double length, double hzStart, double hzEnd)
{
int bitRate = 44100;
int l = (int)(bitRate * length);
double f = 1.0 / bitRate;
Int16[] data = new Int16[l];
double hz;
double hzDelta = hzEnd - hzStart;
for (int i = 0; i < l; i++)
{
hz = hzStart + ((double)i / l) * hzDelta * 0.5; // why *0.5 ?
data[i] = (Int16)(Math.Sin(hz * i * f * Math.PI * 2) * Int16.MaxValue);
}
return new Wave(false, Wave.MakeInt16WaveData(data));
}
Now, when I swooped from 200 to 100 Hz, the sound played from 200 to 0 hertz. For some reason I had to multiply the delta by 0.5 to get the correct output. What might be the issue here ? Is this an audio thing or is there a bug in my code ?
Thanks
Edit by TaW: I take the liberty to add screenshots of the data in a chart which illustrate the problem, the first is with the 0.5 factor, the 2nd with 1.0 and the 3rd & 4th with 2.0 and 5.0:
Edit: here is an example, a swoop from 200 to 100 hz:
Debug values:
Wave clearly does not end at 100 hz

Digging out my rusty math I think it may be because:
Going in L steps from frequency F1 to F2 you have a frequency of
Fi = F1 + i * ( F2 - F1 ) / L
or with
( F2 - F1 ) / L = S
Fi = F1 + i * S
Now to find out how far we have progressed we need the integral over i, which would be
I(Fi) = i * F1 + 1/2 * i ^ 2 * S
Which give or take resembles the term inside your formula for the sine.
Note that you can gain efficiency by moving the constant part (S/2) out of the loop..

Find the min and max for quadratic equation

how to find the min and max for quadratic equation using c# ??
f(x,y) = x^2 + y^2 + 25 * (sin(x)^2 + sin(y)^2) ,where (x,y) from (-2Pi, 2Pi) ??
in the manual solving I got min is = 0 , max = 8Pi^2 = 78.957 .
I tried to write the code based on liner quadratic code but something goes totally wrong
this code give the min = -4.?? and the max = 96 could you help to know where is my mistake please ??
I uploaded the code to dropbox if anyone can have look : https://www.dropbox.com/s/p7y6krk2gk29i9e/Program.cs
double[] X, Y, Result; // Range array and result array.
private void BtnRun_Click(object sender, EventArgs e)
{
//Set any Range for the function
X = setRange(-2 * Math.PI, 2 * Math.PI, 10000);
Y = setRange(-2 * Math.PI, 2 * Math.PI, 10000);
Result = getOutput_twoVariablesFunction(X, Y);
int MaxIndex = getMaxIndex(Result);
int MinIndex = getMinIndex(Result);
TxtMin.Text = Result[MinIndex].ToString();
TxtMax.Text = Result[MaxIndex].ToString();
}
private double twoVariablesFunction(double x,double y)
{
double f;
//Set any two variables function
f = Math.Pow(x, 2) + Math.Pow(y, 2) + 25 * (Math.Pow(Math.Sin(x), 2) + Math.Pow(Math.Sin(y), 2));
return f;
}
private double[] setRange(double Start, double End, int Sample)
{
double Step = (End - Start) / Sample;
double CurrentVaue = Start;
double[] Array = new double[Sample];
for (int Index = 0; Index < Sample; Index++)
{
Array[Index] = CurrentVaue;
CurrentVaue += Step;
}
return Array;
}
private double[] getOutput_twoVariablesFunction(double[] X, double[] Y)
{
int Step = X.Length;
double[] Array = new double[Step];
for (int Index = 0; Index < X.Length ; Index++)
{
Array[Index] = twoVariablesFunction(X[Index], Y[Index]);
}
return Array;
}
private int getMaxIndex(double[] ValuesArray)
{
double M = ValuesArray.Max();
int Index = ValuesArray.ToList().IndexOf(M);
return Index;
}
private int getMinIndex(double[] ValuesArray)
{
double M = ValuesArray.Min();
int Index = ValuesArray.ToList().IndexOf(M);
return Index;
}

Do you want to compute (sin(x))^2 or sin(x^2)? In your f(x,y) formula it looks like (sin(x))^2, but in your method twoVariablesFunction like sin(x^2).

c# LOESS/LOWESS regression [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed last year.
Improve this question
Do you know of a .net library to perform a LOESS/LOWESS regression? (preferably free/open source)

Port from java to c#
public class LoessInterpolator
{
public static double DEFAULT_BANDWIDTH = 0.3;
public static int DEFAULT_ROBUSTNESS_ITERS = 2;
/**
* The bandwidth parameter: when computing the loess fit at
* a particular point, this fraction of source points closest
* to the current point is taken into account for computing
* a least-squares regression.
*
* A sensible value is usually 0.25 to 0.5.
*/
private double bandwidth;
/**
* The number of robustness iterations parameter: this many
* robustness iterations are done.
*
* A sensible value is usually 0 (just the initial fit without any
* robustness iterations) to 4.
*/
private int robustnessIters;
public LoessInterpolator()
{
this.bandwidth = DEFAULT_BANDWIDTH;
this.robustnessIters = DEFAULT_ROBUSTNESS_ITERS;
}
public LoessInterpolator(double bandwidth, int robustnessIters)
{
if (bandwidth < 0 || bandwidth > 1)
{
throw new ApplicationException(string.Format("bandwidth must be in the interval [0,1], but got {0}", bandwidth));
}
this.bandwidth = bandwidth;
if (robustnessIters < 0)
{
throw new ApplicationException(string.Format("the number of robustness iterations must be non-negative, but got {0}", robustnessIters));
}
this.robustnessIters = robustnessIters;
}
/**
* Compute a loess fit on the data at the original abscissae.
*
* #param xval the arguments for the interpolation points
* #param yval the values for the interpolation points
* #return values of the loess fit at corresponding original abscissae
* #throws MathException if some of the following conditions are false:
* <ul>
* <li> Arguments and values are of the same size that is greater than zero</li>
* <li> The arguments are in a strictly increasing order</li>
* <li> All arguments and values are finite real numbers</li>
* </ul>
*/
public double[] smooth(double[] xval, double[] yval)
{
if (xval.Length != yval.Length)
{
throw new ApplicationException(string.Format("Loess expects the abscissa and ordinate arrays to be of the same size, but got {0} abscisssae and {1} ordinatae", xval.Length, yval.Length));
}
int n = xval.Length;
if (n == 0)
{
throw new ApplicationException("Loess expects at least 1 point");
}
checkAllFiniteReal(xval, true);
checkAllFiniteReal(yval, false);
checkStrictlyIncreasing(xval);
if (n == 1)
{
return new double[] { yval[0] };
}
if (n == 2)
{
return new double[] { yval[0], yval[1] };
}
int bandwidthInPoints = (int)(bandwidth * n);
if (bandwidthInPoints < 2)
{
throw new ApplicationException(string.Format("the bandwidth must be large enough to accomodate at least 2 points. There are {0} " +
" data points, and bandwidth must be at least {1} but it is only {2}",
n, 2.0 / n, bandwidth
));
}
double[] res = new double[n];
double[] residuals = new double[n];
double[] sortedResiduals = new double[n];
double[] robustnessWeights = new double[n];
// Do an initial fit and 'robustnessIters' robustness iterations.
// This is equivalent to doing 'robustnessIters+1' robustness iterations
// starting with all robustness weights set to 1.
for (int i = 0; i < robustnessWeights.Length; i++) robustnessWeights[i] = 1;
for (int iter = 0; iter <= robustnessIters; ++iter)
{
int[] bandwidthInterval = { 0, bandwidthInPoints - 1 };
// At each x, compute a local weighted linear regression
for (int i = 0; i < n; ++i)
{
double x = xval[i];
// Find out the interval of source points on which
// a regression is to be made.
if (i > 0)
{
updateBandwidthInterval(xval, i, bandwidthInterval);
}
int ileft = bandwidthInterval[0];
int iright = bandwidthInterval[1];
// Compute the point of the bandwidth interval that is
// farthest from x
int edge;
if (xval[i] - xval[ileft] > xval[iright] - xval[i])
{
edge = ileft;
}
else
{
edge = iright;
}
// Compute a least-squares linear fit weighted by
// the product of robustness weights and the tricube
// weight function.
// See http://en.wikipedia.org/wiki/Linear_regression
// (section "Univariate linear case")
// and http://en.wikipedia.org/wiki/Weighted_least_squares
// (section "Weighted least squares")
double sumWeights = 0;
double sumX = 0, sumXSquared = 0, sumY = 0, sumXY = 0;
double denom = Math.Abs(1.0 / (xval[edge] - x));
for (int k = ileft; k <= iright; ++k)
{
double xk = xval[k];
double yk = yval[k];
double dist;
if (k < i)
{
dist = (x - xk);
}
else
{
dist = (xk - x);
}
double w = tricube(dist * denom) * robustnessWeights[k];
double xkw = xk * w;
sumWeights += w;
sumX += xkw;
sumXSquared += xk * xkw;
sumY += yk * w;
sumXY += yk * xkw;
}
double meanX = sumX / sumWeights;
double meanY = sumY / sumWeights;
double meanXY = sumXY / sumWeights;
double meanXSquared = sumXSquared / sumWeights;
double beta;
if (meanXSquared == meanX * meanX)
{
beta = 0;
}
else
{
beta = (meanXY - meanX * meanY) / (meanXSquared - meanX * meanX);
}
double alpha = meanY - beta * meanX;
res[i] = beta * x + alpha;
residuals[i] = Math.Abs(yval[i] - res[i]);
}
// No need to recompute the robustness weights at the last
// iteration, they won't be needed anymore
if (iter == robustnessIters)
{
break;
}
// Recompute the robustness weights.
// Find the median residual.
// An arraycopy and a sort are completely tractable here,
// because the preceding loop is a lot more expensive
System.Array.Copy(residuals, sortedResiduals, n);
//System.arraycopy(residuals, 0, sortedResiduals, 0, n);
Array.Sort<double>(sortedResiduals);
double medianResidual = sortedResiduals[n / 2];
if (medianResidual == 0)
{
break;
}
for (int i = 0; i < n; ++i)
{
double arg = residuals[i] / (6 * medianResidual);
robustnessWeights[i] = (arg >= 1) ? 0 : Math.Pow(1 - arg * arg, 2);
}
}
return res;
}
/**
* Given an index interval into xval that embraces a certain number of
* points closest to xval[i-1], update the interval so that it embraces
* the same number of points closest to xval[i]
*
* #param xval arguments array
* #param i the index around which the new interval should be computed
* #param bandwidthInterval a two-element array {left, right} such that: <p/>
* <tt>(left==0 or xval[i] - xval[left-1] > xval[right] - xval[i])</tt>
* <p/> and also <p/>
* <tt>(right==xval.length-1 or xval[right+1] - xval[i] > xval[i] - xval[left])</tt>.
* The array will be updated.
*/
private static void updateBandwidthInterval(double[] xval, int i, int[] bandwidthInterval)
{
int left = bandwidthInterval[0];
int right = bandwidthInterval[1];
// The right edge should be adjusted if the next point to the right
// is closer to xval[i] than the leftmost point of the current interval
int nextRight = nextNonzero(weights, right);
if (nextRight < xval.Length && xval[nextRight] - xval[i] < xval[i] - xval[left])
{
int nextLeft = nextNonzero(weights, bandwidthInterval[0]);
bandwidthInterval[0] = nextLeft;
bandwidthInterval[1] = nextRight;
}
}
/**
* Compute the
* tricube
* weight function
*
* #param x the argument
* #return (1-|x|^3)^3
*/
private static double tricube(double x)
{
double tmp = Math.abs(x);
tmp = 1 - tmp * tmp * tmp;
return tmp * tmp * tmp;
}
/**
* Check that all elements of an array are finite real numbers.
*
* #param values the values array
* #param isAbscissae if true, elements are abscissae otherwise they are ordinatae
* #throws MathException if one of the values is not
* a finite real number
*/
private static void checkAllFiniteReal(double[] values, bool isAbscissae)
{
for (int i = 0; i < values.Length; i++)
{
double x = values[i];
if (Double.IsInfinity(x) || Double.IsNaN(x))
{
string pattern = isAbscissae ?
"all abscissae must be finite real numbers, but {0}-th is {1}" :
"all ordinatae must be finite real numbers, but {0}-th is {1}";
throw new ApplicationException(string.Format(pattern, i, x));
}
}
}
/**
* Check that elements of the abscissae array are in a strictly
* increasing order.
*
* #param xval the abscissae array
* #throws MathException if the abscissae array
* is not in a strictly increasing order
*/
private static void checkStrictlyIncreasing(double[] xval)
{
for (int i = 0; i < xval.Length; ++i)
{
if (i >= 1 && xval[i - 1] >= xval[i])
{
throw new ApplicationException(string.Format(
"the abscissae array must be sorted in a strictly " +
"increasing order, but the {0}-th element is {1} " +
"whereas {2}-th is {3}",
i - 1, xval[i - 1], i, xval[i]));
}
}
}
}

Since I'm unable to comment on other people's posts (new user), and people seem to think I should do that with this instead of editing the above answer, I'm simply going to write it as an answer even though I know this is better as a comment.
The updateBandwidthInterval method in the above answer forgets to check the left side as written in the method comment. This can give NaN issues for sumWeights. The below should fix that. I encountered this when doing a c++ implementation based on the above.
/**
* Given an index interval into xval that embraces a certain number of
* points closest to xval[i-1], update the interval so that it embraces
* the same number of points closest to xval[i]
*
* #param xval arguments array
* #param i the index around which the new interval should be computed
* #param bandwidthInterval a two-element array {left, right} such that: <p/>
* <tt>(left==0 or xval[i] - xval[left-1] > xval[right] - xval[i])</tt>
* <p/> and also <p/>
* <tt>(right==xval.length-1 or xval[right+1] - xval[i] > xval[i] - xval[left])</tt>.
* The array will be updated.
*/
private static void updateBandwidthInterval(double[] xval, int i, int[] bandwidthInterval)
{
int left = bandwidthInterval[0];
int right = bandwidthInterval[1];
// The edges should be adjusted if the previous point to the
// left is closer to x than the current point to the right or
// if the next point to the right is closer
// to x than the leftmost point of the current interval
if (left != 0 &&
xval[i] - xval[left - 1] < xval[right] - xval[i])
{
bandwidthInterval[0]++;
bandwidthInterval[1]++;
}
else if (right < xval.Length - 1 &&
xval[right + 1] - xval[i] < xval[i] - xval[left])
{
bandwidthInterval[0]++;
bandwidthInterval[1]++;
}
}

Hope someone after 5 years find this useful. This is the original code posted by Tutcugil but with the missing methods and updated.
using System;
using System.Linq;
namespace StockCorrelation
{
public class LoessInterpolator
{
public static double DEFAULT_BANDWIDTH = 0.3;
public static int DEFAULT_ROBUSTNESS_ITERS = 2;
/**
* The bandwidth parameter: when computing the loess fit at
* a particular point, this fraction of source points closest
* to the current point is taken into account for computing
* a least-squares regression.
*
* A sensible value is usually 0.25 to 0.5.
*/
private double bandwidth;
/**
* The number of robustness iterations parameter: this many
* robustness iterations are done.
*
* A sensible value is usually 0 (just the initial fit without any
* robustness iterations) to 4.
*/
private int robustnessIters;
public LoessInterpolator()
{
this.bandwidth = DEFAULT_BANDWIDTH;
this.robustnessIters = DEFAULT_ROBUSTNESS_ITERS;
}
public LoessInterpolator(double bandwidth, int robustnessIters)
{
if (bandwidth < 0 || bandwidth > 1)
{
throw new ApplicationException(string.Format("bandwidth must be in the interval [0,1], but got {0}", bandwidth));
}
this.bandwidth = bandwidth;
if (robustnessIters < 0)
{
throw new ApplicationException(string.Format("the number of robustness iterations must be non-negative, but got {0}", robustnessIters));
}
this.robustnessIters = robustnessIters;
}
/**
* Compute a loess fit on the data at the original abscissae.
*
* #param xval the arguments for the interpolation points
* #param yval the values for the interpolation points
* #return values of the loess fit at corresponding original abscissae
* #throws MathException if some of the following conditions are false:
* <ul>
* <li> Arguments and values are of the same size that is greater than zero</li>
* <li> The arguments are in a strictly increasing order</li>
* <li> All arguments and values are finite real numbers</li>
* </ul>
*/
public double[] smooth(double[] xval, double[] yval, double[] weights)
{
if (xval.Length != yval.Length)
{
throw new ApplicationException(string.Format("Loess expects the abscissa and ordinate arrays to be of the same size, but got {0} abscisssae and {1} ordinatae", xval.Length, yval.Length));
}
int n = xval.Length;
if (n == 0)
{
throw new ApplicationException("Loess expects at least 1 point");
}
checkAllFiniteReal(xval, true);
checkAllFiniteReal(yval, false);
checkStrictlyIncreasing(xval);
if (n == 1)
{
return new double[] { yval[0] };
}
if (n == 2)
{
return new double[] { yval[0], yval[1] };
}
int bandwidthInPoints = (int)(bandwidth * n);
if (bandwidthInPoints < 2)
{
throw new ApplicationException(string.Format("the bandwidth must be large enough to accomodate at least 2 points. There are {0} " +
" data points, and bandwidth must be at least {1} but it is only {2}",
n, 2.0 / n, bandwidth
));
}
double[] res = new double[n];
double[] residuals = new double[n];
double[] sortedResiduals = new double[n];
double[] robustnessWeights = new double[n];
// Do an initial fit and 'robustnessIters' robustness iterations.
// This is equivalent to doing 'robustnessIters+1' robustness iterations
// starting with all robustness weights set to 1.
for (int i = 0; i < robustnessWeights.Length; i++) robustnessWeights[i] = 1;
for (int iter = 0; iter <= robustnessIters; ++iter)
{
int[] bandwidthInterval = { 0, bandwidthInPoints - 1 };
// At each x, compute a local weighted linear regression
for (int i = 0; i < n; ++i)
{
double x = xval[i];
// Find out the interval of source points on which
// a regression is to be made.
if (i > 0)
{
updateBandwidthInterval(xval, weights, i, bandwidthInterval);
}
int ileft = bandwidthInterval[0];
int iright = bandwidthInterval[1];
// Compute the point of the bandwidth interval that is
// farthest from x
int edge;
if (xval[i] - xval[ileft] > xval[iright] - xval[i])
{
edge = ileft;
}
else
{
edge = iright;
}
// Compute a least-squares linear fit weighted by
// the product of robustness weights and the tricube
// weight function.
// See http://en.wikipedia.org/wiki/Linear_regression
// (section "Univariate linear case")
// and http://en.wikipedia.org/wiki/Weighted_least_squares
// (section "Weighted least squares")
double sumWeights = 0;
double sumX = 0, sumXSquared = 0, sumY = 0, sumXY = 0;
double denom = Math.Abs(1.0 / (xval[edge] - x));
for (int k = ileft; k <= iright; ++k)
{
double xk = xval[k];
double yk = yval[k];
double dist;
if (k < i)
{
dist = (x - xk);
}
else
{
dist = (xk - x);
}
double w = tricube(dist * denom) * robustnessWeights[k];
double xkw = xk * w;
sumWeights += w;
sumX += xkw;
sumXSquared += xk * xkw;
sumY += yk * w;
sumXY += yk * xkw;
}
double meanX = sumX / sumWeights;
double meanY = sumY / sumWeights;
double meanXY = sumXY / sumWeights;
double meanXSquared = sumXSquared / sumWeights;
double beta;
if (meanXSquared == meanX * meanX)
{
beta = 0;
}
else
{
beta = (meanXY - meanX * meanY) / (meanXSquared - meanX * meanX);
}
double alpha = meanY - beta * meanX;
res[i] = beta * x + alpha;
residuals[i] = Math.Abs(yval[i] - res[i]);
}
// No need to recompute the robustness weights at the last
// iteration, they won't be needed anymore
if (iter == robustnessIters)
{
break;
}
// Recompute the robustness weights.
// Find the median residual.
// An arraycopy and a sort are completely tractable here,
// because the preceding loop is a lot more expensive
System.Array.Copy(residuals, sortedResiduals, n);
//System.arraycopy(residuals, 0, sortedResiduals, 0, n);
Array.Sort<double>(sortedResiduals);
double medianResidual = sortedResiduals[n / 2];
if (medianResidual == 0)
{
break;
}
for (int i = 0; i < n; ++i)
{
double arg = residuals[i] / (6 * medianResidual);
robustnessWeights[i] = (arg >= 1) ? 0 : Math.Pow(1 - arg * arg, 2);
}
}
return res;
}
public double[] smooth(double[] xval, double[] yval)
{
if (xval.Length != yval.Length)
{
throw new Exception($"xval and yval len are different");
}
double[] unitWeights = Enumerable.Repeat(1.0, xval.Length).ToArray();
return smooth(xval, yval, unitWeights);
}
/**
* Given an index interval into xval that embraces a certain number of
* points closest to xval[i-1], update the interval so that it embraces
* the same number of points closest to xval[i]
*
* #param xval arguments array
* #param i the index around which the new interval should be computed
* #param bandwidthInterval a two-element array {left, right} such that: <p/>
* <tt>(left==0 or xval[i] - xval[left-1] > xval[right] - xval[i])</tt>
* <p/> and also <p/>
* <tt>(right==xval.length-1 or xval[right+1] - xval[i] > xval[i] - xval[left])</tt>.
* The array will be updated.
*/
private static void updateBandwidthInterval(double[] xval, double[] weights,
int i,
int[] bandwidthInterval)
{
int left = bandwidthInterval[0];
int right = bandwidthInterval[1];
// The right edge should be adjusted if the next point to the right
// is closer to xval[i] than the leftmost point of the current interval
int nextRight = nextNonzero(weights, right);
if (nextRight < xval.Length && xval[nextRight] - xval[i] < xval[i] - xval[left])
{
int nextLeft = nextNonzero(weights, bandwidthInterval[0]);
bandwidthInterval[0] = nextLeft;
bandwidthInterval[1] = nextRight;
}
}
private static int nextNonzero(double[] weights, int i)
{
int j = i + 1;
while (j < weights.Length && weights[j] == 0)
{
++j;
}
return j;
}
/**
* Compute the
* tricube
* weight function
*
* #param x the argument
* #return (1-|x|^3)^3
*/
private static double tricube(double x)
{
double tmp = Math.Abs(x);
tmp = 1 - tmp * tmp * tmp;
return tmp * tmp * tmp;
}
/**
* Check that all elements of an array are finite real numbers.
*
* #param values the values array
* #param isAbscissae if true, elements are abscissae otherwise they are ordinatae
* #throws MathException if one of the values is not
* a finite real number
*/
private static void checkAllFiniteReal(double[] values, bool isAbscissae)
{
for (int i = 0; i < values.Length; i++)
{
double x = values[i];
if (Double.IsInfinity(x) || Double.IsNaN(x))
{
string pattern = isAbscissae ?
"all abscissae must be finite real numbers, but {0}-th is {1}" :
"all ordinatae must be finite real numbers, but {0}-th is {1}";
throw new ApplicationException(string.Format(pattern, i, x));
}
}
}
/**
* Check that elements of the abscissae array are in a strictly
* increasing order.
*
* #param xval the abscissae array
* #throws MathException if the abscissae array
* is not in a strictly increasing order
*/
private static void checkStrictlyIncreasing(double[] xval)
{
for (int i = 0; i < xval.Length; ++i)
{
if (i >= 1 && xval[i - 1] >= xval[i])
{
throw new ApplicationException(string.Format(
"the abscissae array must be sorted in a strictly " +
"increasing order, but the {0}-th element is {1} " +
"whereas {2}-th is {3}",
i - 1, xval[i - 1], i, xval[i]));
}
}
}
}
}

Array data normalization

I have an array of values (between -1.0 and 1.0) that represent intensity (Black to White). I need a way to map the double values from -1.0 through 1.0 to 0 through 255 and back.
More generalized, I have an array of data and I need to map from the min and max value of the data to a supplied min and max. Basic structure should be like:
private static int[] NormalizeData(double[] data, int min, int max)
{
var sorted = data.OrderBy(d => d);
double dataMax = sorted.First();
double dataMin = sorted.Last();
int[] ret = new int[data.Length];
for (int i = 0; i < data.Length; i++)
{
ret[i] = (int)data[i]; // Normalization here
}
return ret;
}

This works:
private static int[] NormalizeData(IEnumerable<double> data, int min, int max)
{
double dataMax = data.Max();
double dataMin = data.Min();
double range = dataMax - dataMin;
return data
.Select(d => (d - dataMin) / range)
.Select(n => (int)((1 - n) * min + n * max))
.ToArray();
}
The first select normalizes the input to be from 0 to 1 (0 being minimum, 1 being the maximum). The second select takes that normalized number, and maps it to the new minimum and maximum.
Note that using the LINQ Min() and Max() functions are faster than sorting the input for larger datasets: O(n) vs. O(n * lg(n)).
Also, if you want to go the other way, then you'll want it to return doubles instead of ints.

public static double Scale(this double elementToScale,
double rangeMin, double rangeMax,
double scaledRangeMin, double scaledRangeMax)
{
var scaled = scaledRangeMin + ((elementToScale - rangeMin) * (scaledRangeMax - scaledRangeMin) / (rangeMax - rangeMin));
return scaled;
}
Usage:
// double [-1,1] to int [0-255]
int[] integers = doubles.Select(x => x.Scale(-1,1,0,255)).ToArray();
// int [0-255] to double [-1,1]
double[] doubles = integers.Select(x => ((double)x).Scale(0,255,-1,1)).ToArray();
If you don't know the min and max in advance ([0-255] and [-1,1] in the example), you can use LINQ Min() and Max()

private static int[] NormalizeData(double[] data, int min, int max) {
int[] ret = new int[data.Length];
for (int i = 0; i < data.Length; i++) {
ret[i] = (int)((max * (data[i] + 1)) / 2);
}
return ret;
}
static void Main(string[] args) {
double[] data = { 1.0, -1, 0, -.5, .5 };
int[] normalized = NormalizeData(data, 0, 255);
foreach (var v in normalized) {
Console.WriteLine(v);
}
}

EDIT:
How about this:
private static int[] NormalizeData(double[] data, int min, int max)
{
var sorted = data.OrderBy(d => d);
double dataMax = sorted.First();
double dataMin = sorted.Last();
int[] ret = new int[data.Length];
double avgIn = (double)((min + max) / 2.0);
double avgOut = (dataMax + dataMin) / 2.0);
for (int i = 0; i < data.Length; i++)
{
ret[i] = (int) Math.Round(avgOut * (data[i] + avgIn) / 2);
}
return ret;
}

Assuming a strictly linear transformation and that you want dataMin to map to min and dataMax to map to max:
double dataRange = dataMax - dataMin;
int newRange = max - min;
double pct = (data[i] - dataMin) / dataRange;
int newValue = Math.Round(min + (pct * newRange));
That can certainly be optimized, but it shows the basic idea. Basically, you figure out the position (as a percentage) of the value in the original range and then map that percentage to the target range.
Note that if dataMin is -0.5 and dataMax is 0.5, this might not produce the results that you're looking for because -0.5 will map to 0 and 0.5 will map to 255. If you want things to map exactly as stated, you'll have to define the source range as well.
As an aside, there's no particular reason to sort the items just to get the min and max. You can write:
double dataMax = data.Max();
double dataMin = data.Min();

To be able to normalize your array which in this example acts a vector mathematically you need to define what length the vector is in (how many dimensions).
It's not really clear from the example if you want to normalize the entire array taking all elements in the array into account. If so then you calculate the dot product of the array, store the dot products square root as the length of the array. then you divide every term with that length to normalize the array to a length of 1.0.
In the case above you did not actually describe a normalization of the data but a conversion. To solve that you could use something like the following:
private static double[] convertToScale(double[] data, double oldMin, double oldMax,double min, double max)
{
double oldDiff = 0 - oldMin;
double oldScale = oldMax - oldMin;
double diff = 0 - min;
double scale = max - min;
int[] ret = new double[data.Length];
for (int i = 0; i < data.Length; i++)
{
double scaledFromZeroToOne = (oldDiff+data[i])/oldScale; // Normalization here [0,1]
double value = (scaledFromZeroToOne*scale)-diff;
ret[i] = value;
}
return ret;
}
This function i believe would solve the problem described above.
You can call it like following row:
double[] result = convertToScale(input,-1.0,1.0,0,255);
And then cast everything to int if you'd rather have the values represented as ints.
Hope it helps.