I'm trying to implement logistic regression by myself writing code in C#. I found a library (Accord.NET) that I use to minimize the cost function. However I'm always getting different minimums. Therefore I think something may be wrong in the cost function that I wrote.
static double costfunction(double[] thetas)
{
int i = 0;
double sum = 0;
double[][] theta_matrix_transposed = MatrixCreate(1, thetas.Length);
while(i!=thetas.Length) { theta_matrix_transposed[0][i] = thetas[i]; i++;}
i = 0;
while (i != m) // m is the number of examples
{
int z = 0;
double[][] x_matrix = MatrixCreate(thetas.Length, 1);
while (z != thetas.Length) { x_matrix[z][0] = x[z][i]; z++; } //Put values from the training set to the matrix
double p = MatrixProduct(theta_matrix_transposed, x_matrix)[0][0];
sum += y[i] * Math.Log(sigmoid(p)) + (1 - y[i]) * Math.Log(1 - sigmoid(p));
i++;
}
double value = (-1 / m) * sum;
return value;
}
static double sigmoid(double z)
{
return 1 / (1 + Math.Exp(-z));
}
x is a list of lists that represent the training set, one list for each feature. What's wrong with the code? Why am I getting different results every time I run the L-BFGS? Thank you for your patience, I'm just getting started with machine learning!
That is very common with these optimization algorithms - the minima you arrive at depends on your weight initialization. The fact that you are getting different minimums doesn't necessarily mean something is wrong with your implementation. Instead, check your gradients to make sure they are correct using the finite differences method, and also look at your train/validation/test accuracy to see if they are also acceptable.
Related
I am trying to figure out what the difference between the following for loops is.
The first is code that I wrote while practicing algorithms on codewars.com. It times out when attempting the larger test cases.
The second is one of the top solutions. It seems functionally similar (obviously its more concise) but runs much faster and does not time out. Can anyone explain to me what the difference is? Also, the return statement in the second snippet is confusing to me. What exactly does this syntax mean? Maybe this is where it is more efficient.
public static long findNb(long m)
{
int sum = 0;
int x = new int();
for (int n = 0; sum < m; n++)
{
sum += n*n*n;
x = n;
System.Console.WriteLine(x);
}
if (sum == m)
{
return x;
}
return -1;
}
vs
public static long findNb(long m) //seems similar but doesnt time out
{
long total = 1, i = 2;
for(; total < m; i++) total += i * i * i;
return total == m ? i - 1 : -1;
}
The second approach uses long for the total value. Chances are that you're using an m value that's high enough to exceed the number of values representable by int. So your math overflows and the n value becomes a negative number. You get caught in an infinite loop, where n can never get as big as m.
And, like everyone else says, get rid of the WriteLine.
Also, the return statement in the second snippet is confusing to me. What exactly does this syntax mean?
It's a ternary conditional operator.
Both approaches are roughly the same, except unwanted System.Console.WriteLine(x); which spolis the fun: printing on the Console (UI!) is a slow operation.
If you are looking for a fast solution (esp. for the large m and long loop) you can just precompute all (77936) values:
public class Solver {
static Dictionary<long, long> s_Sums = new Dictionary<long, long>();
private static void Build() {
long total = 0;
for (long i = 0; i <= 77936; ++i) {
total += i * i * i;
s_Sums.Add(total, i);
}
}
static Solver()
Build();
}
public static long findNb(long m) {
return s_Sums.TryGetValue(m, out long result)
? result
: -1;
}
}
When I run into micro optimisation challenges like this, I always use BenchmarkDotnet. It's the tool to use to get all the insights to performance, memory allocations, deviations in .NET Framework versions, 64bit vs 32 bit etc. etc.
But as others write - remember to remove the WriteLine() statement :)
I have been practicing c# nowadays and decided to write a code that converts any decimal to any base representation for practice purpose. And i have some troubles. Since i want to practice i decided to do it with an additional function where calculations take place. First i wanted to use an array to keep my result. But since ,at the beginning, i do not know the length of the array i could not define it.So i decided to use list(somehow i assumed undeclared slots are 0 as default). This is what i end up with.
class MainClass
{
static double number;
static double baseToConvert;
static int counter = 0;
static List<double> converted = new List<double>();
public static void Main(string[] args)
{
Console.WriteLine("Enter a decimal");
number = double.Parse(Console.ReadLine());
Console.WriteLine("Enter a base you want to convert to");
baseToConvert = double.Parse(Console.ReadLine());
ConverterToBase(number);
for (int i = converted.Count - 1; i >= 0; i--)
{
Console.WriteLine(converted[i]);
}
Console.ReadLine();
}
public static void ConverterToBase(double x)
{
double temp = x;
while (x >= baseToConvert)
{
x /= baseToConvert;
counter++;
}
converted[counter] = x;
counter = 0;
if (temp - x * Math.Pow(baseToConvert, Convert.ToDouble(counter)) >= baseToConvert)
{
ConverterToBase(temp - x * Math.Pow(baseToConvert, Convert.ToDouble(counter)));
}
else
{
converted[0] = temp - x * Math.Pow(baseToConvert, Convert.ToDouble(counter));
}
}
}
But after i write inputs console gets stuck without an error. My guess is that since i do not have any elements in the list " converted[counter] " does not make sense. But i do not know maybe the problem is somewhere else.
My question is not about the way i calculate the problem(Of course any suggestions are welcomed). I just want to know what i am doing wrong and how i can handle such situation(unknown array size , use of list , accessing a variable,array,.. etc from another method... ).
Thanks.
My previous answer was wrong as pointed out by #Rufus L. There is no infinite for loop. However upon further review, there seems to be an infinite recursion going on in your code in this line:
if (temp - x * Math.Pow(baseToConvert, Convert.ToDouble(counter)) >= baseToConvert)
{
ConverterToBase(temp - x * Math.Pow(baseToConvert, Convert.ToDouble(counter)));
}
ConverterToBase calls itself and there seems to be no base case nor return statement to end the recursion.
In the method named "ConverterToBase(double x)" you want to set value of 0 element. But you didn't add any element. The converted is Empty.
Firstly add value or values to your list.
This is a formula to approximate arcsine(x) using Taylor series from this blog
This is my implementation in C#, I don't know where is the wrong place, the code give wrong result when running:
When i = 0, the division will be 1/x. So I assign temp = 1/x at startup. For each iteration, I change "temp" after "i".
I use a continual loop until the two next value is very "near" together. When the delta of two next number is very small, I will return the value.
My test case:
Input is x =1, so excected arcsin(X) will be arcsin (1) = PI/2 = 1.57079633 rad.
class Arc{
static double abs(double x)
{
return x >= 0 ? x : -x;
}
static double pow(double mu, long n)
{
double kq = mu;
for(long i = 2; i<= n; i++)
{
kq *= mu;
}
return kq;
}
static long fact(long n)
{
long gt = 1;
for (long i = 2; i <= n; i++) {
gt *= i;
}
return gt;
}
#region arcsin
static double arcsinX(double x) {
int i = 0;
double temp = 0;
while (true)
{
//i++;
var iFactSquare = fact(i) * fact(i);
var tempNew = (double)fact(2 * i) / (pow(4, i) * iFactSquare * (2*i+1)) * pow(x, 2 * i + 1) ;
if (abs(tempNew - temp) < 0.00000001)
{
return tempNew;
}
temp = tempNew;
i++;
}
}
public static void Main(){
Console.WriteLine(arcsin());
Console.ReadLine();
}
}
In many series evaluations, it is often convenient to use the quotient between terms to update the term. The quotient here is
(2n)!*x^(2n+1) 4^(n-1)*((n-1)!)^2*(2n-1)
a[n]/a[n-1] = ------------------- * --------------------- -------
(4^n*(n!)^2*(2n+1)) (2n-2)!*x^(2n-1)
=(2n(2n-1)²x²)/(4n²(2n+1))
= ((2n-1)²x²)/(2n(2n+1))
Thus a loop to compute the series value is
sum = 1;
term = 1;
n=1;
while(1 != 1+term) {
term *= (n-0.5)*(n-0.5)*x*x/(n*(n+0.5));
sum += term;
n += 1;
}
return x*sum;
The convergence is only guaranteed for abs(x)<1, for the evaluation at x=1 you have to employ angle halving, which in general is a good idea to speed up convergence.
You are saving two different temp values (temp and tempNew) to check whether or not continuing computation is irrelevant. This is good, except that you are not saving the sum of these two values.
This is a summation. You need to add every new calculated value to the total. You are only keeping track of the most recently calculated value. You can only ever return the last calculated value of the series. So you will always get an extremely small number as your result. Turn this into a summation and the problem should go away.
NOTE: I've made this a community wiki answer because I was hardly the first person to think of this (just the first to put it down in a comment). If you feel that more needs to be added to make the answer complete, just edit it in!
The general suspicion is that this is down to Integer Overflow, namely one of your values (probably the return of fact() or iFactSquare()) is getting too big for the type you have chosen. It's going to negative because you are using signed types — when it gets to too large a positive number, it loops back into the negative.
Try tracking how large n gets during your calculation, and figure out how big a number it would give you if you ran that number through your fact, pow and iFactSquare functions. If it's bigger than the Maximum long value in 64-bit like we think (assuming you're using 64-bit, it'll be a lot smaller for 32-bit), then try using a double instead.
I hope somebody can be of assistance, thank you in advance.
I am using C# to generate some simulation models for evaluating the medium stationary time of a request in a system and the degree of usage of that serving station from the system.
I am using a function to generate the required numbers:
public double genTP(double miu)
{
Random random = new Random();
double u, x;
u = (double)random.NextDouble();
x = (-1 / miu) * Math.Log(1 - u);
return x;
}
This is the main:
Program p1 = new Program();
double NS = 1000000;
double lambda = 4;
double miu = 10;
double STP = 0;
double STS = 0;
double STL = 0;
double i = 1;
double Ta = 0;
double Tp = 0;
double Dis = 0;
do
{
Tp = p1.genTP(miu);
STP += Tp;
STS += Ta + Tp;
Dis = p1.genDIS(lambda);
if (Dis < Ta + Tp)
{
Ta = Ta + Tp - Dis;
}
else
{
STL += Dis - (Ta + Tp);
Ta = 0;
}
i++;
} while (i <= NS);
Console.WriteLine(STS / NS);
Console.WriteLine((STP/(STP+STL))*100);
1) The medium stationary time (which is r) returned is wrong, I get values like 0.09.. but I should get something like ~0.1665.. The algorithm is ok, I am 100% sure of that, I tried something like that in Matlab and it was good. Also the degree of usage (the last line) returned is ok (around ~39.89), only the r is wrong. Could it be a problem with the function, especially the random function that should generate a number?
2)Regarding my function genTP, if I change the parameter from double to int, then it returns 0 at the end. I used debugger to check why is that, and I saw that when the method calculates the value of x with (-1 / miu), this returns 0 automatically, I have tried to cast to double but with no result. I was thinking that this could be a source of the problem.
You're creating a new instance of Random each time you call genTP. If this is called multiple times in quick succession (as it is) then it will use the same seed each time. See my article on random numbers for more information.
Create a single instance of Random and pass it into the method:
private static double GenerateTP(Random random, double miu)
{
double u = random.NextDouble();
return (-1 / miu) * Math.Log(1 - u);
}
And...
Random random = new Random();
do
{
double tp = GenerateTP(random, miu);
...
}
A few more suggestions:
Declare your variables at the point of first use, with minimal scope
Follow .NET naming conventions
Don't make methods instance methods if they don't use any state
I prefer to create a static random field in the calculation class
static Random random = new Random();
Now I can use it without thinking of quick succession and I don't need to give it as a function parameter (I'm not trying to say it works faster but just similar to mathematical notation)
Regarding your second question, it is because the compiler does an integer division because they are two integers.
int / int = int (and the result is truncated).
If any of the args are floating point types, the operation is promoted to a floating point division. If you change your arg to an int, you should either use -1d (a double with value -1), or cast 'miu' to a double before usage:
x = (-1d / miu) ...
or
x = (-1 / (double)miu) ...
Or, of course, use them both as doubles. I prefer the first way.
I'm looking for a library or existing code to simplify fractions.
Does anyone have anything at hand or any links?
P.S. I already understand the process but really don't want to rewrite the wheel
Update
Ok i've checked out the fraction library on the CodeProject
BUT the problem I have is a little bit tricker than simplifying a fraction.
I have to reduce a percentage split which could be 20% / 50% / 30% (always equal to 100%)
I think you just need to divide by the GCD of all the numbers.
void Simplify(int[] numbers)
{
int gcd = GCD(numbers);
for (int i = 0; i < numbers.Length; i++)
numbers[i] /= gcd;
}
int GCD(int a, int b)
{
while (b > 0)
{
int rem = a % b;
a = b;
b = rem;
}
return a;
}
int GCD(int[] args)
{
// using LINQ:
return args.Aggregate((gcd, arg) => GCD(gcd, arg));
}
I haven't tried the code, but it seems simple enough to be right (assuming your numbers are all positive integers and you don't pass an empty array).
You can use Microsoft.FSharp.Math.BigRational, which is in the free F# Power Pack library. Although it depends on F# (which is gratis and included in VS2010), it can be used from C#.
BigRational reduced = BigRational.FromInt(4)/BigRational.FromInt(6);
Console.WriteLine(reduced);
2/3
Console.WriteLine(reduced.Numerator);
2
Console.WriteLine(reduced.Denominator);
3
This library looks like it might be what you need:
var f = new Fraction(numerator, denominator);
numerator = f.Numerator;
denominator = f.Denominator;
Although, I haven't tested it, so it looks like you may need to play around with it to get it to work.
The best example of Fraction (aka Rational) I've seen is in Timothy Budd's "Classic Data Structures in C++". His implementation is very good. It includes a simple implementation of GCD algorithm.
It shouldn't be hard to adapt to C#.
A custom solution:
void simplify(int[] numbers)
{
for (int divideBy = 50; divideBy > 0; divideBy--)
{
bool divisible = true;
foreach (int cur in numbers)
{
//check for divisibility
if ((int)(cur/divideBy)*divideBy!=cur){
divisible = false;
break;
}
}
if (divisible)
{
for (int i = 0; i < numbers.GetLength(0);i++ )
{
numbers[i] /= divideBy;
}
}
}
}
Example usage:
int [] percentages = {20,30,50};
simplify(percentages);
foreach (int p in percentages)
{
Console.WriteLine(p);
}
Outupts:
2
3
5
By the way, this is my first c# program. Thought it would simply be a fun problem to try a new language with, and now I'm in love! It's like Java, but everything I wish was a bit different is exactly how I wanted it
<3 c#
Edit: Btw don't forget to make it static void if it's for your Main class.