i'm new here and my English isn't very good, so i'll try to explain as well as possible.
I'm doing a web application in ASP.NET and C# about steganalysis.
I was looking for internet a function that calculates the observed significance level, or p-value in a chi-square test
for my algorithm and I found it in Java:
This is the result of mi search:
chi[block]= chiSquareTest(expectedValues, pod);
chiSquareTest(double[] expected, long[] observed)
Returns the observed significance level, or p-value,
associated with a Chi-square goodness of fit test comparing
the observed frequency counts to those in the expected array.
My question is, Are there any equivalent function in C# that returns the same parameter?
Thank you in advance,
Ana.
The MathNet nugetpackage contains the ChiSquared distribution - you can get the cumulative or inverse cumulative.
ChiSquared c = new ChiSquared(degreesOfFreedom);
return c.CumulativeDistribution(testValue);
I doubt there is an inbuilt function.
You should try looking for a library that contains the function or implement it yourself.
A quick search returned me this
http://www.alglib.net/specialfunctions/distributions/chisquare.php
and
http://www.codeproject.com/Articles/432194/How-to-Calculate-the-Chi-Squared-P-Value
I know this is an old question but I figured I'd post anyway.
The Accord.NET framework has a library and NuGet package Accord.Statistics which has a ChiSquareTest class that can be used in a similar manor as mentioned in your question:
ChiSquareTest chiSquareTest = new ChiSquareTest(observedArray, expectedArray, DOF);
chiSquareTest.PValue; // gets p-value
chiSquareTest.Significant; // true if statistically significant
The only thing is that you'll have to calculate your DOF - degrees of freedom.
I have values greater than 1.97626258336499E-323
I cant use BigInteger also as it handler only integer values
Any help is appreciated
Here is the code that failed also failed with some solution given by some users:
BigValue / (Math.Pow((1 + ret), j));
WHere BigValue is something like 15000.25
ret is -0.99197104212554987
And j will go to around 500-600.
I am not gettting how to use Rational Class for this too
BigRational from the base class library team from Microsoft. It uses big integers to store it as a fraction, but supports all kinds of operators.
When it comes to printing it as a decimal, I think you need to write your own implementation for that. I have one written somewhere for this class, but I'd have to find it.
Here is something that may be useful. I used it a while back with no problem. It is a .Net BigDecimal class, you can download it from codeplex(or just look at the source):
http://bigdecimal.codeplex.com/releases/view/44790
It is written in VB.Net (.Net 4.0), but that shouldn't matter.
An example of its use in C#: http://www.dreamincode.net/forums/blog/217/entry-2522-the-madman-scribblings/
You will have to switch languages to one that has a BigFloat type (e.g. Haskel and Python have native packages) or else find a third party big float library with a C# binding. There was some discussion of such a binding for GNU MP, but it faded away. Maybe you'll write one!
See also this discussion where MS BigRational is discussed. However this is different from BigFloat.
One solution might be to do your problems in log space instead.
your example would become:
exp(log(Number) - log(1-0.9999999) * 400)
Learn how to use logs to work with numbers like these. Yes, you CAN use a big float package, but that is overkill almost always. You can usually get what you need using logs.
I'm working on a n image processing library which extends OpenCV, HALCON, ... . The library must be with .NET Framework 3.5 and since my experiences with .NET are limited I would like to ask some questions regarding the performance.
I have encountered a few specific things which I cannot explain to myself properly and would like you to ask a) why and b) what is the best practise to deal with the cases.
My first question is about Math.pow. I already found some answers here on StackOverflow which explains it quite well (a) but not what to do about this(b). My benchmark Program looks like this
Stopwatch watch = new Stopwatch(); // from the Diagnostics class
watch.Start();
for (int i = 0; i < 1000000; i++)
double result = Math.Pow(4,7) // the function call
watch.Stop()
The result was not very nice (~300ms on my computer) (I have run the test 10 times and calcuated the average value).
My first idea was to check wether this is because it is a static function. So I implemented my own class
class MyMath
{
public static double Pow (double x, double y) //Using some expensive functions to calculate the power
{
return Math.Exp(Math.Log(x) * y);
}
public static double PowLoop (double x, int y) // Using Loop
{
double res = x;
for(int i = 1; i < y; i++)
res *= x;
return res;
}
public static double Pow7 (double x) // Using inline calls
{
return x * x * x * x * x * x * x;
}
}
THe third thing I checked were if I would replace the Math.Pow(4,7) directly through 4*4*4*4*4*4*4.
The results are (the average out of 10 test runs)
300 ms Math.Pow(4,7)
356 ms MyMath.Pow(4,7) //gives wrong rounded results
264 ms MyMath.PowLoop(4,7)
92 ms MyMath.Pow7(4)
16 ms 4*4*4*4*4*4*4
Now my situation now is basically like this: Don't use Math for Pow. My only problem is just that... do I really have to implement my own Math-class now? It seems somehow ineffective to implement an own class just for the power function. (Btw. PowLoop and Pow7 are even faster in the Release build by ~25% while Math.Pow is not).
So my final questions are
a) am I wrong if I wouldn't use Math.Pow at all (but for fractions maybe) (which makes me somehow sad).
b) if you have code to optimize, are you really writing all such mathematical operations directly?
c) is there maybe already a faster (open-source^^) library for mathematical operations
d) the source of my question is basically: I have assumed that the .NET Framework itself already provides very optimized code / compile results for such basic operations - be it the Math-Class or handling arrays and I was a little surprised how much benefit I would gain by writing my own code. Are there some other, general "fields" or something else to look out in C# where I cannot trust C# directly.
Two things to bear in mind:
You probably don't need to optimise this bit of code. You've just done a million calls to the function in less than a second. Is this really going to cause big problems in your program?
Math.Pow is probably fairly optimal anyway. At a guess, it will be calling a proper numerics library written in a lower level language, which means you shouldn't expect orders of magnitude increases.
Numerical programming is harder than you think. Even the algorithms that you think you know how to calculate, aren't calculated that way. For example, when you calculate the mean, you shouldn't just add up the numbers and divide by how many numbers you have. (Modern numerics libraries use a two pass routine to correct for floating point errors.)
That said, if you decide that you definitely do need to optimise, then consider using integers rather than floating point values, or outsourcing this to another numerics library.
Firstly, integer operations are much faster than floating point. If you don't need floating point values, don't use the floating point data type. This generally true for any programming language.
Secondly, as you have stated yourself, Math.Pow can handle reals. It makes use of a much more intricate algorithm than a simple loop. No wonder it is slower than simply looping. If you get rid of the loop and just do n multiplications, you are also cutting off the overhead of setting up the loop - thus making it faster. But if you don't use a loop, you have to know
the value of the exponent beforehand - it can't be supplied at runtime.
I am not really sure why Math.Exp and Math.Log is faster. But if you use Math.Log, you can't find the power of negative values.
Basically int are faster and avoiding loops avoid extra overhead. But you are trading off some flexibility when you go for those. But it is generally a good idea to avoid reals when all you need are integers, but in this case coding up a custom function when one already exists seems a little too much.
The question you have to ask yourself is whether this is worth it. Is Math.Pow actually slowing your program down? And in any case, the Math.Pow already bundled with your language is often the fastest or very close to that. If you really wanted to make an alternate implementation that is really general purpose (i.e. not limited to only integers, positive values, etc.), you will probably end up using the same algorithm used in the default implementation anyway.
When you are talking about making a million iterations of a line of code then obviously every little detail will make a difference.
Math.Pow() is a function call which will be substantially slower than your manual 4*4...*4 example.
Don't write your own class as its doubtful you'll be able to write anything more optimised than the standard Math class.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 9 years ago.
I've mainly been doing C# development for the past few years but recently started to do a bit of Python (not Iron Python). But I'm not sure if I've made the mental leap to Python...I kind of feel I'm trying to do things as I would in C#.
Any advice on how I can fully take advantage of Python?
Or any tips\tricks, things to learn more about, things to watch out for?
First, check tgray's and Lundström's advice.
Then, some things you may want to know:
Python is dynamically typed, so unlike C#, you will not
check type, but behavior. You may want to google about duck
typing. It implies you do not have to deal with boxing and
unboxing.
Python is fully object oriented, but the syntax does not
enforce this paradigm. You can write Python without using
the word "class".
The GUI library featured with Python can't compare with
C#'s. Check PyQt, GTK or wxPython libraries.
Python has a lot of concepts you may not be familiar with:
list comprehensions, generators ("yield" does exist in C#,
but it is not used much), decorators, metaclasses, etc. Don't
be afraid; you can program in Python without them. They
are just smart tools, not mandatory.
Like in C#, the Python standard library is huge. Always
look at it when you encounter any problem. It is most
likely that someone solved it already.
Python use LATE binding and variable labels. It's far too
early for somebody starting with the language to worry
about it, but remember that one day you will encounter a
behavior with variables that SEEMS illogical, and you'll
have to check that. For the moment:
Just remember to never do the following:
def myfunc(my_list=[]) :
# bla
Instead:
def myfunc(my_list=()) :
my_list = list(my_list)
And you'll be good. There is a good reason for that, but
that's not the point :-)
Python is cross platform, enjoy writing on Mac, and
run on Linux, if you wish.
Python is not provided with a complex IDE (you got IDLE :-)).
If you are a Visual Studio addict, check Glade. This is
not as advanced as Visual Studio, but it's still a good RAD.
If you want to develop some web application in Python,
remember that Python is not .NET. You must add a web
framework to it if you want to compare. I like Django.
Python does not need a huge IDE to work with. SciTE,
Notepad++, IDLE, Kate, gedit...
Lightweight editors are really sufficient.
Python enforces indentation using spaces and line break,
you can't change that. You should avoid using tabs for
indenting and choose spaces instead. The equivalent of
empty bracelets {} is the keyword "pass".
Python does not enforce private variables. You can define a
private var using "__" (two underscores) at the beginning of
the variable name, but it's still bypassable in some tricky
ways. Python usually assume programmers are grown adults
that know what they do and communicate.
Python uses iteration. A lot. A lot of a lot. And so the
itertools module is you best friend.
Python has no built in delegates. The delegate module is
not what you think. For event-driven programming, use a
GUI lib (or code the pattern yourself, it's not that
difficult).
Python has an interpreter: you can test almost anything,
live. It should always be running next to your text
editor. Python basic interpreter is not much, try IPython
for something tasty.
Python is autodocumented: use docstrings in your own code
and consult other's using "help()" in the python interpreter
Module basics:
sys: manipulate system features
os: set credential, manipulate file paths, rename, recursive file walk, etc
shutil: batch file processing (such as recursive delete)
re: regexp
urllib and urllib2: HTTP¨scripting like downloading, post / get resquests, etc.
datetime: manipulate date, time AND DURATION
thread: you guess it
zlib: compression
pickle: serialization
xml: parsing / Writing XML with SAX or DOM
There are hundreds of modules. Enjoy.
Some typical ways to do things in Python:
Loops:
Python coders use massively the equivalent of the foreach C#
loop, and prefer it to any others:
Basic iterations:
for item in collection:
print str(item)
"collection" can be a string, a list, a tuple... Any
iterable: any object defining the .next() method. There are
a lot of iterables in Python. E.g: a typical Python idiom
to read files:
for line in open("/path/to/file") :
print line
A shortcut to the for loop is called "list comprehension".
It's a way to create an new iterable in one line:
Creating a filtered list with list comprehension:
my_list = [item for item in collection if condition]
Creating a new list with a list comprehension:
my_list = [int(item) * 3 for item in collection]
Creating a new generator with a list comprehension:
my_list = (int(item) * 3 for item in collection)
Same as above, but the values will be generated on the fly
at the first iteration then lost. More information about it here.
Ordinary for loop
If you want to express a usual for loop, you can use the
xrange() function. for (int i = 0; i < 5; i++) becomes:
for i in xrange(0,5) :
do while equivalent
There is no "Do While" in Python. I never missed it, but if
you have to use this logic, do the following:
while True : # Yes, this is an infinite loop. Crazy, hu?
# Do your stuff
if condition :
break
Unpacking
Swapping variables:
a, b = b, a
Multiple assignations:
The above is just a result of what we call "unpacking" (here
applied to a tuple). A simple way to explain it is that you
can assign each value of any sequence directly to an equal
number a variables, in one row:
animal1, animal2, animal3, animal4 = ["cow", "dog", "bird", "fish"]
This has a lot of implications. While iterating on a
multidimensional array, you normally get each sub sequence
one by one then use it, for example:
agenda = [("steve", "jobs"), ("linus", "torvald"), ("bill", "gates"),("jon", "skeet")]
for person in agenda:
print person[0], person[1]
But with unpacking, you can assign the values directly to
variables as well:
agenda = [("steve", "jobs"), ("linus", "torvald"), ("bill", "gates"),("jon", "skeet")]
for name, lastname in agenda:
print name, lastname
And that's why if you want to get an index while iterating,
Python coders use the following idioms (enumerate() is a
standard function):
for index, value in enumerate(sequence) :
print index, value
Unpacking in functions calls
This is advanced use, and you can skip it if it bothers you.
You can unpack values using the sign "*" to use a sequence
directly in a function call. E.g:
>>> foo(var1, var1, var3) :
print var1, var2
print var3
>>> seq = (3.14, 42, "yeah")
>>> foo(*seq)
3.14 42
yeah
There is even more than that. You can unpack a dictionary as
named variables, and write function prototypes with *,
** to accept an arbitrary number of arguments. But it not
used enough to deserve to make this post even longer :-).
String formatting:
print "This is a %s on %s about %s" % ("post", "stackoverflow", "python")
print "This is a %(subject)s on %(place)s about %(about)s" % {"subject" : "post", "place" : "stackoverflow", "about" : "python"}
Slicing an iterable:
You can get any part of an iterable using a very concise syntax:
print "blebla"[2:4] # Print "eb"
last = string[:-1] # Getting last element
even = (0,1,2,3,4,5,6,7,8,9)[::2] # Getting evens only (third argument is a step)
reversed = string[::-1] # Reversing a string
Logical checks:
You can check the way you do in C#, but there are "Pythonic"
ways (shorter, clearer :-)):
if 1 in (1, 2, 3, 4) : # Check en element is in a sequence
if var : # check is var is true. Var == false if it's False, 0, (), [], {} or None
if not var : # Contrary of above
if thing is var: # Check if "thing" and "var" label the same content.
if thing is None : # We use that one because None means nothing in Python (almost null)
Combo (print on one line all the words containing an "o" in uppercase ):
sentence = "It's a good day to write some code"
print " ".join([word.upper() for word in sentence.split() if "o" in word])
Output: "GOOD TO SOME CODE"
Easier to ask for forgiveness than permission
Python coders usually don't check if something is possible. They are a bit like Chuck Norris. They do it. Then catch the exception. Typically, you don't check if a file exists, you try to open it, and roll back if it fails:
try :
f = open(file)
except IOerror :
print "no file here !"
Of course Chuck Norris never uses excepts since he never fails.
The else clause
"Else" is a world of many uses in Python. You will find
"else" after "if", but after "except" and "for" as well.
for stuff in bunch :
# Do things
else :
# This always happens unless you hit "break" in the loop
This works for "while" loop too, even if we do not use this
loop as much.
try :
# A crazy stuff
except ToCrazyError :
# This happens if the crazy stuff raises a ToCrazyError Exception
else :
# This will happen if there is no error so you can put only one line after the "try" clause
finally :
# The same as in C#
If you are curious, here is a bunch of advanced quick and
dirty (but nice) Python snippets.
Refrain from using classes. Use dictionaries, sets, list and tuples.
Setters and getters are forbidden.
Don't have exception handlers unless you really need to - let it crash in style.
Pylint can be your friend for more pythonish coding style.
When you're ready - check out list comprehensions, generators and lambda functions.
If you are not new to programming, I would recommend the book "Dive into Python" by Mark Pilgrim. It explains Python in a way that makes it easy to understand how Python techniques and idioms can be applied to build practical applications.
Start by reading The Zen of Python
You can read it at the link above, or just type import this at the Python prompt. =)
Take advantage of Python features not offered* by C#
Such as duck-typing, metaclasses, list comprehension, etc.*
Write simple programs just to test these features. You'll get used (if not addicted) to them in no time.
Look at the Python Standard Library
So you don't reinvent the wheel. Don't try to read the whole thing, even a quick look at the TOC could save you a lot of time.
* I know C# already has some of these features, but from what I can see they're either pretty new or not commonly used by C# developers. Please correct me if I'm wrong.
In case you haven't heard about it yet, Dive Into Python is a great place to start for anyone learning Python. It also has a bunch of Tips & Tricks.
If you are someone who is better learning a new language by taking small incremental steps then I would recommend using IronPython. Otherwise use regular CPython and don't do any more C# coding until you feel like you have a grasp of Python.
I would suggest getting a good editor so that you don't get bitten by whitespace. For simplicity, I just use ActivePython's packages Link, which include an editor and all of the win32api libraries. They are pretty fun to get into if you have been using C#. The win32api in Python can be a little bit simpler. You don't need to do the whole DDLImport thing. Download ActivePython (which comes with CPython), open it up, and start entering some stuff at the console. You will pick it up fairly easy after using C#. For some more interesting Python tidbits, try ActiveState code, which has all sorts of recipes, which can allow you to very simply see different things that you can do with Python.
I'm pretty much in your shoes too, still using C# for most of my work, but using Python more and more for other projects.
#e-satis probably knows Python inside-out and all his advice is top-notch. From my point of view what made the biggest difference to me was the following:
Get back into functional. not necessarily spaghetti code, but learning that not everything has to be in an object, nor should it be.
The interpreter. It's like the immediate window except 10^10 better. Because of how Python works you don't need all the baggage and crap C# makes you put in before you can run things; you can just whack in a few lines and see how things work.
I've normally got an IDLE instance up where I just throw around snippets as I'm working out how the various bits in the language works while I'm editing my files... e.g. busy working out how to do a map call on a list, but I'm not 100% on the lambda I should use... whack in a few lines into IDLE, see how it works and what it does.
And finally, loving into the verbosity of Python, and I don't mean that in the long winded meaning of verbosity, but as e-satis pointed out, using verbs like "in", "is", "for", etc.
If you did a lot of reflection work in C# you'll feel like crying when you see how simple the same stuff is in Python.
Good luck with it.
If you have programming experience and don't feel like spending money I'd recommend How to Think Like a Computer Scientist in Python.
And then something you can benefit from:
IPython shell: Auto completion in the shell. It does batch operations, adds a ton of features, logging and such. >>> Play with the shell - always!
easy_install / pip: So nice and an easy way to install a 3rd party Python application.