OperationGuys my noob brain is having issues following this simple order of operation, would anybody be kind enough to enlight me on how is the computer handling this order of operation?.
Running this on Unity it throws the result printed on the book, but as simple as the operation maybe I cannot see how the computer is doing the operation.
you have to first learn the mathematics BODMAS rule and how the precedence works. Refer to this.
Now coming to your question.
it will become 5 + 1/2.
1/2 will become 0 because both the numerator and denominator are int.
and 5 + 0 is 5
Related
I'm trying to understand how TrainingParameterScheduleDouble works in the CNTK C# API. Unfortunately, there is no documentation and the previous SO thread here appears to be incorrect/incomplete, so I've tried to reverse engineer the behavior myself. Can anyone confirm my conclusions and answer the lingering questions that I have?
Overload #1
TrainingParameterScheduleDouble(value, minibatchSize)
This sets the learning rate to value per minibatchSize number of samples, regardless of the actual minibatch size passed to GetNextMinibatch. Thus, using minibatchSize: 1 is an easy way to specify a per-sample learning rate.
It seems to me that calling the second parameter minibatchSize is very misleading in this context, since it's totally unrelated to the actual size of each minibatch. I think a better name would have been something like perNumSamples, or am I missing something?
Overload #2
TrainingParameterScheduleDouble(value)
This is the same as setting minibatchSize: 0 above, and has the effect of using the "natural" minibatchSize that's passed to GetNextMinibatch as the number of samples.
So if we have GetNextMinibatch(64) then new TrainingParameterScheduleDouble(0.001) will result in a 64x slower learning rate than new TrainingParameterScheduleDouble(0.001, 1).
Overload #3
TrainingParameterScheduleDouble(schedule)
This changes the learning rate over time, using the "natural" minibatch size. So a schedule of (30, 0.321), (1, 0.123) will use a per-actual-minibatch learning rate of 0.321 for the first 30 minibatches and a rate of 0.123 thereafter.
Overload #4
TrainingParameterScheduleDouble(schedule, epochSize)
epochSize causes IsSweepBased() to return False instead of True, but otherwise has no apparent effect on the learning rate or anything else. This is surprising. Can anyone explain the purpose of epochSize in this context?
Overload #5
TrainingParameterScheduleDouble(schedule, epochSize, minibatchSize)
This is the only way to change the learning rate over time without using the natural minibatch size. So a schedule of (30, 0.321), (1, 0.123) with minibatchSize: 1 will use a per-sample learning rate of 0.321 for the first 30 samples (regardless of the actual minibatch size) and a rate of 0.123 thereafter. As before, the epoch size has no apparent effect.
Assuming this is correct, it's not clear to me what happens if the learning rate changes in the middle of a minibatch. Can anyone clarify?
0 1 1 1 1
1 0 1 1 2
1 1 0 1 2
1 1 1 0 2
1 2 2 2 0
The table above gives the value of a relation for each pair of strings. As you can see the matrix is symmetric(relation is commutative).Now i am supposed to find all possible groups (if subsets of a group already found are skipped it is fine.A group can have any size) such that in a given group for any possible string pair in the group the relation value is less than a particular threshold(say 2).
I've tried to do it in c# but it did not cover all possibilities and resulted in too many loops.The reason i didn't turn to any clustering algorithms is because this relation is not a distance metric.
An algorithm or a syntactic element in c# which might make the process easy or a clue about how to approach the problem...Any help will be appreciated.
That is quite outside my league, but I did a quick search around and found these, might give you an enlightment:
"Cluster analysis" with MySQL
This one is an article, in Portuguese though, about using the Tocher method on cluster analysys:
http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0100-204X2007001000008
Sorry for not being able to help more. I'll study it, see if I can come up with something more useful
Not all clustering algorithms require a metric.
For example, most HAC can work with similarities as well as with distances (except for ward apparently).
Your requirement sounds exactly like complete linkage clustering. Even if you don't use clustering, you'll still get the same result as with HAC.
Bad news is that HAC usually is O(n^3). But I believe if you fix the threshold beforehand, it is only O(n^2).
I'm implementing Ng's example of OCR neural network in C#.
I think I've got all formulas correctly implemented [vectorized version] and my app is training the network.
Any advice on how can I see my network improving in recognition - without testing examples manually by drawing them after the training is done? I want to see where my training is going while it's being trained.
I've test my trained weights on a drawn digits, output on all neurons is quite similar(approx. 0.077,or something like that ...on all neurons) ,and the largest value is on the wrong neuron. So the result doesn't match the drawn image.
This is the only test I'm doing so far: Cost Function changes with epochs
So, this is what happens with Cost function (some call it objective function? ) in 50 epochs.
my Lambda value is set to 3.0 , learning rate is 0.01, 5000 examples, I do batch after each epoch i.e. after those 5000 examples. Activation function: sigmoid.
input: 400
hidden: 25
output:10
I don't know what proper values are for lambda and learning rate so that my network can learn without overfitting or underfitting.
Any suggestions how to find out my network is learning well?
Also, what value should J cost function have after all this training?
Should it approach zero?
Should I have more epochs?
Is it bad that my examples are all ordered by digits?
Any help is appreciated.
Q: Any suggestions how to find out my network is learning well?
A: Split the data into three groups training, cross validation and test.Validate your result with test data. This is actually address in the course later.
Q: Also, what value should J cost function have after all this training? Should it approach zero?
A: I recall in the homework Ng mentioned what is the expected value. The regularized cost should not be zero since it includes a sum of all the weights.
Q: Should I have more epochs?
A: If you run your program long enough ( less than 20 minutes? ) you will see the cost is not getting smaller, I assume it reached the local/global optimum so more epochs would not be necessary.
Q: Is it bad that my examples are all ordered by digits?
A: The algorithm modify the weights for every example so different order of data does affect each step in a batch. However the final result should not have much difference.
I have 7 million+ records that i need to apply multiple formula to it.
This is how i am doing it:
I am converting my formula to shunting Yard Algorithm then using "find and replace" the variable then using Reverse Polish notation to convert to a double ...
for instance: y= ( 3 + X + RMT + T ) + 5
replace X by 6
replace RMT by 5.9
replace T by whatever...
it is taking too long.
Can i speed it up ?
thank's
Note: the formula are dynamic and can change...
it is taking too long
Use a profiler, fix your application.
I have a problem seeing a simple formula like that on a properly multi core system with parallelizaton taking more than a tenth off a second to do 7 million updates when a properly programmed interpreter is attached there.
Perfomrance for something like that is an implementation detail, and you lack ANY information.
NOrmally the usual applies - it is too slow, pull up a profiler and FIND OUT WHY.
I have the following simple equation in my C# program to convert a number to a resulting value:
sectorSize = 1 << sectorShift;
Is there some sort of inverse operation that will allow me to go the other way as well?
sectorShift = ???
I know that you can implement a loop, but that's a little bit of an overkill. I've never had to do this before, so I have no idea and I can't find anything online about it. The equation I need only needs to produce valid results when sectorSize is a power of two; the rest of the domain can go to hell for all I care.
Here are five ways to do that in C. Translating them to correct C# is left as an exercise. Be extremely careful.
http://graphics.stanford.edu/~seander/bithacks.html#IntegerLogObvious
Frankly, I would personally always go with the loop. It is not clear to me why you believe that simple and obviously correct code is "overkill".
Logarithms. But since you don't want to do that, use a loop and/or lookup table.