I implemented a simplex noise algorithm (by KdotJPG: OpenSimplex2S) which works fine, but I'd like to add a "function" which can increase/decrease the contrast of the noise. The noise method returns a value between -1 and 1 but the overall result is quite homogeneous. It is not bad at all, but I need to get a different outcome now.
So basically I should "pull" the value of the noise toward the range edges.. this will result more contrasting noise (more distance between the smaller and bigger values). Of course this change must be consistent and proportionally scaled between -1 and 1 (or 0-1) to get natural result.
Actually this is pure mathematical issue, but I'm not good in math at all! I'd like to make it more understandable to give this picture of two graphs:
So, on these graph the Y axis is the noise value (-1 is bottom and +1 it the top) and X axis is the time passed. The left graph shows the original result of the noise generator, and the right is the stretched version what I need to get. As you can see on the right graph everything the same but their values stretched/pulled toward the edge (toward the min, max limit) but still in range.
Is there any math formula or c# built in function to stretch the return value of the noise proportionally respect to the min, max values (-1/1 or 0/1)? If you need the code of the noise you can see it here OpenSimplex2S too, but this is irrelevant in my case, as I just wish to modify its return value. Thanks!
Related
I am developing a little application in Visual Studio 2010 in C# to draw a spectrogram (frequency "heat map").
I have already done the basic things:
Cut a rectangular windowed array out of the input signal array
Feed that array into FFT, which returns complex values
Store magnitude values in an array (spectrum for that window)
Step the window, and store the new values in other arrays, resulting in a jagged array that holds every step of windowing and their spectra
Draw these into a Graphics object, in color that uses the global min/max values of the heat map as relative cold and hot
The LEFT side of the screenshot shows my application, and on the RIGHT there is a spectrogram for the same input (512 samples long) and same rectangular window with size 32 from a program called "PAST - time series analysis" (https://folk.uio.no/ohammer/past/index.html). My 512 long sample array only consists of integer elements ranging from around 100 to 1400.
(Note: the light-blue bar on the very right of the PAST spectrogram is only because I accidentally left an unnecessary '0' element at the end of thats input array. Otherwise they are the same.)
Link to screenshot: https://drive.google.com/open?id=1UbJ4GyqmS6zaHoYZCLN9c0JhWONlrbe3
But I have encountered a few problems here:
The spectrogram seems very undetailed, related to another one that I made in "PAST time series analysis" for reference, and that one looks extremely detailed. Why is that?
I know that for an e.g. 32 long time window, the FFT returns 32 elements, the 0. elem is not needed here, the next 32/2 elements have the magnitude values I need. But this means that the frequency "resolution" on the output for a 32 long window is 16. That is exactly what my program uses. But "PAST" program shows a lot more detail. If you look at the narrow lines in the blue background, you can see that they show a nice pattern in the frequency axis, but in my spectrogram that information remains unseen. Why?
In the beginning (windowSize/2) wide window step-band and the ending (windowSize/2) step-band, there are less values for FFT input, thus there is less output, or just less precision. But in the "PAST" program those parts also seem relatively detailed, not just stretched bars like in mine. How can I improve that?
The 0. element of the FFT return array (the so called "DC" element) is a huge number, which is a lot bigger than the sample average, or even its sum. Why is that?
Why are my values (e.g. the maximum that you see near the color bar) so huge? That is just a magnitude value from the FFT output. Why are there different values in the PAST program? What correction should I use on the FFT output to get those values?
Please share your ideas, if you know more about this topic. I am very new to this. I only read first about Fourier transform a little more than a week ago.
Thanks in advance!
To get more smoothness in the vertical axis, zero pad your FFT so that there are more (interpolated) frequency bins in the output. For instance, zero pad your 32 data points so that you can use a 256 point or larger FFT.
To get more smoothness in the horizontal axis, overlap your FFT input windows (75% overlap, or more).
For both, use a smooth window function (Hamming or Von Hann, et.al.), and try wider windows, more than 32 (thus even more overlapped).
To get better coloring, try using a color mapping table, with the input being the log() of the (non zero) magnitudes.
You can also use multiple different FFTs per graph XY point, and decide which to color with based on local properties.
Hello LimeAndConconut,
Even though I do not know about PAST, I can provide you with some general information about FFT. Here is an answer for each of your points
1- You are right, a FFT performed on 32 elements returns 32 frequencies (null frequency, positive and negative components). It means that you already have all the information in your data, and PAST cannot get more information with the same 32 sized window. That's why I suspect the data to be interpolated for plotting, but this just visual. Once again PAST cannot create more information than the one you have in your data.
2- Once again I agree with you. On the borders, you have access to less frequency components. You can decide different strategies: not show data at the borders, or extend this data with zero-padding or circular padding
3- The zero element of the FFT should be the sum of your 32 windowed array. You need to check FFT normalization, have a look at the documentation of your FFT function.
4- Once again check the FFT normalization. Since PAST colorbar exhibit negative values, it seems to be plotted in logarithmic scale. This is common usage to use logarithm for plotting data with high dynamics in order to enhance details.
Here's a picture to make it a little easier:
The blue line represents some input values that resemble waves with variable amplitudes and lengths. The y axis represents the values, the x axis represents time. Please note that there is quite some jitter in the wave. However, every wave has a certain minimum and maximum length.
The green line shows how the input values should be transformed.
Please note: The above picture is just a hand drawn example to explain the task. In an ideal case, the position of the rising and falling edges of the rectangular (green) wave are close to the blue waves average value. The height/amplitude of the green wave segments should match the values of the blue wave.
How do you calculate the green line?
Do you know of any C# libraries or algorithms to do that? I guess this could be a rather common task for electrical engineers, so there are most likely some common approaches available. If so, how are hey called?
How would you approach this requirements?
Any advice that helps in getting started is welcome.
Take a base frequency (f) at an amplitude (a).
Then add ODD harmonics with the inverse amplitude ie f * a + f3 * a/3 + f5 * a/5 + f7 * a/7 ...
This will tend towards a square wave as you add harmonics.
BTW Try doing the same with even harminics, and with all the harmonics - Great fun!!!
Good luck
Tony
I have a number of non-coplanar 3D points and I want to calculate the nearest plane to them (They will always form a rough plane but with some small level of variation). This can be done by solving simultaneous linear equations, one for each point, of the form:
"Ax + By + Cz + D = 0"
The problem I'm having at the moment is twofold.
Firstly since the points are 3D floats they can't be relied on to be precise due to rounding errors.
Secondly all of the methods to solving linear equations programatically that I have found thus far involve using NXN matrices which severely limits what I would be able to do given that I have 4 unknowns and any number of linear equations (due to the variation in the number of 3D points).
Does anyone have a decent way to either solve the simultaneous linear equations without these constraints or, alternatively, a better way to calculate the nearest plane to non-coplanar points? (The precision of the plane calculation is not too much of a concern)
Thanks! :)
If your points are all close to the plane, you have a choice between ordinary least squares (where you see Z as a function of two independent variables X and Y and you minimize the sum of squared vertical distances to the plane), or total least squares (all variables independent, minimize the sum of normal distances). The latter requires a 3x3 SVD. (See http://en.wikipedia.org/wiki/Total_least_squares, unfortunately not the easiest presentation.)
If some of the points are outliers, you will need to resort to robust fitting methods. One of them is RANSAC: choose three points are random, build their plane and compute the sum of distances of all points to the plane, as a measure of fitness. Keep the best result after N drawings.
There are numerical methods for linear regression, which calculates the nearest line y=mx+c to a set of points. Your solution will be similar, only it has one more dimension and is thus a "planar regression".
If you don't care the mathematical accuracy of the algorithm and just want to get a rough result, then perhaps you'd randomly 3 points to construct a plane vector, then adjust it incrementally as you go through the rest of the points. Just some thoughts...
Here's a somewhat simplified example of what I am trying to do.
Suppose I have a formula that computes credit points, but the formula has no constraints (for example, the score might be 1 to 5000). And a score is assigned to 100 people.
Now, I want to assign a "normalized" score between 200 and 800 to each person, based on a bell curve. So for example, if one guy has 5000 points, he might get an 800 on the new scale. The people with the middle of my point range will get a score near 500. In other words, 500 is the median?
A similar example might be the old scenario of "grading on the curve", where a the bulk of the students perhaps get a C or C+.
I'm not asking for the code, either a library, an algorithm book or a website to refer to.... I'll probably be writing this in Python (but C# is of some interest as well). There is NO need to graph the bell curve. My data will probably be in a database and I may have even a million people to which to assign this score, so scalability is an issue.
Thanks.
The important property of the bell curve is that it describes normal distribution, which is a simple model for many natural phenomena. I am not sure what kind of "normalization" you intend to do, but it seems to me that current score already complies with normal distribution, you just need to determine its properties (mean and variance) and scale each result accordingly.
References:
https://en.wikipedia.org/wiki/Grading_on_a_curve
https://en.wikipedia.org/wiki/Percentile
(see also: gaussian function)
I think the approach that I would try would be to compute the mean (average) and standard deviation (average distance from the average). I would then choose parameters to fit to my target range. Specifically, I would choose that the mean of the input values map to the value 500, and I would choose that 6 standard deviations consume 99.7% of my target range. Or, a single standard deviation will occupy about 16.6% of my target range.
Since your target range is 600 (from 200 to 800), a single standard deviation would cover 99.7 units. So a person who obtains an input credit score that is one standard deviation above the input mean would get a normalized credit score of 599.7.
So now:
# mean and standard deviation of the input values has been computed.
for score in input_scores:
distance_from_mean = score - mean
distance_from_mean_in_standard_deviations = distance_from_mean / stddev
target = 500 + distance_from_mean_in_standard_deviations * 99.7
if target < 200:
target = 200
if target > 800:
target = 800
This won't necessarily map the median of your input scores to 500. This approach assumes that your input is more-or-less normally distributed and simply translates the mean and stretches the input bell curve to fit in your range. For inputs that are significantly not bell curve shaped, this may distort the input curve rather badly.
A second approach is to simply map your input range to our output range:
for score in input_scores:
value = (score - 1.0) / (5000 - 1)
target = value * (800 - 200) + 200
This will preserve the shape of your input, but in your new range.
A third approach is to have your target range represent percentiles instead of trying to represent a normal distribution. 1% of people would score between 200 and 205; 1% would score between 794 and 800. Here you would rank your input scores and convert the ranks into a value in the range 200..600. This makes full use of your target range and gives it an easy to understand interpretation.
I'm developing a histogram container class and I'm trying to determine where the cut off points should be for the bins. I'd like the cutoff points to be nice looking numbers, in much that same way that graphs are scaled.
To distill my request into a basic question: Is there a basic method by which data axis labels can be determined from a list of numbers.
For example:
Array{1,6,8,5,12,15,22}
It would make sense to have 5 bins.
Bin Start Count
0 1
5 3
10 2
15 0
20 1
The bin start stuff is identical to selecting axis labels on a graph in this instance.
For the purpose of this question I don't really care about bins and the histogram, I'm more interested in the graph scale axis label portion of the question.
I will be using C# 4.0 for my app, so nifty solution using linq are welcome.
I've attempted stuff like this in the distant past using some log base 10 scaling stuff, but I never got it to work in great enough detail for this application. I don't want to do log scaling, I just used base 10 to round to nearest whole numbers. I'd like it to work for large numbers and very small numbers and possibly dates too; although dates can be converted to doubles and parsed that way.
Any resources on the subject would be greatly appreciated.
You could start with something simple:
NUM_BINS is a passed argument or constatn (e.g. NUM_BINS = 10)
x is your array of x-values (e.g. int[] x = new int[50])
int numBins = x.Length < NUM_BINS ? x.Length : NUM_BINS;
At this point you could calc a histogram of xPoints, and if the xPoints are heavily weighted to one side of distribution (maybe just count left of midpoint vs. right of midpoint), then use log/exp divisions over range of x[]. If the histogram is flat, use linear divisions.
double[] xAxis = new double[numBins];
double range = x[x.Length-1] - x[0];
CalcAxisValues(xAxis, range, TYPE); //Type is enum of LOG, EXP, or LINEAR
This function would then equally space points based on the TYPE.