Create intervals of data with LINQ - c#

I have an issue in C# that I can't figure out how to solve:
I have a set of data that consist of a TimeStamp and a value.
This is a sample dataset:
<Timestamp>2014-01-06T17:40:08.000Z</TimeStamp>
<Value>200</Value>
<Timestamp>2014-01-06T17:40:09.000Z</TimeStamp>
<Value>234</Value>
<Timestamp>2014-01-06T17:40:11.000Z</TimeStamp>
<Value>214</Value>
<Timestamp>2014-01-06T17:40:12.000Z</TimeStamp>
<Value>264</Value>
<Timestamp>2014-01-06T17:40:13.000Z</TimeStamp>
<Value>300</Value>
<Timestamp>2014-01-06T17:40:15.000Z</TimeStamp>
<Value>276</Value>
What I need to do is to somehow get the average of the values by every 30 seconds. Notice that the recordings of data is not necessarily every second. This is what makes it hard for me to imagine how to do this without having an insecurity in the results.
Is this achievable through a LINQ statement or do you have other suggestions?

Assuming you can figure out how to parse the XML, and you have a collection of objects with Time and Value. For example, I'll use this collection:
var now = DateTime.Now;
var random = new Random();
var times = Enumerable.Range(1, 10000).Select(i => new
{
Time = now.AddHours(random .NextDouble()),
Value = i
});
Using a helper method DateTime RoundUp(DateTime dt, TimeSpan d), and GroupBy:
var interval = TimeSpan.FromSeconds(30);
var intervalAverageValues =
times.GroupBy(t => RoundUp(t.Time, interval))
.Select(g => new
{
AverageValue = g.Average(t => t.Value),
IntervalEndTime = g.Key,
Count = g.Count()
});

Consider this pseudo answer :)
Take your list of values and their timestamps and produce a list of Tuple<int, int>{} this is assuming your values are ints. Where first int is index and second int is the value;
As you are filling the tuples list you need to take the total seconds component of your timestamp and divide it by 30. The result should be rounded to closest int.
At this point you have a list of tuples that has values paired with their corresponding '30 second' group index;
Use a simple group linq to produce your averages. Basically you loop over your groups and sum the values. Then divide by group's item count;
This is brute force approach somewhat. I'm sure there are smarter ways to do this.

Related

Is there a way to find the closest number in an Array to another inputed number?

So I have a Visualstudio Forms where I have a NumericUpDown function that will allow users to input a 5 digit number such as 09456. And I need to be able to compare that number to an already existing array of similar 5 digit numbers, so essentially I need to get the inputted number and find the closest number to that.
var numbers = new List<float> {89456f, 23467f, 86453f, };
// the list is way longer but you get the idea
var target = numericUpDown.3 ;
var closest = numbers.Select(n => new { n, (n - target) })
.OrderBy(p => p.distance)
.First().n;
But the first problem I encounter is that I cannot use a "-" operation on a float. Is there any way I can avoid that error and be able to still find the closest input?
Anonymous type members need names, and you need to use the absolute value of the difference. eg
var numbers = new List<float> { 89456f, 23467f, 86453f, };
var target = 3;
var closest = numbers.Select(n => new { n, distance = Math.Abs(n - target) })
.OrderBy(p => p.distance)
.First().n;
Well, apart from some issues in your sample(like no distance property on float) it should work:
int target = 55555;
float closest = numbers.OrderBy(f => Math.Abs(f - target)).First();
Demo: https://dotnetfiddle.net/gqS50L
The answers that use OrderBy are correct, but have less than optimal performance. OrderBy is an O(N log N) operation, but why sort the whole collection when you only need the top element? By contrast, MinBy will give you the result in O(N) time:
var closest = numbers.MinBy(n => Math.Abs(n - target));
Apart from the compilation errors, using LINQ for this is very slow and time consuming. The entire list has to be scanned once to find the distance, then it needs to be sorted, which scans it all over again and caches the results before returning them in order.
Before .NET 6
A faster way would be to iterate only once, calculating the distance of the current item from the target, and keep track of which number is closest. That's how eg Min and Max work.
public static float? Closest(this IEnumerable<float> list, float target)
{
float? closest=null;
float bestDist=float.MaxValue;
foreach(var n in list)
{
var dist=Math.Abs(n-target);
if (dist<bestDist)
{
bestDist=dist;
closest=n;
}
}
return closest;
}
This will return the closest number in a single pass.
var numbers = new List<float> { 89456f, 23467f, 86453f, };
var closest=numbers.Closest(20000);
Console.WriteLine($"Closest is {closest}");
------------------
Closest is 23467
Using MoreLINQ and MinBy
The same can be done in a single line using the MinBy extension method from the MoreLINQ library:
var closest=numbers.MinBy(n=>Math.Abs(n-target));
Using MinBy
In .NET 6 and later, Enumerable.MinBy was added to the BCL:
var closest=numbers.MinBy(n=>Math.Abs(n-target));
The code is similar to the explicit loop once you look past the generic key selectors and comparers :
while (e.MoveNext())
{
TSource nextValue = e.Current;
TKey nextKey = keySelector(nextValue);
if (nextKey != null && comparer.Compare(nextKey, key) < 0)
{
key = nextKey;
value = nextValue;
}
}

Show non matched values from two arrays

How do I put into the collection of ListBox those items that doesn't have a pair from each array?
For example:
first array = 100 500
second array = 100 200 300 400 500 600 700 800
Now, how do I show those non matched values (200,400,600,700,800) into ListBox?
You can use LINQ and Except method:
int[] result = secondArray.Except(firstArray).ToArray();
yourListBox.DataSource = result;
Also if you want to include values in firstArray that are not in secondArray go with the following query:
var result = firstArray.Except(secondArray).Union(secondArray.Except(firstArray)).ToArray();
HashSet<int> can easily do set operations like this. Go check out the docs on that class and I'm sure you'll have an answer. I believe you will be interested in the method SymmetricExceptWith
Might not be the most efficient, but you can do
var firstArray = new int[2] {100,500};
var secondArray = new int[8] {100,200,300,400,500,600,700,800};
var x = secondArray.Except(firstArray);
foreach(var item in x)
Console.WriteLine(item);
This is more or less a duplicate of Get the symmetric difference from generic lists.
var differences = listA.Except(listB).Union(listB.Except(listA));

Max value from list that is smaller than X

How to retrieve int value from List that is biggest number on it but still smaller than X value to which i am comparing.
Value = 10
Example one: List = {1,4,6,8}; => number 8 biggest on list and smaller than 10
Example two: List = {1,15,17,20}; => number 1 biggest on list and smaller than 10
I was trying using Linq but no success so far.
You can just restrict the values you use to get the "Max" by using a Where clause:
return myList.Where(x => x < 10).Max();
You can use Where to filter your values which less than 10 and use Max to get maximum values in them like;
var list = new List<int>{1, 15, 17, 20};
list.Where(s => s < 10).Max().Dump();
you can filter items less than x and then find the max by using Max() as below
list.where(m => m <= x).Max(p => p);

Get closest/next match in .NET Hashtable (or other structure)

I have a scenario at work where we have several different tables of data in a format similar to the following:
Table Name: HingeArms
Hght Part #1 Part #2
33 S-HG-088-00 S-HG-089-00
41 S-HG-084-00 S-HG-085-00
49 S-HG-033-00 S-HG-036-00
57 S-HG-034-00 S-HG-037-00
Where the first column (and possibly more) contains numeric data sorted ascending and represents a range to determine the proper record of data to get (e.g. height <= 33 then Part 1 = S-HG-088-00, height <= 41 then Part 1 = S-HG-084-00, etc.)
I need to lookup and select the nearest match given a specified value. For example, given a height = 34.25, I need to get second record in the set above:
41 S-HG-084-00 S-HG-085-00
These tables are currently stored in a VB.NET Hashtable "cache" of data loaded from a CSV file, where the key for the Hashtable is a composite of the table name and one or more columns from the table that represent the "key" for the record. For example, for the above table, the Hashtable Add for the first record would be:
ht.Add("HingeArms,33","S-HG-088-00,S-HG-089-00")
This seems less than optimal and I have some flexibility to change the structure if necessary (the cache contains data from other tables where direct lookup is possible... these "range" tables just got dumped in because it was "easy"). I was looking for a "Next" method on a Hashtable/Dictionary to give me the closest matching record in the range, but that's obviously not available on the stock classes in VB.NET.
Any ideas on a way to do what I'm looking for with a Hashtable or in a different structure? It needs to be performant as the lookup will get called often in different sections of code. Any thoughts would be greatly appreciated. Thanks.
A hashtable is not a good data structure for this, because items are scattered around the internal array according to their hash code, not their values.
Use a sorted array or List<T> and perform a binary search, e.g.
Setup:
var values = new List<HingeArm>
{
new HingeArm(33, "S-HG-088-00", "S-HG-089-00"),
new HingeArm(41, "S-HG-084-00", "S-HG-085-00"),
new HingeArm(49, "S-HG-033-00", "S-HG-036-00"),
new HingeArm(57, "S-HG-034-00", "S-HG-037-00"),
};
values.Sort((x, y) => x.Height.CompareTo(y.Height));
var keys = values.Select(x => x.Height).ToList();
Lookup:
var index = keys.BinarySearch(34.25);
if (index < 0)
{
index = ~index;
}
var result = values[index];
// result == { Height = 41, Part1 = "S-HG-084-00", Part2 = "S-HG-085-00" }
You can use a sorted .NET array in combination with Array.BinarySearch().
If you get a non negative value this is the index of exact match.
Otherwise, if result is negative use formula
int index = ~Array.BinarySearch(sortedArray, value) - 1
to get index of previous "nearest" match.
The meaning of nearest is defined by a comparer you use. It must be the same you used when sorting the array. See:
http://gmamaladze.wordpress.com/2011/07/22/back-to-the-roots-net-binary-search-and-the-meaning-of-the-negative-number-of-the-array-binarysearch-return-value/
How about LINQ-to-Objects (This is by no means meant to be a performant solution, btw.)
var ht = new Dictionary<string, string>();
ht.Add("HingeArms,33", "S-HG-088-00,S-HG-089-00");
decimal wantedHeight = 34.25m;
var foundIt =
ht.Select(x => new { Height = decimal.Parse(x.Key.Split(',')[1]), x.Key, x.Value }).Where(
x => x.Height < wantedHeight).OrderBy(x => x.Height).SingleOrDefault();
if (foundIt != null)
{
// Do Something with your item in foundIt
}

Select items from List of structs

I've got List of sctructs. In struct there is field x. I would like to select those of structs, which are rather close to each other by parameter x. In other words, I'd like to clusterise them by x.
I guess, there should be one-line solution.
Thanks in advance.
If I understood correctly what you want, then you might need to sort your list by the structure's field X.
Look at the GroupBy extension method:
var items = mylist.GroupBy(c => c.X);
This article gives a lot of examples using group by.
If you're doing graph-style clustering, the easiest way to do it is by building up a list of clusters which is initially empty. Then loop over the input and, for each value, find all of the clusters which have at least one element which is close to the current value. All those clusters should then be merged together with the value. If there aren't any, then the value goes into a cluster all by itself.
Here is some sample code for how to do it with a simple list of integers.
IEnumerable<int> input;
int threshold;
List<List<int>> clusters = new List<List<int>>();
foreach(var current in input)
{
// Search the current list of clusters for ones which contain at least one
// entry such that the difference between it and x is less than the threshold
var matchingClusters =
clusters.Where(
cluster => cluster.Any(
val => Math.Abs(current - val) <= threshold)
).ToList();
// Merge all the clusters that were found, plus x, into a new cluster.
// Replace all the existing clusters with this new one.
IEnumerable<int> newCluster = new List<int>(new[] { current });
foreach (var match in matchingClusters)
{
clusters.Remove(match);
newCluster = newCluster.Concat(match);
}
clusters.Add(newCluster.ToList());
}

Categories

Resources