Develop Reference

Convert Byte array into collection of items of different types

I'm recieving UDP messages as byte arrays and depending on details in some of the first byte, I need to convert remaining bytes into a multitude of different possible arrangements of other data types.
These data types are primarily 'uint', 'ushort' and just 'byte' but can be in any order, and there can be any number of them. Each of these items will have a named variable.
I've tried using quite a few different options and am getting close, but feel that the methods I've created are not the best they could be. In places I've wanted to use 'sizeof' but don't want to mark the code as 'unsafe'. I've tried to use the 'params' keyword on the input, but this cannot be in conjunction with 'ref'. I've wanted to pass them with a maximum number of T1, T2, etc. using generics, but realised I can't enumerate these. I'm now passing an array of variables in conjunction with the 'ref' keyword, which means creating the array in memory but only the array gets updated with the changed and not the original variables.
byte[] message = new byte[] { }; //some byte array
ushort item1 = default(ushort);
byte item2 = default(byte);
var argumentArray = new object[] { item1, item2 };
ConvertArray(response, ref argumentArray);
private void ConvertArray(byte[] response, ref object[] items)
{
int index = 0;
for (int i = 0; i < items.Length; i++)
{
var item = items[i];
var itemType = item.GetType();
var itemSize = SizeOf(item.GetType());
if (itemSize == 0)
{
continue;
}
else if (itemSize == 1)
{
items[i] = response[index];
}
else
{
var method = typeof(BitConverter).GetMethod($"To{itemType.Name}");
var returned = method.Invoke(null, new object[] { response, index });
items[i] = Convert.ChangeType(returned, itemType);
}
index = index + itemSize;
}
}
private int SizeOf(Type type)
{
switch (type.Name)
{
case nameof(UInt16):
return 2;
case "Byte":
return 1;
default:
return 0;
}
}
So this is partially working in that 'argumentArray' is updating with the values from the 'ConvertArray' method, but I'm sure there is a neater way to do this using Types.
Ideally I wouldn't need to create the 'argumentArray' and just pass the items (e.g. Item1 and Item2) directly as arguments to the method.

Add range dynamically to list

I have a List<byte> that stores the value of a variable byte by byte. I am trying to build up this variable by respect to its original data type.
Example of the result:
List<byte> varBytes = new List<byte>();
varBytes.Add(0x12);
varBytes.Add(0x34);
varBytes.Add(0x56);
varBytes.Add(0x78);
//After the conversion of UInt32:
varReady = 0x78563412;
Here is a snippet of my class that returns the value of the variable.
public static object GetTypedString(List<byte> varBytes, string varType)
{
object varReady;
switch (varType)
{
case "uint16":
UInt16 varReady = BitConverter.ToUInt16(varBytes.ToArray<byte>(), 0);
break;
case "uint32":
UInt32 varReady = BitConverter.ToUInt32(varBytes.ToArray<byte>(), 0);
break;
//repeat case for each data type
}
return varReady ;
}
The problem comes up if my variable is only 2 bytes long and if I want to show that variable as UInt32. The BitConverter.ToUInt32 will throw this exception:
Destination array is not long enough to copy all the items in the collection.
Because the varBytes list only has 2 bytes but BitConverter.ToUInt32 is trying to read 4 bytes. My solution was to add dummy bytes to the end of the list in this case:
.
.
.
case "uint32":
int difference = sizeof(UInt32) - varSize; //we know the variable size already
if(difference > 0)
{
varToDisp.value.AddRange(new byte[difference]);
}
UInt32 varReady = BitConverter.ToUInt32(varBytes.ToArray<byte>(), 0);
break;
.
.
.
This works but didn't seem a good way to me since it will edit the original List and consumes some time. Is there any easier way to achieve this?

You can create the array (not list) with required Length with a help of Linq Concat; I suggest routine redesign as well.
Code:
// Let's implement a generic method: we want, say, uint not object from given list
public static T GetTypedString<T>(List<byte> varBytes) where T: struct {
if (null == varBytes)
throw new ArgumentNullException(nameof(varBytes));
// sizeof alternative
// char is Ascii by default when marshalling; that's why Marshal.SizeOf returns 1
int size = typeof(T) == typeof(char)
? sizeof(char)
: System.Runtime.InteropServices.Marshal.SizeOf(typeof(T));
// if data is too short we should pad it; either from left or from right:
// {0, ..., 0} + data or data + {0, ..., 0}
// to choose the way, let's have a look at endiness
byte[] data = (size >= varBytes.Count)
? BitConverter.IsLittleEndian
? varBytes.Concat(new byte[size - varBytes.Count]).ToArray()
: new byte[size - varBytes.Count].Concat(varBytes).ToArray()
: varBytes.ToArray();
// A bit of reflection: let's find out suitable Converter method
var mi = typeof(BitConverter).GetMethod($"To{typeof(T).Name}");
if (null == mi)
throw new InvalidOperationException($"Type {typeof(T).Name} can't be converted");
else
return (T)(mi.Invoke(null, new object[] { data, 0 })); // or data.Length - size
}
Then you can use it as follow:
List<byte> varBytes = new List<byte>();
varBytes.Add(0x12);
varBytes.Add(0x34);
varBytes.Add(0x56);
varBytes.Add(0x78);
int result1 = GetTypedString<int>(varBytes);
long result2 = GetTypedString<long>(varBytes);
Console.WriteLine(result1.ToString("x"));
Console.WriteLine(result2.ToString("x"));
// How fast it is (Linq and Reflection?)
var sw = new System.Diagnostics.Stopwatch();
int n = 10000000;
sw.Start();
for (int i = 0; i < n; ++i) {
// The worst case:
// 1. We should expand the array
// 2. The output is the longest one
long result = GetTypedString<long>(varBytes);
//Trick: Do not let the compiler optimize the loop
if (result < 0)
break;
}
sw.Stop();
Console.WriteLine($"Microseconds per operation: {(sw.Elapsed.TotalSeconds/n*1000000)}");
Outcome:
78563412
78563412
Microseconds per operation: 0.84716933
Edit: If you insist on type name (string varType) instead of generic parameter <T> first of all let's extract a model (type name - type correspondense):
private static Dictionary<string, Type> s_Types =
new Dictionary<string, Type>(StringComparer.OrdinalIgnoreCase) {
{ "uint16", typeof(UInt16)},
{ "ushort", typeof(UInt16)}, // <- you can add synonyms if you want
{ "int", typeof(Int32)},
{ "int32", typeof(Int32)},
{ "long", typeof(Int64)},
{ "int64", typeof(Int64)},
//TODO: add all the other names and correspondent types
};
Then you can implement it as
public static object GetTypedString(List<byte> varBytes, string varType) {
if (null == varBytes)
throw new ArgumentNullException(nameof(varBytes));
else if (null == varType)
throw new ArgumentNullException(nameof(varType));
Type type = null;
if (!s_Types.TryGetValue(varType, out type))
throw new ArgumentException(
$"Type name {varType} is not a valid type name.",
nameof(varBytes));
// sizeof alternative
// char is Ascii by default when marshalling; that's why Marshal.SizeOf returns 1
int size = typeof(T) == typeof(char)
? sizeof(char)
: System.Runtime.InteropServices.Marshal.SizeOf(typeof(T));
byte[] data = (size >= varBytes.Count)
? BitConverter.IsLittleEndian
? varBytes.Concat(new byte[size - varBytes.Count]).ToArray()
: new byte[size - varBytes.Count].Concat(varBytes).ToArray()
: varBytes.ToArray();
var mi = typeof(BitConverter).GetMethod($"To{type.Name}");
if (null == mi)
throw new InvalidOperationException(
$"Type {type.Name} (name: {varType}) can't be converted");
else
return mi.Invoke(null, new object[] { data, 0 }); // data.Length - size
}
Demo:
string result1 = (GetTypedString(varBytes, "Int64") as IFormattable).ToString("x8", null);

Rather than using .ToArray you could preallocate your array to the correct size and use .CopyTo.
Example:
var byteArray = new byte[sizeof(UInt32)];
varBytes.CopyTo(byteArray);
UInt32 varReady = BitConverter.ToUInt32(byteArray, 0);

You can check for the length of the array and convert it to smaller types then cast the required one
case "uint32":
{
if (varBytes.Count == 1)
{
varReady = (UInt32)varBytes[0];
}
else if (varBytes.Count >= 2 && varBytes.Count < 4)
{
varReady = (UInt32)BitConverter.ToUInt16(varBytes.ToArray<byte>(), 0);
}
else
{
varReady = BitConverter.ToUInt32(varBytes.ToArray<byte>(), 0);
}
break;
}

Assigning to an array field through reflection in c#

I use reflection to get FieldInfos in a class and assign to them with FieldInfo.SetValue. It works fine when i assign primitive types (i use Convert.ChangeType to convert from object to type) but doesn't work if the field is an array. It works if i use Array.Cast but the type is only known at runtime so i cant cast.
I saw lots of topics on this but none of them worked so far.
I get this exception:
ArgumentException: Object type System.Object[] cannot be converted to target type: System.Single[]
I know why it happens i just can't find a way to convert the data. Any ideas?
EDIT: relevant code:
public static object Deserialize (string path){
string[] lines = File.ReadAllLines(path);
ConstructorInfo ci = type.GetConstructor(Type.EmptyTypes);
object newObj = ci.Invoke(new object[] {});
Type type = Type.GetType(lines[0], true);
FieldInfo[] fields = type.GetFields(BindingFlags.Public);
for (int i = 1; i < lines.Length; i++){
FieldInfo thisField = currentType.GetField(lines[i]);
if (thisField != null) {
if (line != "") {
if (fieldType == typeof(string)) {
thisField.SetValue(currentObject, line);
thisField = null;
}
else if (fieldType.IsPrimitive) {
val = Convert.ChangeType(line, fieldType);
thisField.SetValue(currentObject, val);
thisField = null;
}
else if (fieldType.IsArray){
string[] values = ReadUntil(']');
//ReadUntil just returns the values as string[] as read from text file
object[] newValues = Convert.ChangeType(values, fieldType);
thisField.SetValue(newObj, newValues);
}
}
}
}
}
return newObj;
}

You can use something like this
else if (fieldType.IsArray)
{
string[] values = ReadUntil(']');
var elementType = fieldType.GetElementType();
if (elementType == typeof(string))
thisField.SetValue(newObj, values);
else
{
var actualValues = Array.CreateInstance(elementType, values.Length);
for (int i = 0; i < values.Length; i++)
actualValues.SetValue(Convert.ChangeType(values[i], elementType), i);
thisField.SetValue(newObj, actualValues);
}
}
Type.GetElementType method is used to retrieve the type of the array elements, then Array.CreateInstance method to create an array of the desired type, and finally Array.SetValue method to populate the new array elements with the converted values.

Change multiple, similarly named variable values at the same time

First time on stackOverflow, so this might be a really nooby question, but i was wondering if i could change multiple variable values at the same time without having to write out every single one.
Here is my code at the moment:
public string Label1Text()
{
int index;
for (index = 0; index < 32; index++)
{
if (seatChosen[index])
{
_bookedSeats += "A" + (index + 1) + " ";
Properties.Settings.Default.A1 = true;
}
}
string text = _bookedSeats + ".";
//debug
label1.Text = text;
return text;
}
The line
Properties.Settings.Default.A1 = true;
is what i want to change to something like this (theoretical code)
Properties.Settings.Default.A[index] = true;
or
Properties.Settings.Default.A + index = true;
I hope you can understand what I'm trying to accomplish.

Using reflection: (I'm assuming Properties.Settings.Default is a static class, and A1, A2, etc. are public static properties.)
Type type = typeof(Properties.Settings.Default);
var prop = type.GetProperty(index.ToString("\\A0"));
if (prop != null)
prop.SetValue(null, true);
If Default is an instance, you would need to pass it to SetValue instead of null. Also, C# v6 allows a more concise syntax.
Type type = Properties.Settings.Default.GetType();
type.GetProperty($"A{index}")?.SetValue(Properties.Settings.Default, true);

OutOfMemoryException when updating a large list?

I have a large list and I would like to overwrite one value if required. To do this, I create two subsets of the list which seems to give me an OutOfMemoryException. Here is my code snippet:
if (ownRG != "")
{
List<string> maclist = ownRG.Split(',').ToList();
List<IVFile> temp = powlist.Where(a => maclist.Contains(a.Machine)).ToList();
powlist = powlist.Where(a => !maclist.Contains(a.Machine)).ToList(); // OOME Here
temp.ForEach(a => { a.ReportingGroup = ownRG; });
powlist.AddRange(temp);
}
Essentially I'm splitting the list into the part that needs updating and the part that doesn't, then I perform the update and put the list back together. This works fine for smaller lists, but breaks with an OutOfMemoryException on the third row within the if for a large list. Can I make this more efficient?
NOTE
powlist is the large list (>1m) items. maclist only has between 1 and 10 but even with 1 item this breaks.

Solving your issue
Here is how to rearrange your code using the enumerator code from my answer:
if (!string.IsNullOrEmpty(ownRG))
{
var maclist = new CommaSeparatedStringEnumerable(str);
var temp = powlist.Where(a => maclist.Contains(a.Machine));
foreach (var p in temp)
{
p.ReportingGroup = ownRG;
}
}
You should not use ToList in your code.
You don't need to remove thee contents of temp from powlist (you are re-adding them anyway)
Streaming over a large comma-separated string
You can iterate over the list manually instead of doing what you do now, by looking for , characters and remembering the position of the last found one and the one before. This will definitely make your app work because then it won't need to store the entire set in the memory at once.
Code example:
var str = "aaa,bbb,ccc";
var previousComma = -1;
var currentComma = 0;
for (; (currentComma = str.IndexOf(',', previousComma + 1)) != -1; previousComma = currentComma)
{
var currentItem = str.Substring(previousComma + 1, currentComma - previousComma - 1);
Console.WriteLine(currentItem);
}
var lastItem = str.Substring(previousComma + 1);
Console.WriteLine(lastItem);
Custom iterator
If you want to do it 'properly' in a fancy way, you can even write a custom enumerator:
public class CommaSeparatedStringEnumerator : IEnumerator<string>
{
int previousComma = -1;
int currentComma = -1;
string bigString = null;
bool atEnd = false;
public CommaSeparatedStringEnumerator(string s)
{
if (s == null)
throw new ArgumentNullException("s");
bigString = s;
this.Reset();
}
public string Current { get; private set; }
public void Dispose() { /* No need to do anything here */ }
object IEnumerator.Current { get { return this.Current; } }
public bool MoveNext()
{
if (atEnd)
return false;
atEnd = (currentComma = bigString.IndexOf(',', previousComma + 1)) == -1;
if (!atEnd)
Current = bigString.Substring(previousComma + 1, currentComma - previousComma - 1);
else
Current = bigString.Substring(previousComma + 1);
previousComma = currentComma;
return true;
}
public void Reset()
{
previousComma = -1;
currentComma = -1;
atEnd = false;
this.Current = null;
}
}
public class CommaSeparatedStringEnumerable : IEnumerable<string>
{
string bigString = null;
public CommaSeparatedStringEnumerable(string s)
{
if (s == null)
throw new ArgumentNullException("s");
bigString = s;
}
public IEnumerator<string> GetEnumerator()
{
return new CommaSeparatedStringEnumerator(bigString);
}
IEnumerator IEnumerable.GetEnumerator()
{
return this.GetEnumerator();
}
}
Then you can iterate over it like this:
var str = "aaa,bbb,ccc";
var enumerable = new CommaSeparatedStringEnumerable(str);
foreach (var item in enumerable)
{
Console.WriteLine(item);
}
Other thoughts
Can I make this more efficient?
Yes, you can. I suggest to either work with a more efficient data format (you can take a look around databases or XML, JSON, etc. depending on your needs). If you really want to work with comma-separated items, see my code examples above.

There's no need to create a bunch of sub-lists from powlist and reconstruct it. Simply loop over the powlist and update the ReportingGroup property accordingly.
var maclist = new HashSet<string>( ownRG.Split(',') );
foreach( var item in powlist) {
if( maclist.Contains( item.Machine ) ){
item.ReportingGroup = ownRG;
}
}
Since this changes powlist in place, you won't allocate any extra memory and shouldn't run into an OutOfMemoryException.

In a loop find the next ',' char. Take the substring between the ',' and the previous ',' position. At the end of the loop save a reference to the previous ',' position (which is initially set to 0). So you parse the items one-by-one rather than all at once.

You can try looping the items of your lists, but this will increase processing time.
foreach(var item in powlist)
{
//do your opeartions
}