CSV total data and list count equivalence (C#) - c#

I have standard code that reads a csv file full of data that was obtained by reading data from a load cell. I get that data read it. And put each column into a list. The CSV file has a column length of about 118,000 but my list is only 14365. Why might be happening?
Here is my code - class of readings:
public class ReadingClassSample
{
public double Time;
public double Ax;
public double Ay;
public double Az;
public double Bx;
public double By;
public double Bz:
public double Cx;
public double Cy;
public double Cz;
public double Dx;
public double Dy;
public double Dz:
public ReadingClassSample()
{}
public ReadingClassSample(string[] vals)
{
try
{
Double.TryParse(vals[0], out Time);
Double.TryParse(vals[1], out Ax);
Double.TryParse(vals[2], out Ay);
Double.TryParse(vals[3], out Az);
Double.TryParse(vals[6], out Bx);
Double.TryParse(vals[5], out By);
Double.TryParse(vals[6], out Bz);
Double.TryParse(vals[7], out Cx);
Double.TryParse(vals[8], out Cy);
Double.TryParse(vals[9], out Cz);
Double.TryParse(vals[10], out Dx);
Double.TryParse(vals[11], out Dy);
Double.TryParse(vals[12], out Dz);
}
catch
{
MessageBox.Show("There was an error parsing the data into a double.")
}
}
}
CSV File Reader Code
public void ReadLCCSVFile(string samplePath)
{
StreamReader sampleReader;
List<ReadingClassSample> lcOutputList = new List<ReadingClassSample>();
List<double> AxList = new List<double>();
List<double> AyList = new List<double>();
List<double> AzList = new List<double>();
List<double> BxList = new List<double>();
List<double> ByList = new List<double>();
List<double> BzList = new List<double>();
List<double> CxList = new List<double>();
List<double> CyList = new List<double>();
List<double> CzList = new List<double>();
List<double> DxList = new List<double>();
List<double> DyList = new List<double>();
List<double> DzList = new List<double>();
if (File.Exists(samplePath))
{
sampleReader = new StreamReader(File.OpenRead(samplePath));
while (!sampleReader.EndOfStream)
{
var line = sampleReader.ReadLine();
string[] values = line.Split(',');
ReadingClassSample temp = new ReadingClassSample(values);
sampleOutputList.Add(temp);
}
AxList.Add(sampleOutputList[ii].Ax);
AyList.Add(sampleOutputList[ii].Ay);
AzList.Add(sampleOutputList[ii].Az);
BxList.Add(sampleOutputList[ii].Bx);
ByList.Add(sampleOutputList[ii].By);
BzList.Add(sampleOutputList[ii].Bz);
CxList.Add(sampleOutputList[ii].Cx);
CyList.Add(sampleOutputList[ii].Cy);
CzList.Add(sampleOutputList[ii].Cz);
DxList.Add(sampleOutputList[ii].Dx);
DyList.Add(sampleOutputList[ii].Dy);
DzList.Add(sampleOutputList[ii].Dz);
SampleAxList = AxList;
SampleAyList = AyList;
SampleAzList = AzList;
SampleBxList = BxList;
SampleByList = ByList;
SampleBzList = BzList;
SampleCxList = CxList;
SampleCyList = CyList;
SampleCzList = CzList;
SampleDxList = DxList;
SampleDyList = DyList;
SampleDzList = DzList;
}
}
TLDR: when I check the CSV file its has usually over 118,000 values in a column but my code only takes 14,000 - 16,000 for some reason.

StreamReader reads an X amount of bytes from the underlining stream and put it in his internal buffer. The stream position will be after the bytes read, but that is not the position where you already have read, that depends on what in the buffer you have already read. So, avoid dealing direct with the underlining stream (like Seek, Position and EndofStream), because that's not the position where you actually are. StreamWriter is similar, that's why it has to flush the data (transfer from buffer to stream).
So, instead of
while (!sampleReader.EndOfStream)
{
var line = sampleReader.ReadLine();
use
string line = null;
while ((line = sampleReader.ReadLine()) != null)
{
Since ReadLine and Read consumes from the buffer.

Related

C# Newtonsoft Json deserialize array of different inheirted objects from json file

I have an array of objects which I want to write to a json file and recover it later
Point - Visual Basic class:
<Serializable> Public Class Point
Implements IComparable
Public Property X As Integer
Public Property Y As Integer
Public Sub New()
X = 0
Y = 0
End Sub
Public Sub New(ByVal x As Integer, ByVal y As Integer)
Me.X = x
Me.Y = y
End Sub
End Class
Point3D - C# class
[Serializable]
public class Point3D : Point
{
public int Z { get; set; }
public Point3D() : base()
{
var rnd = new Random();
Z = rnd.Next(10);
}
public Point3D(int x, int y, int z) : base(x, y)
{
this.Z = z;
}
}
Serialization works fine but when I try to deserialize json from file, all objects presented as Point3D (even the ones with only X and Y variables).
Before deserialization:
enter image description here
After:
enter image description here
Serialization code block:
using (var fs = new FileStream(dlg.FileName, FileMode.Create, FileAccess.Write))
{
switch (Path.GetExtension(dlg.FileName))
{
...
case ".json":
var jf = new JsonSerializer();
jf.TypeNameHandling= TypeNameHandling.Auto;
using (var w = new StreamWriter(fs))
jf.Serialize(w, points);
break;
}
}
Deserialization code block:
using (var fs = new FileStream(dlg.FileName, FileMode.Open, FileAccess.Read))
{
switch (Path.GetExtension(dlg.FileName))
{
...
case ".json":
var jf = new JsonSerializer();
using (var r = new StreamReader(fs))
points = (Point[])jf.Deserialize(r, typeof(Point3D[]));
break;
}
}
Code I use to initialize array of objects (Point and Point3D):
private void buttonCreate_Click(object sender, EventArgs e)
{
points = new Point[5];
var rnd = new Random();
for (int i = 0; i < points.Length; i++)
{
points[i] = rnd.Next(3) % 2 == 0 ? new Point() : new Point3D();
}
listBox.DataSource = points;
}
And json string
{"$type":"PointLib.Point[], PointLib","$values":[{"$type":"LabOneFormsApp.Point3D, LabOneFormsApp","Z":6,"X":0,"Y":0},{"$type":"LabOneFormsApp.Point3D, LabOneFormsApp","Z":1,"X":0,"Y":0},{"$type":"PointLib.Point, PointLib","X":0,"Y":0},{"$type":"PointLib.Point, PointLib","X":0,"Y":0},{"$type":"PointLib.Point, PointLib","X":0,"Y":0}]}
I tried adding
jf.TypeNameHandling= TypeNameHandling.Auto;
to my code but it does not seem to work for me. Any ideas?
I've decided to do it simple way
case ".json":
var reader = new StreamReader(fs).ReadToEnd();
var jsonFile = JsonConvert.DeserializeObject<List<Dictionary<string, int>>>(reader);
if (jsonFile != null)
{
points = new Point[jsonFile.Count];
for (int i = 0; i < jsonFile.Count; i++)
{
var pointDict = jsonFile[i];
switch (pointDict.Count)
{
case 2:
points[i] = new Point(pointDict["X"], pointDict["Y"]);
break;
case 3:
points[i] = new Point3D(pointDict["X"], pointDict["Y"], pointDict["Z"]);
break;
}
}
}
else
{
points = null;
}
Thanks everyone for the answers!

How to pass Multiple two-dimensional list, lists and variables using JsonResult as return from a controller to a view in Asp.net MVC 5

I am trying to pass the single and two-dimensional list of data from a controller to view using JsonResult in asp.net MVC 5
In Controller
public JsonResult floatingFloorCalc(double roomlen, double roomwid, double roomhgt, int nmOpSde, double roomvol, int msurmnt, double[] tAreaArrVal, int[] dIndxArrVal, int spl63val, int spl125val, int spl250val, int spl500val, int spl1kval, int spl2kval, int spl4kval, int spl8kval, int afiltCor, double equiplen, double equipwid, double equiphgt, int plintReq, double plintlen, double plintwid, double plinthgt, double plintdnsty, double distnFloor1, double distnFloor2, double distnFloor3, double distnFloor4, int mthrslbthck, int fltflrslbthck, double re63, double re125, double re250, double re500, double re1k, double re2k, double re4k, double re8k, int ncrid)
{
tbic tb = new tbic();
tb.tbic_calc(roomlen, roomwid, roomhgt, roomvol, msurmnt, tAreaArrVal, dIndxArrVal, out double totalRoomArea, out double roomVolInFt, out List<List<double>> listAllCoefficeint, out List<List<double>> listAreaCoefMulti, out List<double> listAreaCoefSum, out List<double> airAbsorpConst, out List<double> listAirAbsorption, out List<double> listRoomObsoprtion, out List<double> listRoomConstant);
List<double> splLvlList = new List<double> { spl63val, spl125val, spl250val, spl500val, spl1kval, spl2kval, spl4kval, spl8kval };
bool aFiltr = Convert.ToBoolean(afiltCor);
List<double> DistanceFromFloorList = new List<double> { distnFloor1, distnFloor2, distnFloor3, distnFloor4 };
List<double> roomEffectList = new List<double> { re63, re125, re250, re500, re1k, re2k, re4k, re8k };
tspl tspl = new tspl();
tspl.chiller1(splLvlList, aFiltr, nmOpSde, DistanceFromFloorList, listRoomConstant, roomEffectList, ncrid, mthrslbthck, fltflrslbthck);
loadCalculations lc = new loadCalculations();
lc.calc_Plinth(plintlen, plintwid, plinthgt, plintdnsty);
return Json(? , JsonRequestBehavior.AllowGet);
}
-- ? = How to pass so many output parameters to view.
Class tbic.tbic_calc
Following are the output variables returned from the method
public class tbic
{
//Creating the instance of the dbmodel
db_eaciEntities _eaciEntities = new db_eaciEntities();
public void tbic_calc(double roomlen, double roomwid, double roomhgt, double roomvol, int msurmnt, double[] tAreaArrVal, int[] dIndxArrVal, out double totalRoomArea, out double roomVolInFt, out List<List<double>> listAllCoefficeint, out List<List<double>> listAreaCoefMulti, out List<double> listAreaCoefSum, out List<double> airAbsorpConst, out List<double> listAirAbsorption, out List<double> listRoomObsoprtion, out List<double> listRoomConstant)
{
#region Intialize the output variables
totalRoomArea = 0;
double totalRoomAreaInFt = 0;
roomVolInFt = 0;
listAllCoefficeint = new List<List<double>>();
listAreaCoefMulti = new List<List<double>>();
listAreaCoefSum = new List<double>();
listAirAbsorption = new List<double>();
listRoomObsoprtion = new List<double>();
listRoomConstant = new List<double>();
#endregion
for (int x = 0; x < dIndxArrVal.Count(); x++)
{
//fetch coeffcient id
int indx = dIndxArrVal[x];
//Fetch the total area of same surface type
}
Similarly the output variables from Tspl.chiller
Which are both lists and two-dimensional lists and some variables
The output lists contain data that needed to be displayed on to the view.
Tried to pass these to another list so that a single list can be passed view ajax data but when passing two-dimensional list it gives me an error can't convert double[] to double.
I have encountered a roadblock on;
How to pass these lists data from a controller to view to display as I am using JsonResult to calculation on button click

How to read 2D int array from binary file in c#?

I have a 2D integer array to store x,y coordinates. I checked out a few functions to write 2D array into a file but cannot find anything that is able to read that binary file on load and push it into a new 2 dimensional integer array.
This is my world generator function which saves it to the file:
public WorldGenerator()
{
int worldSizeX = 100;
int worldSizeY = 100;
int[,] world = new int[worldSizeX*worldSizeY, 2];
Logger.log("Generating world...");
for(int x = 0; x < worldSizeX; x++)
{
for(int y = 0; y < 2; y++)
{
System.Random random = new System.Random();
int itemID = random.Next(0, 1);
world[x, y] = itemID;
}
}
FileStream fs = new FileStream(Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData) + "/ConsoleGame/world/default.wd", FileMode.OpenOrCreate, FileAccess.Write);
BinaryWriter bw = new BinaryWriter(fs);
for (int x = 0; x < worldSizeX; x++)
{
for (int y = 0; y < 2; y++)
{
bw.Write(world[x, y]);
}
}
bw.Close();
fs.Close();
Logger.log("World generated.");
}
Any good idea that could work for reading this file in? I should get back a 2D integer array and world[0,0] should get me the itemid. I am new to c# and this would be just a basic console application.
I have also seen others answering similar questions but none of them are worked for me yet. Maybe because this save function is wrong or something else.
EDIT:
Here is how I load the file:
using (var filestream = File.Open(Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData) + "/ConsoleGame/world/default.wd", FileMode.Open))
using (var binaryStream = new BinaryReader(filestream))
{
while (binaryStream.PeekChar() != -1)
{
Console.WriteLine(binaryStream.ReadInt32());
}
}
need Newtonsoft.Json
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Newtonsoft.Json;
namespace ConsoleApp18
{
class Program
{
static void Main(string[] args)
{
int worldSizeX = 100;
int worldSizeY = 100;
int[,] world = new int[worldSizeX * worldSizeY, 2];
System.Random random = new System.Random();
for (int x = 0; x < worldSizeX; x++)
{
for (int y = 0; y < 2; y++)
{
int itemID = random.Next(0, 2);
world[x, y] = itemID;
}
}
string json = JsonConvert.SerializeObject(world, Formatting.Indented);
System.IO.File.WriteAllText("WriteText.txt", json);
string text = System.IO.File.ReadAllText("WriteText.txt");
int[,] deserialized = JsonConvert.DeserializeObject<int[,]>(text);
//use "deserialized"
}
}
}
What you need is called "Serialization".
Start with the simple builtin binary serializer.
Serializable attribute does the magic here.
By the moment you'll realize it is not the best option, you'll be able to use something more suiting your needs, like proto-buf.
I've also changed ints to shorts in your example. I doubt you need 32 bits for each world cell, so we can save a bit of hard drive space.
[Serializable]
public class WorldState
{
public short[,] Items { get; set; }
public void Save(string filename)
{
if (filename == null) throw new ArgumentNullException(nameof(filename));
using (var file = File.Create(filename))
{
var serializer = new BinaryFormatter();
serializer.Serialize(file, this);
}
}
public static WorldState Load(string filename)
{
if (filename == null) throw new ArgumentNullException(nameof(filename));
if (!File.Exists(filename)) throw new FileNotFoundException("File not found", filename);
using (var file = File.OpenRead(filename))
{
var serializer = new BinaryFormatter();
return serializer.Deserialize(file) as WorldState;
}
}
}
public class WorldStateTests
{
[Fact]
public void CanSaveAndLoad()
{
var ws = new WorldState
{
Items = new short[,]
{
{ 1, 2, 3, 4 },
{ 1, 2, 3, 4 },
{ 1, 2, 3, 4 },
{ 1, 2, 3, 4 }
}
};
// save the world state to file. Find it and see what's inside
ws.Save("./ws.bin");
// load the world back
var loaded = WorldState.Load("./ws.bin");
// check a new world state got loaded
Assert.NotNull(loaded);
// and it still has items
Assert.NotEmpty(loaded.Items);
// and the items are the same as we saved
Assert.Equal(ws.Items, loaded.Items);
}
}

Combining var and Dynamic statements

Heres a fun problem I have.
I have a function that returns a var of items;
var Items = new { sumList = SumList, ratesList = List, sum = List.Sum() };
return Items;
From a function that is dynamic:
public override dynamic GetRates()
and I return it to a function I else where and try to apply it to my code:
dynamic res = cl.mainC.GetRates();
List<double> MashkantaSumList = res.sumList;
Now when I try to apply it, it says the object doesnt exist. But if I look in the debugger the items are happily there as a generic list or what not.
How do I resolve this?
EDIT:
as per request I'll post the full code:
//virtual
public virtual dynamic TotalMashkanta(int i, double sum, double ribit, string[] discount)
{
return 0;
}
//override
public override dynamic TotalMashkanta(int i, double sum, double ribit, string[] discount)
{
double SumTemp = sum;
double monthlyRibit = ribit / 12;
Double permPayPerMont = Financial.Pmt(monthlyRibit, i, sum, 0, DueDate.EndOfPeriod);
List<double> MashkantaList = new List<double>();
List<double> MashkantaSumList = new List<double>();
for (int j = 1; j <= i; j++)
{
MashkantaList.Add(Mashkanta(j, sum, ribit, permPayPerMont) * (1 - CalcDiscount((j / 12) + 1, discount)));
SumTemp = getSum(j, sum, ribit, permPayPerMont * -1); ;
MashkantaSumList.Add(SumTemp);
}
var K_Mashkanta = new { sumList = MashkantaSumList, ratesList = MashkantaList, sum = MashkantaList.Sum() };
return K_Mashkanta;
}
//Function that calls the results
public void GetSilukinTable(string Path, string ClientID, DAL.Client client, string partner_checked, string insurance_Amount, string Premiya_Structure_Mashkanta, string Premiya_Life_Mashkanta, string Discount_Life_Mashkanta, string Loan_Period,string Loan_EndDate, string Bank, string Loan_Interest, string Loan_Amount, string Discount_Loan, string AgentNotes, string ManID)
{
BL.CalculateLogic.Companies t = BL.CalculateLogic.Companies.כלל;
if(ManID == "211") t = BL.CalculateLogic.Companies.הפניקס;
if(ManID == "207") t = BL.CalculateLogic.Companies.הראל;
if(ManID == "206") t = BL.CalculateLogic.Companies.מנורה;
if(ManID == "208") t = BL.CalculateLogic.Companies.הכשרה;
BL.CalculateLogic cl = new BL.CalculateLogic(client, t);
DateTime LoanEnd = DateTime.Now;
int months = 0;
if (DateTime.TryParse(Loan_EndDate, out LoanEnd))
months = BL.Calculating_Companies.Company.GetMonthsBetween(DateTime.Now, LoanEnd);
else
months = Int32.Parse(Loan_Period) * 12;
string[] Discount = Discount_Loan.Split('-');
dynamic res = cl.mainC.TotalMashkanta(months, Double.Parse(Loan_Amount), Double.Parse(Loan_Interest.Trim('%')), Discount);
var MashkantaSumList = res.sumList;
List<double> MashkantaList = res.ratesList;
List<double> MashkantaSumListPartner = new List<double>();
List<double> MashkantaListPartner = new List<double>();
List<double> MashkantaListSum = res.ratesList;
}
The compiler is happy about it because dynamic is compiled and checked at run time. Whatever the problem is, the types don't match. It evaluates this at run time, so you won't see issues at compile time. (Advice: use dynamic only when you really must! Else you will have this kind of problems all the time!)
I tried your code using this and it works fine:
static dynamic GetRates()
{
List<double> SumList = new List<double>();
List<double> List = new List<double>();
var Items = new { sumList = SumList, ratesList = List, sum = List.Sum() };
return Items;
}
static void Main(string[] args)
{
dynamic res = GetRates();
List<double> MashkantaSumList = res.sumList;
}

Why DFT from AForge.Math doesn't work?

I tried to get fft and then get dft of a wave using this code:
string s = textBox1.Text;
double[] source = SourceToDouble(s);
listBox2.DataSource = source;
ToPowerOfTwo(ref source);
List<Complex> Source = DoubleToComplex(source);
Complex[] sou = Source.ToArray();
FourierTransform.FFT(sou, FourierTransform.Direction.Forward);
listBox1.DataSource = sou;
FourierTransform.DFT(sou, FourierTransform.Direction.Forward);
DoPlaySound(sou);
SourceToDouble(s):
private double[] SourceToDouble(string s)
{
List<double> Final = new List<double>();
EricOulashin.WAVFile audioFile = new EricOulashin.WAVFile();
String warning = audioFile.Open(s, WAVFile.WAVFileMode.READ);
if (warning == "")
{
short audioSample = 0;
for (int sampleNum = 0; sampleNum < audioFile.NumSamples; ++sampleNum)
{
audioSample = audioFile.GetNextSampleAs16Bit();
Final.Add((double)audioSample);
}
}
else
{
throw new Exception(warning);
}
return Final.ToArray();
}
ToPowerOfTwo(ref source):
private void ToPowerOfTwo(ref double[] source)
{
List<long> TwoPowers = GetTwoPowers(100);
long pCount = 0;
for (int i = 0; i <= 100; i++)
{
if (source.Count() <= TwoPowers[i])
{
pCount = TwoPowers[i];
break;
}
}
List<double> f = new List<double>(source);
while (f.Count < pCount)
{
f.Add(0.0);
}
//f.Add(0.0);
source = f.ToArray();
}
DoubleToComplex(source):
private static List<Complex> DoubleToComplex(double[] source)
{
List<Complex> Source = new List<Complex>();
foreach (double dob in source)
{
Complex c = new Complex(dob, 0.0);
Source.Add(c);
}
return Source;
}
DoPlaySound(sou):
private void DoPlaySound(Complex[] c)
{
FourierTransform.DFT(c, FourierTransform.Direction.Forward);
double wav = c[0].Re;
List<double> Big = ToBigger(100000, new double[] { wav });
MakeWavFile(Big, "tmp.wav");
System.Media.SoundPlayer s = new SoundPlayer("tmp.wav");
s.PlayLooping();
}
the problem is this: when I give a wav file to general code, after a long time the method trying to play the final wav (tmp.wav) but it isn't like the general file.
Update 1:
I also tried FourierTransform.DFT(sou, FourierTransform.Direction.Backward); but it didn't work too!
An FFT of an entire audio file is not a good way of analyzing voice or other non-stationary information. It would also be very slow. A more common technique would be to analyse short overlapping frames of a few to few dozen milliseconds in length using shorter FFTs, or to process successive frames using overlap-add/save FFT fast convolution.
A single IFFT would be the proper inverse function to an FFT. Otherwise you might end up with a mis-scaled backwards result.
It seems you are running a forward DFT three times on your data between reading it and playing it. So of course it's not going to sound like the original.
I would rewrite the DoubleToComplex function as:
private static Complex[] DoubleToComplex(double[] source)
{
Complex[] complexSource = new Complex[source.Length];
for(int i =0; i< source.Length; i++ )
{
complexSource[i] = new Complex(source[i], 0.0);
}
return complexSource;
}
Why first create a list to create all the Complex object, to bring it later on back to an Array? To use an Array from the beginning is more efficient.

Categories

Resources