Extract data into datatable to draw multiple graphs on one chart - c#

I am working on a project that aims to extract and treat data for statitics purpose.
Let's say I have many elements "E" and each element has a list of fields {F1, F2, F3, ...}.
My main table looks like the following:
I need to extract data by elementID into a data table with "Date" as Key.
[{"key": "date1",
"F1": "value",
"F2": "value"}
,{"key": "date2",
"F1": "value",
"F2": "value"}
,{"key": "date3",
"F1": "value",
"F2": "value"}
,{....}]
My current implementation is the following does the next:
1) Query from database by field and order by date in a Dictionary<DateTime, double>
2) Check and fill missing values in each Dictionary.
3) Loop through the list or Dictionary by key and fill a DataRow a row by row.
I don't think that this is the ultimate solution, I have been trying to optimize code. But I am not really sure in wich layer I should focus. Is there any possible way to get the required structure using a select from database ( no need to further loops ) ?

It's not that clear, however, i assume that you want to group by the date-part of the DateTime and select a dictionary where this date is the key and the value is a List<double>.
Then you don't need a DataTable at all. Instead of a double i would use custom classes with all properties. Assuming that mainTable is already a DataTable with the raw data:
public class ElementMeasurement
{
public Element Element { get; set; }
public double Value { get; set; }
public string Field { get; set; }
public DateTime Date { get; set; }
}
public class Element
{
public int ElementID { get; set; }
public string Name { get; set; }
}
Now you can use Enumerable.GroupBy which is part of System.Linq:
Dictionary<DateTime, List<ElementMeasurement>> dateMeasurements = mainTable.AsEnumerable()
.GroupBy(row => row.Field<DateTime>("Date").Date)
.ToDictionary(g =>
g.Key,
g => g.Select(row => new ElementMeasurement
{
Element = new Element { ElementId = row.Field<int>("ElementId") },
Field = row.Field<string>("Field"),
Value = row.Field<double>("Value"),
TimeOfMeasurement = row.Field<DateTime>("Date")
}).ToList()
);
Edit: "Well I should let you know that it is a huge table with thousands of miles of rows!"
I didnt know that the table was not already in memory. Then this might be too memory expensive. So maybe a loop on the DataReader which yields rows ordered by Date is more efficient. Then you could still use my classes above to keep it readable and maintainable but fill the Dictionary step-by-step.
For example (assuming SQL-Server):
var dateMeasurements = new Dictionary<DateTime, List<ElementMeasurement>>();
using(var con = new SqlConnection("ConnectionString"))
using (var cmd = new SqlCommand("SELECT e.* FROM Elements e ORDER BY Date,ElementId,Field,Value", con))
{
con.Open();
using (var rd = cmd.ExecuteReader())
while(rd.Read())
{
DateTime timeOfMeasurement = rd.GetDateTime(rd.GetOrdinal("Date"));
DateTime dateOfMeasurement = timeOfMeasurement.Date;
List<ElementMeasurement> measurements = null;
if (!dateMeasurements.TryGetValue(dateOfMeasurement, out measurements))
{
measurements = new List<ElementMeasurement>();
dateMeasurements.Add(dateOfMeasurement, measurements);
}
var measurement = new ElementMeasurement
{
Element = new Element { ElementId = rd.GetInt32(rd.GetOrdinal("ElementId")) },
Field = rd.GetString(rd.GetOrdinal("Field")),
Value = rd.GetDouble(rd.GetOrdinal("Value")),
TimeOfMeasurement = timeOfMeasurement
};
measurements.Add(measurement);
}
}

Related

How can I assign a set of values to a string or Int value?

I have a problem with C#.
I managed it to parse out the price of a material from a web page (with a list of materials and prices) but I don't know exactly how I can extract single values to single strings or ints. I have assigned the "price" to a string but it this string contains a array of values.
Here's the C# Code:
var products = htmlDocument.DocumentNode.Descendants("div")
.Where(node => node.GetAttributeValue("id", "")
.Equals("accordion1")).ToList();
var productListItems = products[0].Descendants("tr")
.Where(node => node.GetAttributeValue("data-name", "")
.Contains("")).ToList();
foreach (var productListItem in productListItems)
{
Console.WriteLine(productListItem.GetAttributeValue("data-name", ""));
var tds = productListItem.Descendants("td").ToList();
var name = tds[0].InnerText;
var price = tds[1].InnerText.Trim('$');
Console.WriteLine(name);
Console.WriteLine(price);
}
Console:
clay_r
Lehm
199
coal_r
Kohle
427
copper_r
Kupfer
312
etc.
Now I want, that I can call a single material (like clay) with a single string. So that I have for every materialPrice a int.
Something like:
int clayPrice = int.Parse(the first price)
int coalPrice = int.Parse(the second price)
int copperPrice = int.Parse(the third price)
etc.
I'm pretty new to C#, so I don't know much about it. I hope someone understands my problem and can help me.
Define a separate class that contains the pricing information.
For example:
public class PriceInfo
{
public string Name {get; set;}
public string RawPrice {get; set;}
public int Price => int.Parse(RawPrice.Trim('$'));
}
Define a Dictionary as a class member. Dictionary.
Dictionary<string, PriceInfo> priceData = new Dictionary<string, PriceInfo>();
In your loop, add the prices to the dictionary.
foreach (var productListItem in productListItems)
{
Console.WriteLine(productListItem.GetAttributeValue("data-name", ""));
var tds = productListItem.Descendants("td").ToList();
var name = tds[0].InnerText;
var price = tds[1].InnerText;
// add to Dictionary
priceData[name] = new PriceInfo
{
Name = name,
RawPrice = price
};
}
Use the value:
priceData["somename"].Price

Sorting List Array based on an index of array

I want to sort a List Array on the basis of an array item.
I have a List Array of Strings as below:
List<String>[] MyProjects = new List<String>[20];
Through a loop, I have added five strings
(Id, Name, StartDate, EndDate, Status)
to each of the 20 projects from another detailed List source.
for(int i = 0; i<20; i++){
MyProjects[i].Add(DetailedProjectList.Id.ToString());
MyProjects[i].Add(DetailedProjectList.Name);
MyProjects[i].Add(DetailedProjectList.StartDate);
MyProjects[i].Add(DetailedProjectList.EndDate);
MyProjects[i].Add(DetailedProjectList.Status)}
The Status values are
"Slow", "Normal", "Fast", "Suspended" and "" for unknown status.
Based on Status, I want to sort MyProject List Array.
What I have done is that I have created another List as below
List<string> sortProjectsBy = new List<string>(){"Slow", "Normal", "Fast", "", "Suspended"};
I tried as below to sort, however unsuccessful.
MyProjects = MyProjects.OrderBy(x => sortProjectsBy.IndexOf(4));
Can anyone hint in the right direction. Thanks.
I suggest you to create class Project and then add all the fields inside it you need. It's much nicer and scalable in the future. Then create a List or an Array of projects and use the OrderBy() function to sort based on the field you want.
List<Project> projects = new List<>();
// Fill the list...
projects.OrderBy(project => project.Status);
The field Status has to be a primitive type or needs to implement the interface IComparable in order for the sorting to work. I suggest you add an enum for Status with int values.
First consider maybe to use Enum for status and put it in a different file lite (utils or something) - better to work like that.
enum Status {"Slow"=1, "Normal", "Fast", "", "Suspend"}
Now about the filtering you want to achieve do it like this (you need to tell which attribute of x you are referring to. In this case is status)
MyProjects = MyProjects.OrderBy(x => x.status == enum.Suspend);
Read about enums :
https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/enum
Read about lambda expressions :
https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/statements-expressions-operators/lambda-expressions
First of all, storing project details as List is not adivisable. You need to create a Custom Class to represent them.
For example,
public class DetailedProjectList
{
public string Name {get;set;}
public eStatus Status {get;set;}
// rest of properties
}
Then You can use
var result = MyProjects.OrderBy(x=> sortProjectsBy.IndexOf(x.Status));
For example
List<string> sortProjectsBy = new List<string>(){"Slow", "Normal", "Fast", "", "Suspended"};
var MyProjects= new List<DetailedProjectList>{
new DetailedProjectList{Name="abc1", Status="Fast"},
new DetailedProjectList{Name="abc2", Status="Normal"},
new DetailedProjectList{Name="abc3", Status="Slow"},
};
var result = MyProjects.OrderBy(x=> sortProjectsBy.IndexOf(x.Status));
Output
abc3 Slow
abc2 Normal
abc1 Fast
A better approach thought would be to use Enum to represent Status.
public enum eStatus
{
Slow,
Normal,
Fast,
Unknown,
Suspended
}
Then your code can be simplified as
var MyProjects= new List<DetailedProjectList>{
new DetailedProjectList{Name="abc1", Status=eStatus.Fast},
new DetailedProjectList{Name="abc2", Status=eStatus.Normal},
new DetailedProjectList{Name="abc3", Status=eStatus.Slow},
};
var result = MyProjects.OrderBy(x=> x.Status);
Ok so you have a collection of 20 items. Based on them you need to create a list of strings(20 DetailedProjectList items).
What you can do to solve your problem is to SORT YOUR COLLECTION before you create your list of strings. In this way your list of strings will be sorted.
But your code is not optimal at all. So you should concider optimization on many levels.
Lets say you have ProjectDetail class as follow:
private class ProjectDetail
{
public int Id {get;set;}
public string Name {get;set;}
DateTime StartDate {get;set;} = DateTime.Now;
DateTime EndDate {get;set;} = DateTime.Now;
public string Status {get;set;}
public string toString => $"{Id} - {Name} - {StartDate} - {EndDate} - {Status}";
}
Notice that I have added a toString attribute to make things easier, and I also have added default values.
Then your program could be like:
static void Main(string[] args)
{
var projectDetails = MockProjectItems();
Console.WriteLine("Before sortig:");
foreach (var item in projectDetails)
{
Console.WriteLine(item.toString);
}
var myProjects = projectDetails.OrderBy(p => p.Status).Select(p => p.toString);
Console.WriteLine("\n\nAfter sorting:");
foreach (var item in myProjects)
{
Console.WriteLine(item);
}
}
where the helper method is
private static List<ProjectDetail> MockProjectItems()
{
var items = new List<ProjectDetail>(20);
for(int i = 0; i < 20 ; i += 4){
items.Add(new ProjectDetail{Id = i, Name = "RandomName "+i, Status = "Slow"});
items.Add(new ProjectDetail{Id = i+1, Name = "RandomName "+(i+1), Status = "Normal"});
items.Add(new ProjectDetail{Id = i+2, Name = "RandomName "+(i+2), Status = "Fast"});
items.Add(new ProjectDetail{Id = i+3, Name = "RandomName "+(i+3), Status = "Suspended"});
}
return items;
}
Then your program should print the following:

How to process XML file into C# List<T> on conditional basis?

Here is my sample XML schema that is generated by SQL Server. I want it to assign to List<class> in C# using LINQ. I tried it but I am getting six rows but after parsing this XML. I should get only one rows in list.
There some conditions to parse this XML.
TYPE == 1 then SMS,
TYPE == 3 then DATA
TYPE == 2 then MINUTES
XML:
<summary>
<period>Jul-2016</period>
<providerid>7</providerid>
<type>1</type>
<volume>2981655</volume>
</summary>
<summary>
<period>Jul-2016</period>
<providerid>7</providerid>
<type>2</type>
<volume>6449570</volume>
</summary>
<summary>
<period>Jul-2016</period>
<providerid>7</providerid>
<type>3</type>
<volume>7702484</volume>
</summary>
Here is my C# class structure. I want to parse this XML schema into a List<UsageSummary>. After parsing this there will be only two rows into List<UsageSummary>. I have used XDocument.Parse for XML parsing. After that, I am using Linq .Descendants method but I am getting six rows but output should contain only one rows.
public class UsageSummary
{
public int carrierID { get; set; }
public Int64 minutes { get; set; }
public Int64 sms { get; set; }
public Int64 data { get; set; }
public string period { get; set; }
}
I have used this code:
List<UsageSummary> obj = new List<UsageSummary>();
obj = (from res in xmlDoc.Descendants("summary")
select new UsageSummary
{
carrierID = (Convert.ToInt16(res.Element("providerid").Value)),
period = res.Element("period").Value.ToString(),
sms = (Convert.ToInt64(res.Element("SMS").Value)),
data = (Convert.ToInt64(res.Element("DATA").Value)),
minutes = (Convert.ToInt64(res.Element("MINUTES").Value))
}).ToList();
Kindly help me with this problem.
You need to group your elements. You can split the problem into 3 parts so you're not trying to this all at once.
First, map your XML onto a structure that mirrors the XML structure:
var items =
from summary in doc.Descendants("summary")
select new
{
Period = (string) summary.Element("period"),
CarrierId = (int) summary.Element("providerid"),
Type = (int) summary.Element("type"),
Volume = (long) summary.Element("volume")
};
Then group these by CarrierId and Period. For Volume, you want a lookup of all the volumes by Type:
var volumesByPeriodAndCarrier =
from item in items
group item by new {item.CarrierId, item.Period}
into grouping
select new
{
grouping.Key.CarrierId,
grouping.Key.Period,
VolumeByType = grouping.ToLookup(x => x.Type, x => x.Volume)
};
Then you can easily create your summaries. CarrierId and Period map directly to your class, and the 3 volumes are where you use your mapping from types to fields and sum the totals:
var summaries = volumesByPeriodAndCarrier
.Select(x => new UsageSummary
{
carrierID = x.CarrierId,
minutes = x.VolumeByType[2].Sum(),
sms = x.VolumeByType[1].Sum(),
data = x.VolumeByType[3].Sum(),
period = x.Period
}).ToList();
See this fiddle for a working demo.
This solution is implemented using a linq "pivoting" technic.
Pivoting consists in collecting different types of data, each of them corresponding to a given data record (or xml nodes in our example), in the fields of one record (or object..).
In our case, each pivot object contains the "set" of different types of data (on which you can operate several aggregation operators) of a specific grouping of the orginal data set.
In particular, it allows to sum volume values by type, providerid and period, and it gets zero(0) for missing type xml element values.
List<UsageSummary> obj = new List<UsageSummary>();
obj = (from xmlsummary in doc.Descendants("summary")
group xmlsummary by new { id = xmlsummary.Element("providerid").Value, period = xmlsummary.Element("period").Value } into summaryGrouped
select new UsageSummary{
carrierID = Convert.ToInt16(summaryGrouped.Key.id),
period = summaryGrouped.Key.period,
sms = (summaryGrouped.Where(sg => sg.Element("type").Value.Equals("1")).Select(v => Convert.ToInt64(v.Element("volume").Value)).Sum()),
minutes = (summaryGrouped.Where(sg => sg.Element("type").Value.Equals("2")).Select(v => Convert.ToInt64(v.Element("volume").Value)).Sum()),
data = (summaryGrouped.Where(sg => sg.Element("type").Value.Equals("3")).Select(v => Convert.ToInt64(v.Element("volume").Value)).Sum())
}
).ToList();

Cannot implicitly convert type '.List<AnonymousType#1>' to '.List<WebApplication2.Customer>'

In the following code that returns a list:
public List<Customer> GeAllCust()
{
var results = db.Customers
.Select(x => new { x.CustName, x.CustEmail, x.CustAddress, x.CustContactNo })
.ToList()
return results;
}
I get an error reporting that C# can't convert the list:
Error: Cannot implicitly convert type System.Collections.Generic.List<AnonymousType#1> to System.Collections.Generic.List<WebApplication2.Customer>
Why is that?
Here's a screenshot showing some additional information that Visual Studio provides in a tooltip for the error:
Is it right way to return some columns instead of whole table....?
public object GeAllCust()
{
var results = db.Customers.Select(x => new { x.CustName, x.CustEmail, x.CustAddress, x.CustContactNo }).ToList();
return results;
}
When you look the code:
x => new { ... }
This creates a new anonymous type. If you don't need to pull back only a particular set of columns, you can just do the following:
return db.Customers.ToList();
This assumes that Customers is an IEnumerable<Customer>, which should match up with what you are trying to return.
Edit
You have noted that you only want to return a certain subset of columns. If you want any sort of compiler help when coding this, you need to make a custom class to hold the values:
public class CustomerMinInfo
{
public string Name { get; set; }
public string Email { get; set; }
public string Address { get; set; }
public int? ContactNumber { get; set; }
}
Then change your function to the following:
public List<CustomerMinInfo> GetAllCust()
{
var results = db.Customers.Select(x => new CustomerMinInfo()
{
Name = x.CustName,
Email = x.Email,
Address = x.Address,
ContactNumber = x.CustContactNo
})
.ToList();
return results;
}
This will work, however, you will lose all relationship to the database context. This means if you update the returned values, it will not stick it back into the database.
Also, just to repeat my comment, returning more columns (with the exception of byte arrays) does not necessarily mean longer execution time. Returning a lot of rows means more execution time. Your function is returning every single customer in the database, which when your system grows, will start to hang your program, even with the reduced amount of columns.
You are selecting to an anonymous type, which is not a Customer.
If you want to do (sort of) this, you can write it like this:
return db.Customers.Select(x => new Customer { Name = x.CustName, Email = x.CustEmail, Address = x.CustAddress, ContactNo = x.ContactNo }).ToList();
This assumes the properties on your Customer object are what I called them.
** EDIT ** Per your comment,
If you want to return a subset of the table, you can do one of two things:
Return the translated form of Customer as I specified above, or:
Create a new class for your business layer that only has only those four fields, and change your method to return a List<ShrunkenCustomer> (assuming ShunkenCustomer is the name that you choose for your new class.)
GetAllCust() is supposed to return a List of Customer, Select New will create a list of Anonymous Types, you need to return a list of Customer from your query.
try:
var results = db.Customers.Select( new Customer{CustName = x.CustName}).ToList(); //include other fields
I guess Customer is a class you have defined yourself?
The my suggestion would be to do something like the following:
var results = db.Customers.Select(x => new Customer(x.Custname, x.CustEmail, x.CustAddress, x.CustContactNo)).ToList();
The reason is that you are trying to return a list of Customer but the results from your link is an anonymous class containing those four values.
This would of course require that you have a constructor that takes those four values.
Basically whatever u got in var type, loop on that and store it in list<> object then loop and achieve ur target.Here I m posting code for Master details.
List obj = new List();
var orderlist = (from a in db.Order_Master
join b in db.UserAccounts on a.User_Id equals b.Id into abc
from b in abc.DefaultIfEmpty()
select new
{
Order_Id = a.Order_Id,
User_Name = b.FirstName,
Order_Date = a.Order_Date,
Tot_Qty = a.Tot_Qty,
Tot_Price = a.Tot_Price,
Order_Status = a.Order_Status,
Payment_Mode = a.Payment_Mode,
Address_Id = a.Address_Id
});
List<MasterOrder> ob = new List<MasterOrder>();
foreach (var item in orderlist)
{
MasterOrder clr = new MasterOrder();
clr.Order_Id = item.Order_Id;
clr.User_Name = item.User_Name;
clr.Order_Date = item.Order_Date;
clr.Tot_Qty = item.Tot_Qty;
clr.Tot_Price = item.Tot_Price;
clr.Order_Status = item.Order_Status;
clr.Payment_Mode = item.Payment_Mode;
clr.Address_Id = item.Address_Id;
ob.Add(clr);
}
using(ecom_storeEntities en=new ecom_storeEntities())
{
var Masterlist = en.Order_Master.OrderByDescending(a => a.Order_Id).ToList();
foreach (var i in ob)
{
var Child = en.Order_Child.Where(a => a.Order_Id==i.Order_Id).ToList();
obj.Add(new OrderMasterChild
{
Master = i,
Childs = Child
});
}
}

Get SQL LINQ Results Based off of String List

Lets start off with a list of strings that will be used to filter the results:
List<String> RadioNames = new List<String>();
RadioNames.AddRange(new String[] { "abc", "123", "cba", "321" });
I want to be able to filter a LINQ to SQL database table based on RadioNames but the catch is that I want RadioNames to be a partial match (meaning it will catch Radio123 and not just 123).
The source that I need to filter is below:
var ChannelGrants = from cg in sddc.ChannelGrants
select new
{
cg.ID,
cg.Timestamp,
cg.RadioID,
cg.Radio
};
So I need to perform something similar to below (outside of the original ChannelGrants results as this is a conditional search)
if(RadioNamesToSearch != null)
{
List<String> RadioNames = new List<String>();
// Here I split all the radio names from RadioNamesToSearch based on a command separator and then populate RadioNames with the results
ChannelGrants = from cg in ChannelGrants
where ???
select cg;
}
I need help where ??? is in the code above (or if ChannelGrants = ... is invalid all together). Repeating above, I need to filter ChannelGrants to return any matches from RadioNames but it will do partial matches (meaning it will catch Radio123 and not just 123).
All the code is contained in a method as such...
public static DataTable getBrowseChannelGrants(int Count = 300, String StartDate = null, String StartTime = null, String EndDate = null, String EndTime = null, String RadioIDs = null, String RadioNamesToSearch = null, String TalkgroupIDs = null, String TalkgroupNames = null, bool SortAsc = false)
What field in ChannelGrants are you comparing RadioNames to?
To retrieve entries that are only in your RadioNames list, you'd use the contains method like this
ChannelGrants = from cg in ChannelGrants
where RadioNames.Contains(cg.Radio)
select cg;
(If you wanted to find all rows that had one of your RadioNames in the Radio property. Replace cg.Radio with the appropriate column you are matching)
This gives you a similar outcome if you had this where clause in SQL
where cg.Radio in ("abc", "123", "cba", "321")
from this link How to do SQL Like % in Linq?
it looks like you can combo it with like matching as well, but adding slashes, by it's not something I've done personally.
in place of the ???
RadioNames.Where(rn=>cg.Radio.ToLower().Contains(rn.ToLower())).Count() > 0
That should do it...
The ToLower() calls are optional, of course.
EDIT: I just wrote this and it worked fine for me in a Console Application. The result contained one item and the WriteLine spit out "cbaKentucky". Not sure what to tell ya.
class Program
{
static void Main(string[] args)
{
List<String> RadioNames = new List<String>();
RadioNames.AddRange(new String[] { "abc", "123", "cba", "321" });
List<ChannelGrants> grants = new List<ChannelGrants>();
grants.Add(new ChannelGrants() { ID = 1, Radio = "cbaKentucky", RadioID = 1, TimeStamp = DateTime.Now });
var result = from cg in grants
where RadioNames.Where(rn=>cg.Radio.ToLower().Contains(rn.ToLower())).Count() > 0
select cg;
foreach (ChannelGrants s in result)
{
Console.WriteLine(s.Radio);
}
}
}
class ChannelGrants
{
public int ID { get; set; }
public DateTime TimeStamp { get; set; }
public int RadioID { get; set; }
public string Radio { get; set; }
}
At the moment, there doesn't seem to be a best way so I'll answer this until a new answer that doesn't repeat the other answers that don't work on this thread.

Categories

Resources