I have a class storedetails which get different values from database. fetching these values from database and calculating takes a long time(due to poor design of the database over 76 different queries for one store).
My implementation for generating the report is as below.
string week = "32";
string year = "2013";
string[] storecode = getallstoreforweekandyear(week, year); // get all store code that were active for the week and year
ArrayList ar = new ArrayList();
foreach (string item in storecode)
{
storedetails sd = new storedetails(item, year, week);// this initializion I want to move to another thread because it is taking time.
ar.Add(sd);
}
ar.TrimToSize();
DataTable dt = getdtfromarraylist(ar);// convert the arraylist of class to datatable
Gridview1.Datasourc=dt;
My implementation of class is in over 2000 line of code
class storedetails
{
public storedetails(string store,string year, string week)
{
//time taking implementation
}
}
Is it possible that initialization of classes occur in separate threads, so I can gain some speed?
Have you checked out the Task Parallel Libraries in .NET 4.0 - they simplify threading a lot.
http://msdn.microsoft.com/en-us/library/dd460717(v=vs.110).aspx
object locker = new object();
Parallel.ForEach (storecode => item
{
storedetails sd = new storedetails(item, year, week);
lock(locker)
{
ar.Add(sd);
}
});
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 6 months ago.
Improve this question
I have a CSV File with these headers:
date;clock;value
My aim is to select the CSV line with a specific date to get the corresponding value.
For example:
I want to select date 20.08.22 and the result should be 130
15.08.22;07:05;100
20.08.22;08:04;130
21.08.22;10:04;150
With this code snippet I read the lines of the csv file:
private void Werte_aus_CSV_auslesen()
{
var path = #"E:\werte.csv";
using (TextFieldParser csvParser = new TextFieldParser(path))
{
csvParser.CommentTokens = new string[] { "#" };
csvParser.SetDelimiters(new string[] { ";" });
csvParser.HasFieldsEnclosedInQuotes = true;
// Skip the row with the column names
csvParser.ReadLine();
while (!csvParser.EndOfData)
{
// Read current line fields, pointer moves to the next line.
fields = csvParser.ReadFields();
datum.Add(fields[0]);
uhrzeit.Add(fields[1]);
wert.Add(double.Parse(fields[2], CultureInfo.InvariantCulture));
}
}
}
The approach you are using is going to have to scan the entire CSV every time you lookup a value. This might be a performance problem if this method is called multiple times. It would be better to build a dictionary that maps the date to the value that can be built once and reused for each subsequent lookup.
I maintain a couple libraries that make this pretty easy: Sylvan.Data and Sylvan.Data.Csv. Here is a complete C# 10 console app that demonstrates how to accomplish this:
using Sylvan.Data.Csv;
using Sylvan.Data;
// data: would normally use CsvDataReader.Create(csvFileName, opts);
var data =
new StringReader(
#"date;clock;value
15.08.22;07:05;100
20.08.22;08:04;130
21.08.22;10:04;150
");
// parameter:
var selectDate = new DateTime(2022, 8, 20);
// configure settings so the csv reader understands your data
var opts = new CsvDataReaderOptions
{
DateTimeFormat = "dd'.'MM'.'yy",
// ignore clock, as it isn't used
Schema = new CsvSchema(Schema.Parse("date:date,clock,value:int"))
};
var csvReader = CsvDataReader.Create(data, opts);
// create a dictionary to cache the CSV data for quick lookups
// creating the dictionary scans the whole dataset, but subsequent lookups will
// be blazing fast.
{
var dict =
csvReader
.GetRecords<Record>() // bind the CSV data to the Record class
.ToDictionary(r => r.Date, r => r.Value);
Console.WriteLine(dict.TryGetValue(selectDate, out var value) ? value.ToString() : "Value not found");
}
class Record
{
public DateTime Date { get; set; }
public int Value { get; set; }
}
Matched arrays/lists like datum, uhrzeit, and wert that relate values within each collection based on index is an anti-pattern... something to avoid. So much better to create a class with fields for each of the values, and then have one collection to hold the class.
public class MyData
{
public DateTime date {get;set;}
public int value {get;set;}
}
(Of course, give it a better name than "MyData")
Newer code might also use a record instead of a class.
We can further improve this by separating the code to read the csv data from the code that composes the objects. Start with something like this:
private IEnumerable<string[]> Werte_aus_CSV_auslesen(string path)
{
using (TextFieldParser csvParser = new TextFieldParser(path))
{
csvParser.CommentTokens = new string[] { "#" };
csvParser.SetDelimiters(new string[] { ";" });
csvParser.HasFieldsEnclosedInQuotes = true;
// Skip the row with the column names
csvParser.ReadLine();
while (!csvParser.EndOfData)
{
// Read current line fields, pointer moves to the next line.
yield return csvParser.ReadFields();
}
}
}
Notice how it accepts an input and returns an object (the enumerable with the data). Also notice how it avoids anything to do with processing the individual rows. It is only concerned with parsing the CSV/SSV inputs. It doesn't care what fields you expect to find, and can handle any file input with a header line, hash comments, and semi-colon field separators.
Since this gives us string[] values, we also add a method to transform a string[] into a class instance. I like to start out with this as a static method of the class itself, but as a project grows to have many of these methods they may eventually be moved to their own static type:
public class MyData
{
public DateTime date {get;set;}
public int value {get;set}
public static MyData FromCSVRow(string[] input)
{
return new MyData() {
date = DateTime.ParseExact($"{input[0]} {input[1]}", "dd.MM.yy HH:mm", null),
value = int.Parse(input[2])
};
}
}
And now with all that out of the way, we can finally put it all together to get your answer:
var targetDate = new DateTime(2022, 8, 20);
var csv = Werte_aus_CSV_auslesen(#"E:\werte.csv");
var rows = csv.Select(MyData.FromCSV);
var result = rows.Where(r => r.date.Date == targetDate);
If we really wanted to, we could even treat all that as a single line of code (it's probably better to keep it separate, for readability/maintainability):
var result = Werte_aus_CSV_auslesen(#"E:\werte.csv").
Select(MyData.FromCSV).
Where(r => r.date.Date == new DateTime(2022, 8, 20));
Note result is still an IEnumerable<MyData>, because there might be more than one row matching the criteria. If you are really sure there will only be one matching record, you can use this:
var result = rows.Where(r => r.date.Date == targetDate).FirstOrDefault();
or this:
var result = rows.Where(r => r.date.Date == targetDate).First();
depending on what you want to happen if no match is found.
One of the nice features here is this checks each record as it reads the file, and will stop reading the file as soon as it finds a match, which is potentially a very nice performance win.
I started to make a football game app in C#. I should make 15 Matchdays. Every matchday has a number, date and two lists with teams in it. In every MatchDay the teams with the same index in the lists play against eachother. (TeamList1[0] plays against TeamList2[0], ...). I make 15 matchdays with a for loop and switch the teams in the lists with the SwitchTeams() method. First i make a matchday-object with the two lists in it, then I put this matchday in a MatchDayList and then i change the lists and make the next matchday object. But when i switch the teams in the list in the normal code, the list in my MatchDay object changes as well.
I don't know how this is possible and how to make the normal lists change and the lists that are in the MatchDay object remain the same. Does anyone have a solution for this? Thank you
//variables
List<Team> teamList1 = new List<Team>();
List<Team> teamList2 = new List<Team>();
List<MatchDay> matchDayList = new List<MatchDay>();
//Making the 15 Matchdays
private void ComposeGamesMenuItem_Click(object sender, RoutedEventArgs e)
{
DateTime date = new DateTime(DateTime.Now.Year, 7, 31);
while (date.DayOfWeek != DayOfWeek.Saturday)
{
date = date.AddDays(-1);
}
for (int i = 1; i <= 15; i++) {
List<Team> helpList1 = teamList1;
List<Team> helpList2 = teamList2;
MatchDay helpMatchDay = new MatchDay(i, helpList1, helpList2, date);
matchDayList.Add(helpMatchDay);
SwitchTeams();
date = date.AddDays(7);
}
}
// Switching teams in original lists
private void SwitchTeams()
{
teamList2.Insert(0, teamList1[1]);
teamList1.RemoveAt(1);
teamList1.Add(teamList2[8]);
teamList2.RemoveAt(8);
}
//Matchday class
public class MatchDay
{
private int DayNumber;
private List<Team> TeamsList1;
private List<Team> TeamsList2;
private List<int> ScoresList1;
private List<int> ScoresList2;
private DateTime Date;
public MatchDay(int dayNumber, List<Team> teamsList1, List<Team> teamsList2, DateTime date)
{
DayNumber = dayNumber;
TeamsList1 = teamsList1;
TeamsList2 = teamsList2;
Date = date;
}
}
I expect the List in TeamList1 and TeamList2 in the Matchday Class not the change after the SwitchTeams() method. Only the TeamList1 and TeamList2 that are variables should change and put in the new Matchday Object and also not change ofcourse.
Because it's the same list. In your program you have exactly two lists of teams:
List<Team> teamList1 = new List<Team>();
List<Team> teamList2 = new List<Team>();
You never create any more. When you use them to construct your MatchDay object:
new MatchDay(i, helpList1, helpList2, date)
That object now has a reference to each list. Any changes made to a list will be visible by all references to that list. They haven't been copied.
You can create new lists with the .ToList() extension method. For example:
new MatchDay(i, helpList1.ToList(), helpList2.ToList(), date)
This would create new lists, but they would of course still have references to the same objects. So you can modify one list, and it won't modify the other one. But if you modify a property on one of the objects in that list it will modify the object in the original list as well, and all for the same reason as the original problem you described.
In general, assigning a reference-type object (anything other than primitive types like int, bool, double, etc.) to a variable does not create a copy of that object, it just sets that variable as a reference to that same object.
I'm trying to build a standalone application that creates a custom report for Encompass360 without needing to put certain fields into the reporting database.
So far I have only found one way to do it, but it is extremely slow. (Much slower than a normal report within encompass when retrieving data outside of the reporting database.) It takes almost 2 minutes to pull the data for 5 loans doing this:
int count = 5;
StringList fields = new StringList();
fields.Add("Fields.317");
fields.Add("Fields.3238");
fields.Add("Fields.313");
fields.Add("Fields.319");
fields.Add("Fields.2");
// lstLoans.Items contains the string location of the loans(i.e. "My Pipeline\Dave#6")
foreach (LoanIdentity loanID in lstLoans.Items)
{
string[] loanIdentifier = loanID.ToString().Split('\\');
Loan loan = Globals.Session.Loans.Folders[loanIdentifier[0]].OpenLoan(loanIdentifier[1]);
bool fundingPlus = true; // if milestone == funding || shipping || suspended || completion;
if (!fundingPlus)
continue;
bool oneIsChecked = false;
LogMilestoneEvents msEvents = loan.Log.MilestoneEvents;
DateTime date;
MilestoneEvent ms = null; // better way to do this probably
if (checkBox4.Checked)
{
ms = msEvents.GetEventForMilestone("Completion");
if (ms.Completed)
{
oneIsChecked = true;
}
}
else if (checkBox3.Checked)
{
ms = msEvents.GetEventForMilestone("Suspended");
if (ms.Completed)
{
oneIsChecked = true;
}
}
else if (checkBox2.Checked)
{
ms = msEvents.GetEventForMilestone("Shipping");
if (ms.Completed)
{
oneIsChecked = true;
}
}
else if (checkBox1.Checked)
{
ms = msEvents.GetEventForMilestone("Funding");
if (ms.Completed)
{
oneIsChecked = true;
}
}
if (!oneIsChecked)
continue;
string LO = loan.Fields["317"].FormattedValue;
string LOid = loan.Fields["3238"].FormattedValue;
string city = loan.Fields["313"].FormattedValue;
string address = loan.Fields["319"].FormattedValue;
string loanAmount = loan.Fields["2"].FormattedValue;
if (loanAmount == "")
{
Console.WriteLine(LO);
continue;
}
int numLoans = 1;
addLoanFieldToListView(LO, numLoans, city, address, loanAmount);
if (--count == 0)
break;
}
}
I haven't been able to figure out how to use any of the pipeline methods to retrieve data outside the reporting database, but when all of the fields I am looking for are in the reporting database, it hardly takes a couple seconds to retrieve the contents of hundreds of loans using these tools:
session.Reports.SelectReportingFieldsForLoans(loanGUIDs, fields);
session.Loans.QueryPipeline(selectedDate, PipelineSortOrder.None);
session.Loans.OpenPipeline(PipelineSortOrder.None);
What would really help me is if somebody provided a simple example for retrieving data outside of the reporting database by using the encompass sdk that doesn't take longer than it ought to for retrieving the data.
Note: I am aware I can add the fields to the reporting database that aren't in it currently, so this is not the answer I am looking for.
Note #2: Encompass360 doesn't have it's own tag, if somebody knows of better tags that can be added for the subject at hand, please add them.
I use the SelectFields method on Loans to retrieve loan field data that is not in the reporting database in Encompass. It is very performant compared to opening loans up one by one but the results are returned as strings so it requires some parsing to get the values in their native types. Below is the example from the documentation for using this method.
using System;
using System.IO;
using EllieMae.Encompass.Client;
using EllieMae.Encompass.BusinessObjects;
using EllieMae.Encompass.Query;
using EllieMae.Encompass.Collections;
using EllieMae.Encompass.BusinessObjects.Loans;
class LoanReader
{
public static void Main()
{
// Open the session to the remote server
Session session = new Session();
session.Start("myserver", "mary", "maryspwd");
// Build the query criterion for all loans that were opened this year
DateFieldCriterion dateCri = new DateFieldCriterion();
dateCri.FieldName = "Loan.DateFileOpened";
dateCri.Value = DateTime.Now;
dateCri.Precision = DateFieldMatchPrecision.Year;
// Perform the query to get the IDs of the loans
LoanIdentityList ids = session.Loans.Query(dateCri);
// Create a list of the specific fields we want to print from each loan.
// In this case, we'll select the Loan Amount and Interest Rate.
StringList fieldIds = new StringList();
fieldIds.Add("2"); // Loan Amount
fieldIds.Add("3"); // Rate
// For each loan, select the desired fields
foreach (LoanIdentity id in ids)
{
// Select the field values for the current loan
StringList fieldValues = session.Loans.SelectFields(id.Guid, fieldIds);
// Print out the returned values
Console.WriteLine("Fields for loan " + id.ToString());
Console.WriteLine("Amount: " + fieldValues[0]);
Console.WriteLine("Rate: " + fieldValues[1]);
}
// End the session to gracefully disconnect from the server
session.End();
}
}
You will highly benefit from adding these fields to the reporting DB and using RDB query instead. Internally, Encompass has to open / parse files when you read fields without RDB, which is a slow process. Yet it just does a SELECT query on fields in RDB which is a very fast process. This tool will allow you quickly checking / finding which fields are in RDB so that you can create a plan for your query as well as a plan to update RDB: https://www.encompdev.com/Products/FieldExplorer
You query RDB via Session.Loans.QueryPipeline() very similarly to your use of Loan Query. Here's a good example of source code (in VB): https://www.encompdev.com/Products/AlertCounterFieldPlugin
This question already has answers here:
Remove duplicates from a List<T> in C#
(32 answers)
Closed 7 years ago.
I am currently working on a project and have hit a snag.
I'm taking in an unformatted string[] of football teams and I'm trying to filter the data and return it in a more orgainised format.
I'm fine with most of it (splitting the string to get the relevant values and sorting the format) except i have created a Team object which holds most of this data and as i loop through the original string i create a new team object each time i see a team name. Then i check to see if i've seen that object before and if i have i don't create a new object. After creating the Team or skipping that part i add the relevant info to the team object and continue.
My issue is the list i'm using to hold the final team info has many duplicates mean my check to see if the object exists or not doesn't work. The code is :
After splitting string,
List<Team> teams = new List<Team>();
for (int i = 1; i <= matches.Length - 1; i++)
{
string fullStr = matches[i];
string[] score = fullStr.Split(',');
string[] team1 = score[0].Split('!');
string team1Name = team1[0];
Team teams1 = new Team(team1Name);
if (teams.Contains(teams1) != true)
{
teams.Add(teams1);
}
string team1Score = team1[1];
int team1ScoreInt = int.Parse(team1Score);
string[] team2 = scores[1].Split('!');
string team2Name = team2[1];
Team teams2 = new Team(team2Name);
if (!teams.Contains(teams2))
{
teams.Add(teams2);
}
When i print the list i get the format i want but multiple Germanys etc. And only the score etc of that 1 game rather than them all adding to 1 Germany Team object.
Any ideas how i can stop the duplicates and maintain using only the 1 Team object every time i see that team name?
Thanks
You can implement IEqualityComparer for Team class and check equality of objects based on its values. Something like below.
using System.Collections.Generic;
public class Team{
public string Name {get;set;}
public int score {get;set;}
}
//Create some dummy data
public List<Team> lstTeam = new List<Team>{
new Team{Name="A", score=1},
new Team{Name="A", score=1},
new Team{Name="B", score=1},
new Team{Name="C", score=2},
new Team{Name="A", score=2},
new Team{Name="C", score=2}
};
List<Team> lstDistictTeams = lstTeam.Distinct<Team>(new DistinctComparer()).ToList();
foreach(Team t in lstDistictTeams)
{
Console.WriteLine("Team {0} has Score {1}",t.Name,t.score);
}
//This class provides a way to compare two objects are equal or not
public class DistinctComparer : IEqualityComparer<Team>
{
public bool Equals(Team x, Team y)
{
return (x.Name == y.Name && x.score == y.score); // Here you compare properties for equality
}
public int GetHashCode(Team obj)
{
return (obj.Name.GetHashCode() + obj.score.GetHashCode());
}
}
Here is running example : http://csharppad.com/gist/6428fc8738629a36163d
Looks like the problem is you're creating a new Team object with the name of the team for each result.
When you then compare against the list to see if it is contained, you're checking to see if there's a reference to that object in the list, and as you've just created it, there won't be, so every Team object you create will be added to the list.
You'll need to check whether the list contains a Team object with the name, rather than just check for the instance of the object.
I have some stock data that I'm trying to sort using linq, however I'm very unfamiliar with linq and cannot understand the documentation.
Right now I have a list of bars (a class I've created that holds the stock data) and it's all of the stock data for each day since 1990. Now I'm trying to group this stock data by year and month so I can turn daily stock data into monthly stock data (the resolution of the stock).
public class Stock
{
private string stockSymboll;
private string period;
private List<bar> aBar = new List<bar>();
private DateTime startingDate;
private DateTime endingDate;
enum period { DAILY, WEEKLY, MONTHLY };
private period PeriodType;
}
public class bar
{
private double open;
private double high;
private double low;
private double close;
private double volume;
private DateTime stockDate;
}
within the stock class I have a function that I'm trying to use to convert from a list of daily data to a list of monthly data, in order from most recent to least recent.
Here is what I've attempted:
stock convertPeriod(Period pt)
{
stock newStock = new Stock(stockName, startingDate, endingDate, period);
if (pt == Periode.MONTHLY)
{
List<bar> monthlyGroup = new List<bar>();
var group1 = (from b in bar group c by b.getDate().Month);
var group2 = from g in group1 group g by g.getDate().Year)
return...;
}
}
However I've discovered that you cannot sort a var. So I was thinking the best way would be to attempt a nested query in linq, however I can hardly seem to even get basic queries to work. Any help would be greatly appreciated.
I'm not 100% clear on what contract you're supposed to be setting up, but as I understand it, you want to:
Receive an IEnumerable<Bar>, and
Return an IEnumerable<Bar> such that StockDate is the first day of the year and month for the summarized stocks, all sorted descending by date.
As I understand it, Stock is more or less irrelevant to your true question here. If I'm incorrect in that, let me know and I can help you take this that one step further.
You have a good start on this in your LINQ. I'm a little confused about your use of private fields in the types, but I'm assuming those to be typos, and that your actual code uses public, probably properties.
I'll do this as a couple separate methods just to make what I'm doing more clear, but you may want to bring them together in the interest of performance. Particularly since I order the collection twice, effectively.
This method groups and sorts the data, based on the first day of their year-month set. Note that you can actually perform a grouping on an anonymous type. Sorting doesn't work on the anonymous object itself, as you noted, but it does work on its properties.
public IEnumerable<IEnumerable<Bar>> GroupIntoMonths(IEnumerable<Bar> bars)
{
return bars.GroupBy(c => new { c.StockDate.Year, c.StockDate.Month })
.OrderByDescending(c => c.Key.Year)
.ThenByDescending(c => c.Key.Month);
}
The choice is yours whether you prefer to group on an instantiated DateTime object with a date set of one, or what I've done here. I don't touch the Key property again, so I was fine effectively losing track of it after I left the method. Other implementations might drive you to make a different decision on that.
Once you've got that, it's a matter of converting an IEnumerable<Bar> into a single Bar that summarizes the whole period.
public IEnumerable<Bar> GroupIntoBars(IEnumerable<IGrouping<DateTime, Bar>> groups)
{
return groups.Select(GetBar);
}
public Bar GetBar(IEnumerable<Bar> bars)
{
Bar ret = new Bar();
Bar last = null;
int index = -1;
foreach(var v in bars.OrderBy(c => c.StartingDate))
{
index++;
if(index == 0)
{
ret.Open = v.Open;
ret.StockDate = v.StockDate;
ret.High = v.High;
ret.Low = v.Low;
}
else
{
ret.High = Math.Max(ret.High, v.High);
ret.Low= Math.Max(ret.Low, v.Low);
}
last = v;
}
if(last == null) throw new ArgumentException("Collection cannot be empty!");
ret.Close = last.Close;
return ret;
}
I think that method is pretty straight-forward, but let me know if I can clear anything up.
You can group by multiple properties at once by specifying them in an anonymous object that will be the group key:
var monthlyGroup = aBar.OrderBy(bar => bar.stockDate)
.GroupBy(bar => new { Year = bar.stockDate.Year, Month = bar.stockDate.Month })
//create bars from groups
.Select(g => new bar()
{
open = g.First().open,
high = g.Max(b => b.high),
low = g.Min(b => b.low),
close = g.Last().close,
volume = g.Average(b => b.volume),
stockDate = new DateTime(g.Key.Year, g.Key.Month, 1)
})
.ToList();
Sorry but I prefer the functions syntax of linq.
I noticed that fields in bar class are private so they will be inaccessible. However, I assume that you have properties for each field.
In that case you will have to replace field names with property names in the code above.