I have a method Comparer, with I compare some properties of the objects of two collections.
public IEnumerable<Product> Comparer(IEnumerable<Product> collection, IEnumerable<Product> target, string comparissonKey)
{
var count = 0;
var stopWatch = new Stopwatch();
var result = new ConcurrentBag<Product>();
var parallelOptions = new ParallelOptions { MaxDegreeOfParallelism = Environment.ProcessorCount * 2 };
Parallel.ForEach(collection, parallelOptions, obj =>
{
count++;
if (count == 60000)
{
stopwatch.Stop();
//breakpoint
var aux = stopwatch.Elapsed;
}
var comparableObj = obj;
comparableObj.IsDifferent = false;
bool hasTargetObject = false;
comparableObj.Exist = true;
Product objTarget = null;
foreach (Product p in target)
{
if (obj.Key == p.Key)
{
objTarget = p;
break;
}
}
if (objTarget != null)
{
//Do stuff
}
if (hasTargetObject) return;
if (comparableObj.IsDifferent)
{
//Do Stuff
}
});
return result.ToList();
}
If I execute this method like this, im getting almost 50 seconds to the breakpoint in aux variable breaks.
If I comment the second foreach (inside the Parallel.Foreach) it breaks in less than 1 second.
I need to find the corresponding object in the target collection using the Key, so I made the second foreach. I used LINQ where clause but I got no better results. Any suggestions to improve this method performance?
You can improve performance by using a dictionary:
public IEnumerable<Product> Comparer(IEnumerable<Product> collection, IEnumerable<Product> target, string comparissonKey)
{
var count = 0;
var stopWatch = new Stopwatch();
var result = new ConcurrentBag<Product>();
var parallelOptions = new ParallelOptions { MaxDegreeOfParallelism = Environment.ProcessorCount * 2 };
// create a dictionary for fast lookup
var targetDictionary = target.ToDictionary(p => p.Key);
Parallel.ForEach(collection, parallelOptions, obj =>
{
count++;
if (count == 60000)
{
stopwatch.Stop();
//breakpoint
var aux = stopwatch.Elapsed;
}
var comparableObj = obj;
comparableObj.IsDifferent = false;
bool hasTargetObject = false;
comparableObj.Exist = true;
Product objTarget = null;
// lookup using dictionary
if (targetDictionary.TryGetValue(obj.Key, out objTarget))
{
//Do stuff
}
if (hasTargetObject) return;
if (comparableObj.IsDifferent)
{
//Do Stuff
}
});
return result.ToList();
}
If Key is indeed a key
Then use HashSet as it has IntersetWith and is smoking fast
http://msdn.microsoft.com/en-us/library/bb359438.aspx
On your class Product you will need to overwrite GetHashCode and Equals
Use the Key for the GetHashCode
Override GetHashCode on overriding Equals
Related
I would like to ask you if there's a way by Linq to check discontinuity of multiple ranges, for example we have a class AgeRange:
public class AgeRange
{
public int firstValue {get;set;}
public int secondValue {get;set;}
}
var ageRange1 = new AgeRange(0,2); // interval [0,2]
var ageRange2 = new AgeRange(4,10); // interval [4,10]
var ageRange3 = new AgeRange(11,int.MaxValue); // interval [11,+oo[
var ageRangeList = new List<AgeRange>();
ageRangeList.Add(ageRange1);
ageRangeList.Add(ageRange2);
ageRangeList.Add(ageRange3);
in this example we have a discontinuity between first range and second range.
is there a way in Linq to check discontinuity between elements in ageRangeList ?
Thanks for you help.
Assuming firstValue always <= secondValue (for the same element), you can try to use Aggregate:
var start = ageRangeList
.OrderBy(a => a.firstValue).Dump()
.First();
var result = ageRangeList
.OrderBy(a => a.firstValue)
.Aggregate(
(hasGap: false, s: start.secondValue),
(tuple, range) =>
{
if (tuple.hasGap)
{
return tuple;
}
else
{
var max = Math.Max(tuple.s, tuple.s+1); //hacky overflow protection
if (max < range.firstValue)
{
return (true, tuple.s);
}
else
{
return (false, Math.Max(tuple.s, range.secondValue));
}
}
})
.hasGap;
The downside of such approach is that it still will need to loop through all age ranges.
If you want to find first discontinuity and use that information elsewhere
public static IEnumerable<AgeRange> FindDiscontinuity(List<AgeRange> ageRangeList) {
foreach(var ageRange in ageRangeList.Zip(ageRangeList.Skip(1), (a, b) => new {Prev = a, Current = b})) {
if(ageRange.Prev.SecondValue != ageRange.Current.FirstValue) {
yield return ageRange.Prev;
yield return ageRange.Current;
break;
}
}
}
public static void Main()
{
var ageRange1 = new AgeRange(0, 2);
var ageRange2 = new AgeRange(4, 10);
var ageRange3 = new AgeRange(11, int.MaxValue);
var ageRangeList = new List<AgeRange>();
ageRangeList.Add(ageRange1);
ageRangeList.Add(ageRange2);
ageRangeList.Add(ageRange3);
var result = FindDiscontinuity(ageRangeList);
foreach(var ageRange in result) {
Console.WriteLine("{0}, {1}", ageRange.FirstValue, ageRange.SecondValue);
}
}
You can change the function so it can return boolean value instead of data.
I have a function that gets x numbers og items from a SharePoint list. It takes the items in batches. After each batch I do something with the items, destroys everything and take the next batch for calculation. I currently consider to use events. So have an event for each batch. Is this the correct strategy or is there a better way to do it? I am thinking anonymous functions or something similar?
public static List<Item> GetAllItems(this List list, int rowLimit, List<string> fields, bool includeRoleAssignments, ILogger logger)
{
var result = new List<Item>();
var ctx = list.Context;
ListItemCollectionPosition position = null;
var camlQuery = new CamlQuery();
camlQuery.ViewXml =
#"<View Scope='RecursiveAll'>
<Query>
<OrderBy Override='TRUE'><FieldRef Name='ID'/></OrderBy>
</Query>
<ViewFields></ViewFields>" +
"<RowLimit Paged='TRUE'>" + rowLimit + "</RowLimit>" +
"</View>";
System.Net.ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12;
do
{
try
{
using (var clonedCtx = ctx.Clone(ctx.Url))
{
List listWithClonedContext = clonedCtx.Web.Lists.GetByTitle(list.Title);
clonedCtx.Load(listWithClonedContext);
clonedCtx.ExecuteQuery();
ListItemCollection listItems = null;
camlQuery.ListItemCollectionPosition = position;
listItems = listWithClonedContext.GetItems(camlQuery);
foreach (string field in fields)
{
clonedCtx.Load(listItems, includes => includes.Include(i => i[field]));
}
if (!includeRoleAssignments) {
clonedCtx.Load(listItems, item => item.ListItemCollectionPosition);
}
else {
clonedCtx.Load(listItems, item =>
item.ListItemCollectionPosition,
item => item.Include(
i => i.RoleAssignments.Include(
ra => ra.Member,
ra => ra.Member.LoginName,
ra => ra.RoleDefinitionBindings.Include(rd => rd.Description, rd => rd.Name))));
}
clonedCtx.Load(listItems, item => item.ListItemCollectionPosition);
clonedCtx.ExecuteQueryWithIncrementalRetry(3, 1, logger);
// here i want to do something with items before next loop/batch
position = listItems.ListItemCollectionPosition;
if (position != null)
{
logger.WriteTrace(string.Format("Iteration on getting items performed: {0}", position.PagingInfo), SeverityLevel.Verbose);
}
else
{
logger.WriteTrace("Getting all items finished.", SeverityLevel.Verbose);
}
logger.Flush();
}
}
catch (Exception ex)
{
logger.WriteException(ex);
}
}
while (position != null);
return result;
}
Maybe events are an option but there might also be an easier way to "stream" them out instead of returning all at once with a list. Therefore use yield and change to IEnumerable<Item>:
public static IEnumerable<Item> EnumerateItems(this List list, int rowLimit, List<string> fields, bool includeRoleAssignments, ILogger logger)
{
// ...
do
{
try
{
using (var clonedCtx = ctx.Clone(ctx.Url))
{
//...
camlQuery.ListItemCollectionPosition = position;
listItems = listWithClonedContext.GetItems(camlQuery);
// ...
foreach(Item x in listItems)
{
yield return x;
}
position = listItems.ListItemCollectionPosition;
// ...
}
while (position != null);
}
On this way you can start processing them while you are still fetching them or you could filter them, for example with Where,Skip or Take without loading all into memory first.
I have a report screen where I do group by over 200 MB of data in DB and sometimes i get timeout, sometimes runs to completion.
How can i optimize it or make it faster?
Below is the code I have :
public ActionResult CleanReport()
{
var count = _db.Surveys.AsNoTracking().Count();
var result = _db.Surveys.AsNoTracking().GroupBy(x => x.Clean).Select(group => new { Clean = group.Key, Count = group.Count() });
var items = new List<PieChartModel>();
foreach (var item in result)
{
if (item.Clean.HasValue == false)
{
continue;
}
var chartModel = new PieChartModel();
if (item.Clean == Clean.IdareEder)
{
chartModel.color = "#F7464A";
chartModel.highlight = "#FF5A5E";
}
if (item.Clean == Clean.Iyiydi)
{
chartModel.color = "#46BFBD";
chartModel.highlight = "#5AD3D1";
}
if (item.Clean == Clean.Kötüydü)
{
chartModel.color = "#642EFE";
chartModel.highlight = "#642EFE";
}
if (item.Clean == Clean.CevapYok)
{
chartModel.color = "#FFCC00";
chartModel.highlight = "#FFCC66";
}
chartModel.label = item.Clean.Value.DisplayName();
double per = (item.Count * 100.0) / count;
chartModel.value = (int)Math.Round(per);
items.Add(chartModel);
}
return Json(items, JsonRequestBehavior.AllowGet);
}
I have created a MySQL Database with a vast number of products and their cost. I utilize EF6 to wrap the database.
Based on the given input, I need to generate at random, a correct selection that meets the described criteria.
For example:
10 Items, Total Value $25
I am at a loss as how to properly go about iterating through the database to produce the required results.
What I am currently doing seems terribly inefficent:
using (var db = new Database())
{
var packageSelected = false;
var random = new Random();
var minItemId = (from d in db.products select d.id).Min();
var maxItemId = (from d in db.products select d.id).Max();
var timer = new Stopwatch();
timer.Start();
Console.WriteLine("Trying to make package...");
while (!packageSelected)
{
var currentItems = new List<int>();
for (var i = 0; i <= 9; i++)
{
var randomItem = random.Next(minItemId, maxItemId);
currentItems.Add(randomItem);
}
decimal? packageValue = 0;
currentItems.ForEach(o =>
{
var firstOrDefault = db.products.FirstOrDefault(s => s.id == o);
if (firstOrDefault != null)
{
var value = firstOrDefault.MSRP;
packageValue += value;
}
});
if (!(packageValue >= 25) || !(packageValue <= 26)) continue;
packageSelected = true;
timer.Stop();
Console.WriteLine("Took {0} seconds.", timer.Elapsed.TotalSeconds);
currentItems.ForEach(o =>
{
var firstOrDefault = db.products.FirstOrDefault(s => s.id == o);
if (firstOrDefault != null)
Console.WriteLine("Item: {0} - Price: ${1}", firstOrDefault.DESCRIPTION,
firstOrDefault.MSRP);
});
}
}
What about something like this:
public virtual TEntity GetRandom()
{
return DBSet.OrderBy(r => Guid.NewGuid()).Take(1).First();
}
public List<TEntity> Random(int amount, int maxprice)
{
var list = new List<TEntity>();
var tempPrice = 0;
for (int i = 0 ; i < amount; i++)
{
var element = GetRandom();
tempPrice += element.Price;
if (tempPrice > maxprice)
{
return list;
}
list.Add(element);
}
return list;
}
hope this helps
EDIT: If the maxprice is reached before the required amount of elements, the for-loop will stop and you won't get the full amount of elements.
I want to update record in ArrayList in c#, I want to update my status to inactive of particular record of ArrayList, here is the code below but this isn't working fine as it doesn't update status to inactive.
public Boolean changeAccStatus(int userid)
{
Boolean result = false;
for (int i = 0; i < list.Count; i++)
{
AccountDto dto = (AccountDto)list[i];
if (dto.Userid == userid) {
dto.Status = "Unactive";
list[i] = dto;
result = true;
}
}
return result;
}
Do (AccountDto)list[i].Status = "inactive";
try with below code
foreach (var temp in list)
{
var t = temp as AccountDto;
if (t != null && t.Userid.Trim().Equals(userid.Trim()))
t.Status = "inactive";
}
also think about doing re factoring to generic List< AccountDto> , this will give you the power of Linq directly
if you can able to convert list to generic list than you can do this
Func<int, AccountDto, bool> UpdateRecord= (y, x) =>
{
if( x.Userid== y){x.Status="Unactive";
return true;
};
List<AccountDto> mylist = new List<AccountDto>();
var v = from x in mylist
where UpdateRecord(userit-intvalue, x)
select x;
also you can try this
List<AccountDto> mylist = new List<AccountDto>();
list.ForEach(x => {
if( x.Userid== userid)
x.Status="Unactive";
});