Develop Reference

NEST - IndexMany doesn't index my objects

I've used NEST for elasticsearch for a while now and up until now I've used the regular ElasticSearchClient.Index(...) function, but now I want to index many items in a bulk operation.
I found the IndexMany(...) function, but I must do something wrong because nothing is added to the elastic search database as it does with the regular Index(...) function?
Does anyone have any idea?
Thanks in advance!

I found the problem. I had to specifiy the index name in the call to IndexMany
var res = ElasticClient.CreateIndex("pages", i => i.Mappings(m => m.Map<ESPageViewModel>(mm => mm.AutoMap())));
var page = new ESPageViewModel
{
Id = dbPage.Id,
PageId = dbPage.PageId,
Name = dbPage.Name,
Options = pageTags,
CustomerCategoryId = saveTagOptions.CustomerCategoryId,
Link = dbPage.Link,
Price = dbPage.Price
};
var pages = new List<ESPageViewModel>() { page };
var res2 = ElasticClient.IndexManyAsync<ESPageViewModel>(pages, "pages");
This works as expected. Guess I could specify a default index name in the configuration to avoid specifying the index for the IndexMany call.

If you are using C# you should create a list of objects that you want to insert then call the IndexMany function.
Example :
List<Business> businessList = new List<Business>();
#region Fill the business list
...............................
#endregion
if (businessList.Count == 1000) // the size of the bulk.
{
EsClient.IndexMany<Business>(businessList, IndexName);
businessList.Clear();
}
And in the end check again
if (businessList.Count > 0)
{
EsClient.IndexMany<Business>(businessList, IndexName);
}

LINQ - Deserialize JSON column and filter

How to deserialize/serialize a property with JSON string array value and then filter (using where clause) in LINQ inside a lambda expression?
void Main()
{
var regionList = new List<Row>() {
new Row { RegionJsonList = "[\"QLD\",\"NSW\"]" },
new Row { RegionJsonList = "[\"TAZ\",\"SA\"]" },
new Row { RegionJsonList = "[\"QLD\",\"VIC\"]" }
};
var filterRegionList = new List<string>() {
"QLD", "NSW"
};
var queryable = regionList.AsQueryable();
// this is obviously wrong, i just want to find the Row that contains one on filterRegionList
var result = queryable.Where(r => JsonConvert.DeserializeObject<string[]>(r.RegionJsonList).Contains(filterRegionList));
result.Count().Dump(); // should be 2
}
class Row
{
public string RegionJsonList { get;set; }
}

Following would work:
var result =
filterRegionList.Aggregate(regionList,(current,filter) =>
current.Where( r => r.RegionJsonList.Contains(filter)).ToList())
Aggregating the filterRegionList and regionList and thus applying filters for the final result. I did not find a requirement to Deserialize the RegionJsonList, since this would work as is, but you may add that part in case you are keen.
Also we are applying And filter via aggregation, it checks for the rows which contains both the filters, and thus provide the result, you may modify filter to achieve more number of rows, like following will select two entries from original regionList
var filterRegionList = new List<string>() { "QLD" };

To filter for rows that contain at least one of the entries from filterRegionList, you can use Enumerable.Intersect and check for non-empty intersections:
var resultAny = queryable.Where(r => JsonConvert.DeserializeObject<string[]>(r.RegionJsonList).Intersect(filterRegionList).Any());
To filter for rows that contain all of the entries from filterRegionList, you can use Enumerable.Except to remove the row's entries from the filter list. If everything gets removed, it's a match:
var resultAll = queryable.Where(r => !filterRegionList.Except(JsonConvert.DeserializeObject<string[]>(r.RegionJsonList)).Any());
(It wasn't entirely clear from your question which you wanted.)

Lambda SQL Query / Manipulate String At Query Or After Result

I'm using C#, EF5, and Lambda style queries against SQL.
I have the usual scenario of binding data to gridviews. Some of the results for my columns may be too long (character count) and so I only want to display the first 'n' characters. Let's say 10 characters for this example. When I truncate a result, I'd like to indicate this by appending "...". So, let's say the following last names are returned:
Mercer, Smith, Garcia-Jones
I'd like them to be returned like this:
Mercer, Smith, Garcia-Jon...
I was doing something like this:
using (var context = new iaiEntityConnection())
{
var query = context.applications.Where(c => c.id == applicationPrimaryKey);
var results = query.ToList();
foreach (var row in results)
{
if (row.employerName.Length > 10)
{
row.employerName = row.employerName.Substring(0, Math.Min(10, row.employerName.ToString().Length)) + "...";
}
if (row.jobTitle.Length > 10)
{
row.jobTitle = row.jobTitle.Substring(0, Math.Min(10, row.jobTitle.ToString().Length)) + "...";
}
}
gdvWorkHistory.DataSource = results;
gdvWorkHistory.DataBind();
However, if I change my query to select specific columns like this:
var query2 = context.applications.Select(c => new
{
c.id,
c.applicationCode,
c.applicationCategoryLong,
c.applicationType,
c.renew_certification.PGI_nameLast,
c.renew_certification.PGI_nameFirst,
c.renew_certification.PAI_homeCity,
c.renew_certification.PAI_homeState,
c.reviewStatusUser,
c.dateTimeSubmittedByUser
})
The result appears to become read-only if specific columns are selected, and I really should be selecting just the columns I need. I'm losing my ability to edit the result set.
So, I'm rethinking the entire approach. There must be away to select the first 'n' characters on select, right? Is there anyway to append the "..." if the length is > 10 on select? That seems trickier. Also, I guess I could parse through the gridview after bind and make this adjustment. Or, perhaps there is a way to maintain my ability to edit the result set when selecting specific columns?
I welcome your thoughts. Thanks!

To quote MSDN
Anonymous types provide a convenient way to encapsulate a set of read-only properties into a single object without having to explicitly define a type first.
So you would have to define a class and select into that if you want read write capability.
e.g.
public class MyClass {
public int id { get; set; }
public string applicationCode {get; set; }
// rest of property defintions.
}
var query2 = context.applications.Select(c => new MyClass {
id = c.id,
applicationCode = c.applicationCode,
// Rest of assignments
};
As to just providing 10 character limit with ... appended. I'm going to assume you mean on the applicationcategoryLog field but you can use the same logic on other fields.
var query2 = context.applications.Select(c => new
{
c.id,
c.applicationCode,
applicationCategoryLong = (c.applicationCategoryLong ?? string.Empty).Length <= 10 ?
c.applicationCategoryLong :
c.applicationCategoryLong.Substring(0,10) + "...",
c.applicationType,
c.renew_certification.PGI_nameLast,
c.renew_certification.PGI_nameFirst,
c.renew_certification.PAI_homeCity,
c.renew_certification.PAI_homeState,
c.reviewStatusUser,
c.dateTimeSubmittedByUser
})

Remove duplicate items from a List<String[]> in C#

I have an issue here a bit complex than I'm trying to resolve since some days ago. I'm using the PetaPoco ORM and didn't found any other way to do a complex query like this:
var data = new List<string[]>();
var db = new Database(connectionString);
var memberCapabilities = db.Fetch<dynamic>(Sql.Builder
.Select(#"c.column_name
,CASE WHEN c.is_only_view = 1
THEN c.is_only_view
ELSE mc.is_only_view end as is_only_view")
.From("capabilities c")
.Append("JOIN members_capabilities mc ON c.capability_id = mc.capability_id")
.Where("mc.member_id = #0", memberID)
.Where("c.table_id = #0", tableID));
var roleCapabilities = db.Fetch<dynamic>(Sql.Builder
.Select(#"c.column_name
,CASE WHEN c.is_only_view = 1
THEN c.is_only_view
ELSE rc.is_only_view end as is_only_view")
.From("capabilities c")
.Append("JOIN roles_capabilities rc ON c.capability_id = rc.capability_id")
.Append("JOIN members_roles mr ON rc.role_id = mr.role_id")
.Where("mr.member_id = #0", memberID)
.Where("c.table_id = #0", tableID));
I'm trying to get the user capabilities, but my system have actually to ways to assign an user a capability, or direct to that user or attaching the user to a role. I wanted to get this merged list using a stored procedure but I needed cursors and I thought maybe should be easier and faster doing this on the web application. So I get that two dynamics and the members capabilities have priority to the roles capabilities, so I need to check if that using loops. And I did like this:
for (int i = 0; i < roleCapabilities.Count; i++)
{
bool added = false;
for (int j = 0; j < memberCapabilities.Count; j++)
if (roleCapabilities[i].column_name == memberCapabilities[j].column_name)
{
data.Add(new string[2] { memberCapabilities[j].column_name, Convert.ToString(memberCapabilities[j].is_only_view) });
added = true;
break;
}
if (!added)
data.Add(new string[2] { roleCapabilities[i].column_name, Convert.ToString(roleCapabilities[i].is_only_view) });
}
So now the plan is delete the duplicate entries. I have try using the following methods with no results:
data = data.Distinct();
Any help? Thanks

Make sure that your object either implements System.IEquatable or overrides Object.Equals and Object.GetHashCode. In this case, it looks like you're storing the data as string[2], which won't give you the desired behavior. Create a custom object to hold the data, and do one of the 2 options listed above.

If I understand your question correctly you want to get a distinct set of arrays of strings, so if the same array exists twice, you only want one of them? The following code will return arrays one and three while two is removed as it is the same as one.
var one = new[] {"One", "Two"};
var two = new[] {"One", "Two"};
var three = new[] {"One", "Three"};
List<string[]> list = new List<string[]>(){one, two, three};
var i = list.Select(l => new {Key = String.Join("|", l), Values = l})
.GroupBy(l => l.Key)
.Select(l => l.First().Values)
.ToArray();

You might have to use ToList() after Distinct():
List<string[]> distinct = data.Distinct().ToList();

Update One List of Objects Attribute with Other List using LINQ

I have a scenario as think
class a
{
String Username;
String val;
}
List<a> lst = new List<a>();
List<a> lstnew = new List<a>();
What i required is to that in lstnew i have some updated values in val Attribute (Only in Several Objects) , what i required is to update the lst with updated values in lstnew as the Username Attribute using LINQ

You can join the two lists on UserName, and then update the Values in the first list with those in the second.
For example, given this class and lists:
public class a
{
public string UserName { get; set; }
public string Value { get; set; }
}
List<a> list = new List<a>
{
new a { UserName = "Perry", Value = "A" },
new a { UserName = "Ferb", Value = "B" },
new a { UserName = "Phineas", Value = "C" }
};
List<a> newList = new List<a>
{
new a { UserName = "Phineas", Value = "X" },
new a { UserName = "Ferb", Value = "Y" },
new a { UserName = "Candace", Value = "Z" }
};
You can join to get the elements with common UserNames:
var common = from a1 in list
join a2 in newList on a1.UserName equals a2.UserName
select new { A1 = a1, A2 = a2 };
At this point, if I understand you correctly, you want to update the elements from the original list:
foreach(var c in common)
{
c.A1.Value = c.A2.Value;
}
at which point the elements in list look like:
UserName Value
-----------------
Perry A
Ferb Y
Phineas X

It sounds like you have two lists. One of which is named lst and contains a full list of usernames and a second one named lstnew that contains a list of usernames who have had their val property updated. I suggest unioning the untouched usernames with the ones that have been updated. This represents the most LINQ-friendly solution I can think of.
var updatedList = Enumerable.Union(
lst.Where(x => !lstnew.Any(y => y.Username == x.Username)),
lstnew).ToList();

you should be able to use the .Zip() method to execute this.
lst.Zip(lstNew, (orig, new) => {
orig.Username = new.Username;
return orig;
});
the idea that you are getting each pair together, then instead of returning a new one, changing the orig.Username value and return the orig.

This should also do the trick. Zip method, propsed by Alastair Pitts assumes that both collections have the same order of elements and each element from first list has correspondent element in second list. My approach is more generic, it simply looks for corresponding element by comparing Username property. Still it assumes that for each element in lstNew there is corresponding element in lst.
lstNew.ForEach(new => lst.First(orig => orig.Username == new.Username).val = new.val);

I know this is an old question but a more elegant solution that I have developed, which is a slight improvement over the one given by #JeffOgata would be:
var newList= lst.GroupJoin(lstnew ,
i => i.UserName ,
j => j.UserName ,
(i, j) => j.FirstOrDefault()?? i );
Where lst is the original list and lstnew is the new list.
This will just replace the entire object in the first list with the corresponding object in the second list (the join) if one exists.
It is a slight improvement over the answer given by #JeffOgata
The result is the same.
If you have complex objects then iterating through each object then going through all the properties was a problem, simply replacing the old object with the new one was quicker.
This hopefully will help someone.