I need to return JUST all categories DISTINCTS (without repeat any) from a document using NEST.
In SQL it looks like this:
SELECT DISTINCT Category
FROM Log
ORDER BY Category ASC
Inside ElasticSearch I do this way:
GET log/_search
{
"size":"0",
"aggs" : {
"alias_category" : {
"terms" : { "field" : "category.keyword" }
}
}
}
How can I do that using NEST?
public ICollection<string> SelectAllCategoriesDistinct(ElasticClient client)
{
var searchResponse = client.Search<LogElasticSearch>(s => s
.Query(q => q
.Terms(t => t
.Field(f => f.Category)
)
)
);
return (ICollection<string>)searchResponse;
}
I found a way to do it. Is not a elegant way, but I found in the elastico site (https://discuss.elastic.co/t/c-nest-best-way-of-accessing-properties-of-iaggregate-object/85384/2) and I did this way:
public ICollection<string> SelectAllCategoriesDistinct(ElasticClient client)
{
var searchResponse =
client.Search<LogElasticSearch>(s => s
.Size(0)
.Aggregations(agg => agg
.Terms("categories", t => t
.Field("category.keyword")
)
)
);
var aggregation = searchResponse.Aggregations.Values;
var listOfCategories = new List<string>();
if (searchResponse.Aggregations.Values.FirstOrDefault().GetType() == typeof(BucketAggregate))
{
foreach (IBucket bucket in ((BucketAggregate)aggregation.FirstOrDefault()).Items)
{
if (bucket.GetType() == typeof(KeyedBucket<object>))
{
var valueKey = ((KeyedBucket<object>)bucket).Key;
listOfCategories.Add(valueKey.ToString());
}
}
}
return listOfCategories.OrderBy(c => c).ToList();
}
If someone knows a better way to do it, help me to improve, but that way it reaches the goal.
I would use a composite aggregation to fetch all the terms. In contrast to the terms aggregation, the composite aggregation supports pagination, so multiple requests can be made if there are a lot of terms to fetch
private static void Main()
{
var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
var settings = new ConnectionSettings(pool);
var client = new ElasticClient(settings);
var categories = new List<string>();
GetAllCategories(client, categories);
// do something with the categories
foreach(var category in categories)
Console.WriteLine(category);
}
private static void GetAllCategories(IElasticClient client, List<string> categories, CompositeKey after = null)
{
// number of terms to fetch in each request
var size = 10_000;
var response = client.Search<LogElasticSearch>(s => s
.Size(0)
.Aggregations(a => a
.Composite("categories", c => c
.After(after)
.Size(size)
.Sources(so => so
.Terms("category", t => t
.Field("category.keyword")
.Order(SortOrder.Ascending)
)
)
)
)
);
var compositeAgg = response.Aggregations.Composite("categories");
var buckets = compositeAgg.Buckets;
foreach (var bucket in buckets)
{
if (bucket.Key.TryGetValue("category", out string category))
{
categories.Add(category);
}
}
// there may be more
if (buckets.Count == size)
{
GetAllCategories(client, categories, compositeAgg.AfterKey);
}
}
Related
Is it possible to select two rows into one anonymous object DTO with two properties?
With a model like:
public class Document
{
public int Id { get; set; }
public string Text { get; set; }
// Other properties
}
I am writing a method that finds the difference between two versions of a document:
public Task<string> CompareVersions(int initialId, int finalId)
So I need to retrieve the text of exactly two Documents by Id, and I need know which was which.
Currently I am constructing a Dictionary<int, string> by doing:
var dto = await _context.Documents
.Where(doc => doc.Id == initialId
|| doc.Id == finalId)
.ToDictionaryAsync(x => x.Id, x => x.Text);
and then calling dto[initialId] to get the text. However, this feels very cumbersome. Is there any way to take the two Ids and select them into one DTO in the form
{
InitialText,
FinalText
}
You have to use SelectMany
var query =
from initial in _context.Documents
where initial.Id = initialId
from final in _context.Documents
where final.Id = finalId
select new
{
InitialText = initial.Text,
FinalText = final.Text
};
var result = await query.FirstOrDefaultAsync();
Aggregate can do it too
var dto = (await _context.Documents
.Where(doc => doc.Id == initialId || doc.Id == finalId).ToListAsync())
.Aggregate(
new { InitialText = "", FinalText = "" },
(seed, doc) => {
if(doc.Id == initialId)
seed.InitialText = doc.Text;
else
seed.FinalText = doc.Text;
}
);
I'm not sure I like it any more than I do your dictionary approach, but with an actual dto at the end rather than the dictionary:
var d = await _context.Documents
.Where(doc => doc.Id == initialId || doc.Id == finalId)
.ToDictionaryAsync(x => x.Id, x => x.Text);
var dto = new { InitialText = d[initialId], FinalText = d[finalId] };
You could also perhaps just:
var dto = new {
InitialText = await context.Documents
.FindAsync(initialId),
FinalText = await context.Documents
.FindAsync(finalId)
};
What I'm doing wrong in this method below? I created a group with linq because I need to group the list by 2 columns and for this grouping I will have a list of files.
[HttpGet]
[Route("versions-by-period")]
public IActionResult GetVersionsByPeriodId(int entityId, int periodId)
{
var versionsInvoiceBillet = db.RemittanceInvoiceBilletVersionsCompacts
.Where(x => x.LegalEntityId == entityId && x.PeriodId == periodId && x.IsCurrent && x.DownloadHash != null)
.GroupBy(x => new { x.LifePolicyNumber, x.LegalEntityGroupNumber },
i => new { i.DownloadHash, i.FileTypeEnum, i.DueDate }, (key, group) => new
{
LifePolicyNumber = key.LifePolicyNumber,
LegalEntityGroupNumber = key.LegalEntityGroupNumber,
Files = group.ToList()
});
return Ok(versionsInvoiceBillet.Select(x => new {
lifePolicyNumber = x.LifePolicyNumber,
legalEntityGroupNumber = x.LegalEntityGroupNumber,
invoiceAndBillet = x.Files.Select(f => new {
downloadHash = f.DownloadHash,
fileTypeEnum = f.FileTypeEnum,
dueDatet = f.DueDate
})
}));
}
If I try to call this method with Postman, the body comes empty. The problem is in invoiceAndBillet information that is returned, if I change to below, the body comes filled.
return Ok(versionsInvoiceBillet.Select(x => new {
lifePolicyNumber = x.LifePolicyNumber,
legalEntityGroupNumber = x.LegalEntityGroupNumber,
invoiceAndBillet = x.Files.Select
}));
If I try to debug the selection that I'm trying to return, I get this message below:
I need to convert the following query into c# in using NEST.
"aggs": {
"number_of_weeks": {
"date_histogram": {
"field": "#timestamp",
"interval": "week"
}
}
}
in Kibana the output is
I wrote the following query but it give me zero bucket while in Kibana it return many result in buckets
var query3 = EsClient.Search<doc>(q => q
.Index("SomeIndex")
.Size(0)
.Aggregations(agg => agg.DateHistogram("group_by_week", e => e.Field(p => p.timestamp) .Interval(DateInterval.Week)
)) ;
var resultquery3 = query3.Aggregations.DateHistogram("group_by_week");
in vs studio the output is
The problem is likely that
e => e.Field(p => p.timestamp)
does not serialize to the "#timestamp" field in Elasticsearch. For this to work, you would need to either map it with an attribute on the POCO
public class Doc
{
[Date(Name = "#timestamp")]
public DateTime timestamp { get; set; }
}
or map it on ConnectionSettings
var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
var settings = new ConnectionSettings(pool)
.DefaultMappingFor<Doc>(m => m
.PropertyName(e => e.timestamp, "#timestamp")
);
var client = new ElasticClient(settings);
Alternatively, you can simply pass a string to .Field(), which implicitly converts
.Field("#timestamp")
Below is my code
var dbClaimLink = this.Context.Set<ClaimLink>();
var claims = await DbSet
.Include(claim => claim.Parent)
.Include(link => link.ParentLinks)
.ToListAsync();
var newClaimLink = await dbClaimLink.ToListAsync();
var processedClaims = claims.Select(x =>
{
var claimLinks = x.ParentLinks;
if (!claimLinks.Any())
{
return x;
}
var hiddenParents = claimLinks.Select(p => claims.Find(t => t.Id == p.ClaimLinkId));
x.HiddenParents = hiddenParents;
return x;
});
foreach (var objClaim in processedClaims)
{
if (objClaim.Children == null)
objClaim.Children = new List<Claim>();
var lst = newClaimLink.Where(k=> k.ClaimLinkId == objClaim.Id).ToList();
if (lst.Any())
{
foreach (var item in lst)
{
IEnumerable<Claim> newChildren = claims.Where(p => p.Id == item.ClaimId);
objClaim.Children.Concat(newChildren);
}
}
}
it always return old children set without concatenate with new children. I want to those old and new children set concatenate in side of foreach loop
the Concat method returns a new collection with both values and does not alter the original.
Concat will return new object - result of concatination, so you need to save it somewhere: var result = objClaim.Children.Concat(newChildren);
Where is lazy operation, it does not execute in place, only after materialization (ToArray, or foreach call): claims.Where(p => p.Id == item.ClaimId).ToArray()
Given a return of type "AccountItem", I want to filter and sort to a new list of type FDKeyValue<>
I am trying to do this without looping and I thought I could do something like this:
var monthlyList = accountList.Where(x => x.RoleType == "Metric")
.OrderBy(x => x.EntityName)
.Select(new FDKeyValue<long, string>{}
{
"Field", "Field"
}
);
here is what I have working with a loop
var accountList = DBEntity.ReturnAccountListBySearch((int)this.PageLanguageType, "");
var monthlyList = accountList.Where(x => x.RoleType == "Metric").OrderBy(x => x.EntityName).ToList();
this.MonthlyAccountList = new FDKeyValue<long,string>();
foreach (var item in monthlyList)
{
this.MonthlyAccountList.Add(item.EntityID, item.EntityName);
}
This syntax must help
var monthlyList = accountList.Where(x => x.RoleType == "Metric")
.OrderBy(x => x.EntityName)
.Select(x => new FDKeyValue<long, string>
{
x.EntityID, x.EntityName
}
);