I am using Elasticsearch .Nest.
I need to create a search query that will return fileProperties objects but only of the latest version of each fileProperties.
Each FileId have many versions. So what I need is to do a groupBy FileId and get the whole document object with the Max version for each FileId.
This my class:
public class FileProperties
{
public string FileName { get; set; }
public string FileId { get; set; }
public long? Version { get; set; }
public string FileType { get; set; }
public Dictionary<string,string> DynamicProperties { get; set; }
}
and my code:
var searchResponse = client.Search<FileProperties>(s => s.AllIndices().Size(100)
.Query(q =>
q.Match(m => m
.Field(f => f.FileName)
.Query(fileId))
&& q.Match(m => m
.Field(f => f.Version)
.Query(version))).Aggregations(fstAgg => fstAgg
.Terms("file", f => f
.Field(z => z.FileId.Suffix("keyword"))
.ExecutionHint(TermsAggregationExecutionHint.GlobalOrdinals)
.Aggregations(sums => sums
.Max("version", son => son
.Field(f4 => f4.Version )
)
)
) ) );
But the results only show the key and the max value for that key.
what I want is the full document object if that is possible...
Any ideas?
Related
I have the following classes:
public class LogViews
{
public string DateYYMMDD { get; set; }
public string Mode { get; set; }
public int LearnViews { get; set; }
public int PracticeViews { get; set; }
public int QuizViews { get; set; }
}
public class Views2Model
{
public string DateYYMMDD { get; set; }
public int Devices { get; set; }
public int? LearnViews { get; set; }
public int? PracticeViews { get; set; }
public int? QuizViews { get; set; }
}
What I would like to do is to get a count when grouped by the date:
var ViewCount = CreateExcelFile.ListToDataTable(Views
.GroupBy(x => x.DateYYMMDD)
.OrderBy(g => g.Key)
.Select(g => new Views2Model
{
DateYYMMDD = $"20{g.Key.Substring(0, 2)}/{g.Key.Substring(2, 2)}/{g.Key.Substring(4, 2)}",
Devices = g.Count(),
LearnViews = g.LearnViews.Count(),
PracticeViews = g.PracticeViews.Count(),
QuizViews = g.QuizViews.Count(),
})
.ToList());
But I am getting errors for g.LearnViews, g.PracticeViews and g.QuizViews
GetViews.cs(36,36): Error CS1061: 'IGrouping<string, LogViews>' does not contain a definition for 'LearnViews' and no accessible extension method 'LearnViews' accepting a first argument of type 'IGrouping<string, LogViews>' could be found (are you missing a using directive or an assembly reference?) (CS1061) (Cosmos)
Can anyone give advice on what I am doing wrong?
You're treating a group as if it's a single record. In your select g represents a group. A single group object has a common Key, but it contains many "View" items. Therefore you need to Sum the individual record values for LearnViews, etc.
var ViewCount = CreateExcelFile.ListToDataTable(Views
.GroupBy(x => x.DateYYMMDD)
.OrderBy(g => g.Key)
.Select(g => new Views2Model
{
DateYYMMDD = $"20{g.Key.Substring(0, 2)}/{g.Key.Substring(2, 2)}/{g.Key.Substring(4, 2)}",
Devices = g.Count(),
LearnViews = g.Sum(gi => gi.LearnViews),
PracticeViews = g.Sum(gi => gi.PracticeViews),
QuizViews = g.Sum(gi => gi.QuizViews),
})
.ToList());
Maybe you've messed up using succesive LINQ function. g is not LogViews, but it's IEnumerable<LogViews>. As I see in your code, You used g.Count(), but there is no Count function defined in LogViews, right?
You can only do a Count() on an IEnumerable object.
LearnViews is an int.
What you want to do is : LearnViews = g.Sum(v => v.LearnViews)
I have my models as:
class Entity
{
public List<Prop> PropList { get; set; }
public List<string> Props { get; set; }
// others
}
class Prop
{
public string Name { get; set; }
public List<string> Tags { get; set; }
}
I want to copy all Prop's.Names in PropList to List<string> Props so that they are include in _source.
Is that possible with elasticsearch copy_to?
I tried the following, which didn't work,
.Mappings(des => des
.AutoMap()
.Properties(propsDes => propsDes
.Object<Prop>(objDes => objDes
.Name(propDes => propDes.PropList)
.AutoMap()
.Properties(objPropsDes => objPropsDes
.Text(s => s
.Name(e => e.Name)
.CopyTo(fieldDes => fieldDes.Field("props"))
)
)
)
)
)
i want to get latest 10 unique records whose last 12 characters should be unique.
sample data
json data
[{"timestamp":"2017-03-20T05:27:01.688Z","dataFrame":"ACnrAAAAAAAAAAA=","fcnt":165,"port":3,"rssi":-85,"snr":7,"sf_used":12,"id":1489987621688,"decrypted":true},{"timestamp":"2017-03-20T05:27:41.675Z","dataFrame":"ACntAAAAAAAAAAA=","fcnt":169,"port":3,"rssi":-85,"snr":9,"sf_used":12,"id":1489987661675,"decrypted":true},..
AGMDAQo1/wSsCPU=
AGMEAQo1/wSsCPU=
AGMFAQo1/wSsCPU=
AGMGAQo1/wSsCPU=
AGMHAQo1/wSsCPU=
ASHAAQo2FgSsBxc=
getting output like this , but it should only be one because last 12 characters are same.
AGMDAQo1/wSsCPU=,
AGMEAQo1/wSsCPU=,
AGMFAQo1/wSsCPU=
desired output
AGMDAQo1/wSsCPU=
ASHAAQo2FgSsBxc=
code
var Pirs = Newtonsoft.Json.JsonConvert.DeserializeObject<List<AssetDetail>>(responseString);
var items = Pirs.Where(a => !a.dataFrame.EndsWith("AAAAAAAAAAA="))
.GroupBy(a => a.dataFrame)
.Select(g => g.First())
.OrderByDescending(a => a.timestamp)
.Take(10);
model
public class AssetDetail
{
public long id { get; set; }
public DateTime timestamp { get; set; }
public string dataFrame { get; set; }
public long fcnt { get; set; }
public int port { get; set; }
public int rssi { get; set; }
public string snr { get; set; }
public string sf_used { get; set; }
public bool decrypted { get; set; }
}
You could use a.dataFrame.Substring(a.dataFrame.Length - 12) inside the GroupBy function to group by the AssetDetails that have the same 12 characters at the end on their dataFrame property.
var Pirs = Newtonsoft.Json.JsonConvert.DeserializeObject<List<AssetDetail>>(responseString);
var items = Pirs.Where(a => !a.dataFrame.EndsWith("AAAAAAAAAAA="))
.GroupBy(a => a.dataFrame.Substring(a.dataFrame.Length - 12))
.Select(g => g.First())
.OrderByDescending(a => a.timestamp)
.Take(10);
There is no need to use the Distinct() function if you are using GroupBy()
Use Distinct():
Newtonsoft.Json.JsonConvert.DeserializeObject<List<AssetDetail>>(responseString);
var items = Pirs.Where(a => !a.dataFrame.EndsWith("AAAAAAAAAAA="))
.GroupBy(a => a.dataFrame)
.Select(g => g.First())
.Distinct()
.OrderByDescending(a => a.timestamp)
.Take(10);
Hope it helps!
For example, I have those entities:
public class Book
{
[Key]
public string BookId { get; set; }
public List<BookPage> Pages { get; set; }
public string Text { get; set; }
}
public class BookPage
{
[Key]
public string BookPageId { get; set; }
public PageTitle PageTitle { get; set; }
public int Number { get; set; }
}
public class PageTitle
{
[Key]
public string PageTitleId { get; set; }
public string Title { get; set; }
}
How should I load all PageTitles, if I know only the BookId?
Here it is how I'm trying to do this:
using (var dbContext = new BookContext())
{
var bookPages = dbContext
.Book
.Include(x => x.Pages)
.ThenInclude(x => x.Select(y => y.PageTitle))
.SingleOrDefault(x => x.BookId == "some example id")
.Pages
.Select(x => x.PageTitle)
.ToList();
}
But the problem is, that it throws exception
ArgumentException: The properties expression 'x => {from Pages y
in x select [y].PageTitle}' is not valid. The expression should represent
a property access: 't => t.MyProperty'. When specifying multiple
properties use an anonymous type: 't => new { t.MyProperty1,
t.MyProperty2 }'. Parameter name: propertyAccessExpression
What's wrong, what exactly should I do?
Try accessing PageTitle directly in ThenInclude:
using (var dbContext = new BookContext())
{
var bookPages = dbContext
.Book
.Include(x => x.Pages)
.ThenInclude(y => y.PageTitle)
.SingleOrDefault(x => x.BookId == "some example id")
.Select(x => x.Pages)
.Select(x => x.PageTitle)
.ToList();
}
Given two persisted entities
public class Header
{
public virtual int Id { get; set; }
public virtual string Name { get; set; }
public virtual List<Detail> Details { get; set; }
}
and
public class Detail
{
public virtual int Id { get; set; }
public virtual Header MyHeader { get; set; }
public virtual string Text { get; set; }
public virtual object LotsOfPropertiesIDontNeed { get; set; }
}
I want to populate a new object
public class MiniHeader
{
public string Name { get; set; }
public Dictionary<int, string> DetailTexts { get; set; }
}
with only the name from the Header, and with a dictionary relating detail IDs to the associated texts. Note that Detail also has LotsOfPropertiesIDontNeed, which I would prefer not to pull
across the wire or even request from SQL Server.
With the code
IEnumerable<MiniHeader> mini =
ctx.Details.Include(d => d.MyHeader)
.GroupBy(d => d.MyHeader)
.Select(g => new MiniHeader()
{
Name = g.Key.Name,
DetailTexts = g.ToDictionary(d => d.Id, d => d.Text)
});
I get the expected
LINQ to Entities does not recognize the method 'System.Collections.Generic.Dictionary`2[System.Int32,System.String]
since .ToDictionary cannot execute on the database side. I can make it work like this:
IEnumerable<MiniHeader> mini =
ctx.Details.Include(d => d.MyHeader)
.GroupBy(d => d.MyHeader)
.AsEnumerable()
.Select(g => new MiniHeader()
{
Name = g.Key.Name,
DetailTexts = g.ToDictionary(d => d.Id, d => d.Text)
});
but I presume that LotsOfPropertiesIDontNeed will be requested of SQL Server and pulled across the wire.
Is there a way to make this work, without pulling the unnecessary fields?
You can project your results to an anonymous type and then apply AsEnumerable and later project to your class like:
IEnumerable<MiniHeader> mini =
ctx.Details.Include(d => d.MyHeader)
.GroupBy(d => d.MyHeader)
.Select(g => new
{
Name = g.Key.Name,
Details = g.Select(i => new { i.Id, i.Text }),
})
.AsEnumerable()
.Select(e => new MiniHeader()
{
Name = e.Name,
DetailTexts = e.Details.ToDictionary(d => d.Id, d => d.Text)
});
This will let you get only those field that you need and later you can use ToDictionary on an in-memory collection.