I am writing a weather app and need to go through 2 nested loops. For the return value I want to iterate over the first list, looking at the corresponding second list data. When the data in the second list matches the bool, I need to get data from the corresponding first list. Now I think that my code works... but would like to ask if this is a good way to do this. I am also not sure if this LINQ query will work in general, with even more nested lists. Here's my approach in LINQ:
public static async Task<string> UpdateWeather(string lat, string lon)
{
WeatherObject weather = await WeatherAPI.GetWeatherAsync(lat, lon);
var first = (from l in weather.list
from w in l.weather
where w.id == 800
select l.city.name).First();
return first;
}
Your code is OK, it is a LINQ query.But one more thing. Use FirstOrDefault() instead of First(). First() will throw an exception if no matched element is found, but FirstOrDefault() will return the element or the default value.
You can also write in LINQ Method syntax if you prefer this.
public static async Task<string> UpdateWeather(string lat, string lon)
{
WeatherObject weather = await WeatherAPI.GetWeatherAsync(lat, lon);
var first = weather.list.Where(l => l.weather.Any(w => w.id == 800))
.Select(l => l.city.name)
.FirstOrDefault();
return first;
}
I believe your query should work, and it should generally work with more nested lists following a similar structure. As to if it is a good way to do this - it depends on the data structure and any data constraints.
For example, if two elements in weather.list have an element in their nested weather list might have the same id, then your code will only return the first one - which may not be correct.
e.g. in json:
[
{
city : {
name : "Chicago"
},
weather : [
{
id = 799
},
{
id = 800
}
]
},
{
city : {
name : "New York"
},
weather : [
{
id = 800
},
{
id = 801
}
]
}
}
For this dataset, your code will return "Chicago", but "New York" also matches. This may not be possible with the data API you are accessing, but given that there are no data constraints to ensure exclusivity of the nested lists, you might want to defensively check that there is only 0 or 1 elements in the returned list that match the expected criteria.
Another suggestion
On another note, not strictly an answer to your question - if you think your code will work but aren't sure, write a unit test. In this case, you'd wrap the call to WeatherAPI in a class that implements an interface you define. Update your method to call the method on a reference to the interface.
For your real application, ensure that an instance of the wrapper/proxy class is set on the reference.
For the unit test, use a framework like Moq to create a mock implementation of the interface that returns a known set of data and use that instead. You can then define a suite of unit tests that use mocks that return different data structures and ensure your code works under all expected structures.
This will be a lot easier if your class is not a static method as well, and if you can use dependency injection (Ninject, Autofac or one of many others...) to manage injecting the appropriate implementation of the service.
Further explanations of unit testing, dependency injection and mocking will take more than I can write in this answer, but I recommend reading up on it - you'll never find yourself thinking "I think this code works" again!
Related
I am using HotChocolate library to work with GraphQL via .NET. I already can get all objects, that are stored in db, using this query:
query
{
news
{
title
description
}
}
But I need to have an opportunity, to select object with specific id like in this query:
query
{
news(id: 5)
{
title
description
}
}
But I'm getting the following exception
Unknown argument "id" on field "Query.news".
I use this code to get all news from database and return it to a client:
[UseDbContext(typeof(Context.Context))]
[UseFiltering]
[UseSorting]
public IQueryable<Entities.News> GetNews([ScopedService] Context.Context context)
{
return context.News;
}
Also I tried to get an object by id using this code:
[UseDbContext(typeof(Context.Context))]
[UseFiltering]
[UseSorting]
public Entities.News GetNews(int id, [ScopedService] Context.Context context)
{
return context.News.Find(id);
}
But I іtill have the exception when trying to get it by id
With the UseFiltering attribute, this exposes a new schema "descriptor" of WHERE and you can then use this in your Graph QL query
query ($searchId: Int!)
{
news(where: {id: {eq: $searchId}})
{
title
description
}
}
variables {
"searchId" : 1
}
The short answer: no, you cannot proceed that way.
The explanation: the attribute [UseFiltering] can be applied to something that presents a collection of items only. In the second method with 'id', you are trying to return the single entity along with [UseFiltering]. It's not possible.
So, in total, you have the following options (which all do not conform to your needs exactly):
You follow the way in that answer - as for me, it is the most logically correct approach.
You can add the parameter 'id' to the method GetNews but still return IQueryable containing the single element or nothing - as for me, the most non-intuitive approach because the selection by single id is logically incompatible with filtering by WHERE, results ordering and pagination.
Create a separate method GetUserById returning a single element - as for me, the correct approach which is often used along with p.1.
consider the following scenario:
public class DBEntry {
public string Id;
}
public class ComputedEntry {
public string Id;
public int ComputedIndex;
}
IQueryable<DBEntry> databaseQueryable; // Somewhere hidden behind the API
IQueryable<ComputedEntry> entryQueryable; // Usable with the API
Let's assume each DBEntry has a unique Id and not much else. A ComputedEntry has a 1:n relationship with DBEntry, meaning that a DBEntrycan be expanded into more than a single ComputedEntryduring execution of the query.
Now, I am trying to query entryQueryable to get a range of computed indices, e.g:
entryQueryable.Where(dto => dto.ComputedIndex < 10 && dto.Id == "some-id");
What I'm looking for is a way of separating the given query expression to only push down the relevant parts of the query to the databaseQueryable. In the example above something like this should happen (probably in the implementation of IQueryableProvider.Execute when using the entryQueryable):
var results = databaseQueryable.Where(e => e.Id == "some-id").ToList();
int i = 0;
return results.Select(e => new ComputedEntry(e.Id, i++));
So basically I'd like the query to be separated and the relevant/compatible parts should be pushed down to the databaseQueryable.
The obvious question would be: How should I approach this? I tried to figure out a way of separating the expression with an ExpressionVisitor, but haven't been very successful here and it seems like this is a rather complex task.
Any ideas? Maybe there is an already existing method of optimizing/translating the query I am not aware of? I have looked through the documentation but couldn't find anything useful here.
Many thanks for your suggestions!
Please see the following post for some background: MongoDB C# Driver - Return last modified rows only
After almost two years of running this code, we've been experiencing performance problems lately and as much as I keep on saying that the code is not the issue, Infrastructure are insisting it's because I'm doing full table scans.
The thing is that the problem is environment specific. Our QA environment runs like a dream all the time but Dev and Prod are very slow at times and fine at other - it's very erratic. They have the same data and code on but Dev and Prod have another app that is also running on the database.
My data has an Id as well as an _id (or AuditId) - I group the data by Id and then return the last _id for that record where it was not deleted. We have multiple historic records for the same ID and I would like to return the last one (see original post).
So I have the following method:
private static FilterDefinition<T> ForLastAuditIds<T>(IMongoCollection<T> collection) where T : Auditable, IMongoAuditable
{
var pipeline = new[] { new BsonDocument { { "$group", new BsonDocument { { "_id", "$Id" }, { "LastAuditId", new BsonDocument { { "$max", "$_id" } } } } } } };
var lastAuditIds = collection.Aggregate<Audit>(pipeline).ToListAsync().Result.ToList().Select(_ => _.LastAuditId);
var forLastAuditIds = Builders<T>.Filter.Where(_ => lastAuditIds.Contains(_.AuditId) && _.Status != "DELETE");
return forLastAuditIds;
}
This method is called by the one below, which accepts an Expression that it appends to the FilterDefinition created by ForLastAuditIds.
protected List<T> GetLatest<T>(IMongoCollection<T> collection,
Expression<Func<T, bool>> filter, ProjectionDefinition<T, T> projection = null,
bool disableRoleCheck = false) where T : Auditable, IMongoAuditable
{
var forLastAuditIds = ForLastAuditIds(collection);
var limitedList = (
projection != null
? collection.Find(forLastAuditIds & filter, new FindOptions()).Project(projection)
: collection.Find(forLastAuditIds & filter, new FindOptions())
).ToListAsync().Result.ToList();
return limitedList;
}
Now, all of this works really well and is re-used by all of my code that calls Collections, but this specific collection is a lot bigger than the others and we are getting slowdowns just on that one.
My question is: Is there a way for me to take the aggregate and Filter Builder and combine them to return a single FilterDefinition that I could use without running the full table scan first?
I really hope I am making sense.
Assuming I fully understand what you want, this should be as easy as this:
First, put a descending index on the LastAuditId field:
db.collection.createIndex{ "LastAuditId": -1 /* for sorting */ }
Or even extend the index to cover for other fields that you have in your filter:
db.collection.createIndex{ "Status": 1, "LastAuditId": -1 /* for sorting */ }
Make sure, however, that you understand how indexes can/cannot support certain queries. And always use explain() to see what's really going on.
The next step is to realize that you must always filter as much as possible as the very first step to reduce the amount of sorting required.
So, if you need to e.g. filter by Name then by all means do it as the very first step if your business requirements permit it. Be careful, however, that filtering at the start changes your semantics in the sense that you will get the last modified documents per each Id that passed the preceeding $match stage as opposed to the last documents per each Id that happen to also pass the following $match stage.
Anyway, most importantly, once you've got a sorted set, you can easily and quickly get the latest full document by using $group with $first which - with the right index in place - will not do a collection scan anymore (it'll be an index scan for now and hence way faster).
Finally, you want to run the equivalent of the following MongoDB query through C# leveraging the $$ROOT variable in order to avoid a second query (I can put the required code together for you once you post your Audit, Auditable and IMongoAuditable types as well as any potential serializers/conventions):
db.getCollection('collection').aggregate({
$match: {
/* some criteria that you currently get in the "Expression<Func<BsonDocument, bool>> filter" */
}
}, {
$sort: {
"ModifiedDate": -1 // this will use the index!
}
}, {
$group: {
"_id": "$Id",
"document": { $first: "$$ROOT" } // no need to do a separate subsequent query or a $max/$min across the entire group because we're sorted!
}
}, {
$match: { // some additional filtering depending on your needs
"document.Status": { $ne: "Delete" }
}
})
Lastly, kindly note that it might be a good idea to move to the latest version of MongoDB because they are currently putting a lot of effort into optimizing aggregation cases like yours, e.g. this one: https://jira.mongodb.org/browse/SERVER-9507
I have a mongo database with documents that look like this:
{
PublishedDate: [date],
PublishedDateOverride: [NullableDate],
...
}
The reason I have the override as a separate field is that it is important to know the original published date as well as the overridden one.
When I get these documents back I want to sort them by their "apparent" published date. That is if there is an override it should use that, otherwise use the original.
Our current system just sorts by PublishedDateOverride and then by PublishedDate which of course groups all of those with a null override together.
For a concrete example take the following four documents:
A = {
PublishedDate: 2014-03-14,
PublishedDateOverride: 2014-03-24,
...
}
B = {
PublishedDate: 2014-01-21,
PublishedDateOverride: 2014-02-02,
...
}
C = {
PublishedDate: 2014-03-01,
PublishedDateOverride: null,
...
}
D = {
PublishedDate: 2014-03-27,
PublishedDateOverride: null,
...
}
The desired sort order would be D (2014-03-27), A (2014-03-14), C (2014-03-01), B (2014-02-02).
I need to be able to do this in the database since I am also paging this data so I can't just sort after getting it out of the database.
So the question:
What is the best way to achieve this goal? Is there a way to sort by an expression? Is there a way to have a calculated field such that whenever I update a document it will put the appropriate date in there to sort on?
I'm doing this in C# in case that is relevant but I would assume any solution would be a mongo one, not in my client code.
If you want a projection of only the valid and greater date then use aggregate with the $cond operator and the $gt operator. A basic shell example for translation (which is not hard) :
db.collection.aggregate([
{ "$project": {
"date": { "$cond": [
{ "$gt": [
"$PublishedDate",
"$PublishedDateOverride"
]},
"$PublishedDate",
"$PublishedDateOverride"
]}
}},
{ "$sort": { "date": 1 } }
])
So that basically breaks down your documents to having the "date" field set to which ever of those two fields had the greater value. Then you can sort on the result. All processed server side.
try
datesCollection.OrderBy(d => d.PublishedDateOverride!= null? d.PublishedDateOverride: d.PublishedDate)
Use Indexes to Sort Query Results
To sort on multiple fields, create a compound index.
[The question is similar to this: What is the difference between passing It.IsAny<int>() and the value of It.IsAny<int>() to a method setup - needing additional clarification]
Short version
(More definitions below)
I am in a situation where I need to setup mocks in a factory. Depending on the value, I want to choose between using it or It.IsAny<TTypeOfValue>().
This is what I (naively) wanted to do:
moq.Setup(() => mockWebService.WebServiceMethod(
webServiceMethodObject.Some ?? It.IsAny<string>(), // that fails
...,
out webServiceMethodObject.That,
...)).Returns(webServiceMethodObject.ReturnEnum);
I have such large parameterlists (old legacy webservice methods) and so much different combination of values that I don't want to write it by hand, nor using It.IsAny<T>() everywhere, as I want to control the return value depending on parameters.
A possible alternative should be the version, where I could match concrete values with concrete return types, and when no concrete values can be matched, it falls back to the broadest version (with all parameters substituted with It.IsAny<T>(). (Short example: in a login test I'd like to test different return values of different input parameters of a login method. In all other tests I'd just like to return LoginSuccess).
How can I achieve this? Only with some Expression/Reflection magic?
Longer explanation
This is how our typical legacy webservice looks like:
ReturnValueEnum WebServiceMethod(string some, int thing, ..., out int that, out byte[] those, ...) { ... }
We had so many webservice method calls needed and they are so bloated with parameters, that we had to come up with encapsulating them in objects.
Example:
public class WebServiceMethodObject
{
public string Some { get; set; }
public int Thing { get; set; }
...
public ReturnValue ReturnEnum { get; set; }
}
The strategy should be this: we are creating a default version of this object. In the tests we fill up values that needed to be matched. We would like to pass the object to a to-be-written method, which sets up the mock accordingly:
if a property is set: use that value
else: use It.IsAny<T>() (see the Setup above!).
I would've thought that if
webServiceMethodObject.Some ?? It.IsAny<string>()
is what you want but doesn't work, the simple substitution would be
It.Is<string>(v =>
webServiceMethodObject.Some == null ||
webServiceMethodObject.Some == v)
If the logic for the parameters is so complex you can safely use It.IsAny<T> for all parameters and define a custom delegate to process the method call:
moq
.Setup(() => WebServiceMethod(It.IsAny<string>(), It.IsAny<int>(), ...))
.Returns((some, thing, ...) => {
if (some == webServiceMethodObject.Some || webServiceMethodObject.Some == null) ...
{
return webServiceMethodObject.ReturnEnum;
}
});
If you look into the list of Returns overloads you see that there is a wealth of options, the "hardcoded" .Returns(some value) being only one of those.
As far as I know the the whole thing passed to setup is an expression tree. Moq then inspects the expression subtrees for the parameters. If they are It based expressions they are used for advanced logic. If not they are evaluated and the value is matched with the input. This is probably the/one reason why your naïve version is not working.