I'm very new to MongoDB, but have what I believed to be a very simple query.
I have a Protein object that inherits from IProtein (yes, my naming sucks)
public interface IProtein
{
int Count { get; set; }
DateTime Date { get; set; }
}
I want to return the FirstOrDefault from the collection based on a date comparison of the Date field of the protein object and Today
public IProtein GetProteinForDay(DateTime day)
{
var collection = _db.GetCollection<IProtein>(DB_COLLECTION);
var query = collection.AsQueryable<IProtein>()
.Where(p => p.Date == day.Date);
var protein = query.FirstOrDefault();
return protein;
}
Unfortunately, I've gone through so many different variations of trying to match dates using MongoDB (some using Linq, some not) that I've completely lost focus on how far I got with each one.
This is my current code, it returns the error Unable to determine the serialization information for the expression: p.Date
What is wrong with my query (yes, it probably is something very simple) and how do I actually compare dates with a MongoDB / Linq query?
Well, It's disappointing that .net DateTime doesn't work seamlessly with MongoDB driver. I believe support should be baked into driver.
Anyway you'll need to take couple of steps to make .net and MongoDB work together.
1) Decorate Date field in your interface with BsonDateTimeOptions attribute to tell MongoDB driver how to serialize .net DateTime. See BsonDateTimeOptions Documentation
Interface should looks like
public interface IProtein
{
int Count { get; set; }
[BsonDateTimeOptions(Kind = DateTimeKind.Local)]
DateTime Date { get; set; }
}
2) In your GetProteinForDay function, replace
var collection = _db.GetCollection<IProtein>(DB_COLLECTION);
var query = collection.AsQueryable<IProtein>()
.Where(p => p.Date == day.Date);
with
var collection = db.GetCollection<Protein>(DB_COLLECTION);
var query = collection.Find(p => p.Date == day);
Notice that, I have replaced interface IProtein with concrete implementation of interface, in my case Protein.
Update: Full program is attached as reference.
Source document:
{
_id: ObjectID('5964ebf315c46ab80b2c20f3),
Count: 10,
Date: '2017-07-11 00:00:00.000'
}
Test Program:
using System;
using MongoDB.Bson;
using MongoDB.Bson.Serialization.Attributes;
using MongoDB.Driver;
namespace mongoT
{
public interface IProtein
{
ObjectId Id { get; set; }
int Count { get; set; }
[BsonDateTimeOptions(Kind = DateTimeKind.Local)]
DateTime Date { get; set; }
}
public class Protein : IProtein
{
public ObjectId Id { get; set; }
public int Count { get; set; }
public DateTime Date { get; set; }
public override string ToString()
{
return $"{nameof(Id)}: {Id}, {nameof(Count)}: {Count}, {nameof(Date)}: {Date}";
}
}
class Program
{
private static string DB = "ProteinsDB";
private static string COLLECTION = "Proteins";
static void Main(string[] args)
{
var result = GetProteinForDay(DateTime.Now.Date);
Console.WriteLine(result);
}
public static IProtein GetProteinForDay(DateTime day)
{
var client = new MongoClient();
var db = client.GetDatabase(DB);
var collection = db.GetCollection<Protein>(COLLECTION);
var query = collection.Find(p => p.Date == day.Date);
var protein = query.FirstOrDefault();
return protein;
}
}
}
Related
I follow this page to find a better way to query based on a timestamp
I use the Cosmonaut library and this is the cosmos DB settings
var cosmosSettings = new CosmosStoreSettings(cosmosDbName, endpointUrl, key, settings: setting =>
{
setting.IndexingPolicy = new IndexingPolicy(
new RangeIndex(DataType.String, precision: -1),
new RangeIndex(DataType.Number, precision: -1));
});
and then I try to query base on the datetime as following
public async Task<IEnumerable<collectionNameObject>> GetAsync(GetCollection query)
{
var result = await _objectStore
.Query(new FeedOptions {PartitionKey = new PartitionKey(query.x)})
.Where(r =>
r.y == query.y
&& r.z == query.z
&& r.Timestamp.Date >= query.Date.Date)
.ToListAsync();
return result;
}
and here is what I have saved in CosmosDb
[CosmosCollection("collectionNameObject")]
public class collectionNameObject: Entity
{
[CosmosPartitionKey]
[JsonProperty("x")]
public string x{ get; set; }
[JsonProperty("z")] public string z{ get; set; }
[JsonProperty("y")] public string y{ get; set; }
[JsonProperty("timestamp")] public DateTime Timestamp { get; set; }
}
The problem is that the query result is always empty, however, if I remove the timestamp filter, I get what I expected. It is not clear what I miss, so I wonder if anyone has a better suggestion or a hint?
I guess since you are storing it as a string, it does not get reflected when you query it as a DateTime. The easiest way to do this is to implement a custom serializer & deserializer for dealing with JSON.
[JsonConverter(typeof(EpochDateTimeConverter))]
public DateTime Timestamp { get; set; }
I would recommend you to go through Working with Dates in Azure Cosmos DB and
Example
I have an object PurchaseOrder that has a List<VendorStatus> VendorStatuses, VendorStatus has an EstimatedShipDate column. I need to filter a purchase order by the latest EstimatedShipDate. How can I do this?
I've tried:
Filter<PurchaseOrder>.Range(r => r
.OnField(x => x.VendorStatuses.OrderByDescending(v => v.StatusUpdateDate).First().EstimatedShipDate)
.GreaterOrEquals(fromDate)
.LowerOrEquals(toDate));
and
var f = Filter<PurchaseOrder>.Range(r => r
.OnField("VendorStatuses.EstimatedShipDate")
.GreaterOrEquals(fromDate)
.LowerOrEquals(toDate));
and
var newFilter = Filter<PurchaseOrder>.Nested(nfd => nfd.Path(x => x.VendorStatuses.First().EstimatedShipDate))
.Filter(f2 => f2.Range(x => x.OnField(f => f.VendorStatuses.First().EstimatedShipDate)
.GreaterOrEquals(fromDate)
.LowerOrEquals(toDate))));
But none of these worked. How can I do this?
I'm not even able to get the date range working when running against elastic directly. The following query returns no results even though when using the head plugin i can see them listed under "po"
{
"query": {
"range": {
"po.vendorStatuses.estimatedShipDate": {
"gte": "2016-10-01",
"lte": "2016-11-01",
}
}
}
}
We're using Nest version 1.7.2 in our project with elastic 1.4.5.
Here's some more info on our types:
//that builds our search query, contains many methods to build up the FilterContainer
public class PurchaseOrderSearchQueryBuilder
{
protected FilterContainer Filter { get; set; }
public PurchaseOrderSearchQueryBuilder WithExpectedShipDate(DateTime fromDate, DateTime toDate)
{
var newFilter = Filter<PurchaseOrder>.
//method i need to implment
Filter &= newFilter;
return this;
}
}
//example working filter method
public PurchaseOrderSearchQueryBuilder WithCustomerId(long customerId)
{
if (customerId > 0)
Filter &= Filter && Filter<PurchaseOrder>.Term(p => p.CustomerId, customerId);
return this;
}
public class PurchaseOrder
{
...
public long CustomerId { get; set; }
public List<PoVendorStatus> VendorStatuses { get; set; }
}
public class PoVendorStatus
{
public long Id { get; set; }
...
public DateTime? EstimatedShipDate { get; set; }
public DateTime StatusUpdateDate { get; set; }
}
Using nest this will return all the parent object that have one inner object that has met the condition. If you want to query nested object as well the need to be mapped as nested and you need to use nested query.
//Fluent
client.Search<PurchaseOrder>(s=>s.Query(
q=>q.DateRange(
dr=>dr.Field(p=>p.VendorStatuses.First().StatusUpdateDate)
.GreaterThan(fromDate)
.LessThan(toDate))));
//Object
client.Search<PurchaseOrder>(new SearchRequest<PurchaseOrder>()
{
Query =new DateRangeQuery
{
//If you used all default mappings this might be camelCase
Field = "VendorStatuses.StatusUpdateDate",
GreaterThan = fromDate,
LessThan = toDate
}
});
I am aggregating data that I retrieve from multiple identical web services. The same row count and data points are returned with only a variance in the Value. The GroupBy clause I am using is not condensing any of the rows. I have the same row count before and after the GroupBy.
MyWebServiceUrls
.AsParallel()
.SelectMany(url => GetMetricItemData(url))
.GroupBy(item => new { item.DateTime, item.Group, item.Metric }, item => item.Value)
.Select(grp => new MetricItem()
{
DateTime = grp.Key.DateTime,
Group = grp.Key.Group,
Metric = grp.Key.Metric,
Value = // type = decimal?
grp.Any(mi => mi.HasValue)
? grp.Key.Metric.AggregationType == Metric.MetricAggregationTypes.Sum
? grp.Sum(mi => mi.Value)
: grp.Average(mi => mi)
: null
})
.AsEnumerable();
The syntax looks correct based on other examples I have found.
I send this data back to my database and can aggregate with the statement GROUP BY [DateTime], [Group], [Metric] and everything works great. While I can use the database to solve this issue, I would like to know how to correctly use LINQ in this instance.
What am I missing to get this LINQ expression to work?
UPDATE:
This is the relevant MetricItem and Metric class definition:
public class MetricItem
{
public string Group { get; set; }
public DateTime DateTime { get; set; }
public Metric Metric { get; set; }
public Decimal? Value { get; set; }
}
public class Metric
{
public string Code { get; set; }
public string Label { get; set; }
private List<string> SumMetrics = new List<string>(new string[] { "TPI", "TPO", "TPIO" });
public enum MetricAggregationTypes { Sum, Average };
public MetricAggregationTypes AggregationType
{
get
{
if (SumMetrics.IndexOf(this.Code) >= 0)
return MetricAggregationTypes.Sum;
else
return MetricAggregationTypes.Average;
}
}
}
You need to override Equals and GetHashCode on the Metric class. Most Linq methods use hash codes for comparison operations, so for most objects you define yourself, you need to override this class if you plan to use something like GroupBy, Union, etc.
My code like bellow:
This Main class :
public class Product {
public string Id { set; get; }
public IList<Attr> Attributes { set; get; }
}
This child class of main class :
public class Attr
{
public string Key { set; get; }
public object Value { set; get; }
}
Filter item class:
public class Filter
{
public CompareType Type { set; get; }
public string Key { set; get; }
public object Value { set; get; }
}
Linq extension fuction for querying :
public static class LINQExtension
{
public static bool isMatch(this Product prod, this List<Filter> filters)
{
foreach(Filter F in filters){
Attr attribute = prod.Attributes.Any(A => A.Key == F.Key);
switch(F.Type){
case CompareType.CONTAIN: return ((string)attribute.Value).Contains(F.Value);
case ....
default: return false;
}
}
}
}
Filtering products result: (Not working)
public ActionResult FilterProducts(string word, decimal min, decimal max){
List<Filter> Conditions = new List<Filter> {
new Filter {Key = "Price", Type = CompareType.BETWEEN, Value = new decimal[] {min, max} },
new Filter {Key = "Title", Type = CompareType.CONTAIN, Value = word }
...
};
var Result = Session.Query<Product>().Where(P => P.isMatch(Conditions)).ToList();
return View(Result);
}
When it tried to run give errors like below:
{"Could not understand expression: .Where(P => P.isMatch(value(App.Controllers.HomeController+<>c__DisplayClass2).Conditions)).ToList()"}
In general, RavenDB's linq provider implementation is not equal to Linq-to-Objects provider.
Under the hood, Raven's client API serializes linq query experssion to Lucene query, then makes a REST call to server with that query. (You can use Fiddler to see it happen)
For example, given a database named Test with Northwind sample data and the query code (and assuming you have Fiddler active)
using (var store = new DocumentStore
{
Url = "http://localhost.fiddler:8080",
DefaultDatabase = "Test"
})
{
store.Initialize();
using (var session = store.OpenSession())
{
var result = session.Query<Order>().Where(x =>
x.Company == "companies/58" && x.Freight < 30m).ToList();
}
}
you will see the following REST call to the server (after url decoding)
http://localhost:8080/databases/Test/indexes/dynamic/Orders?&query=Company:companies/58 AND Freight_Range:{* TO Dx30}&pageSize=128&SortHint-Freight_Range=Double
What you see highlighted in the url is Linq query "serialized" into Lucene query.
In your case the error that you are seeing is simply Raven's linq implementation cannot understand how to transform your custom code into Lucene query
I'm trying to use the following code to get all trace data for the last hour from Azure:
StorageCredentialsAccountAndKey storageCredentialsAccountAndKey = new StorageCredentialsAccountAndKey(accountName, key);
CloudStorageAccount csa = new CloudStorageAccount(storageCredentialsAccountAndKey, true);
TableServiceContext tableServiceContext = new TableServiceContext(csa.TableEndpoint.ToString(), csa.Credentials);
var results = tableServiceContext.CreateQuery<TableServiceEntity>("WADLogsTable").Where(
x => x.Timestamp > DateTime.UtcNow.AddHours(-1)).ToList();
However, I'm finding that no results are found when I know that there is data in the table for the last hour (I'm comparing the output to Cerebrata's Azure Diagnostics Manager).
I have two questions:
Is this the right way to query WADLogsTable? Why am I not seeing any
results?
What is the correct type to pass in as the generic
parameter? TableServiceEntity is a base class that only defines
three columns. I'd like to know if there is a type that represents a
WADLogsTable entity specifically. Do I just create a type with
properties the same as the column names?
There is no out of the box type (class) that would represent WADLogs entity. Using the base class you will only get the PartionKey, RowKey and Timestamp properties. You have to define it by yourself. Here a sample that I use:
public class WadLogEntity
: Microsoft.WindowsAzure.StorageClient.TableServiceEntity
{
public WadLogEntity()
{
PartitionKey = "a";
RowKey = string.Format("{0:10}_{1}", DateTime.MaxValue.Ticks - DateTime.Now.Ticks, Guid.NewGuid());
}
public string Role { get; set; }
public string RoleInstance { get; set; }
public int Level { get; set; }
public string Message { get; set; }
public int Pid { get; set; }
public int Tid { get; set; }
public int EventId { get; set; }
public DateTime EventDateTime
{
get
{
return new DateTime(long.Parse(this.PartitionKey.Substring(1)));
}
}
}
Also, when I was struggling with the WADLogs table, I managed to get it showing the results (for the last 24 hours) with this code:
var dtThen = DateTime.UtcNow.AddHours(-24);
var dtNow = DateTime.UtcNow;
var logs = this._wadLogs.WadLogs.Where(
wl =>
wl.Level == 2
&& String.Compare(wl.PartitionKey,"0" + dtThen.Ticks.ToString()) >=0
&& String.Compare(wl.PartitionKey, "0" + dtNow.Ticks.ToString()) < 0
).Take(200);
I noted that there is a "0" prefix in the partition key before the ticks count.
For users of the latest (2014) Azure Storage client:
http://blogs.msdn.com/b/tilovell/archive/2014/02/11/how-to-view-azure-diagnostics-traces-from-wadlogstable-in-your-local-console-app.aspx
tl;dr you can use Timestamp for filtering.
...
var query = table.CreateQuery<GenericTableEntity>()
.Where(e => e.Timestamp > DateTime.UtcNow.AddMinutes(-120));
By extending the entity in the linked example, you can expose the Message and Date variables:
public class LogEntity : GenericTableEntity
{
// Since Timestamp is a DateTimeOffset
public DateTime LogDate
{
get { return Timestamp.UtcDateTime; }
}
public string Message
{
get { return Properties["Message"].StringValue; }
}
}