I follow this page to find a better way to query based on a timestamp
I use the Cosmonaut library and this is the cosmos DB settings
var cosmosSettings = new CosmosStoreSettings(cosmosDbName, endpointUrl, key, settings: setting =>
{
setting.IndexingPolicy = new IndexingPolicy(
new RangeIndex(DataType.String, precision: -1),
new RangeIndex(DataType.Number, precision: -1));
});
and then I try to query base on the datetime as following
public async Task<IEnumerable<collectionNameObject>> GetAsync(GetCollection query)
{
var result = await _objectStore
.Query(new FeedOptions {PartitionKey = new PartitionKey(query.x)})
.Where(r =>
r.y == query.y
&& r.z == query.z
&& r.Timestamp.Date >= query.Date.Date)
.ToListAsync();
return result;
}
and here is what I have saved in CosmosDb
[CosmosCollection("collectionNameObject")]
public class collectionNameObject: Entity
{
[CosmosPartitionKey]
[JsonProperty("x")]
public string x{ get; set; }
[JsonProperty("z")] public string z{ get; set; }
[JsonProperty("y")] public string y{ get; set; }
[JsonProperty("timestamp")] public DateTime Timestamp { get; set; }
}
The problem is that the query result is always empty, however, if I remove the timestamp filter, I get what I expected. It is not clear what I miss, so I wonder if anyone has a better suggestion or a hint?
I guess since you are storing it as a string, it does not get reflected when you query it as a DateTime. The easiest way to do this is to implement a custom serializer & deserializer for dealing with JSON.
[JsonConverter(typeof(EpochDateTimeConverter))]
public DateTime Timestamp { get; set; }
I would recommend you to go through Working with Dates in Azure Cosmos DB and
Example
Related
In my Xamarin.Forms app, I am generating queries at api side and I directly want to execute it in my local db.
I have a table like this.
[Table("TblUnit")]
public class TblUnit
{
[AutoIncrement, PrimaryKey]
public int Id { get; set; }
public string Name { get; set; }
public string Description { get; set; }
public bool IsFavourite { get; set; }
public int WebId { get; set; }
public int CompanyId { get; set; }
public DateTime OpeningStockDate { get; set; }
}
When I try to add object using InsertAsync(object) method, object is inserted with proper date. But when I try to fire query, date is not added to database. After firing query when I try to fetch data from this table OpeningStockDate comes as 01/01/0001.
//this is generated at api side and I directly want to fire query in my local sqlite db
var query = "insert into [TblUnit] (Name,Description,IsFavourite,WebId,CompanyId,OpeningStockDate) values('Bag','Description',true,12,185,'2021-03-04 10:00:00')";
I tried changing date format to mm/dd/yyyy as well. But no success.
I tried setting storeDateTimeAsTicks flag to false.
static readonly Lazy<SQLiteAsyncConnection> lazyInitializer = new Lazy<SQLiteAsyncConnection>(() =>
{
return new SQLiteAsyncConnection(Constants.DatabasePath, Constants.Flags,false);
});
But then my entire app starts behaving incorrect. I can not get data from this table. It shows datetime parsing exception. Also can not add data to the table using InsertAsync method.
What I am doing wrong here ? Please tell me.
Thanks
I have a .NET Core 3.1 application with Entity Framework talking to PostgreSQL database.
I use Npgsql library 3.1.0, code-first pattern and LINQ to make queries.
So, I have a table Meetings with object like this:
public class Meeting
{
[Key]
public string Id { get; set; }
public string CreatorId { get; set; }
public List<string> Members { get; set; }
}
My query is:
var userId = "...";
using var db = new DatabaseContext();
var meetings = db.Meetings.Where(m => m.CreatorId == userId || m.Members.Contains(userId));
And it returns all records that matche first criteria: m.CreatorId == userId, but no records for the second: m.Members.Contains(userId).
This also doesn't work:
var meetings = db.Meetings.Where(m => m.Members.Contains(userId));
Returns zero records. But there are definitely matching records, because this:
var meetings = db.Meetings.ToList().Where(m => m.Members.Contains(userId));
Returns several records as expected.
Why does it happen? How can I use Contains in query like this?
Ok, I think I've figured it out myself.
According to the documentation ...Contains() query should transform to WHERE 3 = ANY(c."SomeArray") SQL operator, but there is an annotaion below:
Note that operation translation on List<> is limited at this time, but will be improved in the future. It's recommended to use an array for now.
I changed my model to:
public class Meeting
{
[Key]
public string Id { get; set; }
public string CreatorId { get; set; }
public string[] Members { get; set; }
}
and now it works.
I'm very new to MongoDB, but have what I believed to be a very simple query.
I have a Protein object that inherits from IProtein (yes, my naming sucks)
public interface IProtein
{
int Count { get; set; }
DateTime Date { get; set; }
}
I want to return the FirstOrDefault from the collection based on a date comparison of the Date field of the protein object and Today
public IProtein GetProteinForDay(DateTime day)
{
var collection = _db.GetCollection<IProtein>(DB_COLLECTION);
var query = collection.AsQueryable<IProtein>()
.Where(p => p.Date == day.Date);
var protein = query.FirstOrDefault();
return protein;
}
Unfortunately, I've gone through so many different variations of trying to match dates using MongoDB (some using Linq, some not) that I've completely lost focus on how far I got with each one.
This is my current code, it returns the error Unable to determine the serialization information for the expression: p.Date
What is wrong with my query (yes, it probably is something very simple) and how do I actually compare dates with a MongoDB / Linq query?
Well, It's disappointing that .net DateTime doesn't work seamlessly with MongoDB driver. I believe support should be baked into driver.
Anyway you'll need to take couple of steps to make .net and MongoDB work together.
1) Decorate Date field in your interface with BsonDateTimeOptions attribute to tell MongoDB driver how to serialize .net DateTime. See BsonDateTimeOptions Documentation
Interface should looks like
public interface IProtein
{
int Count { get; set; }
[BsonDateTimeOptions(Kind = DateTimeKind.Local)]
DateTime Date { get; set; }
}
2) In your GetProteinForDay function, replace
var collection = _db.GetCollection<IProtein>(DB_COLLECTION);
var query = collection.AsQueryable<IProtein>()
.Where(p => p.Date == day.Date);
with
var collection = db.GetCollection<Protein>(DB_COLLECTION);
var query = collection.Find(p => p.Date == day);
Notice that, I have replaced interface IProtein with concrete implementation of interface, in my case Protein.
Update: Full program is attached as reference.
Source document:
{
_id: ObjectID('5964ebf315c46ab80b2c20f3),
Count: 10,
Date: '2017-07-11 00:00:00.000'
}
Test Program:
using System;
using MongoDB.Bson;
using MongoDB.Bson.Serialization.Attributes;
using MongoDB.Driver;
namespace mongoT
{
public interface IProtein
{
ObjectId Id { get; set; }
int Count { get; set; }
[BsonDateTimeOptions(Kind = DateTimeKind.Local)]
DateTime Date { get; set; }
}
public class Protein : IProtein
{
public ObjectId Id { get; set; }
public int Count { get; set; }
public DateTime Date { get; set; }
public override string ToString()
{
return $"{nameof(Id)}: {Id}, {nameof(Count)}: {Count}, {nameof(Date)}: {Date}";
}
}
class Program
{
private static string DB = "ProteinsDB";
private static string COLLECTION = "Proteins";
static void Main(string[] args)
{
var result = GetProteinForDay(DateTime.Now.Date);
Console.WriteLine(result);
}
public static IProtein GetProteinForDay(DateTime day)
{
var client = new MongoClient();
var db = client.GetDatabase(DB);
var collection = db.GetCollection<Protein>(COLLECTION);
var query = collection.Find(p => p.Date == day.Date);
var protein = query.FirstOrDefault();
return protein;
}
}
}
This is the situation. I have a Domain object Product like this...
[DataContract]
public class Product : IStorableEntity
{
[DataMember]
public String Id { get; set; }
[DataMember]
public String RemoteId { get; set; }
[DataMember]
public String LanguageId { get; set; }
[DataMember]
public DateTime? CreationDate { get; set; }
[DataMember]
public DateTime? LastUpdate { get; set; }
ETC..ETC...
}
into my repository layer I have the following method.
public IEnumerable<TElement> Read()
{
var mongoCollection = _mongoDatabase.GetCollection<TElement>(_partitionName);
return mongoCollection.AsQueryable<TElement>();
}
With this method I want to expose via LINQ my repository layer without exporting information about technology.
In this way I can do this:
var _repository = new MyRepositoryFactory<Product>();
var result = _repository.Read().Where(p=>p.RemoteId == "1")
this query it takes 1 or 2 milliseconds.
instead...
var _repository = new MyRepositoryFactory<Product>();
var result = _repository.Read().Where(p=>p.RemoteId == "29000")
it takes 2800 milliseconds!!!
I have correctly created a unique index with the command
db.products.ensureIndex({"RemoteId":1, unique:true})
NB: Yes, I have also rebuilt the indexes with .reIndex() command
Here the strange thing...
Avoiding LINQ and modifying the repository method in...
public IEnumerable<TElement> Read(string remoteId)
{
var mongoCollection = _mongoDatabase.GetCollection<TElement>(_partitionName);
var query = Query<TElement>.EQ(p => p.RemoteId, remoteId);
return mongoCollection.Find(query);
}
if then I invoke the method whit the same id before..
var _repository = new MyMongoRepository<Product>();
var result = _repository.Read("29000")
it takes 1 or 2 milliseconds. WHY??
Why with the first approach do I have a performance degradation as the id increases instead with the second is not it?
Ps. Erm... really sorry for my english
As WiredPrainie stated in comments you should use IQueryable instead of IEnumerable otherwise the whole collection will be retrieved.
Read this guide carefully
http://docs.mongodb.org/ecosystem/tutorial/use-linq-queries-with-csharp-driver/
I'm trying to use the following code to get all trace data for the last hour from Azure:
StorageCredentialsAccountAndKey storageCredentialsAccountAndKey = new StorageCredentialsAccountAndKey(accountName, key);
CloudStorageAccount csa = new CloudStorageAccount(storageCredentialsAccountAndKey, true);
TableServiceContext tableServiceContext = new TableServiceContext(csa.TableEndpoint.ToString(), csa.Credentials);
var results = tableServiceContext.CreateQuery<TableServiceEntity>("WADLogsTable").Where(
x => x.Timestamp > DateTime.UtcNow.AddHours(-1)).ToList();
However, I'm finding that no results are found when I know that there is data in the table for the last hour (I'm comparing the output to Cerebrata's Azure Diagnostics Manager).
I have two questions:
Is this the right way to query WADLogsTable? Why am I not seeing any
results?
What is the correct type to pass in as the generic
parameter? TableServiceEntity is a base class that only defines
three columns. I'd like to know if there is a type that represents a
WADLogsTable entity specifically. Do I just create a type with
properties the same as the column names?
There is no out of the box type (class) that would represent WADLogs entity. Using the base class you will only get the PartionKey, RowKey and Timestamp properties. You have to define it by yourself. Here a sample that I use:
public class WadLogEntity
: Microsoft.WindowsAzure.StorageClient.TableServiceEntity
{
public WadLogEntity()
{
PartitionKey = "a";
RowKey = string.Format("{0:10}_{1}", DateTime.MaxValue.Ticks - DateTime.Now.Ticks, Guid.NewGuid());
}
public string Role { get; set; }
public string RoleInstance { get; set; }
public int Level { get; set; }
public string Message { get; set; }
public int Pid { get; set; }
public int Tid { get; set; }
public int EventId { get; set; }
public DateTime EventDateTime
{
get
{
return new DateTime(long.Parse(this.PartitionKey.Substring(1)));
}
}
}
Also, when I was struggling with the WADLogs table, I managed to get it showing the results (for the last 24 hours) with this code:
var dtThen = DateTime.UtcNow.AddHours(-24);
var dtNow = DateTime.UtcNow;
var logs = this._wadLogs.WadLogs.Where(
wl =>
wl.Level == 2
&& String.Compare(wl.PartitionKey,"0" + dtThen.Ticks.ToString()) >=0
&& String.Compare(wl.PartitionKey, "0" + dtNow.Ticks.ToString()) < 0
).Take(200);
I noted that there is a "0" prefix in the partition key before the ticks count.
For users of the latest (2014) Azure Storage client:
http://blogs.msdn.com/b/tilovell/archive/2014/02/11/how-to-view-azure-diagnostics-traces-from-wadlogstable-in-your-local-console-app.aspx
tl;dr you can use Timestamp for filtering.
...
var query = table.CreateQuery<GenericTableEntity>()
.Where(e => e.Timestamp > DateTime.UtcNow.AddMinutes(-120));
By extending the entity in the linked example, you can expose the Message and Date variables:
public class LogEntity : GenericTableEntity
{
// Since Timestamp is a DateTimeOffset
public DateTime LogDate
{
get { return Timestamp.UtcDateTime; }
}
public string Message
{
get { return Properties["Message"].StringValue; }
}
}