I'm trying to use the following code to get all trace data for the last hour from Azure:
StorageCredentialsAccountAndKey storageCredentialsAccountAndKey = new StorageCredentialsAccountAndKey(accountName, key);
CloudStorageAccount csa = new CloudStorageAccount(storageCredentialsAccountAndKey, true);
TableServiceContext tableServiceContext = new TableServiceContext(csa.TableEndpoint.ToString(), csa.Credentials);
var results = tableServiceContext.CreateQuery<TableServiceEntity>("WADLogsTable").Where(
x => x.Timestamp > DateTime.UtcNow.AddHours(-1)).ToList();
However, I'm finding that no results are found when I know that there is data in the table for the last hour (I'm comparing the output to Cerebrata's Azure Diagnostics Manager).
I have two questions:
Is this the right way to query WADLogsTable? Why am I not seeing any
results?
What is the correct type to pass in as the generic
parameter? TableServiceEntity is a base class that only defines
three columns. I'd like to know if there is a type that represents a
WADLogsTable entity specifically. Do I just create a type with
properties the same as the column names?
There is no out of the box type (class) that would represent WADLogs entity. Using the base class you will only get the PartionKey, RowKey and Timestamp properties. You have to define it by yourself. Here a sample that I use:
public class WadLogEntity
: Microsoft.WindowsAzure.StorageClient.TableServiceEntity
{
public WadLogEntity()
{
PartitionKey = "a";
RowKey = string.Format("{0:10}_{1}", DateTime.MaxValue.Ticks - DateTime.Now.Ticks, Guid.NewGuid());
}
public string Role { get; set; }
public string RoleInstance { get; set; }
public int Level { get; set; }
public string Message { get; set; }
public int Pid { get; set; }
public int Tid { get; set; }
public int EventId { get; set; }
public DateTime EventDateTime
{
get
{
return new DateTime(long.Parse(this.PartitionKey.Substring(1)));
}
}
}
Also, when I was struggling with the WADLogs table, I managed to get it showing the results (for the last 24 hours) with this code:
var dtThen = DateTime.UtcNow.AddHours(-24);
var dtNow = DateTime.UtcNow;
var logs = this._wadLogs.WadLogs.Where(
wl =>
wl.Level == 2
&& String.Compare(wl.PartitionKey,"0" + dtThen.Ticks.ToString()) >=0
&& String.Compare(wl.PartitionKey, "0" + dtNow.Ticks.ToString()) < 0
).Take(200);
I noted that there is a "0" prefix in the partition key before the ticks count.
For users of the latest (2014) Azure Storage client:
http://blogs.msdn.com/b/tilovell/archive/2014/02/11/how-to-view-azure-diagnostics-traces-from-wadlogstable-in-your-local-console-app.aspx
tl;dr you can use Timestamp for filtering.
...
var query = table.CreateQuery<GenericTableEntity>()
.Where(e => e.Timestamp > DateTime.UtcNow.AddMinutes(-120));
By extending the entity in the linked example, you can expose the Message and Date variables:
public class LogEntity : GenericTableEntity
{
// Since Timestamp is a DateTimeOffset
public DateTime LogDate
{
get { return Timestamp.UtcDateTime; }
}
public string Message
{
get { return Properties["Message"].StringValue; }
}
}
Related
I have this view
[Table("vw_Entity", Schema = "c")]
public partial class vw_Entity : BaseEntity
{
public long? PredictedEntityTypeID { get; set; }
public bool IsManuallyChanged { get; set; }
}
where BaseEntity is the class that stores only my ID and UUID.
This is my DTO return object:
public class EntityDTO
{
public long ID { get; set; }
public LookupTableDetails PredictedEntityTypeId { get; set; }
public bool IsManuallyChanged { get; set; }
}
where LookupTableDetails looks like:
public class LookupTableDetails
{
public long Id { get; set; }
public string Name { get; set; }
}
Now I have this stored procedure that does basically a PATCH. I call it using the following snippet:
var data = await _context.vw_Entity.FromSqlRaw("EXECUTE core.Update_Entity #EntityID", parameters)
.Select(x => new EntityDTO()
{
ID = x.ID,
PredictedEntityTypeId = new LookupTableDetails() { Id = x.PredictedEntityTypeId, Name = x.PredictedEntityTypeId == 1 ? "Entity1" : "Entity2" },
IsManuallyChanged = x.IsManuallyChanged
}).ToListAsync();
However, this crashes with an error
FromSqlRaw or FromSqlInterpolated was called with non-composable SQL and with a query composing over it
I'm aware what this error does, if I have a object of some other class inside my view then the stored procedure couldn't map it properly and return the error but in this case, my view is clear from obstacles of that type and all I need to do is just return the LookupTableDetails in my DTO object. The error is in
PredictedEntityTypeId = new LookupTableDetails() { Id = x.PredictedEntityTypeId, Name = x.PredictedEntityTypeId == 1 ? "Entity1" : "Entity2" }
I tried most of the solutions that the Internet offers, such as wrapping it with IgnoreFilters.., AsEnumerable() etc.
Any ideas what is the cause and how can I prevent it from happening again in the future i.e fix it? :D
Since You iterate over result from the first query, try changing to:
var data = _context.vw_Entity.FromSqlRaw("EXECUTE core.Update_Entity #EntityID", parameters)
.ToList() // force change from IQueryable to IEnumerable and resolve result
.Select(x => new EntityDTO() // You are anyway using all results, so this is ok
{
ID = x.ID,
PredictedEntityTypeId = new LookupTableDetails() { Id = x.PredictedEntityTypeId, Name = x.PredictedEntityTypeId == 1 ? "Entity1" : "Entity2" },
IsManuallyChanged = x.IsManuallyChanged
}).ToList();
I follow this page to find a better way to query based on a timestamp
I use the Cosmonaut library and this is the cosmos DB settings
var cosmosSettings = new CosmosStoreSettings(cosmosDbName, endpointUrl, key, settings: setting =>
{
setting.IndexingPolicy = new IndexingPolicy(
new RangeIndex(DataType.String, precision: -1),
new RangeIndex(DataType.Number, precision: -1));
});
and then I try to query base on the datetime as following
public async Task<IEnumerable<collectionNameObject>> GetAsync(GetCollection query)
{
var result = await _objectStore
.Query(new FeedOptions {PartitionKey = new PartitionKey(query.x)})
.Where(r =>
r.y == query.y
&& r.z == query.z
&& r.Timestamp.Date >= query.Date.Date)
.ToListAsync();
return result;
}
and here is what I have saved in CosmosDb
[CosmosCollection("collectionNameObject")]
public class collectionNameObject: Entity
{
[CosmosPartitionKey]
[JsonProperty("x")]
public string x{ get; set; }
[JsonProperty("z")] public string z{ get; set; }
[JsonProperty("y")] public string y{ get; set; }
[JsonProperty("timestamp")] public DateTime Timestamp { get; set; }
}
The problem is that the query result is always empty, however, if I remove the timestamp filter, I get what I expected. It is not clear what I miss, so I wonder if anyone has a better suggestion or a hint?
I guess since you are storing it as a string, it does not get reflected when you query it as a DateTime. The easiest way to do this is to implement a custom serializer & deserializer for dealing with JSON.
[JsonConverter(typeof(EpochDateTimeConverter))]
public DateTime Timestamp { get; set; }
I would recommend you to go through Working with Dates in Azure Cosmos DB and
Example
I'm very new to MongoDB, but have what I believed to be a very simple query.
I have a Protein object that inherits from IProtein (yes, my naming sucks)
public interface IProtein
{
int Count { get; set; }
DateTime Date { get; set; }
}
I want to return the FirstOrDefault from the collection based on a date comparison of the Date field of the protein object and Today
public IProtein GetProteinForDay(DateTime day)
{
var collection = _db.GetCollection<IProtein>(DB_COLLECTION);
var query = collection.AsQueryable<IProtein>()
.Where(p => p.Date == day.Date);
var protein = query.FirstOrDefault();
return protein;
}
Unfortunately, I've gone through so many different variations of trying to match dates using MongoDB (some using Linq, some not) that I've completely lost focus on how far I got with each one.
This is my current code, it returns the error Unable to determine the serialization information for the expression: p.Date
What is wrong with my query (yes, it probably is something very simple) and how do I actually compare dates with a MongoDB / Linq query?
Well, It's disappointing that .net DateTime doesn't work seamlessly with MongoDB driver. I believe support should be baked into driver.
Anyway you'll need to take couple of steps to make .net and MongoDB work together.
1) Decorate Date field in your interface with BsonDateTimeOptions attribute to tell MongoDB driver how to serialize .net DateTime. See BsonDateTimeOptions Documentation
Interface should looks like
public interface IProtein
{
int Count { get; set; }
[BsonDateTimeOptions(Kind = DateTimeKind.Local)]
DateTime Date { get; set; }
}
2) In your GetProteinForDay function, replace
var collection = _db.GetCollection<IProtein>(DB_COLLECTION);
var query = collection.AsQueryable<IProtein>()
.Where(p => p.Date == day.Date);
with
var collection = db.GetCollection<Protein>(DB_COLLECTION);
var query = collection.Find(p => p.Date == day);
Notice that, I have replaced interface IProtein with concrete implementation of interface, in my case Protein.
Update: Full program is attached as reference.
Source document:
{
_id: ObjectID('5964ebf315c46ab80b2c20f3),
Count: 10,
Date: '2017-07-11 00:00:00.000'
}
Test Program:
using System;
using MongoDB.Bson;
using MongoDB.Bson.Serialization.Attributes;
using MongoDB.Driver;
namespace mongoT
{
public interface IProtein
{
ObjectId Id { get; set; }
int Count { get; set; }
[BsonDateTimeOptions(Kind = DateTimeKind.Local)]
DateTime Date { get; set; }
}
public class Protein : IProtein
{
public ObjectId Id { get; set; }
public int Count { get; set; }
public DateTime Date { get; set; }
public override string ToString()
{
return $"{nameof(Id)}: {Id}, {nameof(Count)}: {Count}, {nameof(Date)}: {Date}";
}
}
class Program
{
private static string DB = "ProteinsDB";
private static string COLLECTION = "Proteins";
static void Main(string[] args)
{
var result = GetProteinForDay(DateTime.Now.Date);
Console.WriteLine(result);
}
public static IProtein GetProteinForDay(DateTime day)
{
var client = new MongoClient();
var db = client.GetDatabase(DB);
var collection = db.GetCollection<Protein>(COLLECTION);
var query = collection.Find(p => p.Date == day.Date);
var protein = query.FirstOrDefault();
return protein;
}
}
}
I'm new to RaveDB (Actually I've started learn it only yesterday). And try to implement some functionality.
Lets we have next class hierarchy:
public abstract class Transaction
{
public string Id { get; set; }
}
public class CategoryTransaction : Transaction
{
public string AccountId { get; set; }
public string Name { get; set; }
public string Category { get; set; }
public decimal Amount { get; set; }
}
public class ExchangeTransaction : Transaction
{
public string DebitAccountId { get; set; }
public string CreditAccountId { get; set; }
public decimal DebitAmount { get; set; }
public decimal CreditAmount { get; set; }
}
Everything storing excelent int db. I event add Conventions = FindTypeTagName ... for store documents as 'transactions/*' document id.
But I want to create index for select all transactions for specific Account (by field AccountId). I've create index:
public class AccountTransactionResult
{
public string AccId { get; set; }
}
public class Account_Transactions : AbstractMultiMapIndexCreationTask<AccountTransactionResult>
{
public Account_Transactions()
{
this.AddMap<CategoryTransaction>( transactions => from transaction in transactions
select new
{
AccId = transaction.AccountId
} );
this.AddMap<ExchangeTransaction>( transactions => from transaction in transactions
select new
{
AccId = transaction.DebitAccountId
} );
this.AddMap<ExchangeTransaction>( transactions => from transaction in transactions
select new
{
AccId = transaction.CreditAccountId
} );
}
}
This index works well I can't get all type of transactions from DB (Exchange and Category) as single IEnumerable<Transaction>. That is great.
But I want to use the index to return transactions only for particular account. Because ExchangeTransaction can belong to two Accounts. I want to see ExchangeTransaction for both Accounts. I can make query in Raven Studio by AccId (index field) and it works greate! But I can't create the same request in the code :(.
Can someone help me? How can I use index field AccId in C# LINQ query?
This code
var query = session.Query<Transaction, Account_Transactions>()
return all type of transaction, but I don't how to filter the transactions by index field in the DB.
Thanks in advance. And please sorry me my English, it is not my native language.
My fault, in documentation everythings is clear. I missed some useful paragraph.
The easiest way to do this is by writing a multi-map index like this
one:
public class AnimalsIndex : AbstractMultiMapIndexCreationTask
{
public AnimalsIndex()
{
AddMap<Cat>(cats => from c in cats
select new { c.Name });
AddMap<Dog>(dogs => from d in dogs
select new { d.Name });
}
}
And query it like this:
var results = session.Advanced.LuceneQuery<object>("AnimalsIndex")
.WhereEquals("Name", "Mitzy");
>
In my case I should use next construction:
var transactions = session.Advanced.LuceneQuery<Transaction>( "Account/Transactions" )
.WhereEquals( "AccId", account2.Id )
And everything is working now.
I'm trying to do something that should be real simple but I'm getting errors I don't know how to correct. Also, I don't even know if I'm doing things the "correct" way.
I have 3 entities. My first entity is called Bill and it's my main one. The other two are Repeat and Type. Bill has foreign keys (i.e. TypeId) which point to their respective primary keys. Both Type and Repeat are similar. They have their Id and description.
What I'm trying to do with EF and LINQ to Entity: Get all bills with all the properties of Bill but instead of the TypeId, I want TypeDesc.
This is an N tier Solution with 3 projects
Thus far I have my data layer I call Model, business layer I call BLL and my presentation later I call GUI.
namespace BillApp.Model
{
public class BillModel
{
public List<BillType> GetAllBills()
{
using (BillsEntities be = new BillsEntities())
{
var results = from o in be.Bills
.Include("Type")
.Include("Repeat")
select new
{
BillId = o.BillId,
Name = o.Name,
URL = o.URL,
PaymentAmount = o.PaymentAmount,
Balance = o.Balance,
DueDate = o.DueDate,
TypeDesc = o.Type.TypeDesc,
RepeatDesc = o.Repeat.RepeatDesc,
Username = o.UserName
};
return results.ToList();
}
}
}
public class BillType
{
#region Properties
public int BillId { get; set; }
public string Name { get; set; }
public string URL { get; set; }
public decimal PaymentAmount { get; set; }
public decimal Balance { get; set; }
public DateTime DueDate { get; set; }
public string TypeDesc { get; set; }
public string RepeatDesc { get; set; }
public string Username { get; set; }
#endregion
}
}
results returns an error Cannot implicitly convert type System.Linq.IQueryable<AnonymousType#1> to System.Collections.Generic.List<BillApp.Model.BillType>
Next I try and pass the object to my BLL with this code.
namespace BillApp.BLL
{
public class BillBLL
{
public List<BillType> GetAllBills()
{
BillModel b = new BillModel();
return b.GetAllBills();
}
}
}
But BillType has an error:The type or namespace name 'BillType' could not be found (are you missing a using directive or an assembly reference?)
What am I missing? What can I do better?
I think the error messages are pretty clear: results is of type IQueryable<T> and you're trying to return a List<T>. To return a list instead, you can use the ToList method. You'll also need to create instances of BillType instead of anonymous objects, so that it conforms to the List<BillType> return type. Your GetAllBills method would look like this:
public List<BillType> GetAllBills()
{
using (BillsEntities be = new BillsEntities())
{
var results = from o in be.Bills
.Include("Type")
.Include("Repeat")
select new BillType
{
BillId = o.BillId,
Name = o.Name,
URL = o.URL,
PaymentAmount = o.PaymentAmount,
Balance = o.Balance,
DueDate = o.DueDate,
TypeDesc = o.Type.TypeDesc,
RepeatDesc = o.Repeat.RepeatDesc,
Username = o.UserName
};
return results.ToList();
}
}
For the second error you're probably just missing a using directive to have access to your BillType type which is in a different namespace. You'd have this line at the top of the file:
using BillApp.Model;