Need example for Azure Table query using ExecuteQuerySegmentedAsync - c#

I'm simply trying to get all entities in an Azure Table matching a certain field name and can't seem to find any recent examples I can learn from.
Ultimately I want to display the results in an HTML table back to my view so I was thinking a List<ActivityModel> would make sense. Either that or an IEnumerable since I don't need to change any of the entities (just read). Either way I'm struggling with the basic concepts.
public async Task<List<ActivityModel>> GetActivitiesAsync(string domainName)
{
CloudTable cloudTable = TableConnection("NodeEvents");
TableQuery<ActivityModel> query = new TableQuery<ActivityModel>().Where(
TableQuery.GenerateFilterCondition("DomainName", QueryComparisons.Equal, domainName)
);
//tried many examples of continuation tokens/etc
}
The problem I'm running into is that all the latest SDK's use async calls with the use of a ContinuationToken and I can't seem to find any good examples. This seems like it should be a simple task. Any help is appreciated.
Tried to follow along here but the examples made reference to methods and operations not available anymore.

This worked for me. I wasn't able to find anything out there showing recent use of the Azure Tables SDK. Hopefully this works for others. Couple more things to note here:
The table has thousands of entities and I only wanted about 500 in total. While the code below sort of works, it needs some work on possibly the use of .Take(int). It's my understanding the "Take" method only throttles how many records are returned per query. Since I likely have a continuation token to deal with, setting it to something like 5 would only take 5 entities per request until the end of the list.
I added an index counter to break out in case we get more than 500 records however my testing is showing 2000+ records versus 5000+ without it so maybe this is a result of the async nature of the call?
public async Task<List<ActivityModel>> GetActivitiesAsync(string domainName)
{
List<ActivityModel> activities = new List<ActivityModel>();
CloudTable cloudTable = TableConnection("NodeEvents");
string filter = TableQuery.GenerateFilterCondition("DomainName", QueryComparisons.Equal, domainName);
TableContinuationToken continuationToken = null;
do
{
var result = await cloudTable.ExecuteQuerySegmentedAsync(new TableQuery<ActivityModel>().Where(filter), continuationToken);
continuationToken = result.ContinuationToken;
int index = 0;
if (result.Results != null)
{
foreach (ActivityModel entity in result.Results)
{
activities.Add(entity);
index++;
if (index == 500)
break;
}
}
} while (continuationToken != null);
return activities;
}

Related

Avoid parameter duplication in CQRS RESTful ASP.NET Core API

I'm trying to build a RESTful API using CQRS (with MediatR) in ASP.NET Core 6.
The issue I'm facing is when trying to expose child entities.
Consider a ItemProperty to have many ItemPropertyOptions.
The query I'm trying to expose is the following:
public record GetItemPropertyOptionsQuery(string PropertyReference, int PageNumber = 1, int PageSize= 10) : IRequest<PaginatedList<ItemPropertyOptionDto>>;
So in the controller my initial attempt was the following:
[HttpGet]
[Route("ItemProperties/{propertyReference}/Options")]
public async Task<ActionResult<PaginatedList<ItemPropertyOptionDto>>> GetAsync([FromQuery]GetItemPropertyOptionsQuery query, string propertyReference)
{
if (query.PropertyReference != propertyReference)
{
return BadRequest("You are querying for a different object.");
}
return Ok(await Mediator.Send(query));
}
As you can see, the main issue is that I need the property reference to process my query, but that is also part of the URL, so I have them duplicated. This means that the consumer of the API has to include them twice and I have to check for them to be equal.
The resulting call is something like:
/ItemProperties/test/Options?PropertyReference=test
This is a smell to me but I'm not sure how to solve it.
A first idea was to not expose directly the query but a different object that does not include this parameter and the create the Query myself, but I would need to create an extra record and match it to the query/command fields for every endpoint I create, and that doesn't sound very clean from a maintainability PoV since every time I change the Query to include a new parameter I need to do the same in the Request.
Or do you think it would be ok this way?
public record GetItemPropertyOptionsRequest(int PageNumber = 1, int PageSize= 10);
[HttpGet]
[Route("ItemProperties/{propertyReference}/Options")]
public async Task<ActionResult<PaginatedList<ItemPropertyOptionDto>>> GetAsync([FromQuery]GetItemPropertyOptionsRequest request, string propertyReference)
{
var query = new GetItemPropertyOptionsQuery(propertyReference, request.PageNumber, request.PageSize);
return Ok(await Mediator.Send(query));
}
What are you trying to achieve with this, return a list of
ItemPropertyOptions for a given ItemProperty?
#Peppermintology yes
In which case you do not require the additional (and superfluous second) propertyReferece parameter.
All you require is the first propertyReference parameter. That should operate as the lookup key for the relevant ItemProperty entity in your database and then all you need to do is retrieve the releated ItemPropertyOptions as a relationship.
So your URL would look as follows:
/item-properties/{propertyId}/options
Then you would use the propertyId parameter in your query handler to obtain your records and relation opptions. That might look something like:
var results = await _dbContext.PropertyItems
.Where(item => item.Id == propertyId)
.Include(item => item.Options)
.ToListAsync();

Strange Issue With GUID Search using C#

I'm trying to learn to use ABP with Blazor and MongoDB. I've got a set of objects in a database that I display as a list but I also want to be able to retrieve objects from the database by the object's GUID and this is where I'm running into problems.
This is the task I wrote to retrieve objects based on a filter from the database:
private async Task GetObjectAsync(string filter)
{
var result = await ObjectAppService.GetListAsync(
new GetObjectListDto
{
MaxResultCount = PageSize,
SkipCount = CurrentPage * PageSize,
Sorting = CurrentSorting,
Filter = filter
}
);
ObjectList = result.Items;
TotalCount = (int)result.TotalCount;
errorVal = TotalCount.ToString();
This is my GetListAsync() method where I retrieve objects from the database according to the filter string:
public async Task<PagedResultDto<ObjectDto>> GetListAsync(GetObjectListDto input)
{
if (input.Sorting.IsNullOrWhiteSpace())
{
input.Sorting = nameof(Object.Payload);
}
var objects = await _objectRepository.GetListAsync(
input.SkipCount,
input.MaxResultCount,
input.Sorting,
input.Filter
);
var totalCount = input.Filter == null
? await _objectRepository.CountAsync()
: await _objectRepository.CountAsync(
** SEARCH METHODS GO HERE **
return new PagedResultDto<ObjectDto>(
totalCount,
ObjectMapper.Map<List<Object>, List<ObjectDto>>(objects)
);
}
These are the search methods I've been trying:
object => object.Name.Contains(input.Filter));
object => object.Id.ToString().Contains(input.Filter));
object => object.Id.Equals(Guid.Parse(input.Filter)));
The first search method (search by name and return any DB results which contain it) works perfectly except for the fact I want to be able to search by GUID. The second search method would be ideal except it throws an exception when I use it:
(System.ArgumentException: Unsupported filter: {document}{_id}.ToString().Contains("70f35018-a504-89f1-51c0-3a04cb49cb97").).
The third search method definitely retrieves something from the database (I have verified) but it doesn't display this result on the page for some reason and I obviously have to enter a complete GUID to return the object (not ideal).
I'm sure this is possible and I'm doing something wrong here but I'm not sure what that is. Can anyone tell me the correct way to search by GUID here? I expected Id.ToString().Contains(Filter) to work.
UPDATE:
I've sort of solved this problem - I needed to look at the MongoObjectRepository.cs file:
public async Task<List<Object>> GetListAsync(
int skipCount,
int maxResultCount,
string sorting,
string filter = null)
{
var queryable = await GetMongoQueryableAsync();
return await queryable
.WhereIf<Object, IMongoQueryable<Object>>(
!filter.IsNullOrWhiteSpace(),
obj => obj.Id.Equals(filter)
)
.OrderBy(sorting)
.As<IMongoQueryable<Object>>()
.Skip(skipCount)
.Take(maxResultCount)
.ToListAsync();
}
For some reason, I was filtering based on the Payload property rather than Id.
I'm still confused, however, why obj => obj.Id.ToString().Contains((filter)) doesn't work - is this not allowed when retrieving items from a database? I've tried this on DateTimeOffset fields as well (e.g. obj => obj.InsertTimestamp.ToString().Contains((filter))) and get similar problems. When I try to add the .FirstOrDefault() to the end of the .WhereIf<Object, IMongoQueryable<Object>>() statement it invalidates the function (has to be chained with all the other methods - maybe that's the problem there?)

Trying to reference a query in a c# class

I'm trying to accomplish 2 things with the below snippet of code (from ApplicationDataService.lsml.cs in the server project of my Lightswitch 2013 solution).
partial void Query1_PreprocessQuery(ref IQueryable<CandidateBasic> query)
{
query = from item in query where item.CreatedBy == this.Application.User.Name select item;
}
partial void CandidateBasics_Validate(CandidateBasic entity, EntitySetValidationResultsBuilder results)
{
var newcandidateCount = this.DataWorkspace.ApplicationData.Details.GetChanges().AddedEntities.OfType<CandidateBasic>().Count();
var databasecandidateCount = this.CandidateBasics.GetQuery().Execute().Count();
const int maxcandidateCount = 1;
if (newcandidateCount + databasecandidateCount > maxcandidateCount)
{
results.AddEntityError("Error: you are only allowed to have one candidate record");
}
}
Firstly, I want to make sure each user can only see things that he has made. This, together with a preprocess query on the table in question, works perfectly.
The next bit is designed to make sure that each user can only create one record in a certain table. Unfortunately, it seems to be looking at the whole table, and not the query I made that shows only the user's own records.
How can I get that second bit of code to limit only the user's own records, and not the global table?
You're not actually calling that query though are you? Your query is called Query1 based on the code provided yet you don't seem to be calling it. I'd do something like:
int count = DataWorkspace.ApplicationData.Query1().Count();

LINQ to Entities- SaveChanges take too much time

Currently, I am struggling with an issue regarding Entity Framework (LINQ to Entities). Most of the time when I try to execute entity.SaveChanges() everything works fine but at some points entity.SaveChanges() takes too much and timesouts. I searched a lot but was unable to find out the answer.
(According to companies policy, I cannot copy code somewhere else. So, I do not have the exact code but I will try to layout the basic structure. I hope it helps you to figure out the problem but if i doesn't then let me know.)
Task:
My task is to scan the whole network for some specific files. Match content of each file with the content of database and based on the matching either insert or update the database with the content of the file. I have around 3000 files on the network.
Problem:
public void PerformAction()
{
DbTransaction tran = null;
entity.Connection.Open(); //entity is a global variable declared like myDatabaseEntity entity = new myDatabaseEntity();
tran = entity.Connection.BeginTransaction();
foreach(string path in listOfPaths)
{
//returns 1 - Multiple matching in database OR
// 2 - One matching file in database OR
// 3 - No Matching found.
int returnValue = SearchDatabase();
if(returnValue == 1)
DoSomething(); //All inserts/updates work perfectly. Save changes also works correctly.
else if(returnValue == 2)
DoSomething(); //Again, everything ok. SaveChanges works perfectly here.
else
{
//This function uses some XML file to generate all the queries dynamically
//Forexample INSERT INTO TABLEA(1,2,3);
GenerateInsertQueriesFromXML();
ExecuteQueries();
SaveChanges(); <---- Problem here. Sometimes take too much time.
}
//Transaction commit/rollback code here
}
}
public bool ExecuteQueries()
{
int result = 0;
foreach(string query in listOfInsertQueries)
{
result = entity.ExecuteStoreCommand(query); //Execute the insert queries
if(result <=0)
return false;
}
entity.TestEntityA a = new entity.TestEntityA();
a.PropertyA = 123;
a.PropertyB = 345;
//I have around 25 properties here
entity.AddToTestEntityA(a);
return true;
}
Found the issue.
The main table where i was inserting all the data had a trigger on INSERT and DELETE.
So, whenever i inserted some new data in the main table, the trigger was firing in the backend and was taking all the time.
Entity framework is FAST and INNOCENT :D

Castle ActiveRecord - 2nd level cache - need explanation of this behaviour

I am just doing some experiments on Castle AR and 2nd level cache of NH. In the following two methods, I can see caching working fine but only for the repetition of the call of each. In other words if I call RetrieveByPrimaryKey twice for same PK, the object is found in cache. And if I call RetrieveAll twice, I see SQL issued only once.
But if I call RetrieveAll and then RetrieveByPrimaryKey with some PK, I see two SQL statements getting issued. My question is, Why AR does not look for that entity in cache first? Sure it would have found it there as a result of previous call to RetrieveAll.
public static T RetrieveByPrimaryKey(Guid id)
{
var res = default(T);
var findCriteria = DetachedCriteria.For<T>().SetCacheable(true);
var eqExpression = NHibernate.Criterion.Expression.Eq("Id", id);
findCriteria.Add(eqExpression);
var items = FindAll(findCriteria);
if (items != null && items.Length > 0)
res = items[0];
return res;
}
public static T[] RetrieveAll()
{
var findCriteria = DetachedCriteria.For<T>().SetCacheable(true);
var res = FindAll(findCriteria);
return res;
}
You're using caching on specific queries. that means that cache lookup is done in the following way:
search the cahce for results of a query with identical syntax AND the same parameters. If found- use cached results.
nHibernate (this has nothing to do with AR, by the way) doesn't know that logically, one query 'contains' the other. so this is why you're getting 2 db trips.
I would suggest using ISession.Get to retreive items by ID (it's the recommended method). I think (not tested it though) that Get can use items cached by other queries.
here's a nice blog post from ayende about it.

Categories

Resources