Develop Reference

Validation for amount of returned rows in API [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
I have an API which takes its model from the request. It transfers that to a small stored procedure. That SP selects some record by using some of the model's parameters. It returns the matching records as a list of objects. Finally the controller returns the response.
Simplified example:
public IHttpActionResult Get([FromBody]TestModel testModel)
{
var responseList = _testService.GetResponse(testModel);
return Ok(responseList );
}
The API should not return more than 500 records at a time. If we have more than that the API should return with an error: "Your request would return more than 500 records".
What is the best solution to add validation for the number of retrieved records?
Two selects with same conditions and joins, but one of them
returns only the count(*)?
Or return all data to the application and then count it there?
Or something else?

Normally when you are doing a REST API there are too many approaches to do this.
Approach #1:
One is at the SQL level, you need to pass to your sql logic pagination parameters like
public int PageNumber { get; set; } = 1;
private int PageSize = 500;
So you can select and skip the number of elements you want. What is the advantage on that? you can add some execution plans to your SQL queries, see what indexes are missing, you can optimize your query on the SQL side mostly. DBAs are normally in charge of these approaches in big organizations.
Approach #2
You can still write your query from your backend, at least this one is simple, and it will be executed on the SQL server side, below is an extension method you can use to do it.
public static IQueryable<T> PageBy<T>(this IQueryable<T> query, int skipCount, int maxResultCount)
{
if (query == null)
{
throw new ArgumentNullException("query");
}
return query.Skip(skipCount).Take(maxResultCount);
}
Approach #3:
Bring the data into memory(in this case will be executed on the server-side, not in the SQL server) and to the pagination from there. The only thing you need to do is to call a ToList() and that will materialize your query and bring it into memory where you can use the same idea of the Approach #2.
Approach #4:
This example will apply for .Net Core but sure there is a version of the package that can do the same thing for .NetFramework.
Use OData nugget package
Install-Package Microsoft.AspNetCore.OData
Add EnableQueryAttribute and pass ODataQueryOptions<TestModel> options, Odata use some extension methods over IQueryble
[EnableQuery]
public IHttpActionResult Get(ODataQueryOptions<TestModel> options)
{
//this should return an IQueryble<T>
var responseList = _testService.GetResponse(testModel);
var ret= options.ApplyToWithDefaultTop(responseList, ignoreQueryOptions);
return Ok(ret);
}
here is my extension method to apply a default count of elements to bring
public static IQueryable ApplyToWithDefaultTop<T>(this ODataQueryOptions<T> options,
IQueryable<T> query, AllowedQueryOptions ignoreQueryOptions)
{
if (options.Top == null)
{
//this is the default
query = query.Take(500);
}
//odata extension method
return options.ApplyTo(query, ignoreQueryOptions);
}
This will do the same thing as Approach #2, but you don't need to worry about the parameters model, they are part of the Odata syntax, for example, here I am returning 40 elements of that resource and skipping the first 2 pages.
https://localhost:44335/api/weather_forecast?$top=40&$skip=2
Only adding 4 lines of code you are good to put on top of your API the Odata syntax, on your Configure method
app.UseMvc(routerBuilder =>
{
routerBuilder.EnableDependencyInjection();
//adding MaxTop from appsettings with a value of 100
//routerBuilder.Select().OrderBy().Filter().Count().MaxTop(Configuration.GetValue<int>("Odata:MaxTop"));
routerBuilder.Expand().Select().Count().OrderBy().Filter().MaxTop(Configuration.GetValue<int>("500"));
});
and in ConfigureServices add services.AddOData();
For more references see this article: OData article
Recommendation
If your API has more than 500 you should not throw an error to the client, just give him the first 500
Also, you can pass the PageSize and validate that number without going to your database and return the error in case you still want to go with this approach. Don't let the application fail when you can validate the input. You can use as well FluentValidation nugget package in order to validate your DTOs
Hope this helps

You can pass an Output to the SP. and set it if count is greater than 500 and select records else don't do anything in SP. in C# code just first check the output value first and then process accordingly.

As #viveknuna pointed out you can use an out parameter to indicate the retrieved row count.
I would add some small additions to his suggestion:
1) It was suggested to set that parameter only if the predefined threshold is exceeded. I would suggest to set it every time and check it inside your application code. It gives you much more flexibility.
2) You should not need to use two selects. ##ROWCOUNT or ROWCOUNT_BIG() I would suggest to read this excellent article about this topic.

Scripting functoid returns input instead of return-value in Biztalk mapper

I have a scripting functoid with the following code:
public string MyConcat(string product)
{
string retStr= "01";
product = product.ToUpper();
if(product.Contains("CONDITION")){
retStr= "02";
}
return retStr;
}
This works perfect when I run it in LinqPad, but when I test the map it returns the product string instead of the retStr, which I find really weird. Any help is much appreciated.

You probably have another Scripting functoid that has the same signature, i.e. is called MyConcat, returns a string, has a single string input. In that case it will execute the first version created with the input linked to it.
Please make sure you give your function names a unique and descriptive name to avoid this.
If you do need to use the same function multiple times in your map, this feature of it re-using the the function is quite useful, but I usually make sure to add a comment to all the subsequent copies stating that only the first version has the code.

linq query that selects any object in the model

Edit
To understand what i am trying to do i will give an example
Suppose you used entity framework to generate multiple tables. Now you have MyDbEntities with objects (client, user, product, singer [attributes don't matter]). Now you create a method. This method gets an object as a parameter + MyDbEntities. The object would be a client, user, product, singer. In this method you have a query that given the object would return a list of that object. I am trying to create this method but i am still trying either with linq or sql but in vain so far
Update
I tried this, it works, but not the way i want it to. to understand me check the code below.
ViewBag.ok = LibMethods.get_all(MyDbEntities, "Client"); //Params = Entity & String
this how i can my method below is my method
public static Object get_all (MyDbEntitiesce, String given_entity_type)
{
if (given_entity_type.Equals("Client"))
{
var get_all = from clt in ce.Client
select clt;
return get_all.ToList().First();
}
else if (given_entity_type.Equals("Product"))
{
var get_all = from pdt in ce.Product
select pdt;
return get_all.ToList().First();
}
return null;
}
As you can see this method checks the object given and creates and returns a list of objects of that given_entity_type. here i return the first element just because i am doing this for tests.
now suppose that i add a new object to MyDbEntities & call it Developer i would have to update that method but i don't want to do that i want something generic that would work on any object i give it that exists in MyDbEntities. that's what i mean by generic/dynamic.

Would the OfType query be what you want? I'm not sure I clearly understand your question, but this query will filter the list of objects by the given type.

Although I do not understand what your ultimate goal is, I would suggest that you look into DbSet.Set() method:
You can then use it like this Context.Set<Singer>() or Context.Set<Client>() depending on your target table.
That is the most generic you can get. Try showing more Code to make your scenario more clear.

Getting A field or Entity Generically Based on a String Value

I am trying to write a wrapper function for an application.
This application will continually add entities so it would be better if we could write one generic function rather than have to carve out an exception for each item.
For certain reasons we maintain both a GUID and an int key.
When an int key gets updated, we need to update it both in the parent record the child records but since it is possible that at any given time there could be multiple child records with the same int key, we need to get a list of Guids of what we want to update.
Here is some psuedo code of what I am trying to do.
List<string> depenedents = new List<string>();
depenedents.add(table1);
depenedents.add(table2);
depenedents.add(table3);
for(item in depenedents)
{
context.set<type item>();
entities.getguid();
}
Obviously the issue here is with the for loop.
Is it possible to get a list of entities knowing only the string of the entity type? Luckily all of our entities are wrapped to a base class that has a get guid method, but I need to find a way to actually get the entities.
Any help is greatly appreciated.

Something like:
for(item in depenedents)
{
context.GetMethod("Set")
.MakeGenericType(Type.GetType(item))
.Invoke(context, new object[0]);
entities.GetType.GetMethod("GetGuid").Invoke(entities, new object[0]);
}
...should be with roughly what you need. This will invoke the correct type specialisation of the Set generic instance method. Then it will invoke the instance method called "GetGuid" on the entities object.
Or possibly:
foreach(var entity in entities)
{
entity.GetType.GetMethod("GetGuid").Invoke(entities, new object[0]);
}
You'll maybe want to do something with the values returned, but hopefully this answer will point you in the right direction!
(and clearly you could optimise this code substantially to cache reflected types and methods, or to use compiled expressions rather than Invoke() calls)

Use EF Code First Local extention

The following method works properly in my service layer:
public override IList<City> GetAll()
{
var query = from item in _tEntities
select item;
query.Load();
return _tEntities.Local;
}
but when i try to run following method, the method returns old data loaded by GetAll() method.
public override IList<City> GetAll(Func<City, bool> predicate)
{
var query = from item in _tEntities
select item;
query.Where<City>(predicate);
query.Load();
return _tEntities.Local;
}
What is the problem and how can i fix them?
How can i use local method in this sample and reload new data into local(cache)?

You are looking at the wrong problem. What you are most likely seeing is a result of the fact that when you do the first query, the local cache is empty. So it only returns the results from your query. But when you do the second, it's returning the results of your first query AND your second query.
This comes down to the fact that you are using a shared DbContext between all your methods. Local contains a cache of all records the context has retrieved, not just the most recent query.
The correct solution is to not use Local in this manner. Even better, don't use a shared context since this can lead to context cache bloat.

I'm not too sure what you are trying to achieve with a .Load method here but it seems like you want the following.
public override IList<City> GetAll(Func<City, bool> predicate)
{
return _tEntities.Where<City>(predicate).ToList();
}

query.Where<City>(predicate);
This doesn't change query. The query.Load() on the next line ignores the predicate: you're calling query.Load() and not query.Where<City>(predicate).Load(). It's as if you had written
int i = 3;
i + 1;
Console.WriteLine(i); // still prints 3
In that example, C# does not really actually allow an addition to be used as a statement, but .Where(predicate) is a method call, and method calls can be used as such, even if they return values.
This is not your only issue (see the other answers), but my guess is that this issue is the one that leads to the unexpected results you're seeing.