I am posting this question using an automatic translation.
Please forgive any grammatical errors.
I have built an application using the .NET framework and the ASP.net Web API.
I have split the virtual path for each customer region within a site running on IIS and copied the same binary to run as separate applications.
The applications run in the same application pool.
Recently, some customers have been making a very large number of requests in a matter of minutes.
(I suspect a glitch in the system on the customer's end).
I am thinking of adding a static class to my current application that keeps track of the number of requests per customer in a given time period and blocks them if the threshold is exceeded.
From past StackOverFlow articles I have found that "information in the static class is lost if the application pool is recycled", but I have determined that this is not a problem in this case.
For my purposes, I only need to be able to retain information for a few minutes.
However, I still have a few questions that I can't find answers to, so I'd like to ask you all a few questions.
Even if the same binary is running in the same application pool, will the static class information be kept separately for different applications?
Will the static constructor of a static class be executed even after the application pool is recycled?
Is there a problem if I reference a field in Global.asax from within a static class?
Is there a problem with referencing the contents of web.config from within a static class?
Attached below is the source of my experimental implementation.
I plan to call the static method "ExcessiveRequestCheck.isExcessiveRequest" of this static class after the Web API receives the request and identifies the user ID.
Any advice would be sincerely appreciated.
P.S.
I understand that this approach does not work well in a load balancing environment. Currently my system only runs on one virtual machine. If you are moving to the cloud or deploying a load balancer, you will probably need a different approach than this one.
public static class ExcessiveRequestCheck
{
private static Dictionary<string, ExcessiveRequestInfo> dicExcessiveRequestCheckInfo = new Dictionary<string, ExcessiveRequestInfo>();
private static object initLock = new object();
private static object dicExcessiveRequestCheckInfoLock = new object();
//If possible, I want this process to be a static constructor
public static Dictionary<int, int> dicExcessiveRequestSkipConditions
{
get
{
lock (initLock)
{
if (ExcessiveRequestCheck._dicExcessiveRequestSkipConditions == null)
{
//if possible, I want to set this value from Web.config.
ExcessiveRequestCheck._dicExcessiveRequestSkipConditions = new Dictionary<int, int>() {
{ 5, 3 }, { 15, 5 }, { 45, 10 }, { 120, 20 }
};
}
return ExcessiveRequestCheck._dicExcessiveRequestSkipConditions;
}
}
}
private static Dictionary<int, int> _dicExcessiveRequestSkipConditions = null;
public const int BUFFER_CLEAR_MINUTES = 5;
public static bool isExcessiveRequest(string userId)
{
ExcessiveRequestCheck.refreshExcessiveRequestCheckInfo();
lock (ExcessiveRequestCheck.dicExcessiveRequestCheckInfoLock)
{
if (ExcessiveRequestCheck.dicExcessiveRequestCheckInfo.ContainsKey(userId) == false)
{
ExcessiveRequestCheck.dicExcessiveRequestCheckInfo.Add(userId, new ExcessiveRequestInfo() { countRequest = 1 });
return false;
}
bool doSkip = false;
ExcessiveRequestCheck.dicExcessiveRequestCheckInfo[userId].countRequest++;
foreach (KeyValuePair<int, int> pair in ExcessiveRequestCheck.dicExcessiveRequestSkipConditions)
{
if (ExcessiveRequestCheck.dicExcessiveRequestCheckInfo[userId].lastRequesttTime.AddSeconds(pair.Key) > DateTime.Now)
{
if (ExcessiveRequestCheck.dicExcessiveRequestCheckInfo[userId].countRequest > pair.Value)
{
ExcessiveRequestCheck.dicExcessiveRequestCheckInfo[userId].wasRequestSkip = true;
doSkip = true;
}
}
}
ExcessiveRequestCheck.dicExcessiveRequestCheckInfo[userId].lastRequesttTime = DateTime.Now;
return doSkip;
}
}
public static void refreshExcessiveRequestCheckInfo()
{
lock (ExcessiveRequestCheck.dicExcessiveRequestCheckInfoLock)
{
var keyList = ExcessiveRequestCheck.dicExcessiveRequestCheckInfo.Keys;
foreach (string key in keyList)
{
if (ExcessiveRequestCheck.dicExcessiveRequestCheckInfo.ContainsKey(key))
{
var value = ExcessiveRequestCheck.dicExcessiveRequestCheckInfo[key];
if (value.lastRequesttTime.AddMinutes(BUFFER_CLEAR_MINUTES) < DateTime.Now)
{
if (value.wasRequestSkip)
{
//this NLog instance was created in Global.asax.cs
WebApiApplication.logger.Fatal("skip request! user id=" + key);
}
ExcessiveRequestCheck.dicExcessiveRequestCheckInfo.Remove(key);
}
}
}
}
}
}
class ExcessiveRequestInfo
{
public DateTime requestStartTime { get; set; } = DateTime.Now;
public DateTime lastRequesttTime { get; set; } = DateTime.Now;
public int countRequest { get; set; } = 0;
public bool wasRequestSkip { get; set; } = false;
}
Your questions
Even if the same binary is running in the same application pool, will the static class information be kept separately for different applications?
Yes, they are separate
Will the static constructor of a static class be executed even after the application pool is recycled?
Yes, the static constructor is guaranteed to be called before any of the static methods are executed
Is there a problem if I reference a field in Global.asax from within a static class?
No more than accessing it from anywhere else
Is there a problem with referencing the contents of web.config from within a static class?
No more than accessing it from anywhere else
Your general approach
DoS
If you're trying to mitigate a denial-of-service attack or credential stuffing attack, your approach probably won't work, since requests to your service will still result in load being added to your server, and if they are performing a credential stuffing attack, it'll fill up your dictionary with millions of entries and possibly cause your application to crash.
If you want to mitigate a denial-of-service attack effectively, you will probably need a more network-oriented solution, such as a smart firewall or a WAF.
Rate limiting
If on the other hand you are attempting to throttle specific users' activities (i.e. rate limiting), again, your approach probably isn't the greatest, because it does not support load balancing-- your list is held in in-process memory. For per-user rate limiting you will probably need to track user activity in a central data store accessible to all of your servers.
Static constructors
As a general rule, you should try to avoid static constructors, or keep them very simple, as a failure in a static constructor will cause your entire application to fail to start. Be careful!
even if the same binary is running in the same application pool, will the static class information be kept separately for different applications?
If by different applications, you mean separate web sites? yes, it will be kept separate to each web site you have running for that app pool.
Will the static constructor of a static class be executed even after the application pool is recycled?
Hum, that's a bit confusing. The constructor will only be executed if you call the class and that given constructor. Since there is never a instance of the class created, then the "initialize/new" event is never used nor triggered. So, any method with parameters will run and work fine - including the constructor. I would suggest that there is not some "event" that gets triggered on first use - it would not and does not make sense in the context of a static class, since you never create an instance. So, if you have some methods with parameters then fine.
So, constructor in the context of new instance of the class makes no sense - (did not even think that is possible with static).
There is no concept of "new" event that triggers, so I fail to see how this issue can ever matter.
Is there a problem if I reference a field in Global.asax from within a static class?
Well, values in that class are global to ALL users. But, those values can go out of scope just about any old time you please. As a result, ZERO use of public members is practial. While a app-pool re-start will re-set those class values? They can go out of scope just about any old time. They are global to all and every user. So, persisting values, or attempting to persit values in a static class is NOT a viable choice for production code. You can have methods (code) in that class, but any public persisting values really can't be relied upon to persist correctly. I'm not 100% sure, but even just general .net garbage collection would likely cause a re-set.
If you need this information to persist, then you can't use static, you have to create a instance of that class and persist it in session(). And session is per user.
A static class public values will apply to EVERY user - not just the current user. In effect those values are global to all users - but without any real ccontrol or garrutee that the values will persit - you have no control over this and thus you can't adopt this concpet and design for any system of practial value.
Is there a problem with referencing the contents of web.config from within a static class?
Reading values? No problem. Update or modify values? - a MASSIVE different issue. You modify web.config, that will trigger a app pool restart.
So, you free to read any file - text files, xml or whatever, and that includes web.config. As long as you not modify such files, then no problems.
The main issue here?
It simple not practical to assume, or build a design in which public static class values are to persist. The ZERO control you have when such values may go out of scope is somthing you have ZERO control over, and thus such designs can't use nor rely on values persisting.
And of course on many web hosting systems? They are now adopting cloud computing. This means from one post back to the next, you might be using a different server, and again, that means such values can't persist in memory, since from one post-back to the next, or one web service call to the next? You may well be hitting a different server anyway (and they don't share memory) (so, this suggests say using SQL server based sessions, or at the very least persisting such values in a database).
In fact, if you need such persisting values and data? Then use a database. The WHOLE idea of web based software is you do NOT have state between post-backs. And you are attempting to go even down a worse road, but hope on a wing and a prayer that some global values "might" and "sort of" and "maybe" will persist between calls to the web site.
Answer:
You really can't do this with any realm of reliably.
So, most of your questions don't really matter. What matters is these values are to persist, and you can't rely on such a design. If you need some persisting values, then you have to adopt a system and design that supports that concept (viewstate, cookies, or session()).
Now, I suppose you can give this a try, and then come back with a detailed report and how your experience turned out. But, there are too many pit falls, and without any code or system control over persisting values in memory, I don't think I would go down this road.
In web land, it makes next to no sense to have public variables that you attempt to persist in a static class. You can have code, you can have cool methods, you can use session(). But, the concept of persisting values in static class is a design choice that does not make sense, and can't be relied upon.
Web software is assumed to be state-less, and that VERY much is the assumption you have to make in regards to a static class, or in fact general use of such code.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I am working through C# in a Nutshell from Joseph Albahari & Ben Albahari which has been a great book by the way and am reading through the topic on static fields in C#. They have this example code,
public class Panda
{
public string name;
public static int population;
public Panda(string n)
{
name = n;
population = population + 1;
}
}
So I understand that the more instances of Panda that you instantiate the greater population will be become since it is shared amongst all objects of type Panda but now to my question.
Why? I just can't understand why I would ever want to utilize such behavior in an application. It seems like a confusing way to track a global variable within the object itself. Am I misunderstanding the potential benefits of a static field? What are some cases where this would be useful and not confusing?
I think it's best to review what happens under the hood first.
If you create a static class, a single instance is created at runtime. It happens whenever you try to use the type the first time and is used from there on. This can come in handy if you want to, say, lazy load a shared resource for instance. It also guarantees (via compiler and runtime) that you have one and only one instance at all times.
If the class is not static but you use static members, you can construct new instances, but a "static version" is maintained for you in the background. This is useful for situations in which you need to either keep track of something or if you want to share something across instances or even other code if you make the member public.
In terms of performance for instance, it could be really useful if you need to speed up your program and realize (through object count) that you are instantiating an object that never changes 100 times. Maybe you want to show your user how many Pandas have been born. You could in theory keep a count somewhere else but if you think about it, you will need another object anyways so it makes sense to keep all related information and logic together. Besides, you could have a more general type that breaks into derived ones and you may want to track all of them without having to keep adding logic.
Consider the following example:
public abstract class Animal
{
private static int _count;
protected Animal()
{
IncrementCount();
}
protected static void IncrementCount()
{
_count++;
}
public int WorldPopulation()
{
return _count;
}
}
public class Dog : Animal
{
}
public class Cat : Animal
{
}
public class Bird : Animal
{
}
If I was to create a Dog, Cat and Bird instance and then check the value of the WorldPopulation() method, I would get 3.
The Singleton pattern is also commonly implemented using this approach. It allows you to maintain a single instance while containing the construction internally:
public class SingletonSample
{
private SingletonSample()
{
}
private static SingletonSample _instance;
public static SingletonSample Instance
{
get
{
if(_instance == null)
_instance = new SingletonSample();
return _instance;
}
}
public bool IsThisTrue()
{
return true;
}
}
Notice you can't access the IsThisTrue() method via Class name, you need an instance and it cannot be created directly. It can only be created internally by the class itself:
//Object construction occurs the first time you access the "Instance" property
SingletonSample.Instance.IsThisTrue();
I hope that helps.
I just can't understand why I would ever want to utilize such
behavior in an application.
You'd never want to know the panda-count in a game? What about high-score?
Now, whether static fields are the best approach is a different manner - there are alternative patterns, but they tend to be far more complex to build and manage.
Short answer:
Consider that a cache is a place to store the result of computation. Caches are useful whenever the computation is expensive, but the storage is cheap. In C#, a static variable is just a cache for computations about a live system.
Longer answer:
Theoretically, we could discover anything that we wanted to know about a running system by searching for all objects, and then performing a computation with respect to some subset. Since this is exactly what the garbage collector does, a hypothetical CLI that provided the right hooks into the garbage collector would obviate the need for static variables.
For example, suppose we wanted to know how many Widget objects that we’ve created. Well, all we would need to do is ask the GC for a list of all of the live objects, then filter the list for objects of type Widget, and then count the Widgets.
But there are a couple of problems in the example: Firstly, some of the Widget objects might not be live (not reachable, thus not able to undergo state changes), but we would need to keep them around just for counting purposes. Even if the size of each Widget instance was only a single bit, we would still need 122KB of memory if we needed to keep a count of, say, one million Widgets (since a CLR object is at least 4 bytes, we would need almost 4MB just to keep track of the count). On the other hand, a 20-bit variable is enough to count up one million. This is a savings of 99.99% (actually 99.99999% in the case of the actual CLR). Secondly, garbage collection can be an expensive operation. Even if we avoid the overhead of memory compaction, we would just need to pause the system in general.
So, hopefully, it’s now easy to see why we would want to have the ability to cache certain computations about a live system, and hence the usefulness of static variables.
Having said all that, it is often the case that it's better to just recompute things rather than caching the results in a static variables because of the way CPU caching works.
here is an example of how i used static objects.
I had task to create an uploader handler with progress bar.
and progress bar had show up to all users that are in the site.
so a created the upload operation in a new thread and then appended the result of the operation to a static object(Progress bar) that are outside the thread, the progress bar will show up to all users that are viewing the site.
more info and exemplar could be found here
What is the use of static variable in C#? When to use it? Why can't I declare the static variable inside method?
I created a class for authentication of my project.
I want to use it in my website and also in mobile application but why this class has a static constructor and should be get instance one time, it didn't work well, I mean I want to get instance for each application once.
I want to know how fix it?
It sounds to me like you're looking for a singleton pattern.
"I mean I want to get instance for each application once."
I assume you mean that you want to construct an instance once per application.
The easiest implementation is to use this:
public class MyAuth
{
private static readonly MyAuth instance = new MyAuth();
public static MyAuth Instance { get { return instance; } }
private MyAuth()
{
// initialization goes here and will be called once
}
// Members.
}
So first off, I'm not sure if this is a good solution. Authorization is usually updated from different applications, so it might be better to flush the authentication once in a while.
Second, note that member functions can be called from multiple threads. If you want to do it like this, you have to use locking for fields that you use (either directly or indirectly). Be sure to know what you're doing.
There is an alternative solution that solves all this in a neat way (and a ton of other issues). It's called a database; you might want to consider using that.
I have an application running in IIS (ASP.NET) and a significant amount of times, the page loads but rendered and populated with information for a user different than the user who actually requested the page.
The data used to render and populate the page belongs to a user who also initiated a request at roughly the same point in time.
Could this be due to static variables and what I am now seeing referred to as thread agility?
This isn't an issue with thread agility, it's an issue with using static variables. Static variables are static for the PROCESS, not one thread. So, a static variable in ASP.NET is shared by EVERY request being made to your site. So, ask yourself: is this static variable MEANT to be shared among every requester, or is it specific to ONE requester? For example:
static int TotalPageHits; //Count the number of times a page has been requested
Clearly, this is perfectly fine data to share for ALL users. On the other hand:
static string CurrentUserId; //DANGER WILL ROBINSON!!!!
Any variable specific to one requester you want to put in HttpContext.Current.Items.
Static variables that contain user-specific data would explain the behavior you are seeing.
Thread agility might exascerbate the problem (e.g. make it more noticeable or lead to different manifestations of the problem, cause issues with ThreadStatic usage, etc.), but I would start by examining your usage of static members.
// bad - any caller can access this in any sequence
public static int CurrentUserId
{
get;
set;
}
// okay, because the backing storage is safe/segmented
public static int CurrentUserId
{
get { return (int)Session["CurrentUserId"]; }
set { Session["CurrentUserId"] = value; }
}
// data that you want to be shared
public static List<string> SomeValuesToBeShared
{
// safe for reading (if properly initialized)
// safe for writing only if appropriate locks are used
}
IF your code is really thread-safe this won't happen...
Use of static variables in such a context might a good starting point for investigation...
IF you mean anything marked ThreadStatic then this is a good starting point too...
To help in any specific way you need to provide much more details...
I have recently inherited a system that uses a very basic approach to processing workitems, basically, it does them one by one. To be honest, up until recently this worked well. However, we are looking to implement a similiar process for another type of workitem and I have been looking into Task Parallel Library and think that will fit the bill. However, I have some concerns about Thread Safety and to be honest, this is an area that I lack knowledge, so I am asking only my 2nd question on here in hope that someone can give me some good points as I have yet to find a definitive yes or no answer for this.
So we have our 'WorkItem' class
public class WorkItem
{
public int Id {get; set;}
public string data { get; set;}
}
A List<WorkItem> will be generated and these will then be processed using a Parallel.Foreach loop.
The Parallel.Foreach will call a private method, which in turn will call static methods from another assembly;
//Windows service that will run the Parallel.Foreach
private int MainMethod(WorkItem item)
{
item.Data = Processor.ProcessWorkItemDataProcess1(item.data);
item.Data = Processor.ProcessWorkItemDataProcess2(item.data);
SendToWorkFlow(item);
}
public static class Processor
{
public static string ProcessWorkItemDataProcess1(string data)
{
//Process it here
return string
}
public static string ProcessWorkItemDataProcess2(string data)
{
//Process it here
return string
}
}
And so on. All of these methods have logic in them to process the WorkItem instance at various different stages. Once complete, the MainMethod will send the processed WorkItem off to a Workflow System.
We will be processing these in batches of up to 30 in order not to overload the other systems. My concerns are basically the potential of 30 instances of WorkItem accessing the same static methods could cause some data integrity issues. For example, ProcessWorkItemDataProcess2 is called with WorkItem1.Data and is subsequently called with WorkItem2.Data and somehow WorkItem2.Data is returned when it should be WorkItem1.Data
All of the static methods are self-contained in so far as they have defined logic and will only (in theory) use the WorkItem that it was called with. There are no methods such as DB access, file access, etc.
So, hopefully that explains what I am doing. Should I have any concerns? If so, will creating an instance of the Processor class for each WorkItem solve any potential problems?
Thanks in advance
The scenario you describe doesn't sound like it has any blatant threading issues. Your worries about a static method being called on two different threads and getting the data mixed up is unfounded, unless you write code to mix things up. ;>
Since the methods are static, they don't have any shared object instance to worry about. That's good. You have isolated the work into self-contained work items. That is good.
You will need to check to make sure that none of the static methods access any global state, like static variables or properties, or reading from a file (the same file name for multiple work items). Reading of global state is less of a concern, writing is what will throw a wrench in the works.
You should also review your code to see how data is assigned to your work items and whether any of the code that processes the work items modifies the work item data. If the work items are treated as strictly read only by the methods, that's good. If the methods write changes back to fields or properties of the work items, you will need to double check that the data in the work items is not shared with any other work items. If the code that constructs the work item instances assigns a cached value to a property of multiple work items, and the static methods modify properties of that value, you will have threading conflicts. If the work item construction always constructs new instances of values that are assigned to properties of the work item, this shouldn't be an issue.
In a nutshell, if you have multiple threads accessing shared state, and at least one is writing, then you need to worry about thread safety. If not then you're golden.
If I've got this class defined as part of an app in ASP.NET 2.0:
public class Foo
{
private static int _seed = 100;
private static object myLock = new object();
public Foo()
{
lock (myLock)
{
this.MyInt = _seed;
_seed++;
}
}
public int MyInt {get; set;}
}
(Edit: updated to account for thread safety concerns as pointed out by answers)
How will that static member behave? Will it start at 100 and increment separately for every session, or will it increment separately for every page refresh, or is it global...?
Note: I'm asking this because I'm using classes to model data for the first time in my ASP.NET app, and I've already discovered that C#'s by-reference nature appears to be ignored by ViewState serialization, so I want to know what other weirdness I can expect. For example, if I have this class defined (assume Bar is another class):
public class OtherFoo
{
public List<Bar> Bars {get; set;}
}
and I do this on my page:
OtherFoo _myFoo = new OtherFoo();
//Code here to instantiate the list member and add some instances of Bar
Bar b = _myFoo.Bars[0];
ViewState["myFoo"] = _myFoo; //Assume both are [Serializable]
ViewState["myBar"] = b;
When I get those out of ViewState on the next postback, b and _myFoo.Bars[0] are no longer the same object.
ASP.NET is not magic. It doesn't magically turn the C# programming language (or any other language) into a language that is aware of web development (sessions, requests, etc).
Your code will behave exactly as it would in any other kind of application, with the addition that it can be called by multiple threads at the same time (so that using "++" is not safe).
Again, there is no magic. Just like every other application, the lifetime of a static is restricted to the lifetime of the AppDomain in which the type containing the static is loaded.
An AppDomain in an ASP.NET application is created the first time the application is accessed (unless the IIS settings force it to pre-start), and only ends at certain times, like when an assembly in the bin folder is changed, or the web.config is changed, or when the IIS settings say that the AppPool needs to be recycled.
It will increase the seed every time the constructor is invoked. Note that this can happen in multiple threads, so you better make it thread safe.
Deserialization will cause the (default) constructor to be invoked. If you serialize it to the ViewState, then ASP.NET will deserialize the object on postback, and thus invoke the constructor.
Please note that the C# language and the asp.net framework are on a whole other level. The framework is written (largely) in C#, and it will do a lot for you behind the scenes, but it still follows the rules of the language and the runtime.
Serialization is nothing more than encoding the information of an object (or graph of objects) to a stream. If you deserialize it, you will have the same information back, but it is not the same object you started with. Again, it is no magic, you could write your own serialization library using attributes and reflection.