Develop Reference

.Net Session (StateServer mode) not synchronizing if manipulated after request end

I'm getting a bit frustrated with this problem:
I have a web site that manage some files to download, cause these files are very big, and must be organized in folders and then compacted, I build an Ajax structure that do this job in background, and when these files is ready to be downloaded, this job changes the status of an object in the user session (bool isReady = true, simple like that).
To achieve this, when the user clicks "download", a jquery Post is send to an API, and this API starts the "organizer" job and finish the code (main thread, the request scoped one), leaving a background thread doing the magic (it's so beautiful haha).
This "organizer" job is a background thread that receive HttpSessionState (HttpContext.Current.Session) by parameter. It organize and ZIP the files, create a download link and, in the end, change an object in the session using the HttpSessionState that received by param.
This works great when I'm using the session "InProc" mode (I was very happy to deploy this peace of art in production after the tests).
But, my nightmares started when I have deployed the project in production environment, cause we use "StateServer" mode in this environment.
In these environment, the changes is not applied.
What I have noticed, until now, is that in the StateServer, every change I make in the background thread is not "commited" to the session when the changes occurs AFTER the user request ends (the thread that starts the thread).
If i write a thread.join() to wait the thread to finish, the changes made inside the thread is applied.
I'm thinking about use the DB to store these values, but, I will lose some performance :(
[HttpPost]
[Route("startDownloadNow")]
public void StartDownloadNow(DownloadStatusProxy input)
{
//some pieces of code...
...
//add the download request in the user session
Downloads.Add(data);
//pass the session as parameter to the thread
//cause the thread itself don't know the current httpcontext session
HttpSessionState session = HttpContext.Current.Session;
Thread thread = new Thread(() => ProccessDownload(data, session));
thread.Start();
//here, if I put a thread.join(), the changes inside the thread are applied correctly, but I can't do this, otherwise, it ceases to be ajax
}
private void ProccessDownload(DownloadStatus currentDownload, HttpSessionState session)
{
List<DownloadStatus> listDownload = ((List<DownloadStatus>)session["Downloads"]);
try
{
//just make the magic...
string downloadUrl = CartClient.CartDownloadNow(currentDownload.idRegion, currentDownload.idUser, currentDownload.idLanguage, currentDownload.listCartAsset.ToArray(), currentDownload.listCartAssetThumb.ToArray());
listDownload.Find(d => d.hashId == currentDownload.hashId).downloadUrl = downloadUrl;
listDownload.Find(d => d.hashId == currentDownload.hashId).isReady = true;
//in this point, if I inspect the current session, the values are applied but, in the next user request, these values are in the previous state... sad... .net bad dog...
}
catch (Exception e)
{
listDownload.Find(d => d.hashId == currentDownload.hashId).msgError = Utils.GetAllErrors(e);
LogService.Log(e);
}
//this was a desesperated try, I retrieve the object, manipulated and put it back again to the session, but it doesn't works too...
session["Downloads"] = listDownload;
}

Better Technique: Reading Data in a Thread

I've got a routine called GetEmployeeList that loads when my Windows Application starts.
This routine pulls in basic employee information from our Active Directory server and retains this in a list called m_adEmpList.
We have a few Windows accounts set up as Public Profiles that most of our employees on our manufacturing floor use. This m_adEmpList gives our employees the ability to log in to select features using those Public Profiles.
Once all of the Active Directory data is loaded, I attempt to "auto logon" that employee based on the System.Environment.UserName if that person is logged in under their private profile. (employees love this, by the way)
If I do not thread GetEmployeeList, the Windows Form will appear unresponsive until the routine is complete.
The problem with GetEmployeeList is that we have had times when the Active Directory server was down, the network was down, or a particular computer was not able to connect over our network.
To get around these issues, I have included a ManualResetEvent m_mre with the THREADSEARCH_TIMELIMIT timeout so that the process does not go off forever. I cannot login someone using their Private Profile with System.Environment.UserName until I have the list of employees.
I realize I am not showing ALL of the code, but hopefully it is not necessary.
public static ADUserList GetEmployeeList()
{
if ((m_adEmpList == null) ||
(((m_adEmpList.Count < 10) || !m_gotData) &&
((m_thread == null) || !m_thread.IsAlive))
)
{
m_adEmpList = new ADUserList();
m_thread = new Thread(new ThreadStart(fillThread));
m_mre = new ManualResetEvent(false);
m_thread.IsBackground = true;
m_thread.Name = FILLTHREADNAME;
try {
m_thread.Start();
m_gotData = m_mre.WaitOne(THREADSEARCH_TIMELIMIT * 1000);
} catch (Exception err) {
Global.LogError(_CODEFILE + "GetEmployeeList", err);
} finally {
if ((m_thread != null) && (m_thread.IsAlive)) {
// m_thread.Abort();
m_thread = null;
}
}
}
return m_adEmpList;
}
I would like to just put a basic lock using something like m_adEmpList, but I'm not sure if it is a good idea to lock something that I need to populate, and the actual data population is going to happen in another thread using the routine fillThread.
If the ManualResetEvent's WaitOne timer fails to collect the data I need in the time allotted, there is probably a network issue, and m_mre does not have many records (if any). So, I would need to try to pull this information again the next time.
If anyone understands what I'm trying to explain, I'd like to see a better way of doing this.
It just seems too forced, right now. I keep thinking there is a better way to do it.

I think you're going about the multithreading part the wrong way. I can't really explain it, but threads should cooperate and not compete for resources, but that's exactly what's bothering you here a bit. Another problem is that your timeout is too long (so that it annoys users) and at the same time too short (if the AD server is a bit slow, but still there and serving). Your goal should be to let the thread run in the background and when it is finished, it updates the list. In the meantime, you present some fallbacks to the user and the notification that the user list is still being populated.
A few more notes on your code above:
You have a variable m_thread that is only used locally. Further, your code contains a redundant check whether that variable is null.
If you create a user list with defaults/fallbacks first and then update it through a function (make sure you are checking the InvokeRequired flag of the displaying control!) you won't need a lock. This means that the thread does not access the list stored as member but a separate list it has exclusive access to (not a member variable). The update function then replaces (!) this list, so now it is for exclusive use by the UI.
Lastly, if the AD server is really not there, try to forward the error from the background thread to the UI in some way, so that the user knows what's broken.
If you want, you can add an event to signal the thread to stop, but in most cases that won't even be necessary.

Problems getting newly created thread to send outputs to asp:panel in ASP.NET C#

I'm creating a file processor for use in an intranet.
I described it in another question - ERR_EMPTY_RESPONSE when processing a large number of files in ASP.Net using C#
Now, as suggested on above question's answer, I'm trying to use threads to execute the file processing task.
But there is a problem. I need the newly created thread to write feedbacks to a component in page (asp:panel, or div, or whatever). Those feedbacks would be results from several database operations.
The application reads those txts, interprets each line of it, and insert data in database. Each line inserted in database must return a feedback, like "registry 'regname' inserted successfully", or "i got problems inserting registry 'regname' in file 'filename', skipping to next registry".
I did test with something very simple:
protected void DoImport()
{
try
{
MainBody.Style.Add(HtmlTextWriterStyle.Cursor, "wait");
int x = 0;
while (x < 10000)
{
ReturnMessage(String.Format("Number {0}<hr />", x), ref pnlConfirms);
x++;
}
}
catch (Exception ex)
{
ReturnMessage(String.Format("<font style='color:red;'><b>FATAL ERROR DURING DATA IMPORT</b></font><br /><br /><font style='color:black;'><b>Message:</b></font><font style='color:orange;'> {0}</font><br />{1}", ex.Message, ex.StackTrace), ref pnlErrors);
}
finally
{
MainBody.Style.Add(HtmlTextWriterStyle.Cursor, "default");
}
}
This function is called from Page_Load, and fills an asp:panel called "pnlConfirms" with a row of numbers, but all at once, on load.
I changed it to:
protected void DoImport()
{
try
{
MainBody.Style.Add(HtmlTextWriterStyle.Cursor, "wait");
ThreadPool.QueueUserWorkItem(new WaitCallback(DoWork));
}
catch (Exception ex)
{
ReturnMessage(String.Format("<font style='color:red;'><b>FATAL ERROR DURING DATA IMPORT</b></font><br /><br /><font style='color:black;'><b>Message:</b></font><font style='color:orange;'> {0}</font><br />{1}", ex.Message, ex.StackTrace), ref pnlErrors);
}
finally
{
MainBody.Style.Add(HtmlTextWriterStyle.Cursor, "default");
}
}
private void DoWork(Object stateInfo)
{
int x = 0;
while (x < 10000)
{
ReturnMessage(String.Format("Number {0}<hr />", x), ref pnlConfirms);
x++;
}
}
And both uses this function:
public void ReturnMessage(string message, ref Panel panel, bool reset = false)
{
if (reset)
{
panel.Controls.Clear();
}
Label msg = new Label();
msg.Attributes.Add("width", "100%");
msg.Text = message;
panel.Controls.Add(msg);
}
I need ThreadPool.QueueUserWorkItem(new WaitCallback(DoWork)); to fill those asp:panels with feedbacks - like insertion errors and warnings.
My code already has those feedbacks under try...catch statements, but they're not getting output to any asp:panel from threadpool (it works when invoked directly from DoImport() function, like in the first example I posted).
I'm doing something very wrong, but I can't find out what (and I'm researching this for almost 2 weeks). Please, help!

In ASP.NET, when a browser requests a page, that page is rendered and sent to the browser as soon as its processing finishes, so the browser will show the page as it's finally rendered.
According to your code you're trying to render a page, show a wait cursor, and expect it's shown on the browser and then, the cursor is changed by a default cursor. As I explained, independently from using or not additional threads, the page won't be sent to the browser until it's completely rendered. So you'l never see the wait cursor on the client side.
The easiest wait to get what you're trying to do is to use web services (traditional .asmx or WCF) and AJAX (jquery os ASP.NET AJAX).
1) create a web service that does the processing
2) create a page which is sent to the browser, and, using javascript (jQuery or ASP.NET AJAX) make a call to the web service, and show something to let the user know that the request is being processed. (a wait cursor, or even better an animated gif)
3) when the process finishes, your javascript will get the responde from the web service, and you can update the page to let the user know the process has finished.
if you don't have experience on javascript, you can make most of this task using:
ScriptManager which can be used to create a javascript web service proxy for your client side (other interesting article) and is required for the rest of the controls
some javascript (or jquery) which can be use to update the "process running/ process finished hints" on the client side. I.e. when the call to the web service ends, you can use javascript to update the page using DOM, or load a new page or the same page with an special parameter to show the result of the process
In this way you can do what you want:
1) show a page in a state that shows the process is running
2) show the same, or other page, in a state that shows the end of the process
The trick is comunicating the browser with the server, and this can only be done using some of the available ajax techniques.
Another typical technique is using jQuery.ajax, like explained in encosia.com
According to the OP message, the process of all the files would be so slow that it would tiemout the web service call. If this is the case, you can use this solution:
1) Create a web service that process one (or a batch) of the pending files, and returns at least the number of pending files when it finishes the processing of the current file (or batch).
2) from the client side (javascript), call the web service. When it finishes, update the page showing the number of pending files, and, if this number is greater than zero, call the web service again.
3) when the call to the web service returns 0 pending files, you can update the page to show the work is finished, and don't call it any more.
If you process all the files at once, there will be no feedback on the client side, and there will also be a timeout. Besides, IIS can decide to stop the working thread which is making the work. IIS does this for several reasons.
A more reliable solution, but harder to implement, is:
1) implement a Windows Service, that does the file processing
2) implement a web service that returns the number of pending files (you can communicate the Windows Service and Web App indirectly using the file system, a database table or something like that)
3) use a timer (ajax timer, or javascript setInterval) from your web page to poll the server every N seconds using the web service, until the number of pending files is 0.
An even harder way to do this is hosting a WCF service in your Windows Service, instead of the indirect communication between your web app and windows service. This case is much more complicated because you need to use threads to do the work, and attend the calls to the wcf service. If you can use indirect communitacion it's much easier to implemente. The dtabse table is a simple and effective solution: your working process updates a row a table whenever it process a file, and the web service reads the progress state from this table.
There are many different soultions for a not so simple problem.

You are starting new thread (or more precise running your code on one of free threads in thread pool)and not waiting for results in main thread. Something like Thread.Join (if you would use manual thread creation) or other synchronization mechanism as events need to be used if you want to go this route.
The question you've linked to suggests using asynchronous pages which you are not doing. You would start processing request, kick off the task and release the thread, when the task is finished you complete request.
Side note: consider simply doing all conversion on main thread that handles request. Unless you expect slow I/O to complete the task moving CPU work from one thread to another may not produce significant gains. Please measure performance of your current solution and confirm that it does not meet performance goals you have set up for your application. (this does not apply if you doing it for fun/educational purposes).

Editing File in a thread

I'm working on a xml service at the moment , which is a sum of 20+ other xml's from other site's services.
So at first it was just ;
GetherDataAndCreateXML();
But obviously getting 20+ other xml , editing and serving it takes time , so i decided to cache it for like 10 minutes and added a final.xml file with a DateTime attribute to check if it's out of date etc. So it became something like ;
var de = DateTime.Parse(x.Element("root").Attribute("DateTime").Value).AddSeconds(10.0d);
if (de >= DateTime.Now)
return finalXML();
else
{
RefreshFinalXml();
return finalXML();
}
The problem now , is that any request after that 10 minute obviously takes too much time as it's waiting for my looong RefreshFinalXml() function. So i did this;
if (ndt >= DateTime.Now)
return finalXML();
else
{
ThreadStart start = RefreshFinalXml;
var thr = new Thread(start);
thr.IsBackground = true;
thr.Start();
return finalXML();
}
This way , even at the 11th minute i simply return the old final.xml but meanwhile i start another thread to refresh current xml at the background. So after something like 13th minute , users get fresh data without any delay.
But still there is a problem with this ; it creates a new thread for every single request between 10 to 13th minutes ( while first RefreshFinalXml is still working at the background ) and obviously i can't let that happen , right? And since I don't know much about locking files and detecting if it's lock , i added a little attribute , "Updating" to my final xml ;
if (ndt >= DateTime.Now)
return finalXML();
else
{
if (final.Element("root").Attribute("Updating").Value != "True")
{
final.Element("root").SetAttributeValue("Updating", "True");
final.Save(Path);
ThreadStart start = RefreshFinalXml;
//I change Updating Attribute back to False at the end of this function , right before saving Final Xml
var thr = new Thread(start);
thr.IsBackground = true;
thr.Start();
}
return finalXML();
}
So ,
0-10 minutes = return from cache
10~13 minutes = return from cache while just one thread is refreshing final.xml
13+ minutes = returns from cache
It works and seems decent at the moment , but the question/problem is ; I'm extremely inexperienced in these kind of stuff ( xml services , threading , locks etc ) so i'm not really sure if it'll work flawlessly under tougher situations. For example , will my custom locking create problems under heavy traffic, should i switch to lock file etc.
So I'm looking for any advice/correction about this process , what would be the "best practice" etc.
Thanks in advance
Full Code : http://pastebin.com/UH94S8t6
Also apologies for my English as it's not my mother language and it gets even worse when I'm extremely sleepless/tired as I'm at the moment.
EDIT : Oh I'm really sorry but somehow i forgot to mention a crucial thing ; this is all working on Asp.Net Mvc2. I think i could have done a little better if it wasn't a web application but i think that changes many things right?

You've got a couple of options here.
Approach #1
First, you can use .NET's asychronous APIs for fetching the data. Assuming you're using HttpWebRequest you'd want to take a look at BeginGetResponse and EndGetResponse, as well as the BeginRead and EndRead methods on the Stream you get back the response.
Example
var request = WebRequest.Create("http://someurl.com");
request.BeginGetResponse(delegate (IAsyncResult ar)
{
Stream responseStream = request.EndGetResponse(ar).GetResponseStream();
// use async methods on the stream to process the data -- omitted for brevity
});
Approach #2
Another approach is to use the thread pool to do your work, rather than creating and managing your own threads. This will effectively cap the number of threads you're running, as well as removing the performance hit you'd normally get when you create a new thread.
Now, you're right about not wanting to repeatedly fire updates while you wait for
Example #2
Your code might look something like this:
// We use a dictionary here for efficiency
var Updating = new Dictionary()<TheXMLObjectType, object>;
...
if (de >= DateTime.Now)
{
return finalXML();
}
else
{
// Lock the updating dictionary to prevent other threads from
// updating it before we're done.
lock (Updating)
{
// If the xml is already in the updating dictionary, it's being
// updated elsewhere, so we don't need to do anything.
// On the other hand, if it's not already being updated we need
// to queue RefreshFinalXml, and set the updating flag
if (!Updating.ContainsKey(xml))
{
// Use the thread pool for the work, rather than managing our own
ThreadPool.QueueUserWorkItem(delegate (Object o)
{
RefreshFinalXml();
lock(Updating)
{
Updating.Remove(xml);
}
});
// Set the xml in the updating dictionary
Updating[xml] = null;
}
}
return finalXML();
}
Hopefully that's enough for you to work off of.

I would go for a different method assuming the following
Your service is always running
You can afford/are allowed to getting the XML files even if you don't have any request to your service currently.
The XML files you fetch are the same files for all your requests. (that is the total number of XML files you need for all your responses are those 20 files)
The resulting XML file is not too big to keep in memory all the time
1
First of all I would not store the resulting XML in a file on disk but rather in a static variable.
2
Second I would create a timer set on 10 minutes that updates the cache even if you have no calls to your service. That way you always have quite recent data ready and cached even if your service was not called for a while. It also removes the need to think about if you already have a refresh "ongoing".
3
Third I would consider using threading/async calls to fetch all your 20 XML's in parallel. This is only useful if you want to reduce the refresh time. It could allow you to reduce the refresh interval from 10 to maybe 1-2 minutes if that is improving your service.
I would recommend 1 and 2, but 3 is more optional.

Threading using AJAX

When the user clicks on a link to generate report I make an AJAX call which generates a pdf file in the background.Now the files are huge running upto 10mb or more.So it takes some time.In the mean time the user should be able to navigate other links as if nothing has happened.So I need to implement in such a way that the pdf generation process gets started & user doesn't have to wait for the process to finish.Is this possible?I am using AJAX Pro with c# with dot net framework 2.0
The problem here is that as soon as the AJAX activity begins the browser enters into a hung stage & the user has to wait although he clicks on a different link.

I would probably create a 'queue' or an 'inbox' for the user ...
start your pdf generation routine with a ThreadPool.QueueUserWorkItem (you would also need to modify your generation method to output to their inbox)
then on each http request check that inbox and notify the user of the item ... you can always poll the server on an interval or somthing

Sure, but once the user navigates to another page, the Javascript that is waiting for the Ajax response is no longer running, so that request is lost. You'd have to either find a way to keep that page open (using frames or exclusively Ajaxified navigiation), or find a way to store the response and notify the user of its completion on the next page view. For instance, storing a session variable that indicates that the operation is completed, or storing it in a database with (perhaps) an "unread" boolean value.

You can have asynchronous Ajax call with which you can do other tasks while response objects returns from the Ajax page.
Here is some example, testAjax.aspx is the Ajax page here :
http_request.onreadystatechange = function() { alertContents(http_request); };
http_request.open('GET', 'testAjax.aspx?', true);
http_request.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
http_request.send(null);
function alertContents(http_request)
{//debugger;
if (http_request.readyState == 4)
{
if (http_request.status == 200)
{
var vResult;
vResult=http_request.responseText;
//Write your logic after successful Ajax call here.
}
else
{
alert('There was a problem with the request.');
}
}
}