I'm trying to write a program that catches the HTTP get requests.
I have found Fiddler-core a genius library that should do exactly what I want.
The thing is, I'm trying to execute a big piece code-work inside the void FiddlerApplication_BeforeRequest(Session oSession) and it seems to block all the request and damage my surfing speed a great deal.
I have tried to use threads/tasks with no avail.
What am I doing wrong?
This is my code:
public event RequestCapture RequestCaptured;
private CancellationTokenSource cancelTokenSource = new CancellationTokenSource();
//...stat public function:
public void RunWatch() {
Fiddler.FiddlerApplication.BeforeRequest += FiddlerApplication_BeforeRequest;
Fiddler.FiddlerApplication.Startup(0, FiddlerCoreStartupFlags.Default);
}
void FiddlerApplication_BeforeRequest(Session oSession)
{
if (RequestCaptured != null)
{
CancellationToken ct = cancelTokenSource.Token;
Task.Factory.StartNew(() =>RequestCaptured(oSession.fullUrl), ct);
//Handle the event in a new thread, so the Listener will continue to listen
}
}
//close public function:
public void Close() {
try
{
FiddlerApplication.Shutdown();
cancelTokenSource.Cancel();
}
catch { }
}
now i have i different class that do that:
public Form1()
{
Listiner = new HttpWatcher.Listner();
Listiner.RequestCaptured += RequestCaptured;
Listiner.RunWatch();
}
void RequestCaptured(string url)
{
System.Threading.Thread.Sleep(10000);
}
edit
The question is: Is there a better way using fiddler-core? or am i to build a simple proxy for that? using something else? Thanks!
edit2
I have edited the code, so it would fill the missing parts.
Just to be clear here, FiddlerCore processes each Session on a threadpool thread. If you need blocking behavior, there's no need to spin up an additional thread or anything like that. If you don't need to process things in a blocking manner, then feel free to queue the data on a background queue and use tasks or another asynchronous mechanism to perform processing.
You should explain exactly what you mean when you say damage my surfing speed a great deal, and whether or not you see different behavior when using Fiddler rather than your application.
Related
I want to open a thread to do the things it needs to do until a new command is given by the user. Then this thread should either close or receive a new command.
I have seen many posts that sending a variable to a running thread is hard, that is why I decided to kill the thread and start it again with the new variable.
I used the following post: https://stackoverflow.com/a/1327377 but without success. When I start the thread again (after it has done abort()) it gives me an exception: System.Threading.ThreadStateException.
private static Thread t = new Thread(Threading);
private static bool _running = false;
static void Main(string[] args)
{
[get arg]
if (CanRedo(arg))
{
if (t.IsAlive)
{
_running = false;
t.Interrupt();
if (t.Join(2000)) // with a '!' like in the post, abort() would not be called
{
t.Abort();
}
}
_running = true;
t.Start(arg); // gives System.Threading.ThreadStateException
}
}
private static void Threading(object obj)
{
_stopped = false;
string arg = obj.ToString();
while(_running)
{
if (bot._isDone)
{
ExecuteInstruction(arg);
}
}
}
What am I doing wrong?
I'm going to guess that you don't literally mean to abort the thread and start that same thread again. That's because if we start a thread to do some work we don't care which thread it is. If you cancel one thing and start something else, you probably don't care if it's the same thread or a different one. (In fact it's probably better if you don't care. If you need precise control over which thread is doing what then something has gotten complicated.) You can't "abort" a thread and restart it anyway.
Regarding Thread.Abort:
The Thread.Abort method should be used with caution. Particularly when you call it to abort a thread other than the current thread, you do not know what code has executed or failed to execute when the ThreadAbortException is thrown, nor can you be certain of the state of your application or any application and user state that it is responsible for preserving. For example, calling Thread.Abort may prevent static constructors from executing or prevent the release of unmanaged resources.
It's like firing an employee by teleporting them out of the building without warning. What if they were in the middle of a phone call or carrying a stack of papers? That might be okay in an emergency, but it wouldn't be a normal way to operate. It would be better to let the employee know that they need to wrap up what they're doing immediately. Put down what you're carrying. Tell the customer that you can't finish entering their order and they'll need to call back.
You're describing an expected behavior, so it would be better to cancel the thread in an orderly way.
That's where we might use a CancellationToken. In effect you're passing an object to the thread and telling it to check it from time to time to see if it should cancel what it's doing.
So you could start your thread like this:
class Program
{
static void Main(string[] args)
{
using (var cts = new CancellationTokenSource())
{
ThreadPool.QueueUserWorkItem(DoSomethingOnAnotherThread, cts.Token);
// This is just for demonstration. It allows the other thread to run for a little while
// before it gets canceled.
Thread.Sleep(5000);
cts.Cancel();
}
}
private static void DoSomethingOnAnotherThread(object obj)
{
var cancellationToken = (CancellationToken) obj;
// This thread does its thing. Once in a while it does this:
if (cancellationToken.IsCancellationRequested)
{
return;
}
// Keep doing what it's doing.
}
}
Whatever the method is that's running in your separate thread, it's going to check IsCancellationRequested from time to time. If it's right in the middle of doing something it can stop. If it has unmanaged resources it can dispose them. But the important thing is that you can cancel what it does in a predictable way that leaves your application in a known state.
CancellationToken is one way to do this. In other really simple scenarios where the whole thing is happening inside one class you could also use a boolean field or property that acts as a flag to tell the thread if it needs to stop. The separate thread checks it to see if cancellation has been requested.
But using the CancellationToken makes it more manageable if you want to refactor and now the method executing on another thread is a in separate class. When you use a known pattern it makes it easier for the next person to understand what's going on.
Here's some documentation.
What about doing it this way:
private static Task t = null;
private static CancellationTokenSource cts = null;
static void Main(string[] args)
{
[get arg]
if (CanRedo(out var arg))
{
if (t != null)
{
cts.Cancel();
t.Wait();
}
// Set up a new task and matching cancellation token
cts = new CancellationTokenSource();
t = Task.Run(() => liveTask(arg, cts.Token));
}
}
private static void liveTask(object obj, CancellationToken ct)
{
string arg = obj.ToString();
while(!ct.IsCancellationRequested)
{
if (bot._isDone)
{
ExecuteInstruction(arg);
}
}
}
Tasks are cancellable, and I can see nothing in your thread that requires the same physical thread to be re-used.
I'm trying to understand the differences among WebClient.OpenRead, WebClient.OpenReadAsync and WebClient.OpenReadTaskAsync.
It looks like these have differences regarding blocking a thread, but I don't understand it well.
Could you please explain the differences? It would be great if you could give me some example (examples don't have to be sample code, but would be great if you could provide)
As you said, the difference is in thread blocking behavior. First one (OpenRead()) is thread blocking operation, other two - not. For example, let assume, that your network latency to reach google.com is 300ms. When you do var stream = webClient.OpenRead(#"https://www.google.com"); your application is "paused" for this 300ms, so code next to this line is not executed until your webClient return a stream to stream variable. This is calling Thread blocking.
When you do this in UI-thread (in example: in a button click handler) - your application become freezing and not responding to user actions. This is bad user experience, so never-ever call a thread blocking stuff in your UI. Here is example for console application:
var address = #"https://www.google.com/";
Console.WriteLine($"Opening stream from {address}");
using (var stream = webClient.OpenRead(address)) // this will block for 300ms
{
Console.WriteLine("Stream is open!");
// some other code
}
Second method (OpenReadAsync()) is asynchronous and return nothing immediately after a call, so your thread is not blocked. After awhile (300ms) OpenReadCompleted event will be raised by your webClient and all attached listeners will handle opened stream one-by-one. Here is an example:
public partial class MainForm : Form
{
private WebClient _client = new WebClient();
public MainForm()
{
InitializeComponents();
_client.OpenReadCompleted += OpenReadCompletedHandler;
}
private void ButtonClickHandler(object sender, EventArgs e)
{
_client.OpenReadAsync(#"https://www.google.com/");
}
private void OpenReadCompletedHandler(object sender, OpenReadCompletedEventArgs e)
{
// this event will be raiesed 300ms after 'Button' click
var stream = e.Result; // <- here is your stream
// some other code
}
}
The last one (OpenReadTaskAsync()) is all about TPL (Task Parallel Library) and async/await keywords. It runs all stuff in a Task which is returned by this method. Here is an example:
public partial class MainForm : Form
{
private WebClient _client = new WebClient();
public MainForm()
{
InitializeComponents();
}
private async void ButtonClickHandler(object sender, EventArgs e)
{
// after 'await' keyword, execution will be returned from this method immediately
// meanwhile, actual acquiring of 'stream' is running in a background thread
using (var stream = await _client.OpenReadTaskAsync(#"https://www.google.com/"))
{
// after 300ms, this code will be continued in UI thread
// result will be automaticly unpacked to 'stream' variable
// some other code
}
}
}
Hope this helps.
I'd suggest WebClient is more or less obsolete now, HttpClient is more appropriate for anything targeting .NET framework 4.5+ or .NET core. Just watch out that the latter does not automatically throw exceptions on HTTP error codes (400+).
I need to a background long running thread from the asp.net application.
BTW,the method fetches something from external data source period and maybe exception happens, I provide some ways to fullfill this task, please advice which way is best, please advice if there is better way.
Way1=Looping when exception happens.
static void LongRunningMethod()
{
do
{
try
{
//fetch something from external period and maybe exception happens.
Thread.sleep(100000);
}
catch(Exception ex)
{
//log exception..
}
} while (true);
}
Way2=Running the following method period by something like timer, and open a new thread when exception happens.
static void LongRunningMethod()
{
try
{
//fetch something from external period and maybe exception happens.
Thread.sleep(100000);
}
catch(Exception ex)
{
//log exception..
Thread T2 = new Thread(LongRunningMethod);
T2.Start();
}
}
Way3=Call itself when exception happens.
static void LongRunningMethod()
{
try
{
//fetch something from external period and maybe exception happens.
Thread.sleep(100000);
}
catch(Exception ex)
{
//log exception..
LongRunningMethod();
}
}
I will use none of the three. In fact, I seldom build interval tasks base on ASP.NET because:
1. The start of the task is out of control
Based on ASP.NET the task has to be started/registered on Application_Start method, which is triggered by the first request, that means your tasks is started when the first request comes. Theoretically it can be long time after your application is deployed.
2. The end of the task is out of control
The web servers (thinking IIS) can be configured as recycle-on-demand (AFAIK this is the default behavior). That is, your thread may be killed when executing. Under most circumstances you need to deal with the persistence of your tasks and states, and add retrying codes and etc... That's a nightmare!
For me it's a better idea to use a windows service, or the task scheduler function of windows, the latter is even easier because you don't need to write a timer or interval call codes. It's more stable, less code, and friendly logs in the event viewer! That's really make everything easier.
I like to use System.ComponentModel.BackgroundWorker:
static BackgroundWorker looper = new BackgroundWorker();
static bool isRunning = true;//you can set this to false when closing
public static void initialize(){
looper.DoWork+= doLoop;
looper.RunRunWorkerAsync();
}
private static void doLoop(object sender, DoWorkEventArgs e){
while(isRunning){
//do looping code
System.Threading.Thread.Sleep(5000);
if(!isRunning)
break;
}
}
PS - sorry for weird spacing - I edited this directly in the code window.
Is Looping inside a task really recommended?
example code:
public void doTask(){
Task.Factory.StartNew(() => {
do{
// do tasks here.... call webservice
}while(true till cancelled)
});
}
any answers would be great! :)
because it is a case for my webservice calling right now, and the memory consumption goes out of control.
So may I ask, is looping inside a task really good or not recommended at all?
As Requested by SLC, heres the code:
CancellationTokenSource tokenSrc;
Task myTask;
private void btnStart_Click(object sender, EventArgs e)
{
isPressed = !isPressed;
if(isPressed)
{
tokenSrc = new CancellationTokenSource();
myTask = Task.Factory.StartNew(() =>
{
do{
checkMatches(tokenSrc.Token);
}while(tokenSrc.IsCancellationRequested != true);
}, tokenSrc.Token);
}
else {
try{
tokenSrc.Cancel();
// Log to notepad
}
catch(Exception err){
// Log to notepad
}
finally {
if(myTask.IsCanceled || myTask.IsCompleted || myTask.isFaulted) {
myTask.Dispose();
}
}
}
}
private void checkMatches(CancellationTokenSource token)
{
try
{
if(!token.IsCancellationRequested)
{
//Create Endpoint...
//Bypass ServCertValidation for test purposes
ServicePointManager.ServerCertificateValidationCallback = new RemoteCertificateValidationCallback(delegate {return true;});
using(WebServiceAsmx.SoapClient client = new....)
{
client.CheckResp response = client.chkMatch();
// if's here for the response then put to logs
}
}
}
catch(Exception err)
{
// err.toLogs
}
}
It's perfectly fine to do this, especially if your task runs constantly, for example picking up a message queue.
while (not shutting down)
get next email to send
if exists next email to send
send
else
wait for 10 seconds
wend
Ensure that you have a way to get out if you need to cancel it, like you've done with a flag, and you should be fine.
Regarding webservices:
You should have no problem calling the webservice repeatedly, nor should it cause any memory spikes. However, you should make sure your initialisation code is not inside the loop:
BAD
while (notShuttingDown)
make a new connection
initialise
make a call to the service()
wend
GOOD
make a new connection
initialise
while (notShuttingDown)
make a call to the service
wend
Depending on your webservice it might be more optimal to create a batch operation, for example if your service is HTTP then hitting it repeatedly involves a lot of overhead. A persistent TCP connection might be better because it could be creating and destroying a lot of objects to make the calls.
For example
slow, lots of overhead:
myRecords = { cat, dog, mouse }
foreach record in myRecords
webservice check record
endforeach
faster:
myRecords = { cat, dog, mouse }
webservice check [myRecords] // array of records is passed instead of one by one
Debugging: The most likely risk is that somehow the task is not being disposed correctly - can you add this to your method to debug?
myTask = Task.Factory.StartNew(() =>
{
Console.Writeline("Task Started");
do{
checkMatches(tokenSrc.Token);
Thread.Sleep(10); // Some pause to stop your code from going as fast as it possibly can and putting your CPU usage to 100% (or 100/number of cores%)
}while(tokenSrc.IsCancellationRequested != true);
Console.Writeline("Task Stopped");
}
You might have to change that so it writes to a file or similar depending on if you have a console.
Then run it and make sure that only 1 task is being created.
In my application, I used to create along string of async operations, which passed in functions like this:
public void LotsOfAsync()
{
DoAsync1( ()=> { DoAsync2( ()=> { doAsync3( ()=> { finalAction();}) } ) } );
}
However, now I have moved many of those async operations into separate classes and objects but I want the same results. Mainly I have moved those async operations into classes which become part of a collection.
I'd like my new code to look like this:
public void lotsOfAsync()
{
DoAsync1();
for each ( MyClass in Async1List)
{
MyClass.DoAsyn2();
}
if (allAsyncOperationsAreComplete)
{
FinalAction();
}
}
What things do I need to do, to get this to work? Thanks.
Using the Answer below about Tasks, something still seems to be lacking. The program never continues even after throwing everything into a BackgroundWorker.
You can use Task<T> (using the Task Parallel Library for Silverlight) - something like this maybe:
List<Task> tasks = new List<Task>();
Task.Factory.StartNew(() => DoAsync1()).ContinueWith(_ =>
{
foreach (MyClass myClass in Async1List)
{
tasks.Add(Task.Factory.StartNew(() => myClass.DoSomething()));
}
Task.WaitAll(tasks.ToArray());
FinalAction();
});
Im not familiar with wp7, but you may use counter as static field and check if it's equal to 0 in final action.
Every MyClass.DoAsyn2() should fire maybe a event, or any other code to signal that it is finished.
Another option is to move all async to task and call Task.WaitAll
http://msdn.microsoft.com/en-us/library/dd270695.aspx
Have you had a look at the CountdownEvent in .Net 4? This is a signalling construct where one thread will block and only proceed once other threads have completed and called set on the count down event. You initialise it with the number of signals you need before the thread calling Wait on the construct will proceed. E.g.:
CountdownEvent countdown = new CountdownEvent(3);
will only let the thread calling Wait to proceed once 3 other threads have called Signal.
So your example would perhaps look something like this:
public void lotsOfAsync()
{
Int32 numberOfAsyncProcesses = Async1List.Length + 1;
CountdownEvent countdown = new CountdownEvent (numberOfAsyncProcesses);
DoAsync1(countdown); // call countdown.signal() in the async method once complete.
for each ( MyClass in Async1List)
{
// call countdown.signal() in the async method once complete.
MyClass.DoAsyn2(countdown);
}
if(countDown.Wait(TimeSpan.FromSeconds(3))
{
FinalAction();
}
}
I've also added a timeout where the calling thread will unblock after 3 seconds if failed to get a response from all processing threads. In this case, the final action will not be performed.
You can reproduce this with Monitor/Pulse if you are not targeting .Net 4.
There is a nice example here too. Hope this helps!
After looking through all the previous answers, I was unable to solve my problem.
Instead what I needed to do, was create custom Events within my classes, which triggered upon a successful completion of the asynchronous tasks.
The website that proved the most useful to me to accomplish this was: http://www.marten-online.com/csharp/simple-custom-event-handling.html
My final code looked something like this:
public void lotsOfAsync()
{
DoAsync1();
for each ( MyClass in Async1List)
{
MyClass.PropertyChange += new MyClass.PropertyChangeHandler(Async2Complete);
MyClass.DoAsyn2();
}
}
public void Async2Complete(object sender, PropertyChangeEventArgs data)
{
if (data.PropertyName == "AsyncComplete")
{
totalAsyncCompleted++;
if (totalAsyncCompleted >= Async1List.Count)
{
FinalAction();
}
}
}
Have you heard of the Deferred pattern often used in Javascript?
It is simple to work with and very dynamic and you should be able to implement it on Windows phone aswell.
Have a look at this guide
http://twistedmatrix.com/documents/current/core/howto/defer.html
Regards
Tobias