How can i see HttpWebRequest object as string before calling GetResponse method? I want to see raw format of request something like this as in fiddler:
Content-Type: multipart/form-data; boundary=---------------------------2600251021003
Content-Length: 338
-----------------------------2600251021003 Content-Disposition: form-data; name="UPLOAD_FILEName"; filename="Searchlight062210 w price.csv" Content-Type: application/vnd.ms-excel
,,,,,
-----------------------------2600251021003
Content-Disposition: form-data; name="submit"
submit
-----------------------------2600251021003--
I tried following code, but not worked because stream is not readable.
string GetRequestString(HttpWebRequest req)
{
Stream stream2 = req.GetRequestStream();
StreamReader reader2 = new StreamReader(stream2);
return reader2.ReadToEnd();
}
If it is for logging purposes you could activate tracing by putting this in your app/web.config:
<system.diagnostics>
<sources>
<source name="System.Net.Sockets" tracemode="protocolonly">
<listeners>
<add name="System.Net.Sockets" type="System.Diagnostics.TextWriterTraceListener" initializeData="network.log" />
</listeners>
</source>
</sources>
<switches>
<add name="System.Net.Sockets" value="Verbose"/>
</switches>
<trace autoflush="true" />
</system.diagnostics>
Run your code and look at the generated log file.
Related
I'm trying to log all outgoing HTTP requests and responses a .NET application makes. I am following this article from the Microsoft docs: https://learn.microsoft.com/en-us/dotnet/framework/network-programming/how-to-configure-network-tracing
I am providing a short example of my code and app.config.
The following works as expected and produces a file network.log which includes the full request and response body. However this only works on Windows and .NET Framework. Running the same code on Linux using mono version 5.18.1.0 - mono Program.exe does not create the network.log file and does not throw any exception.
C# Code:
using System;
using System.IO;
using System.Net;
namespace TraceTest
{
class Program
{
static void Main(string[] args)
{
WebRequest request = HttpWebRequest.Create("http://example.com");
using (WebResponse response = request.GetResponse())
using (Stream responseStream = response.GetResponseStream())
using (StreamReader streamReader = new StreamReader(responseStream))
{
Console.WriteLine("Received: " + streamReader.ReadToEnd());
}
}
}
}
App.config
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.7.2" />
</startup>
<system.diagnostics>
<sources>
<source name="System.Net" tracemode="includehex" maxdatasize="1024">
<listeners>
<add name="System.Net"/>
</listeners>
</source>
</sources>
<switches>
<add name="System.Net" value="Verbose"/>
</switches>
<sharedListeners>
<add name="System.Net"
type="System.Diagnostics.TextWriterTraceListener"
initializeData="network.log"
traceOutputOptions="ProcessId, DateTime"
/>
</sharedListeners>
<trace autoflush="true"/>
</system.diagnostics>
</configuration>
Is there any way to log all network traffic on Mono from the application itself? I do not want to run an external program to capture the traffic.
I realise this question has been asked a hundred times, but sadly I still have the same issue. So I've converted someone's Vb.Net solution on here to C#, but I get as far as the smtpClient.Send and my application hangs until it times out. I've allowed the exe through the firewall, so that isn't the issue. Any assistance in what I've missed/got wrong would be greatly appreciated. Here's my code
MailMessage mailMsg = new MailMessage();
mailMsg.To.Add("a#b.com");
// From
MailAddress mailAddress = new MailAddress("x#y.com");
mailMsg.From = mailAddress;
// Subject and Body
mailMsg.Subject = "subject";
mailMsg.Body = "body";
SmtpClient smtpClient = new SmtpClient("smtp.virginmedia.com", 465);
System.Net.NetworkCredential credentials =
new System.Net.NetworkCredential("x#y.com", "MyPassword");
smtpClient.EnableSsl = true;
smtpClient.Credentials = credentials;
//smtpClient.
smtpClient.Send(mailMsg);
Cheers in advance
Robbo
Add the following to your app.config.
Run the applicatin again. Now, you will see exactly where the timeout occurs in the log. If it's during connect, you're blocked by a firewall or using the wrong port. But maybe it's happing at another time?
<system.diagnostics>
<sources>
<source name="System.Net">
<listeners>
<add name="TraceFile"/>
<add name="TraceConsole"/>
</listeners>
</source>
<source name="System.Net.Sockets">
<listeners>
<add name="TraceFile"/>
</listeners>
</source>
</sources>
<sharedListeners>
<add name="TraceFile" type="System.Diagnostics.TextWriterTraceListener" initializeData="trace.log"/>
<add name="TraceConsole" type="System.Diagnostics.ConsoleTraceListener"/>
</sharedListeners>
<switches>
<add name="System.Net" value="Verbose"/>
<add name="System.Net.Sockets" value="Verbose"/>
</switches>
<trace autoflush="true"/>
</system.diagnostics>
Related: Make Https call using HttpClient
Here's an example that illustrates how I'm doing Web Service calls:
class Program
{
static void Main(string[] args)
{
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
var client = new HttpClient();
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue(#"text/javascript") { CharSet = "utf-8" });
client.BaseAddress = new Uri("https://...");
HttpResponseMessage msg = client.GetAsync("jobs").Result;
string json = msg.Content.ReadAsStringAsync().Result;
// Deserialize this for later use
}
}
As I understand it from the answers there, as well as from this, this will establish a secure TLS connection and messages will be encrypted.
My understanding from the Wikipedia description of TLS 1.2 as well as the previously-mentioned documents and Q&A is that it uses public-key cryptography to authenticate the identity of the server and AES to do the exchange. However, Wikipedia's vague as to the exact details (e.g. the AES key size and mode, how the key is exchanged and stored, etc.). How is this handled in HttpClient? What AES key size and mode is it using (if it is, in fact, using AES), how does it do the key exchange, and how does it store the key?
To summarize the comments (as a Community Wiki answer since I'm summarizing other people's material), the content will be encrypted, but the exact details of the encryption are negotiated with the server (meaning that there's not one correct answer - it depends on configuration).
Microsoft has instructions on how to find out details here. They suggest adding the following to your App.config:
<system.diagnostics>
<sources>
<source name="System.Net" tracemode="includehex" maxdatasize="1024">
<listeners>
<add name="System.Net"/>
</listeners>
</source>
<source name="System.Net.Cache">
<listeners>
<add name="System.Net"/>
</listeners>
</source>
<source name="System.Net.Http">
<listeners>
<add name="System.Net"/>
</listeners>
</source>
<source name="System.Net.Sockets">
<listeners>
<add name="System.Net"/>
</listeners>
</source>
<source name="System.Net.WebSockets">
<listeners>
<add name="System.Net"/>
</listeners>
</source>
</sources>
<switches>
<add name="System.Net" value="Verbose"/>
<add name="System.Net.Cache" value="Verbose"/>
<add name="System.Net.Http" value="Verbose"/>
<add name="System.Net.Sockets" value="Verbose"/>
<add name="System.Net.WebSockets" value="Verbose"/>
</switches>
<sharedListeners>
<add name="System.Net"
type="System.Diagnostics.TextWriterTraceListener"
initializeData="network.log"
/>
</sharedListeners>
<trace autoflush="true"/>
</system.diagnostics>
This results in highly detailed logs about the connection (including very specific details about what encryption schemes are being used). For example, I was able to discover that the AES key length is 128 bits, that they're using Elliptic Curve Diffie-Hellman for the key agreement, and various other details of the security.
I have a method that does a POST like below
var response = await client.PostAsJsonAsync(url, entity);
if (response.IsSuccessStatusCode)
{
// read the response as strongly typed object
return await response.Content.ReadAsAsync<T>();
}
My question is how can I obtain the actual JSON that got posted from the entity object. I would like to log the JSON that gets POSTED, so it will be nice to have that without me having to do a json serialize myself.
An example of how you could do this:
Some notes:
LoggingHandler intercepts the request before it handles it to HttpClientHandler which finally writes to the wire.
PostAsJsonAsync extension internally creates an ObjectContent and when ReadAsStringAsync() is called in the LoggingHandler, it causes the formatter
inside ObjectContent to serialize the object and that's the reason you are seeing the content in json.
Logging handler:
public class LoggingHandler : DelegatingHandler
{
public LoggingHandler(HttpMessageHandler innerHandler)
: base(innerHandler)
{
}
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
Console.WriteLine("Request:");
Console.WriteLine(request.ToString());
if (request.Content != null)
{
Console.WriteLine(await request.Content.ReadAsStringAsync());
}
Console.WriteLine();
HttpResponseMessage response = await base.SendAsync(request, cancellationToken);
Console.WriteLine("Response:");
Console.WriteLine(response.ToString());
if (response.Content != null)
{
Console.WriteLine(await response.Content.ReadAsStringAsync());
}
Console.WriteLine();
return response;
}
}
Chain the above LoggingHandler with HttpClient:
HttpClient client = new HttpClient(new LoggingHandler(new HttpClientHandler()));
HttpResponseMessage response = client.PostAsJsonAsync(baseAddress + "/api/values", "Hello, World!").Result;
Output:
Request:
Method: POST, RequestUri: 'http://kirandesktop:9095/api/values', Version: 1.1, Content: System.Net.Http.ObjectContent`1[
[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Headers:
{
Content-Type: application/json; charset=utf-8
}
"Hello, World!"
Response:
StatusCode: 200, ReasonPhrase: 'OK', Version: 1.1, Content: System.Net.Http.StreamContent, Headers:
{
Date: Fri, 20 Sep 2013 20:21:26 GMT
Server: Microsoft-HTTPAPI/2.0
Content-Length: 15
Content-Type: application/json; charset=utf-8
}
"Hello, World!"
NOTE: This works with .NET Framework ONLY!
May be working with .NET 7+ (https://github.com/dotnet/runtime/issues/23937)
See http://mikehadlow.blogspot.com/2012/07/tracing-systemnet-to-debug-http-clients.html
To configure a System.Net listener to output to both the console and a log file, add the following to your assembly configuration file:
<system.diagnostics>
<trace autoflush="true" />
<sources>
<source name="System.Net">
<listeners>
<add name="MyTraceFile"/>
<add name="MyConsole"/>
</listeners>
</source>
</sources>
<sharedListeners>
<add
name="MyTraceFile"
type="System.Diagnostics.TextWriterTraceListener"
initializeData="System.Net.trace.log" />
<add name="MyConsole" type="System.Diagnostics.ConsoleTraceListener" />
</sharedListeners>
<switches>
<add name="System.Net" value="Verbose" />
</switches>
</system.diagnostics>
Network tracing also available for next objects (see article on msdn)
System.Net.Sockets
Some public methods of the Socket, TcpListener, TcpClient, and Dns classes
System.Net
Some public methods of the HttpWebRequest, HttpWebResponse, FtpWebRequest, and FtpWebResponse classes, and SSL debug information (invalid certificates, missing issuers list, and client certificate errors.)
System.Net.HttpListener
Some public methods of the HttpListener, HttpListenerRequest, and HttpListenerResponse classes.
System.Net.Cache
Some private and internal methods in System.Net.Cache.
System.Net.Http
Some public methods of the HttpClient, DelegatingHandler, HttpClientHandler, HttpMessageHandler, MessageProcessingHandler, and WebRequestHandler classes.
System.Net.WebSockets.WebSocket
Some public methods of the ClientWebSocket and WebSocket classes.
Put the following lines of code to the configuration file:
<configuration>
<system.diagnostics>
<sources>
<source name="System.Net" tracemode="includehex" maxdatasize="1024">
<listeners>
<add name="System.Net"/>
</listeners>
</source>
<source name="System.Net.Cache">
<listeners>
<add name="System.Net"/>
</listeners>
</source>
<source name="System.Net.Http">
<listeners>
<add name="System.Net"/>
</listeners>
</source>
<source name="System.Net.Sockets">
<listeners>
<add name="System.Net"/>
</listeners>
</source>
<source name="System.Net.WebSockets">
<listeners>
<add name="System.Net"/>
</listeners>
</source>
</sources>
<switches>
<add name="System.Net" value="Verbose"/>
<add name="System.Net.Cache" value="Verbose"/>
<add name="System.Net.Http" value="Verbose"/>
<add name="System.Net.Sockets" value="Verbose"/>
<add name="System.Net.WebSockets" value="Verbose"/>
</switches>
<sharedListeners>
<add name="System.Net"
type="System.Diagnostics.TextWriterTraceListener"
initializeData="network.log"
/>
</sharedListeners>
<trace autoflush="true"/>
</system.diagnostics>
</configuration>
The easiest solution would be to use Wireshark and trace the HTTP tcp flow.
I'm new with TraceSource so I'm doing some investigation into how it can/ can't be used (basically pros and cons).
Something I do like is that I can get dumps from within the .NET framework itself, so I've made a little app to test that and using my own custom source together (as that's how I'd expect it to be used), like so:
class Program
{
static void Main(string[] args)
{
SmtpClient smtp = new SmtpClient();
var mm = new MailMessage();
mm.To.Add("me#my-site.com");
mm.Subject = "Trace Testing";
smtp.Send(mm);
var ts = new TraceSource("MyCustomTracer");
ts.TraceEvent(TraceEventType.Error, 0, "This is an error");
ts.TraceEvent(TraceEventType.Information, 0, "Just debugging now");
}
}
I've then added some listeners into the App.config like this:
<system.diagnostics>
<trace autoflush="true" />
<sources>
<source name="MyCustomTracer"
switchValue="Information, ActivityTracing">
<listeners>
<add name="sdt"
type="System.Diagnostics.XmlWriterTraceListener"
initializeData= "traceOutput.log" />
</listeners>
</source>
<source name="System.Net"
switchValue="Information, ActivityTracing, Critical">
<listeners>
<add name="sdt"
type="System.Diagnostics.XmlWriterTraceListener"
initializeData= "traceOutput.log" />
</listeners>
</source>
</sources>
</system.diagnostics>
But for some reason when I run the app the 2 events I'm logging via MyCustomTracer aren't going into the log file unless I comment out the SmtpClient stuff (ie - only have my custom tracer used).
I would have expected that multiple TraceSources can be used in the manner in which I'm trying to use them, I'm just not sure what's going wrong.
Found the problem, a complete noob mistake, both my TraceSource items have a Listener which is writing to the same file. Although I'm not sure exactly the error, but it'd be some kind of clash when writing.
If you want to have multiple sources using the same listener you need to use the <sharedListeners /> like this:
<system.diagnostics>
<trace autoflush="true" />
<sources>
<source name="MyCustomTracer"
switchValue="Information, ActivityTracing">
<listeners>
<add name="sdt" />
</listeners>
</source>
<source name="System.Net"
switchValue="Information, ActivityTracing, Critical">
<listeners>
<add name="sdt" />
</listeners>
</source>
</sources>
<sharedListeners>
<add name="sdt"
type="System.Diagnostics.XmlWriterTraceListener"
initializeData= "traceOutput.log" />
</sharedListeners>
</system.diagnostics>