WebRequest.GetRequestStream and LOH - c#

I am using code below to upload large file to server and noticed that copying FileStream to GetRequestStream the bytes array is created and hold in memory. This increase large object heap and I don't want it. Maybe someone know how to solve this?
Stream formData = new FileStream(.....)
HttpWebRequest request = WebRequest.Create(postUrl) as HttpWebRequest;
using (Stream requestStream = request.GetRequestStream())
{
Helpers.CopyStream(formData, requestStream);
requestStream.Close();
}
public static void CopyStream(Stream fromStream, Stream toStream)
{
try
{
int bytesRead;
byte[] buffer = new byte[32768];
while (fromStream != null && (bytesRead = fromStream.Read(buffer, 0, buffer.Length)) > 0)
{
toStream.Write(buffer, 0, bytesRead);
}
}
catch (IOException)
{
//suppress empty stream response
}
}
Memory profiler graph. bytes array create internally in GetRequestStream

You can use the HttpWebRequest.AllowWriteStreamBuffering to disable internal buffering:
HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
request.AllowWriteStreamBuffering = false;
using (Stream formData = File.Open(fileName, FileMode.Open))
using (Stream requestStream = request.GetRequestStream())
{
formData.CopyTo(requestStream, 32768);
}

Related

Reading data from System.IO.Stream to text or a file

I am using HttpWebRequest.GetStreamData() to get data from stream and then writing a couple of files in that variable.
Before calling the GetResponse() method on the stream, I need to get the copy of data and put it in text format.
Here's my code snippet
HttpWebRequest wr = (HttpWebRequest)WebRequest.Create(url);
wr.Method = "POST";
string requestbody = "Some Body Text"
Stream rs = wr.GetRequestStream();
foreach (fragment in Documents).ToList())
{
byte[] requestBytes = System.Text.Encoding.UTF8.GetBytes(requestbody);
rs.Write(requestBytes, 0, requestBytes.Length);
FileStream fileStream = new FileStream(fragment.FilePath, FileMode.Open, FileAccess.Read);
byte[] buffer = new byte[4096];
int bytesRead = 0;
while ((bytesRead = fileStream.Read(buffer, 0, buffer.Length)) != 0)
{
rs.Write(buffer, 0, bytesRead);
}
fileStream.Flush();
fileStream.Close();
fileStream = null;
byte[] trailer = System.Text.Encoding.ASCII.GetBytes("Trailer Text");
rs.Write(trailer, 0, trailer.Length);
}
}
using (StreamReader strm = new StreamReader(rs))
{
string text = strm.ReadToEnd();
}
rs.Close();
rs = null;
The issue is I get error "stream was not readable".
When I check the property rs.canread, it is set to false.
Can anyone help me reading the content of my rs stream.
I have tried a couple of methods but none seems to be working as of now.

Exporting byte[] results to corrupted .zip

I am trying to make a CLR with .NET 2.0 integrated into MS SQL Server 2008. I call an API with and I should receive a .zip as response. I store the response into a Stream and I want to export this file to a physical .zip file.
I tried exporting the file with C# and SQL (BCB OR OLE) and all resulted into a corrupted file. So, I believe I am doing something wrong with in the making of the stream.
The C# code is the following:
private static byte[] GetStreamFileResult(Cookie loginCookie, Guid fileGuid, String baseUri)
{
byte[] output = null;
//String result = null;
string url = "some url"
CookieContainer cookies = new CookieContainer();
cookies.Add(new Uri(url), loginCookie);
WebRequest request = WebRequest.Create(url);
(request as HttpWebRequest).CookieContainer = cookies;
WebResponse response = request.GetResponse();
HttpWebResponse resp = response as HttpWebResponse;
Stream dataStream = response.GetResponseStream();
using (MemoryStream ms = new MemoryStream())
{
CopyStream(dataStream, ms);
output = ms.ToArray();
}
dataStream.Close();
response.Close();
return output;
}
The C# code to export the zip is the following:
File.WriteAllBytes("C:\\folder\\t.zip", stream); // Requires System.IO
The copy from stream to stream:
public static void CopyStream(Stream input, Stream output)
{
if (input != null)
{
using (StreamReader reader = new StreamReader(input))
using (StreamWriter writer = new StreamWriter(output))
{
writer.Write(reader.ReadToEnd());
}
}
}
Your CopyStream is broken. You need to talk binary. You're currently treating binary zip data as though it were text:
byte[] buffer = new byte[2048];
int bytesRead;
while((bytesRead = input.Read(buffer, 0, buffer.Length)) > 0) {
output.Write(buffer, 0, bytesRead);
}

Use fileinfo in stream

This is my code am trying to make a stream from the url and and i want to cut file into 6 pieces and download , because of that i want to connect the stream to file info but am getting some errors. thanks
FileInfo fs;
DateTime startTime = DateTime.UtcNow;
WebRequest request = WebRequest.Create("http://tegos.ru/new/mp3_full/David_Guetta_feat_Ne-Yo_and_Akon_-_Play_Hard.mp3");
WebResponse response = request.GetResponse();
using (Stream responseStream = response.GetResponseStream())
using (Stream fileStream = fs new FileInfo("")))
error 'System.IO.FileInfo' is a 'type' but is used like a 'variable'
Assuming you want to copy the stream from your HTTP response to a file:
using (Stream output = File.OpenWrite("pah_to_file"))
{
using (Stream input = http.Response.GetResponseStream())
{
byte[] buffer = new byte[2048]; // some buffer
int bytesRead;
while ((bytesRead = input.Read(buffer, 0, buffer.Length)) > 0)
{
output.Write(buffer, 0, bytesRead);
}
}
}
If it's not the case, do rephrase your question.

Download a file and read from it in the same time

I download a file with this method:
WebClient WC = new WebClient();
WC.DownloadFile(url, filePath);
And i want that in the same time to read the file in the same time with:
var fs = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.None);
but i get allways :
The process cannot access the file 'filePath' because it is being used by another process.
It's possible to download and read in the same time?
Edit
I now download the file with:
var req = (HttpWebRequest)HttpWebRequest.Create(url);
var fileStream = new FileStream(filePath,
FileMode.Create, FileAccess.ReadWrite, FileShare.ReadWrite);
using (var resp = req.GetResponse())
{
using (var stream = resp.GetResponseStream())
{
byte[] buffer = new byte[0x10000];
int len;
while ((len = stream.Read(buffer, 0, buffer.Length)) > 0)
{
fileStream.Write(buffer, 0, len);
}
}
}
And still get the error....
var req = (HttpWebRequest)HttpWebRequest.Create(url);
using (var resp = req.GetResponse())
{
using (var stream = resp.GetResponseStream())
{
byte[] buffer = new byte[0x10000];
int len;
while ((len = stream.Read(buffer, 0, buffer.Length))>0)
{
//Do with the content whatever you want
// ***YOUR CODE***
fileStream.Write(buffer, 0, len);
}
}
}
Try this way :
HttpWebRequest request = (HttpWebRequest) WebRequest.Create("http://www.domain.com/file.txt");
HttpWebResponse response = (HttpWebResponse) request.GetResponse();
var stream = response.GetResponseStream();
byte[] buffer = new byte[8192];
int bytesRead = 0;
while ((bytesRead = stream.Read(buffer, 0, 8192)) > 0)
{
// handle your raw data
// you needed raw data = buffer[0 to bytesRead - 1]
}
stream.Close();
response.Close();

How to upload a file on server in c# fastly than this

Currently i am using following code , is there any better (means fast) way to upload a file, here is my complete code, it is called at each file upload:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("request_uri_string");
FileStream fileStream = new FileStream("path_to_my_file",
FileMode.Open, FileAccess.Read);
Stream requestStream = request.GetRequestStream();
byte[] buffer = new Byte[checked((uint)Math.Min(4096,
(int)fileStream.Length))];
int bytesRead = 1;
while (bytesRead != 0)
{
bytesRead = fileStream.Read(buffer, 0, buffer.Length);
if (bytesRead > 0)
{
requestStream.Write(buffer, 0, bytesRead);
}
}
requestStream.Close();
fileStream.Close();
String responseFromServer = null;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
if (((HttpWebResponse)response).StatusDescription.Contains("OK"))
{
Encoding encode = System.Text.Encoding.GetEncoding(((HttpWebResponse)response).CharacterSet);
Stream dataStream = response.GetResponseStream();
StreamReader reader = new StreamReader(dataStream, encode);
responseFromServer = reader.ReadToEnd();
reader.Close();
dataStream.Close();
}
response.Close();
Probably not faster, but easier:
WebClient client = new WebClient();
//client.Credentials = new NetworkCredential("username", "password");
client.UploadFile(uriString,fileName);

Categories

Resources