My company run an application that have to archive many kinds of files into some distants servers. The application works well but can't handle files larger than 1GB.
Here is the current function use to load the files to be uploaded to the distant server :
FileStream fs = File.OpenRead(fileToUploadPath);
byte[] fileArray = new byte[fs.Length];
fs.Read(fileArray, 0, fs.Length);
The byte array (when loaded successfully) was then splited into 100Mb bytes arrays and sent to the local server (using some WSDL web services) with the following function :
localServerWebService.SendData(subFileArray, filename);
I changed the function responsible for the file reading to use BufferendStream and I also wanted to improve the Webservice part so that it doesn't have to create a new stream at each call. I thought of somethings like this :
FileInfo source = new FileInfo(fileName);
using (FileStream reader = File.OpenRead(fileName))
{
using (FileStream distantWriter = localServerWebService.CreateWriteFileStream(fileName))
{
using (BufferedStream buffReader = new BufferedStream(reader))
{
using (BufferedStream buffWriter = new BufferedStream(distantWriter))
{
byte[] buffer = new byte[BUFFER_SIZE];
int bytesRead = 0;
long bytesToRead = source.Length;
while (bytesToRead > 0)
{
int nbBytesRead = buffReader.Read(buffer, 0, buffer.Length);
buffWriter.Write(buffer, 0, nbBytesRead);
bytesRead += nbBytesRead;
bytesToRead -= nbBytesRead;
}
}
}
}
}
But this code can't compile and always give me the error Cannot convert MyNameSpace.FileStream into System.IO.FileStream at line using (FileStream distantWriter = localServerWebService.CreateWriteFileStream(fileName)). I can't cast MyNameSpace.FileStream into System.IO.FileStream either.
The web service method :
[WebMethod]
public FileStream CreateWriteFileStream(String fileName)
{
String RepVaultUP =
System.Configuration.ConfigurationSettings.AppSettings.Get("SAS_Upload");
String desFile = Path.Combine(RepVaultUP, fileName);
return File.Open(desFile, FileMode.Create, FileAccess.Write);
}
So can you guys please explain to me why is this not working?
P.S.: English is not my mothertong so I hope what i wrote is clearly undestandable.
Related
From SharePoint, I get a "Stream" for a file. I want to copy this entire file from the internet to a local file on the PC, but I want to have status while this download is occurring. How? FileStream and StreamReader seem to have bytes vs char differences when not doing a full "CopyTo" which doesn't have progress updates. There has to be a cleaner way...
using (var sr = new StreamReader(fileData.Value))
{
using (FileStream fs = new FileStream(localFile + "_tmp", FileMode.Create))
{
byte[] block = new byte[1024];
// only guessing something like this is necessary...
int count = (int)Math.Min(sr.BaseStream.Length - sr.BaseStream.Position, block.Length);
while (sr.BaseStream.Position < sr.BaseStream.Length)
{
// read requires char[]
sr.Read(block, 0, count);
// write requires byte[]
fs.Write(block, 0, count);
Log("Percent complete: " + (sr.BaseStream.Position / sr.BaseStream.Length));
count = (int)Math.Min(sr.BaseStream.Length - sr.BaseStream.Position, block.Length);
}
}
}
Just had to use BinaryReader instead of StreamReader. Easy Peasy.
using (Stream fileStream = System.IO.File.OpenWrite("D:\sample.svg"))
{
byte[] buffer = new byte[8 * 1024];
int len;
while ((len = SvgStream.Read(buffer, 0, buffer.Length)) > 0)
{
fileStream.Write(buffer, 0, len);
}
}
I have this code for converting a PPT to SVG. SvgStream contains the PPT Slides. I don't want to store the converted SVG file on a physical path like D:\sample.svg. Is it possible to store it on an object that is not a physical path?
Is it possible to store it on an object that is not a physical path?
Sure. Create a MemoryStream instead of a FileStream if you want to stream into memory rather than the file system.
And while I'm here: you might want to use the CopyTo method rather than this tedious business of buffering it over one page at a time.
I saved it to a string:
SvgStream = new MemoryStream();
sld.WriteAsSvg(SvgStream);
SvgStream.Position = 0;
string[] svgArray = new string[MAX];
//instead of using(var fileStream = File.IO.......
//I came up with this:
var sr = new StreamReader(SvgStream);
svgArray[i] = sr.ReadToEnd();
I am attempting to create a new FileStream object from a byte array. I'm sure that made no sense at all so I will try to explain in further detail below.
Tasks I am completing:
1) Reading the source file which was previously compressed
2) Decompressing the data using GZipStream
3) copying the decompressed data into a byte array.
What I would like to change:
1) I would like to be able to use File.ReadAllBytes to read the decompressed data.
2) I would then like to create a new filestream object usingg this byte array.
In short, I want to do this entire operating using byte arrays. One of the parameters for GZipStream is a stream of some sort, so I figured I was stuck using a filestream. But, if some method exists where I can create a new instance of a FileStream from a byte array - then I should be fine.
Here is what I have so far:
FolderBrowserDialog fbd = new FolderBrowserDialog(); // Shows a browser dialog
fbd.ShowDialog();
// Path to directory of files to compress and decompress.
string dirpath = fbd.SelectedPath;
DirectoryInfo di = new DirectoryInfo(dirpath);
foreach (FileInfo fi in di.GetFiles())
{
zip.Program.Decompress(fi);
}
// Get the stream of the source file.
using (FileStream inFile = fi.OpenRead())
{
//Create the decompressed file.
string outfile = #"C:\Decompressed.exe";
{
using (GZipStream Decompress = new GZipStream(inFile,
CompressionMode.Decompress))
{
byte[] b = new byte[blen.Length];
Decompress.Read(b,0,b.Length);
File.WriteAllBytes(outfile, b);
}
}
}
Thanks for any help!
Regards,
Evan
It sounds like you need to use a MemoryStream.
Since you don't know how many bytes you'll be reading from the GZipStream, you can't really allocate an array for it. You need to read it all into a byte array and then use a MemoryStream to decompress.
const int BufferSize = 65536;
byte[] compressedBytes = File.ReadAllBytes("compressedFilename");
// create memory stream
using (var mstrm = new MemoryStream(compressedBytes))
{
using(var inStream = new GzipStream(mstrm, CompressionMode.Decompress))
{
using (var outStream = File.Create("outputfilename"))
{
var buffer = new byte[BufferSize];
int bytesRead;
while ((bytesRead = inStream.Read(buffer, 0, BufferSize)) != 0)
{
outStream.Write(buffer, 0, bytesRead);
}
}
}
}
Here is what I ended up doing. I realize that I did not give sufficient information in my question - and I apologize for that - but I do know the size of the file I need to decompress as I am using it earlier in my program. This buffer is referred to as "blen".
string fi = #"C:\Path To Compressed File";
// Get the stream of the source file.
// using (FileStream inFile = fi.OpenRead())
using (MemoryStream infile1 = new MemoryStream(File.ReadAllBytes(fi)))
{
//Create the decompressed file.
string outfile = #"C:\Decompressed.exe";
{
using (GZipStream Decompress = new GZipStream(infile1,
CompressionMode.Decompress))
{
byte[] b = new byte[blen.Length];
Decompress.Read(b,0,b.Length);
File.WriteAllBytes(outfile, b);
}
}
}
I have some code that downloads gzipped files, and decompresses them. The problem is, I can't get it to decompress the whole file, it only reads the first 4096 bytes and then about 500 more.
Byte[] buffer = new Byte[4096];
int count = 0;
FileStream fileInput = new FileStream("input.gzip", FileMode.Open, FileAccess.Read, FileShare.Read);
FileStream fileOutput = new FileStream("output.dat", FileMode.Create, FileAccess.Write, FileShare.None);
GZipStream gzipStream = new GZipStream(fileInput, CompressionMode.Decompress, true);
// Read from gzip steam
while ((count = gzipStream.Read(buffer, 0, buffer.Length)) > 0)
{
// Write to output file
fileOutput.Write(buffer, 0, count);
}
// Close the streams
...
I've checked the downloaded file; it's 13MB when compressed, and contains one XML file. I've manually decompressed the XML file, and the content is all there. But when I do it with this code, it only outputs the very beginning of the XML file.
Anyone have any ideas why this might be happening?
EDIT
Try not leaving the GZipStream open:
GZipStream gzipStream = new GZipStream(fileInput, CompressionMode.Decompress,
false);
or
GZipStream gzipStream = new GZipStream(fileInput, CompressionMode.Decompress);
I ended up using a gzip executable to do the decompression instead of a GZipStream. It can't handle the file for some reason, but the executable can.
Same thing happened to me. In my case only reads up to 6 lines and then reached end of file. So I realized that although the extension is gz, it was compressed by another algorithm not supported by GZipStream. So I used SevenZipSharp library and it worked. This is my code
You can use SevenZipSharp library
using (var input = File.OpenRead(lstFiles[0]))
{
using (var ds = new SevenZipExtractor(input))
{
//ds.ExtractionFinished += DsOnExtractionFinished;
var mem = new MemoryStream();
ds.ExtractFile(0, mem);
using (var sr = new StreamReader(mem))
{
var iCount = 0;
String line;
mem.Position = 0;
while ((line = sr.ReadLine()) != null && iCount < 100)
{
iCount++;
LstOutput.Items.Add(line);
}
}
}
}
Are you calling Close or Flush on fileOutput? (Or just wrap it in a using, which is recommended practice.) If you don't the file might not be flushed to disk when your program ends.
I am wanting to write a WCF web service that can send files over the wire to the client. So I have one setup that sends a Stream response. Here is my code on the client:
private void button1_Click(object sender, EventArgs e)
{
string filename = System.Environment.CurrentDirectory + "\\Picture.jpg";
if (File.Exists(filename))
File.Delete(filename);
StreamServiceClient client = new StreamServiceClient();
int length = 256;
byte[] buffer = new byte[length];
FileStream sink = new FileStream(filename, FileMode.CreateNew, FileAccess.Write);
Stream source = client.GetData();
int bytesRead;
while ((bytesRead = source.Read(buffer,0,length))> 0)
{
sink.Write(buffer,0,length);
}
source.Close();
sink.Close();
MessageBox.Show("All done");
}
Everything processes fine with no errors or exceptions. The problem is that the .jpg file that is getting transferred is reported as being "corrupted or too large" when I open it.
What am I doing wrong?
On the server side, here is the method that is sending the file.
public Stream GetData()
{
string filename = Environment.CurrentDirectory+"\\Chrysanthemum.jpg";
FileStream myfile = File.OpenRead(filename);
return myfile;
}
I have the server configured with basicHttp binding with Transfermode.StreamedResponse.
I think the problem is this:
while ((bytesRead = source.Read(buffer,0,length))> 0)
{
sink.Write(buffer,0,length);
}
Imagine you're reading the last bit of your file - maybe it's not 256 bytes anymore, but only 10.
You read those last 10 bytes, bytesRead will be 10, but on the sink.Write operation, you're still using the fixed value length (256). So when you read the last block of data, you're writing out a block that might be too large.
You need to change the line for the sink to:
sink.Write(buffer, 0, bytesRead);
and then it should work.