Develop Reference

Correct way to use GZipStream in dotNET C#

I'm working with GZipStream at the moment using .net 3.5.
I have two methods listed below. As input file I use text file which consists of chars 's'. Size of the file is 2MB. This code works fine if I use .net 4.5 but with .net 3.5 after compress and decompress I get file of size 435KB which of course isn't the same with source file.
If I try to decompress file via WinRAR it is also looks good (the same with source file).
If I try decompress file using GZipStream from .net4.5 (file compressed via GZipStream from .net 3.5) the result is bad.
UPD:
In general I really need to read the file as several separate gzip chunks, in this case all the bytes of copressed files are read at one call of the Read() method so I still don't understand why decompressing doesn't works.
public void CompressFile()
{
string fileIn = #"D:\sin2.txt";
string fileOut = #"D:\sin2.txt.pgz";
using (var fout = File.Create(fileOut))
{
using (var fin = File.OpenRead(fileIn))
{
using (var zip = new GZipStream(fout, CompressionMode.Compress))
{
var buffer = new byte[1024 * 1024 * 10];
int n = fin.Read(buffer, 0, buffer.Length);
zip.Write(buffer, 0, n);
}
}
}
}
public void DecompressFile()
{
string fileIn = #"D:\sin2.txt.pgz";
string fileOut = #"D:\sin2.1.txt";
using (var fsout = File.Create(fileOut))
{
using (var fsIn = File.OpenRead(fileIn))
{
var buffer = new byte[1024 * 1024 * 10];
int n;
while ((n = fsIn.Read(buffer, 0, buffer.Length)) > 0)
{
using (var ms = new MemoryStream(buffer, 0, n))
{
using (var zip = new GZipStream(ms, CompressionMode.Decompress))
{
int nRead = zip.Read(buffer, 0, buffer.Length);
fsout.Write(buffer, 0, nRead);
}
}
}
}
}
}

You're trying to decompress each "chunk" as if it's a separate gzip file. Don't do that - just read from the GZipStream in a loop:
using (var fsout = File.Create(fileOut))
{
using (var fsIn = File.OpenRead(fileIn))
{
using (var zip = new GZipStream(fsIn, CompressionMode.Decompress))
{
var buffer = new byte[1024 * 32];
int bytesRead;
while ((bytesRead = zip.Read(buffer, 0, buffer.Length)) > 0)
{
fsout.Write(buffer, 0, bytesRead);
}
}
}
}
Note that your compression code should look similar, reading in a loop rather than assuming a single call to Read will read all the data.
(Personally I'd skip fsIn, and just use new GZipStream(File.OpenRead(fileIn)) but that's just a personal preference.)

First, as #Jon Skeet mentioned, you are not using Stream.Read method correctly. It doesn't matter if your buffer is big enough or not, the stream is allowed to return less bytes than requested, with zero indicating no more, so reading from stream should always be performed in a loop.
However the main problem in your decompress code is the way you share the buffer. Your read the input into a buffer, than wrap it in a MemoryStream (note that the constructor used does not make a copy of the passed array, but actually sets it as it's internal buffer), and then you try to read and write to that buffer at the same time. Taking into account that decompressing writes data "faster" than reading, it's surprising that your code works at all.
The correct implementation is quite simple
static void CompressFile()
{
string fileIn = #"D:\sin2.txt";
string fileOut = #"D:\sin2.txt.pgz";
using (var input = File.OpenRead(fileIn))
using (var output = new GZipStream(File.Create(fileOut), CompressionMode.Compress))
Write(input, output);
}
static void DecompressFile()
{
string fileIn = #"D:\sin2.txt.pgz";
string fileOut = #"D:\sin2.1.txt";
using (var input = new GZipStream(File.OpenRead(fileIn), CompressionMode.Decompress))
using (var output = File.Create(fileOut))
Write(input, output);
}
static void Write(Stream input, Stream output, int bufferSize = 10 * 1024 * 1024)
{
var buffer = new byte[bufferSize];
for (int readCount; (readCount = input.Read(buffer, 0, buffer.Length)) > 0;)
output.Write(buffer, 0, readCount);
}

C# - Decrypt large files in memory using BouncyCastle

I have a requirement where I need to decrypt large files in memory because these files contain sensitive data (SSNs, DOBs, etc). In other words the decrypted data cannot be at rest (on disk). I was able to use the BouncyCastle API for C# and managed to make it to work for files up to 780 MB. Basically here's the code that works:
string PRIVATE_KEY_FILE_PATH = #"c:\pgp\privatekey.gpg";
string PASSPHRASE = "apassphrase";
string pgpData;
string[] pgpLines;
string pgpFilePath = #"C:\test\test.gpg";
Stream inputStream = File.Open(pgpFilePath, FileMode.Open);
Stream privateKeyStream = File.Open(PRIVATE_KEY_FILE_PATH, FileMode.Open);
string pgpData = CryptoHelper.DecryptPgpData(inputStream, privateKeyStream, PASSPHRASE);
string[] pgpLines = pgpData.Split(new string[] { "\r\n", "\n" }, StringSplitOptions.None);
Console.WriteLine(pgpLines.Length);
Console.ReadLine();
foreach (var x in pgpLines)
{
Console.WriteLine(x);
Console.ReadLine();
}
In the above code the entire decrypted data is stored in the pgpData string and that's fine for files up to 780MB as stated previously.
However, I will be getting much larger files and my solution above does not work as I get OutOfMemoryExeception exception.
I've been trying the code below, and I keep getting errors when the DecryptPgpData method below is invoked. When using 512 as the chunksize I get a "Premature end of stream in PartialInputStream" exception. When using 1024 as the chunksize I get a "Exception starting decryption" exception. My question is, is this the correct way to decerypt chunks of data using BouncyCastle? The pgp file I'm trying to decrypt was encrypted using the gpg.exe utility. Any help is much appreciated.....It's been almost two days that I've been trying to make this to work with no success.
string PRIVATE_KEY_FILE_PATH = #"c:\pgp\privatekey.gpg";
string PASSPHRASE = "apassphrase";
string pgpData;
string[] pgpLines;
string pgpFilePath = #"C:\test\test.gpg";
string decryptedData = string.Empty;
FileInfo inFile = new FileInfo(pgpFilePath);
FileStream fs = null;
fs = inFile.OpenRead();
int chunkSize = 1024;
byte[] buffer = new byte[chunkSize];
int totalRead = 0;
while (totalRead < fs.Length)
{
int readBytes = fs.Read(buffer, 0, chunkSize);
totalRead += readBytes;
Stream stream = new MemoryStream(buffer);
decryptedData = CryptoHelper.DecryptPgpData(stream, privateKeyStream, PASSPHRASE);
Console.WriteLine(decryptedData);
}
fs.Close();
Console.WriteLine(totalRead);
Console.ReadLine();

ASP .Net C#: Get Binaries from fileupoad to generate XML

I am currently at the end of my program but am coming into a section that I have NO experience in. I've been able to convert the PDF to a Base64 in other methods but those are not permitted as per the instructions I've been given. I've posted my code below in the hopes that someone can maybe get me started in the right direction. I do not have any clue how to put in the volumes either so ANY assistance would be awesome! Contained is the Get Binaries and the IO class which it reads.
Default.cs.aspx
private static List<Binary> GetBinaries()
{
return new List<Binary>
{
new Binary
{
//hardcoded but need to call from fileUpload1
BinaryBase64Object = IO.ReadFromFile(#"..\..\EFACTS eRecord Technical Specification.pdf"), **<-- How do I get this to read from fileupload1**
BinaryID = "BIN1234", //hardcoded
BinarySizeValue = 56443, //hardcoded
FileName = " test.my.pdf", //hardcoded
PageRange = "23-89", //hardcoded
NoOfPages = 14, //hardcoded
TotalVolumes = 1, //hardcoded
Volume = 1 //hardcoded
}
};
}
IO.cs
using System;
using System.IO;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Electronic_Filing_of_Appeals;
namespace Electronic_Filing_of_Appeals
{
public static class IO
{
public static void ReadWriteStream(MemoryStream readStream, Stream writeStream)
{
using (writeStream)
{
int Length = 256;
Byte[] buffer = new Byte[Length];
readStream.Position = 0;
int bytesRead = readStream.Read(buffer, 0, Length);
// write the required bytes
while (bytesRead > 0)
{
writeStream.Write(buffer, 0, bytesRead);
bytesRead = readStream.Read(buffer, 0, Length);
}
readStream.Close();
writeStream.Close();
}
}
public static MemoryStream FileToMemoryStream(string filename)
{
FileStream inStream = System.IO.File.OpenRead(filename);
MemoryStream outStream = new MemoryStream();
outStream.SetLength(inStream.Length);
inStream.Read(outStream.GetBuffer(), 0, (int)inStream.Length);
outStream.Flush();
inStream.Close();
return outStream;
}
public static MemoryStream ConvertStreamToMemoryStream(Stream stream)
{
MemoryStream memoryStream = new MemoryStream();
if (stream != null)
{
byte[] buffer = stream.ReadFully();
if (buffer != null)
{
var binaryWriter = new BinaryWriter(memoryStream);
binaryWriter.Write(buffer);
}
}
return memoryStream;
}
public static byte[] ReadFromFile(string filePath)
{
FileStream fs = new FileStream(filePath, FileMode.Open, FileAccess.Read);
BinaryReader br = new BinaryReader(fs);
byte[] fileRD = br.ReadBytes((int)fs.Length);
br.Close();
fs.Close();
return fileRD;
}
public static void SaveToFile(byte[] byteData, string fileName)
{
FileStream fs = new FileStream(fileName, FileMode.Create);
fs.Write(byteData, 0, byteData.Length);
fs.Close();
}
public static byte[] ReadFully(this Stream input)
{
byte[] buffer = new byte[16 * 1024];
using (MemoryStream ms = new MemoryStream())
{
int read;
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
return ms.ToArray();
}
}
}
}

You should ask for some more assistance. There are probably some library for reading PDF-files already in use. Try to find out what it is, and read the code to see how it is used.
If this is the first time in the project anyone had to read PDF-files, you will have to adopt some library. Writing one on your own is very time-consuming. What you are looking for is a library that can read the meta-data from the PDF.
Two such libraries are iTextSharp and PDFSharp. Here are an example with PDFSharp:
var doc = PdfReader.Open(#"icpc_briefing_1_sep_10.pdf");
Console.WriteLine("Number of pages: {0}", doc.PageCount);
Console.WriteLine();
Console.WriteLine("=== INFO ===");
foreach (KeyValuePair<string, PdfItem> pair in doc.Info)
{
Console.WriteLine("{0}: {1}", pair.Key, pair.Value);
}
byte[] xmlData = GetMetadata(doc);
if (xmlData != null)
{
Console.WriteLine();
Console.WriteLine("=== XMP ===");
Console.Write(Encoding.UTF8.GetString(xmlData));
Console.WriteLine();
}
byte[] GetMetadata(PdfDocument doc)
{
// PdfSharp does not have direct support for the XML metadata, but it does
// allow you to go poking into the internal structure.
PdfItem metadataItem;
if (doc.Internals.Catalog.Elements.TryGetValue("/Metadata", out metadataItem))
{
var metadataRef = (PdfReference) metadataItem;
var metadata = (PdfDictionary) metadataRef.Value;
return metadata.Stream.Value;
}
return null;
}
Example output:
Number of pages: 27
=== INFO ===
/CreationDate: D:20100910191635+08'00'
/Author: Steven Halim
/Creator: PScript5.dll Version 5.2.2
/Producer: Acrobat Distiller 8.1.0 (Windows)
/ModDate: D:20130610173442+02'00'
/Title: Microsoft PowerPoint - icpc_briefing_1_sep_10 [Compatibility Mode]
=== XMP ===
<?xpacket begin="?" id="W5M0MpCehiHzreSzNTczkc9d"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 4.0-c316 44.253921, Sun Oct 01 2006 17:14:39">
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<rdf:Description rdf:about=""
xmlns:xap="http://ns.adobe.com/xap/1.0/">
<xap:CreatorTool>PScript5.dll Version 5.2.2</xap:CreatorTool>
<xap:ModifyDate>2010-09-10T19:16:35+08:00</xap:ModifyDate>
<xap:CreateDate>2010-09-10T19:16:35+08:00</xap:CreateDate>
</rdf:Description>
<rdf:Description rdf:about=""
xmlns:dc="http://purl.org/dc/elements/1.1/">
<dc:format>application/pdf</dc:format>
<dc:title>
<rdf:Alt>
<rdf:li xml:lang="x-default">Microsoft PowerPoint - icpc_briefing_1_sep_10 [Compatibility Mode]</rdf:li>
</rdf:Alt>
</dc:title>
<dc:creator>
<rdf:Seq>
<rdf:li>Steven Halim</rdf:li>
</rdf:Seq>
</dc:creator>
</rdf:Description>
<rdf:Description rdf:about=""
xmlns:pdf="http://ns.adobe.com/pdf/1.3/">
<pdf:Producer>Acrobat Distiller 8.1.0 (Windows)</pdf:Producer>
</rdf:Description>
<rdf:Description rdf:about=""
xmlns:xapMM="http://ns.adobe.com/xap/1.0/mm/">
<xapMM:DocumentID>uuid:b5ffc85b-6578-48fb-bddf-a5215ffa9d35</xapMM:DocumentID>
<xapMM:InstanceID>uuid:f45826a4-8c58-4a54-bac1-7f3192866d57</xapMM:InstanceID>
</rdf:Description>
</rdf:RDF>
</x:xmpmeta>
<?xpacket end="w"?>
Here are some more examples: http://www.pdfsharp.net/wiki/PDFsharpSamples.ashx
To get the data from a <asp:FileUpload> control, you just need to access FileUpload.FileBytes (byte[]) or FileUpload.FileContent (Stream). Alternatively, you can access FileUpload.PostedFile or Request.Files for more information.
var doc = PdfReader.Open(FileUpload1.FileContent);

Decompressing gzipstream from

I've made a simple proxy server. Works fine, but not the gzip decompressing:
The magic number is not correct. Be sure .
No error, but the gzipstream won't decompress.
It works as follows:
private void HandleTraffic()
{
while ((bytesRead = realServer.Receive(buffer, 0, BODYBUFFERSIZE, SocketFlags.None)) > 0)
{
decompressedBuffer = new byte[BODYBUFFERSIZE];
if (t == "gzip")
{
MemoryStream outputStream = new MemoryStream(), inputStream = new MemoryStream();
inputStream.Write(buffer, 0, bytesRead);
inputStream.Position = 0;
inputStream.Flush();
using (GZipStream gz = new GZipStream(inputStream, CompressionMode.Decompress))
{
gz.CopyTo(outputStream);
}
decompressedBuffer = outputStream.ToArray();
MessageBox.Show(ASCIIEncoding.ASCII.GetString(decompressedBuffer));
}
}
}
InputStream is filled, but outputStream isn't decompressed or gives me the error 'The magic number is not correct'.
In this job, sockets has been required.

You don't need to process the data in a proxy server in any way whatsoever, unzipping, deflating, whatsoever. Just copy the bytes. The only data you need to even look at is the first line from the client, the HTTP CONNECT command.

The solution to this for myself was to turn off Fiddler.

C# 4.0: Convert pdf to byte[] and vice versa

How do I convert a pdf file to a byte[] and vice versa?

// loading bytes from a file is very easy in C#. The built in System.IO.File.ReadAll* methods take care of making sure every byte is read properly.
// note that for Linux, you will not need the c: part
// just swap out the example folder here with your actual full file path
string pdfFilePath = "c:/pdfdocuments/myfile.pdf";
byte[] bytes = System.IO.File.ReadAllBytes(pdfFilePath);
// munge bytes with whatever pdf software you want, i.e. http://sourceforge.net/projects/itextsharp/
// bytes = MungePdfBytes(bytes); // MungePdfBytes is your custom method to change the PDF data
// ...
// make sure to cleanup after yourself
// and save back - System.IO.File.WriteAll* makes sure all bytes are written properly - this will overwrite the file, if you don't want that, change the path here to something else
System.IO.File.WriteAllBytes(pdfFilePath, bytes);

using (FileStream fs = new FileStream("sample.pdf", FileMode.Open, FileAccess.Read))
{
byte[] bytes = new byte[fs.Length];
int numBytesToRead = (int)fs.Length;
int numBytesRead = 0;
while (numBytesToRead > 0)
{
// Read may return anything from 0 to numBytesToRead.
int n = fs.Read(bytes, numBytesRead, numBytesToRead);
// Break when the end of the file is reached.
if (n == 0)
{
break;
}
numBytesRead += n;
numBytesToRead -= n;
}
numBytesToRead = bytes.Length;
}

Easiest way:
byte[] buffer;
using (Stream stream = new IO.FileStream("file.pdf"))
{
buffer = new byte[stream.Length - 1];
stream.Read(buffer, 0, buffer.Length);
}
using (Stream stream = new IO.FileStream("newFile.pdf"))
{
stream.Write(buffer, 0, buffer.Length);
}
Or something along these lines...