I was looking around for a way to copy a portion of a file from a determined address to another address, is there a way to do that in C#?
For example let's say i have a file like this:
and I want to copy from 0xA0 to 0xB0 and then paste it to another file.
Something like this perhaps:
long start = 0xA0;
int length = 0xB0 - start;
byte[] data = new byte[length];
using (FileStream fs = File.OpenRead(#"C:\Temp\InputFile.txt"))
{
fs.Seek(start, SeekOrigin.Begin);
fs.Read(data, 0, length);
}
File.WriteAllBytes(#"C:\Temp\OutputFile.txt", data);
It should be something like:
// input data
string inputName = "input.bin";
long startInput = 0xa0;
long endInput = 0xb0; // excluding 0xb0 that is not copied
string outputName = "output.bin";
long startOutput = 0xa0;
// begin of code
long count = endInput - startInput;
using (var fs = File.OpenRead(inputName))
using (var fs2 = File.OpenWrite(outputName))
{
fs.Seek(startInput, SeekOrigin.Begin);
fs2.Seek(startOutput, SeekOrigin.Begin);
byte[] buf = new byte[4096];
while (count > 0)
{
int read = fs.Read(buf, 0, (int)Math.Min(buf.Length, count));
if (read == 0)
{
// end of file encountered
throw new IOException("end of file encountered");
}
fs2.Write(buf, 0, read);
count -= read;
}
}
Related
I'm having issues with an "out of memory" exception that I can't reproduce, but a build server that runs unit tests hits every time. Running the unit tests on my machine doesn't cause the exception. The changes made were because the original code was having an odd issue with large PDF's in the passed in stream. If you have an idea of why the original code had issues with the large PDF's or why the new code would cause an "out of memory" exception then let me know.
Original Code:
// stream is a valid Stream and parentKey is a valid int
// Reset the stream position
stream.Position = 0;
int sequenceNumber = 0;
int StreamReadSize = short.MaxValue;
byte[] buffer = new byte[StreamReadSize];
MemoryStream outStream = null;
try
{
long previousStreamPosition = 0;
long DataBlockSize = 52428800;
int read;
while ((read = stream.Read(buffer, 0, buffer.Length)) > 0)
{
if (outStream == null)
outStream = new MemoryStream(new byte[System.Math.Min(stream.Length - previousStreamPosition, DataBlockSize)]);
previousStreamPosition = stream.Position;
outStream.Write(buffer, 0, read);
if (outStream.Position <= (DataBlockSize - StreamReadSize) && stream.Position < stream.Length)
continue;
var dataRow = dataSet.Tables["table_name"].NewRow();
dataRow["parent_key"] = parentKey;
dataRow["key"] = FuncThatReturnsNextAvailableKey();
dataRow["sequence_number"] = ++sequenceNumber;
// Reset the position and Zip up the data
outStream.Position = 0;
dataRow["data_segment"] = FuncThatZipsAStreamToByteArray(outStream);
dataSet.Tables["table_name"].Rows.Add(dataRow);
outStream.Flush();
outStream.Dispose();
outStream = null;
}
}
finally
{
if (outStream != null)
outStream.Dispose();
}
New Code:
// stream is a valid Stream and parentKey is a valid int
// Reset the stream position and create the variables needed for saving the file data
stream.Position = 0;
int sequenceNumber = 0;
int bytesRead;
int DataBlockSize = 52428800;
byte[] buffer = new byte[DataBlockSize];
while ((bytesRead = stream.Read(buffer, 0, DataBlockSize)) > 0)
{
sequenceNumber++;
// Create and initialize the row
var dataRow = dataSet.Tables["table_name"].NewRow();
dataRow["parent_key"] = parentKey;
dataRow["key"] = FuncThatReturnsNextAvailableKey(); ;
dataRow["sequence_number"] = sequenceNumber;
// If the stream reads in less data than the size of the buffer then create an appropriately sized version of the buffer
// that will only hold the data that was read in
if (bytesRead != DataBlockSize)
{
var shrunkBuffer = new byte[bytesRead];
Array.Copy(buffer, shrunkBuffer, bytesRead);
using (var memoryStream = new MemoryStream(shrunkBuffer))
dataRow["data_segment"] = FuncThatZipsAStreamToByteArray(memoryStream);
}
else
{
using (var memoryStream = new MemoryStream(buffer))
dataRow["data_segment"] = FuncThatZipsAStreamToByteArray(memoryStream);
}
// Add the finished row
dataSet.Tables["table_name"].Rows.Add(dataRow);
}
It makes sense that two different environments might generate a different result. it could be that your build server has less memory than your personal coding environment.
It could be that you are keeping your byte arrays in memory via:
dataRow["data_segment"] = FuncThatZipsAStreamToByteArray(memoryStream);
Your are disposing the output stream, but i am assuming your data row stays in memory, hence your are keeping a reference to that byte array. it could be so that multiple PDFs reach the maximum amount of allocation your process can allocate for itself.
Use a class memorytributary
From sources at https://gist.github.com/bittercoder/3588074
using (System.IO.FileStream stream = new System.IO.FileStream(fileName, System.IO.FileMode.Open, System.IO.FileAccess.Read))
{
using (MemoryTributary memT = new MemoryTributary())
{
memT.ReadFrom(stream, stream.Length);
return memT.ToArray();
}
}
I have several .gz files, and I want to decompress them one by one.
I have writen a simple code using GzipStream in C#, but got failed. I wonder a correct and useful method to achieve what I want. Thanks a lot.
private string Extrgz(string infile)
{
string dir = Path.GetDirectoryName(infile);
string decompressionFileName = dir + Path.GetFileNameWithoutExtension(infile) + "_decompression.bin";
using (GZipStream instream = new GZipStream(File.OpenRead(infile), CompressionMode.Compress))// ArgumentException...
{
using (FileStream outputStream = new FileStream(decompressionFileName, FileMode.Append, FileAccess.Write))
{
int bufferSize = 8192, bytesRead = 0;
byte[] buffer = new byte[bufferSize];
while ((bytesRead = instream.Read(buffer, 0, bufferSize)) > 0)
{
outputStream.Write(buffer, 0, bytesRead);
}
}
}
return decompressionFileName;
}
You need to decompress but you set CompressionMode.Compress, replace it with CompressionMode.Decompress.
Example here.
Here:
public static void DeCompressFile(string CompressedFile, string DeCompressedFile)
{
byte[] buffer = new byte[1024 * 1024];
using (System.IO.FileStream fstrmCompressedFile = System.IO.File.OpenRead(CompressedFile)) // fi.OpenRead())
{
using (System.IO.FileStream fstrmDecompressedFile = System.IO.File.Create(DeCompressedFile))
{
using (System.IO.Compression.GZipStream strmUncompress = new System.IO.Compression.GZipStream(fstrmCompressedFile,
System.IO.Compression.CompressionMode.Decompress))
{
int numRead = strmUncompress.Read(buffer, 0, buffer.Length);
while (numRead != 0)
{
fstrmDecompressedFile.Write(buffer, 0, numRead);
fstrmDecompressedFile.Flush();
numRead = strmUncompress.Read(buffer, 0, buffer.Length);
} // Whend
//int numRead = 0;
//while ((numRead = strmUncompress.Read(buffer, 0, buffer.Length)) != 0)
//{
// fstrmDecompressedFile.Write(buffer, 0, numRead);
// fstrmDecompressedFile.Flush();
//} // Whend
strmUncompress.Close();
} // End Using System.IO.Compression.GZipStream strmUncompress
fstrmDecompressedFile.Flush();
fstrmDecompressedFile.Close();
} // End Using System.IO.FileStream fstrmCompressedFile
fstrmCompressedFile.Close();
} // End Using System.IO.FileStream fstrmCompressedFile
} // End Sub DeCompressFile
// http://www.dotnetperls.com/decompress
public static byte[] Decompress(byte[] gzip)
{
byte[] baRetVal = null;
using (System.IO.MemoryStream ByteStream = new System.IO.MemoryStream(gzip))
{
// Create a GZIP stream with decompression mode.
// ... Then create a buffer and write into while reading from the GZIP stream.
using (System.IO.Compression.GZipStream stream = new System.IO.Compression.GZipStream(ByteStream
, System.IO.Compression.CompressionMode.Decompress))
{
const int size = 4096;
byte[] buffer = new byte[size];
using (System.IO.MemoryStream memory = new System.IO.MemoryStream())
{
int count = 0;
count = stream.Read(buffer, 0, size);
while (count > 0)
{
memory.Write(buffer, 0, count);
memory.Flush();
count = stream.Read(buffer, 0, size);
}
baRetVal = memory.ToArray();
memory.Close();
}
stream.Close();
} // End Using System.IO.Compression.GZipStream stream
ByteStream.Close();
} // End Using System.IO.MemoryStream ByteStream
return baRetVal;
} // End Sub Decompress
I'm looking for a good way to get filesize from the Microsoft.SharePoint.Client.File object.
The Client object does not have a Length member.
I tried this:
foreach (SP.File file in files)
{
string path = file.Path;
path = path.Substring(this.getTeamSiteUrl().Length);
FileInformation fileInformation = SP.File.OpenBinaryDirect(this.Context, path);
using (MemoryStream memoryStream = new MemoryStream())
{
CopyStream(fileInformation.Stream, memoryStream);
file.Size = memoryStream.Length;
}
}
Which gave me a length through using the MemoryStream, but it's not good for performance. This file also does not belong to a document library. Since it's an attached file, I can't convert it to a ListItem object using ListItemAllFields. If I could convert it to a ListItem, I could get its size using: ListItem["File_x0020_Size"]
How do I get the filesize of the Client object in SharePoint using C#?
Load the File_x0020_Size field information to get it.
This is what I do when I want to list all the files in a Sharepoint 2010 folder:
//folderPath is something like /yoursite/yourlist/yourfolder
Microsoft.SharePoint.Client.Folder spFolder = _ctx.Web.GetFolderByServerRelativeUrl(folderPath);
_ctx.Load(spFolder);
_ctx.ExecuteQuery();
FileCollection fileCol = spFolder.Files;
_ctx.Load(fileCol);
_ctx.ExecuteQuery();
foreach (Microsoft.SharePoint.Client.File spFile in fileCol)
{
//In here, specify all the fields you want retrieved, including the file size one...
_ctx.Load(spFile, file => file.Author, file => file.TimeLastModified, file=>file.TimeCreated,
file => file.Name, file => file.ServerRelativeUrl, file => file.ListItemAllFields["File_x0020_Size"]);
_ctx.ExecuteQuery();
int fileSize = int.Parse((string)spFile.ListItemAllFields["File_x0020_Size"]);
}
_ctx is obviously the ClientContext you have initiated.
Here's an extended list of all Sharepoint internal fields
Can't you just use the Stream property's length?
file.Size = fileInformation.Stream.Length;
I don't know if this question was ever solved, but for the people who are looking for an answer (like I did) :
... CODE FOR GETTING THE SP.FILE ...
SP.FileInformation fileInfo = SP.File.OpenBinaryDirect(ctx, mySPFile.ServerRelativeUrl);
byte[] bodyString = ReadToEnd(fileInfo.Stream);
int length = bodyString.Length;
Console.Write(length.ToString());
... DO THE OTHER STUFF YOU NEED TO DO ....
public static byte[] ReadToEnd(System.IO.Stream stream)
{
long originalPosition = 0;
if (stream.CanSeek)
{
originalPosition = stream.Position;
stream.Position = 0;
}
try
{
byte[] readBuffer = new byte[4096];
int totalBytesRead = 0;
int bytesRead;
while ((bytesRead = stream.Read(readBuffer, totalBytesRead, readBuffer.Length - totalBytesRead)) > 0)
{
totalBytesRead += bytesRead;
if (totalBytesRead == readBuffer.Length)
{
int nextByte = stream.ReadByte();
if (nextByte != -1)
{
byte[] temp = new byte[readBuffer.Length * 2];
Buffer.BlockCopy(readBuffer, 0, temp, 0, readBuffer.Length);
Buffer.SetByte(temp, totalBytesRead, (byte)nextByte);
readBuffer = temp;
totalBytesRead++;
}
}
}
byte[] buffer = readBuffer;
if (readBuffer.Length != totalBytesRead)
{
buffer = new byte[totalBytesRead];
Buffer.BlockCopy(readBuffer, 0, buffer, 0, totalBytesRead);
}
return buffer;
}
finally
{
if (stream.CanSeek)
{
stream.Position = originalPosition;
}
}
}
Hope this will help other people, because I couldn't find a direct answer on the internet!
I'm looking for a good way to get filesize from the Microsoft.SharePoint.Client.File object.
The Client object does not have a Length member.
I tried this:
foreach (SP.File file in files)
{
string path = file.Path;
path = path.Substring(this.getTeamSiteUrl().Length);
FileInformation fileInformation = SP.File.OpenBinaryDirect(this.Context, path);
using (MemoryStream memoryStream = new MemoryStream())
{
CopyStream(fileInformation.Stream, memoryStream);
file.Size = memoryStream.Length;
}
}
Which gave me a length through using the MemoryStream, but it's not good for performance. This file also does not belong to a document library. Since it's an attached file, I can't convert it to a ListItem object using ListItemAllFields. If I could convert it to a ListItem, I could get its size using: ListItem["File_x0020_Size"]
How do I get the filesize of the Client object in SharePoint using C#?
Load the File_x0020_Size field information to get it.
This is what I do when I want to list all the files in a Sharepoint 2010 folder:
//folderPath is something like /yoursite/yourlist/yourfolder
Microsoft.SharePoint.Client.Folder spFolder = _ctx.Web.GetFolderByServerRelativeUrl(folderPath);
_ctx.Load(spFolder);
_ctx.ExecuteQuery();
FileCollection fileCol = spFolder.Files;
_ctx.Load(fileCol);
_ctx.ExecuteQuery();
foreach (Microsoft.SharePoint.Client.File spFile in fileCol)
{
//In here, specify all the fields you want retrieved, including the file size one...
_ctx.Load(spFile, file => file.Author, file => file.TimeLastModified, file=>file.TimeCreated,
file => file.Name, file => file.ServerRelativeUrl, file => file.ListItemAllFields["File_x0020_Size"]);
_ctx.ExecuteQuery();
int fileSize = int.Parse((string)spFile.ListItemAllFields["File_x0020_Size"]);
}
_ctx is obviously the ClientContext you have initiated.
Here's an extended list of all Sharepoint internal fields
Can't you just use the Stream property's length?
file.Size = fileInformation.Stream.Length;
I don't know if this question was ever solved, but for the people who are looking for an answer (like I did) :
... CODE FOR GETTING THE SP.FILE ...
SP.FileInformation fileInfo = SP.File.OpenBinaryDirect(ctx, mySPFile.ServerRelativeUrl);
byte[] bodyString = ReadToEnd(fileInfo.Stream);
int length = bodyString.Length;
Console.Write(length.ToString());
... DO THE OTHER STUFF YOU NEED TO DO ....
public static byte[] ReadToEnd(System.IO.Stream stream)
{
long originalPosition = 0;
if (stream.CanSeek)
{
originalPosition = stream.Position;
stream.Position = 0;
}
try
{
byte[] readBuffer = new byte[4096];
int totalBytesRead = 0;
int bytesRead;
while ((bytesRead = stream.Read(readBuffer, totalBytesRead, readBuffer.Length - totalBytesRead)) > 0)
{
totalBytesRead += bytesRead;
if (totalBytesRead == readBuffer.Length)
{
int nextByte = stream.ReadByte();
if (nextByte != -1)
{
byte[] temp = new byte[readBuffer.Length * 2];
Buffer.BlockCopy(readBuffer, 0, temp, 0, readBuffer.Length);
Buffer.SetByte(temp, totalBytesRead, (byte)nextByte);
readBuffer = temp;
totalBytesRead++;
}
}
}
byte[] buffer = readBuffer;
if (readBuffer.Length != totalBytesRead)
{
buffer = new byte[totalBytesRead];
Buffer.BlockCopy(readBuffer, 0, buffer, 0, totalBytesRead);
}
return buffer;
}
finally
{
if (stream.CanSeek)
{
stream.Position = originalPosition;
}
}
}
Hope this will help other people, because I couldn't find a direct answer on the internet!
How do I convert a pdf file to a byte[] and vice versa?
// loading bytes from a file is very easy in C#. The built in System.IO.File.ReadAll* methods take care of making sure every byte is read properly.
// note that for Linux, you will not need the c: part
// just swap out the example folder here with your actual full file path
string pdfFilePath = "c:/pdfdocuments/myfile.pdf";
byte[] bytes = System.IO.File.ReadAllBytes(pdfFilePath);
// munge bytes with whatever pdf software you want, i.e. http://sourceforge.net/projects/itextsharp/
// bytes = MungePdfBytes(bytes); // MungePdfBytes is your custom method to change the PDF data
// ...
// make sure to cleanup after yourself
// and save back - System.IO.File.WriteAll* makes sure all bytes are written properly - this will overwrite the file, if you don't want that, change the path here to something else
System.IO.File.WriteAllBytes(pdfFilePath, bytes);
using (FileStream fs = new FileStream("sample.pdf", FileMode.Open, FileAccess.Read))
{
byte[] bytes = new byte[fs.Length];
int numBytesToRead = (int)fs.Length;
int numBytesRead = 0;
while (numBytesToRead > 0)
{
// Read may return anything from 0 to numBytesToRead.
int n = fs.Read(bytes, numBytesRead, numBytesToRead);
// Break when the end of the file is reached.
if (n == 0)
{
break;
}
numBytesRead += n;
numBytesToRead -= n;
}
numBytesToRead = bytes.Length;
}
Easiest way:
byte[] buffer;
using (Stream stream = new IO.FileStream("file.pdf"))
{
buffer = new byte[stream.Length - 1];
stream.Read(buffer, 0, buffer.Length);
}
using (Stream stream = new IO.FileStream("newFile.pdf"))
{
stream.Write(buffer, 0, buffer.Length);
}
Or something along these lines...