Im trying to read a DAT file with BinaryReady but I get an exception and don't see why. " Unable to read beyond the end of the stream" is the message I get. My code looks like this:
private void button1_Click(object sender, EventArgs e)
{
OpenFileDialog OpenFileDialog = new OpenFileDialog();
OpenFileDialog.Title = "Open File...";
OpenFileDialog.Filter = "Binary File (*.dat)|*.dat";
OpenFileDialog.InitialDirectory = #"C:\";
if (OpenFileDialog.ShowDialog() == DialogResult.OK)
{
FileStream fs = new FileStream(OpenFileDialog.FileName, FileMode.Open);
BinaryReader br = new BinaryReader(fs);
label1.Text = br.ReadString();
label2.Text = br.ReadInt32().ToString();
fs.Close();
br.Close();
}
I hat a certain DAT file with a lot of information and was hoping to be able to read it out and maybe even place it in a table and plot the data. But its been a while I worked with C#. So if anyone could help me I would highly appreciate
BinaryReader is very rarely a good choice for reading an external file; it only really makes sense when used in parallel with code that writes the file using BinaryWriter, since they use the same conventions.
I imagine that what is happening here is that your call to ReadString is trying to use conventions that aren't valid for your file - specifically, it will be reading some bytes (I want to say 4, big endian?) as a length prefix for the string, and then trying to read that many bytes as the string. But if that isn't what the file contents are: that could easily try to read gibberish, interpret it as a huge number, and then fail to read that-many bytes.
If you're processing an arbitrary file (nothing to do with BinaryWriter), then you really need to know a lot about the protocol/format. Given that file extensions are ambiguous, I'm not going to infer anything from ".dat" - what matters is: what is the data and where did it come from?. Only with that information can a sensible comment on reading it be made.
From the comments, here's some (untested) code that should get you started in terms of parsing the contents as a span:
public static YourResultType Process(string path)
{
byte[] oversized = null;
try
{
int len, offset = 0, read;
// read the file into a leased buffer, for simplicity
using (var stream = File.OpenRead(path))
{
len = checked((int)stream.Length);
oversized = ArrayPool<byte>.Shared.Rent(len);
while (offset < len &&
(read = stream.Read(oversized, offset, len - offset)) > 0)
{
offset += read;
}
}
// now process the payload from the buffered data
return Process(new ReadOnlySpan<byte>(oversized, 0, len));
}
finally
{
if (oversized is object)
ArrayPool<byte>.Shared.Return(oversized);
}
}
private static YourResultType Process(ReadOnlySpan<byte> payload)
=> throw new NotImplementedException(); // your code here
Related
I tried to read file (650 megabytes) from SQL Server:
using (var reader = command.ExecuteReader(CommandBehavior.SequentialAccess))
{
if (reader.Read())
{
using (var dbStream = reader.GetStream(0))
{
if (!reader.IsDBNull(0))
{
stream.Position = 0;
dbStream.CopyTo(stream, 256);
}
dbStream.Close();
}
}
reader.Close();
}
But I got OutOfMemoryException on CopyTo().
With small files, this code snippet works fine. How can I handle large file?
You can read and write data to some temp file in small chunks. You can see example on MSDN - Retrieving Binary Data.
//Column Index in the result set
const int colIdx = 0;
// Writes the BLOB to a file (*.bmp).
FileStream stream;
// Streams the BLOB to the FileStream object.
BinaryWriter writer;
// Size of the BLOB buffer.
int bufferSize = 100;
// The BLOB byte[] buffer to be filled by GetBytes.
byte[] outByte = new byte[bufferSize];
// The bytes returned from GetBytes.
long retval;
// The starting position in the BLOB output.
long startIndex = 0;
// Open the connection and read data into the DataReader.
connection.Open();
SqlDataReader reader = command.ExecuteReader(CommandBehavior.SequentialAccess);
while (reader.Read())
{
// Create a file to hold the output.
stream = new FileStream(
"some-physical-file-name-to-dump-data.bmp", FileMode.OpenOrCreate, FileAccess.Write);
writer = new BinaryWriter(stream);
// Reset the starting byte for the new BLOB.
startIndex = 0;
// Read bytes into outByte[] and retain the number of bytes returned.
retval = reader.GetBytes(colIdx, startIndex, outByte, 0, bufferSize);
// Continue while there are bytes beyond the size of the buffer.
while (retval == bufferSize)
{
writer.Write(outByte);
writer.Flush();
// Reposition start index to end of last buffer and fill buffer.
startIndex += bufferSize;
retval = reader.GetBytes(colIdx, startIndex, outByte, 0, bufferSize);
}
// Write the remaining buffer.
writer.Write(outByte, 0, (int)retval);
writer.Flush();
// Close the output file.
writer.Close();
stream.Close();
}
// Close the reader and the connection.
reader.Close();
connection.Close();
Make sure you are using SqlDataReader with CommandBehavior.SequentialAccess, note this line in above code snippet.
SqlDataReader reader = command.ExecuteReader(CommandBehavior.SequentialAccess);
More information on CommandBehavior enum can be found here.
EDIT:
Let me clarify myself. I agreed with #MickyD, cause of the problem is not whether you are using CommandBehavior.SequentialAccess or not, but reading the large file at-once.
I emphasized on this because it is commonly missed by developers, they tend to read files in chunks but without setting CommandBehavior.SequentialAccess they will encounter other problems. Although it is already posted with original question, but highlighted in my answer to give point to any new comers.
#MatthewWatson yeah var stream = new MemoreStream(); What is not right with it? – Kliver Max 15 hours ago
Your problem is not whether or not you are using:
`command.ExecuteReader(CommandBehavior.SequentialAccess)`
...which you are as we can see; or that your stream copy buffer size is too big (it's actually tiny) but rather that you are using MemoryStream as you indicated in the comments above. More than likely you are loading in the 650MB file twice, once from SQL and another to be stored in the MemoryStream thus leading to your OutOfMemoryException.
Though the solution is to instead write to a file stream, the cause of the problem wasn't highlighted in the accepted answer. Unless you know the cause of a problem, you won't learn to avoid such issues in the future.
I am in needed to use the text data in to a program when someone prints a file.
I have basic ideas about the TCP/IP client and listener programming.
Already I could send and receive txt files between two machines.
But how to receive the file contents if the files were in docx,xlx,pdf or any other format?
My requirement is,
I wanted to use the contents (texts) of a file in to another program when someone prints a file.
Please suggest me if there is some alternative ways to do it.
Thank in advance.
Since you haven't posted any code I'll write the code part in "my way" but you should have a bit of understanding after reading this.
First on both of the ends ( client and server ) you should apply unified protocol which will describe what data you're sending. Example could be:
[3Bytes - ASCII extension][4Bytes - lengthOfTheFile][XBytes - fileContents]
Then in your sender you can receive data according to the protocol which means first your read 3 bytes to decide what format file has, then you read 4 bytes which will basically inform you how large file is incomming. Lastly you have to read the content and write it directly to the file. Example sender could look like this :
byte[] extensionBuffer = new byte[3];
if( 3 != networkStream.Read(extensionBuffer, 0, 3))
return;
string extension = Encoding.ASCII.GetString(extensionBuffer);
byte[] lengthBuffer = new byte[sizeof(int)];
if(sizeof(int) != networkStream.Read(lengthBuffer, 0, 3))
return;
int length = BitConverter.ToInt32(lengthBuffer, 0);
int recv = 0;
using (FileStream stream = File.Create(nameOfTheFile + "." + extension))
{
byte #byte = 0x00;
while( (#byte = (byte)networkStream.ReadByte() ) != 0x00)
{
stream.WriteByte(#byte);
recv++;
}
stream.Flush();
}
On the sender part you can read the file extension then open up the file stream get the length of the stream then send the stream length to the client and "redirect" each byte from FileStream into a networkStream. This can look something like :
FileInfo meFile = //.. get the file
byte[] extBytes = Encoding.ASCII.GetBytes(meFile.Extension);
using(FileStream stream = meFile.OpenRead())
{
networkStream.Write(extBytes, 0, extBytes.Length);
networkStream.Write(BitConverter.GetBytes(stream.BaseStream.Length));
byte #byte = 0x00;
while ( stream.Position < stream.BaseStream.Length )
{
networkStream.WriteByte((byte)stream.ReadByte());
}
}
This approach is fairly easy to implement and doesn't require big changes if you want to send different file types. It lack some validator but I think you do not require this functionality.
I'm trying to convert a .db file to binary so I can stream it across a web server. I'm pretty new to C#. I've gotten as far as looking at code snippets online but I'm not really sure if the code below puts me on the right track. How I can write the data once I read it? Does BinaryReader automatically open up and read the entire file so I can then just write it out in binary format?
class Program
{
static void Main(string[] args)
{
using (FileStream fs = new FileStream("output.bin", FileMode.Create))
{
using (BinaryWriter bw = new BinaryWriter(fs))
{
long totalBytes = new System.IO.FileInfo("input.db").Length;
byte[] buffer = null;
BinaryReader binReader = new BinaryReader(File.Open("input.db", FileMode.Open));
}
}
}
}
Edit: Code to stream the database:
[WebGet(UriTemplate = "GetDatabase/{databaseName}")]
public Stream GetDatabase(string databaseName)
{
string fileName = "\\\\computer\\" + databaseName + ".db";
if (File.Exists(fileName))
{
FileStream stream = File.OpenRead(fileName);
if (WebOperationContext.Current != null)
{
WebOperationContext.Current.OutgoingResponse.ContentType = "binary/.bin";
}
return stream;
}
return null;
}
When I call my server, I get nothing back. When I use this same type of method for a content-type of image/.png, it works fine.
All the code you posted will actually do is copy the file input.db to the file output.bin. You could accomplish the same using File.Copy.
BinaryReader will just read in all of the bytes of the file. It is a suitable start to streaming the bytes to an output stream that expects binary data.
Once you have the bytes corresponding to your file, you can write them to the web server's response like this:
using (BinaryReader binReader = new BinaryReader(File.Open("input.db",
FileMode.Open)))
{
byte[] bytes = binReader.ReadBytes(int.MaxValue); // See note below
Response.BinaryWrite(bytes);
Response.Flush();
Response.Close();
Response.End();
}
Note: The code binReader.ReadBytes(int.MaxValue) is for demonstrating the concept only. Don't use it in production code as loading a large file can quickly lead to an OutOfMemoryException. Instead, you should read in the file in chunks, writing to the response stream in chunks.
See this answer for guidance on how to do that
https://stackoverflow.com/a/8613300/141172
I'm writing an interface to a web service where we need to upload configuration files. The documentation only provides a sample in C#.net which I am not familiar with. I'm trying to implement this in PHP.
Can someone familiar with both languages point me in the right direction? I can figure out all the basics, but I'm trying to figure out suitable PHP replacements for the FileStream, ReadBytes, and UploadDataFile functions. I believe that the RecService object contains the URL for the web service. Thanks for your help!
private void UploadFiles() {
clientAlias = “<yourClientAlias>”;
string filePath = “<pathToYourDataFiles>”;
string[] fileList = {"Config.txt", "ProductDetails.txt", "BrandNames.txt", "CategoryNames.txt", "ProductsSoldOut.txt", "Sales.txt"};
RecommendClient RecService = new RecommendClient();
for (int i = 0; i < fileList.Length; i++) {
bool lastFile = (i == fileList.Length ‐ 1); //start generator after last file
try {
string fileName = filePath + fileList[i];
if (!File.Exists(fileName))
continue; // file not found
}
// set up a file stream and binary reader for the selected file and convert to byte array
FileStream fStream = new FileStream(fileName, FileMode.Open, FileAccess.Read); BinaryReader br = new BinaryReader(fStream);
byte[] data = br.ReadBytes((int)numBytes); br.Close();
// pass byte array to the web service
string result = RecService.UploadDataFile(clientAlias, fileList[i], data, lastFile); fStream.Close(); fStream.Dispose();
} catch (Exception ex) {
// log an error message
}
}
}
For reading files, both on the local system and remotely over HTTP, you can use file_get_contents.
For POSTing to a web service, you should probably use cURL. This article looks like a pretty good explanation of how to go about it.
I have an ASP .Net (3.5) website. I have the following code that uploads a file as a binary to a SQL Database:
Print("
protected void UploadButton_Click(object sender, EventArgs e)
{
//Get the posted file
Stream fileDataStream = FileUpload.PostedFile.InputStream;
//Get length of file
int fileLength = FileUpload.PostedFile.ContentLength;
//Create a byte array with file length
byte[] fileData = new byte[fileLength];
//Read the stream into the byte array
fileDataStream.Read(fileData, 0, fileLength);
//get the file type
string fileType = FileUpload.PostedFile.ContentType;
//Open Connection
WebSysDataContext db = new WebSysDataContext(Contexts.WEBSYS_CONN());
//Create New Record
BinaryStore NewFile = new BinaryStore();
NewFile.BinaryID = "1";
NewFile.Type = fileType;
NewFile.BinaryFile = fileData;
//Save Record
db.BinaryStores.InsertOnSubmit(NewFile);
try
{
db.SubmitChanges();
}
catch (Exception)
{
throw;
}
}");
The files that will be uploaded are PDFs, Can you please help me in writing the code to get the PDF out of the SQL database and display it in the browser. (I am able to get the binary file using a linq query but not sure how to process the bytes)
So are you really just after how to serve a byte array in ASP.NET? It sounds like the database part is irrelevant, given that you've said you are able to get the binary file with a LINQ query.
If so, look at HttpResponse.BinaryWrite. You should also set the content type of the response appropriately, e.g. application/pdf.
How big are the files? Huge buffers (i.e. byte[fileLength]) are usually a bad idea.
Personally, I'd look at things like this and this, which show reading/writing data as streams (the second shows pushing the stream as an http response). But updated to use varchar(max) ;-p
protected void Test_Click(object sender, EventArgs e)
{
WebSysDataContext db = new WebSysDataContext(Contexts.WEBSYS_CONN());
var GetFile = from x in db.BinaryStores
where x.BinaryID == "1"
select x.BinaryFile;
FileStream MyFileStream;
long FileSize;
MyFileStream = new FileStream(GetFile, FileMode.Open);
FileSize = MyFileStream.Length;
byte[] Buffer = new byte[(int)FileSize];
MyFileStream.Read(Buffer, 0, (int)FileSize);
MyFileStream.Close();
Response.Write("<b>File Contents: </b>");
Response.BinaryWrite(Buffer);
}
I tryed this and this did not work. I get a compile error on this line "MyFileStream = new FileStream(GetFile, FileMode.Open);"
I not sure where i am going wrong, is it due to the way i have stored it?
When you store binary files in SQL Server it adds an OLE Header to the binary-data. So you must strip that header before actually reading the byte[] into file. Here's how you do this.
// First Strip-Out the OLE header
const int OleHeaderLength = 78;
int strippedDataLength = datarow["Field"].Length - OleHeaderLength;
byte[] strippedData = new byte[strippedDataLength];
Array.Copy(datarow["Field"], OleHeaderLength,
strippedData , 0, strippedDataLength );
Once you run this code, strippedData will contain the actual file data. You can then use MemoryStream or FileStream to perform I/O on the byte[].