I am having issues with FileStreams. I'm in the process of writing a C# serial interface for FPGA project I'm working on which receives a packet (containing 16 bytes) creates and writes the bytes to a file and subsequently appends to the created file.
The program is not throwing any errors but doesn't appear to get past creating the file and does not write any data to it.
Any Ideas? IS there a better way to OpenOrAppend a file?
Thanks in Advance,
Michael
private void SendReceivedDataToFile(int sendBytes)
{
if (saveFileCreated == false)
{
FileStream writeFileStream = new FileStream(tbSaveDirectory.Text, FileMode.Create);
writeFileStream.Write(oldData, 0, sendBytes);
writeFileStream.Flush();
writeFileStream.Close();
saveFileCreated = true;
readByteCount = readByteCount + sendBytes;
}
else
{
using (var writeFilestream2 = new FileStream(tbSaveDirectory.Text, FileMode.Append))
{
writeFilestream2.Write(oldData, 0, sendBytes);
writeFilestream2.Flush();
writeFilestream2.Close();
readByteCount = readByteCount + sendBytes;
}
}
if (readByteCount == readFileSize) // all data has been recieved so close file.
{
saveFileCreated = false;
}
}
FileMode.Append already means "create or append", so really you only need the else {} part of your if. You also don't need to call Flush() or Close() - disposing the stream will do that for you.
Not sure about not writing data... did you try to trace your code?
So first I would reduce your code to
private void SendReceivedDataToFile(int sendBytes)
{
using (var fs = new FileStream(tbSaveDirectory.Text, FileMode.Append))
fs.Write(oldData, 0, sendBytes);
readByteCount += sendBytes;
}
then try to figure what exactly in the oldData.
Related
From SharePoint, I get a "Stream" for a file. I want to copy this entire file from the internet to a local file on the PC, but I want to have status while this download is occurring. How? FileStream and StreamReader seem to have bytes vs char differences when not doing a full "CopyTo" which doesn't have progress updates. There has to be a cleaner way...
using (var sr = new StreamReader(fileData.Value))
{
using (FileStream fs = new FileStream(localFile + "_tmp", FileMode.Create))
{
byte[] block = new byte[1024];
// only guessing something like this is necessary...
int count = (int)Math.Min(sr.BaseStream.Length - sr.BaseStream.Position, block.Length);
while (sr.BaseStream.Position < sr.BaseStream.Length)
{
// read requires char[]
sr.Read(block, 0, count);
// write requires byte[]
fs.Write(block, 0, count);
Log("Percent complete: " + (sr.BaseStream.Position / sr.BaseStream.Length));
count = (int)Math.Min(sr.BaseStream.Length - sr.BaseStream.Position, block.Length);
}
}
}
Just had to use BinaryReader instead of StreamReader. Easy Peasy.
I have some code where I'm trying to read a line within a Mainframe file before I download it from the host. I create an instance of the Stream class as an object called reader, retrieve the FTP data stream from the host and place it into the Stream object, and then create a copy of the original Stream object and its data into another Stream object called readerCopy. My issue, I think, is that when I pass readerCopy into a method that retrieves some data from the Stream(RetrieveDateFromFile), that the resources for both readerCopy and reader are disposed of after the method ends. So when my calling method tries to use reader later on it throws the following:
System.ObjectDisposedException: 'Cannot access a disposed object. Object name: 'System.Net.Sockets.NetworkStream''
I thought that encapsulating all of the Stream objects in using statements would make it so that the resources wouldn't be disposed of until the end of those statements are reached but it seems like they might be disposed of sooner.
What am I missing?
Calling method:
public void FtpFile()
{
// Gets the FTP data stream and stores it into reader, creates a new Stream object called readercopy.
using (Stream reader = request.GetResponse().GetResponseStream(), readerCopy = new MemoryStream())
{
if (reader != null)
{
reader.CopyTo(readerCopy);// Copies the original Stream to readerCopy.
readerCopy.Position = 0; //Sets the position to be beginning of the Stream.
SMDPPITrigger trigger = new SMDPPITrigger(); //Custom class
using (StreamReader fileReader = new StreamReader(readerCopy))
{
using (FileStream fileStream = new FileStream(ftpFileDestination, FileMode.Create))
{
if (trigger.CheckIfExists(RetrieveDateFromFile(fileReader)) == false)
while (true)
{
bytesRead = reader.Read(buffer, 0, buffer.Length); //<--- error occurs here.
if (bytesRead == 0)
break;
fileStream.Write(buffer, 0, bytesRead);
}
}
}
}
}
}
Method to retrieve data from stream:
public DateTime RetrieveDateFromFile(StreamReader mainframeFile)
{
string lineParsed = "";
// StreamReader fileReader = new StreamReader(mainframeFile);
for (int i = 0; i <= 2; i++)
switch (i)
{
case 2:
string line = mainframeFile.ReadLine();
if (line != null)
{
lineParsed = line.Substring(124);
break;
}
else
{
break;
}
default:
{
mainframeFile.ReadLine();
break;
}
}
return DateTime.Parse(lineParsed);
}
I am suspecting on below two lines of code causing problem.
reader.CopyTo(readerCopy);// Copies the original Stream to readerCopy.
readerCopy.Position = 0; //Sets the position to be beginning of the Stream.
Can you try specifying length of stream you want to copy into readerCopy?
e.g
reader.CopyTo(readerCopy,124);
Closing the loop on my question. It turns out the issue and resolution is exactly what #KlausGütter had said. Using reader.CopyTo(readerCopy) was setting my position to the end of the stream and my error message wasn't that it had been disposed but instead that there was nothing left to read. Using the stream that I copied over to, readerCopy, solved my issue since the stream is seekable.
I want to read file continuously like GNU tail with "-f" param. I need it to live-read log file.
What is the right way to do it?
More natural approach of using FileSystemWatcher:
var wh = new AutoResetEvent(false);
var fsw = new FileSystemWatcher(".");
fsw.Filter = "file-to-read";
fsw.EnableRaisingEvents = true;
fsw.Changed += (s,e) => wh.Set();
var fs = new FileStream("file-to-read", FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
using (var sr = new StreamReader(fs))
{
var s = "";
while (true)
{
s = sr.ReadLine();
if (s != null)
Console.WriteLine(s);
else
wh.WaitOne(1000);
}
}
wh.Close();
Here the main reading cycle stops to wait for incoming data and FileSystemWatcher is used just to awake the main reading cycle.
You want to open a FileStream in binary mode. Periodically, seek to the end of the file minus 1024 bytes (or whatever), then read to the end and output. That's how tail -f works.
Answers to your questions:
Binary because it's difficult to randomly access the file if you're reading it as text. You have to do the binary-to-text conversion yourself, but it's not difficult. (See below)
1024 bytes because it's a nice convenient number, and should handle 10 or 15 lines of text. Usually.
Here's an example of opening the file, reading the last 1024 bytes, and converting it to text:
static void ReadTail(string filename)
{
using (FileStream fs = File.Open(filename, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
// Seek 1024 bytes from the end of the file
fs.Seek(-1024, SeekOrigin.End);
// read 1024 bytes
byte[] bytes = new byte[1024];
fs.Read(bytes, 0, 1024);
// Convert bytes to string
string s = Encoding.Default.GetString(bytes);
// or string s = Encoding.UTF8.GetString(bytes);
// and output to console
Console.WriteLine(s);
}
}
Note that you must open with FileShare.ReadWrite, since you're trying to read a file that's currently open for writing by another process.
Also note that I used Encoding.Default, which in US/English and for most Western European languages will be an 8-bit character encoding. If the file is written in some other encoding (like UTF-8 or other Unicode encoding), It's possible that the bytes won't convert correctly to characters. You'll have to handle that by determining the encoding if you think this will be a problem. Search Stack overflow for info about determining a file's text encoding.
If you want to do this periodically (every 15 seconds, for example), you can set up a timer that calls the ReadTail method as often as you want. You could optimize things a bit by opening the file only once at the start of the program. That's up to you.
To continuously monitor the tail of the file, you just need to remember the length of the file before.
public static void MonitorTailOfFile(string filePath)
{
var initialFileSize = new FileInfo(filePath).Length;
var lastReadLength = initialFileSize - 1024;
if (lastReadLength < 0) lastReadLength = 0;
while (true)
{
try
{
var fileSize = new FileInfo(filePath).Length;
if (fileSize > lastReadLength)
{
using (var fs = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
fs.Seek(lastReadLength, SeekOrigin.Begin);
var buffer = new byte[1024];
while (true)
{
var bytesRead = fs.Read(buffer, 0, buffer.Length);
lastReadLength += bytesRead;
if (bytesRead == 0)
break;
var text = ASCIIEncoding.ASCII.GetString(buffer, 0, bytesRead);
Console.Write(text);
}
}
}
}
catch { }
Thread.Sleep(1000);
}
}
I had to use ASCIIEncoding, because this code isn't smart enough to cater for variable character lengths of UTF8 on buffer boundaries.
Note: You can change the Thread.Sleep part to be different timings, and you can also link it with a filewatcher and blocking pattern - Monitor.Enter/Wait/Pulse. For me the timer is enough, and at most it only checks the file length every second, if the file hasn't changed.
This is my solution
static IEnumerable<string> TailFrom(string file)
{
using (var reader = File.OpenText(file))
{
while (true)
{
string line = reader.ReadLine();
if (reader.BaseStream.Length < reader.BaseStream.Position)
reader.BaseStream.Seek(0, SeekOrigin.Begin);
if (line != null) yield return line;
else Thread.Sleep(500);
}
}
}
so, in your code you can do
foreach (string line in TailFrom(file))
{
Console.WriteLine($"line read= {line}");
}
You could use the FileSystemWatcher class which can send notifications for different events happening on the file system like file changed.
private void button1_Click(object sender, EventArgs e)
{
if (folderBrowserDialog.ShowDialog() == DialogResult.OK)
{
path = folderBrowserDialog.SelectedPath;
fileSystemWatcher.Path = path;
string[] str = Directory.GetFiles(path);
string line;
fs = new FileStream(str[0], FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
tr = new StreamReader(fs);
while ((line = tr.ReadLine()) != null)
{
listBox.Items.Add(line);
}
}
}
private void fileSystemWatcher_Changed(object sender, FileSystemEventArgs e)
{
string line;
line = tr.ReadLine();
listBox.Items.Add(line);
}
If you are just looking for a tool to do this then check out free version of Bare tail
I am trying to create a "virtual printer" application in C# that receives print jobs over the network, parses the raw print data for certain information, and then saves the document into a database. A modified version of the following class is working for postscript print jobs (it saves the incoming data to a valid .prn file, just as though the printer was set to print to the "FILE:" port.) When I try to capture .XPS documents from Microsoft XPS Document Writer, though, the documents cannot be opened. Valid XPS files should also be valid ZIP files if the extension is renamed, and this doesn't work either. When I print the same document to the FILE: port and then to my application, and I compare the results in Notepad++, there is a 5-character difference in the length of the data, but it looks identical (it is not plaintext so it's difficult to look at, but the first few characters and last few characters appear to be the same). The file saved the "normal" way works fine, but the file generated by my code does not.
More generally speaking, I'm trying to receive arbitrary data through a TCP port and write it to a file. My solution is "close" but not working. I don't know what kind of encoding XPS uses, but I am using ASCII for postscript and I have tried ASCII and UTF8 for this XPS version.
Any help is greatly appreciated! Here is the relevant part of my code:
class XPSListener
{
private TcpListener tcpListener;
private Thread listenThread;
private string instanceName = "";
private string fileShare = (Settings.Default.SharedPath.Substring(Settings.Default.SharedPath.Length - 1) == #"\") ? Settings.Default.SharedPath : Settings.Default.SharedPath + #"\"; // use SharedPath setting value - append backslash if it isn't already there.
public XPSListener(string initInstanceName, Int32 initPort)
{
this.instanceName = initInstanceName;
this.tcpListener = new TcpListener(IPAddress.Any, initPort);
this.listenThread = new Thread(new ThreadStart(ListenForClients));
this.listenThread.Start();
}
private void ListenForClients()
{
try
{
this.tcpListener.Start();
}
catch (Exception e)
{
MessageBox.Show("Socket Error 1 - " + e.StackTrace);
}
while (true)
{
//blocks until a client has connected to the server
TcpClient client = this.tcpListener.AcceptTcpClient();
//create a thread to handle communication with connected client
Thread clientThread = new Thread(new ParameterizedThreadStart(AcceptXPSData));
clientThread.Start(client);
}
}
private void AcceptXPSData(object client)
{
TcpClient tcpClient = (TcpClient)client;
NetworkStream clientStream = tcpClient.GetStream();
string tempFilePath = fileShare + "XPStemp_" + instanceName + ".oxps";
byte[] message = new byte[65536];
int bytesRead;
string input;
while (true)
{
bytesRead = 0;
try
{
//blocks until a client sends a message
bytesRead = clientStream.Read(message, 0, 65536);
Debug.WriteLine("Bytes read: " + bytesRead.ToString());
}
catch
{
//a socket error has occured
break;
}
if (bytesRead == 0)
{
//the client has disconnected from the server
break;
}
//message has successfully been received
if (instanceName != "DontPrint")
{
Debug.WriteLine(instanceName + " Receiving Data");
//ASCIIEncoding encoder = new ASCIIEncoding();
UTF8Encoding encoder = new UTF8Encoding();
using (FileStream fs = new FileStream(tempFilePath, FileMode.Append, FileAccess.Write))
{
using (StreamWriter sw = new StreamWriter(fs))
{
input = encoder.GetString(message, 0, bytesRead);
sw.Write(input);
// first capture this input and write it to an xps file. This file can be converted to PDF at a later time by Ghostscript
// but we will still have access to the temp file for parsing purposes.
}
}
}
}
tcpClient.Close();
// processXPS();
}
You have at least two problems in your code, one of them almost certainly the reason the file you write is incorrect:
You keep reopening the file you're writing to, rather than just opening it once.
You are interpreting the bytes you receive as text and then re-encoding them.
The first issue is more of an efficiency/file-locking issue than a correctness problem. But the second is a big problem.
As you seem to be aware, an XPS file is basically a .zip file. That means that while the underlying data is XML (i.e. UTF8), the file itself is a compressed binary file. You can't interpret that as text in any meaningful way.
You should simply write the bytes you read straight to the file. A better version of your code would look like this:
private void AcceptXPSData(object client)
{
string tempFilePath = fileShare + "XPStemp_" + instanceName + ".oxps";
using (TcpClient tcpClient = (TcpClient)client)
using (NetworkStream clientStream = tcpClient.GetStream())
using (FileStream fs = new FileStream(tempFilePath, FileMode.Create, FileAccess.Write))
{
clientStream.CopyTo(fs);
}
// processXPS();
}
If you actually want to monitor the I/O as it occurs, you can deal with it explicitly, but still much more simply than your code was:
private void AcceptXPSData(object client)
{
string tempFilePath = fileShare + "XPStemp_" + instanceName + ".oxps";
using (TcpClient tcpClient = (TcpClient)client)
using (NetworkStream clientStream = tcpClient.GetStream())
using (FileStream fs = new FileStream(tempFilePath, FileMode.Create, FileAccess.Write))
{
byte[] message = new byte[65536];
int bytesRead;
while ((bytesRead = clientStream.Read(message, 0, message.Length)) > 0)
{
fs.Write(message, 0, bytesRead);
// Add logging or whatever here
}
}
// processXPS();
}
Note that if you want to handle exceptions, you need to handle only those you specifically expect might happen, and for which you have a reasonable way to deal with. Bare catch clauses, or broad catch (Exception) should be avoided in code like this.
My company run an application that have to archive many kinds of files into some distants servers. The application works well but can't handle files larger than 1GB.
Here is the current function use to load the files to be uploaded to the distant server :
FileStream fs = File.OpenRead(fileToUploadPath);
byte[] fileArray = new byte[fs.Length];
fs.Read(fileArray, 0, fs.Length);
The byte array (when loaded successfully) was then splited into 100Mb bytes arrays and sent to the local server (using some WSDL web services) with the following function :
localServerWebService.SendData(subFileArray, filename);
I changed the function responsible for the file reading to use BufferendStream and I also wanted to improve the Webservice part so that it doesn't have to create a new stream at each call. I thought of somethings like this :
FileInfo source = new FileInfo(fileName);
using (FileStream reader = File.OpenRead(fileName))
{
using (FileStream distantWriter = localServerWebService.CreateWriteFileStream(fileName))
{
using (BufferedStream buffReader = new BufferedStream(reader))
{
using (BufferedStream buffWriter = new BufferedStream(distantWriter))
{
byte[] buffer = new byte[BUFFER_SIZE];
int bytesRead = 0;
long bytesToRead = source.Length;
while (bytesToRead > 0)
{
int nbBytesRead = buffReader.Read(buffer, 0, buffer.Length);
buffWriter.Write(buffer, 0, nbBytesRead);
bytesRead += nbBytesRead;
bytesToRead -= nbBytesRead;
}
}
}
}
}
But this code can't compile and always give me the error Cannot convert MyNameSpace.FileStream into System.IO.FileStream at line using (FileStream distantWriter = localServerWebService.CreateWriteFileStream(fileName)). I can't cast MyNameSpace.FileStream into System.IO.FileStream either.
The web service method :
[WebMethod]
public FileStream CreateWriteFileStream(String fileName)
{
String RepVaultUP =
System.Configuration.ConfigurationSettings.AppSettings.Get("SAS_Upload");
String desFile = Path.Combine(RepVaultUP, fileName);
return File.Open(desFile, FileMode.Create, FileAccess.Write);
}
So can you guys please explain to me why is this not working?
P.S.: English is not my mothertong so I hope what i wrote is clearly undestandable.