I writing a simple sample program, that should write a data to file and read in real time when there is some data. I write this code:
using System;
using System.IO;
using System.Reflection;
using System.Threading;
namespace Sample
{
class Program
{
static void Main(string[] args)
{
const string file = "sample.txt";
var thread = new Thread(() =>
{
var r = new Random();
using (var sw = new StreamWriter(new FileStream(file, FileMode.Create, FileAccess.ReadWrite, FileShare.ReadWrite)) {AutoFlush = true})
{
while (true)
{
Thread.Sleep(r.Next(100, 500));
sw.WriteLine(DateTime.Now);
}
}
});
thread.Start();
using (var watcher = new FileSystemWatcher
{
Path = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location),
Filter = file,
NotifyFilter = NotifyFilters.LastWrite
})
{
var mse = new ManualResetEventSlim(false);
watcher.Changed += (sender, eventArgs) =>
mse.Set();
watcher.EnableRaisingEvents = true;
using (var sr = new StreamReader(new FileStream(file, FileMode.Open, FileAccess.ReadWrite, FileShare.ReadWrite)))
{
while (true)
{
mse.Wait();
ProcessData(sr.ReadToEnd());
mse.Reset();
}
}
}
}
private static void ProcessData(string s)
{
Console.WriteLine(s);
}
}
}
But it seems that watcher works only when file is opened, but doesn't when it is populated with info (even with AutoFlush flag enabled on StreamWriter). Data is physically on the disk but watcher doesn't raise an event File changed.
I just want to avoid infinite loop and process data only when it is written.
If you don't close them I think you should dispose of both the FileStream and the StreamWriter by putting each in a using clause:
string file = "sample.txt";
using (var fs =new FileStream(file, FileMode.Create,
FileAccess.ReadWrite, FileShare.ReadWrite))
using (var sw = new StreamWriter(fs) {AutoFlush = true})
{
..
..
}
Before closing directly or by disposing, the file system will not be informed that the changes are done. Otherwise you would be flooded with changed events..
Related
I have a function to write some data into a binary file
private void writeToBinFile (List<MyClass> myObjList, string filePath)
{
FileStream fs = new FileStream(filePath, FileMode.Create);
BinaryWriter bw = new BinaryWriter(fs);
foreach (MyClass myObj in myObjList)
{
bw.Write(JsonConvert.SerializeObject(myObj));
}
bw.Close();
fs.Close();
}
I am looking something like
FileStream fs = new FileStream(filePath, FileMode.Create);
BinaryReader bw = new BinaryReader(fs);
while (!filePath.EOF)
{
List<MyClass> myObjList = br.Read(myFile);
}
anyone can help with this?
thanks in advance
JSON can be saved with no formatting (no new lines), so you can save 1 record per row of a file. Thus, my suggested solution is to ignore binary files and instead use a regular StreamWriter:
private void WriteToFile(List<MyClass> myObjList, string filePath)
{
using (StreamWriter sw = File.CreateText(filePath))
{
foreach (MyClass myObj in myObjList)
{
sw.Write(JsonConvert.SerializeObject(myObj, Newtonsoft.Json.Formatting.None));
}
}
}
private List<MyClass> ReadFromFile(string filePath)
{
List<MyClass> myObjList = new List<MyClass>();
using (StreamReader sr = File.OpenText(filePath))
{
string line = null;
while ((line = sr.ReadLine()) != null)
{
myObjList.Add(JsonConvert.DeserializeObject<MyClass>(line));
}
}
return myObjList;
}
If you really want to use the binary writer to save JSON, you could change it to be like this:
private void WriteToBinFile(List<MyClass> myObjList, string filePath)
{
using (FileStream fs = new FileStream(filePath, FileMode.Create))
using (BinaryWriter bw = new BinaryWriter(fs))
{
foreach (MyClass myObj in myObjList)
{
bw.Write(JsonConvert.SerializeObject(myObj));
}
}
}
private List<MyClass> ReadFromBinFile(string filePath)
{
List<MyClass> myObjList = new List<MyClass>();
using (FileStream fs = new FileStream(filePath, FileAccess.Read))
using (BinaryReader br = new BinaryReader(fs))
{
while (fs.Length != fs.Position) // This will throw an exception for non-seekable streams (stream.CanSeek == false), but filestreams are seekable so it's OK here
{
myObjList.Add(JsonConvert.DeserializeObject<MyClass>(br.ReadString()));
}
}
return myObjList;
}
Notes:
I've added using around your stream instantiations so that the files are properly closed when memory is freed
To check the stream is at the end, you have to compare Length to Position.
I have a txt file like this
#header1
#header2
#header3
....
#headerN
ID Value Pvalue
a 0.1 0.002
b 0.2 0.002
...
My code will try to parse
FileStream fs = new FileStream(file, FileMode.Open, FileMode.Read);
......
Table t = Table.Load(fs);
what I want is to make the start position of the Stream right before "ID", so I can feed the stream to the code and make a new table. But I am not sure what is the correct way to do it.
Thanks in advance
Ideally, you should convert Table.Load to take an IEnumerable<string> or at least a StreamReader, not a raw Stream.
If this is not an option, you can read the whole file into memory, skip its header, and write the result into MemoryStream:
MemoryStream stream = new MemoryStream();
using (var writer = new StreamWriter(stream, Encoding.UTF8);
foreach (var line in File.ReadLines(fileName).SkipWhile(s => s.StartsWith("#"))) {
writer.WriteLine(line);
}
}
stream.Position = 0;
Table t = Table.Load(stream);
Try this code
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
namespace ConsoleApplication57
{
class Program
{
const string file = "";
static void Main(string[] args)
{
FileStream fs = new FileStream(file, FileMode.Open, FileAccess.Read);
StreamReader reader = new StreamReader(fs);
string inputline = "";
State state = State.FIND_HEADER;
while((inputline = reader.ReadLine()) != null)
{
switch (state)
{
case State.FIND_HEADER:
if (inputline.StartsWith("#header"))
{
state = State.READ_TABLE;
}
break;
case State.READ_TABLE:
Table t = Table.Load(fs);
break;
}
}
}
enum State
{
FIND_HEADER,
READ_TABLE
}
}
}
I have an application A that generates a text file for tracing.
While, an application B needs read the same text file and attach in a mailmessage.
But I get the following error, when application B try read the text file:
IOException: The process cannot access the file 'filename' because it
is being used by another process
Any suggestions ? Maybe better use for FileMode and FileAccess?
Application A
if (File.Exists(nFile2)) File.Delete(nFile2);
traceFile2 = File.Open(nFile2, FileMode.Create, FileAccess.ReadWrite, FileShare.ReadWrite);
if (traceFile2 != null)
{
var twt2 = new TextWriterTraceListener(traceFile2);
// http://www.helixoft.com/blog/archives/20
try
{
if (twt2.Writer is StreamWriter)
{
(twt2.Writer as StreamWriter).AutoFlush = true;
}
}
catch { }
var indiceTraceFile2 = Trace.Listeners.Add(twt2);
System.Diagnostics.Trace.WriteLine("INICIO: " + DateTime.Now.ToString());
Application B
using (FileStream fileStream = File.Open(f, FileMode.Open, FileAccess.ReadWrite, FileShare.Read))
{
var messageAttachment = new Attachment(fileStream, Path.GetFileName(f));
msgMail.Attachments.Add(messageAttachment);
}
You need to make sure that both the service and the reader open the log file non-exclusively. Notice line 2 of App A and Line 1 of App B
Application A:
if (File.Exists(nFile2))
File.Delete(nFile2);
traceFile2 = new FileStream(nFile2, FileMode.Open, FileAccess.Write, FileShare.ReadWrite);
if (traceFile2 != null)
{
var twt2 = new TextWriterTraceListener(traceFile2);
// http://www.helixoft.com/blog/archives/20
try
{
if (twt2.Writer is StreamWriter)
{
(twt2.Writer as StreamWriter).AutoFlush = true;
}
}
catch { }
var indiceTraceFile2 = Trace.Listeners.Add(twt2);
System.Diagnostics.Trace.WriteLine("INICIO: " + DateTime.Now.ToString());
and Application B:
using (FileStream fileStream = new FileStream(f, FileMode.Open,
FileAccess.Read,
FileShare.ReadWrite))
{
var messageAttachment = new Attachment(fileStream, Path.GetFileName(f));
msgMail.Attachments.Add(messageAttachment);
}
Of course you can read and write from/to the same file at the same time (by different threads/processes).
Here is a sample code. Just see how FileStream is created.
string fname = "a.txt";
//WRITER
Task.Factory.StartNew(() =>
{
var f = new FileStream(fname, FileMode.OpenOrCreate, FileAccess.Write, FileShare.ReadWrite);
var s = new StreamWriter(f);
long l = 0;
while (true)
{
s.WriteLine(l++);
s.Flush();
Task.Delay(1000).Wait();
}
});
//READER
Task.Factory.StartNew(() =>
{
Task.Delay(1000).Wait();
var f = new FileStream(fname, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
var s = new StreamReader(f);
while (true)
{
var line = s.ReadLine();
if (line == null) { Task.Delay(100).Wait(); continue; };
Console.WriteLine("> " + line + " <");
}
});
It seems that you are not using the Dispose() and Close() methods of StreamWriter class to release the file.
You need to release control of the file from Program A. Try closing or disposing the streamwriter when you finish.
Or you might attempt using as is described in the answer to this question: Releasing access to files
I have 2 applications, one is writing to a file, and the other one reads the file. It's a log file, so the writer will be logging until the program stops, while the reader could be invoked any time to get the content of the file.
I thought that when the writer opens the file with FileShare.Read, the reader would be able to access the file, but it produces an error saying that the file is being used by another process.
Writer Application:
FileStream fs = new FileStream("file.log", FileMode.Open, FileAccess.Write, FileShare.Read);
BinaryWriter writer = new BinaryWriter(fs);
Reader Application:
BinaryReader reader = new BinaryReader(File.OpenRead("file.log"));
How do I prevent this error?
Can you try specifying FileShare.Read while reading the file also? Instead of directly using File.OpenRead use FileStream with this permission.
Also, for logging, you can use log4Net or any other free logging framework which manages logging so efficiently and we do not have to manage writing to files.
o read a locked file you are going to need to provide more flags for the FileStream.
Code such as below.
using (var reader = new FileStream("d:\\test.txt", FileMode.Open, FileAccess.Read,FileShare.ReadWrite))
{
using (var binary = new BinaryReader(reader))
{
//todo: add your code
binary.Close();
}
reader.Close();
}
This would open the file for reading only with the share mode of read write. This can be tested with a small app. (Using streamreader\write instead of binary)
static Thread writer,
reader;
static bool abort = false;
static void Main(string[] args)
{
var fs = File.Create("D:\\test.txt");
fs.Dispose();
writer = new Thread(new ThreadStart(testWriteLoop));
reader = new Thread(new ThreadStart(testReadLoop));
writer.Start();
reader.Start();
Console.ReadKey();
abort = true;
}
static void testWriteLoop()
{
using (FileStream fs = new FileStream("d:\\test.txt", FileMode.Open, FileAccess.Write, FileShare.Read))
{
using (var writer = new StreamWriter(fs))
{
while (!abort)
{
writer.WriteLine(DateTime.Now.ToString());
writer.Flush();
Thread.Sleep(1000);
}
}
}
}
static void testReadLoop()
{
while (!abort)
{
Thread.Sleep(1000);
using (var reader = new FileStream("d:\\test.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
using (var stream = new StreamReader(reader))
{
Console.WriteLine(stream.ReadToEnd());
stream.Close();
}
reader.Close();
}
}
}
I realize the example above is pretty simple but the fact still remains that the "testWriteLoop" never releases the lock.
Hope this helps
In Linux, a lot of IPC is done by appending to a file in 1 process and reading the new content from another process.
I want to do the above in Windows/.NET (Too messy to use normal IPC such as pipes). I'm appending to a file from a Python process, and I want to read the changes and ONLY the changes each time FileSystemWatcher reports an event. I do not want to read the entire file content into memory each time I'm looking for changes (the file will be huge)
Each append operation appends a row of data that starts with a unique incrementing counter (timestamp+key) and ends with a newline.
using (FileStream fs = new FileStream
(fileName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
using (StreamReader sr = new StreamReader(fs))
{
while (someCondition)
{
while (!sr.EndOfStream)
ProcessLinr(sr.ReadLine());
while (sr.EndOfStream)
Thread.Sleep(100);
ProcessLinr(sr.ReadLine());
}
}
}
this will help you read only appended lines
You can store the offset of the last read operation and seek the file to that offset when you get a changed file notification. An example follows:
Main method:
public static void Main(string[] args)
{
File.WriteAllLines("test.txt", new string[] { });
new Thread(() => ReadFromFile()).Start();
WriteToFile();
}
Read from file method:
private static void ReadFromFile()
{
long offset = 0;
FileSystemWatcher fsw = new FileSystemWatcher
{
Path = Environment.CurrentDirectory,
Filter = "test.txt"
};
FileStream file = File.Open(
"test.txt",
FileMode.Open,
FileAccess.Read,
FileShare.Write);
StreamReader reader = new StreamReader(file);
while (true)
{
fsw.WaitForChanged(WatcherChangeTypes.Changed);
file.Seek(offset, SeekOrigin.Begin);
if (!reader.EndOfStream)
{
do
{
Console.WriteLine(reader.ReadLine());
} while (!reader.EndOfStream);
offset = file.Position;
}
}
}
Write to file method:
private static void WriteToFile()
{
for (int i = 0; i < 100; i++)
{
FileStream writeFile = File.Open(
"test.txt",
FileMode.Append,
FileAccess.Write,
FileShare.Read);
using (FileStream file = writeFile)
{
using (StreamWriter sw = new StreamWriter(file))
{
sw.WriteLine(i);
Thread.Sleep(100);
}
}
}
}