I simply want to merge all text files in a given directory, similar to the following command prompt command:
cd $directory
copy * result.txt
I've written the following code, which almost accomplishes what I want, but it's doing something strange. When the StreamWriter writes the first file (or when i = 0), it doesn't actually write any content - the file size remains 0 bytes, despite the first file being ~300 KB. However, the other file writes execute successfully.
If I compare the output from the command prompt to the output from the C# code in diff, you can see that a large block of text is missing. Additionally, the command prompt result is 1,044 KB where the C# result is 700 KB.
string[] txtFiles = Directory.GetFiles(filepath);
using (StreamWriter writer = new StreamWriter(filepath + "result.txt"))
{
for (int i = 0; i < txtFiles.Length; i++)
{
using (StreamReader reader = File.OpenText(txtFiles[i]))
{
writer.Write(reader.ReadToEnd());
}
}
}
Am I using the StreamWriter / StreamReader incorrectly?
Minimalistic implementation, reading the bytes and writing them instead of using a stream for reading - please note, that you should handle the IOException correctly to avoid misbehavior:
var newline = Encoding.ASCII.GetBytes(Environment.NewLine);
var files = Directory.GetFiles(filepath);
try
{
using (var writer = File.Open(Path.Combine(filepath, "result.txt"), FileMode.Create))
foreach (var text in files.Select(File.ReadAllBytes))
{
writer.Write(text, 0, text.Length);
writer.Write(newline, 0, newline.Length);
}
}
catch (IOException)
{
// File might be used by different process or you have insufficient permissions
}
Here, hope it helps you. Note: By copying from a stream to another you save some ram and greatly improve performance.
class Program
{
static void Main(string[] args)
{
string filePath = #"C:\Users\FunkyName\Desktop";
string[] txtFiles = Directory.GetFiles(filePath, "*.txt");
using (Stream stream = File.Open(Path.Combine(filePath, "result.txt"), FileMode.OpenOrCreate))
{
for (int i = 0; i < txtFiles.Length; i++)
{
string fileName = txtFiles[i];
try
{
using (Stream fileStream = File.Open(fileName, FileMode.Open, FileAccess.Read))
{
fileStream.CopyTo(stream);
}
}
catch (IOException e)
{
// Handle file open exception
}
}
}
}
}
I wrote your code , it works properly! only change the line :
using (StreamWriter writer = new StreamWriter(filepath + "result.txt"))
to:
using (StreamWriter writer = new StreamWriter(filepath + "/result.txt"))
i guess you can't see the file because it is saved in another folder .
Related
I have files (from 3rd parties) that are being FTP'd to a directory on our server. I download them and process them even 'x' minutes. Works great.
Now, some of the files are .zip files. Which means I can't process them. I need to unzip them first.
FTP has no concept of zip/unzipping - so I'll need to grab the zip file, unzip it, then process it.
Looking at the MSDN zip api, there seems to be no way i can unzip to a memory stream?
So is the only way to do this...
Unzip to a file (what directory? need some -very- temp location ...)
Read the file contents
Delete file.
NOTE: The contents of the file are small - say 4k <-> 1000k.
Zip compression support is built in:
using System.IO;
using System.IO.Compression;
// ^^^ requires a reference to System.IO.Compression.dll
static class Program
{
const string path = ...
static void Main()
{
using(var file = File.OpenRead(path))
using(var zip = new ZipArchive(file, ZipArchiveMode.Read))
{
foreach(var entry in zip.Entries)
{
using(var stream = entry.Open())
{
// do whatever we want with stream
// ...
}
}
}
}
}
Normally you should avoid copying it into another stream - just use it "as is", however, if you absolutely need it in a MemoryStream, you could do:
using(var ms = new MemoryStream())
{
stream.CopyTo(ms);
ms.Position = 0; // rewind
// do something with ms
}
You can use ZipArchiveEntry.Open to get a stream.
This code assumes the zip archive has one text file.
using (FileStream fs = new FileStream(path, FileMode.Open))
using (ZipArchive zip = new ZipArchive(fs) )
{
var entry = zip.Entries.First();
using (StreamReader sr = new StreamReader(entry.Open()))
{
Console.WriteLine(sr.ReadToEnd());
}
}
using (ZipArchive archive = new ZipArchive(webResponse.GetResponseStream()))
{
foreach (ZipArchiveEntry entry in archive.Entries)
{
Stream s = entry.Open();
var sr = new StreamReader(s);
var myStr = sr.ReadToEnd();
}
}
Looks like here is what you need:
using (var za = ZipFile.OpenRead(path))
{
foreach (var entry in za.Entries)
{
using (var r = new StreamReader(entry.Open()))
{
//your code here
}
}
}
You can use SharpZipLib among a variety of other libraries to achieve this.
You can use the following code example to unzip to a MemoryStream, as shown on their wiki:
using ICSharpCode.SharpZipLib.Zip;
// Compresses the supplied memory stream, naming it as zipEntryName, into a zip,
// which is returned as a memory stream or a byte array.
//
public MemoryStream CreateToMemoryStream(MemoryStream memStreamIn, string zipEntryName) {
MemoryStream outputMemStream = new MemoryStream();
ZipOutputStream zipStream = new ZipOutputStream(outputMemStream);
zipStream.SetLevel(3); //0-9, 9 being the highest level of compression
ZipEntry newEntry = new ZipEntry(zipEntryName);
newEntry.DateTime = DateTime.Now;
zipStream.PutNextEntry(newEntry);
StreamUtils.Copy(memStreamIn, zipStream, new byte[4096]);
zipStream.CloseEntry();
zipStream.IsStreamOwner = false; // False stops the Close also Closing the underlying stream.
zipStream.Close(); // Must finish the ZipOutputStream before using outputMemStream.
outputMemStream.Position = 0;
return outputMemStream;
// Alternative outputs:
// ToArray is the cleaner and easiest to use correctly with the penalty of duplicating allocated memory.
byte[] byteArrayOut = outputMemStream.ToArray();
// GetBuffer returns a raw buffer raw and so you need to account for the true length yourself.
byte[] byteArrayOut = outputMemStream.GetBuffer();
long len = outputMemStream.Length;
}
Ok so combining all of the above, suppose you want to in a very simple way take a zip file called
"file.zip" and extract it to "C:\temp" folder. (Note: This example was only tested for compress text files) You may need to do some modifications for binary files.
using System.IO;
using System.IO.Compression;
static void Main(string[] args)
{
//Call it like this:
Unzip("file.zip",#"C:\temp");
}
static void Unzip(string sourceZip, string targetPath)
{
using (var z = ZipFile.OpenRead(sourceZip))
{
foreach (var entry in z.Entries)
{
using (var r = new StreamReader(entry.Open()))
{
string uncompressedFile = Path.Combine(targetPath, entry.Name);
File.WriteAllText(uncompressedFile,r.ReadToEnd());
}
}
}
}
I am doing a project Windows form for assignment in Uni, I want to search an already created text file to match a first name and last name then write some additional information if the name and last name exist. I have the code constructed and showing no errors, however when I run and attempt to add information I am being provided with an error which essentially says the next process (Streamreader writer can not access the file as it is already in use by another process) I assume this process is streamreader, I have tried to code it to stop reading to no avail. I am in my first 3 months learning coding and would appreciate some assistance if possible, I have put a snippet of my code below.
//check if there is a file with that name
if (File.Exists(sFile))
{
using (StreamReader sr = new StreamReader(sFile))
{
//while there is more data to read
while (sr.Peek() != -1)
{
//read first name and last name
sFirstName = sr.ReadLine();
sLastName = sr.ReadLine();
}
{
//does this name match?
if (sFirstName + sLastName == txtSearchName.Text)
sr.Close();
}
//Process write to file
using (StreamWriter sw = new StreamWriter(sFile, true))
{
sw.WriteLine("First Name:" + sFirstName);
sw.WriteLine("Last Name:" + sLastName);
sw.WriteLine("Gender:" + sGender);
}
You are using your writer inside the reader, using the same file.
A using disposes the object inside it, after the closing curly braces.
using(StreamReader reader = new StreamReader("foo")){
//... some stuff
using(Streamwriter writer = new StreamWriter("foo")){
}
}
Do it like so :
using(StreamReader reader = new StreamReader("foo")){
//... some stuff
}
using(Streamwriter writer = new StreamWriter("foo")){
}
As per my comment regarding the using statement.
Rearrange to the below. I've tested locally and it seems to work.
using (StreamReader sr = new StreamReader(sfile))
{
//while there is more data to read
while (sr.Peek() != -1)
{
//read first name and last name
sFirstName = sr.ReadLine();
sLastName = sr.ReadLine();
//does this name match?
if (sFirstName + sLastName == txtSearchName.Text)
break;
}
}
using (StreamWriter sw = new StreamWriter(sfile, true))
{
sw.WriteLine("First Name:" + sFirstName);
sw.WriteLine("Last Name:" + sLastName);
sw.WriteLine("Gender:" + sGender);
}
I've replaced the sr.Close with a break statement to exit out. Closing the reader causes the subsequent peek to error as it's closed.
Also, I've noticed that you are not setting gender? unless its set elsewhere.
hope that helps
You can use FileStream. It gives you many options to work with file:
var fileStream = new FileStream("FileName", FileMode.Open,
FileAccess.Write, FileShare.ReadWrite);
var fileStream = new FileStream("fileName", FileMode.Open,
FileAccess.ReadWrite, FileShare.ReadWrite);
I think this is what you want/need. You can't append to a file the way you are trying to do it. Instead you'll want to read your input file, and write a temp file as you are reading through. And, whenever your line matches your requirements, then you can write the line with your modifications.
string inputFile = "C:\\temp\\StreamWriterSample.txt";
string tempFile = "C:\\temp\\StreamWriterSampleTemp.txt";
using (StreamWriter sw = new StreamWriter(tempFile))//get a writer ready
{
using (StreamReader sr = new StreamReader(inputFile))//get a reader ready
{
string currentLine = string.Empty;
while ((currentLine = sr.ReadLine()) != null)
{
if (currentLine.Contains("Clients"))
{
sw.WriteLine(currentLine + " modified");
}
else
{
sw.WriteLine(currentLine);
}
}
}
}
//now lets crush the old file with the new file
File.Copy(tempFile, inputFile, true);
I'm currently trying to read a file, modify a few placeholders within and then write the file into an output stream. As its the output stream for a page response in aspx.net I'm using the OutputStream.Write method there (the file is an attachment in the end).
Originally I had:
using (FileStream fs = new FileStream(filename, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
while (readBytes < fs.Length)
{
tmpReadBytes = fs.Read(bytes, 0, bytes.Length);
if (tmpReadBytes > 0)
{
readBytes += tmpReadBytes;
page.Response.OutputStream.Write(bytes, 0, tmpReadBytes);
}
}
}
After thinking things over I came up with the following:
foreach(string line in File.ReadLines(filename))
{
string modifiedLine = line.Replace("#PlaceHolder#", "NewValue");
byte[] modifiedByteArray = System.Text.Encoding.UTF8.GetBytes(modifiedLine);
page.Response.OutputStream.Write(modifiedByteArray, 0, modifiedByteArray.length);
}
But it looks inefficient especially with the conversions. So my question is: Is there any better way of doing this?
As note the file itself is not very big, it's an about 3-4 KB sized textfile.
You don't need to handle the bytes your self.
If you know the file is and always will be small,
this.Response.Write(File.ReadAllText("path").Replace("old", "new"));
otherwise
using (var stream = new FileStream("path", FileMode.Open))
{
using (var streamReader = new StreamReader(stream))
{
while (streamReader.Peek() != -1)
{
this.Response.Write(streamReader.ReadLine().Replace("old", "new"));
}
}
}
To get the lines in a string array:
string[] lines = File.ReadAllLines(file);
To alter the lines, use a loop.
for (int i = 0; i < lines.Length; i++)
{
lines[i] = lines[i].Replace("#PlaceHolder#", "NewValue");
}
And to save the new text, first create a string with all the lines.
string output = "";
foreach(string line in lines)
{
output+="\n"+line;
}
And then save the string to the file.
File.WriteAllText(file,output);
I have files (from 3rd parties) that are being FTP'd to a directory on our server. I download them and process them even 'x' minutes. Works great.
Now, some of the files are .zip files. Which means I can't process them. I need to unzip them first.
FTP has no concept of zip/unzipping - so I'll need to grab the zip file, unzip it, then process it.
Looking at the MSDN zip api, there seems to be no way i can unzip to a memory stream?
So is the only way to do this...
Unzip to a file (what directory? need some -very- temp location ...)
Read the file contents
Delete file.
NOTE: The contents of the file are small - say 4k <-> 1000k.
Zip compression support is built in:
using System.IO;
using System.IO.Compression;
// ^^^ requires a reference to System.IO.Compression.dll
static class Program
{
const string path = ...
static void Main()
{
using(var file = File.OpenRead(path))
using(var zip = new ZipArchive(file, ZipArchiveMode.Read))
{
foreach(var entry in zip.Entries)
{
using(var stream = entry.Open())
{
// do whatever we want with stream
// ...
}
}
}
}
}
Normally you should avoid copying it into another stream - just use it "as is", however, if you absolutely need it in a MemoryStream, you could do:
using(var ms = new MemoryStream())
{
stream.CopyTo(ms);
ms.Position = 0; // rewind
// do something with ms
}
You can use ZipArchiveEntry.Open to get a stream.
This code assumes the zip archive has one text file.
using (FileStream fs = new FileStream(path, FileMode.Open))
using (ZipArchive zip = new ZipArchive(fs) )
{
var entry = zip.Entries.First();
using (StreamReader sr = new StreamReader(entry.Open()))
{
Console.WriteLine(sr.ReadToEnd());
}
}
using (ZipArchive archive = new ZipArchive(webResponse.GetResponseStream()))
{
foreach (ZipArchiveEntry entry in archive.Entries)
{
Stream s = entry.Open();
var sr = new StreamReader(s);
var myStr = sr.ReadToEnd();
}
}
Looks like here is what you need:
using (var za = ZipFile.OpenRead(path))
{
foreach (var entry in za.Entries)
{
using (var r = new StreamReader(entry.Open()))
{
//your code here
}
}
}
You can use SharpZipLib among a variety of other libraries to achieve this.
You can use the following code example to unzip to a MemoryStream, as shown on their wiki:
using ICSharpCode.SharpZipLib.Zip;
// Compresses the supplied memory stream, naming it as zipEntryName, into a zip,
// which is returned as a memory stream or a byte array.
//
public MemoryStream CreateToMemoryStream(MemoryStream memStreamIn, string zipEntryName) {
MemoryStream outputMemStream = new MemoryStream();
ZipOutputStream zipStream = new ZipOutputStream(outputMemStream);
zipStream.SetLevel(3); //0-9, 9 being the highest level of compression
ZipEntry newEntry = new ZipEntry(zipEntryName);
newEntry.DateTime = DateTime.Now;
zipStream.PutNextEntry(newEntry);
StreamUtils.Copy(memStreamIn, zipStream, new byte[4096]);
zipStream.CloseEntry();
zipStream.IsStreamOwner = false; // False stops the Close also Closing the underlying stream.
zipStream.Close(); // Must finish the ZipOutputStream before using outputMemStream.
outputMemStream.Position = 0;
return outputMemStream;
// Alternative outputs:
// ToArray is the cleaner and easiest to use correctly with the penalty of duplicating allocated memory.
byte[] byteArrayOut = outputMemStream.ToArray();
// GetBuffer returns a raw buffer raw and so you need to account for the true length yourself.
byte[] byteArrayOut = outputMemStream.GetBuffer();
long len = outputMemStream.Length;
}
Ok so combining all of the above, suppose you want to in a very simple way take a zip file called
"file.zip" and extract it to "C:\temp" folder. (Note: This example was only tested for compress text files) You may need to do some modifications for binary files.
using System.IO;
using System.IO.Compression;
static void Main(string[] args)
{
//Call it like this:
Unzip("file.zip",#"C:\temp");
}
static void Unzip(string sourceZip, string targetPath)
{
using (var z = ZipFile.OpenRead(sourceZip))
{
foreach (var entry in z.Entries)
{
using (var r = new StreamReader(entry.Open()))
{
string uncompressedFile = Path.Combine(targetPath, entry.Name);
File.WriteAllText(uncompressedFile,r.ReadToEnd());
}
}
}
}
In Linux, a lot of IPC is done by appending to a file in 1 process and reading the new content from another process.
I want to do the above in Windows/.NET (Too messy to use normal IPC such as pipes). I'm appending to a file from a Python process, and I want to read the changes and ONLY the changes each time FileSystemWatcher reports an event. I do not want to read the entire file content into memory each time I'm looking for changes (the file will be huge)
Each append operation appends a row of data that starts with a unique incrementing counter (timestamp+key) and ends with a newline.
using (FileStream fs = new FileStream
(fileName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
using (StreamReader sr = new StreamReader(fs))
{
while (someCondition)
{
while (!sr.EndOfStream)
ProcessLinr(sr.ReadLine());
while (sr.EndOfStream)
Thread.Sleep(100);
ProcessLinr(sr.ReadLine());
}
}
}
this will help you read only appended lines
You can store the offset of the last read operation and seek the file to that offset when you get a changed file notification. An example follows:
Main method:
public static void Main(string[] args)
{
File.WriteAllLines("test.txt", new string[] { });
new Thread(() => ReadFromFile()).Start();
WriteToFile();
}
Read from file method:
private static void ReadFromFile()
{
long offset = 0;
FileSystemWatcher fsw = new FileSystemWatcher
{
Path = Environment.CurrentDirectory,
Filter = "test.txt"
};
FileStream file = File.Open(
"test.txt",
FileMode.Open,
FileAccess.Read,
FileShare.Write);
StreamReader reader = new StreamReader(file);
while (true)
{
fsw.WaitForChanged(WatcherChangeTypes.Changed);
file.Seek(offset, SeekOrigin.Begin);
if (!reader.EndOfStream)
{
do
{
Console.WriteLine(reader.ReadLine());
} while (!reader.EndOfStream);
offset = file.Position;
}
}
}
Write to file method:
private static void WriteToFile()
{
for (int i = 0; i < 100; i++)
{
FileStream writeFile = File.Open(
"test.txt",
FileMode.Append,
FileAccess.Write,
FileShare.Read);
using (FileStream file = writeFile)
{
using (StreamWriter sw = new StreamWriter(file))
{
sw.WriteLine(i);
Thread.Sleep(100);
}
}
}
}