Develop Reference

How do I replicate the functionality of tail -f in C# [duplicate]

I want to read file continuously like GNU tail with "-f" param. I need it to live-read log file.
What is the right way to do it?

More natural approach of using FileSystemWatcher:
var wh = new AutoResetEvent(false);
var fsw = new FileSystemWatcher(".");
fsw.Filter = "file-to-read";
fsw.EnableRaisingEvents = true;
fsw.Changed += (s,e) => wh.Set();
var fs = new FileStream("file-to-read", FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
using (var sr = new StreamReader(fs))
{
var s = "";
while (true)
{
s = sr.ReadLine();
if (s != null)
Console.WriteLine(s);
else
wh.WaitOne(1000);
}
}
wh.Close();
Here the main reading cycle stops to wait for incoming data and FileSystemWatcher is used just to awake the main reading cycle.

You want to open a FileStream in binary mode. Periodically, seek to the end of the file minus 1024 bytes (or whatever), then read to the end and output. That's how tail -f works.
Answers to your questions:
Binary because it's difficult to randomly access the file if you're reading it as text. You have to do the binary-to-text conversion yourself, but it's not difficult. (See below)
1024 bytes because it's a nice convenient number, and should handle 10 or 15 lines of text. Usually.
Here's an example of opening the file, reading the last 1024 bytes, and converting it to text:
static void ReadTail(string filename)
{
using (FileStream fs = File.Open(filename, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
// Seek 1024 bytes from the end of the file
fs.Seek(-1024, SeekOrigin.End);
// read 1024 bytes
byte[] bytes = new byte[1024];
fs.Read(bytes, 0, 1024);
// Convert bytes to string
string s = Encoding.Default.GetString(bytes);
// or string s = Encoding.UTF8.GetString(bytes);
// and output to console
Console.WriteLine(s);
}
}
Note that you must open with FileShare.ReadWrite, since you're trying to read a file that's currently open for writing by another process.
Also note that I used Encoding.Default, which in US/English and for most Western European languages will be an 8-bit character encoding. If the file is written in some other encoding (like UTF-8 or other Unicode encoding), It's possible that the bytes won't convert correctly to characters. You'll have to handle that by determining the encoding if you think this will be a problem. Search Stack overflow for info about determining a file's text encoding.
If you want to do this periodically (every 15 seconds, for example), you can set up a timer that calls the ReadTail method as often as you want. You could optimize things a bit by opening the file only once at the start of the program. That's up to you.

To continuously monitor the tail of the file, you just need to remember the length of the file before.
public static void MonitorTailOfFile(string filePath)
{
var initialFileSize = new FileInfo(filePath).Length;
var lastReadLength = initialFileSize - 1024;
if (lastReadLength < 0) lastReadLength = 0;
while (true)
{
try
{
var fileSize = new FileInfo(filePath).Length;
if (fileSize > lastReadLength)
{
using (var fs = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
fs.Seek(lastReadLength, SeekOrigin.Begin);
var buffer = new byte[1024];
while (true)
{
var bytesRead = fs.Read(buffer, 0, buffer.Length);
lastReadLength += bytesRead;
if (bytesRead == 0)
break;
var text = ASCIIEncoding.ASCII.GetString(buffer, 0, bytesRead);
Console.Write(text);
}
}
}
}
catch { }
Thread.Sleep(1000);
}
}
I had to use ASCIIEncoding, because this code isn't smart enough to cater for variable character lengths of UTF8 on buffer boundaries.
Note: You can change the Thread.Sleep part to be different timings, and you can also link it with a filewatcher and blocking pattern - Monitor.Enter/Wait/Pulse. For me the timer is enough, and at most it only checks the file length every second, if the file hasn't changed.

This is my solution
static IEnumerable<string> TailFrom(string file)
{
using (var reader = File.OpenText(file))
{
while (true)
{
string line = reader.ReadLine();
if (reader.BaseStream.Length < reader.BaseStream.Position)
reader.BaseStream.Seek(0, SeekOrigin.Begin);
if (line != null) yield return line;
else Thread.Sleep(500);
}
}
}
so, in your code you can do
foreach (string line in TailFrom(file))
{
Console.WriteLine($"line read= {line}");
}

You could use the FileSystemWatcher class which can send notifications for different events happening on the file system like file changed.

private void button1_Click(object sender, EventArgs e)
{
if (folderBrowserDialog.ShowDialog() == DialogResult.OK)
{
path = folderBrowserDialog.SelectedPath;
fileSystemWatcher.Path = path;
string[] str = Directory.GetFiles(path);
string line;
fs = new FileStream(str[0], FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
tr = new StreamReader(fs);
while ((line = tr.ReadLine()) != null)
{
listBox.Items.Add(line);
}
}
}
private void fileSystemWatcher_Changed(object sender, FileSystemEventArgs e)
{
string line;
line = tr.ReadLine();
listBox.Items.Add(line);
}

If you are just looking for a tool to do this then check out free version of Bare tail

C# unzip a file: Error with 'Thumbs.db'

I wrote a program that unzip a file (.zip) using SharpZipLib...
The following code:
public void UnZip(string zipFilePath, string extractionPath)
{
FastZip fz = new FastZip();
fz.ExtractZip(zipFilePath, extractionPath, null);
}
I get the following Exception:
Additional information: The access to the path "C:\Program files (x86)\... Thumbs.db" was refused.
The program starts with Admin rights and the file "Thumbs.db" does not exist in the .zip archive.
Who knows further?
Greets and thanks!

I would ignore the "Thumbs.db" file as its an OS file.
Maybe something like this:
using ICSharpCode.SharpZipLib.Core;
using ICSharpCode.SharpZipLib.Zip;
public void ExtractZipFile(string archiveFilenameIn, string password, string outFolder) {
ZipFile zf = null;
try {
FileStream fs = File.OpenRead(archiveFilenameIn);
zf = new ZipFile(fs);
if (!String.IsNullOrEmpty(password)) {
zf.Password = password; // AES encrypted entries are handled automatically
}
foreach (ZipEntry zipEntry in zf) {
if (!zipEntry.IsFile) {
continue; // Ignore directories
}
String entryFileName = zipEntry.Name;
// to remove the folder from the entry:- entryFileName = Path.GetFileName(entryFileName);
// Optionally match entrynames against a selection list here to skip as desired.
// The unpacked length is available in the zipEntry.Size property.
byte[] buffer = new byte[4096]; // 4K is optimum
Stream zipStream = zf.GetInputStream(zipEntry);
// Manipulate the output filename here as desired.
String fullZipToPath = Path.Combine(outFolder, entryFileName);
string directoryName = Path.GetDirectoryName(fullZipToPath);
if (directoryName.Length > 0)
Directory.CreateDirectory(directoryName);
// Unzip file in buffered chunks. This is just as fast as unpacking to a buffer the full size
// of the file, but does not waste memory.
// The "using" will close the stream even if an exception occurs.
using (FileStream streamWriter = File.Create(fullZipToPath)) {
StreamUtils.Copy(zipStream, streamWriter, buffer);
}
}
} finally {
if (zf != null) {
zf.IsStreamOwner = true; // Makes close also shut the underlying stream
zf.Close(); // Ensure we release resources
}
}
}

Get zip file entries in .NET 2.0

I need to get the number of entries and original file size of each entry in a ZIP file. On .NET 4.5, we can use the C# classes ZipArchive and ZipFile as shown below, but these classes are not available in .NET 2.0. What are my options then?
using (ZipArchive archive = ZipFile.OpenRead(fileName))
{
foreach (ZipArchiveEntry entry in archive.Entries)
{
Console.WriteLine("[INFO] Zip entry {0} original file size: [{1}]", entry.name, entry.Length);
}
}
Thanks in advance.

Using [ICsharpZipLib] (https://icsharpcode.github.io/SharpZipLib/)
using ICSharpCode.SharpZipLib.Core;
using ICSharpCode.SharpZipLib.Zip;
public void ExtractZipFile(string archiveFilenameIn, string password, string outFolder) {
ZipFile zf = null;
try {
FileStream fs = File.OpenRead(archiveFilenameIn);
zf = new ZipFile(fs);
if (!String.IsNullOrEmpty(password)) {
zf.Password = password; // AES encrypted entries are handled automatically
}
foreach (ZipEntry zipEntry in zf) {
if (!zipEntry.IsFile) {
continue; // Ignore directories
}
String entryFileName = zipEntry.Name;
// to remove the folder from the entry:- entryFileName = Path.GetFileName(entryFileName);
// Optionally match entrynames against a selection list here to skip as desired.
// The unpacked length is available in the zipEntry.Size property.
byte[] buffer = new byte[4096]; // 4K is optimum
Stream zipStream = zf.GetInputStream(zipEntry);
// Manipulate the output filename here as desired.
String fullZipToPath = Path.Combine(outFolder, entryFileName);
string directoryName = Path.GetDirectoryName(fullZipToPath);
if (directoryName.Length > 0)
Directory.CreateDirectory(directoryName);
// Unzip file in buffered chunks. This is just as fast as unpacking to a buffer the full size
// of the file, but does not waste memory.
// The "using" will close the stream even if an exception occurs.
using (FileStream streamWriter = File.Create(fullZipToPath)) {
StreamUtils.Copy(zipStream, streamWriter, buffer);
}
}
} finally {
if (zf != null) {
zf.IsStreamOwner = true; // Makes close also shut the underlying stream
zf.Close(); // Ensure we release resources
}
}
}

Unzipping a Stream in C#

I'm working in C#, and I'm downloading for the internet a zip file with one XML file in it. and I wish to load this XML file. This is what I have so far:
byte[] data;
WebClient webClient = new WebClient();
try {
data = webClient.DownloadData(downloadUrl);
}
catch (Exception ex) {
Console.WriteLine("Error in DownloadData (Ex:{0})", ex.Message);
throw;
}
if (data == null) {
Console.WriteLine("Bulk data is null");
throw new Exception("Bulk data is null");
}
//Create the stream
MemoryStream stream = new MemoryStream(data);
XmlDocument document = new XmlDocument();
//Gzip
GZipStream gzipStream = new GZipStream(stream, CompressionMode.Decompress);
//Load report straight from the gzip stream
try {
document.Load(gzipStream);
}
catch (Exception ex) {
Console.WriteLine("Error in Load (Ex:{0})", ex.Message);
throw;
}
in document.Load I'm always getting the following exception:
The magic number in GZip header is not correct. Make sure you are passing in a GZip stream.
What I'm doing wrong?

Apparently SharpZipLib is now unmaintained and you probably want to avoid it:
https://stackoverflow.com/a/593030
In .NET 4.5 there is now built in support for zip files, so for your example it would be:
var data = new WebClient().DownloadData(downloadUrl);
//Create the stream
var stream = new MemoryStream(data);
var document = new XmlDocument();
//zip
var zipArchive = new ZipArchive(stream);
//Load report straight from the zip stream
document.Load(zipArchive.Entries[0].Open());

If you have a byte array that contains a zip archive with a single file, you can use the ZipArchive class to get an unzipped byte array with the file's data.
ZipArchive is contained in .NET 4.5, in the assembly System.IO.Compression.FileSystem (you need to reference it explicitly).
The following function, adapted from this answer, works for me:
public static byte[] UnzipSingleEntry(byte[] zipped)
{
using (var memoryStream = new MemoryStream(zipped))
{
using (var archive = new ZipArchive(memoryStream))
{
foreach (ZipArchiveEntry entry in archive.Entries)
{
using (var entryStream = entry.Open())
{
using (var reader = new BinaryReader(entryStream))
{
return reader.ReadBytes((int)entry.Length);
}
}
}
}
}
return null; // To quiet my compiler
}

I am using SharpZipLib and it's working great !
Below is a function that encapsulate the library
public static void Compress(FileInfo sourceFile, string destinationFileName,string destinationTempFileName)
{
Crc32 crc = new Crc32();
string zipFile = Path.Combine(sourceFile.Directory.FullName, destinationTempFileName);
zipFile = Path.ChangeExtension(zipFile, ZIP_EXTENSION);
using (FileStream fs = File.Create(zipFile))
{
using (ZipOutputStream zOut = new ZipOutputStream(fs))
{
zOut.SetLevel(9);
ZipEntry entry = new ZipEntry(ZipEntry.CleanName(destinationFileName));
entry.DateTime = DateTime.Now;
entry.ZipFileIndex = 1;
entry.Size = sourceFile.Length;
using (FileStream sourceStream = sourceFile.OpenRead())
{
crc.Reset();
long len = sourceFile.Length;
byte[] buffer = new byte[bufferSize];
while (len > 0)
{
int readSoFar = sourceStream.Read(buffer, 0, buffer.Length);
crc.Update(buffer, 0, readSoFar);
len -= readSoFar;
}
entry.Crc = crc.Value;
zOut.PutNextEntry(entry);
len = sourceStream.Length;
sourceStream.Seek(0, SeekOrigin.Begin);
while (len > 0)
{
int readSoFar = sourceStream.Read(buffer, 0, buffer.Length);
zOut.Write(buffer, 0, readSoFar);
len -= readSoFar;
}
}
zOut.Finish();
zOut.Close();
}
fs.Close();
}
}

As the others have mentioned GZip and Zip are not the same so you might need to use a zip library. I use a library called: DotNetZip - available from the below site:
http://dotnetzip.codeplex.com/

From GZipStream Class description:
Compressed GZipStream objects written to a file with an extension of .gz can be decompressed using many common compression tools; however, this class does not inherently provide functionality for adding files to or extracting files from .zip archives
So unless you control server-side files, I'd suggest looking for specific zip-targeted library (SharpZipLib for example).

.Net Zip Up files

Whats the best way to zip up files using C#? Ideally I want to be able to seperate files into a single archive.

You can use DotNetZip to archieve this. It´s free to use in any application.
Here´s some sample code:
try
{
// for easy disposal
using (ZipFile zip = new ZipFile())
{
// add this map file into the "images" directory in the zip archive
zip.AddFile("c:\\images\\personal\\7440-N49th.png", "images");
// add the report into a different directory in the archive
zip.AddFile("c:\\Reports\\2008-Regional-Sales-Report.pdf", "files");
zip.AddFile("ReadMe.txt");
zip.Save("MyZipFile.zip");
}
}
catch (System.Exception ex1)
{
System.Console.Error.WriteLine("exception: " + ex1);
}

This is now built into the framework if you have version 4.5+
Otherwise, use Ionic.
Namespace is System.IO.Packaging.ZIPPackage.
See http://visualstudiomagazine.com/articles/2012/05/21/net-framework-gets-zip.aspx for a story.

Have you looked at SharpZipLib?
I believe you can build zip files with classes in the System.IO.Packaging namespace - but every time I've tried to look into it, I've found it rather confusing...

Take a look at this library:
http://www.icsharpcode.net/OpenSource/SharpZipLib/
It is pretty comprehensive, it deals with many formats, is open-source, and you can use in closed-source commercial applications.
It is very simple to use:
byte[] data1 = new byte[...];
byte[] data2 = new byte[...];
/*...*/
var path = #"c:\test.zip";
var zip = new ZipOutputStream(new FileStream(path, FileMode.Create))
{
IsStreamOwner = true
}
zip.PutNextEntry("File1.txt");
zip.Write(data1, 0, data1.Length);
zip.PutNextEntry("File2.txt");
zip.Write(data2, 0, data2.Length);
zip.Close();
zip.Dispose();

There are a few librarys around - the most popular of which are DotNetZip and SharpZipLib.

Hi i created two methods with the ShapLib library (you can download it here http://www.icsharpcode.net/opensource/sharpziplib/) that would like to share, they are very easy to use just pass source and target path (fullpath including folder/file and extension). Hope it help you!
//ALLYOURNAMESPACESHERE
using ...
//SHARPLIB
using ICSharpCode.SharpZipLib.Zip;
using ICSharpCode.SharpZipLib.Zip.Compression.Streams;
public static class FileUtils
{
/// <summary>
///
/// </summary>
/// <param name="sourcePath"></param>
/// <param name="targetPath"></param>
public static void ZipFile(string sourcePath, string targetPath)
{
string tempZipFilePath = targetPath;
using (FileStream tempFileStream = File.Create(tempZipFilePath, 1024))
{
using (ZipOutputStream zipOutput = new ZipOutputStream(tempFileStream))
{
// Zip with highest compression.
zipOutput.SetLevel(9);
DirectoryInfo directory = new DirectoryInfo(sourcePath);
foreach (System.IO.FileInfo file in directory.GetFiles())
{
// Get local path and create stream to it.
String localFilename = file.FullName;
//ignore directories or folders
//ignore Thumbs.db file since this probably will throw an exception
//another process could be using it. e.g: Explorer.exe windows process
if (!file.Name.Contains("Thumbs") && !Directory.Exists(localFilename))
{
using (FileStream fileStream = new FileStream(localFilename, FileMode.Open, FileAccess.Read, FileShare.Read))
{
// Read full stream to in-memory buffer.
byte[] buffer = new byte[fileStream.Length];
fileStream.Read(buffer, 0, buffer.Length);
// Create a new entry for the current file.
ZipEntry entry = new ZipEntry(file.Name);
entry.DateTime = DateTime.Now;
// set Size and the crc, because the information
// about the size and crc should be stored in the header
// if it is not set it is automatically written in the footer.
// (in this case size == crc == -1 in the header)
// Some ZIP programs have problems with zip files that don't store
// the size and crc in the header.
entry.Size = fileStream.Length;
fileStream.Close();
// Update entry and write to zip stream.
zipOutput.PutNextEntry(entry);
zipOutput.Write(buffer, 0, buffer.Length);
// Get rid of the buffer, because this
// is a huge impact on the memory usage.
buffer = null;
}
}
}
// Finalize the zip output.
zipOutput.Finish();
// Flushes the create and close.
zipOutput.Flush();
zipOutput.Close();
}
}
}
public static void unZipFile(string sourcePath, string targetPath)
{
if (!Directory.Exists(targetPath))
Directory.CreateDirectory(targetPath);
using (ZipInputStream s = new ZipInputStream(File.OpenRead(sourcePath)))
{
ZipEntry theEntry;
while ((theEntry = s.GetNextEntry()) != null)
{
if (theEntry.Name != String.Empty)
{
using (FileStream streamWriter = File.Create(targetPath + "\\" + theEntry.Name))
{
int size = 2048;
byte[] data = new byte[2048];
while (true)
{
size = s.Read(data, 0, data.Length);
if (size > 0)
{
streamWriter.Write(data, 0, size);
}
else
{
break;
}
}
}
}
}
}
}
}