Convert Windows file format to Linux format - c#

I am using Linux machine for development and then deploying my script files to Azure function app which is a Windows machine.
Then I am copying shell script files to wasb from Azure function app.
I am getting following error when running shell script in Edge node on Azure HDI cluster
$'\r': command not found
My script files are not executed properly.
What is the best way to convert dos2unix option in C# Azure function?


You don't have to do anything special to read a file that contains Unix-style newlines (\n) instead of \r\n. .NET IO methods treat both as newlines.
You could write
var lines-File.ReadAllLines("myUnixText.txt");
or
using(var reader=File.OpenText("myUnixText.txt"))
{
string line;
while( (line=reader.ReadLine()) !=null)
{
// Do something
}
}
to read lines whether the line ending is \r or \n
To prove it :
var numbers = new[]{1,2,3,4,5,6};
var lines=String.Join("\n",numbers);
File.WriteAllText("myUnixText.txt",lines);
var newLines=File.ReadAllLines("myUnixText.txt");
Debug.Assert(newLines.Length==6);
Even though only a single string was written, File.ReadAllLines read 6 lines from the file


You could upload your file to Linux and change file format.
dos2unix <filename>
Another easy solution is that if you install Notepad++, it supports automatic conversion format to Unix(LF).


I am calling Dos2Unix(filePath) method before copying file from Azure function to WASB.
Call method:-
Dos2Unix(D:\home\site\repository\sample.sh);
Following method actually works for me in C# Azure function.
private void Dos2Unix(string fileName)
{
const byte CR = 0x0D;
const byte LF = 0x0A;
byte[] data = File.ReadAllBytes(fileName);
using (FileStream fileStream = File.OpenWrite(fileName))
{
BinaryWriter bw = new BinaryWriter(fileStream);
int position = 0;
int index = 0;
do
{
index = Array.IndexOf<byte>(data, CR, position);
if ((index >= 0) && (data[index + 1] == LF))
{
// Write before the CR
bw.Write(data, position, index - position);
// from LF
position = index + 1;
}
}
while (index >= 0);
bw.Write(data, position, data.Length - position);
fileStream.SetLength(fileStream.Position);
}
}
Update 1:-
Here is my code to upload files(*.sh, *.jar, *.img, etc) to blob storage.
public bool UploadBlobFile(string containerName, string blobName, string filePath)
{
try{
CloudBlobContainer container = blobClient.GetContainerReference(containerName);
CloudBlockBlob blob = container.GetBlockBlobReference(blobName);
// convert dos2unix format
Dos2Unix(filePath);
using (var fileStream = System.IO.File.OpenRead(filePath))
{
blob.UploadFromStream(fileStream);
}
return true;
} catch (Exception e) {
log.Info("Exception: " + e);
return false;
}
}


It seems that you have Windows style line endings (\r\n) - you need to change them to unix style (\n).
Refer: '\r': command not found and Convert DOS line endings to Linux line endings in vim


There is a tool in sourceforge that can do it.
Is has been actualized recently a runs very well.

Related

Outputing to .txt file

I am a little stuck. I have to use the getItems method and output it to PrintItems.txt, but I am not sure how to appoach this problem.
This is my GetItems method:
public string getItems()
{
string strout = "stock items " + "\n";
foreach (Stock s in stock)
{
strout = strout + s.ToString() + "\n";
}
return strout;
}
This is my PrintItems method:
string filename = "printitems.txt";
int count = 0;
using (StreamWriter sw = new StreamWriter(filename))
{
while (count < 5 && count < stock.Count)
{
Stock t = stock[count];
sw.WriteLine(t);
count++;
}
}
It doesn't work because it doesn't write to a file at all.

You code generally should work.
But since you haven't specified full path to the text file - it will be created in the same folder where your executable file is.
If you running it from Visual Studio - it should be in your_project\bin\Debug or your_project\bin\Release folder.

You could use:
File.WriteAllText(#"C:\dummy.txt", strout);
(As long as you don't expect the string to be massive - i.e. 10MB)
'File' uses System.IO

If you made no copy & paste error then there are 4 possibilities for your pheonomenon:
string filename = "printitems.txt";
int count = 0;
using (StreamWriter sw = new StreamWriter(filename))
{
while (count < 5 && count < stock.Count)
{
Stock t = stock[count];
sw.WriteLine(t.ToString());
count++;
}
}
You had no { after the streamwriter using line was something I saw first. The second thing is that Stock t normally can't be written directly but instead you need to do the same thing as when printing it out to the console.
Third: your code does not say anything about if stock is filled or not.
Fourth: The file: You should specify a directory (not only the filename) as else it can be that it is tried to create the file in a location where you have no permissions to create a file in (normally if you put no additional path info in the same path as where the application runs in is used [if you start from the visual studio itself then the appropraite bin path] as location where the file would be created).
Additionally: You should put a try catch block around the whole thing as there can be unexpected phenomenons which result in an exception.

How to parse a file into a list of string to use for auto-complete in a windows forms application in C#?

I am building an application in C# that has a textbox field. In this field, a user will write text and the text will autocomplete from a file found on a remote repository. I am using a library called SharpSVN and I am trying to find a method where I can fetch that file from the repository based on a certain path I provide, then parse the content into strings that will be added to the list in the autocomplete of the textbox mentioned previously.

There are two ways:
Download the file text using the repository url. If you want the file at a specific revision, try entering in "?r=12345" as to get the file's appearance at a specific revision number:
string fileText = new WebClient().DownloadFile("https://myrepo.com/myfile.txt", localFilename);
Or, you could also use SharpSVN, removing the revision options if you want the latest version:
public string GetFileContentsAsString(long revisionNumber)
{
return new StreamReader(GetFileContents(revisionNumber)).ReadToEnd();
}
private MemoryStream GetFileContents(long revisionNumber)
{
SvnRevision rev = new SvnRevision(revisionNumber);
MemoryStream stream = new MemoryStream();
using (SvnClient client = GetClient())
{
client.FileVersions(SvnTarget.FromUri(RemotePath), new SvnFileVersionsArgs() { Start = rev, End = rev }, (s, e) =>
{
e.WriteTo(stream);
});
}
stream.Position = 0;
return stream;
}
When you have the file as a text string, you can use .NET's String.Split() method to split the text into a list of lines using the '\n' line-break character as the delimiter:
string[] fileAsLines = fileText.Split(new char[] {'\n'});

Universal Microsoft Word document parser without Word installation

In order to parse : Microsoft Word 97/2003 (.doc) & Microsoft Word 2007/2010 (.docx) using C# and WPF without Word installation, I need to know if someone can give me a serious library to use in order to achieve that.
Technically I iterate throught ZipEntry elements like that :
foreach (string file in _listPathFiles)
{
using (Ionic.Zip.ZipFile zip = ZipFile.Read(file))
{
try
{
zip.ToList().ForEach(entry =>
{
if (entry.FileName.EndsWith(".doc") ||
entry.FileName.EndsWith(".docx"))
{
// Extract file into disk
entry.FileName = System.IO.Path.GetFileName(entry.FileName);
entry.Extract(baseStoragePath);
// Get data from file with Parser
string filePath = baseStoragePath + entry.FileName;
// Remove extracted filess
if (File.Exists(filePath))
{
File.Delete(filePath);
Console.WriteLine("Delete : " + filePath);
}
}
});
}
catch (Exception e)
{
Console.WriteLine("Fail to unzip Exception : " + e.StackTrace);
}
}
}
I'm not sure that I can use ZipEntry directly to get the document, may be I will have to unzip it before parsing ?!
And my goal is to get data located after a "Heading 1" Microsoft Word style, so the library should be able to get this kind of properties.
Libraries ideas and code samples are welcome..

GroupDocs.Parser for .NET can be used in your case for extracting the text from Word documents without installing MS Word. The extraction can be performed line by line or at once.
// extracting all the text
WordsTextExtractor extractor = new WordsTextExtractor("sample.docx");
Console.Write(extractor.ExtractAll());
// OR
// Extract text line by line
string line = extractor.ExtractLine();
// If the line is null, then the end of the file is reached
while (line != null)
{
// Print a line to the console
Console.Write(line);
// Extract another line
line = extractor.ExtractLine();
}
Disclosure: I work as Developer Evangelist at GroupDocs.

Check out NPOI (A .NET Port of the Apache NOI API):
http://npoi.codeplex.com/
or
Download OpenXML SDK for reading Office Documents such as MS Word.

How to read a text file line by line Windows RT?

i just cant find a clean explanation on how to read a text file, line by line in Windows RT ( for Windows 8 Store).
Lets say i have a folder: MyFolder/Notes.txt
And i want to read the data from Notes.txt and add it to an array of strings.
How do i read/write from that file? I was using StreamReader before, but now it is very confusing. And the dev samples dont help that much.

Ive managed to find the answer myself. Thanks for the help.
// READ FILE
public async void ReadFile()
{
// settings
var path = #"MyFolder\MyFile.txt";
var folder = Windows.ApplicationModel.Package.Current.InstalledLocation;
// acquire file
var file = await folder.GetFileAsync(path);
var readFile = await Windows.Storage.FileIO.ReadLinesAsync(file);
foreach (var line in readFile)
{
Debug.WriteLine("" + line.Split(';')[0]);
}
}
MyFile.txt has:
Test1;Description1;
Test2;Description2;
//Output for ReadFile()
Test1
Test2

You don't have to use File.ReadLines. You try an implementation like this if you prefer:
using (StreamReader reader = new StreamReader("notes.txt"))
{
while (reader.Peek() >= 0)
{
Console.WriteLine(reader.ReadLine());
}
}

foreach (var line in File.ReadLines("MyFolder/Notes.txt"))
{
...
}
reads the file line by line. This is different from File.ReadAllLines which reads the entire file at once.
If you want to read everything at once, into an array, use the latter.

How do I create 7-Zip archives with .NET?

How can I create 7-Zip archives from my C# console application? I need to be able to extract the archives using the regular, widely available 7-Zip program.
Here are my results with the examples provided as answers to this question
"Shelling out" to 7z.exe - this is the simplest and most effective approach, and I can confirm that it works nicely. As workmad3 mentions, I just need to guarantee that 7z.exe is installed on all target machines, which is something I can guarantee.
7Zip in memory compression - this refers to compressing cookies "in-memory" before sending to the client; this method seems somewhat promising. The wrapper methods (wrapping the LZMA SDK) return type byte[]. When I write the byte[] array to a file, I can't extract it using 7-Zip (File.7z is not supported archive).
7zSharp Wrapper (found on CodePlex) - this wraps the 7z exe/LZMA SDK. I referenced the project from my app, and it successfully created some archive files, but I was unable to extract the files using the regular 7-Zip program (File.7z is not supported archive).
7Zip SDK aka LZMA SDK - I guess I'm not smart enough to figure out how to use this (which is why I posted here)... Any working code examples that demonstrate creating a 7zip archive that is able to be extracted by the regular 7zip program?
CodeProject C# (.NET) Interface for 7-Zip Archive DLLs - only supports extracting from 7zip archives... I need to create them!
SharpZipLib - According to their FAQ, SharpZipLib doesn't support 7zip.

EggCafe 7Zip cookie example This is an example (zipping cookie) with the DLL of 7Zip.
CodePlex Wrapper
This is an open source project that warp zipping function of 7z.
7Zip SDK The official SDK for 7zip (C, C++, C#, Java) <---My suggestion
.Net zip library by SharpDevelop.net
CodeProject example with 7zip
SharpZipLib Many zipping

If you can guarantee the 7-zip app will be installed (and in the path) on all target machines, you can offload by calling the command line app 7z. Not the most elegant solution but it is the least work.

SevenZipSharp is another solution. Creates 7-zip archives...

Here's a complete working example using the SevenZip SDK in C#.
It will write, and read, standard 7zip files as created by the Windows 7zip application.
PS. The previous example was never going to decompress because it never wrote the required property information to the start of the file.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using SevenZip.Compression.LZMA;
using System.IO;
using SevenZip;
namespace VHD_Director
{
class My7Zip
{
public static void CompressFileLZMA(string inFile, string outFile)
{
Int32 dictionary = 1 << 23;
Int32 posStateBits = 2;
Int32 litContextBits = 3; // for normal files
// UInt32 litContextBits = 0; // for 32-bit data
Int32 litPosBits = 0;
// UInt32 litPosBits = 2; // for 32-bit data
Int32 algorithm = 2;
Int32 numFastBytes = 128;
string mf = "bt4";
bool eos = true;
bool stdInMode = false;
CoderPropID[] propIDs = {
CoderPropID.DictionarySize,
CoderPropID.PosStateBits,
CoderPropID.LitContextBits,
CoderPropID.LitPosBits,
CoderPropID.Algorithm,
CoderPropID.NumFastBytes,
CoderPropID.MatchFinder,
CoderPropID.EndMarker
};
object[] properties = {
(Int32)(dictionary),
(Int32)(posStateBits),
(Int32)(litContextBits),
(Int32)(litPosBits),
(Int32)(algorithm),
(Int32)(numFastBytes),
mf,
eos
};
using (FileStream inStream = new FileStream(inFile, FileMode.Open))
{
using (FileStream outStream = new FileStream(outFile, FileMode.Create))
{
SevenZip.Compression.LZMA.Encoder encoder = new SevenZip.Compression.LZMA.Encoder();
encoder.SetCoderProperties(propIDs, properties);
encoder.WriteCoderProperties(outStream);
Int64 fileSize;
if (eos || stdInMode)
fileSize = -1;
else
fileSize = inStream.Length;
for (int i = 0; i < 8; i++)
outStream.WriteByte((Byte)(fileSize >> (8 * i)));
encoder.Code(inStream, outStream, -1, -1, null);
}
}
}
public static void DecompressFileLZMA(string inFile, string outFile)
{
using (FileStream input = new FileStream(inFile, FileMode.Open))
{
using (FileStream output = new FileStream(outFile, FileMode.Create))
{
SevenZip.Compression.LZMA.Decoder decoder = new SevenZip.Compression.LZMA.Decoder();
byte[] properties = new byte[5];
if (input.Read(properties, 0, 5) != 5)
throw (new Exception("input .lzma is too short"));
decoder.SetDecoderProperties(properties);
long outSize = 0;
for (int i = 0; i < 8; i++)
{
int v = input.ReadByte();
if (v < 0)
throw (new Exception("Can't Read 1"));
outSize |= ((long)(byte)v) << (8 * i);
}
long compressedSize = input.Length - input.Position;
decoder.Code(input, output, compressedSize, outSize, null);
}
}
}
public static void Test()
{
CompressFileLZMA("DiscUtils.pdb", "DiscUtils.pdb.7z");
DecompressFileLZMA("DiscUtils.pdb.7z", "DiscUtils.pdb2");
}
}
}

I used the sdk.
eg:
using SevenZip.Compression.LZMA;
private static void CompressFileLZMA(string inFile, string outFile)
{
SevenZip.Compression.LZMA.Encoder coder = new SevenZip.Compression.LZMA.Encoder();
using (FileStream input = new FileStream(inFile, FileMode.Open))
{
using (FileStream output = new FileStream(outFile, FileMode.Create))
{
coder.Code(input, output, -1, -1, null);
output.Flush();
}
}
}

string zipfile = #"E:\Folderx\NPPES.zip";
string folder = #"E:\TargetFolderx";
ExtractFile(zipfile,folder);
public void ExtractFile(string source, string destination)
{
// If the directory doesn't exist, create it.
if (!Directory.Exists(destination))
Directory.CreateDirectory(destination);
//string zPath = ConfigurationManager.AppSettings["FileExtactorEXE"];
// string zPath = Properties.Settings.Default.FileExtactorEXE; ;
string zPath=#"C:\Program Files\7-Zip\7zG.exe";
try
{
ProcessStartInfo pro = new ProcessStartInfo();
pro.WindowStyle = ProcessWindowStyle.Hidden;
pro.FileName = zPath;
pro.Arguments = "x \"" + source + "\" -o" + destination;
Process x = Process.Start(pro);
x.WaitForExit();
}
catch (System.Exception Ex) { }
}
Just Install 7 zip from source and pass the parameter to the method.
Thanks. Please like the answer.

I use this code
string PZipPath = #"C:\Program Files\7-Zip\7z.exe";
string sourceCompressDir = #"C:\Test";
string targetCompressName = #"C:\Test\abc.zip";
string CompressName = targetCompressName.Split('\\').Last();
string[] fileCompressList = Directory.GetFiles(sourceCompressDir, "*.*");
if (fileCompressList.Length == 0)
{
MessageBox.Show("No file in directory", "Important Message");
return;
}
string filetozip = null;
foreach (string filename in fileCompressList)
{
filetozip = filetozip + "\"" + filename + " ";
}
ProcessStartInfo pCompress = new ProcessStartInfo();
pCompress.FileName = PZipPath;
if (chkRequestPWD.Checked == true)
{
pCompress.Arguments = "a -tzip \"" + targetCompressName + "\" " + filetozip + " -mx=9" + " -p" + tbPassword.Text;
}
else
{
pCompress.Arguments = "a -tzip \"" + targetCompressName + "\" \"" + filetozip + "\" -mx=9";
}
pCompress.WindowStyle = ProcessWindowStyle.Hidden;
Process x = Process.Start(pCompress);
x.WaitForExit();

Some additional test-info on #Orwellophile code using a 17.9MB textfile.
Using the property values in the code-example "as is" will have a HUGE negative impact on performance, it takes 14.16 sec.
Setting the properties to the following do the same job at 3.91 sec (i.a. the archive will have the same container info which is: you can extract and test the archive with 7zip but there are no filename information)
Native 7zip 2 sec.
CoderPropID[] propIDs = {
//CoderPropID.DictionarySize,
//CoderPropID.PosStateBits,
//CoderPropID.LitContextBits,
//CoderPropID.LitPosBits,
//CoderPropID.Algorithm,
//CoderPropID.NumFastBytes,
//CoderPropID.MatchFinder,
CoderPropID.EndMarker
};
object[] properties = {
//(Int32)(dictionary),
//(Int32)(posStateBits),
//(Int32)(litContextBits),
//(Int32)(litPosBits),
//(Int32)(algorithm),
//(Int32)(numFastBytes),
//mf,
eos
};
I did another test using native 7zip and a 1,2GB SQL backup file (.bak)
7zip (maximum compression): 1 minute
LZMA SDK (#Orwellophile with above property-setting): 12:26 min :-(
Outputfile roughly same size.
So I guess I'll myself will use a solution based on the c/c++ engine, i.a. either call the 7zip executable from c# or use squid-box/SevenZipSharp, which is a wrapper around the 7zip c/c++ dll file, and seems to be the newest fork of SevenZipSharp.
Haven't tested the wrapper, but I hope is perform just as the native 7zip. But hopefully it will give the possibility to compress stream also which you obvious cannot if you call the exe directly. Otherwise I guess there isn't mush advantage over calling the exe. The wrapper have some additional dependencies so it will not make your published project "cleaner".
By the way it seems the .Net Core team consider implementing LZMA in the system.io class in .Core ver. 5, that would be great!
(I know this is kind of a comment and not an answer but to be able to provide the code snippet it couldn't be a comment)

SharpCompress is in my opinion one of the smartest compression libraries out there. It supports LZMA (7-zip), is easy to use and under active development.
As it has LZMA streaming support already, at the time of writing it unfortunately only supports 7-zip archive reading. BUT archive writing is on their todo list (see readme). For future readers: Check to get the current status here: https://github.com/adamhathcock/sharpcompress/blob/master/FORMATS.md

Install the NuGet package called SevenZipSharp.Interop
Then:
SevenZipBase.SetLibraryPath(#".\x86\7z.dll");
var compressor = new SevenZip.SevenZipCompressor();
var filesToCompress = Directory.GetFiles(#"D:\data\");
compressor.CompressFiles(#"C:\archive\abc.7z", filesToCompress);

These easiest way is to work with .zip files instead of .7z and use Dot Net Zip
When spinning off 7zip commands to shell there are other issues like user privileges, I had issue with SevenZipSharp.
Private Function CompressFile(filename As String) As Boolean
Using zip As New ZipFile()
zip.AddFile(filename & ".txt", "")
zip.Save(filename & ".zip")
End Using
Return File.Exists(filename & ".zip")
End Function

Here is code to create and extract 7zip (based on LZMA SDK - C#)
Note: 7z archives created with same code can be unarchived. As code uses managed LZMA using earlier version of LZMA SDK

Categories

Resources