Develop Reference

C#/.NET console applications that behave like normal Unix tools

This question is regarding designing console applications in .NET that behave like normal Unix tools. The basic fundamental of Unix tools is that they can be chained together with 1 tool being able to take input from any compatible stream and giving output to any compatible stream.
If I'm writing console applications in .NET for Windows, what are the general things I need to follow for my tools to be of the same type?

"Do only one thing" is definitely one, but there are more:
Do only one thing and do it well
Output nothing on success (other than the result, of course)
Use stdin for the input, stdout for the output, and stderr for errors
Use non-zero exit codes to communicate failure
With this in mind, here's what is, in my opinion, a more "unixy" "to-uppercase" program in C#:
using System;
class Program
{
static int Main(string[] args)
{
try
{
var buf = new char[4096];
while (true)
{
int read = Console.In.Read(buf, 0, buf.Length);
if (read == 0)
break;
for (int i = 0; i < read; i++)
buf[i] = char.ToUpper(buf[i]);
Console.Out.Write(buf, 0, read);
}
return 0;
}
catch (Exception e)
{
Console.Error.WriteLine("ERROR: " + e.Message);
return 1;
}
}
}
Like a typical unixy program, you can run it without arguments and then it will be interactive, allowing you to type input manually on the console, terminated with Ctrl+Z, printing output whenever it receives a chunk of your input. Or you could pass it a file to process: uppercase.exe <input.txt and it will print the output to the console. Or you could redirect the output to a file too. Or you could pipe the input into it. Etc.

The main principle behind *nix tools is do one thing, and do it well.
Let's say I set out to create a *nix style tool that converted the input to uppercase. It's a trivial example, but that allows me to post the whole program here.
Here's the source code:
using System;
using System.Diagnostics.Contracts;
namespace Upperc {
class Program {
static void Main(string[] args) {
var input = Console.ReadLine();
Contract.Assert(input != null);
Console.WriteLine(input.ToUpperInvariant());
}
}
}
I took advantage of the fact that Console methods handle input and output and the standard streams. Example usage is:
> type example.txt | Upperc.exe > uppercased.txt
The input file is a plain text file:
example text file before processing
and the output file:
EXAMPLE TEXT FILE BEFORE PROCESSING

Why does the console close after my last input?

My program allows the user to put in 20 prices and to display the average of those values. Why does the console close after I enter my last input? Below is the code I'm running:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace machineproblem4
{
class Program
{
static void Main(string[] args)
{
int sum = 0;
double average = 0;
Console.WriteLine("\t\t\t INPUT PRICES \n");
int[] price = new int[20];
Console.WriteLine("\t\t\t Please enter 20 prices \n");
for (int ctr = 0; ctr < 20; ctr++)
{
Console.Write("Enter price {0} : ", ctr + 1);
price[ctr] = Convert.ToInt32(Console.ReadLine());
}
// [...calculate sum...]
//average
Console.WriteLine("\n----------------------------");
Console.WriteLine("||average of the prices||");
average = sum / 20;
Console.WriteLine("average of the prices: {0}", average);
//more code that outputs statistics about the inputs
//exit
//Edit: This is what fixed my problem
Console.WriteLine("press any key to exit ..");
Console.ReadKey();
}
}
}

use Console.Readline();
Read(), ReadLine() and ReadKey() are basically static methods, and they comes under the Console class. That's why we use these methods like:
Console.Read():-- method accept the String and return the integer.
Console.ReadLine():--method accept the String and return string .
Console.ReadKey():--method accept the Character and also return Character.
That's why we mostly use the Console.ReadKey() method, for come back to source code from output window .
Because when we only press the character we directly come on source code. If you will use the Console.Read() and Console.ReadLine method then
you need to press Enter, come back to the source code rather then any character.

You can place a Console.Read() at the last statement. You can also place a breakpoint at your last statement

Generally, it is not a good idea to wait for user input from a console application. This is okay for debugging, but not definitely for release.
So, first find out if your application is in debug or release config using
private static bool IsDebug()
{
object[] customAttributes = Assembly.GetExecutingAssembly().GetCustomAttributes(typeof(DebuggableAttribute), false);
if ((customAttributes != null) && (customAttributes.Length == 1))
{
DebuggableAttribute attribute = customAttributes[0] as DebuggableAttribute;
return (attribute.IsJITOptimizerDisabled && attribute.IsJITTrackingEnabled);
}
return false;
}
Then use,
if (IsDebug())
Console.Readline();
This eliminates the need to edit the code for different build configurations. Alternative is to put a breakpoint and debug the console app, as suggested by #Erwin

Put:
Console.Readline();
at the end of your main function, so it waits until you press enter before it closes.

None of the previous answers actually directly answer the question of why this is happening. The reason why the console is closing after your last input is that the rest of the code runs very quickly and when it reaches the end of your program, the console closes. This is correct behavior and should be expected when running a console application. As the other answers have stated, you can work around this by requiring a final input before closing the console, but that is all it is, a work around.
If you were to output to a text file rather than just the console, you would see that all of the output is generated as you would expect. The console output and close is just too fast for you to see it without some sort of pause in the code.
Additionally, a solution that has not been mentioned yet is to run the project from Visual Studio without debugging, which will automatically output "Press any key to continue..." when it finishes processing before closing the console. That way you can see what it is outputting without extraneous code that you wouldn't want to have in production code.

Checking the values of a long string from a specific point

This is a follow on from my last question.
The string I am trying to compare comes through as follows:
Discovered Peripherial: <CBPeripheral: 0x39b8b50 identifier = DEEE65FB-FF1F-A6A9-4C3C-5784F41B0D39, Name = "rawr", state = connecting>
What I'm trying to do is check the identifier number to that which I'm storing in my program. To do that, I have done the following:
private void AppendString(string message)
{
message.Substring(message.Length-77, message.Length);
outputContent.text += "\n" + message;
}
The \n is in there because I'm reading in 6 different devices and they all generate the above line. So I needed something to make it easier to read.
Then in my update function I am checking to see if they are similar like so:
if(String.Equals(myValues["UUID"], outputContent.text, StringComparison.OrdinalIgnoreCase))
{
Instantiate(model1, new Vector3(-2.5f, 3.0f,0), Quaternion.identity);
}
However when I run this on my iPad, xCode generates the following message:
ArgumentOutOfRangeException: startIndex + length > this.length
Parameter name: length
Which I'm guessing means I have miscounted the amount of characters I need to count back from in the substring.
My question is this:
Is there a better way in which I can compare my stored value with that of a specific part of a really long string or have I done something silly in my code which is generating that error?
This is a Unity project which I'm building onto an iPad and makes use of some functionality I can't replicate on a mac.

in your code i observed that you are calling Substring() method but not getting/saving its result into some variable .
message.Substring(message.Length-77, message.Length);
as string is an immutable which can not be modified.
hence when you call the Substring() method it cuts the required text and returns, which you have to save it on some variable to proceed further.
as below:
replace your function AppendString() as below :
private void AppendString(string message)
{
message= message.Substring(message.Length-77, message.Length);
outputContent.text += "\n" + message;
}

Search String takes a long time the first time only?

No shortage of search for string performance questions out there yet I still can not make heads or tails out of what the best approach is.
Long story short, I have committed to moving from 4NT to PowerShell. In leaving the 4NT I am going to miss the console super quick string searching utility that came with it called FFIND. I have decided to use my rudimentary C# programming skills to try an create my own utility to use in PowerShell that is just as quick.
So far search results on a string search in 100's of directories across a few 1000 files, some of which are quite large, are FFIND 2.4 seconds and my utility 4.4 seconds..... after I have ran mine at least once????
The first time I run them FFIND does it near the same time but mine takes over a minute? What is this? Loading of libraries? File indexing? Am I doing something wrong in my code? I do not mind waiting a little longer but the difference is extreme enough that if there is a better language or approach I would rather start down that path now before I get too invested.
Do I need to pick another language to write a string search that will be lighting fast
I have the need to use this utility to search through 1000 of files for strings in web code, C# code, and another propitiatory language that uses text files. I also need to be able to use this utility to find strings in very large log files, MB size.
class Program
{
public static int linecounter;
public static int filecounter;
static void Main(string[] args)
{
//
//INIT
//
filecounter = 0;
linecounter = 0;
string word;
// Read properties from application settings.
string filelocation = Properties.Settings.Default.FavOne;
// Set Args from console.
word = args[0];
//
//Recursive search for sub folders and files
//
string startDIR;
string filename;
startDIR = Environment.CurrentDirectory;
//startDIR = "c:\\SearchStringTestDIR\\";
filename = args[1];
DirSearch(startDIR, word, filename);
Console.WriteLine(filecounter + " " + "Files found");
Console.WriteLine(linecounter + " " + "Lines found");
Console.ReadKey();
}
static void DirSearch(string dir, string word, string filename)
{
string fileline;
string ColorOne = Properties.Settings.Default.ColorOne;
string ColorTwo = Properties.Settings.Default.ColorTwo;
ConsoleColor valuecolorone = (ConsoleColor)Enum.Parse(typeof(ConsoleColor), ColorOne);
ConsoleColor valuecolortwo = (ConsoleColor)Enum.Parse(typeof(ConsoleColor), ColorTwo);
try
{
foreach (string f in Directory.GetFiles(dir, filename))
{
StreamReader file = new StreamReader(f);
bool t = true;
int counter = 1;
while ((fileline = file.ReadLine()) != null)
{
if (fileline.Contains(word))
{
if (t)
{
t = false;
filecounter++;
Console.ForegroundColor = valuecolorone;
Console.WriteLine(" ");
Console.WriteLine(f);
Console.ForegroundColor = valuecolortwo;
}
linecounter++;
Console.WriteLine(counter.ToString() + ". " + fileline);
}
counter++;
}
file.Close();
file = null;
}
foreach (string d in Directory.GetDirectories(dir))
{
//Console.WriteLine(d);
DirSearch(d,word,filename);
}
}
catch (System.Exception ex)
{
Console.WriteLine(ex.Message);
}
}
}
}

If you want to speed up your code run a performance analysis and see what is taking the most time. I can almost guaruntee the longest step here will be
fileline.Contains(word)
This function is called on every line of the file, on every file. Naively searching for a word in a string can taken len(string) * len(word) comparisons.
You could code your own Contains method, that uses a faster string comparison algorithm. Google for "fast string exact matching". You could try using a regex and seeing if that gives you a performance enhancement. But I think the simplest optimization you can try is :
Don't read every line. Make a large string of all the content of the file.
StreamReader streamReader = new StreamReader(filePath, Encoding.UTF8);
string text = streamReader.ReadToEnd();
Run contains on this.
If you need all the matches in a file, then you need to use something like Regex.Matches(string,string).
After you have used regex to get all the matches for a single file, you can iterate over this match collection (if there are any matches). For each match, you can recover the line of the original file by writing a function that reads forward and backward from the match object index attribute, to where you find the '\n' character. Then output that string between those two newlines, to get your line.
This will be much faster, I guarantee it.
If you want to go even further, some things I've noticed are :
Remove the try catch statement from outside the loop. Only use it exactly where you need it. I would not use it at all.
Also make sure your system is running, ngen. Most setups usually have this, but sometimes ngen is not running. You can see the process in process explorer. Ngen generates a native image of the C# managed bytecode so the code does not have to be interpreted each time, but can be run natively. This speeds up C# a lot.
EDIT
Other points:
Why is there a difference between first and subsequent run times? Seems like caching. The OS could have cached the requests for the directories, for the files, for running and loading programs. Usually one sees speedups after a first run. Ngen could also be playing a part here, too, in generating the native image after compilation on the first run, then storing that in the native image cache.
In general, I find C# performance too variable for my liking. If the optimizations suggested are not satisfactory and you want more consistent performance results, try another language -- one that is not 'managed'. C is probably the best for your needs.

Do __LINE__ __FILE__ equivalents exist in C#?

For logging purposes
__LINE__
__FILE__
were my friends in C/C++. In Java to get that information I had to throw an exception and catch it. Why are these old standbys so neglected in the modern programming languages? There is something magical about their simplicity.

Caller Information has been added to .NET 4.5. This will be compiled, a big improvement over having to examine the stack trace manually.
public void Log(string message,
[CallerFilePath] string filePath = "",
[CallerLineNumber] int lineNumber = 0)
{
// Do logging
}
Simply call it in this manner. The compiler will fill in the file name and line number for you:
logger.Log("Hello!");

It is uglier, but you can do something like this in C# using the StackTrace and StackFrame classes:
StackTrace st = new StackTrace(new StackFrame(true));
Console.WriteLine(" Stack trace for current level: {0}", st.ToString());
StackFrame sf = st.GetFrame(0);
Console.WriteLine(" File: {0}", sf.GetFileName());
Console.WriteLine(" Method: {0}", sf.GetMethod().Name);
Console.WriteLine(" Line Number: {0}", sf.GetFileLineNumber());
Console.WriteLine(" Column Number: {0}", sf.GetFileColumnNumber());
Of course, this comes with some overhead.

With Caller Information (introduced in .NET 4.5) you can create the equivalent of __LINE__ and __FILE__ in C#:
static int __LINE__([System.Runtime.CompilerServices.CallerLineNumber] int lineNumber = 0)
{
return lineNumber;
}
static string __FILE__([System.Runtime.CompilerServices.CallerFilePath] string fileName = "")
{
return fileName;
}
The only thing to remember is that these are functions and not compiler directives.
So for example:
MessageBox.Show("Line " + __LINE__() + " in " + __FILE__());
If you were to use this in practise then I'd suggest different names. I've used the C/C++ names just to make it clearer what they are returning, and something like CurrentLineNumber() and CurrentFileName() might be better names.
The advantage of using Caller Information over any solution that uses the StackTrace is that the line and file information is available for both debug and release.

The closest thing to those is the fact that you can create a StackTrace object and find out the name of the method at the top of the stack, so you can get close to the functionality of the __FUNCTION__ macro.
StackTrace stackTrace = new StackTrace();
StackFrame[] stackFrames = stackTrace.GetFrames();
foreach (StackFrame stackFrame in stackFrames)
Console.WriteLine(stackFrame.GetMethod().Name);
To reduce the cost of typing this out by hand, and also the runtime code, you can write a helper method:
[Conditional("Debug")]
public void LogMethodName()
{
Trace.WriteLine("Entering:" + new StackTrace().GetFrame(1).GetMethod().Name);
}
Note how we get frame 1, as frame 0 would be LogMethodName itself. By marking it as Conditional("Debug") we ensure that the code is removed from release builds, which is one way to avoid the runtime cost where it may not be needed.

Because the stack trace contains most of what you need. It will not give you the name of the file but it will give you the class/method name. It also contains the line number. It is not neglected it is automatic. You just need to throw an exception like you do it in Java

Here's a way to get the line number: http://askville.amazon.com/SimilarQuestions.do?req=line-numbers-stored-stack-trace-C%2523-application-throws-exception
If you use log4net, you can get the line number and file name in your logs, but:
it can decrease your app. performance
you have to have .PDB files together with your assemblies.

There are already some suggestions to achieve what you want. Either use the StackTrace object or better log4net.
In Java to get that information I had to throw an exception and catch it.
That's not quite true. You can have it without throwing exceptions, too. Have a look to log4j. It even logs your method and class name, without polluting your code with hard coded strings containing the current method name (at least I have seen this in some occasions).
Why are these old standbys so neglected in the modern programming languages?
Java and C# don't make use (in the latter: excessive use) of preprocessors. And I think it's good. Abusing preprocessors to make unreadable code is very easy. And if programmers can abuse some technique ... they will abuse it.
Just a note about performance, which is very likely to be the next thing, which pops up in your mind:
If you use StackTrace or log4net you will always will read or hear that it is slow, because it uses Reflection. I am using log4net and I never encountered logging as a performance bottle neck. If it would be, I can declaratively deactivate (parts of) logging -- without changing the source code. That's pure beauty compared to delete all the logging lines in C/C++ code! (Besides: If performance is a primary goal, I would use C/C++ ... it will never die despite of Java and C#.)

Having used the FILE and LINE macros for many years for logging in C/C++ I really wanted a similar solution in C#. This is my solution. I prefer it to #fostandy suggestion of creating many overloads with varying number of parameters. This seems the less intrusive and does not limit the number of formatted parameters. You just have to be willing to accept the addition of the F.L() parameter at start of every Log.Msg() call.
using System;
using System.Runtime.CompilerServices;
namespace LogFileLine
{
public static class F
{
// This method returns the callers filename and line number
public static string L([CallerFilePath] string file = "", [CallerLineNumber] int line = 0)
{
// Remove path leaving only filename
while (file.IndexOf("\\") >= 0)
file = file.Substring(file.IndexOf("\\")+1);
return String.Format("{0} {1}:", file, line);
}
}
public static class Log
{
// Log a formatted message. Filename and line number of location of call
// to Msg method is automatically appended to start of formatted message.
// Must be called with this syntax:
// Log.Msg(F.L(), "Format using {0} {1} etc", ...);
public static void Msg(string fileLine, string format, params object[] parms)
{
string message = String.Format(format, parms);
Console.WriteLine("{0} {1}", fileLine, message);
}
}
class Program
{
static void Main(string[] args)
{
int six = 6;
string nine = "nine";
string dog = "dog";
string cat = "cats";
Log.Msg(F.L(), "The {0} chased the {1} {2}", dog, 5, cat);
Log.Msg(F.L(), "Message with no parameters");
Log.Msg(F.L(), "Message with 8 parameters {0} {1} {2} {3} {4} {5} {6} {7}",
1, 2, 3, "four", 5, 6, 7, 8.0);
Log.Msg(F.L(), "This is a message with params {0} and {1}", six, nine);
}
}
}
Here's the output from this code above
Program.cs 41: The dog chased the 5 cats
Program.cs 43: Message with no parameters
Program.cs 45: Message with 8 parameters 1 2 3 four 5 6 7 8
Program.cs 48: This is a message with params 6 and nine