Develop Reference

Fastest way to find which is the longest string from a set contained in an input string

I have a very large List<MyClass> (approximatively 600.000 records +) in which i need to extract the record where MyClass.Property1 is the exact match or the closest of my input string. However even if it seems it, this is not a fuzzy string matching problem, so i can't use the Levenshtein distance. To clear the things a bit i'll give you an example.
Suppose that the following is my data set (listing only MyClass.Property1):
242
2421
2422
24220
24221
24222
24223
24224
Now what i expect is, if i have in input 2422 i expect the third record to be given in output. If i get in input 24210 i expect in output the second record, which is the longest string contained in my output. To make the things faster, when i fill the List<MyClass>, i have saved in a Dictionary<int,int> the index at which the first number in the string change (example from 19999 to 20000) so i can reduce the size of the dataset i'm going to search for the match. What i wonder is: Which is the fastest way to reach my goal?
The only thing i can think is something like that:
Since i'm sure that the List<MyClass> is ordered by the MyClass.Property1 like in the example, and supposing that i have extracted a List<MyClass> called SubSet based on the dictionary i mentioned before, i would do
MyClass result = null;
foreach(MyCLass m in SubSet)
{
if (input.Contains(m.Property1))
{
// if the 2 strings are equal i've found the exact match
if(input == m.Property1)
return m.Property1;
else
result = m;
}
else
return result;
}
The most obvious problem i can see here is the fact that if the desidered result is at the end of the SubSet i need to loop over thousands of records Can you think any better way to reach my goal or a way to improve my current code?

Maybe,You can use Linq method in recursive function like
public string test(string input)
{
string result = Subset.FirstOrDefault(a => a == input);
if (result == null)
return test(input.Substring(0, input.Length - 2));
else
return result;
}

Getting the latest item in an observable sequence using RX in C#

take the following as an example:
var ob = Observable.Interval(TimeSpan.FromSeconds(1)).StartWith(500).Replay(1).RefCount();
What I'm trying to achieve here is to obtain the value of the latest item in the sequence at any given time "synchronously". Which means extensions like FirstAsync can't make it up for me.
The StartWith and Replay bit ensures that there will always be a value, and the RefCount bit is necessary in my actual code to detect when I can do some disposal actions.
So to simulate this "any given time" part, let's try getting the latest value after 5 seconds:
Observable.Timer(TimeSpan.FromSeconds(5)).Subscribe(x =>
{
// Try to get latest value from "ob" here.
});
So with a 5 second delay, I need to get the value 5 out of the sequence and these are what I have tried so far with no success:
ob.First() - returns 500
ob.Latest().Take(1) - same as above
ob.MostRecent(-1).First() - same as above
ob.MostRecent(-1) - gives me an IEnumerable<long> full of "500"
ob.Last() - never returns because it's waiting for the sequence to complete which it never will
ob.Latest().Last() - same as above
ob.ToTask().Result - same as above
ob.ToEnumerable() - same as above
ob.MostRecent().Last() same as above
It seems there's not much resources around that people can actually do this. The closest I can find is this: "Rx: operator for getting first and most recent value from an Observable stream", but it is not a synchronous call after all (still using a subscription) so it doesn't work for me.
Does any body know if this is actually doable?

To point out why your code probably isn't working as you expect it to
var ob = Observable.Interval(TimeSpan.FromSeconds(1)).StartWith(500).Replay(1).RefCount();
//Note at this point `ob` has never been subscribed to,
// so the Reference-count is 0 i.e. has not be connected.
Observable.Timer(TimeSpan.FromSeconds(5)).Subscribe(x =>
{
// Try to get latest value from "ob" here.
//Here we make our first subscription to the `ob` sequence.
// This will connect the sequence (invoke subscribe)
// which will
// 1) invoke StartWith
// 2) invoke onNext(500)
// 3) invoke First()
// 4) First() will then unsubscribe() as it has the single value it needs
// 5) The refCount will now return to 0
// 6) The sequence will be unsubscribed to.
ob.First().Dump();
//Any future calls like `ob.First()` will thus always get the value 500.
});
Potentially what you want is
var ob = Observable.Interval(TimeSpan.FromSeconds(1))
.Publish(500);
var connection = ob.Connect();
//Note at this point `ob` has never been subscribed to, so the ReferenceCount is 0 i.e. has not be connected.
var subscription = Observable.Timer(TimeSpan.FromSeconds(5)).Subscribe(x =>
{
// Try to get latest value from "ob" here.
ob.First().Dump();
});
//Sometime later
subscription.Dispose();
connection.Dispose()
HOWEVER, You really don't want to be mixing Synchronous calls with Rx. You also generally don't want to be subscribing within a subscription (as .First() is a subscription). What you probably mean to be doing is getting the latest value, and stashing it somewhere. Using .First() is just a slippery slope. You probably would be better writing something like
var subscription = Observable.Timer(TimeSpan.FromSeconds(5))
.SelectMany(_=>ob.Take(1))
.Subscribe(x =>
{
//Do something with X here.
x.Dump();
});

You need to do something like this:
var ob = Observable.Interval(TimeSpan.FromSeconds(1)).StartWith(500);
var latestAndThenTheRest =
Observable
.Create<long>(o =>
{
var bs = new BehaviorSubject<long>(1);
var s1 = ob.Subscribe(bs);
var s2 = bs.Subscribe(o);
return new CompositeDisposable(s1, s2);
});
The only thing that you need to consider here is that ob must be a hot observable for this to even make sense. If it were cold then every subscriber would get a brand new subscription to the start of the ob sequence.

Just to clarify this a bit, and thanks for #LeeCampbell's answer.
What was not working:
var ob = Observable.Interval(TimeSpan.FromSeconds(1)).StartWith(500).Replay(1).RefCount();
Observable.Timer(TimeSpan.FromSeconds(5)).Subscribe(x =>
{
ob.First().Dump();
// This gives you 500.
// Because this is the first time any one subscribes to the observable,
// so it starts right here and gives you the initial value.
});
What would actually work:
var ob = Observable.Interval(TimeSpan.FromSeconds(1)).StartWith(500).Replay(1).RefCount();
ob.Subscribe(); // Subscribe to start the above hot observable immediately.
Observable.Timer(TimeSpan.FromSeconds(5)).Subscribe(x =>
{
ob.First().Dump();
// This would give you either 3 or 4, depending on the speed and timing of your computer.
});

I'm not sure if this answer helps you, but have you looked into BehaviorSubject? It's an IObservable that remembers its latest value. It's a bit like a combination of a regular variable and an observable in one.
Otherwise, why don't you subscribe to 'ob' and store the latest value in a variable yourself?

Debugging LINQ on a per-element basis

I love LINQ statements for the expressive syntax and other convenient features. However, I find it very troublesome to debug them sometimes. Specifically, when I run a LINQ statement on a collection and one of the elements in the collection causes an exception, how can I figure out what the problem input was and where the problem came from?
Imagine I have a text file with 1000 real numbers:
0.46578
12.314213
1.444876
...
I am reading this as a List<string> and loading it into a more specific data structure:
var file_contents = File.ReadAllLines("myfile.txt");
var data = file_contents.Select(s => double.Parse(s));
Now, for this particular input, I didn't bother to look at it carefully and it turns out the 876th line contains (line numbers shown):
875 5.56786450
876 Error: Could not calculate value.
878 0.0316213
For whatever reason (perhaps the file was generated by a script that malfunctioned). My LINQ method chain will of course throw an exception. The problem is, how do I figure which element of the list caused the exception, and what its value was?
To clarify, if instead I used a for-loop:
var data = new List<double>();
foreach(var row in file_contents)
{
var d = double.Parse(row);
data.Add(d);
}
Then the exception would highlight the string which calls double.Parse, and I would be able to mouse over row to easily see what the problem input was.
I can, of course, use Resharper to convert my LINQ statements into for-loops, and then debug them, but is there a better way?

Put a conditional breakpoint on the lambda function, where the condition is s.StartsWith("5.56"). You just need to have your cursor on the lambda and press F9. Assuming you're using visual studio.

var data = file_contents.Select(s => {
try
{
return double.Parse(s);
}
catch
{
throw; //breakpoint?
}
});

Disclaimer: I work for OzCode
LINQ debugging is hard borderline impossible using Visual Studio. I suggest you try using OzCode.
This is what your code looks when debugging (the exception in on the 6th item).
You can tell which item caused the exception by investigating the items that where passed to the Select clause - and since the last one triggered the exception - it's easy to find the offending value.
If you're interested you can try OzCode's LINQ debugging - we've just started an EAP

I would just use a tryparse personally.
var data = new List<string>
{
"0.46578",
"12.314213",
"Error: Could not calculate value.",
"1.444876",
};
double d;
var good = data.Where(s => Double.TryParse(s, out d)).Select(Double.Parse);
var bad = data.Where(s => !Double.TryParse(s, out d)).Select(x => new
{
key = data.IndexOf(x),
value = x
}).ToDictionary(x => x.key, x => x.value);
textBox1.AppendTextAddNewLine("Good Data:");
WriteDataToTextBox(good);
textBox1.AppendTextAddNewLine(String.Format("{0}{0}Bad Data:", Environment.NewLine));
WriteDataToTextBox(bad);
The AppendTextAddNewLine is simply an extension method I wrote for my little proof of concept test program
public static void AppendTextAddNewLine(this TextBox textBox, string textToAppend)
{
textBox.AppendText(textToAppend + Environment.NewLine);
}
Edit
The WriteDataToTextbox is a generic method that writes an IEnumerble<T> out to the text box.
void WriteDataToTextBox<T>(IEnumerable<T> data )
{
foreach (var row in data)
{
textBox1.AppendTextAddNewLine(row.ToString());
}
}
Forgot to put the output here so I figure I should do that. It shows the index of the bad data and the data itself that caused the problem.
Good Data:
0.46578
12.314213
1.444876
Bad Data:
[2, Error: Could not calculate value.]

I'm not sure why you don't like foreach loop here. LINQ uses it internally anyway, and as you've already realized there are some pros and cons of using LINQ and debugging is one of cons.
I would probably mix LINQ with foreach and end up with following:
// read all lines from file //
var file_contents = File.ReadAllLines("myfile.txt");
// set initial data list length to number of lines for better performance
var data = new List<double>(file_contents.Length);
// list for incorrect line numbers
var incorrectRows = new List<int>();
foreach (var x in file_contents.Select((s, i) => new {s, i}))
{
// x.s - line string
// x.i - line number
double value;
if (double.TryParse(x.s, out value))
data.Add(value); // add value, which was OK
else
incorrectRows.Add(x.i); // add index of incorrect value
}
That will prevent an exception at all and will give you line numbers for all incorrect values. It also iterate over file_contents just once and every value is being parsed only once.

Rhino Mocks : How to match array arguments in an expectation?

Again at the Rhino Mocks Noob Wall
mockUI.Expect( x => x.Update( new Frame[] {Frame.MakeIncompleteFrame(1, 5)} ) );
This is the exact argument that I need to match. Via trace statements, I have verified that is the actual output as well i.e. the code behaves as intended but the test disagrees. RhinoMocks responds with
TestBowlingScorer.TestGamePresenter.TestStart:
Rhino.Mocks.Exceptions.ExpectationViolationException : IScoreObserver.Update([Frame# 1, Score = 0 Rolls [ 5, PENDING, ]]); Expected #1, Actual #0.
A Frame object contains few properties but doesn't override Equals() yet (overridden ToString() seen above). Update receives an array of Frames;
How do I setup this expectation? I see an Is.Matching constraint.. not sure how to use it or rather am concerned with the verbose nature of it.
I have a helper NUnit style custom Assert
public static void AssertFramesAreEqual(Frame[] expectedFrames, Frame[] actualFrames)
{
// loop over both collections
// compare attributes
}

#Gishu,
Yeah, that's it. I just also learned about the Arg<> static class which should allow you to do something like this:
mockUI.Expect( x => x.Update(Arg<Frame[]>
.Matches(fs=>HelperPredicates.CheckFrames ( expected, fs)) ));
There is also the Arg<>.List configuration starting point which I have not explored yet but might be even better for what you want

Verified works.. don't know if this is THE RhinoMocks way
var expectedFrames = new Frame[] { Frame.MakeIncompleteFrame(1, 5) };
mockUI.Expect( x => x.Update(null) )
.IgnoreArguments()
.Constraints( Is.Matching<Frame[]>( frames => HelperPredicates.CheckFramesMatch(expectedFrames, frames) ) );
The helper predicate is just a function that returns a boolean value - True on an exact match else false.
public static bool CheckFramesMatch(Frame[] expectedFrames, Frame[] actualFrames)
{
// return false if array lengths differ
// loop over corresponding elements
// return false if any attribute differs
// return true
}

Recursive woes - reducing an input string

I'm working on a portion of code that is essentially trying to reduce a list of strings down to a single string recursively.
I have an internal database built up of matching string arrays of varying length (say array lengths of 2-4).
An example input string array would be:
{"The", "dog", "ran", "away"}
And for further example, my database could be made up of string arrays in this manner:
(length 2) {{"The", "dog"},{"dog", "ran"}, {"ran", "away"}}
(length 3) {{"The", "dog", "ran"}.... and so on
So, what I am attempting to do is recursively reduce my input string array down to a single token. So ideally it would parse something like this:
1) {"The", "dog", "ran", "away"}
Say that (seq1) = {"The", "dog"} and (seq2) = {"ran", "away"}
2) { (seq1), "ran", "away"}
3) { (seq1), (seq2)}
In my sequence database I know that, for instance, seq3 = {(seq1), (seq2)}
4) { (seq3) }
So, when it is down to a single token, I'm happy and the function would end.
Here is an outline of my current program logic:
public void Tokenize(Arraylist<T> string_array, int current_size)
{
// retrieve all known sequences of length [current_size] (from global list array)
loc_sequences_by_length = sequences_by_length[current_size-min_size]; // sequences of length 2 are stored in position 0 and so on
// escape cases
if (string_array.Count == 1)
{
// finished successfully
return;
}
else if (string_array.Count < current_size)
{
// checking sequences of greater length than input string, bail
return;
}
else
{
// split input string into chunks of size [current_size] and compare to local database
// of known sequences
// (splitting code works fine)
foreach (comparison)
{
if (match_found)
{
// update input string and recall function to find other matches
string_array[found_array_position] = new_sequence;
string_array.Removerange[found_array_position+1, new_sequence.Length-1];
Tokenize(string_array, current_size)
}
}
}
// ran through unsuccessfully, increment length and try again for new sequence group
current_size++;
if (current_size > MAX_SIZE)
return;
else
Tokenize(string_array, current_size);
}
I thought it was straightforward enough, but have been getting some strange results.
Generally it appears to work, but upon further review of my output data I'm seeing some issues. Mainly, it appears to work up to a certain point...and at that point my 'curr_size' counter resets to the minimum value.
So it is called with a size of 2, then 3, then 4, then resets to 2.
My assumption was that it would run up to my predetermined max size, and then bail completely.
I tried to simplify my code as much as possible, so there are probably some simple syntax errors in transcribing. If there is any other detail that may help an eagle-eyed SO user, please let me know and I'll edit.
Thanks in advance

One bug is:
string_array[found_array_position] = new_sequence;
I don't know where this is defined, and as far as I can tell if it was defined, it is never changed.
In your if statement, when if match_found ever set to true?
Also, it appears you have an extra close brace here, but you may want the last block of code to be outside of the function:
}
}
}
It would help if you cleaned up the code, to make it easier to read. Once we get past the syntactic errors it will be easier to see what is going on, I think.

Not sure what all the issues are, but the first thing I'd do is have your "catch-all" exit block right at the beginning of your method.
public void Tokenize(Arraylist<T> string_array, int current_size)
{
if (current_size > MAX_SIZE)
return;
// Guts go here
Tokenize(string_array, ++current_size);
}
A couple things:
Your tokens are not clearly separated from your input string values. This makes it more difficult to handle, and to see what's going on.
It looks like you're writing pseudo-code:
loc_sequences_by_length is not used
found_array_position is not defined
Arraylist should be ArrayList.
etc.
Overall I agree with James' statement:
It would help if you cleaned up the
code, to make it easier to read.
-Doug