Develop Reference

How to properly Clear a Queue containing structs?

I have declared a basic struct like this
private struct ValLine {
public string val;
public ulong linenum;
}
and declared a Queue like this
Queue<ValLine> check = new Queue<ValLine>();
Then in a using StreamReader setup where I'm reading through the lines of an input file using ReadLine in a while loop, among other things, I'm doing this to populate the Queue:
check.Enqueue(new ValLine { val = line, linenum = linenum });
("line" is a string containing the text of each line, "linenum" is just a counter that is initialized at 0 and is incremented each time through the loop.)
The purpose of the "check" Queue is that if a particular line meets some criteria, then I store that line in "check" along with the line number that it occurs on in the input file.
After I've finished reading through the input file, I use "check" for various things, but then when I'm finished using it I clear it out in the obvious manner:
check.Clear();
(Alternatively, in my final loop through "check" I could just use .Dequeue(), instead of foreach'ing it.)
But then I got to thinking - wait a minute, what about all those "new ValLine" I generated when populating the Queue in the first place??? Have I created a memory leak? I've pretty new to C#, so it's not coming clear to me how to deal with this - or even if it should be dealt with (perhaps .Clear() or .Dequeue() deals with the now obsoleted structs automatically?). I've spent over an hour with our dear friend Google, and just not finding any specific discussion of this kind of example in regard to the clearing of a collection of structs.
So... In C# do we need to deal with wiping out the individual structs before clearing the queue (or as we are dequeueing), or not? And if so, then what is the proper way to do this?
(Just in case it's relevant, I'm using .NET 4.5 in Visual Studio 2013.)
UPDATE: This is for future reference (you know, like if this page comes up in a Google search) in regard to proper coding. To make the struct immutable as per recommendation, this is what I've ended up with:
private struct ValLine {
private readonly string _val;
private readonly ulong _linenum;
public string val { get { return _val; } }
public ulong linenum { get { return _linenum; } }
public ValLine(string x, ulong n) { _val = x; _linenum = n; }
}
Corresponding to that change, the queue population line is now this:
check.Enqueue(new ValLine(line,linenum));
Also, though not strictly necessary, I did get rid of my foreach on the queue (and the check.Clear();, and changed it to this
while (check.Count > 0) {
ValLine ll = check.Dequeue();
writer.WriteLine("[{0}] {1}", ll.linenum, ll.val);
}
so that the queue is emptied out as the information is output.
UPDATE 2: Okay, yes, I'm still a C# newbie (less than a year). I learn a lot from the Internet, but of course, I'm often looking at examples from more than a year ago. I have changed my struct so now it looks like this:
private struct ValLine {
public string val { get; private set; }
public ulong linenum { get; private set; }
public ValLine(string x, ulong n): this()
{ this.val = x; this.linenum = n; }
}
Interestingly enough, I had actually tried exactly this off the top of my head before coming up with what's in the first update (above), but got a compile error (because I did not have the : this() with the constructor). Upon further suggestion, I checked further and found a recent example showing that : this() for making it work like I tried before, plugged that in, and - Wa La! - clean compile. I like the cleaner look of the code. What the private variables are called is irrelevant to me.

No, you won't have created a memory leak. Calling Clear or Dequeue will clear the memory appropriately - for example, if you had a List<T> then a clear operation might use:
for (int i = 0; i < capacity; i++)
{
array[i] = default(T);
}
I don't know offhand whether Queue<T> is implemented with a circular buffer built on an array, or a linked list - but either way, you'll be fine.
Having said that, I would strongly recommend against using mutable structs as you're doing here, along with mutable fields. While it's not causing the particular problem you're envisaging, they can behave in confusing ways.

How to access the reference values of a HashSet<TValue> without enumeration?

I have this scenario in which memory conservation is paramount. I am trying to read in > 1 GB of Peptide sequences into memory and group peptide instances together that share the same sequence. I am storing the Peptide objects in a Hash so I can quickly check for duplication, but found out that you cannot access the objects in the Set, even after knowing that the Set contains that object.
Memory is really important and I don't want to duplicate data if at all possible. (Otherwise I would of designed my data structure as: peptides = Dictionary<string, Peptide> but that would duplicate the string in both the dictionary and Peptide class). Below is the code to show you what I would like to accomplish:
public SomeClass {
// Main Storage of all the Peptide instances, class provided below
private HashSet<Peptide> peptides = new HashSet<Peptide>();
public void SomeMethod(IEnumerable<string> files) {
foreach(string file in files) {
using(PeptideReader reader = new PeptideReader(file)) {
foreach(DataLine line in reader.ReadNextLine()) {
Peptide testPep = new Peptide(line.Sequence);
if(peptides.Contains(testPep)) {
// ** Problem Is Here **
// I want to get the Peptide object that is in HashSet
// so I can add the DataLine to it, I don't want use the
// testPep object (even though they are considered "equal")
peptides[testPep].Add(line); // I know this doesn't work
testPep.Add(line) // THIS IS NO GOOD, since it won't be saved in the HashSet which i use in other methods.
} else {
// The HashSet doesn't contain this peptide, so we can just add it
testPep.Add(line);
peptides.Add(testPep);
}
}
}
}
}
}
public Peptide : IEquatable<Peptide> {
public string Sequence {get;private set;}
private int hCode = 0;
public PsmList PSMs {get;set;}
public Peptide(string sequence) {
Sequence = sequence.Replace('I', 'L');
hCode = Sequence.GetHashCode();
}
public void Add(DataLine data) {
if(PSMs == null) {
PSMs = new PsmList();
}
PSMs.Add(data);
}
public override int GethashCode() {
return hCode;
}
public bool Equals(Peptide other) {
return Sequence.Equals(other.Sequence);
}
}
public PSMlist : List<DataLine> { // and some other stuff that is not important }
Why does HashSet not let me get the object reference that is contained in the HashSet? I know people will try to say that if HashSet.Contains() returns true, your objects are equivalent. They may be equivalent in terms of values, but I need the references to be the same since I am storing additional information in the Peptide class.
The only solution I came up with is Dictionary<Peptide, Peptide> in which both the key and value point to the same reference. But this seems tacky. Is there another data structure to accomplish this?

Basically you could reimplement HashSet<T> yourself, but that's about the only solution I'm aware of. The Dictionary<Peptide, Peptide> or Dictionary<string, Peptide> solution is probably not that inefficient though - if you're only wasting a single reference per entry, I would imagine that would be relatively insignificant.
In fact, if you remove the hCode member from Peptide, that will safe you 4 bytes per object which is the same size as a reference in x86 anyway... there's no point in caching the hash as far as I can tell, as you'll only compute the hash of each object once, at least in the code you've shown.
If you're really desperate for memory, I suspect you could store the sequence considerably more efficiently than as a string. If you give us more information about what the sequence contains, we may be able to make some suggestions there.
I don't know that there's any particularly strong reason why HashSet doesn't permit this, other than that it's a relatively rare requirement - but it's something I've seen requested in Java as well...

Use a Dictionary<string, Peptide>.

c# Memory performance and speedup

i have a situation, i need to process an jagged array of 20k registers every time a user press a key. I have a grid and while the user is typing the system shows a filtered result in a grid. so. So i have a jagged array filled with all 20k registers. and the i have a list (global to the control) and it´s cleaned up every time the user press a key and filled up with just the filtered registers and then show then in the grid.
Here is the code
the model
public struct PlayerLookUpAdapter
{
[Browsable(false)]
public decimal Id { get; set; }
[DisplayName("Número")]
public String Number { get; set; }
[DisplayName("Nombre")]
public String Name { get; set; }
[DisplayName("Apellido")]
public String Surname { get; set; }
[DisplayName("DNI")]
public String Document { get; set; }
[DisplayName("Estado")]
public String Status { get; set; }
}
private PlayerLookUpAdapter[] _source; // here are the 20k registers
List<PlayerLookUpAdapter> filteredOut = new List<PlayerLookUpAdapter>(); // here the filtered ones
// this code is executed every time the user press a key
private void tb_nro_KeyUp(object sender, KeyEventArgs e)
{
if (!(e.KeyCode.Equals(Keys.Enter) || e.KeyCode.Equals(Keys.Down)) && _source!=null)
{
String text = tb_nro.Text.ToUpper();
if (String.IsNullOrEmpty(text))
{
fg.DataSource = _source;
fg.Refresh();
return;
}
fg.DataSource = null;
filteredOut.Clear();
int length = _source.Length;
for (int i = 0; i < length; i++)
{
PlayerLookUpAdapter cur = _source[i];
if (cur.Number.ToUpper().StartsWith(text) || cur.Surname.ToUpper().StartsWith(text) || cur.Name.ToUpper().StartsWith(text))
filteredOut.Add(cur);
}
fg.DataSource = filteredOut;
SetGridColumnsProperties();
fg.Refresh();
}
else
{
fg.Focus();
}
}
is it a good solution in terms of memory usage and performance? have you got any advice? How can i gain more speed. It works realy good, but what about if i got 100k registers instead of 20k?
Thanks in advance.

I think this should be a prime example for using a tree.
If you lay your Data down in a Tree (i actually don't know if C#/.Net supports a Tree Data-Structure, or you have get your own hands dirty).
The Speed you search in a Tree will increase in comparison for searching in an Array (because a Tree gots a search-speed of somehting like O(n)=n*log(n))
The Theory is easy: if a User Types in a Literal, the Tree goes to the Node starting with this Literal, on this nodes are all possible other nodes and so on. For example: The User types in an "t" you go to the "t" Node, then he types in an "e" you go to the subnode "te", there are some other subnodes like "test" and the system will propose the User these subnodes.

firts of all you could improve a bit your code: the StartWith method has an overload who takes the string comparison as well. you could set it as "OrdinalIgnoreCase" to avoid to upper all the strings but I don't think you will gain a lot.
The only way you have to speed up you search is go for a Search engine as Lucene.net.
http://www.codeproject.com/KB/library/IntroducingLucene.aspx

You want a prefix tree for this.
Here is one implementation:
A Reusable Prefix Tree using Generics in C# 2.0

You could probably use the StringComparison.OrdinalIgnoreCase option on your string comparisons and avoid having to call ToUpper on all your strings 20k times.
Ideally, first you need to decide how slow is too slow based on your best estimates for typical usage of your program. After all premature optimisation is the root of all evil.

Precalculate the ToUpper() call so you dont have to do it every time. You could maintain a second list where all the strings are stored uppercase.
Secondly you should search the filtered list (instead of the whole list) in case a key is added to the search string. The new (longer) string can never be outside of the filtered results.

C#, how to handle constant tables

It seems strange that the language apparently includes no suitable functionality.
I find myself with data that would best be expressed as a multi-dimensional array but it's utterly constant, there is no way anyone could want to change it without also changing the associated code. Faced with such stuff in Delphi the answer is obvious--a constant whose value is the table. However, C# doesn't seem to support anything like this.
Google shows many people griping about this, no good answers.
How do people handle this sort of situation?
(And don't say that constants don't belong in code--the last one I bumped into was all possible permutations of 4 items. Unless the very nature of spacetime changes this is set in stone.)
What happened?? There was an answer that came pretty close, I was asking about a detail and it vanished! Simply declaring an array sort of does the job--the only problem is that the array allocation is going to run every time. The one in front of me contains 96 values--how do I get it to initialize only once? Do I just have to accept scoping it far wider than it should be? (As it stands it's in one 3-line routine that's inside what amounts to an O(n^3) routine.)

ReadOnlyCollection

There's a page in in the C# FAQ about this specific thing.
They suggest using a static readonly array:
static readonly int[,] constIntArray = new int[,] { { 1, 2, 3 }, { 4, 5, 6 }};
However, be aware that this is only sort of constant - you can still reassign individual elements within the array. Also, this has to be specified on the class level since it's a static, but it will work fairly well.

You could use a readonly Hashtable. The only downside is that readonly does not prevent you from changing the value of a particular item in the Hashtable. So it is not truly const.
readonly Hashtable table = new Hashtable(){{1,"One"},{2,"Two"}};
Or an array
public readonly string[,] arry = new string[,]{{"1","2"},{"2","4"}};
Yes, you will need to declare the variable in the appropriate scope so it does not get initialized more than once.

Like they say, just add another layer of indirection. C# doesn't need to provide a specialized data structure as a language primitive, although one does, at times, wish there was a way to make any class immutable, but that's another discussion.
Now you didn't mention if you need to store different things in there. In fact you didn't mention anything other than multi-dimensional and no ability to change the values or the arrays. I don't even know if the access pattern (a single int,int,int indexer) is appropriate.
But in general, for a 3-dimensional jagged array, the following works (but it isn't pretty).
One caveat is the type you construct it with also needs to be immutable, but that's your problem. You can just create your own read-only wrapper.
public static readonly ReadOnlyThreeDimensions<int> MyGlobalThree
= new ReadOnlyThreeDimensions<int>(IntInitializer);
public class ReadOnlyThreeDimensions<T>
{
private T[][][] _arrayOfT;
public ReadOnlyThreeDimensions(Func<T[][][]> initializer)
{
_arrayOfT = initializer();
}
public ReadOnlyThreeDimensions(T[][][] arrayOfT)
{
_arrayOfT = arrayOfT;
}
public T this [int x, int y, int z]
{
get
{
return _arrayOfT[x][y][z];
}
}
}
And then you just need to provide some initializer method, or assign it in a static constructor.
public static int[][][] IntInitializer()
{
return xyz // something that constructs a [][][]
}

Enumerations, surely.

Well, I've taken the approach of the following, it's a little nasty to read but easy to edit.
public struct Something
{
public readonly int Number;
public readonly string Name;
public Something(int num, string name) { this.Number = num; this.Name = name; }
}
public readonly Something[] GlobalCollection = new Something[]
{
new Something(1, "One"),
new Something(2, "Two"),
};

Generating the next available unique name in C#

If you were to have a naming system in your app where the app contains say 100 actions, which creates new objects, like:
Blur
Sharpen
Contrast
Darken
Matte
...
and each time you use one of these, a new instance is created with a unique editable name, like Blur01, Blur02, Blur03, Sharpen01, Matte01, etc. How would you generate the next available unique name, so that it's an O(1) operation or near constant time. Bear in mind that the user can also change the name to custom names, like RemoveFaceDetails, etc.
It's acceptable to have some constraints, like restricting the number of characters to 100, using letters, numbers, underscores, etc...
EDIT: You can also suggest solutions without "filling the gaps" that is without reusing the already used, but deleted names, except the custom ones of course.

I refer you to Michael A. Jackson's Two Rules of Program Optimization:
Don't do it.
For experts only: Don't do it yet.
Simple, maintainable code is far more important than optimizing for a speed problem that you think you might have later.
I would start simple: build a candidate name (e.g. "Sharpen01"), then loop through the existing filters to see if that name exists. If it does, increment and try again. This is O(N2), but until you get thousands of filters, that will be good enough.
If, sometime later, the O(N2) does become a problem, then I'd start by building a HashSet of existing names. Then you can check each candidate name against the HashSet, rather than iterating. Rebuild the HashSet each time you need a unique name, then throw it away; you don't need the complexity of maintaining it in the face of changes. This would leave your code easy to maintain, while only being O(N).
O(N) will be good enough. You do not need O(1). The user is not going to click "Sharpen" enough times for there to be any difference.

I would create a static integer in action class that gets incremented and assigned as part of each new instance of the class. For instance:
class Blur
{
private static int count = 0;
private string _name;
public string Name
{
get { return _name; }
set { _name = value; }
}
public Blur()
{
_name = "Blur" + count++.ToString();
}
}
Since count is static, each time you create a new class, it will be incremented and appended to the default name. O(1) time.
EDIT
If you need to fill in the holes when you delete, I would suggest the following. It would automatically queue up numbers when items are renamed, but it would be more costly overall:
class Blur
{
private static int count = 0;
private static Queue<int> deletions = new Queue<int>();
private string _name;
public string Name
{
get { return _name; }
set
{
_name = value;
Delete();
}
}
private int assigned;
public Blur()
{
if (deletions.Count > 0)
{
assigned = deletions.Dequeue();
}
else
{
assigned = count++;
}
_name = "Blur" + assigned.ToString();
}
public void Delete()
{
if (assigned >= 0)
{
deletions.Enqueue(assigned);
assigned = -1;
}
}
}
Also, when you delete an object, you'll need to call .Delete() on the object.
CounterClass Dictionary version
class CounterClass
{
private int count;
private Queue<int> deletions;
public CounterClass()
{
count = 0;
deletions = new Queue<int>();
}
public string GetNumber()
{
if (deletions.Count > 0)
{
return deletions.Dequeue().ToString();
}
return count++.ToString();
}
public void Delete(int num)
{
deletions.Enqueue(num);
}
}
you can create a Dictionary to look up counters for each string. Just make sure you parse out the index and call .Delete(int) whenever you rename or delete a value.

You can easily do it in O(m) where m is the number of existing instances of the name (and not dependent on n, the number of items in the list.
Look up the string S in question. If S isn't in the list, you're done.
S exists, so construct S+"01" and check for that. Continue incrementing (e.g. next try S+"02" until it doesn't exist.
This gives you unique names but they're still "pretty" and human-readable.
Unless you expect a large number of duplicates, this should be "near-constant" time because m will be so small.
Caveat: What if the string naturally ends with e.g. "01"? In your case this sounds unlikely so perhaps you don't care. If you do care, consider adding more of a suffix, e.g. "_01" instead of just "01" so it's easier to tell them apart.

You could do something like this:
private Dictionary<string, int> instanceCounts = new Dictionary<string, int>();
private string GetNextName(string baseName)
{
int count = 1;
if (instanceCounts.TryGetValue(baseName, out count))
{
// the thing already exists, so add one to it
count++;
}
// update the dictionary with the new value
instanceCounts[baseName] = count;
// format the number as desired
return baseName + count.ToString("00");
}
You would then just use it by calling GetNextName(...) with the base name you wanted, such as
string myNextName = GetNextName("Blur");
Using this, you wouldn't have to pre-init the dictionary.
It would fill in as you used the various base words.
Also, this is O(1).

I would create a dictionary with a string key and a integer value, storing the next number to use for a given action. This will be almost O(1) in practice.
private IDictionary<String, Int32> NextFreeActionNumbers = null;
private void InitializeNextFreeActionNumbers()
{
this.NextFreeActionNumbers = new Dictionary<String, Int32>();
this.NextFreeActionNumbers.Add("Blur", 1);
this.NextFreeActionNumbers.Add("Sharpen", 1);
this.NextFreeActionNumbers.Add("Contrast", 1);
// ... and so on ...
}
private String GetNextActionName(String action)
{
Int32 number = this.NextFreeActionNumbers[action];
this.NextFreeActionNumbers[action] = number + 1;
return String.Format("{0} {1}", action, number);
}
And you will have to check against collisions with user edited values. Again a dictionary might be a smart choice. There is no way around that. What ever way you generate your names, the user can always change a existing name to the next one you generate unless you include all existing names into the generation schema. (Or use a special character that is not allowed in user edited names, but that would be not that nice.)
Because of the comments on reusing the holes I want to add it here, too. Don't resuse the holes generated be renaming or deletion. This will confuse the user because names he deleted or modified will suddenly reappear.

I would look for ways to simplify the problem.
Are there any constraints that can be applied? As an example, would it be good enough if each user can only have one (active) type of action? Then, the actions could be distinguished using the name (or ID) of the user.
Blur (Ben F)
Blur (Adrian H)
Focus (Ben F)
Perhaps this is not an option in this case, but maybe something else would be possible. I would go to great lengths in order to avoid the complexity in some of the proposed solutions!

If you want O(1) time then just track how many instances of each you have. Keep a hashtable with all of the possible objects, when you create an object, increment the value for that object and use the result in the name.

You're definitely not going to want to expose a GUID to the user interface.
Are you proposing an initial name like "Blur04", letting the user rename it, and then raising an error message if the user's custom name conflicts? Or silently renaming it to "CustomName01" or whatever?
You can use a Dictionary to check for duplicates in O(1) time. You can have incrementing counters for each effect type in the class that creates your new effect instances. Like Kevin mentioned, it gets more complex if you have to fill in gaps in the numbering when an effect is deleted.