The most common word in spaceless string

The most common word in spaceless string - c#

I have a very long string of text that is many words separated by camelCase like so:
AedeagalAedilityAedoeagiAefaldnessAegeriidaeAeginaAeipathyAeneolithicAeolididaeAeonialAerialityAerinessAerobia
I need to find the most common word and the number of times it has been used, I am unaware how to do this due to the lack of spaces and being new to C#.
I have tried many methods but none seem to work, any advice you have I'd be very grateful.
I have a github repo with the file being downloaded and a few tests already done here: https://github.com/Imstupidpleasehelp/C-code-test
Thank you.

You can try querying the string with a help of regular expressions and Linq:
string source = ...
var result = Regex
.Matches(source, "[A-Z][a-z]*")
.Cast<Match>()
.Select(match => match.Value)
.GroupBy(word => word)
.Select(group => (word : group.Key, count : group.Count()))
.OrderByDescending(pair => pair.count)
.First();
Console.Write($"{result.word} appears {result.count} time");

string[] split = Regex.Split(exampleString, "(?<=[A-Za-z])(?=[A-Z][a-z])");
var result = split.GroupBy(s => s)
.Where(g=> g.Count()>=1 )
.OrderByDescending(g => g.Count())
.Select(g => new{ Word = g.Key, Occurrences = g.Count()});
var result will contain pairs of (Word, Occurrences) for all words.
If you want just the first one (the one with the most occurrences) use
var result = split.GroupBy(s => s)
.Where(g=> g.Count()>=1 )
.OrderByDescending(g => g.Count())
.Select(g => new{ Word = g.Key, Occurrences = g.Count()}).First();
Have in mind that it can happen that you have 2 or more words with the same number of occurrences, so using First() would only give you one of those.

A non-linq approach using for loop and IsUpper to separate the words.
string data = "AedeagalAedilityAedoeagiAefaldness";
var words = new List<string>();
var temp = new StringBuilder();
for(int i = 0;i < data.Length;i++)
{
temp.Append(data[i]);
if (i == data.Length-1 || char.IsUpper(data[i+1]))
{
words.Add(temp.ToString());
temp.Clear();
}
}

Related

Extract values from a string into arrays

I have a string like this:
john "is my best buddy" and he loves "strawberry juice"
I want to-
Extract texts within double-quotes into a string array array1
Split texts outside of double-quotes by spaces and then insert them into another string array (array2).
Output:
array1[0]: is my best buddy
array1[1]: strawberry juice
array2[0]: john
array2[1]: and
array2[2]: he
array2[3]: loves
Any help is appreciated.

Clearly, this is a call for Regular Expressions:
var str = #"john ""is my best buddy"" and he loves ""strawberry juice""";
var regex = new Regex("(\"(?'quoted'[^\"]+)\")|(?'word'\\w+)",
RegexOptions.Singleline|RegexOptions.Compiled);
var matches = regex.Matches(str);
var quotes = matches.Cast<Match>()
.SelectMany(m => m.Groups.Cast<Group>())
.Where(g => g.Name == "quoted" && g.Success)
.Select(g => g.Value)
.ToArray();
var words = matches.Cast<Match>()
.SelectMany(m => m.Groups.Cast<Group>())
.Where(g => g.Name == "word" && g.Success)
.Select(g => g.Value)
.ToArray();

Check if String Contains Match in Enumerable.Range Filter List

I want to check if a string contains a word or number from a list and remove it from the string.
I want to use Enumerable.Range() to create the filter list and use it to filter many different strings.
I'm trying to combine two previous answers:
https://stackoverflow.com/a/49733139/6806643
https://stackoverflow.com/a/49740832/6806643
The sentence I want to filter:
This is a A05B09 hello 02 100 test
Filter
A00B00-A100B100, 01-100, 000-100, hello
Should read:
This is a test
Old Way
For Loop - Works
http://rextester.com/BJL70824
New Way
Enumerable Range List - Does not work
http://rextester.com/ZSCM64375
C#
List<List<string>> filters = Enumerable.Range(0, 101)
.SelectMany(a => Enumerable.Range(0, 101).Select(b => "A{0:00}B{1:00}"))
.Select(i => Enumerable.Range(0, 10).Select(c => string.Empty).ToList())
.SelectMany(a => Enumerable.Range(0, 101).Select(b => "{0:000}"))
.SelectMany(a => Enumerable.Range(0, 101).Select(b => "{0:00}"))
.SelectMany(a => Enumerable.Range(0, 1).Select(b => "hello"))
.ToList();
List<string> matches = new List<string>();
// Sentence
string sentence = "This is a A05B09 hello 02 100 test";
string newSentence = string.Empty;
// Find Matches
for (int i = 0; i < filters.Count; i++)
{
// Add to Matches List
if (sentence.Contains(filters[i].ToString()))
{
matches.Add(filters[i]);
}
}
// Filter Sentence
newSentence = Regex.Replace(
sentence
, #"(?<!\S)(" + string.Join("|", matches) + #")(?!\S)"
, ""
, RegexOptions.IgnoreCase
);
// Display New Sentence
Console.WriteLine(newSentence);

I think creating a list of all possible combinations is a very bad approach. You are creating huge lists which will make your process use a lot of RAM and be very slow without any good reason. Why not just create a good Regex? For example, with this expression, you get your desired string:
\b(A\d\dB\d\d|A100B100|0?\d\d|100|hello)\b\s*
That is assuming you don't want to replace stuff like A101B101 or 123.
If you want to replace those as well, the regex is a bit simpler:
\b(A\d\d\d?B\d\d\d?|\d\d\d?|hello)\b\s*

Your this line seems not meet your requirements..SelectMany(a => Enumerable.Range(0, 101).Select(b => "A{0:00}B{1:00}"))
Can you try this Linq?
List<string> filters = Enumerable.Range(0, 101)
.SelectMany(a => Enumerable.Range(0, 101).Select(b => $"A{a:00}B{b:00}"))
.Union(Enumerable.Range(0, 101).Select(b => $"{b:000}"))
.Union(Enumerable.Range(0, 101).Select(b => $"{b:00}"))
.Union(new List<string> {"hello"})
.ToList();
This verion can give you expected result on rextester
List<string> filters = Enumerable.Range(0, 101)
.SelectMany(a => Enumerable.Range(0, 101).Select(b => string.Format("A{0:00}B{1:00}", a, b)))
.Union(Enumerable.Range(0, 101).Select(b => string.Format("{0:000}", b)))
.Union(Enumerable.Range(0, 101).Select(b => string.Format("{0:00}", b)))
.Union(new List<string> { "hello" })
.ToList();

Check value in datatable with Linq

I am creating a Word Cloud and so I am splitting my sentences in Linq using Regex and grouping the words and taking the count of them. However, I don't want some blacklist words to appear in my cloud, so I get those words in a datatable (dtBlackList) and check with Linq as shown in the code below
var result = (Regex.Split(StringsForWordCloud, #"\W+")
.GroupBy(s => s, StringComparer.InvariantCultureIgnoreCase)
.Where(q => q.Key.Trim() != "")
.Where(q => (dtBlackList.Select("blacklistword = '" + q.Key.Trim() + "'").Count() == 0))
.OrderByDescending(g => g.Count())
.Select(p => new { Word = p.Key, Count = p.Count() })
).Take(200);
Will this query affect my performance badly? Is this the right way to check against a datatable?

A LINQ query as this one will execute a query for each word found with the Regex.Split operation. I'm referring to this line of code:
.Where(q => (dtBlackList.Select("blacklistword = '" + q.Key.Trim() + "'").Count() == 0))
I've had to deal with a lot of performance problems on the project I'm working right now, caused by situations similar to this one.
In general, performing a query to check or complete the data extracted in your database is not a good practice.
In your case, I think it's much better to write a single query that will extract the blacklist words and then exclude that list from the dataset you have just extracted. As follows:
var words = Regex.Split(StringsForWordCloud, #"\W+")
.GroupBy(s => s, StringComparer.InvariantCultureIgnoreCase)
.Where(q => q.Key.Trim() != "")
.OrderByDescending(g => g.Count())
.Select(p => new { Word = p.Key, Count = p.Count() });
// Now extract all the word in the blacklist
IEnumerable<string> blackList = dtBlackList...
// Now exclude them from the set of words all in once
var result = words.Where(w => !blackList.Contains(w.Word)
.OrderByDescending(g => g.Count())
.Take(200);

Identifying and grouping similar items in a collection of strings

I have a collection of strings like the following:
List<string> codes = new List<string>
{
"44.01", "44.02", "44.03", "44.04", "44.05", "44.06", "44.07", "44.08", "46", "47.10"
};
Each string is made up of two components separated by a full stop - a prefix code and a subcode. Some of the strings don't have sub codes.
I want to be able combine the strings whose prefixes are the same and output them as follows with the other codes also:
44(01,02,03,04,05,06,07,08),46,47.10
I'm stuck at the first hurdle of this, which is how to identify and group together the codes whose prefix values are the same, so that I can combine them into a single string as you can see above.

You can do:
var query = codes.Select(c =>
new
{
SplitArray = c.Split('.'), //to avoid multiple split
Value = c
})
.Select(c => new
{
Prefix = c.SplitArray.First(), //you can avoid multiple split if you split first and use it later
PostFix = c.SplitArray.Last(),
Value = c.Value,
})
.GroupBy(r => r.Prefix)
.Select(grp => new
{
Key = grp.Key,
Items = grp.Count() > 1 ? String.Join(",", grp.Select(t => t.PostFix)) : "",
Value = grp.First().Value,
});
This is how it works:
Split each item in the list on the delimiter and populate an anonymous type with Prefix, Postfix and original value
Later group on Prefix
after that select the values and the post fix values using string.Join
For output:
foreach (var item in query)
{
if(String.IsNullOrWhiteSpace(item.Items))
Console.WriteLine(item.Value);
else
Console.WriteLine("{0}({1})", item.Key, item.Items);
}
Output would be:
44(01,02,03,04,05,06,07,08)
46
47.10

Try this:-
var result = codes.Select(x => new { SplitArr = x.Split('.'), OriginalValue = x })
.GroupBy(x => x.SplitArr[0])
.Select(x => new
{
Prefix= x.Key,
subCode = x.Count() > 1 ?
String.Join(",", x.Select(z => z.SplitArray[1])) : "",
OriginalValue = x.First().OriginalValue
});
You can print your desired output like this:-
foreach (var item in result)
{
Console.Write("{0}({1}),",item.Prefix,item.subCode);
}
Working Fiddle.

Outlined idea:
Use Dictionary<string, List<string>> for collecting your result
in a loop over your list, use string.split() .. the first element will be your Dictionary key ... create a new List<string> there if the key doesn't exist yet
if the result of split has a second element, append that to the List
use a second loop to format that Dictionary to your output string
Of course, linq is possible too, e.g.
List<string> codes = new List<string>() {
"44.01", "44.05", "47", "42.02", "44.03" };
var result = string.Join(",",
codes.OrderBy(x => x)
.Select(x => x.Split('.'))
.GroupBy(x => x[0])
.Select((x) =>
{
if (x.Count() == 0) return x.Key;
else if (x.Count() == 1) return string.Join(".", x.First());
else return x.Key + "(" + string.Join(",", x.Select(e => e[1]).ToArray()) + ")";
}).ToArray());
Gotta love linq ... haha ... I think this is a monster.

You can do it all in one clever LINQ:
var grouped = codes.Select(x => x.Split('.'))
.Select(x => new
{
Prefix = int.Parse(x[0]),
Subcode = x.Length > 1 ? int.Parse(x[1]) : (int?)null
})
.GroupBy(k => k.Prefix)
.Select(g => new
{
Prefix = g.Key,
Subcodes = g.Where(s => s.Subcode.HasValue).Select(s => s.Subcode)
})
.Select(x =>
x.Prefix +
(x.Subcodes.Count() == 1 ? string.Format(".{0}", x.Subcodes.First()) :
x.Subcodes.Count() > 1 ? string.Format("({0})", string.Join(",", x.Subcodes))
: string.Empty)
).ToArray();
First it splits by Code and Subcode
Group by you Code, and get all Subcodes as a collection
Select it in the appropriate format
Looking at the problem, I think you should stop just before the last Select and let the data presentation be done in another part/method of your application.

The old fashioned way:
List<string> codes = new List<string>() {"44.01", "44.05", "47", "42.02", "44.03" };
string output=""
for (int i=0;i<list.count;i++)
{
string [] items= (codes[i]+"..").split('.') ;
int pos1=output.IndexOf(","+items[0]+"(") ;
if (pos1<0) output+=","+items[0]+"("+items[1]+")" ; // first occurence of code : add it
else
{ // Code already inserted : find the insert point
int pos2=output.Substring(pos1).IndexOf(')') ;
output=output.Substring(0,pos2)+","+items[1]+output.Substring(pos2) ;
}
}
if (output.Length>0) output=output.Substring(1).replace("()","") ;

This will work, including the correct formats for no subcodes, a single subcode, multiple subcodes. It also doesn't assume the prefix or subcodes are numeric, so it leaves leading zeros as is. Your question didn't show what to do in the case you have a prefix without subcode AND the same prefix with subcode, so it may not work in that edge case (44,44.01). I have it so that it ignores the prefix without subcode in that edge case.
List<string> codes = new List<string>
{
"44.01", "44.02", "44.03", "44.04", "44.05", "44.06", "44.07", "44.08", "46", "47.10"
};
var result=codes.Select(x => (x+".").Split('.'))
.Select(x => new
{
Prefix = x[0],
Subcode = x[1]
})
.GroupBy(k => k.Prefix)
.Select(g => new
{
Prefix = g.Key,
Subcodes = g.Where(s => s.Subcode!="").Select(s => s.Subcode)
})
.Select(x =>
x.Prefix +
(x.Subcodes.Count() == 0 ? string.Empty :
string.Format(x.Subcodes.Count()>1?"({0})":".{0}",
string.Join(",", x.Subcodes)))
).ToArray();

General idea, but i'm sure replacing the Substring calls with Regex would be a lot better as well
List<string> newCodes = new List<string>()
foreach (string sub1 in codes.Select(item => item.Substring(0,2)).Distinct)
{
StringBuilder code = new StringBuilder();
code.Append("sub1(");
foreach (string sub2 in codes.Where(item => item.Substring(0,2) == sub1).Select(item => item.Substring(2))
code.Append(sub2 + ",");
code.Append(")");
newCodes.Add(code.ToString());
}

You could go a couple ways... I could see you making a Dictionary<string,List<string>> so that you could have "44" map to a list of {".01", ".02", ".03", etc.} This would require you processing the codes before adding them to this list (i.e. separating out the two parts of the code and handling the case where there is only one part).
Or you could put them into a a SortedSet and provide your own Comparator which knows that these are codes and how to sort them (at least that'd be more reliable than grouping them alphabetically). Iterating over this SortedSet would still require special logic, though, so perhaps the Dictionary to List option above is still preferable.
In either case you would still need to handle a special case "46" where there is no second element in the code. In the dictionary example, would you insert a String.Empty into the list? Not sure what you'd output if you got a list {"46", "46.1"} -- would you display as "46(null,1)" or... "46(0,1)"... or "46(,1)" or "46(1)"?

Counting the Frequency of Specific Words in Text File

I have a text file stored as a string variable. The text file is processed so that it only contains lowercase words and spaces. Now, say I have a static dictionary, which is just a list of specific words, and I want to count, from within the text file, the frequency of each word in the dictionary. For example:
Text file:
i love love vb development although i m a total newbie
Dictionary:
love, development, fire, stone
The output I'd like to see is something like the following, listing both the dictionary word and its count. If it makes coding simpler, it can also only list the dictionary word that appeared in the text.
===========
WORD, COUNT
love, 2
development, 1
fire, 0
stone, 0
============
Using a regex (eg "\w+") I can get all the word matches, but I have no clue how to get the counts that are also in the dictionary, so I'm stuck. Efficiency is crucial here since the dictionary is quite large (~100,000 words) and the text files are not small either (~200kb each).
I appreciate any kind help.

You can count the words in the string by grouping them and turning it into a dictionary:
Dictionary<string, int> count =
theString.Split(' ')
.GroupBy(s => s)
.ToDictionary(g => g.Key, g => g.Count());
Now you can just check if the words exist in the dictionary, and show the count if it does.

var dict = new Dictionary<string, int>();
foreach (var word in file)
if (dict.ContainsKey(word))
dict[word]++;
else
dict[word] = 1;

Using Groovy regex facilty, i would do it as below :-
def input="""
i love love vb development although i m a total newbie
"""
def dictionary=["love", "development", "fire", "stone"]
dictionary.each{
def pattern= ~/${it}/
match = input =~ pattern
println "${it}" + "-"+ match.count
}

Try this. The words variable is obviously your string of text. The keywords array is a list of keywords you want to count.
This won't return a 0 for dictionary words that aren't in the text, but you specified that this behavior is okay. This should give you relatively good performance while meeting the requirements of your application.
string words = "i love love vb development although i m a total newbie";
string[] keywords = new[] { "love", "development", "fire", "stone" };
Regex regex = new Regex("\\w+");
var frequencyList = regex.Matches(words)
.Cast<Match>()
.Select(c => c.Value.ToLowerInvariant())
.Where(c => keywords.Contains(c))
.GroupBy(c => c)
.Select(g => new { Word = g.Key, Count = g.Count() })
.OrderByDescending(g => g.Count)
.ThenBy(g => g.Word);
//Convert to a dictionary
Dictionary<string, int> dict = frequencyList.ToDictionary(d => d.Word, d => d.Count);
//Or iterate through them as is
foreach (var item in frequencyList)
Response.Write(String.Format("{0}, {1}", item.Word, item.Count));
If you want to achieve the same thing without using RegEx since you indicated you know everything is lower case and separated by spaces, you could modify the above code like so:
string words = "i love love vb development although i m a total newbie";
string[] keywords = new[] { "love", "development", "fire", "stone" };
var frequencyList = words.Split(' ')
.Select(c => c)
.Where(c => keywords.Contains(c))
.GroupBy(c => c)
.Select(g => new { Word = g.Key, Count = g.Count() })
.OrderByDescending(g => g.Count)
.ThenBy(g => g.Word);
Dictionary<string, int> dict = frequencyList.ToDictionary(d => d.Word, d => d.Count);

Develop Reference

C# (C-Sharp) is a programming language developed by Microsoft that runs on the .NET Framework.

The most common word in spaceless string - c#

Related

Extract values from a string into arrays

Check if String Contains Match in Enumerable.Range Filter List

Check value in datatable with Linq

Identifying and grouping similar items in a collection of strings

Counting the Frequency of Specific Words in Text File

Categories

Resources