I fall into a surprising issue.
I loaded a text file in my application and I have some logic which compares the value having µ.
And I realized that even if the texts are same the compare value is false.
Console.WriteLine("μ".Equals("µ")); // returns false
Console.WriteLine("µ".Equals("µ")); // return true
In later line the character µ is copy pasted.
However, these might not be the only characters that are like this.
Is there any way in C# to compare the characters which look the same but are actually different?
Because it is really different symbols even they look the same, first is the actual letter and has char code = 956 (0x3BC) and the second is the micro sign and has 181 (0xB5).
References:
Unicode Character 'GREEK SMALL LETTER MU' (U+03BC)
Unicode Character 'MICRO SIGN' (U+00B5)
So if you want to compare them and you need them to be equal, you need to handle it manually, or replace one char with another before comparison. Or use the following code:
public void Main()
{
var s1 = "μ";
var s2 = "µ";
Console.WriteLine(s1.Equals(s2)); // false
Console.WriteLine(RemoveDiacritics(s1).Equals(RemoveDiacritics(s2))); // true
}
static string RemoveDiacritics(string text)
{
var normalizedString = text.Normalize(NormalizationForm.FormKC);
var stringBuilder = new StringBuilder();
foreach (var c in normalizedString)
{
var unicodeCategory = CharUnicodeInfo.GetUnicodeCategory(c);
if (unicodeCategory != UnicodeCategory.NonSpacingMark)
{
stringBuilder.Append(c);
}
}
return stringBuilder.ToString().Normalize(NormalizationForm.FormC);
}
And the Demo
In many cases, you can normalize both of the Unicode characters to a certain normalization form before comparing them, and they should be able to match. Of course, which normalization form you need to use depends on the characters themselves; just because they look alike doesn't necessarily mean they represent the same character. You also need to consider if it's appropriate for your use case — see Jukka K. Korpela's comment.
For this particular situation, if you refer to the links in Tony's answer, you'll see that the table for U+00B5 says:
Decomposition <compat> GREEK SMALL LETTER MU (U+03BC)
This means U+00B5, the second character in your original comparison, can be decomposed to U+03BC, the first character.
So you'll normalize the characters using full compatibility decomposition, with the normalization forms KC or KD. Here's a quick example I wrote up to demonstrate:
using System;
using System.Text;
class Program
{
static void Main(string[] args)
{
char first = 'μ';
char second = 'µ';
// Technically you only need to normalize U+00B5 to obtain U+03BC, but
// if you're unsure which character is which, you can safely normalize both
string firstNormalized = first.ToString().Normalize(NormalizationForm.FormKD);
string secondNormalized = second.ToString().Normalize(NormalizationForm.FormKD);
Console.WriteLine(first.Equals(second)); // False
Console.WriteLine(firstNormalized.Equals(secondNormalized)); // True
}
}
For details on Unicode normalization and the different normalization forms refer to System.Text.NormalizationForm and the Unicode spec.
They both have different character codes: Refer this for more details
Console.WriteLine((int)'μ'); //956
Console.WriteLine((int)'µ'); //181
Where, 1st one is:
Display Friendly Code Decimal Code Hex Code Description
====================================================================
μ μ μ μ Lowercase Mu
µ µ µ µ micro sign Mu
For the specific example of μ (mu) and µ (micro sign), the latter has a compatibility decomposition to the former, so you can normalize the string to FormKC or FormKD to convert the micro signs to mus.
However, there are lots of sets of characters that look alike but aren't equivalent under any Unicode normalization form. For example, A (Latin), Α (Greek), and А (Cyrillic). The Unicode website has a confusables.txt file with a list of these, intended to help developers guard against homograph attacks. If necessary, you could parse this file and build a table for “visual normalization” of strings.
Search both characters in a Unicode database and see the difference.
One is the Greek small Letter µ and the other is the Micro Sign µ.
Name : MICRO SIGN
Block : Latin-1 Supplement
Category : Letter, Lowercase [Ll]
Combine : 0
BIDI : Left-to-Right [L]
Decomposition : <compat> GREEK SMALL LETTER MU (U+03BC)
Mirror : N
Index entries : MICRO SIGN
Upper case : U+039C
Title case : U+039C
Version : Unicode 1.1.0 (June, 1993)
Name : GREEK SMALL LETTER MU
Block : Greek and Coptic
Category : Letter, Lowercase [Ll]
Combine : 0
BIDI : Left-to-Right [L]
Mirror : N
Upper case : U+039C
Title case : U+039C
See Also : micro sign U+00B5
Version : Unicode 1.1.0 (June, 1993)
EDIT After the merge of this question with How to compare 'μ' and 'µ' in C#
Original answer posted:
"μ".ToUpper().Equals("µ".ToUpper()); //This always return true.
EDIT
After reading the comments, yes it is not good to use the above method because it may provide wrong results for some other type of inputs, for this we should use normalize using full compatibility decomposition as mentioned in wiki. (Thanks to the answer posted by BoltClock)
static string GREEK_SMALL_LETTER_MU = new String(new char[] { '\u03BC' });
static string MICRO_SIGN = new String(new char[] { '\u00B5' });
public static void Main()
{
string Mus = "µμ";
string NormalizedString = null;
int i = 0;
do
{
string OriginalUnicodeString = Mus[i].ToString();
if (OriginalUnicodeString.Equals(GREEK_SMALL_LETTER_MU))
Console.WriteLine(" INFORMATIO ABOUT GREEK_SMALL_LETTER_MU");
else if (OriginalUnicodeString.Equals(MICRO_SIGN))
Console.WriteLine(" INFORMATIO ABOUT MICRO_SIGN");
Console.WriteLine();
ShowHexaDecimal(OriginalUnicodeString);
Console.WriteLine("Unicode character category " + CharUnicodeInfo.GetUnicodeCategory(Mus[i]));
NormalizedString = OriginalUnicodeString.Normalize(NormalizationForm.FormC);
Console.Write("Form C Normalized: ");
ShowHexaDecimal(NormalizedString);
NormalizedString = OriginalUnicodeString.Normalize(NormalizationForm.FormD);
Console.Write("Form D Normalized: ");
ShowHexaDecimal(NormalizedString);
NormalizedString = OriginalUnicodeString.Normalize(NormalizationForm.FormKC);
Console.Write("Form KC Normalized: ");
ShowHexaDecimal(NormalizedString);
NormalizedString = OriginalUnicodeString.Normalize(NormalizationForm.FormKD);
Console.Write("Form KD Normalized: ");
ShowHexaDecimal(NormalizedString);
Console.WriteLine("_______________________________________________________________");
i++;
} while (i < 2);
Console.ReadLine();
}
private static void ShowHexaDecimal(string UnicodeString)
{
Console.Write("Hexa-Decimal Characters of " + UnicodeString + " are ");
foreach (short x in UnicodeString.ToCharArray())
{
Console.Write("{0:X4} ", x);
}
Console.WriteLine();
}
Output
INFORMATIO ABOUT MICRO_SIGN
Hexa-Decimal Characters of µ are 00B5
Unicode character category LowercaseLetter
Form C Normalized: Hexa-Decimal Characters of µ are 00B5
Form D Normalized: Hexa-Decimal Characters of µ are 00B5
Form KC Normalized: Hexa-Decimal Characters of µ are 03BC
Form KD Normalized: Hexa-Decimal Characters of µ are 03BC
________________________________________________________________
INFORMATIO ABOUT GREEK_SMALL_LETTER_MU
Hexa-Decimal Characters of µ are 03BC
Unicode character category LowercaseLetter
Form C Normalized: Hexa-Decimal Characters of µ are 03BC
Form D Normalized: Hexa-Decimal Characters of µ are 03BC
Form KC Normalized: Hexa-Decimal Characters of µ are 03BC
Form KD Normalized: Hexa-Decimal Characters of µ are 03BC
________________________________________________________________
While reading information in Unicode_equivalence I found
The choice of equivalence criteria can affect search results. For instance some typographic ligatures like U+FB03 (ffi), ..... so a search for U+0066 (f) as substring would succeed in an NFKC normalization of U+FB03 but not in NFC normalization of U+FB03.
So to compare equivalence we should normally use FormKC i.e. NFKC normalization or FormKD i.e NFKD normalization.
I was little curious to know more about all the Unicode characters so I made sample which would iterate over all the Unicode character in UTF-16 and I got some results I want to discuss
Information about characters whose FormC and FormD normalized values were not equivalent
Total: 12,118
Character (int value): 192-197, 199-207, 209-214, 217-221, 224-253, ..... 44032-55203
Information about characters whose FormKC and FormKD normalized values were not equivalent
Total: 12,245
Character (int value): 192-197, 199-207, 209-214, 217-221, 224-228, ..... 44032-55203, 64420-64421, 64432-64433, 64490-64507, 64512-64516, 64612-64617, 64663-64667, 64735-64736, 65153-65164, 65269-65274
All the character whose FormC and FormD normalized value were not equivalent, there FormKC and FormKD normalized values were also not equivalent except these characters
Characters: 901 '΅', 8129 '῁', 8141 '῍', 8142 '῎', 8143 '῏', 8157 '῝', 8158 '῞'
, 8159 '῟', 8173 '῭', 8174 '΅'
Extra character whose FormKC and FormKD normalized value were not equivalent, but there FormC and FormD normalized values were equivalent
Total: 119
Characters: 452 'DŽ' 453 'Dž' 454 'dž' 12814 '㈎' 12815 '㈏' 12816 '㈐' 12817 '㈑' 12818 '㈒'
12819 '㈓' 12820 '㈔' 12821 '㈕', 12822 '㈖' 12823 '㈗' 12824 '㈘' 12825 '㈙' 12826 '㈚'
12827 '㈛' 12828 '㈜' 12829 '㈝' 12830 '㈞' 12910 '㉮' 12911 '㉯' 12912 '㉰' 12913 '㉱'
12914 '㉲' 12915 '㉳' 12916 '㉴' 12917 '㉵' 12918 '㉶' 12919 '㉷' 12920 '㉸' 12921 '㉹' 12922 '㉺' 12923 '㉻' 12924 '㉼' 12925 '㉽' 12926 '㉾' 13056 '㌀' 13058 '㌂' 13060 '㌄' 13063 '㌇' 13070 '㌎' 13071 '㌏' 13072 '㌐' 13073 '㌑' 13075 '㌓' 13077 '㌕' 13080 '㌘' 13081 '㌙' 13082 '㌚' 13086 '㌞' 13089 '㌡' 13092 '㌤' 13093 '㌥' 13094 '㌦' 13099 '㌫' 13100 '㌬' 13101 '㌭' 13102 '㌮' 13103 '㌯' 13104 '㌰' 13105 '㌱' 13106 '㌲' 13108 '㌴' 13111 '㌷' 13112 '㌸' 13114 '㌺' 13115 '㌻' 13116 '㌼' 13117 '㌽' 13118 '㌾' 13120 '㍀' 13130 '㍊' 13131 '㍋' 13132 '㍌' 13134 '㍎' 13139 '㍓' 13140 '㍔' 13142 '㍖' .......... ﺋ' 65164 'ﺌ' 65269 'ﻵ' 65270 'ﻶ' 65271 'ﻷ' 65272 'ﻸ' 65273 'ﻹ' 65274'
There are some characters which can not be normalized, they throw ArgumentException if tried
Total:2081
Characters(int value): 55296-57343, 64976-65007, 65534
This links can be really helpful to understand what rules govern for Unicode equivalence
Unicode_equivalence
Unicode_compatibility_characters
Most likely, there are two different character codes that make (visibly) the same character. While technically not equal, they look equal. Have a look at the character table and see whether there are multiple instances of that character. Or print out the character code of the two chars in your code.
You ask "how to compare them" but you don't tell us what you want to do.
There are at least two main ways to compare them:
Either you compare them directly as you are and they are different
Or you use Unicode Compatibility Normalization if your need is for a comparison that finds them to match.
There could be a problem though because Unicode compatibility normalization will make many other characters compare equal. If you want only these two characters to be treated as alike you should roll your own normalization or comparison functions.
For a more specific solution we need to know your specific problem. What is the context under which you came across this problem?
If I would like to be pedantic, I would say that your question doesn't make sense, but since we are approaching christmas and the birds are singing, I'll proceed with this.
First off, the 2 entities that you are trying to compare are glyphs, a glyph is part of a set of glyphs provided by what is usually know as a "font", the thing that usually comes in a ttf, otf or whatever file format you are using.
The glyphs are a representation of a given symbol, and since they are a representation that depends on a specific set, you can't just expect to have 2 similar or even "better" identical symbols, it's a phrase that doesn't make sense if you consider the context, you should at least specify what font or set of glyphs you are considering when you formulate a question like this.
What is usually used to solve a problem similar to the one that you are encountering, it's an OCR, essentially a software that recognize and compares glyphs, If C# provides an OCR by default I don't know that, but it's generally a really bad idea if you don't really need an OCR and you know what to do with it.
You can possibly end up interpreting a physics book as an ancient greek book without mentioning the fact that OCR are generally expensive in terms of resources.
There is a reason why those characters are localized the way they are localized, just don't do that.
It's possible to draw both of chars with the same font style and size with DrawString method. After two bitmaps with symbols has been generated, it's possible to compare them pixel by pixel.
Advantage of this method is that you can compare not only absolute equal charcters, but similar too (with definite tolerance).
Related
I have the following string (output from sox --info command):
Input File : 'C:\Users\source\repos\dotnetcore\audio\1000.wav'
Channels : 1
Sample Rate : 44100
Precision : 16-bit
Duration : 00:05:11.64 = 13743363 samples = 23373.1 CDDA sectors
File Size : 27.5M
Bit Rate : 706k
Sample Encoding: 16-bit Signed Integer PCM
I need to extract the file path (without the single quote), channels, sample rate etc.
I have a method where I pass in the whole string (the output) and the property I want to extract. Like this:
private static string Extract(string inputStr, string property)
{
string pattern = string.Format(#"\s+{0}\s+: '?(.*)\r\n", property);
Match result = Regex.Match(inputStr, pattern);
if (result.Success)
{
return result.Groups[1].Value;
}
return string.Empty;
}
This almost returns what I need, except for the last single quote in the Input File. How do I not include that in the pattern
Extract(output, "Input File") //returns C:\Users\source\repos\dotnetcore\audio\1000.wav' --> How to remove the last single quote
Extract(output, "Channels") //returns 1 --> Good
Extract(output, "Sample Rate") // returns 44100 --> Good
I have tried these patterns also
\s+Input File\s+: '?(.*)'? //Still returns with the last single quote
\s+Input File\s+: '?(.*)'+ //This works for Input File but doesn't work for other properties
Edit: Based on the original author’s comment and my not inspecting all the lines closely enough
\s+{0}\s*: '?([^\r\n']*)'?
Original:
\s+{0}\s+: '?([^\r\n']*)'?
This is because * is greedy--it will keep pulling as many characters as it can. Because the ? allows 0 or 1 characters, it doesn't stop the * from continuing to pull in characters.
Try the following expression: \s*(?<name>[^:]+?)\s*:\s*(?<value>('[^']+')|.+)
See demo: https://regex101.com/r/w7b2oO/1
A couple of differences:
(?<name>...) gives the capture group a name so that you can reference it by name instead of index
('[^']+')|.+) makes captures values enclosed in a string ('[^']+') or (|) without (.+)
I'm trying to figure out how to write code to let the user input three values (string, int, int) in one line with space to separate the values.
I thought of doing it with String.Split Method but that only works if all the values have the same datatype.
How can I do it with different datatypes?
For example:
The user might want to input
Hello 23 54
I'm using console application C#
Well the first problem is that you need to decide whether the text the user enters itself can contain spaces. For example, is the following allowed?
Hello World, it's me 08 15
In that case, String.Split will not really be helpful.
What I'd try is using a regular expression. The following may serve as a starting point:
Match m = Regex.Match(input, #"^(?<text>.+) (?<num1>(\+|\-)?\d+) (?<num2>(\+|\-)?\d+)$");
if (m.Success)
{
string stringValue = m.Groups["text"].Value;
int num1 = Convert.ToInt32(m.Groups["num1"].Value);
int num2 = Convert.ToInt32(m.Groups["num2"].Value);
}
BTW: The following part of your question makes me frown:
I thought of doing it with String.Split Method but that only works if all the values have the same datatype.
A string is always just a string. Whether it contains a text, your email-address or your bank account balance. It is always just a series of characters. The notion that the string contains a number is just your interpretation!
So from a program's point of view, the string you gave is a series of characters. And for splitting that it doesn't matter at all what the real semantics of the content are.
That's why the splitting part is separate from the conversion part. You need to tell your application that that the first part is a string, the second and third parts however are supposed to be numbers. That's what you need type conversions for.
You are confusing things. A string is either null, empty or contains a sequence of characters. It never contains other data types. However, it might contain parts that could be interpreted as numbers, dates, colors etc... (but they are still strings). "123" is not an int! It is a string containing a number.
In order to extract these pieces you need to do two things:
Split the string into several string parts.
Convert string parts that are supposed to represent whole numbers into a the int type (=System.Int32).
string input = "Abc 123 456"
string[] parts = input.Split(); //Whitespaces are assumed as separators by default.
if (parts.Count == 3) {
Console.WriteLine("The text is \"{0}\"", parts[0]);
int n1;
if (Int32.TryParse(parts[1], out n1)) {
Console.WriteLine("The 1st number is {0}", n1);
} else {
Console.WriteLine("The second part is supposed to be a whole number.");
}
int n2;
if (Int32.TryParse(parts[2], out n2)) {
Console.WriteLine("The 2nd number is {0}", n2);
} else {
Console.WriteLine("The third part is supposed to be a whole number.");
}
} else {
Console.WriteLine("You must enter three parts separated by a space.");
}
What you have to do is get "Hello 23 54" in a string variable. Split by " " and treat them.
string value = "Hello 23 54";
var listValues = value.Split(' ').ToList();
After that you have to parse each item from listValues to your related types.
Hope it helps. ;)
I have string "H20" (chemical formula for water). I would like to change it so that all the numbers in the string would be small (the number 2 would be small index next to the letter H). How can I do that?
Assuming that you have the means of displaying the subscript unicode characters, you could easily write your own extension method for subscripting:
public static string Subscript(this string normal)
{
if(normal == null) return normal;
var res = new StringBuilder();
foreach(var c in normal)
{
char c1 = c;
// I'm not quite sure if char.IsDigit(c) returns true for, for example, '³',
// so I'm using the safe approach here
if (c >= '0' && c <= '9')
{
// 0x208x is the unicode offset of the subscripted number characters
c1 = (char)(c - '0' + 0x2080);
}
res.Append(c1);
}
return res.ToString();
}
As has been pointed out in the comments, you should typically use some presentation technology for this kind of formatting. For example, in HTML, you could display your text through:
<span>H<sub>2</sub>O</span>
However, Unicode allocates a superscripts and subscripts block for hexadecimal characters which you can take advantage of. Since Unicode is supported natively in .NET, including within string literals, you can use your desired character directly:
text = text.Replace("H2O", "H₂O");
Note: Using the Unicode subscript character would guarantee that your H₂O string would render correctly in any Unicode-aware application, irrespective of its formatting technology (HTML, RTF, PDF, XPS, etc).
Below is a screenshot showing how the string renders in a TextBox under Windows Forms. To improve legibility, font has been changed to Cambria, 11.25pt.
Edit: If you want to convert all numerals (not just 2 into ₂), you could use #Tobias’s code. Here is a regex adaptation of it. I’ve included a lookbehind since I assume that all numerals to be subscripted must be preceded by a letter.
text = Regex.Replace(text, #"(?<=[A-Za-z])\d",
match => ((char)(match.Value[0] - '0' + '₀')).ToString());
The above would transform a string like
CF3CH2Cl + Br2 → CF3CHBrCl + HBr
into
CF₃CH₂Cl + Br₂ → CF₃CHBrCl + HBr
Im trying to convert non-English string (greek) to title string
I did try as this link suggest but with no luck, all the chars got Upper case
Converting string to title case
how can i work with Unicode chars ?
All chars are Unicode chars. We English speakers don't use magical non-Unicode chars from another universe, nor are char characters used in English so obscure as to not be in Unicode yet.
You don't detail precisely what you tried with TextInfo, and the answer you link to isn't very detailed. When I try:
CurrentCulture.TextInfo.ToTitleCase("English here, then some Greek: Ποικιλόθρον', ἀθάνατ' ἀφρόδιτα, παῖ δίος, δολόπλοκε, λίσσομαί σε μή μ' ἄσαισι μήτ' ὀνίαισι δάμνα, πότνια, θῦμον·")
I get back:
English Here, Then Some Greek: Ποικιλόθρον', Ἀθάνατ' Ἀφρόδιτα, Παῖ Δίος, Δολόπλοκε, Λίσσομαί Σε Μή Μ' Ἄσαισι Μήτ' Ὀνίαισι Δάμνα, Πότνια, Θῦμον·
However, if I start with upper-case:
System.Globalization.CultureInfo.CurrentCulture.TextInfo.ToTitleCase("ENGLISH HERE, THEN SOME GREEK: ΠΟΙΚΙΛΌΘΡΟΝ', ἈΘΆΝΑΤ' ἈΦΡΌΔΙΤΑ, ΠΑῖ ΔΊΟΣ, ΔΟΛΌΠΛΟΚΕ, ΛΊΣΣΟΜΑΊ ΣΕ ΜΉ Μ' ἌΣΑΙΣΙ ΜΉΤ' ὈΝΊΑΙΣΙ ΔΆΜΝΑ, ΠΌΤΝΙΑ, ΘῦΜΟΝ·")
I all upper-case like you describe. Are you also starting with upper-case?
Title case leaves all-upper-case words untouched to avoid damaging acronyms and abbreviations like ".NET", "NATO", "ΙΧΘΥΣ", etc. If you need to deal with this, do ToLower first:
var ti = System.Globalization.CultureInfo.CurrentCulture.TextInfo;
return ti.ToTitleCase(ti.ToLower("ENGLISH HERE, THEN SOME GREEK: ΠΟΙΚΙΛΌΘΡΟΝ', ἈΘΆΝΑΤ' ἈΦΡΌΔΙΤΑ, ΠΑῖ ΔΊΟΣ, ΔΟΛΌΠΛΟΚΕ, ΛΊΣΣΟΜΑΊ ΣΕ ΜΉ Μ' ἌΣΑΙΣΙ ΜΉΤ' ὈΝΊΑΙΣΙ ΔΆΜΝΑ, ΠΌΤΝΙΑ, ΘῦΜΟΝ·"));
Greek is not the easiest case for the ToTitleCase linguistically.
TextInfo ti = new CultureInfo("el-GR", false).TextInfo;
experiment 1:
Console.WriteLine(ti.ToTitleCase("εθνικό χρέος"));
the output is: Εθνικό Χρέος
experiment 2:
Console.WriteLine(ti.ToTitleCase("ΕΘΝΙΚΟ ΧΡΕΟΣ"));
the output is: ΕΘΝΙΚΟ ΧΡΕΟΣ
experiment 3:
Console.WriteLine(ti.ToTitleCase("ΕΘΝΙΚΟ ΧΡΕΟΣ".ToLower()));
the output is: Εθνικο Χρεοσ
Output 1 and 3 are different. Output 3 is missing the diacritics (tonos in Greek) at ό and έ and uses σ at the end of a word instead of ς (final s - teliko sigma in Greek). According to the above results, I suggest you to title case only lowered case phrases and leave the uppercase ones as they are, because the result will have for sure many mistakes that your Greek audience will not like. Alternatively you can find a Greek guy to help you on results linguistic accuracy.
For the record "εθνικό χρέος" means national debt - the primary reason to move to another not just country but continent with my family.
I can't tell from the question if it's always in sentence case when it comes in but if you need to split in addition to Title case the string, maybe this method might help you get started.
private static string ToTitleCase(string example)
{
var fromSnakeCase = example.Replace("_", " ");
var lowerToUpper = Regex.Replace(fromSnakeCase, #"(\p{Ll})(\p{Lu})", "$1 $2");
var sentenceCase = Regex.Replace(lowerToUpper, #"(\p{Lu}+)(\p{Lu}\p{Ll})", "$1 $2");
return new CultureInfo("el-GR", false).TextInfo.ToTitleCase(sentenceCase);
}
I have some code to dump strings to stdout to check their encoding, it looks like this:
private void DumpString(string s)
{
System.Console.Write("{0}: ", s);
foreach (byte b in s)
{
System.Console.Write("{0}({1}) ", (char)b, b.ToString("x2"));
}
System.Console.WriteLine();
}
Consider two strings, each of which appear as "ë", but with different encodings. DumpString will produce the following output:
ë: e(65)(08)
ë: ë(eb)
The code looks like this:
DumpString(string1);
DumpString(string2);
How can I convert string2, using the System.Text.Encoding, to be byte equivalent to string1.
They don't have different encodings. Strings in C# are always UTF-16 (thus, you shouldn't use byte to iterate over strings because you'll lose the top 8 bits). What they have is different normalization forms.
Your first string is "\u0065\u0308": LATIN SMALL LETTER E + COMBINING DIAERESIS. This is the decomposed form (NFD).
The second is "\u00EB": LATIN SMALL LETTER E WITH DIAERESIS. This is the precomposed form (NFC).
You can convert between them with string.Normalize.
You're looking for the String.Normalize method.