Develop Reference

Superpower: match a string with tokenizer only if it begins a line

When tokenizing in superpower, how to match a string only if it is the first thing in a line (note: this is a different question than this one) ?
For example, assume I have a language with only the following 4 characters (' ', ':', 'X', 'Y'), each of which is a token. There is also a 'Header' token to capture cases of the following regex pattern /^[XY]+:/ (any number of Xs and Ys followed by a colon, only if they start the line).
Here is a quick class for testing (the 4th test-case fails):
using System;
using Superpower;
using Superpower.Parsers;
using Superpower.Tokenizers;
public enum Tokens { Space, Colon, Header, X, Y }
public class XYTokenizer
{
static void Main(string[] args)
{
Test("X", Tokens.X);
Test("XY", Tokens.X, Tokens.Y);
Test("X Y:", Tokens.X, Tokens.Space, Tokens.Y, Tokens.Colon);
Test("X: X", Tokens.Header, Tokens.Space, Tokens.X);
}
public static readonly Tokenizer<Tokens> tokenizer = new TokenizerBuilder<Tokens>()
.Match(Character.EqualTo('X'), Tokens.X)
.Match(Character.EqualTo('Y'), Tokens.Y)
.Match(Character.EqualTo(':'), Tokens.Colon)
.Match(Character.EqualTo(' '), Tokens.Space)
.Build();
static void Test(string input, params Tokens[] expected)
{
var tokens = tokenizer.Tokenize(input);
var i = 0;
foreach (var t in tokens)
{
if (t.Kind != expected[i])
{
Console.WriteLine("tokens[" + i + "] was Tokens." + t.Kind
+ " not Tokens." + expected[i] + " for '" + input + "'");
return;
}
i++;
}
Console.WriteLine("OK");
}
}

I came up with a custom Tokenizer based on the example found here. I added comments throughout the code so you can follow what's happening.
public class MyTokenizer : Tokenizer<Tokens>
{
protected override IEnumerable<Result<Tokens>> Tokenize(TextSpan input)
{
Result<char> next = input.ConsumeChar();
bool checkForHeader = true;
while (next.HasValue)
{
// need to check for a header when starting a new line
if (checkForHeader)
{
var headerStartLocation = next.Location;
var tokenQueue = new List<Result<Tokens>>();
while (next.HasValue && (next.Value == 'X' || next.Value == 'Y'))
{
tokenQueue.Add(Result.Value(next.Value == 'X' ? Tokens.X : Tokens.Y, next.Location, next.Remainder));
next = next.Remainder.ConsumeChar();
}
// only if we had at least one X or one Y
if (tokenQueue.Any())
{
if (next.HasValue && next.Value == ':')
{
// this is a header token; we have to return a Result of the start location
// along with the remainder at this location
yield return Result.Value(Tokens.Header, headerStartLocation, next.Remainder);
next = next.Remainder.ConsumeChar();
}
else
{
// this isn't a header; we have to return all the tokens we parsed up to this point
foreach (Result<Tokens> tokenResult in tokenQueue)
{
yield return tokenResult;
}
}
}
if (!next.HasValue)
yield break;
}
checkForHeader = false;
if (next.Value == '\r')
{
// skip over the carriage return
next = next.Remainder.ConsumeChar();
continue;
}
if (next.Value == '\n')
{
// line break; check for a header token here
next = next.Remainder.ConsumeChar();
checkForHeader = true;
continue;
}
if (next.Value == 'A')
{
var abcStart = next.Location;
next = next.Remainder.ConsumeChar();
if (next.HasValue && next.Value == 'B')
{
next = next.Remainder.ConsumeChar();
if (next.HasValue && next.Value == 'C')
{
yield return Result.Value(Tokens.ABC, abcStart, next.Remainder);
next = next.Remainder.ConsumeChar();
}
else
{
yield return Result.Empty<Tokens>(next.Location, $"unrecognized `AB{next.Value}`");
}
}
else
{
yield return Result.Empty<Tokens>(next.Location, $"unrecognized `A{next.Value}`");
}
}
else if (next.Value == 'X')
{
yield return Result.Value(Tokens.X, next.Location, next.Remainder);
next = next.Remainder.ConsumeChar();
}
else if (next.Value == 'Y')
{
yield return Result.Value(Tokens.Y, next.Location, next.Remainder);
next = next.Remainder.ConsumeChar();
}
else if (next.Value == ':')
{
yield return Result.Value(Tokens.Colon, next.Location, next.Remainder);
next = next.Remainder.ConsumeChar();
}
else if (next.Value == ' ')
{
yield return Result.Value(Tokens.Space, next.Location, next.Remainder);
next = next.Remainder.ConsumeChar();
}
else
{
yield return Result.Empty<Tokens>(next.Location, $"unrecognized `{next.Value}`");
next = next.Remainder.ConsumeChar(); // Skip the character anyway
}
}
}
}
And you can call it like this:
var tokens = new MyTokenizer().Tokenize(input);

Parse WwwAuthenticate challenge string

I am working on a client for a RESTful service, using .NET Core 2.0. The remote service returns challenges like this:
WwwAuthenticate: Bearer realm="https://somesite/auth",service="some site",scope="some scope"
Which need to get turned into token requests like:
GET https://somesite/auth?service=some%20site&scope=some%20scope
Parsing the header to get a scheme and parameter is easy with AuthenticationHeaderValue, but that just gets me the realm="https://somesite/auth",service="some site",scope="some scope" string. How can I easily and reliably parse this to the individual realm, service, and scope components? It's not quite JSON, so deserializing it with NewtonSoft JsonConvert won't work. I could regex it into something that looks like XML or JSON, but that seems incredibly hacky (not to mention unreliable).
Surely there's a better way?

Since I don't see a non-hacky way. Maybe this hacky way may help
string input = #"WwwAuthenticate: Bearer realm=""https://somesite/auth"",service=""some site"",scope=""some, scope""";
var dict = Regex.Matches(input, #"[\W]+(\w+)=""(.+?)""").Cast<Match>()
.ToDictionary(x => x.Groups[1].Value, x => x.Groups[2].Value);
var url = dict["realm"] + "?" + string.Join("&", dict.Where(x => x.Key != "realm").Select(x => x.Key + "=" + WebUtility.UrlEncode(x.Value)));
OUTPUT
url => https://somesite/auth?service=some+site&scope=some%2C+scope
BTW: I added a , in "scope"

Possible duplicate of How to parse values from Www-Authenticate
Using the schema defined in RFC6750 and RFC2616, a slightly more precise parser implementation is included below. This parser takes into account the possibility that strings might contain =, ,, and/or escaped ".
internal class AuthParamParser
{
private string _buffer;
private int _i;
private AuthParamParser(string param)
{
_buffer = param;
_i = 0;
}
public static Dictionary<string, string> Parse(string param)
{
var state = new AuthParamParser(param);
var result = new Dictionary<string, string>();
var token = state.ReadToken();
while (!string.IsNullOrEmpty(token))
{
if (!state.ReadDelim('='))
return result;
result.Add(token, state.ReadString());
if (!state.ReadDelim(','))
return result;
token = state.ReadToken();
}
return result;
}
private string ReadToken()
{
var start = _i;
while (_i < _buffer.Length && ValidTokenChar(_buffer[_i]))
_i++;
return _buffer.Substring(start, _i - start);
}
private bool ReadDelim(char ch)
{
while (_i < _buffer.Length && char.IsWhiteSpace(_buffer[_i]))
_i++;
if (_i >= _buffer.Length || _buffer[_i] != ch)
return false;
_i++;
while (_i < _buffer.Length && char.IsWhiteSpace(_buffer[_i]))
_i++;
return true;
}
private string ReadString()
{
if (_i < _buffer.Length && _buffer[_i] == '"')
{
var buffer = new StringBuilder();
_i++;
while (_i < _buffer.Length)
{
if (_buffer[_i] == '\\' && (_i + 1) < _buffer.Length)
{
_i++;
buffer.Append(_buffer[_i]);
_i++;
}
else if (_buffer[_i] == '"')
{
_i++;
return buffer.ToString();
}
else
{
buffer.Append(_buffer[_i]);
_i++;
}
}
return buffer.ToString();
}
else
{
return ReadToken();
}
}
private bool ValidTokenChar(char ch)
{
if (ch < 32)
return false;
if (ch == '(' || ch == ')' || ch == '<' || ch == '>' || ch == '#'
|| ch == ',' || ch == ';' || ch == ':' || ch == '\\' || ch == '"'
|| ch == '/' || ch == '[' || ch == ']' || ch == '?' || ch == '='
|| ch == '{' || ch == '}' || ch == 127 || ch == ' ' || ch == '\t')
return false;
return true;
}
}

Serialize arithmetic expression-tree

I'm doing a simple task for investigation purposes. The problem is as follows:
Create an arithmetic expression with variables.
Build an AST for the expression.
Send it to the server (using Sockets)
Calculate an result of the server side and return the results.
Now I can to build a tree. This method doing it:
private readonly Stack<Expression> expressionStack = new Stack<Expression>();
private readonly Stack<Symbol> operatorStack = new Stack<Symbol>();
private readonly List<string> parameters = new List<string>();
public Expression<Func<decimal[], decimal>> Parse(string expression)
{
if (string.IsNullOrWhiteSpace(expression))
{
return s => 0;
}
var arrayParameter = Expression.Parameter(typeof(decimal[]), "args");
parameters.Clear();
operatorStack.Clear();
expressionStack.Clear();
using (var reader = new StringReader(expression))
{
int peek;
while ((peek = reader.Peek()) > -1)
{
var next = (char)peek;
if (char.IsDigit(next))
{
expressionStack.Push(ReadOperand(reader));
continue;
}
if (char.IsLetter(next))
{
expressionStack.Push(ReadParameter(reader, arrayParameter));
continue;
}
if (Operation.IsDefined(next))
{
if (next == '-' && expressionStack.Count == 0)
{
reader.Read();
operatorStack.Push(Operation.UnaryMinus);
continue;
}
var currentOperation = ReadOperation(reader);
EvaluateWhile(() => operatorStack.Count > 0 && operatorStack.Peek() != Parentheses.Left &&
currentOperation.Precedence <= ((Operation)operatorStack.Peek()).Precedence);
operatorStack.Push(currentOperation);
continue;
}
if (next == '(')
{
reader.Read();
operatorStack.Push(Parentheses.Left);
if (reader.Peek() == '-')
{
reader.Read();
operatorStack.Push(Operation.UnaryMinus);
}
continue;
}
if (next == ')')
{
reader.Read();
EvaluateWhile(() => operatorStack.Count > 0 && operatorStack.Peek() != Parentheses.Left);
operatorStack.Pop();
continue;
}
if (next == ' ')
{
reader.Read();
}
else
{
throw new ArgumentException(string.Format("Encountered invalid character {0}", next),
"expression");
}
}
}
EvaluateWhile(() => operatorStack.Count > 0);
return Expression.Lambda<Func<decimal[], decimal>>(expressionStack.Pop(), arrayParameter);
}
This method works, and returns the expected result.
Before being sent to the server I want to serialize a tree to binary type. My question is as follows. Which is the simplest way I can apply for this?
I found a lot of solutions for LINQ serialization, but they are too big. I don't need the full functionality of these solutions. In addition, usually, they provide the JSON or XML-serialization, but I need a binary serialization.
Can somebody suggest a simple and easy solution for this problem?

C# checking if expression is brackets valid [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
The expression:
"( a[i]+{-1}*(8-9) )"
should return true since it is valid to write syntax like this. Every left bracket has a right closer in the correct place and all brackets are at legal positions.
I tried to do this via one stack and I know where I'm wrong but I want to know a relevant way to solve this.
thx!
My poor poor wrong code:
string expression = "( a[i]+{-1}*(8-9) ) ";
Stack<char> expStack = new Stack<char>();
List<char> rightBracketsHolder = new List<char>();
for (int i = 0; i < expression.Length; i++)
{
if (expression[i] == '{')
{
expStack.Push('}');
Console.Write("}" + " ");
}
else if (expression[i] == '(')
{
expStack.Push(')');
Console.Write(")" + " ");
}
else if (expression[i] == '[')
{
expStack.Push(']');
Console.Write("]" + " ");
}
}
Console.WriteLine();
for (int i = 0; i < expression.Length; i++)
{
if (expression[i] == '}')
{
rightBracketsHolder.Add('}');
Console.Write(expression[i] + " ");
}
else if (expression[i] == ')')
{
rightBracketsHolder.Add(')');
Console.Write(expression[i] + " ");
}
else if (expression[i] == ']')
{
rightBracketsHolder.Add(']');
Console.Write(expression[i] + " ");
}
}
Console.WriteLine();
bool stackResult = checkValidity(expStack, rightBracketsHolder);
if (stackResult)
Console.WriteLine("Expression is Valid.");
else
Console.WriteLine("\nExpression is not valid.");
Console.ReadKey();
}
private static bool checkValidity(Stack<char> expStack, List<char> leftBracketsHolder)
{
Console.WriteLine();
int length = leftBracketsHolder.Count;
for (int i = 0; i < length; i++)
{
if (expStack.Peek().ToString().Contains(leftBracketsHolder.ToString()))
{
leftBracketsHolder.Remove(expStack.Peek());
expStack.Pop();
}
}
if (expStack.Count == 0 && leftBracketsHolder.Count ==0)
{
return true;
}
return false;
}
}

This code will solve your purpose -
static void Main(string[] args)
{
bool error = false;
var str = "( a[i]+{-1}*(8-9) )";
Stack<char> stack = new Stack<char>();
foreach (var item in str.ToCharArray())
{
if (item == '(' || item == '{' || item == '[')
{
stack.Push(item);
}
else if(item == ')' || item == '}' || item == ']')
{
if (stack.Peek() != GetComplementBracket(item))
{
error = true;
break;
}
}
}
if (error)
Console.WriteLine("Incorrect brackets");
else
Console.WriteLine("Brackets are fine");
Console.ReadLine();
}
private static char GetComplementBracket(char item)
{
switch (item)
{
case ')':
return '(';
case '}':
return '{';
case ']':
return '[';
default:
return ' ';
}
}

You need to pop things off the stack as the closing occurs. Try the following code. It will push an open brace/bracket/parenthesis on the stack and the first thing then it will be popped from the stack by a corresponding close. Otherwise it is invalid. If you have no opens on the stack when a close is encountered, it is invalid. If you have any extra opens when you are complete it is invalid.
I also used a switch statement instead of an if statement just because I thought it was easier to read.
using System;
using System.Collections.Generic;
public class Program
{
public static void Main()
{
string expression = "( a[i]+{-1}*(8-9) ) ";
bool stackResult = checkValidity(expression);
if (stackResult)
Console.WriteLine("Expression is Valid.");
else
Console.WriteLine("\nExpression is not valid.");
}
private static bool checkValidity(string expression)
{
Stack<char> openStack = new Stack<char>();
foreach (char c in expression)
{
switch (c)
{
case '{':
case '(':
case '[':
openStack.Push(c);
break;
case '}':
if (openStack.Count == 0 || openStack.Peek() != '{')
{
return false;
}
openStack.Pop();
break;
case ')':
if (openStack.Count == 0 || openStack.Peek() != '(')
{
return false;
}
openStack.Pop();
break;
case ']':
if (openStack.Count == 0 || openStack.Peek() != '[')
{
return false;
}
openStack.Pop();
break;
default:
break;
}
}
return openStack.Count == 0;
}
}

C# for case in string(easy)

so I have this code. I need to generate a for loop that checks all the characters in the string and checks if they are all valid(So numbers from 0->7). But I don't know how to write it, I tried something but it didn't work. Here are the examples:user enters: 77, code works, user enters 99, code doesn't work, user enters 5., code doesn't work, etc..
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace NALOGA1
{
class Program
{
static string decToOct(int stevilo)//v mojon primere 7
{
string izhod = "";
//7>0 DRŽI
while (stevilo > 0)
{
//izhodi se dodeli ostanek deljenja z 8 keri se spremeni v string
izhod = (stevilo % 8) + izhod;
//7/8;
stevilo /= 8;
}
return izhod;
}
static int Octtodesetisko(string stevilo)
{
double vsota = 0;
for (int i = stevilo.Length - 1; i >= 0; i--)
{
int stevka = stevilo[i] - '0';
vsota += (stevka * Math.Pow(8, i));
}
return (int)vsota;
}
static void Main(string[] args)
{
//3 podprogram-in progress
string prvastevilka = Console.ReadLine();
int prvasprememba = Int32.Parse(prvastevilka);
if (prvasprememba > 0)
{
Console.WriteLine(decToOct(prvasprememba));
}
else
{
Console.WriteLine("Napaka");
}
string drugastevilka = Console.ReadLine();
int drugasprememba = Octtodesetisko(drugastevilka);
foreach (char znak in drugastevilka)
{
if(znak!=1 || znak!=2 || znak!=3 || znak!=4 || znak!=5 || znak!=6 || znak!=7)
{
Console.WriteLine("Napaka");
}
else
{
Console.WriteLine("dela :D");
}
}
Console.ReadKey();
}
}
}

Personally, I would take advantage of the LINQ Enumerable.All method to express this in a very concise and readable way:
if (str.Any() && str.All(c => c >= '0' && c <= '7'))
{
Console.WriteLine("good");
}
else
{
Console.WriteLine("bad");
}
EDIT: No LINQ
It's not hard to translate what the LINQ Enumerable.All method does to a normal loop. It's just more verbose:
bool isValid = true;
foreach (char c in str)
{
if (c < '0' || c > '7')
{
isValid = false;
break;
}
}
if (str.Length != 0 && isValid)
{
Console.WriteLine("good");
}
else
{
Console.WriteLine("bad");
}

Firstly, there seems to be a mistake in the line
if(znak!=1 || znak!=2 || znak!=3 || znak!=4 || znak!=5 || znak!=6 || znak!=7)
I guess it should read
if(znak!='1' || znak!='2' || znak!='3' || znak!='4' || znak!='5' || znak!='6' || znak!='7')
which should be compressed to
if (znak >= '0' && znak <= '7')
You can use linq instead of the for loop here like this:
if (drugastevilka.All(c => c >= '0' && c <= '7')
Console.WriteLine("dela :D");
else
Console.WriteLine("Napaka");
But the best solution is probably to use a regular expression:
Regex regex = new Regex("^[0-7]+$");
if (regex.IsMatch(drugastevilka))
Console.WriteLine("dela :D");
else
Console.WriteLine("Napaka");
Edit: the linq solution shown accepts empty strings, the regex (as shown) needs at least 1 character. Exchange the + with a * and it will accept empty strings, too. But I don't think you want to accept empty strings.

You are messing up with the datatype
Can you try with below code
static string decToOct(int stevilo)//v mojon primere 7
{
int izhod = 0;
//7>0 DRŽI
while (stevilo > 0)
{
//izhodi se dodeli ostanek deljenja z 8 keri se spremeni v string
izhod = (stevilo % 8) + izhod;
//7/8;
stevilo /= 8;
}
return (izhod.ToString());
}

What about something like this?
class Program
{
static void Main(string[] args)
{
string someString = "1234567";
string someOtherString = "1287631";
string anotherString = "123A6F2";
Console.WriteLine(IsValidString(someString));
Console.WriteLine(IsValidString(someOtherString));
Console.WriteLine(IsValidString(anotherString));
Console.ReadLine();
}
public static bool IsValidString(string str)
{
bool isValid = true;
char[] splitString = str.ToCharArray(); //get an array of each character
for (int i = 0; i < splitString.Length; i++)
{
try
{
double number = Char.GetNumericValue(splitString[i]); //try to convert the character to a double (GetNumericValue returns a double)
if (number < 0 || number > 7) //we get here if the character is an int, then we check for 0-7
{
isValid = false; //if the character is invalid, we're done.
break;
}
}
catch (Exception) //this will hit if we try to convert a non-integer character.
{
isValid = false;
break;
}
}
return isValid;
}
}
IsValidString() takes a string, converts it to a Char array, then checks each value as such:
Get the numeric value
Check if the value is between 0-7
GetNumericValue will fail on a non-integer character, so we wrap it in a try/catch - if we hit an exception we know that isValid = false, so we break.
If we get a valid number, and it's not between 0-7 we also know that isValid = false, so we break.
If we make it all the way through the list, the string is valid.
The sample given above returns:
IsValidString(someString) == true
IsValidString(someOtherString) == false
IsValidString(anotherString) == false