Develop Reference

Storing multiline RSA Key in .NET core User Secrets (JSON)

The recommended way to store secrets (like API keys and passwords) during development of an ASP.NET core 3 application is to use user secrets. While this approach works nicely I do have a multiline string which contains an RSA key. The key is copied from a provided .pem file.
Is there any easy way to store the key in secrets.json?
The problem seems to be that json does not support multiline strings. Thus simply copying the key into the file does not work. Workarounds like using an array for the different lines of the key does not play nicely with the Configuration class and binding retrieved secrets to an Options-class.
I have figured out that once I convert the key into a single line string it works. However, the need for a separate tool to convert the key from multiline to single line, seems to me too complex.
By the way, I do need this for building a GitHub-App.

There's nothing stopping you from using a multi-line string with user secrets. You can pass one directly to dotnet user-secrets
For example, some Powershell using a here-string:
$multiVal = #"
Your
Multi-line
Text
"#
dotnet user-secrets set "YourKeyName" "$multiVal"
Or with embedded new-line character `n:
dotnet user-secrets "YourKeyName" "Your`nMulti-line`nValue"
Or you could read in an entire text-based file:
$fileName = "/path/to/file"
$multiVal = Get-Content $fileName -Raw
dotnet user-secrets set "YourKeyName" "$multiVal"
A JSON string property also allows "multi-line" text, just not in the way you're thinking. The literal characters \ and n together inside of a string property will be deserialized as a new-line. For example, the following JSON has a string property with a multi-line value:
{
"YourKeyName": "Your\nMulti-line\nText"
}
You can achieve this in a variety of ways, for example doing a manual find-and-replace or with tools like Notepad++. You could also use some Powershell once again:
$inputFile = "/path/to/file"
$multiVal = Get-Content $inputFile -Raw
$obj = [pscustomobject]#{
YourKeyName = $multiVal
}
$outputFile = "/path/to/secrets.json"
$obj | ConvertTo-Json -Depth 50 | Out-File -FilePath $outputFile
Edit: you mentioned one of the parameters is not working in the final example. It's possible you are somehow running an older version of powershell (pre 3.0). You can try this instead:
$inputFile = "/path/to/file"
# no -Raw flag
$multiVal = (Get-Content $inputFile | Out-String)
# or alternatively
$multiVal = [System.IO.File]::ReadAllText($inputFile)
$obj = [pscustomobject]#{
YourKeyName = $multiVal
}
$outputFile = "/path/to/secrets.json"
# use redirection instead of Out-File
($obj | ConvertTo-Json -Depth 50) > $outputFile
Now with respect to RSA keys, according to this answer and its comments while the RSA spec calls for line breaks within the base-64 encoded payload it's possible that implementations may allow non-conformance. This means that depending on how you're using it, you might be able to get away with stripping out the new-lines entirely. You'd have to try it out to know for sure.
Edit: It turns out that dotnet user-secrets has/had a known bug where values cannot have a leading -. It's fixed now but I think only for 5.0+. I found that a leading space works and I would think that the RSA provider shouldn't balk at that. The following should work:
dotnet user-secrets set "PKeyPowerShell" " $multiVal"

Let the shell parse or do your own parsing (powershell/cmd)

I develop an application with command line parameters and use it in cmd shell and powershell. There it is obvious that the arguments are received differently in main() of my application.
static void Main(string[] args)
{
// In cmd shell: args[0] == "-ipaddress=127.0.0.1"
// In powershell: args[0] == "-ipaddress=127"
// and args[1] == ".0.0.1"
}
Example:
myApp.exe -ipaddress=127.0.0.1
In my C# application the arguments are interpreted differently depending on the shell I start the app in.
In cmd shell: arg[0]=-ipaddress=127.0.0.1
In powershell: arg[0]=-ipaddress=127 and arg[1]=.0.0.1
What is best practice here?
Should I join all args[] and parse the arguments in my application?
Should I rely on the shell's parser?

I would abandon cmd shell support completely and just created proper PowerShell cmdlet. Writing a Windows PowerShell Cmdlet. But I don't know your exact requirements.
In general calling executable from cmd and PowerShell should work same way. For example line, "> ping -n 3 google.com" just works fine no matter where you put it.

tl;dr:
When calling from PowerShell, enclose the entire argument in '...' (single quotes) to ensure that it is passed as-is:
myApp.exe '-ipaddress=127.0.0.1'
You've run into a parsing bug[1]
where PowerShell breaks an unquoted argument that starts with - in two at the first .
The following simplified example demonstrates that:
# Helper function that echoes all arguments.
function Out-Argument { $i = 0; $Args | % { 'arg[{0}]: {1}' -f ($i++), $_ }}
# Pass an unquoted argument that starts with "-" and has an embedded "."
Out-Argument -foo=bar.baz
The above yields:
arg[0]: -foo=bar
arg[1]: .baz
[1] The presumptive bug is present as of Windows PowerShell v5.1 / PowerShell Core v6.0.1 and has been reported on GitHub.
A PSv2 variation of the bug affects . when enclosed in "..." in the interior of the argument - see this answer of mine.

add a key value with Symbols chars in Csharp

I tried to add this line to a key in registry
"C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe"-ExecutionPolicy Bypass -windowstyle hidden -noexit -file "%1"
as you see the value that i want to add contains """" and % and -
So how can i do it please
i know with Registry.SetValue as here
Registry.SetValue(#"HKEY_CLASSES_ROOT\Microsoft.PowerShellScript.1\Shell\Open\Command", "Default", #""C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe"-ExecutionPolicy Bypass -windowstyle hidden -noexit -file "%1"");
but errors popup every time i putted # at the beginning
but didnt work also
this key will allow me if i edited it to run powershell script by double click on it
any ideas ?..thx

The syntax highlighting shows the issue. The third parameter is
""C:\Windows\System32\WindowsPowerShell...
Thus, the compiler sees an empty string "" and unexpected C:\Windows\System32\WindowsPowerShell...
As resolution, use string literal properly
#"C:\Windows\System32\WindowsPowerShell...

Just put another " to escape each "
Registry.SetValue(#"HKEY_CLASSES_ROOT\Microsoft.PowerShellScript.1\Shell\Open\Command", "Default", #"""C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe""-ExecutionPolicy Bypass -windowstyle hidden -noexit -file ""%1""");

command line arguments given to a C# program are parsed wrong

I wrote a little program, that has the following command line for example:
prog.exe -p -z "E:\temp.zip" -v "f:\" -r -o -s –c
prog.exe -p -z "E:\temp.zip" -v f:\ -r -o -s –c
prog.exe -p -z "E:\temp.zip" -v "f:\log" -r -o -s –c
This command line is being generated by another program that inserts the quotation marks around the file and path names automatically to prevent spaces being recognized as argument separators.
The first command line is then being parsed by the .Net framework as:
args {string[5]} string[]
[0] "-p" string
[1] "-z" string
[2] "f:\\temp.zip" string
[3] "-v" string
[4] "f:\" -r -o -s -c" string
But it should be ( the result from the second command line):
args {string[9]} string[]
[0] "-p" string
[1] "-z" string
[2] "f:\\temp.zip" string
[3] "-v" string
[4] "f:\\" string
[5] "-r" string
[6] "-o" string
[7] "-s" string
[8] "-c" string
A possible solution would be to check the file and path names in the calling application if they contain spaces and only then append the quotation marks around the names.
But is there another solution?

I created a little batch file that would display the command line parameters. With your first command line, I get
1 -p
2 -z
3 "E:\temp.zip"
4 -v
5 "f:\"
6 -r
7 -o
8 -s
9 –c
as it should. If .NET parses it differently, that would be a major bug. Just to check, I created a test console app. The results were:
0 -p
1 -z
2 E:\temp.zip
3 -v
4 f:" -r -o -s -c
I didn't think that was possible! So I researched a little and I got to this link: Everyone quotes command line arguments the wrong way which explains why the command line parser is essentially flawed. (I know it's for C++, but it applies) which led me to the rules of parsing for command line parameters which says \" is interpreted as an escaped quote, as opposed to how the operating system sees it (which is gross).
Conclusion: if you want to fix the command line, you need to escape the slash before a quote so instead of "f:\" you need "f:\\". The other solution is to use Environment.CommandLine which gets you the entire command line, executable included, and you can parse it yourself. More on this here: Getting raw (unsplit) command line in .NET.
Just for completeness' sake, I'll throw this in: Split string containing command-line parameters into string[] in C#, which discusses how to split a command line string into parameters using system functions.
After some more research, I realized that there are several standards
of command line argument parsing, depending on compiler and operating
system and its version, and that the only way to be consistent is to
parse the raw command line yourself.
For example, using CommandLineToArgvW on Environment.CommandLine replicated exactly a call to Environment.GetCommandLineArgs() which returns the exact same thing as the args array given to the Main method.
I found a very detailed page about this problem: http://daviddeley.com/autohotkey/parameters/parameters.htm which I believe is the definitive answer to the general question of command line parsing.

SHA1 hashing in SQLite: how?

Working with several DBs in parallel and need to initialize some records with hashed passwords. In MS SQL server there are handy functions that allow to hash on the fly:
HashBytes('SHA1', CONVERT(nvarchar(32), N'admin'))
Is there is a similar function with SQLite?
If not, which is the easiest workaround (such as select from SQL server and somehow insert it into SQLite tables)?
The preferred hashing algorithm is SHA1 and the passwords are stored in a BLOB column.
Update: I use C# language in the current project.

There is no such function built into SQLite3.
But you could define a user function e.g. with sqlite3_create_function if you're using the C interface, and implement SHA-1 with that. (But if you're having a programmable interface perhaps you could just SHA-1 the password outside of the SQL engine.)
You could also try to find / create an extension and load with the load_extension function, but I don't have experience on that.
Edit:
See this answer on SQLiteFunction Simple Not Working for how to define a custom function with System.Data.SQLite in C#.
Use System.Security.Cryptography.SHA1 to compute the SHA-1 hash.

SQLite does not come with SHA1, but it is relatively easily to add. You didn't say what language, you're using, but you can look at the C documentation for create_function and sqlite3_result. You can also take a look at this example of how to add SHA1 to SQLite using Ruby.
With System.Data.SQLite, they're called user-defined functions. You can look at this example on the main site.

Noting that sqlite does have a sha1() extension added in 2017
https://www.sqlite.org/src/file/ext/misc/sha1.c
although it may not be enabled by default.

You can create a custom function for SHA1 in C# like this:
[SQLiteFunction(Name = "Sha1", Arguments = 1, FuncType = FunctionType.Scalar)]
public class Sha1 : SQLiteFunction
{
public override object Invoke(object[] args)
{
var buffer = args[0] as byte[];
if ( buffer == null )
{
var s = args[0] as string;
if ( s != null )
buffer = Encoding.Unicode.GetBytes(s);
}
if ( buffer == null )
return null;
using ( var sha1 = SHA1.Create() )
{
return sha1.ComputeHash(buffer);
}
}
}
This function can be called for binary data or strings. Strings are hashed in their Unicode representation. This should match SQL Server.
The function can be called like this:
select sha1('abc')
select sha1(x'010203')

The following builds latest sqlite with dynamic library support, and compiles sha1 extension. It also assumes debian-based linux distribution:
sudo apt build-dep sqlite3 # fetches dependencies to compile sqlite3
mkdir sqlite-compilation
cd sqlite-compilation
wget -O sqlite.tar.gz https://www.sqlite.org/src/tarball/sqlite.tar.gz?r=release
tar xzf sqlite.tar.gz
mkdir build
cd build
../sqlite/configure
make OPTS='-DSQLITE_ENABLE_LOAD_EXTENSION'
./sqlite3 -cmd 'pragma compile_options;' <<< .exit
cd -
cd sqlite/ext/misc
# https://sqlite.org/src/file?name=ext/misc/sha1.c
sed -i 's/int sqlite3_sha_init(/int sqlite3_extension_init(/' sha1.c # this is needed to give object file custom name, for example libSqlite3Sha1.so:
gcc -g -O2 -shared -fPIC -I ../../../build -o libSqlite3Sha1.so ./sha1.c
cp libSqlite3Sha1.so ../../../build/
cd -
In result you will have:
build/sqlite3 # sqlite3 binary
build/libSqlite3Sha1.so # sha1 extension
Test:
cd build
sqlite3 <<< '
.load ./libSqlite3Sha1
select sha1(1);
.exit
'
# compare output with:
echo -n 1 | sha1sum
cd -

As far as I know, SQLite doesn't have any hashing functions built-in.
There is a way to add custom functions to SQLite, but it's probably easier if you just calculate the SHA1 hash in your program and store it in SQlite.
Creating custom functions for SQLite depends somewhat on the API and the language you're using. I only have experience with creating SQLite functions from Python.