Working with several DBs in parallel and need to initialize some records with hashed passwords. In MS SQL server there are handy functions that allow to hash on the fly:
HashBytes('SHA1', CONVERT(nvarchar(32), N'admin'))
Is there is a similar function with SQLite?
If not, which is the easiest workaround (such as select from SQL server and somehow insert it into SQLite tables)?
The preferred hashing algorithm is SHA1 and the passwords are stored in a BLOB column.
Update: I use C# language in the current project.
There is no such function built into SQLite3.
But you could define a user function e.g. with sqlite3_create_function if you're using the C interface, and implement SHA-1 with that. (But if you're having a programmable interface perhaps you could just SHA-1 the password outside of the SQL engine.)
You could also try to find / create an extension and load with the load_extension function, but I don't have experience on that.
Edit:
See this answer on SQLiteFunction Simple Not Working for how to define a custom function with System.Data.SQLite in C#.
Use System.Security.Cryptography.SHA1 to compute the SHA-1 hash.
SQLite does not come with SHA1, but it is relatively easily to add. You didn't say what language, you're using, but you can look at the C documentation for create_function and sqlite3_result. You can also take a look at this example of how to add SHA1 to SQLite using Ruby.
With System.Data.SQLite, they're called user-defined functions. You can look at this example on the main site.
Noting that sqlite does have a sha1() extension added in 2017
https://www.sqlite.org/src/file/ext/misc/sha1.c
although it may not be enabled by default.
You can create a custom function for SHA1 in C# like this:
[SQLiteFunction(Name = "Sha1", Arguments = 1, FuncType = FunctionType.Scalar)]
public class Sha1 : SQLiteFunction
{
public override object Invoke(object[] args)
{
var buffer = args[0] as byte[];
if ( buffer == null )
{
var s = args[0] as string;
if ( s != null )
buffer = Encoding.Unicode.GetBytes(s);
}
if ( buffer == null )
return null;
using ( var sha1 = SHA1.Create() )
{
return sha1.ComputeHash(buffer);
}
}
}
This function can be called for binary data or strings. Strings are hashed in their Unicode representation. This should match SQL Server.
The function can be called like this:
select sha1('abc')
select sha1(x'010203')
The following builds latest sqlite with dynamic library support, and compiles sha1 extension. It also assumes debian-based linux distribution:
sudo apt build-dep sqlite3 # fetches dependencies to compile sqlite3
mkdir sqlite-compilation
cd sqlite-compilation
wget -O sqlite.tar.gz https://www.sqlite.org/src/tarball/sqlite.tar.gz?r=release
tar xzf sqlite.tar.gz
mkdir build
cd build
../sqlite/configure
make OPTS='-DSQLITE_ENABLE_LOAD_EXTENSION'
./sqlite3 -cmd 'pragma compile_options;' <<< .exit
cd -
cd sqlite/ext/misc
# https://sqlite.org/src/file?name=ext/misc/sha1.c
sed -i 's/int sqlite3_sha_init(/int sqlite3_extension_init(/' sha1.c # this is needed to give object file custom name, for example libSqlite3Sha1.so:
gcc -g -O2 -shared -fPIC -I ../../../build -o libSqlite3Sha1.so ./sha1.c
cp libSqlite3Sha1.so ../../../build/
cd -
In result you will have:
build/sqlite3 # sqlite3 binary
build/libSqlite3Sha1.so # sha1 extension
Test:
cd build
sqlite3 <<< '
.load ./libSqlite3Sha1
select sha1(1);
.exit
'
# compare output with:
echo -n 1 | sha1sum
cd -
As far as I know, SQLite doesn't have any hashing functions built-in.
There is a way to add custom functions to SQLite, but it's probably easier if you just calculate the SHA1 hash in your program and store it in SQlite.
Creating custom functions for SQLite depends somewhat on the API and the language you're using. I only have experience with creating SQLite functions from Python.
Related
Does C# have a library that can reproduce the same thing?
from tensorflow import keras
from tensorflow.keras import layers
Functions to reproduce:
# A utility function to decode the output of the network
def decode_batch_predictions(pred):
input_len = np.ones(pred.shape[0]) * pred.shape[1]
# Use greedy search. For complex tasks, you can use beam search
results = keras.backend.ctc_decode(pred, input_length=input_len, greedy=True)[0][0][:, :max_length]
# Iterate over the results and get back the text
output_text = []
for res in results:
res = tf.strings.reduce_join(num_to_char(res)).numpy().decode("utf-8")
output_text.append(res)
return output_text
The situation is as follows. I trained the model, it works fine. But I haven 't found a way to decode the model 's response in the environment yet .NET ;(
I will be grateful for any help.
The recommended way to store secrets (like API keys and passwords) during development of an ASP.NET core 3 application is to use user secrets. While this approach works nicely I do have a multiline string which contains an RSA key. The key is copied from a provided .pem file.
Is there any easy way to store the key in secrets.json?
The problem seems to be that json does not support multiline strings. Thus simply copying the key into the file does not work. Workarounds like using an array for the different lines of the key does not play nicely with the Configuration class and binding retrieved secrets to an Options-class.
I have figured out that once I convert the key into a single line string it works. However, the need for a separate tool to convert the key from multiline to single line, seems to me too complex.
By the way, I do need this for building a GitHub-App.
There's nothing stopping you from using a multi-line string with user secrets. You can pass one directly to dotnet user-secrets
For example, some Powershell using a here-string:
$multiVal = #"
Your
Multi-line
Text
"#
dotnet user-secrets set "YourKeyName" "$multiVal"
Or with embedded new-line character `n:
dotnet user-secrets "YourKeyName" "Your`nMulti-line`nValue"
Or you could read in an entire text-based file:
$fileName = "/path/to/file"
$multiVal = Get-Content $fileName -Raw
dotnet user-secrets set "YourKeyName" "$multiVal"
A JSON string property also allows "multi-line" text, just not in the way you're thinking. The literal characters \ and n together inside of a string property will be deserialized as a new-line. For example, the following JSON has a string property with a multi-line value:
{
"YourKeyName": "Your\nMulti-line\nText"
}
You can achieve this in a variety of ways, for example doing a manual find-and-replace or with tools like Notepad++. You could also use some Powershell once again:
$inputFile = "/path/to/file"
$multiVal = Get-Content $inputFile -Raw
$obj = [pscustomobject]#{
YourKeyName = $multiVal
}
$outputFile = "/path/to/secrets.json"
$obj | ConvertTo-Json -Depth 50 | Out-File -FilePath $outputFile
Edit: you mentioned one of the parameters is not working in the final example. It's possible you are somehow running an older version of powershell (pre 3.0). You can try this instead:
$inputFile = "/path/to/file"
# no -Raw flag
$multiVal = (Get-Content $inputFile | Out-String)
# or alternatively
$multiVal = [System.IO.File]::ReadAllText($inputFile)
$obj = [pscustomobject]#{
YourKeyName = $multiVal
}
$outputFile = "/path/to/secrets.json"
# use redirection instead of Out-File
($obj | ConvertTo-Json -Depth 50) > $outputFile
Now with respect to RSA keys, according to this answer and its comments while the RSA spec calls for line breaks within the base-64 encoded payload it's possible that implementations may allow non-conformance. This means that depending on how you're using it, you might be able to get away with stripping out the new-lines entirely. You'd have to try it out to know for sure.
Edit: It turns out that dotnet user-secrets has/had a known bug where values cannot have a leading -. It's fixed now but I think only for 5.0+. I found that a leading space works and I would think that the RSA provider shouldn't balk at that. The following should work:
dotnet user-secrets set "PKeyPowerShell" " $multiVal"
I develop an application with command line parameters and use it in cmd shell and powershell. There it is obvious that the arguments are received differently in main() of my application.
static void Main(string[] args)
{
// In cmd shell: args[0] == "-ipaddress=127.0.0.1"
// In powershell: args[0] == "-ipaddress=127"
// and args[1] == ".0.0.1"
}
Example:
myApp.exe -ipaddress=127.0.0.1
In my C# application the arguments are interpreted differently depending on the shell I start the app in.
In cmd shell: arg[0]=-ipaddress=127.0.0.1
In powershell: arg[0]=-ipaddress=127 and arg[1]=.0.0.1
What is best practice here?
Should I join all args[] and parse the arguments in my application?
Should I rely on the shell's parser?
I would abandon cmd shell support completely and just created proper PowerShell cmdlet. Writing a Windows PowerShell Cmdlet. But I don't know your exact requirements.
In general calling executable from cmd and PowerShell should work same way. For example line, "> ping -n 3 google.com" just works fine no matter where you put it.
tl;dr:
When calling from PowerShell, enclose the entire argument in '...' (single quotes) to ensure that it is passed as-is:
myApp.exe '-ipaddress=127.0.0.1'
You've run into a parsing bug[1]
where PowerShell breaks an unquoted argument that starts with - in two at the first .
The following simplified example demonstrates that:
# Helper function that echoes all arguments.
function Out-Argument { $i = 0; $Args | % { 'arg[{0}]: {1}' -f ($i++), $_ }}
# Pass an unquoted argument that starts with "-" and has an embedded "."
Out-Argument -foo=bar.baz
The above yields:
arg[0]: -foo=bar
arg[1]: .baz
[1] The presumptive bug is present as of Windows PowerShell v5.1 / PowerShell Core v6.0.1 and has been reported on GitHub.
A PSv2 variation of the bug affects . when enclosed in "..." in the interior of the argument - see this answer of mine.
I have a variable in my psm1 file that's a KVP hash
$subcmdlist = #{
"addhost" = "Add-Host";
"deletehost" = "Remove-Host";
"setparameter" = "Set-Parameter";
}
This psm1 file also has a function called 'newtask' which accepts an argument of $subcommand.
I'm wondering how i can execute the cmdlet Add-Host when
newtask addhost
is issued from the shell.
I tried to just echo it but that didn't do much good at all. Just printed out the value.
Thanks!
Use the & sign (aka the call operator), like this: & "Get-Host". This works at least in Powershell 3.0.
I'm calling the iOS SecKeyGeneratePair method to generate an RSA key pair. Now I want to extract the RSA parameters from it (both public and private data). How can I do that?
I've found the CryptoExercise sample but I don't see that it's actually exporting keys to their raw data. Besides, it's Objective-C which I am really struggling to grok.
I'm doing this in a Xamarin.iOS (C#) mobile app so I need my ultimate solution to be written in C# by calling the necessary interop APIs into iOS (just like my call to SecKeyGeneratePair).
For those familiar with Xamarin, yes, I know I can use new RSACryptoServiceProvider() to do it much more easily. The difference is 2-3 orders of magnitude perf improvement when I use native APIs for RSA key generation (and even the encryption bit itself). So while I must use native APIs for the RSA work, I also need to get the raw data out so that I can be cross-platform.
AFAIK you cannot directly extract the parameters. There was some talk on the Apple developer forums about this (you might want to check). Sadly the Apple engineer answer was the assumptions (about the posted code) were implementation details and subject to change at any time.
Still you might be able to do this indirectly using the available API, e.g. export it as a PKCS#12 and then get the RSA parameters from it.
UPDATE: upon verification you can only import PKCS#12 - so this would not help exporting the private key. I'm back to the there's no supported way of doing it. The only safe option, beside the use of managed code, I can think of is to include native code (3rd party library) to generate the keypairs.
I once tried something similar (it was import, not export) but the time required for the extra operations made my code slower than using only C#. Your situation is different so YMMV.
note: it seemed to me that the keychain access (which is out of process, encrypted...) caused much of the slowdown. It was not worth importing a public key for a one-time use - but if you're (re)using it multiple time then the (one-time) cost might be acceptable to you.
The SecKeyGeneratePair is an older API replaced by SecKeyCreateRandomKey that's only available as of iOS 10. So I will answer using the older functions that the question asks about. I'll stick to Core Foundation APIs for interoperability with C#.
You can export the public and private raw key data by adding it to the keychain and returning it as data bytes. Here's an example:
//Convert key object into data
SecKeyRef givenKey = publicOrPrivateFromSecKeyGeneratePair;
static const uint8_t publicKeyIdentifier[] = "com.company.myTempRSAKey"; //make it unique per key
CFDataRef publicTag = CFDataCreate(kCFAllocatorDefault, publicKeyIdentifier, sizeof(publicKeyIdentifier));
if (publicTag)
{
OSStatus sanityCheck = noErr;
CFDataRef publicKeyBits = NULL;
//Create a dictionary info object describing that we are using RSA
CFMutableDictionaryRef queryPublicKey = CFDictionaryCreateMutable(kCFAllocatorDefault, 5, &kCFTypeDictionaryKeyCallBacks, &kCFTypeDictionaryValueCallBacks);
if (queryPublicKey)
{
CFDictionaryAddValue(queryPublicKey, kSecClass, kSecClassKey);
CFDictionaryAddValue(queryPublicKey, kSecAttrApplicationTag, publicTag);
CFDictionaryAddValue(queryPublicKey, kSecAttrKeyType, kSecAttrKeyTypeRSA);
CFDictionaryAddValue(queryPublicKey, kSecAttrKeyClass, kSecAttrKeyClassPublic); //for public or:
//CFDictionaryAddValue(queryPublicKey, kSecAttrKeyClass, kSecAttrKeyClassPrivate); //for private
CFDictionaryAddValue(queryPublicKey, kSecAttrAccessible, kSecAttrAccessibleWhenUnlockedThisDeviceOnly); //other options...
CFMutableDictionaryRef attributes = CFDictionaryCreateMutableCopy(kCFAllocatorDefault, 7, queryPublicKey);
if (attributes)
{
// Temporarily add key to the Keychain, return as data:
CFDictionaryAddValue(attributes, kSecValueRef, givenKey);
CFDictionaryAddValue(attributes, kSecReturnData, kCFBooleanTrue);
CFTypeRef result = NULL;
sanityCheck = SecItemAdd(attributes, &result);
if (sanityCheck == errSecSuccess)
{
publicKeyBits = (CFDataRef)result; // Use the RAW key here
// Remove the temp key from the Keychain
sanityCheck = SecItemDelete(queryPublicKey);
if (sanityCheck != errSecSuccess)
{
//... Error deleting temporary public key from keychain
}
}
// else - failsafe code if key exists, try to delete first and then add item etc.
CFRelease(attributes);
}
CFRelease(queryPublicKey);
}
CFRelease(publicTag);
}
This gets you the raw data. It's raw because it's missing the header that most systems outside of Apple expect. The ASN.1 OID value for RSA public keys followed by a terminating NULL byte, for example.
//HEX: 30 0d 06 09 2a 86 48 86 f7 0d 01 01 01 05 00
/*
SEQUENCE {
OBJECTIDENTIFIER 1.2.840.113549.1.1.1 (rsaEncryption)
NULL
}
*/
So if you're looking for the raw data you have it. If by "parameters" you're trying to extract the modulus and exponent from the raw data, you can do that too (ex. a bit stream: mod + exp). Let me know if that is what you're after.