Develop Reference

Youtube Pubsub Validate signature

I am creating an open Source Project which uses the youtube pubsub api to get notifications on video uploads. I want to verify that the request does come from youtube and not from a 3rd Party by checking the HMAC SHA1 Signature as described.
So, every time 1 Run my Program, I will generate a Secret later, to debug the problem, i use "test" as my secret string.
I use the following method to check if the provided signature is valid
public static bool Check(string body, string signature)
{
using (var hmac = new HMACSHA1(Encoding.UTF8.GetBytes(Secret)))
{
var hashBytes = hmac.ComputeHash(Encoding.UTF8.GetBytes(body));
var hash = Encoding.UTF8.GetString(hashBytes);
Console.WriteLine("Computed Hash " + hash);
return signature.Equals(hash);
}
}
Where body is the request body and signature is a value provided in the request header.
According to https://pubsubhubbub.github.io/PubSubHubbub/pubsubhubbub-core-0.4.html#rfc.section.7
If the subscriber supplied a value for hub.secret in their subscription request, the hub MUST generate an HMAC signature of the payload and include that signature in the request headers of the content distribution request. The X-Hub-Signature header's value MUST be in the form sha1=signature where signature is a 40-byte, hexadecimal representation of a SHA1 signature [RFC3174]. The signature MUST be computed using the HMAC algorithm [RFC2104] with the request body as the data and the hub.secret as the key.
I supply my Secret as hub.secret in my subscription request.
So if I understand it correctly, the hub SHOULD use that secret to generate a HMACSHA1 of the payload -> the body.
I want to regenerate the HMAC and should get the same value, right?
It does not work. Also the computed hash value logged by console.WriteLine is something completely different, not alphabetic characters at all, so I guess it might be a problem with the encoding, but I can't figure it out. Thanks for all the help!

The documentation says "where signature is a 40-byte, hexadecimal representation" so instead of converting hashBytes to an UTF8 string you should convert it to a hexadecimal string.
public static bool Check(string body, string signature)
{
using (var hmac = new HMACSHA1(Encoding.UTF8.GetBytes(Secret)))
{
var hashBytes = hmac.ComputeHash(Encoding.UTF8.GetBytes(body));
var hash = Convert.ToHexString(hashBytes).ToLowerInvariant();
Console.WriteLine("Computed Hash " + hash);
return signature.Equals(hash);
}
}

S3 download object using pre signed v4 signautre

Is there any example in C# to see how to pre-sign all objects using a start-with policy with AWS v4 signature to let customers download object from their respective folder structure instead signing each document separately.
The documentation says :
https://s3.amazonaws.com/examplebucket/test.txt
?X-Amz-Algorithm=AWS4-HMAC-SHA256
&X-Amz-Credential=<your-access-key-id>/20130721/us-east-1/s3/aws4_request
&X-Amz-Date=20130721T201207Z
&X-Amz-Expires=86400
&X-Amz-SignedHeaders=host
&X-Amz-Signature=<signature-value>
But my signature is not working for GET (download) object, while working correctly for upload
void Main()
{
string bucket = "bucket-name-here";
string s3Key = "s3-key-here";
string s3Secret = "secret-here";
string s3Region = "us-east-1";
string Date = DateTime.UtcNow.ToString("yyyyMMdd");
string xAmzDate = DateTime.UtcNow.ToString("yyyyMMdd") + "T000000Z";
string expiration = DateTime.UtcNow.AddDays(1).ToString("yyyy-MM-ddTHH:mm:ssK");
string policyString = $#"{{""expiration"":""{expiration}"",""conditions"":[{{""bucket"":""{bucket}""}},{{""acl"":""private""}},[""starts-with"",""$key"",""Client_1""],[""starts-with"",""$Content-Type"",""""],[""starts-with"",""$filename"",""""],{{""x-amz-date"":""{xAmzDate}""}},{{""x-amz-credential"":""{s3Key}/{Date}/us-east-1/s3/aws4_request""}},{{""x-amz-algorithm"":""AWS4-HMAC-SHA256""}}]}}";
var policyStringBytes = Encoding.UTF8.GetBytes(policyString);
var policy = Convert.ToBase64String(policyStringBytes);
//policy.Dump();
byte[] signingKey = GetSigningKey(s3Secret, Date, s3Region, "s3");
byte[] signature = HmacSHA256(policy, signingKey);
var sign = ToHexString(signature);
sign.Dump();
}
static byte[] HmacSHA256(String data, byte[] key)
{
String algorithm = "HmacSHA256";
KeyedHashAlgorithm kha = KeyedHashAlgorithm.Create(algorithm);
kha.Key = key;
return kha.ComputeHash(Encoding.UTF8.GetBytes(data));
}
private byte[] GetSigningKey(String key, String dateStamp, String regionName, String serviceName)
{
byte[] kSecret = Encoding.UTF8.GetBytes(("AWS4" + key).ToCharArray());
byte[] kDate = HmacSHA256(dateStamp, kSecret);
byte[] kRegion = HmacSHA256(regionName, kDate);
byte[] kService = HmacSHA256(serviceName, kRegion);
byte[] kSigning = HmacSHA256("aws4_request", kService);
return kSigning;
}
public static string ToHexString(byte[] data)
{
StringBuilder sb = new StringBuilder();
for (int i = 0; i < data.Length; i++)
{
sb.Append(data[i].ToString("x2", CultureInfo.InvariantCulture));
}
return sb.ToString();
}
More about the problem: We have thousands of documents for hundreds of clients on S3 on their respective folder structure as below. Right now, every time the client is looking to download their object they gets signed by our API to created the downloadable link > so each document is signed separately.
Client 1
Client_1/Document1.xyz
Client_1/Document2.xyz
Client 2
Client_2/Document1.xyz
Client_2/Document2.xyz

The signing algorithm for S3 HTML form POST uploads allows you to sign a policy document with constraints like ["starts-with","$key",...] but pre-signed URLs for S3 don't support this. With a pre-signed URL, you sign not a policy document but a "canonical request," which is a canonicalized representation of the browser's exact request. So there is no support for wildcards or prefixes.
There are two alternatives that come to mind.
CloudFront signed URLs and signed cookies do support a policy document when you use a "custom policy" (not a "canned policy," which is more like what S3 supports) and a custom policy allows a * in the URL, similar to ["starts-with","$key",...] but using the URL the browser will be requesting. You only have to do the signing once, and the code running in the browser can reuse that policy and signature. On the back-side of CloudFront, a CloudFront Origin Access Identity is used to sign the requests as they are actually sent to the bucket, after authenticating the request on the front-side of CloudFront using the CloudFront signed URL or signed cookies. (With signed cookies, the browser just makes the request, and automatically submits the cookies, so that works the same but with no browser manipulation of the URLs.
Alternately, the AssumeRole action in Security Token Service could be called by your server, to generate a set of temporary credentials for the client to use, to sign its own individual URLs.
When calling AssumeRole, you can also pass an optional session policy document. If you do, then the generated temporary credentials can only perform actions allowed by both the role policy ("allow read from the bucket") and the session policy ("allow read from the bucket for keys beginning with a specific prefix"). So the role credentials obtained would only allow the user to access to their objects.

JSON Web Token Vulnerability test in C# review

I've attempted to check if the JWT Token Bearer support in WebAPI for .NET Framework 4.5 had the vulnerability as reported by https://auth0.com/blog/critical-vulnerabilities-in-json-web-token-libraries/.
So I wrote the following code that takes an existing, valid JWT token, manipulates the header, regenerates the signature using the certificate public key and resend it to the server.
private static string TestSecurityVulnerability(string originalToken)
{
var parts = originalToken.Split(new[] { '.' }, StringSplitOptions.RemoveEmptyEntries);
var headerPart = Encoding.UTF8.GetString(Convert.FromBase64String(parts.First()));
var payloadPart = Encoding.UTF8.GetString(Convert.FromBase64String(parts.Skip(1).First()));
dynamic headerJson = JsonConvert.DeserializeObject(headerPart);
headerJson["alg"] = "HS256";
headerJson["kid"] = 0;
var tamperedHeaderEncoded = Base64UrlEncode(Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(headerJson)));
var cert = Certificate.Load();
var publicKey = cert.PublicKey.EncodedKeyValue.RawData;
var base64PublicKey = Convert.ToBase64String(publicKey); //Not using it but great for verifying if token is "valid"
var sha = new HMACSHA256(publicKey);
var newSigBinary = sha.ComputeHash(Encoding.UTF8.GetBytes(tamperedHeaderEncoded + "." + parts.Skip(1).First()));
var newSigEncoded = Base64UrlEncode(newSigBinary);
return string.Join(".", tamperedHeaderEncoded,
parts.Skip(1).First(),
newSigEncoded);
}
// from JWT spec
private static string Base64UrlEncode(byte[] input)
{
var output = Convert.ToBase64String(input);
output = output.Split('=')[0]; // Remove any trailing '='s
output = output.Replace('+', '-'); // 62nd char of encoding
output = output.Replace('/', '_'); // 63rd char of encoding
return output;
}
As for Certificate.Load(), all it does is look up the certificate that I'm using on IdentityServer to sign the token and it returns with a System.Security.Cryptography.X509Certificates.X509Certificate2 certificate (PFX with public and private key).
I also inspected the value of base64PublicKey to see what the base64'd version is of the public key, to use that as the 256-bit-secret on jwt.io.
My question is, did I do everything correctly? I've checked my solution on https://jwt.io/ and it seems legit from the point of view that I'm signing the JWT correctly but I don't know if I am extracting the certificate correctly.
Would be great just to have a pair of eyes go through this and let me know.
P.S. with my current test the server came back saying "Unauthorized" ;-)

How to serialize and deserialize a PFX certificate in Azure Key Vault?

I have a bunch of strings and pfx certificates, which I want to store in Azure Key vault, where only allowed users/apps will be able to get them. It is not hard to do store a string as a Secret, but how can I serialize a certificate in such way that I could retrieve it and deserialize as an X509Certificate2 object in C#?
I tried to store it as a key. Here is the Azure powershell code
$securepfxpwd = ConvertTo-SecureString -String 'superSecurePassword' -AsPlainText -Force
$key = Add-AzureKeyVaultKey -VaultName 'UltraVault' -Name 'MyCertificate' -KeyFilePath 'D:\Certificates\BlaBla.pfx' -KeyFilePassword $securepfxpwd
But when I tried to get it with GetKeyAsync method, I couldn't use it.

Here's a PowerShell script for you. Replace the file path, password, vault name, secret name.
$pfxFilePath = 'C:\mycert.pfx'
$pwd = '123'
$flag = [System.Security.Cryptography.X509Certificates.X509KeyStorageFlags]::Exportable
$collection = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2Collection
$collection.Import($pfxFilePath, $pwd, $flag)
$pkcs12ContentType = [System.Security.Cryptography.X509Certificates.X509ContentType]::Pkcs12
$clearBytes = $collection.Export($pkcs12ContentType)
$fileContentEncoded = [System.Convert]::ToBase64String($clearBytes)
$secret = ConvertTo-SecureString -String $fileContentEncoded -AsPlainText –Force
$secretContentType = 'application/x-pkcs12'
Set-AzureKeyVaultSecret -VaultName 'myVaultName' -Name 'mySecretName' -SecretValue $Secret -ContentType $secretContentType
This is a common question, so we are going to polish this up and release as a helper.
The script above strips the password because there's no value in having a password protected PFX and then storing the password next to it.

The original question asked how to retrieve the stored PFX as an X509Certificate2 object. Using a Base64 process similar to that posted by Sumedh Barde above (which has the advantage of stripping the password), the following code will return a X509 object. In a real application, the KeyVaultClient should be cached if you're retrieving multiple secrets, and the individual secrets should also be cached.
public static async Task<X509Certificate2> GetSecretCertificateAsync(string secretName)
{
string baseUri = #"https://xxxxxxxx.vault.azure.net/secrets/";
var provider = new AzureServiceTokenProvider();
var client = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(provider.KeyVaultTokenCallback));
var secretBundle = await client.GetSecretAsync($"{baseUri}{secretName}").ConfigureAwait(false);
string pfx = secretBundle.Value;
var bytes = Convert.FromBase64String(pfx);
var coll = new X509Certificate2Collection();
coll.Import(bytes, "certificatePassword", X509KeyStorageFlags.Exportable);
return coll[0];
}

Here is how I solved this. First, convert your PFX file to a base64 string. You can do that with these two simple PowerShell commands:
$fileContentBytes = get-content 'certificate.pfx' -Encoding Byte
[System.Convert]::ToBase64String($fileContentBytes) | Out-File 'certificate_base64.txt'
Create a secret in Azure Key Vault via the Azure Portal. Copy the certificate base64 string that you created previously and paste it in the secret value field in your Azure Key Vault via the Azure Portal. Then simply call the Azure Key Vault from the code to get the base64 string value and convert that to a X509Certificate2:
private async Task<X509Certificate2> GetCertificateAsync()
{
var azureServiceTokenProvider = new AzureServiceTokenProvider();
var keyVaultClient = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(azureServiceTokenProvider.KeyVaultTokenCallback));
var secret = await keyVaultClient.GetSecretAsync("https://path/to/key/vault").ConfigureAwait(false);
var pfxBase64 = secret.Value;
var bytes = Convert.FromBase64String(pfxBase64);
var coll = new X509Certificate2Collection();
coll.Import(bytes, "certificatePassword", X509KeyStorageFlags.Exportable);
return coll[0];
}

See the following answer, which describes how to do this using the latest .NET Azure SDK client libraries:
How can I create an X509Certificate2 object from an Azure Key Vault KeyBundle

Here is the script for uploading pfx certificate in python using azure cli
azure keyvault secret set --vault-name <Valut name> --secret-name <Secret Name> --value <Content of PFX file>
Getting the content of PFX file in python
fh = open(self.getPfxFilePath(), 'rb')
try:
ba = bytearray(fh.read())
cert_base64_str = base64.b64encode(ba)
password = self.getPassword()
json_blob = {
'data': cert_base64_str,
'dataType': 'pfx',
'password': password
}
blob_data= json.dumps(json_blob)
content_bytes= bytearray(blob_data)
content = base64.b64encode(content_bytes)
return content
finally:
fh.close
fh.close()

PHP validate client signature using client public key

Hi I'm building REST service and planning to erase language restriction for developing both client and server from this project, the REST server currently written using PHP.
For security I'm validating their hash code, and also their client app signature. Currently the hash is successful but for the signature it always failed.
Every client will have their own private key and on the server we will have their public key to verify the signature, each request to server will send a signature.
If the client is a web server there will be only one private key for all user. If the client is a native app ( C# ) then each installed client app will have their unique private key.
PHP client to PHP REST server -> calculate hash ok, verify client signature ok
C# client (winapp) to PHP REST Server -> calculate hash ok, verify client signature failed
In the future I want to try with JAVA and vice versa.
I use easy way to create a self sign certificate
openssl genrsa -des3 -out netclient.key 2048
openssl req -new -x509 -key netclient.key -out netclient.crt
openssl pkcs12 -export -inkey netclient.key -in netclient.crt -out netclient.p12
Here is my code to sign data using C#
public static string (string Base64EncryptedData ) {
X509Certificate2 my;
my = new X509Certificate2("cert/netclient.p12", "abcdefg", X509KeyStorageFlags.Exportable | X509KeyStorageFlags.PersistKeySet);
RSACryptoServiceProvider csp = null;
csp = (RSACryptoServiceProvider)my.PrivateKey;
SHA1Managed sha1 = new SHA1Managed();
UnicodeEncoding encoding = new UnicodeEncoding();
byte[] data = encoding.GetBytes(Base64EncryptedData);
byte[] hash = sha1.ComputeHash(data);
return Convert.ToBase64String(csp.SignHash(hash, CryptoConfig.MapNameToOID("SHA1")));
}
It return a signature (base64) which I send to PHP REST server using GET method.
On PHP Server I recount the hash and verify but it never return 1. Currently I copy manually all the key to the server.
set_include_path(get_include_path() . PATH_SEPARATOR ."lib");
include("Crypt/RSA.php");
$p12cert = array();
$fp=fopen("cert/netclient.p12","r");
$priv_key=fread($fp,8192);
fclose($fp);
openssl_pkcs12_read($priv_key, $p12cert, "abcdefg");
//try using the phpseclib library from http://phpseclib.sourceforge.net
//req() will get all the $_POST, $_GET data req("signature") = $_GET["signature"];
//$hash_request is an base64_encode string same exact value as the one in C#
$rsa = new Crypt_RSA();
$rsa->setHash("sha1");
$rsa->setSignatureMode(CRYPT_RSA_SIGNATURE_PKCS1);
$rsa->loadKey($p12cert["cert"]);
$verified_phpseclib = $rsa->verify($hash_request, base64_decode(req("signature"))) ? "verified" : "diff";
//try using php built in openssl
$verified_builtin = openssl_verify($hash_request, base64_decode(req('signature')), $p12cert["cert");
Non of the result return "verified" or "1"
Could it be because when C# sign the data, the format of the data is byte[] ?
Or is there other reason ?
I'm running all the code in linux
php - using apache2
c# - using monodevelop
Here is my pk12 array after reading using openssl_pkcs12_read you can download it here
Array
(
[cert] => -----BEGIN CERTIFICATE-----
MIIDXTCCAkWgAwIBAgIJANjuyBraeAN0MA0GCSqGSIb3DQEBBQUAMEUxCzAJBgNV
BAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBX
aWRnaXRzIFB0eSBMdGQwHhcNMTMwMzE5MDA0ODI0WhcNMTMwNDE4MDA0ODI0WjBF
MQswCQYDVQQGEwJBVTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50
ZXJuZXQgV2lkZ2l0cyBQdHkgTHRkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB
CgKCAQEAojE1Md/DET9mvKpoNJvb+mbGdTfSYuXQurGxb5pU0ba+gVeARbc8fjaK
D/2EKhUP/7240a31e5qI/2UlgiHO+ghQKDPo+5W5CetDVR1oMbWVA1fp7/gv8DEV
AvNV9nVkYzxnoVPs+W0MhMlWPrhCNSqQ28BbbL0EzXwXESnMw0I2VAhGWDNG0zSl
U03l5zh8jzzRKKK+gw4gIKo0u4lEUqB52MaXPzdwQkOEXRUdT+l1vK3DGfQycMrm
gZEN6IXDvpSFcQfHJlICK06+JNgDM0tYbif2mMfDEPKeFIjGC8S2ZDTZlR92LCk1
/WUGhdDFobbLAOx40M/etOcaEN1bgQIDAQABo1AwTjAdBgNVHQ4EFgQUTr0r49JA
DUKKiIQkRVlog4T952AwHwYDVR0jBBgwFoAUTr0r49JADUKKiIQkRVlog4T952Aw
DAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQUFAAOCAQEAO2IwjeIEVy8aLCOTjCkt
fTyLYuoiex+Xa/Dy2Xdl51Q44uAjCeSVsm1mXko+cZSypngGDu8W97sifzeR+pmW
7r6r+Rj+5pZcQiLQOCUQrXVpqHeiCS8QpOhpxGF+TOLy1O9zcqQwEiohKmzJB++d
1NM8//OBx1/Nyu7eLTKrTqfBdhuXfee963HVF9uBxwV8oa3UTQfhKu2vwJ64pcbU
5OVQw4rfhfSYhB1uu+zMQhCZ3hsroj2rC6Kfb8t3BNzngQwiqDBprmJznmJY38pW
kw0NIx+GrO1NUu+Ydl/7MXChujQajqSkG65Z9iJ8G7eegrY988INPVIfitIopWF+
jg==
-----END CERTIFICATE-----
[pkey] => -----BEGIN PRIVATE KEY-----
MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQCiMTUx38MRP2a8
qmg0m9v6ZsZ1N9Ji5dC6sbFvmlTRtr6BV4BFtzx+NooP/YQqFQ//vbjRrfV7moj/
ZSWCIc76CFAoM+j7lbkJ60NVHWgxtZUDV+nv+C/wMRUC81X2dWRjPGehU+z5bQyE
yVY+uEI1KpDbwFtsvQTNfBcRKczDQjZUCEZYM0bTNKVTTeXnOHyPPNEoor6DDiAg
qjS7iURSoHnYxpc/N3BCQ4RdFR1P6XW8rcMZ9DJwyuaBkQ3ohcO+lIVxB8cmUgIr
Tr4k2AMzS1huJ/aYx8MQ8p4UiMYLxLZkNNmVH3YsKTX9ZQaF0MWhtssA7HjQz960
5xoQ3VuBAgMBAAECggEAB/TgBi1S9XKlyJWXfRU0SmlmTPPLF1zsy2vSJ4ZrqMoN
OG0hdsoRZqOoTDaEmEfmPAaDnY2qIEEpfVXp7CNacvubaw143Xav2CO5buB9bwrY
X4ydhk8nkuHlhPqI+gkyPogFEW37jxTha1YxK+yAGvmWl6EtGv1+0dHHk+j4CZAO
9DH1gZRBOXyeAJAhYuBMxxKfZhBNgGHzLtf9SGoOd2v9ya/h4CuqYh/CoxHGM/w1
uE6M2CKxSuZtqdtiJnoYcmAnDLACDmVHQNbFMEyC3ALrixgTj1p9aYPi8P/9BVKv
+T6cCmRQIl4VY4Ir0PCbai6BQaP6TIbAo0lt9cefUQKBgQDV2KKuc/TsZgrl+ijV
vW1kf46X1Squ8AqPgJnXYM5M4kG+17AecV25sm1oHmzN0ChwvWYotfhLMSqdH+1l
5zMYMwjTle9BkQXELyAvKiKND5I42DtTE04YnHi7HDlaJmOCa3WTTPjJiyIZC+9S
Lpc3gDwP8VSBcq2V2t/SMw9+nQKBgQDCKfIUJWW0WHkESmc8rJDBUWueUVfEWh/d
9ZG/gIapSXnJcTp/fRDZsaqS0pI7c8bvdPJPkQKO0M7Sy2E/OjyKOT5X4+4FguCd
XlwaUhVRycMxGjb1Q4pTlPCnQkoBVKQ5oW4IVQW/ILtT6wuJQPmqkwiXN6CSHzmF
w1RW+pUpNQKBgDXEeHLgmO5vYcIdOfMz47NnFxU59bdyh1U5gnTS1Ewkf19an9+n
pWcxY6zQKY8+DUz7cho+VqWhQROsmWYL0Z7+BfQdOMEFk6uWJcN2FqXdCmjchV4H
9pTdksWI/SqbiF2cYz2cFtml7/bYN140dLTxuyhPB25cxSRumeQiDn1JAoGBAJDf
w2UM0mpSaVmuOoGnMQtNuUMT5qz3ojd3eByvxcqirGCGP+PIab5FNsT+oWYC6Tja
xcJgrMvrOadHYXRP+8QXGlFyHLO4B+jj800gWhAAv8fvi3pNvvTGeRoT+Cwt/6uQ
rA1Dg1otDhl7k8wB00hXFV3ff8wHyF/qcw/DQXDRAoGBAJGvJSl9yIr5v3ooZ6BV
FtDcDTPFGxQCyqvfi5RzKW65Wqu1ZL7jSiOPcY4+418Zifnfyfr/9IZjl9EKPjhC
/fNqptnuA7DKPDKkQIeQSwtHF9HGVoqG0JGOmYCeCZxQK6eZhiuU8VohAAbzsdu8
PgxyS/PRbCsXP4p+OjR1i+3C
-----END PRIVATE KEY-----
)

Try this:
<?php
set_include_path(get_include_path() . PATH_SEPARATOR ."lib");
include('File/X509.php');
$p12cert = array();
$fp=fopen("cert/netclient.p12","r");
$priv_key=fread($fp,8192);
fclose($fp);
openssl_pkcs12_read($priv_key, $p12cert, "abcdefg");
$x509 = new File_X509();
$x509->loadX509($p12cert["cert"]);
$pubkey = $x509->getPublicKey();
$pubkey->setSignatureMode(CRYPT_RSA_SIGNATURE_PKCS1);
$verified_phpseclib = $pubkey->verify($hash_request, base64_decode(req("signature"))) ? "verified" : "diff";