Is there any example in C# to see how to pre-sign all objects using a start-with policy with AWS v4 signature to let customers download object from their respective folder structure instead signing each document separately.
The documentation says :
https://s3.amazonaws.com/examplebucket/test.txt
?X-Amz-Algorithm=AWS4-HMAC-SHA256
&X-Amz-Credential=<your-access-key-id>/20130721/us-east-1/s3/aws4_request
&X-Amz-Date=20130721T201207Z
&X-Amz-Expires=86400
&X-Amz-SignedHeaders=host
&X-Amz-Signature=<signature-value>
But my signature is not working for GET (download) object, while working correctly for upload
void Main()
{
string bucket = "bucket-name-here";
string s3Key = "s3-key-here";
string s3Secret = "secret-here";
string s3Region = "us-east-1";
string Date = DateTime.UtcNow.ToString("yyyyMMdd");
string xAmzDate = DateTime.UtcNow.ToString("yyyyMMdd") + "T000000Z";
string expiration = DateTime.UtcNow.AddDays(1).ToString("yyyy-MM-ddTHH:mm:ssK");
string policyString = $#"{{""expiration"":""{expiration}"",""conditions"":[{{""bucket"":""{bucket}""}},{{""acl"":""private""}},[""starts-with"",""$key"",""Client_1""],[""starts-with"",""$Content-Type"",""""],[""starts-with"",""$filename"",""""],{{""x-amz-date"":""{xAmzDate}""}},{{""x-amz-credential"":""{s3Key}/{Date}/us-east-1/s3/aws4_request""}},{{""x-amz-algorithm"":""AWS4-HMAC-SHA256""}}]}}";
var policyStringBytes = Encoding.UTF8.GetBytes(policyString);
var policy = Convert.ToBase64String(policyStringBytes);
//policy.Dump();
byte[] signingKey = GetSigningKey(s3Secret, Date, s3Region, "s3");
byte[] signature = HmacSHA256(policy, signingKey);
var sign = ToHexString(signature);
sign.Dump();
}
static byte[] HmacSHA256(String data, byte[] key)
{
String algorithm = "HmacSHA256";
KeyedHashAlgorithm kha = KeyedHashAlgorithm.Create(algorithm);
kha.Key = key;
return kha.ComputeHash(Encoding.UTF8.GetBytes(data));
}
private byte[] GetSigningKey(String key, String dateStamp, String regionName, String serviceName)
{
byte[] kSecret = Encoding.UTF8.GetBytes(("AWS4" + key).ToCharArray());
byte[] kDate = HmacSHA256(dateStamp, kSecret);
byte[] kRegion = HmacSHA256(regionName, kDate);
byte[] kService = HmacSHA256(serviceName, kRegion);
byte[] kSigning = HmacSHA256("aws4_request", kService);
return kSigning;
}
public static string ToHexString(byte[] data)
{
StringBuilder sb = new StringBuilder();
for (int i = 0; i < data.Length; i++)
{
sb.Append(data[i].ToString("x2", CultureInfo.InvariantCulture));
}
return sb.ToString();
}
More about the problem: We have thousands of documents for hundreds of clients on S3 on their respective folder structure as below. Right now, every time the client is looking to download their object they gets signed by our API to created the downloadable link > so each document is signed separately.
Client 1
Client_1/Document1.xyz
Client_1/Document2.xyz
Client 2
Client_2/Document1.xyz
Client_2/Document2.xyz
The signing algorithm for S3 HTML form POST uploads allows you to sign a policy document with constraints like ["starts-with","$key",...] but pre-signed URLs for S3 don't support this. With a pre-signed URL, you sign not a policy document but a "canonical request," which is a canonicalized representation of the browser's exact request. So there is no support for wildcards or prefixes.
There are two alternatives that come to mind.
CloudFront signed URLs and signed cookies do support a policy document when you use a "custom policy" (not a "canned policy," which is more like what S3 supports) and a custom policy allows a * in the URL, similar to ["starts-with","$key",...] but using the URL the browser will be requesting. You only have to do the signing once, and the code running in the browser can reuse that policy and signature. On the back-side of CloudFront, a CloudFront Origin Access Identity is used to sign the requests as they are actually sent to the bucket, after authenticating the request on the front-side of CloudFront using the CloudFront signed URL or signed cookies. (With signed cookies, the browser just makes the request, and automatically submits the cookies, so that works the same but with no browser manipulation of the URLs.
Alternately, the AssumeRole action in Security Token Service could be called by your server, to generate a set of temporary credentials for the client to use, to sign its own individual URLs.
When calling AssumeRole, you can also pass an optional session policy document. If you do, then the generated temporary credentials can only perform actions allowed by both the role policy ("allow read from the bucket") and the session policy ("allow read from the bucket for keys beginning with a specific prefix"). So the role credentials obtained would only allow the user to access to their objects.
Related
I am creating an open Source Project which uses the youtube pubsub api to get notifications on video uploads. I want to verify that the request does come from youtube and not from a 3rd Party by checking the HMAC SHA1 Signature as described.
So, every time 1 Run my Program, I will generate a Secret later, to debug the problem, i use "test" as my secret string.
I use the following method to check if the provided signature is valid
public static bool Check(string body, string signature)
{
using (var hmac = new HMACSHA1(Encoding.UTF8.GetBytes(Secret)))
{
var hashBytes = hmac.ComputeHash(Encoding.UTF8.GetBytes(body));
var hash = Encoding.UTF8.GetString(hashBytes);
Console.WriteLine("Computed Hash " + hash);
return signature.Equals(hash);
}
}
Where body is the request body and signature is a value provided in the request header.
According to https://pubsubhubbub.github.io/PubSubHubbub/pubsubhubbub-core-0.4.html#rfc.section.7
If the subscriber supplied a value for hub.secret in their subscription request, the hub MUST generate an HMAC signature of the payload and include that signature in the request headers of the content distribution request. The X-Hub-Signature header's value MUST be in the form sha1=signature where signature is a 40-byte, hexadecimal representation of a SHA1 signature [RFC3174]. The signature MUST be computed using the HMAC algorithm [RFC2104] with the request body as the data and the hub.secret as the key.
I supply my Secret as hub.secret in my subscription request.
So if I understand it correctly, the hub SHOULD use that secret to generate a HMACSHA1 of the payload -> the body.
I want to regenerate the HMAC and should get the same value, right?
It does not work. Also the computed hash value logged by console.WriteLine is something completely different, not alphabetic characters at all, so I guess it might be a problem with the encoding, but I can't figure it out. Thanks for all the help!
The documentation says "where signature is a 40-byte, hexadecimal representation" so instead of converting hashBytes to an UTF8 string you should convert it to a hexadecimal string.
public static bool Check(string body, string signature)
{
using (var hmac = new HMACSHA1(Encoding.UTF8.GetBytes(Secret)))
{
var hashBytes = hmac.ComputeHash(Encoding.UTF8.GetBytes(body));
var hash = Convert.ToHexString(hashBytes).ToLowerInvariant();
Console.WriteLine("Computed Hash " + hash);
return signature.Equals(hash);
}
}
I am trying to call Amazon Selling Partner API Token API to get RDT token. I have working program that is making successful GET requests to the Orders API but REST requests to the Orders API contain the headers and URL parameters only. Token API requests contain (additionally) HTTP body as well in which the list of Restricted Resources is saved.
Amazon replys with
The request signature we calculated does not match the signature you provided.
Check your AWS Secret Access Key and signing method.
Consult the service documentation for details.
The Canonical String for this request should have been
'GET
/tokens/2021-03-01/restrictedDataToken
...
host;x-amz-access-token;x-amz-date\n
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
...
The String-to-Sign should have been
...
...
in my Token API program. e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 is the key to my problem. This is Hashed paylod (see the picture in the coming link) that is computers from the initial set of headers, URL parameters and body content. The Hashed payload which is calculated by my program is the same that Amazon returns and accepts for the requests without body. But Amazon's and my program's Hashed payloads are different for the requests with body. So, I have concluded that my program adds and hashes body differently from the Amazon and that is why Amazon is not accepty my Token API requests.
AWS - What a canonical request is really? has nice picture, how the complete request to the Amazon services is computed.
I am computing and adding HTTP body with the C# code:
Model.RestrictedResource restrictedResource = new Model.RestrictedResource(
Model.RestrictedResource.MethodEnum.GET,
"/orders/v0/orders",
new List<String> {"buyerInfo", "shippingAddress"});
List<Model.RestrictedResource> restrictedResources = new List<Model.RestrictedResource>();
restrictedResources.Add(restrictedResource);
Model.CreateRestrictedDataTokenRequest request = new Model.CreateRestrictedDataTokenRequest("", restrictedResources);
IRestRequest restRequest = new RestRequest(rdt_resource, Method.GET);
restRequest.AddParameter("MarketplaceIds", marketplace_id, ParameterType.QueryString);
restRequest.AddJsonBody(Serialize(request));
//restRequest.AddBody(Serialize(request)); //Alternative solution, the same Amazon error about differing hashed payload values
Here is the code for the calculation of the hashed CanonicalRequest string:
public IRestRequest Sign(IRestRequest request, string host)
{
DateTime signingDate = AwsSignerHelper.InitializeHeaders(request, host);
string signedHeaders = AwsSignerHelper.ExtractSignedHeaders(request);
string hashedCanonicalRequest = CreateCanonicalRequest(request, signedHeaders);
string stringToSign = AwsSignerHelper.BuildStringToSign(signingDate,
hashedCanonicalRequest,
awsCredentials.Region);
string signature = AwsSignerHelper.CalculateSignature(stringToSign,
signingDate,
awsCredentials.SecretKey,
awsCredentials.Region);
AwsSignerHelper.AddSignature(request,
awsCredentials.AccessKeyId,
signedHeaders,
signature,
awsCredentials.Region,
signingDate);
return request;
}
private string CreateCanonicalRequest(IRestRequest restRequest, string signedHeaders)
{
var canonicalizedRequest = new StringBuilder();
//Request Method
canonicalizedRequest.AppendFormat("{0}\n", restRequest.Method);
//CanonicalURI
canonicalizedRequest.AppendFormat("{0}\n", AwsSignerHelper.ExtractCanonicalURIParameters(restRequest.Resource));
//CanonicalQueryString
canonicalizedRequest.AppendFormat("{0}\n", AwsSignerHelper.ExtractCanonicalQueryString(restRequest));
//CanonicalHeaders
canonicalizedRequest.AppendFormat("{0}\n", AwsSignerHelper.ExtractCanonicalHeaders(restRequest));
//SignedHeaders
canonicalizedRequest.AppendFormat("{0}\n", signedHeaders);
// Hash(digest) the payload in the body
canonicalizedRequest.AppendFormat(AwsSignerHelper.HashRequestBody(restRequest));
string canonicalRequest = canonicalizedRequest.ToString();
//Create a digest(hash) of the canonical request
return Utils.ToHex(Utils.Hash(canonicalRequest));
}
//This seems to be the key code - how body is extracted and hashed
public virtual string HashRequestBody(IRestRequest request)
{
Parameter body = request.Parameters.FirstOrDefault(parameter => ParameterType.RequestBody.Equals(parameter.Type));
string value = body != null ? body.Value.ToString() : string.Empty;
return Utils.ToHex(Utils.Hash(value));
}
So, I am missing and I am not following Amazon convention somewhere, but where. There are multiple points in which something can go bad: 1) maybe restRequest.AddJsonBody(Serialize(request)); should be doen differently; 2) maybe HashRequestBody should be implemented differently (I have found the code somewhere in the Internet or generated by Swagger from Selling Partner API specifications.
How to tweak the code to generate the Hashed payload (for the request with the body) that is exactly the same as calculated by Amazon for the same request (i.e. part of the request that is used for the computation and that is canonicalized before the computing of hashed payload).
I managed to solve. One should use the code in the question together with these modifications:
It may be important to set TargetApplication:
Model.CreateRestrictedDataTokenRequest request = new Model.CreateRestrictedDataTokenRequest("", restrictedResources);
request.TargetApplication = "MyApplication";
And the different type code should be used for serialization and adding the body. And it is ultimately important to use POST request, not GET request:
IRestRequest restRequest = new RestRequest(rdt_resource, Method.POST);
restRequest.AddParameter("MarketplaceIds", marketplace_id, ParameterType.QueryString);
String postBody = SerializeBody(request);
restRequest.AddParameter("application/json", postBody, ParameterType.RequestBody);
...
...
public String SerializeBody(object obj)
{
try
{
return obj != null ? JsonConvert.SerializeObject(obj) : null;
}
catch (Exception e)
{
throw new Exception(e.Message);
}
}
No I am getting the error message:
Application does not have access to one or more requested data elements: [buyerInfo, shippingAddress]
But it is fine, because technically the registered user and application is recognized and the request and their signatures are well formed.
I have created a WCF service where the service encrypts an incoming string from client application. How do i return the public key of server to the client so that it can decrypt the encrypted string returned from the WCF service?
Is there any other way of achieving?
You can use JSON Web Token (JWT).
JWTs generally have three parts: a header, a payload, and a signature.
header = '{"alg":"HS256","typ":"JWT"}'
HS256 indicates that this token is signed using HMAC-SHA256
The payload contains the claims that we wish to make:
payload = '{"loggedInAs":"admin","iat":1422779638}'
The signature is calculated by base64url encoding the header and payload and concatenating them with a period as a separator:
key = 'secretkey'
unsignedToken = encodeBase64(header) + '.' + encodeBase64(payload)
signature = HMAC-SHA256(key, unsignedToken)
Now token look like:
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJsb2dnZWRJbkFzIjoiYWRtaW4iLCJpYXQiOjE0MjI3Nzk2Mzh9.gzSraSYS8EXBxLN_oWnFSRgCzcmJmMjLiuyu5CSpyHI
In my WCF application i use JWT .dll for generate tokens like this.
public string GenToken()
{
byte[] key = new byte[];
var payload= new Dictionary<string, object>()
{
{ "idUsr", 1 },
{ "nameUsr", admin},
{"accessTime",DateTime.UtcNow.AddMinutes(30)}
};
string token = JWT.JsonWebToken.Encode(payload, key , JWT.JwtHashAlgorithm.HS256);
return token;
}
I have a Facebook Page Tab app and I'm trying to find out where visitors to the page tab are coming from. I've read on http://developers.facebook.com/docs/authentication/signed_request/ that you can get these from app_data in the signed request but whenever I try getting the signed request app_data isn't there.
I used FB.getLoginStatus to get the signed request when inside the tab on Facebook, but
When I debug the signed request with http://developers.facebook.com/tools/echo I get the error "Bad Signature"
Your signed_request was probably not signed with our app_id of xxxxx Here is the payload:
{
"algorithm": "HMAC-SHA256",
"code": "xxxx",
"issued_at": xxxx,
"user_id": "xxxx2"
}
I'm using the C# SDK with Javascript
You can decode the signed request with the code in this topic:
Decode Signed Request Without Authentication
if (Request.Params["signed_request"] != null)
{
string payload = Request.Params["signed_request"].Split('.')[1];
var encoding = new UTF8Encoding();
var decodedJson = payload.Replace("=", string.Empty).Replace('-', '+').Replace('_', '/');
var base64JsonArray = Convert.FromBase64String(decodedJson.PadRight(decodedJson.Length + (4 - decodedJson.Length % 4) % 4, '='));
var json = encoding.GetString(base64JsonArray);
var o = JObject.Parse(json);
var lPid = Convert.ToString(o.SelectToken("page.id")).Replace("\"", "");
var lLiked = Convert.ToString(o.SelectToken("page.liked")).Replace("\"", "");
var lUserId= Convert.ToString(o.SelectToken("user_id")).Replace("\"", "");
}
It should be easy to get the app_data by adding
var lAppData = Convert.ToString(o.SelectToken("app_data")).Replace("\"", "");
To the have the app_data for your tab app you need to add it to the redirect url when acquiring permissions. You redirect url should something like:
http://facebook.com/YOUR_PAGE?sk=app_YOUR_APP_ID&app_data=add,whatever,parameters,you,want,here
I can only guess that the reason you got this error is because you just pasted your signed request in the address bar instead of the one used by the echo tool. The error is because your signed request is signed by your app_id and you're trying to use it with echo which has another app_id. But that's just a guess :)
My primary language is PHP but hope I was able to help :)
I am developing an application that will upload files to Amazon. Amazon provides
a method WithServerSideEncryptionMethod(ServerSideEncryptionMethod.AES256) to encrypt files but it is not working. It is saving text as a plain text.
public static void UploadFile()
{
new Program();
var key = "a";
//key = ReplaceDblSlashToSingleFwdSlash(key);
//path = ReplaceFwdSlashToBackSlash(path);
var request = new PutObjectRequest();
request.WithBucketName("demo")
.WithContentBody("i am achal kumar")
.WithKey(key)
.WithServerSideEncryptionMethod(ServerSideEncryptionMethod.AES256);
//request.PutObjectProgressEvent += displayFileProgress;
S3Response response = s3Client.PutObject(request);
response.Dispose();
}
Your data is likely encrypted and just being automatically decrypted with your get that you are testing with.
http://aws.typepad.com/aws/2011/10/new-amazon-s3-server-side-encryption.html
Decryption of the encrypted data requires no effort on your part. When
you GET an encrypted object, we fetch and decrypt the key, and then
use it to decrypt your data. We also include an extra header in the
response to the GET to let you know that the data was stored in
encrypted form in Amazon S3.
you can use the following code to check if the is encrypted or not .. because aws s3 they already decrypt the object when they return it to you.
so try the following code to check if the object is encrypted on amazon s3
GetObjectMetadataRequest meta = new GetObjectMetadataRequest();
GetObjectMetadataResponse response = s3Client.GetObjectMetadata(meta);
if(response.ServerSideEncryptionMethod = ServerSideEncryptionMethod.AES256)
{
// your code goes here
}
i hope this could help