Related
If I get a JWT and I can decode the payload, how is that secure? Couldn't I just grab the token out of the header, decode and change the user information in the payload, and send it back with the same correct encoded secret?
I know they must be secure, but I just would really like to understand the technologies. What am I missing?
JWTs can be either signed, encrypted or both. If a token is signed, but not encrypted, everyone can read its contents, but when you don't know the private key, you can't change it. Otherwise, the receiver will notice that the signature won't match anymore.
Answer to your comment: I'm not sure if I understand your comment the right way. Just to be sure: do you know and understand digital signatures? I'll just briefly explain one variant (HMAC, which is symmetrical, but there are many others).
Let's assume Alice wants to send a JWT to Bob. They both know some shared secret. Mallory doesn't know that secret, but wants to interfere and change the JWT. To prevent that, Alice calculates Hash(payload + secret) and appends this as signature.
When receiving the message, Bob can also calculate Hash(payload + secret) to check whether the signature matches.
If however, Mallory changes something in the content, she isn't able to calculate the matching signature (which would be Hash(newContent + secret)). She doesn't know the secret and has no way of finding it out.
This means if she changes something, the signature won't match anymore, and Bob will simply not accept the JWT anymore.
Let's suppose, I send another person the message {"id":1} and sign it with Hash(content + secret). (+ is just concatenation here). I use the SHA256 Hash function, and the signature I get is: 330e7b0775561c6e95797d4dd306a150046e239986f0a1373230fda0235bda8c. Now it's your turn: play the role of Mallory and try to sign the message {"id":2}. You can't because you don't know which secret I used. If I suppose that the recipient knows the secret, he CAN calculate the signature of any message and check if it's correct.
You can go to jwt.io, paste your token and read the contents. This is jarring for a lot of people initially.
The short answer is that JWT doesn't concern itself with encryption. It cares about validation. That is to say, it can always get the answer for "Have the contents of this token been manipulated"? This means user manipulation of the JWT token is futile because the server will know and disregard the token. The server adds a signature based on the payload when issuing a token to the client. Later on it verifies the payload and matching signature.
The logical question is what is the motivation for not concerning itself with encrypted contents?
The simplest reason is because it assumes this is a solved problem for the most part. If dealing with a client like the web browser for example, you can store the JWT tokens in a cookie that is secure (is not transmitted via HTTP, only via HTTPS) and httpOnly (can't be read by Javascript) and talks to the server over an encrypted channel (HTTPS). Once you know you have a secure channel between the server and client you can securely exchange JWT or whatever else you want.
This keeps thing simple. A simple implementation makes adoption easier but it also lets each layer do what it does best (let HTTPS handle encryption).
JWT isn't meant to store sensitive data. Once the server receives the JWT token and validates it, it is free to lookup the user ID in its own database for additional information for that user (like permissions, postal address, etc). This keeps JWT small in size and avoids inadvertent information leakage because everyone knows not to keep sensitive data in JWT.
It's not too different from how cookies themselves work. Cookies often contain unencrypted payloads. If you are using HTTPS then everything is good. If you aren't then it's advisable to encrypt sensitive cookies themselves. Not doing so will mean that a man-in-the-middle attack is possible--a proxy server or ISP reads the cookies and then replays them later on pretending to be you. For similar reasons, JWT should always be exchanged over a secure layer like HTTPS.
Let's discuss from the very beginning:
JWT is a very modern, simple and secure approach which extends for Json Web Tokens. Json Web Tokens are a stateless solution for authentication. So there is no need to store any session state on the server, which of course is perfect for restful APIs.
Restful APIs should always be stateless, and the most widely used alternative to authentication with JWTs is to just store the user's log-in state on the server using sessions. But then of course does not follow the principle that says that restful APIs should be stateless and that's why solutions like JWT became popular and effective.
So now let's know how authentication actually works with Json Web Tokens. Assuming we already have a registered user in our database. So the user's client starts by making a post request with the username and the password, the application then checks if the user exists and if the password is correct, then the application will generate a unique Json Web Token for only that user.
The token is created using a secret string that is stored on a server. Next, the server then sends that JWT back to the client which will store it either in a cookie or in local storage.
Just like this, the user is authenticated and basically logged into our application without leaving any state on the server.
So the server does in fact not know which user is actually logged in, but of course, the user knows that he's logged in because he has a valid Json Web Token which is a bit like a passport to access protected parts of the application.
So again, just to make sure you got the idea. A user is logged in as soon as he gets back his unique valid Json Web Token which is not saved anywhere on the server. And so this process is therefore completely stateless.
Then, each time a user wants to access a protected route like his user profile data, for example. He sends his Json Web Token along with a request, so it's a bit like showing his passport to get access to that route.
Once the request hits the server, our app will then verify if the Json Web Token is actually valid and if the user is really who he says he is, well then the requested data will be sent to the client and if not, then there will be an error telling the user that he's not allowed to access that resource.
All this communication must happen over https, so secure encrypted Http in order to prevent that anyone can get access to passwords or Json Web Tokens. Only then we have a really secure system.
So a Json Web Token looks like left part of this screenshot which was taken from the JWT debugger at jwt.io. So essentially, it's an encoding string made up of three parts. The header, the payload and the signature Now the header is just some metadata about the token itself and the payload is the data that we can encode into the token, any data really that we want. So the more data we want to encode here the bigger the JWT. Anyway, these two parts are just plain text that will get encoded, but not encrypted.
So anyone will be able to decode them and to read them, we cannot store any sensitive data in here. But that's not a problem at all because in the third part, so in the signature, is where things really get interesting. The signature is created using the header, the payload, and the secret that is saved on the server.
And this whole process is then called signing the Json Web Token. The signing algorithm takes the header, the payload, and the secret to create a unique signature. So only this data plus the secret can create this signature, all right?
Then together with the header and the payload, these signature forms the JWT,
which then gets sent to the client.
Once the server receives a JWT to grant access to a protected route, it needs to verify it in order to determine if the user really is who he claims to be. In other words, it will verify if no one changed the header and the payload data of the token. So again, this verification step will check if no third party actually altered either the header or the payload of the Json Web Token.
So, how does this verification actually work? Well, it is actually quite straightforward. Once the JWT is received, the verification will take its header and payload, and together with the secret that is still saved on the server, basically create a test signature.
But the original signature that was generated when the JWT was first created is still in the token, right? And that's the key to this verification. Because now all we have to do is to compare the test signature with the original signature.
And if the test signature is the same as the original signature, then it means that the payload and the header have not been modified.
Because if they had been modified, then the test signature would have to be different. Therefore in this case where there has been no alteration of the data, we can then authenticate the user. And of course, if the two signatures
are actually different, well, then it means that someone tampered with the data.
Usually by trying to change the payload. But that third party manipulating the payload does of course not have access to the secret, so they cannot sign the JWT.
So the original signature will never correspond to the manipulated data.
And therefore, the verification will always fail in this case. And that's the key to making this whole system work. It's the magic that makes JWT so simple,
but also extremely powerful.
The contents in a json web token (JWT) are not inherently secure, but there is a built-in feature for verifying token authenticity. A JWT is three hashes separated by periods. The third is the signature. In a public/private key system, the issuer signs the token signature with a private key which can only be verified by its corresponding public key.
It is important to understand the distinction between issuer and verifier. The recipient of the token is responsible for verifying it.
There are two critical steps in using JWT securely in a web application: 1) send them over an encrypted channel, and 2) verify the signature immediately upon receiving it. The asymmetric nature of public key cryptography makes JWT signature verification possible. A public key verifies a JWT was signed by its matching private key. No other combination of keys can do this verification, thus preventing impersonation attempts. Follow these two steps and we can guarantee with mathematical certainty the authenticity of a JWT.
More reading: How does a public key verify a signature?
I would explain this with an example.
Say I borrowed $10 from you, then I gave you an IOU with my signature on it. I will pay you back whenever you or someone else bring this IOU back to me, I will check the signature to make sure that is mine.
I can't make sure you don't show the content of this IOU to anyone or even give it to a third person, all I care is that this IOU is signed by me, when someone shows this IOU to me and ask me to pay it.
The way how JWT works is quite the same, the server can only make sure that the token received was issued by itself.
You need other measures to make it secure, like encryption in transfer with HTTPS, making sure that the local storage storing the token is secured, setting up origins.
Ref - JWT Structure and Security
It is important to note that JWT are used for authorization and not authentication.
So a JWT will be created for you only after you have been authenticated by the server by may be specifying the credentials. Once JWT has been created for all future interactions with server JWT can be used. So JWT tells that server that this user has been authenticated, let him access the particular resource if he has the role.
Information in the payload of the JWT is visible to everyone. There can be a "Man in the Middle" attack and the contents of the JWT can be changed. So we should not pass any sensitive information like passwords in the payload. We can encrypt the payload data if we want to make it more secure. If Payload is tampered with server will recognize it.
So suppose a user has been authenticated and provided with a JWT. Generated JWT has a claim specifying role of Admin. Also the Signature is generated with
This JWT is now tampered with and suppose the
role is changed to Super Admin
Then when the server receives this token it will again generate the signature using the secret key(which only the server has) and the payload. It will not match the signature
in the JWT. So the server will know that the JWT has been tampered with.
Only JWT's privateKey, which is on your server will decrypt the encrypted JWT. Those who know the privateKey will be able to decrypt the encrypted JWT.
Hide the privateKey in a secure location in your server and never tell anyone the privateKey.
I am not a cryptography specialist and hence (I hope) my answer can help somebody who is neither.
There are two possible ways of using cryptography in programming:
Signing / verifying
Encryption / decryption
We use Signing when we want to ensure that data comes from a trusted source.
We use Encryption when we want to protect the data.
Signing / verifying uses asymmetrical algorithms i.e. we sign with one key (private) and the data receiver uses the other (public) key to verify.
A symmetric algorithm uses the same key to encrypt and decrypt data.
The encryption can be done using both symmetric and asymmetric algorithms.
relatively simple article on subject
The above is common knowledge below is my opinion.
When JWT is used for simple client-to-server identification there is no need for signing or asymmetric encryption. JWT can be encrypted with AES which is fast and supersecure. If the server can decrypt it, it means the server is the one who encrypted it.
Summary: non-encrypted JWT is not secure. Symmetric encryption can be used instead of signing in case no third party is involved.
If I get a JWT and I can decode the payload, how is that secure? Couldn't I just grab the token out of the header, decode and change the user information in the payload, and send it back with the same correct encoded secret?
I know they must be secure, but I just would really like to understand the technologies. What am I missing?
JWTs can be either signed, encrypted or both. If a token is signed, but not encrypted, everyone can read its contents, but when you don't know the private key, you can't change it. Otherwise, the receiver will notice that the signature won't match anymore.
Answer to your comment: I'm not sure if I understand your comment the right way. Just to be sure: do you know and understand digital signatures? I'll just briefly explain one variant (HMAC, which is symmetrical, but there are many others).
Let's assume Alice wants to send a JWT to Bob. They both know some shared secret. Mallory doesn't know that secret, but wants to interfere and change the JWT. To prevent that, Alice calculates Hash(payload + secret) and appends this as signature.
When receiving the message, Bob can also calculate Hash(payload + secret) to check whether the signature matches.
If however, Mallory changes something in the content, she isn't able to calculate the matching signature (which would be Hash(newContent + secret)). She doesn't know the secret and has no way of finding it out.
This means if she changes something, the signature won't match anymore, and Bob will simply not accept the JWT anymore.
Let's suppose, I send another person the message {"id":1} and sign it with Hash(content + secret). (+ is just concatenation here). I use the SHA256 Hash function, and the signature I get is: 330e7b0775561c6e95797d4dd306a150046e239986f0a1373230fda0235bda8c. Now it's your turn: play the role of Mallory and try to sign the message {"id":2}. You can't because you don't know which secret I used. If I suppose that the recipient knows the secret, he CAN calculate the signature of any message and check if it's correct.
You can go to jwt.io, paste your token and read the contents. This is jarring for a lot of people initially.
The short answer is that JWT doesn't concern itself with encryption. It cares about validation. That is to say, it can always get the answer for "Have the contents of this token been manipulated"? This means user manipulation of the JWT token is futile because the server will know and disregard the token. The server adds a signature based on the payload when issuing a token to the client. Later on it verifies the payload and matching signature.
The logical question is what is the motivation for not concerning itself with encrypted contents?
The simplest reason is because it assumes this is a solved problem for the most part. If dealing with a client like the web browser for example, you can store the JWT tokens in a cookie that is secure (is not transmitted via HTTP, only via HTTPS) and httpOnly (can't be read by Javascript) and talks to the server over an encrypted channel (HTTPS). Once you know you have a secure channel between the server and client you can securely exchange JWT or whatever else you want.
This keeps thing simple. A simple implementation makes adoption easier but it also lets each layer do what it does best (let HTTPS handle encryption).
JWT isn't meant to store sensitive data. Once the server receives the JWT token and validates it, it is free to lookup the user ID in its own database for additional information for that user (like permissions, postal address, etc). This keeps JWT small in size and avoids inadvertent information leakage because everyone knows not to keep sensitive data in JWT.
It's not too different from how cookies themselves work. Cookies often contain unencrypted payloads. If you are using HTTPS then everything is good. If you aren't then it's advisable to encrypt sensitive cookies themselves. Not doing so will mean that a man-in-the-middle attack is possible--a proxy server or ISP reads the cookies and then replays them later on pretending to be you. For similar reasons, JWT should always be exchanged over a secure layer like HTTPS.
Let's discuss from the very beginning:
JWT is a very modern, simple and secure approach which extends for Json Web Tokens. Json Web Tokens are a stateless solution for authentication. So there is no need to store any session state on the server, which of course is perfect for restful APIs.
Restful APIs should always be stateless, and the most widely used alternative to authentication with JWTs is to just store the user's log-in state on the server using sessions. But then of course does not follow the principle that says that restful APIs should be stateless and that's why solutions like JWT became popular and effective.
So now let's know how authentication actually works with Json Web Tokens. Assuming we already have a registered user in our database. So the user's client starts by making a post request with the username and the password, the application then checks if the user exists and if the password is correct, then the application will generate a unique Json Web Token for only that user.
The token is created using a secret string that is stored on a server. Next, the server then sends that JWT back to the client which will store it either in a cookie or in local storage.
Just like this, the user is authenticated and basically logged into our application without leaving any state on the server.
So the server does in fact not know which user is actually logged in, but of course, the user knows that he's logged in because he has a valid Json Web Token which is a bit like a passport to access protected parts of the application.
So again, just to make sure you got the idea. A user is logged in as soon as he gets back his unique valid Json Web Token which is not saved anywhere on the server. And so this process is therefore completely stateless.
Then, each time a user wants to access a protected route like his user profile data, for example. He sends his Json Web Token along with a request, so it's a bit like showing his passport to get access to that route.
Once the request hits the server, our app will then verify if the Json Web Token is actually valid and if the user is really who he says he is, well then the requested data will be sent to the client and if not, then there will be an error telling the user that he's not allowed to access that resource.
All this communication must happen over https, so secure encrypted Http in order to prevent that anyone can get access to passwords or Json Web Tokens. Only then we have a really secure system.
So a Json Web Token looks like left part of this screenshot which was taken from the JWT debugger at jwt.io. So essentially, it's an encoding string made up of three parts. The header, the payload and the signature Now the header is just some metadata about the token itself and the payload is the data that we can encode into the token, any data really that we want. So the more data we want to encode here the bigger the JWT. Anyway, these two parts are just plain text that will get encoded, but not encrypted.
So anyone will be able to decode them and to read them, we cannot store any sensitive data in here. But that's not a problem at all because in the third part, so in the signature, is where things really get interesting. The signature is created using the header, the payload, and the secret that is saved on the server.
And this whole process is then called signing the Json Web Token. The signing algorithm takes the header, the payload, and the secret to create a unique signature. So only this data plus the secret can create this signature, all right?
Then together with the header and the payload, these signature forms the JWT,
which then gets sent to the client.
Once the server receives a JWT to grant access to a protected route, it needs to verify it in order to determine if the user really is who he claims to be. In other words, it will verify if no one changed the header and the payload data of the token. So again, this verification step will check if no third party actually altered either the header or the payload of the Json Web Token.
So, how does this verification actually work? Well, it is actually quite straightforward. Once the JWT is received, the verification will take its header and payload, and together with the secret that is still saved on the server, basically create a test signature.
But the original signature that was generated when the JWT was first created is still in the token, right? And that's the key to this verification. Because now all we have to do is to compare the test signature with the original signature.
And if the test signature is the same as the original signature, then it means that the payload and the header have not been modified.
Because if they had been modified, then the test signature would have to be different. Therefore in this case where there has been no alteration of the data, we can then authenticate the user. And of course, if the two signatures
are actually different, well, then it means that someone tampered with the data.
Usually by trying to change the payload. But that third party manipulating the payload does of course not have access to the secret, so they cannot sign the JWT.
So the original signature will never correspond to the manipulated data.
And therefore, the verification will always fail in this case. And that's the key to making this whole system work. It's the magic that makes JWT so simple,
but also extremely powerful.
The contents in a json web token (JWT) are not inherently secure, but there is a built-in feature for verifying token authenticity. A JWT is three hashes separated by periods. The third is the signature. In a public/private key system, the issuer signs the token signature with a private key which can only be verified by its corresponding public key.
It is important to understand the distinction between issuer and verifier. The recipient of the token is responsible for verifying it.
There are two critical steps in using JWT securely in a web application: 1) send them over an encrypted channel, and 2) verify the signature immediately upon receiving it. The asymmetric nature of public key cryptography makes JWT signature verification possible. A public key verifies a JWT was signed by its matching private key. No other combination of keys can do this verification, thus preventing impersonation attempts. Follow these two steps and we can guarantee with mathematical certainty the authenticity of a JWT.
More reading: How does a public key verify a signature?
I would explain this with an example.
Say I borrowed $10 from you, then I gave you an IOU with my signature on it. I will pay you back whenever you or someone else bring this IOU back to me, I will check the signature to make sure that is mine.
I can't make sure you don't show the content of this IOU to anyone or even give it to a third person, all I care is that this IOU is signed by me, when someone shows this IOU to me and ask me to pay it.
The way how JWT works is quite the same, the server can only make sure that the token received was issued by itself.
You need other measures to make it secure, like encryption in transfer with HTTPS, making sure that the local storage storing the token is secured, setting up origins.
Ref - JWT Structure and Security
It is important to note that JWT are used for authorization and not authentication.
So a JWT will be created for you only after you have been authenticated by the server by may be specifying the credentials. Once JWT has been created for all future interactions with server JWT can be used. So JWT tells that server that this user has been authenticated, let him access the particular resource if he has the role.
Information in the payload of the JWT is visible to everyone. There can be a "Man in the Middle" attack and the contents of the JWT can be changed. So we should not pass any sensitive information like passwords in the payload. We can encrypt the payload data if we want to make it more secure. If Payload is tampered with server will recognize it.
So suppose a user has been authenticated and provided with a JWT. Generated JWT has a claim specifying role of Admin. Also the Signature is generated with
This JWT is now tampered with and suppose the
role is changed to Super Admin
Then when the server receives this token it will again generate the signature using the secret key(which only the server has) and the payload. It will not match the signature
in the JWT. So the server will know that the JWT has been tampered with.
Only JWT's privateKey, which is on your server will decrypt the encrypted JWT. Those who know the privateKey will be able to decrypt the encrypted JWT.
Hide the privateKey in a secure location in your server and never tell anyone the privateKey.
I am not a cryptography specialist and hence (I hope) my answer can help somebody who is neither.
There are two possible ways of using cryptography in programming:
Signing / verifying
Encryption / decryption
We use Signing when we want to ensure that data comes from a trusted source.
We use Encryption when we want to protect the data.
Signing / verifying uses asymmetrical algorithms i.e. we sign with one key (private) and the data receiver uses the other (public) key to verify.
A symmetric algorithm uses the same key to encrypt and decrypt data.
The encryption can be done using both symmetric and asymmetric algorithms.
relatively simple article on subject
The above is common knowledge below is my opinion.
When JWT is used for simple client-to-server identification there is no need for signing or asymmetric encryption. JWT can be encrypted with AES which is fast and supersecure. If the server can decrypt it, it means the server is the one who encrypted it.
Summary: non-encrypted JWT is not secure. Symmetric encryption can be used instead of signing in case no third party is involved.
What I have
I'm making a web service using C#.
In order to authenticate users, they have to send their name plus their encrypted password, in order to check if exists in a database.
Then, If it's found, I create a string token, which is a 10 char string randomly generated in order to send it the next times while the session is alive, avoiding to have to send the original credentials anymore.
What is my problem
Using this approach, my problem appears due to the service lifetime.
It's known that web services are not initialized each time a request arrives, but nor is infinite. So there will be a moment, when it'll be destroyed and initialized again.
At this point, my token list would be erased, as well as all the alive connections with it, as this is its function.
So I'm stuck at this point. I'm not sure about how to proceed, maybe I'm just fooling around and there's a simpler way to authenticate users? Or maybe you've and idea about how to don't loose all these alive sessions without having to write them at a DB.
Thank you in advance
Update:
My goal
I aim to create a personal Web Service, just build for me and some friends. Not inside a company nor anything like this. Not in the same LAN neither.
I want to add a bit of security to this service, so I wanted to add authentication to the WS, mainly in order to avoid people pretending to be another and this kind of stuff. So I created User+Password system.
Then, in order to avoid to send them both in each WS Request, I started to write the "token" approach described above.
Notice that I'm using token word because it's similarity with token systems for these cases, but it's a completely created from 0 system, nothing proffesional, so do not assume anything complex about it if I've not said that.
How my system works (or try to)
User -> Auth (user, pass_encrypted) -> WS -> DB (exist? OK)
WS -> token (randomly generated, 10char string) -> User
After that, at each WS request, User sends the token instead of credentials.
After receiving it, WS looks for the token at a List<structureToken>, so it obtains the user which is doing the call, and (for example) the access level, in order to know if the user has rights to run this call.
Your current problem is that you want same list to be persisted through restarts and not persisted to any physical media at the same time. You have to pick one of the choices and live with it: not persisted - just ignore the fact you can have list in memory and make sure token can be validated by itself, if persisting - pick storage and save you list of random numbers.
Since you are building simple system without actual need to have proven verifiable security you can get some ideas from existing systems (like STS and the way it creates token). Basically STS signs information about user (indeed after validation) and than encrypts it with public key of receiving party. So particular server that supposed to get the token can decrypt it (as it has private key part), everyone else may still use it but have to treat as non-verifiable black box token.
Simplest version of this would be no encryption of information, just basic signing. Proper signing requires private/public pair (so external party can validate signature), but since in your case both parties are the same service - you can just SHA256. To prevent external callers to fake you signature you need to have some private information included in hash to "salt" value before hashing. Random number hardcoded into server code (or read from settings) would be good enough. You may also want to include expiration as part of signed value.
So your "token" could look like:
Name,expiration,Base64 of SHA256 of {Name + expiration + secret value}
Bob-2015-06-30-A23BDEDDC56
Since your server code have "secret value" you can always re-compute hash to verify if it is indeed the correct token.
Notes:
do not use it for any real services. Use an existing authentication and make sure to review all security comments related to proper usage of it.
this approach gives you chance to learn a some other concepts - i.e. key update (when your "secret value" need to change, or in real systems signing/encryption certs).
I want encrypt request content to preventing send raw data through WebAPI requests, so I have to implement tow way encryption.
Which encryption algorithm is good for encrypt high rate of request in asp.net web api and how to implement it?
One easy way, requiring no coding on your part, is to use HTTPS instead of HTTP.
The 'best' way would largely be a matter of opinion, but the standard design i go for is as follows:
The client (the thing making calls) side gets a client cert with private keys.
The client gets the public key of the server website certificate.
The client gets the public key of the server encryption certificate.
The server (the thing receiving calls) gets the client cert public key.
The server gets the private and public key of the website cert (to be bound in IIS).
The server gets the private key of the server encryption certificate.
In your code, on the server you validate the requests with the presence of exact (thumbprint) and valid (your logic or X509Certificate2 Verify, but beware of CRL/OCSP (if you care about performance, that is)).
On the client, you validate the server website certificate (again, thumbprint plus verification).
Then on the client, as last step before sending a payload, you use the public key of the server encryption cert to PKCS7 envelope a message.
On the server, add a ActionFilterAttribute (to be cleaner) or refactor actions to receive strings (more procedural) and then use the private key of the server encryption cert to unwrap the message.
As the last step in the response, you use the public key of the client cert to PKCS7 envelope a message.
If you're really keen, you can also sign messages with some other certificates but that would really be an obscurity.
This way without getting access to private keys on TWO systems, one can never fully decipher the chat.
I'm Looking at using Amazon S3 and simpleDB in a desktop application.
The main issue I have is that I either need to store my aws credentials in the application or use some other scheme.
I'm guessing that storing them in the application is out of the question as they would be easily picked out.
Another option is to create a web service that creates the aws authentication signature but this has its own issues.
Does the signature require all the data from a file thats being uploaded? If so I would have to transfer all the data twice.
There would then be a central failure point which was one of the main reasons for using aws.
Any ideas?
UPDATE:
I needed to make it a bit clearer that I'm wanting to store my aws credentials in an application handed out to others. DPAPI or any other encryption would be only stop people simply using reflector to get the credentials. Using any encryption still needs the key that is easy to get.
UPDATE 2 - Sept 2011
Amazon have released some details on using the AWS Security Token Service, which allows for authentication without disclosing your secret key. More details are available on this blog post.
Tim, you're indeed hitting on the two key approaches:
NOT GOOD ENOUGH: store the secret key "secretly" in the app. There is indeed a grave risk of someone just picking it out of the app code. Some mitigations might be to (a) use the DPAPI to store the key outside the app binary, or (b) obtain the key over the wire from your web service each time you need it (over SSL), but never store it locally. No mitigation can really slow down a competent attacker with a debugger, as the cleartext key must end up in the app's RAM.
BETTER: Push the content that needs to be protected to your web service and sign it there. The good news is that only the request name and timestamp need to be signed -- not all the uploaded bits (I guess Amazon doesn't want to spend the cycles on verifying all those bits either!). Below are the relevant code lines from Amazon's own "Introduction to AWS for C# Developers". Notice how Aws_GetSignature gets called only with "PutObject" and a timestamp? You could definitely implement the signature on your own web service without having to send the whole file and without compromising your key. In case you're wondering, Aws_GetSignature is a 9-line function that does a SHA1 hash on a concatenation of the constant string "AmazonS3", the operation name, and the RFC822 representation of the timestamp -- using your secret key.
DateTime timestamp = Aws_GetDatestamp();
string signature = Aws_GetSignature( "PutObject", timestamp );
byte[] data = UnicodeEncoding.ASCII.GetBytes( content );
service.PutObjectInline( "MainBucket", cAWSSecretKey, metadata,
data, content.Length, null,
StorageClass.STANDARD, true,
cAWSAccessKeyId, timestamp, true,
signature, null );
EDIT: note that while you can keep the secret key portion of your Amazon identity hidden, the access key ID portion needs to be embedded in the request. Unless you send the file through your own web service, you'll have to embed it in the app.
The main issue I have is that I either need to store my aws credentials in the application or use some other scheme.
Does Windows have a system-wide service similar to Apple's Keychain Manager? If so, put your credentials there. If not, perhaps you can build a watered-down version of it for storing a strongly-encrypted version of your AWS credentials.
Does the signature require all the data from a file thats being uploaded?
The HMAC SHA-1 signature is an encoded encryption of the HTTP request headers. This signature is a hash value and will be very short relative to your data, only 20 bytes long.
You can encrypt the config file and/or use ProtectedData. Here's my blog post on both.
UPDATE: You might be a be to encrypt your app.config as part of an install step. Sample here: http://www.codeproject.com/KB/security/encryptstrings.aspx. Not great, but the best I've found so far.
Will you let anyone that gets a hold of a copy of your program access the data on S3/SimpleDB? If not, you will need your own authentication scheme that's independent from AWS security.
In that case, you could implement a web service that accepts the credentials that you give your customers (a username/password for example, a digital certificate, etc) and then performs the S3/SimpleDB operations that your program requires. That way, the AWS credentials never leave AWS. If a particular user's credentials are compromised, you can cancel those credentials in your web service.