I am currently building an application in C# that makes use of the AWS SDK for uploading files to S3.
However, I have some users who are getting the "Request time too skewed" error when the application tries to upload a file.
I understand the problem is that the user's clock is out of sync, however, it is difficult to expect a user to change this, so I was wondering, is there any way to get this error not to occur (any .NET functionality to get accurate time with NTP or the alike?)
Below the current code I am using to upload files.
var _s3Config = new AmazonS3Config { ServiceURL = "https://s3-eu-west-1.amazonaws.com" };
var _awsCredentials = new SessionAWSCredentials(credentials.AccessKeyId, credentials.SecretAccessKey, credentials.SessionToken);
var s3Client = new AmazonS3Client(_awsCredentials, _s3Config);
var putRequest = new PutObjectRequest
{
BucketName = "my.bucket.name",
Key = "/path/to/file.txt",
FilePath = "/path/to/local/file.txt"
};
putRequest.StreamTransferProgress += OnUploadProgress;
var response = await s3Client.PutObjectAsync(putRequest);
Getting the time from a timeserver is actually the easier part of your challenge. There is no built-in C# functionality that I'm aware of to get an accurate time from a time server, but a quick search yields plenty of sample code for NTP clients. I found a good comprehensive sample at dotnet-snippets.com (probably overkill for your case), and a very streamlined version on Stack Overflow in a page titled "How to Query an NTP Server using C#?". The latter looks like it might be effective in your case, since all you need is a reasonably accurate idea of the current time.
Now on to the real challenge: using that time with Amazon S3. First, some background, as it's important to understand why this is happening. The time skew restriction is intended to protect against replay attacks, as noted here:
http://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html
Because of this, Amazon built the current timestamp into the authentication signature used in the AWS SDK when constructing the HTTP(S) request. However, the SDK always uses the current time (there's no way to override it in the SDK methods):
https://github.com/aws/aws-sdk-net/blob/master/AWSSDK_DotNet35/Amazon.Runtime/Internal/Auth/AWS3Signer.cs#L119
Note that in all cases, the SDK uses AWSSDKUtils.FormattedCurrentTimestampRFC822 as the timestamp, and there's no way for the caller to pass a different value into the method.
So this leaves you with two options that I can see:
Bypass the Amazon SDK and construct your own HTTP requests using the time you retrieve from an NTP server. This is doable but not easy. Amazon discourages this approach because the SDK provides a lot of helpful wrappers to ensure that you're using the API as a whole correctly, handling a lot of the tedious message processing that you have to do yourself if you go straight HTTP. It also helps with a lot of the error handling and ensuring that things get cleaned up properly if a transfer is interrupted.
Clone the Amazon SDK git repository and create your own fork with a modification to allow you to pass in the current time. This would also be difficult, as you'd have to work out a way to pass the time down through several layers of API objects. And you'd lose the benefit of being able to easily update to a new SDK when one is available.
Sorry there's no easy answer for you, but I hope this helps.
If you're asking this question you have probably taken AWS as far as you can go with the provided code sample.
I have found most of the async upload functionality provided by AWS to be more theoretical, or better suited for limited use cases, instead of being production ready for the mainstream- especially end users with all those browsers and operating systems:)
I would recommend rethinking the design of your program: create your own C# upload turnstile and keep the AWS SDK upload functions running as a background process (or sysadmin function) so that AWS servers are handling only your server's time.
Related
I try to create a game that should help people learn for university. My problem is, how can player share questions between phones and also get them into the right folder. Is there a way for setting up a server to upload files and then download them? I had something in mind like the Levelcode system from "Mario Maker".
If a system like the one in "Mario Maker" would work, what kind of server do I need? I know I need some kind of a Database but do I need something like mySQL? And also how do I set it up? I never learned server programming, but I'm eager to learn.
You need to set up and host a REST API.
There are many different ways to go about it, and it's really up to your preference for your host service, programming language, and a database library.
If you have no preference. I'd recommend:
AWS - powerful and you get 12 months of free hosting
Golang - modern, fast, and great for web apps
SQLite (go-sqlite3) - simple and lightweight
You will need to setup handlers for requests. So for example:
To add a new level:
POST example.com/level/
To get an existing level:
GET example.com/level/:id
In Golang you can handle a request using:
func levelHandler(w http.ResponseWriter, r *http.Request) {
switch r.Method {
case "GET":
// find level id in the database
case "POST":
// add level to database
}
func main() {
http.HandleFunc("/level", levelHandler)
http.ListenAndServe(":8080", nil);
}
To modify an existing level:
PUT example.com/level/:id
The problem now is that anyone in the world will be able to change any level. You will want to add user authentication, and that's a whole other can of worms.
You may consider using REST asset for Unity.
Just one final note: server programming is hard, be prepared to spend a lot of time trying, failing, and learning.
Trying to figure out what options I have available to have a program running locally get its calculations or results of the calculations from a remote source.
Problem:
I have a data acquisition application that reads a lot of instruments and collects data while testing equipment. This data than goes through various forms of aggregation (min,max,average,etc) and than several calculations are applied to it and the result is saved to a database. This process happens several times during a test. This application runs on a machine dedicated to perform this test, but users outside of the test also need to perform the same calculations for experimentation, data analysis etc etc.
In the past, our two applications, (one with the equipment and the one with the users) would get updated every time a calculation changed and then deployed everywhere. This is a pain.
Question:
I'm looking for possible options to solve this problem.
What I've found so far.
1). WCF.
Like:
Only have to update the server and both the programs can now take advantage of the new calculation.
Concern:
The DataContract would contain several classes that would have to be passed to the function(s). ( Total Size of "data" could range from 1 MB to 1 GB depending on the situation). Not sure if the amount of data is an actual problem at this time.
2). Store compiled DLLs and download/load them.
Query the server for a class library. Download it. Load it into memory and use the calculations.
Like:
Do not have to pass a lot of data back and forth.
Concern:
DLL that now resides on each and every computer. People may not be forced to update to the correct version which may cause problems. DLL on the local persons computer may pose a security risk.
That is a pretty tricky question and I think there are quite a number of ways you could try solve it. Here are a few ideas that may get you started
Use Akka.Net and take a look at Akka.Remote which may be able to solve your remote deploy issue. But I'm not sure that this will fit with what you need if you have users deciding to join the cluster ad-hoc rather than having a fixed set of places to distribute calculations?
If the calculations are simple mathematical formulas you could store them as a set of rules and download the formulas. I.e. taking more of a rules engine approach. See perhaps C# expression trees
Browser based solution. This clearly wins from a ease of remote deployment point of view. There are many options if you don't want to use JavaScript, e.g. PureScript, F# (Fable), ClojureScipt, ELM and possibly even C# with JSIL/Bridge. Moving from desktop to web is quite a big shift though so make sure you know what you are in for :)
After trying a few options we found that sending data back and forth was too cumbersome. Instead we chose to load a DLL dynamically with all the calculations we require and use a set of interfaces to define the behaviour.
What I did learn and found was neat is that I can store the DLL in the database and load it directly into memory without having to copy it to the hard drive first thus providing an added layer of protection. Using SQL and FILESTREAM. It also provides a sort of version control in that I can choose which version of the DLL to load from the database and calculate/recalculate the various values.
// Read in File
using (SqlFileStream fileStream = new SqlFileStream(path, transaction, System.IO.FileAccess.Read))
{
byte[] buffer = new byte[size];
int bytesRead = fileStream.Read(buffer, 0, buffer.Length);
// Load Assembly
Assembly DLL = Assembly.Load(buffer);
Type myInterfaceType = DLL.GetType("MyNamespace.MyClass");
// Get access to the root interface
IMyInterface myClass = (IMyInterface)Activator.CreateInstance(myInterfaceType);
}
I am trying to create a basic chat room just to enhance my programming and logical skills, but I can't figure out the functionality here.
The question which is bothering me is whether should I include database or not?
(p.s: I don't want to record any chat sessions).
I tried on my own by using Application["variable"] to post messages like.
Application["Message"] = txtMessage.text;
txtDisplay.text = txtDisplay.text + Application["Message"].ToString();
I know this is not the correct way, there will be some limits to store huge amount of messages and can't be implemented on large traffic. I tested it on LAN and it worked fine, but need a proper way to complete my project.
Need a kick start.
If you want a proper solution of the chat utility (with latest technologies including ASP.net MVC), you should consider WebSockets [http://www.codeproject.com/Articles/618032/Using-WebSocket-in-NET-Part] and SignalR [http://www.asp.net/signalr].
Let's say you have an ASP.NET MVC 4 Web API project. One of your resources, when you call it by URL, waits while it gets performance monitoring data for a specified amount of time and then returns it all in JSON form once it has completed. However, between entering the URL and when the process has completed, is there a way to return data dynamically, ie. at each second when performance data is retrieved and display it in the browser.
Here's the issue:
Calling an API resource via URL is static as far as I know and as far as anyone seems to know. Meaning, the JSON won't appear until the resource has retrieved all of its information, which is not what I want. I want to be able to constantly update the JSON in the browser WHILE the API resource is retrieving data.
Since I'm working in a repository class and a controller class, Javascript is not an option. I've tried using SignalR, but apparently that does not work in this scenario, especially since I'm not able to use Javascript.
Is there any possible way to get real-time data with a URL call to the API?
Case in point:
Google Maps.
The only way you can call the Google Maps API via URL is if you want a "static" map, that displays a single image of a specific location. No interaction of any kind. If you want a dynamic, "real time" map, you need to build a web application and consume the API resource in your application with Javascript in a view page. There is no way to call it via URL.
You can put together an old-school ASP.Net IHttpHandler implementation regardless of MVC controllers and routing pipeleline. http://support.microsoft.com/kb/308001. You would then have full acesss to the response stream and you could buffer or not as you see fit. You've got to consider though whether you want to tie up worker thread for that long and if you're planning on streaming more or less continuously then you definately want to use IAsyncHttpHandler while you await further responses from your repo.
Having said that, Web API supports Async too but it's a little more sophisticated. If you plan on sending back data as-and-when, then I'd strongly recommend you take another look at SignalR which does all this out of the box if you are planning on having some JavaScript client side eventually. It's much, much easier.
If you really want to write Async stuff in Web API though, here's a couple of resources that may help you;
http://blogs.msdn.com/b/henrikn/archive/2012/02/24/async-actions-in-asp-net-web-api.aspx
And this one looks like exactly what you need;
http://blogs.msdn.com/b/henrikn/archive/2012/04/23/using-cookies-with-asp-net-web-api.aspx
In order to use that PushStreamContent() class in the example though, you'll not find that in your System.Net.Http.dll, you'll need to go get that from the Web API stack nightly builds at http://aspnetwebstack.codeplex.com/SourceControl/list/changesets
YMMV - good luck!
I think what you're asking for is a sort of streaming mechanism over HTTP. Of course doing that would require sending a response of unknown content length.
This question deals with that sort of chunked transfer encoding which is probably part of the solution. Not knowing what is on the client side, I can't say how it would deal with the JSON you want to push through.
Great question.
You can certainly start streaming the response back to the browser as soon as you want. It's normally buffered, but it doesn't have to be. I've used this trick in the past. In fact SignalR does something similar in some operational modes, although I should add (now I've re-read your question) that although HTTP supports this, it won't be obvious by default from a Web API controller. I think you'll need to get a little lower into the response handling so you can flush the buffer than simply returning a POCO from your web method if that's what you mean.
Essentially, you'll be wanting to write and flush the buffer after you've gathered each piece of information, so I don't think you'll be able to do that with the typical model. I think you'll need a custom message handler http://www.asp.net/web-api/overview/working-with-http/http-message-handlers to get at the payload in order to do that.
I'm curious though, you say you want to send back JSON but you're not allowed JavaScript?
I am not a software engineer as you will see if you continue reading, however I managed to write a very valuable application that saves our company lots of money. I am not paid to write software, I was not paid for writing this application, nor is my job title software engineer so I would like to have total control over who uses this application if I ever had to leave since as far as I can tell it is not legally theirs (did not write during company hours either).
This may sound childish but I've put much much time into this and I've been maintaining it almost on a daily basis so I feel that I should have some control over it, or at least could sell it to my company if they ever had to let me go, or I wanted to move on.
My current protection scheme on this application looks something like this:
string version;
WebRequest request = WebRequest.Create("http://MyWebSiteURL/Licence text file that either says 'expired' or "not expired'");
WebResponse response = request.GetResponse();
StreamReader stream = new StreamReader(response.GetResponseStream());
version = stream.ReadToEnd();
stream.Close();
response.Close();
if (version == ("not expired") == false)
{
MessageBox.Show(Environment.NewLine + "application expired etc etc", "Version Control");
}
It checks my server for "not expired" (in plain text), and if the webrequest comes back as anything but "not expired", it ultimately pops up another form stating it is expired and allows you to type in a passcode for the day which is a multiplication of some predetermined numbers times the current date to create "day passes" if ever needed (I think Alan Turing just rolled over in his grave).
Not the best security scheme, but I thought it was pretty clever having no experience in software security. I have however heard of hex editing to get around security so I did a little test for science and found this area of my compiled EXE:
"System.Net.WebRequest." Which I filled in with zeros to look like this: System.Net000000000
That was all it took to offset the loading of the application to hiccup during the server check which allowed me to click "continue" and completely bypass all my "security" and go along using the program without it ever expiring.
Now would a normal person go to this length (hex editing) to try to get past my protection scheme? Not likely, however just as a learning experience, what could I do as an added step to make hex editing or any other common workarounds not work unless it was by "professional" cracker?
Again I'm not paranoid, I'm just eager to learn more about security of applications. I was both proud of myself and ashamed at the same time for creating and breaking my own protection.
If commenting, please be kind since I know this is probably a humerus post to those more informed than I as I really have little experience in writing software and have never taken any type of course etc. Thanks for reading!
Another way to bypass the license check is to redirect the checking url to localhost returning always the desired text...
A better way is to make a call to a function doing the same thing but make your server response a signed XML including the server response time-stamp, that you can check on addition with the system datetime (use UTC dates in both sides). It is also a good idea to throw exceptions whenever something is not the way you expect it, and control the flow of your program with exception handling.
Check the following to get a how to clue:
How to: Sign XML Documents with Digital Signatures
How to: Verify the Digital Signatures of XML Documents
Now would a normal person go to this length (hex editing) to try to
get past my protection scheme?
Well i guess, that depends on how useful the application is for that "normal person", and how determines he is to make it work.
Most .net application unless obfuscated can be easily de-compiled to the source code using tools like (Telerik JustDecompile) or they can simple use the ildasm to see the IL code, i heard there are tools to even de-compile obfuscated .net libraries, although i haven't used or found any.
With my little experience, i can suggest two approaches
Enforcing licensing and cracking it in a application which runs plainly on the user machine is a cat and mouse game, you can add some extra protection to your code by moving some part of the applications functionality to the server and expose it as a web service which your client can consume, the part you move to the server must be an important part for the application to work and should be something that is hard to simulate.
The other approach is to add a auto updater feature to your application that will check the server for latest updates, and when ever it finds a new version it will overwrite the older one, thus overriding any cracked version, this can be easily disabled, but if disabled this will also stop any bug fixes you might release
I tried both the approaches, but they are only useful to some extent and you have to decide whether it is worth the effort enforcing or not