Hey guys.. I am writing a Windows application in C# that minifies CSS files and packs JS files as a batch job. One hurdle for the application is, what if the user selects a JavaScript file that has already been packed? It will end up increasing the file size, defeating my purpose entirely!
Is opening the file and looking for the string eval(function(p,a,c,k,e,d) enough? My guess is no, as there are other JS packing methods out there. Help me out!
One might suggest that you compare the size of the pre and post packed JS and return/use the smaller of the two.
UPDATE based on question in comment by GPX on Sep 30 at 1:02
The following is a very simple way to tell. There may be different, or more accurate, ways of determining this, but this should get you going in the right direction:
var unpackedJs = File.ReadAllText(...)
var unpackedSize = jsContent.Length;
var packedJs = ... // Your Packaging routine
File.WriteAllText(pathToFile, unpackedSize < packedJs.Length ? unpackedJs : packedJs)
I would check file size and lines of code (e.g.: average line length). These two information should be enough to know if the code is sufficiently compact.
Try this demo.
I direct you to a post that suggests packing is bad.
http://ejohn.org/blog/library-loading-speed/
Rather use minification. Google Closure compiler can do this via a REST web service. Only use a .min.js extension for minified (not packed).
Gzip will do a better job and will be uncompressed by the browser. Its best to switch on zip compression on the server which will zip a minified file down further.
Of course this raises the question 'How can I tell if my Javascript is already minified!'
When you create/save a minified file, use the standard file name convention of "Filename.min.js". Then when they select the file, you can check for that as a reliable indicator.
I do not think it is wise to go overboard on the dummy-proofing. If a user (who is a developer, at that), is dumb enough to double-pack a file, they should experience problems. I know you should give them the benefit of the doubt, but in this case it does not seem worth the overhead.
If you're using a safe minimization routine, your output should be the same as the input. I would not recommend the routine you mention. MS's Ajax Minifier is a good tool and even provides dll's to use in your project. This would make your concern a non-issue.
I would suggest adding a '.min' prefix to the extension of the packed file, something like 'script.min.js'. Then just check the file name.
Other than that, I would suggest checking how long the lines are, and how many spaces are used. Minified/packed JS typically has almost no spaces (typically in strings) and very long lines.
Related
In the purpose of practicing for an upcoming programming contest, I'm making a very basic search engine in C# that takes a query from the user (e.g. "Markov Decision Process") and searches through a couple of files to find the most relevant one to the query.
The application seems to be working (I used a term-document matrix algorithm).
But now I'd like to test the functionality of the search engine to see if it really is working properly. I tried to take a couple of Wikipedia articles and saving them as .txt files and testing it out, but I just can't see if it's working fast enough (even with some timers).
My question is, is there a website that shows a couple of files to test a search engine on (along with the logically expected result)?
I'm testing with common sense so far, but it would be great to be sure of my results.
Also, how can I get a collection of .txt files (maybe 10 000+ files) about various subjects to see if my application runs fast enough?
I tried copying a few Wikipedia articles, but it would take way too much time to do. I also thought about making a script of some sort to do it for me, but I really don't know how to do that.
So, where can I find a lot of files with separated subjects?
Otherwise, how can I benchmark my application?
Note: I guess a simple big .txt file where each line represents a "file" about a subject would do the job too.
One source of text files would be Project Gutenberg. They supply CD/DVD images if you want to download thousands of files at once. (The page doesn't state it, but I would imagine they are in txt format inside the CD/DVD iso.)
You can get wikipedia pages by using a recursive function and loading the html from every page linked to by one set page.
if you have some experience with c# this should help you:
http://www.csharp-station.com/HowTo/HttpWebFetch.aspx
then loop through the text and collect all the instances of the text: "<a href=\""
and recursively call that method. You should also use a counter to limit the number of recursions.
Also, to prevent OutOfMemory exceptions you should stop the method when it reaches multiples of some number of iterations and write everything to a file. Then flush the old data from a string
You can use the datasets from GroupLens Research's site.
Some samples: movies, books
Is this even possible?
I realize that asking them to enter data when the program runs and saving it in the executable file itself is out. (Or is it?)
Right now I'm considering trying to build the program server-side with php and have it incorporate a separate text file which would contain the information. This seems marginally feasible, though I would have quite a bit of learning to accomplish it.
I was hoping for some other ideas of how I might accomplish this.
I am not interested in separate configuration or text files or putting data in windows registry. I am only looking for solutions where it can be quite-solidly a part of the executable.
Does anybody have any experience with this?
Thank you.
Its perfectly possible, that's how self-extracting zip files work.
Basically, you can add as much stuff to the end of the executable file as you want. Your program can then open its own file up on disk and read it back.
How about using Settings within your app? It depends on what you mean by "storing the user registration" as to how you would best achieve this, though. If you could give some more information about what you actually want to store, that would be useful.
An example would be to save a username, or an authentication token, and use that each time you need to check a "registration". As I say, though, the details of what to store would depend entirely on what you want to do it that data...
You could use it to embed in the unmanaged resources.
I am looking at implementing some performance optimization around my javascript/css. In particular looking to achieve the minification and combining of such. I am developing in .net/c# web applications.
I have a couple of options and looking for feedback on each:
First one is this clever tool I came across Chirpy which via visual studio combines, minifies etc -> http://chirpy.codeplex.com/ This is a visual studio add in but as I am in a team environment, this tool isnt ideal.
My next option is to use an Msbuild task (http://yuicompressor.codeplex.com/) to minify the files and also combine them (maybe read from an xml file what needs to be combined). While this works for minifying fine, the concern I have is that I will have to maintain what must be combined which could be a headache.
3rd option is to use msbuild task just for the minifying and at runtime using some helper classes, combine the files on a per page basis. This would combine the files, give it a name and add a version to it.
Any other options I could consider? My concern with the last option is that it may have performance issues as I would have to open the file from the local drive, read its contents and then combine the files. This is alot of processing at run time. I was looking at something like Squishit - https://github.com/jetheredge/SquishIt/downloads This minifies the files at run time but I would look at doing this at compile time.
So any feedback on my approaches would be great? If the 3rd option would not cause performance issues, I am leading towards it.
We have done something similar with several ASP.NET web applications. Specifically, we use the Yahoo Yui compressor, which has a .NET library version which you can reference in your applications.
The approach we took was to generate the necessary merged/minified files at runtime. We wrapped all this logic up into an ASP.NET control, but that isn't necessary depending on your project.
The first time a request is made for a page, we process through the list of included JS and CSS files. In a separate thread (so the original request returns without delay) we then merged the included files together (1 for JS, 1 for CSS), and then apply the Yui compressor.
The result is then written to disk for fast reference in the future
On subsequent requests, the page first looks for the minified versions. If found, it just serves those up. If not, it goes through the process again.
As some icing to the cake:
For debug purposes, if the query string ?debug=true is present, the merged/minified resources are ignored and the original individual files are served instead (since it can be hard to debug optimized JS)
We have found this process to work exceptionally well. We built it into a library so all our ASP.NET sites can take advantage. The post-build scripts can get complicated if each page has different dependencies, but the run-time can determine this quite easily. And, if someone needs to make a quick fix to a CSS file, they can do so, delete the merged versions of the file, and the process will automatically start over without need to do post-build processing with MSBuild or NAnt.
RequestReduce provides a really nice solution for combining and minifying javascript and css at run time. It will also attempt to sprite your background images. It caches the processed files and serves them using custom ETags and far future headers. RequestReduce uses a response filter to transform the content so no code or configuration is needed for basic functionality. It can be configured to work in a web farm environment and sync content accross several servers and can be configured to point to a CDN. It can be downloaded at http://www.RequestReduce.com or from Visual Studio via Nuget. The source is available at https://github.com/mwrock/RequestReduce.
have you heard of Combres ?
go to : http://combres.codeplex.com and check it out
it minifies your CSS and JS files at Runtime meaning you can change any file and upload it and each request the client does it minifies it.
all you gotta do is add the files u wanna compress to a list in the combres XML file and just call the list from your page / masterpage.
if you are using VS2010 you can easily install it on your project using NuGet
here's the Combres NuGet link: http://combres.codeplex.com/wikipage?title=5-Minute%20Quick%20Start
I did a really nice solution to this a couple of years back but I don't have the source left. The solution was for webforms but it should work fine to port it to MVC. I'll give it a try to explain what I did in some simple step. First we need to register the scripts and we wrote a special controller that did just that. When the controller was rendered it did three things:
Minimize all the files, I think we used the YUI compression
Combine all the files and store as string
Calculate a hash for the string of the combined files and use that as a virtual filename. You store the string of combined files in a cached dictionary on the server with the hash value as key, the html that is rendered needs to point to a special folder where the "scripts" are located.
The next step is to implement a special HttpHandler that handles request for files in the special folder. When a request is made to that special folder you make a lookup in the cached dictionary and returns the string bascially.
One really nice feature of this is that the returned script is always valid so the user will never have to ask you for an update of the script. The reason for that is when you make a change to any of the script files the hash value will change and the client will ask for a new script.
You can use this for css-files as well with no problems. I remebered making it configurable so you could turn off combine files, minimize files, or just exclude one file from the process if you wanted to do some debugging.
I might have missed some details, but it wasn't that hard to implement and it turned out very well.
Update: I've implemented a solution for MVC and released it on nuget and have the source up on github.
Microsoft’s Ajax minifier is suprisingly good as a minification tool. I wrote a blog post on combining files and using their minifier in a javascript and stylesheet handler:
http://www.markistaylor.com/javascript-concatenating-and-minifying/
It's worthwhile combining the files at run time to avoid having to synchronise new versions. However, once they are programmatically combined, cache them to disk. Then the code which runs each time the files are fetched need only check that the files haven't changed before serving the cached version.
If they have changed, then the compression code can run as a one-off.
Whilst there will be a slight performance cost, you will also receive a performance benefit from fewer file requests.
This is the approach that the Minify tool uses to compress JS/CSS, which has worked really well for me. It's Linux/PHP only, but you might get some more ideas there too.
I needed a solution for combining/minifying CSS/JS on a .NET 2.0 web app and SquishIt and other tools I found weren't .NET 2.0-compatible, I created my own solution that uses a syntax similar to SquishIt but is compatible with .NET 2.0. Since I thought other people might find it useful I put it up on Github. You can find it here: https://github.com/AlliterativeAlice/simpleyui
I have a program that requires a few large (~4 or 5mb) files. Once a week, every week, there are new versions of these files with minor changes. Mostly just a few lines added or removed.
When the program starts, if there's an Internet connection, I'd like the program to update these files automatically. Instead of downloading the entire new versions of the files, I'll like to download just a patch based on the client's version of the files that updates them.
How might I do this?
I have total control over the server.
That is a tough problem to solve if you don't have any for knowledge of what is in the file or the server doest have a facility to allow you to request differences. Any program you write that does not have a way to determine the differences with out looking at the old and new file will have to download it anyway.
C# doesn't have any built-in facility to do this, but it sounds like your requirements aren't complicated. Look at how diff and ed on Unix can be used to patch a text file based on an easy-to-grok delta. Of course you should check the resulting file against a hash and fall back to a full download if it isn't correct.
I’m looking for a code snippet that would programmatically format unallocated space on a given drive.
I’m ideally looking for .net code (VB.NET C#) or C++.
Many thanks
The best option seems to be to use WMI. In particular, you'll want to look into the Format method of the Win32_Volume class.
You could instead use the SHFormatDrive function of the Win32 API. This may be the simpler option (especially in C++), though I'm not sure how the functionality compares with the WMI method. Edit: As M. Jahedbozorgan points out, this seems to open the Explorer shell dialog to format the drive.
A third option is to run the command-line format.exe from code (and then read from the stdout stream), but this clearly isn't a very nice solution.
Other suggestions are given in this thread on MSDN forums.
Create a file that fills up the entire drive, then write random data to it, then flush it, then close it.
The first step is probably the hardest to do reliably across multiple configurations (e.g., NTFS compressed volumes or per-user quotas).
Or use "cipher /w" from the command line.
It looks like Windows only provides a method to format an entire volume, not just the unallocated space.
So, if I understand your question right, you could create a partition in the remaining space of the drive, then format that new partition.
You will want to use PInvoke and WMI's Win32_Volume.