Develop Reference

How to use Rx to monitor a project file and files for external changes?

I would like to reproduce the behavior of Visual Studio which informs you when a project file is touched externally and proposes to reload it!
Due to the requirements, I believe reactive is a great match to solve that problem.
I am using a modified reactive FileSystemWatcher described in this post: http://www.jaylee.org/post/2012/08/26/An-update-to-matthieumezil-Rx-and-the-FileSystemWatcher.aspx
public class FileWatcher
{
private static readonly ILog Logger = LogManager.GetLogger(MethodBase.GetCurrentMethod().DeclaringType);
public static IObservable<FileChanged> ObserveFolderChanges(string path, string filter, TimeSpan throttle, Predicate<string> isPartOfProject)
{
return Observable.Create<FileChanged>(
observer =>
{
var fileSystemWatcher = new FileSystemWatcher(path, filter) { EnableRaisingEvents = true, IncludeSubdirectories = true };
var sources = new[]
{
Observable.FromEventPattern<FileSystemEventArgs>(fileSystemWatcher, "Created")
.Where(IsMaybeAProjectFile)
.Select(ev => new FileChanged(ev.EventArgs.FullPath, FileChangeTypes.Added, SourceChangeTypes.FileSystem)),
Observable.FromEventPattern<FileSystemEventArgs>(fileSystemWatcher, "Deleted")
.Where(IsMaybeAProjectFile)
.Select(ev => new FileChanged(ev.EventArgs.FullPath, FileChangeTypes.Deleted, SourceChangeTypes.FileSystem))
};
return sources.Merge()
.Throttle(throttle)
.Do(changed =>
{
if (Logger.IsDebugEnabled)
{
Logger.Debug($"FileWatcher event [{changed.FileChangeType}] {changed.FullPath}");
}
})
.Finally(() => fileSystemWatcher.Dispose())
.Subscribe(observer);
}
);
}
private static bool IsMaybeAProjectFile(EventPattern<FileSystemEventArgs> ev)
{
return ev.EventArgs.FullPath.EndsWith(".zip") || ev.EventArgs.FullPath.EndsWith(".skye");
}
}
public class FileChanged
{
public string FullPath { get; }
public FileChangeTypes FileChangeType { get; }
public SourceChangeTypes SourceChangeType { get; }
public FileChanged(string fullPath, FileChangeTypes fileChangeType, SourceChangeTypes sourceChangeType)
{
FullPath = fullPath;
FileChangeType = fileChangeType;
SourceChangeType = sourceChangeType;
}
}
[Flags]
public enum FileChangeTypes
{
Added = 1,
Deleted = 2
}
[Flags]
public enum SourceChangeTypes
{
FileSystem = 1,
Project = 2
}
Now in my application I created an event
private ProjectChangedEventHandler ProjectChanged { get; set; }
private void OnProjectChanged(FileChanged fileChanged)
{
ProjectChanged?.Invoke(this, fileChanged);
}
public delegate void ProjectChangedEventHandler(object sender, FileChanged fileChanged);
Which is used like this when I delete or a add a file from the project
OnProjectChanged(new FileChanged(archive.Filename, FileChangeTypes.Deleted, SourceChangeTypes.Project));
OnProjectChanged(new FileChanged(archive.Filename, FileChangeTypes.Added, SourceChangeTypes.Project));
Now I can start to leverage those two streams and with a join (which needs fine tuning for the left and right duration selector) I am able to detect which file was modified by my application:
private void ObserveProjectModifications(string projectFilePath)
{
_observeFolderChanges = FileWatcher.ObserveFolderChanges(Path.GetDirectoryName(projectFilePath), "*.*", TimeSpan.FromMilliseconds(500), IsPartOfProject);
_observeProjectChanges = Observable.FromEventPattern<ProjectChangedEventHandler, FileChanged>(h => ProjectChanged += h, h => ProjectChanged -= h).Select(pattern => pattern.EventArgs);
_changes = _observeProjectChanges.Join(_observeFolderChanges, _ => Observable.Never<Unit>(), _ => Observable.Never<Unit>(), ResultSelector).Where(changed => IsPartOfProject(changed.FullPath));
}
private FileChanged ResultSelector(FileChanged fileChanged, FileChanged projectChanged)
{
if (Logger.IsDebugEnabled)
{
Logger.Debug($"ResultSelector File [{fileChanged.FileChangeType}] {fileChanged.FullPath} # Project [{projectChanged.FileChangeType}] {projectChanged.FullPath}");
}
if (fileChanged.FullPath == projectChanged.FullPath)
{
if (fileChanged.FileChangeType == projectChanged.FileChangeType)
{
if (fileChanged.SourceChangeType != projectChanged.SourceChangeType)
{
return projectChanged;
}
return fileChanged;
}
return fileChanged;
}
return fileChanged;
}
private bool IsPartOfProject(string fullPath)
{
if (_projectFileManager.ProjectFilePath.Equals(fullPath)) return true;
return _archives.Values.Any(a => a.Filename.Equals(fullPath));
}
My issue is that I also want to know that a file was modified externally! Any idea would be really helpful! Thanks

Unfortunatelly the FileSystemWatcher doesn't provide information which process has modified the file, so you are bit out of luck there. There are few possibilities that I can think of:
Ignore flag - When your application is doing a change you can set a flag and ignore the events when the flag is set. This is the simplest way, but you might miss some external change if it happens concurrently when the flag is set and also it gets even more complicated due to throttling you have.
Tagging the file - whenever you do a change to the file you generate a guid (or similar) which you will use to tag the file. And then whenever the file change is fired, you check the file property (can be stored either as real filesystem file property - similar for example to jpeg metadata you see in details in file explorer, there are more ways to set such file property) and then if the tag is different from what you have or is missing then you know it is external - there you need to also take care due to throttling and the tag being outdated etc
Minifilter file system driver - This would be the cleanest solution and probably is very close to what Visual studio is using - just a guess though. It is basically a universal windows driver that monitors any I/O change. Microsoft has created reference implementation called minispy, which is small tool to monitor and log any I/O and transaction activity that occurs in the system. You don't have to implement the driver yourself as there is already a 3rd party FileSystemWatcher implemented using this approach on github. That file system watcher provides information which process has modified the file. The only problem here is that the driver itself needs to be installed, before it can be used, so you need admin privileged installer of sort.
At the moment that's all I can think of.

Ignoring files in MVC Bundles

We are using feature flags (set in Web.Config) to toggle the visibility in the UI of certain features that aren't yet complete. I'd also like my MVC bundles to NOT include the related JS files since they would just be useless files the client would have to download when the feature isn't enabled.
So far I've found IgnoreList.Ignore but I can't seem to get it to work. This is basically what I'm doing:
public static void RegisterBundles(BundleCollection bundles)
{
if (!appConfiguration.FeatureXEnabled)
{
//Skip these files if the Feature X is not enabled!
//When this flag is removed, these lines can be deleted and the files will be included like normal
bundles.IgnoreList.Ignore("~/App/Organization/SomeCtrl.js", OptimizationMode.Always);
bundles.IgnoreList.Ignore("~/App/Filters/SomeFilter.js", OptimizationMode.Always);
}
var globalBundle = new ScriptBundle("~/bundles/app-global").Include(
"~/App/RootCtrl.js",
"~/App/Directives/*.js",
"~/App/Services/*.js",
"~/App/Application.js"
);
bundles.Add(globalBundle);
var appOrgBundle = new ScriptBundle("~/bundles/app-org");
appOrgBundle.Include(
"~/App/Filters/*.js",
"~/App/Organization/*.js"
);
bundles.Add(appOrgBundle);
}
With the above code, the files in the ignore list are still being included! What am I doing wrong here?

I have fought the ignore list when using expressions as well. A simple work around I have found is to implement an IBundleOrderer that will exclude the files I do not want, and apply some ordering to how the files get included. Although this is not really its intended use, I find it fills the gap.
The IBundleOrderer gets access to the full list of files (expression expanded to what files it matched).
public class ApplicationOrderer : IBundleOrderer {
public IEnumerable<BundleFile> OrderFiles(BundleContext context, IEnumerable<BundleFile> files)
{
if (!AppSettings.FeatureFlag_ServiceIntegrationsEnabled)
{
//Skip these files if the Service Integrations Feature is not enabled!
//When this flag is removed, these lines can be deleted and the files will be included like normal
var serviceIntegrationPathsToIgnore = new[]
{
"/App/ServiceIntegrations/IntegrationSettingsModel.js",
"/App/ServiceIntegrations/IntegrationSettingsService.js",
"/App/ServiceIntegrations/ServiceIntegrationsCtrl.js"
};
files = files.Where(x => !serviceIntegrationPathsToIgnore.Contains(x.VirtualFile.VirtualPath));
}
return files;
}
}
Example Usage:
public static void RegisterBundles(BundleCollection bundles)
{
var appBundle = new ScriptBundle("~/bundles/app")
.IncludeDirectory("~/App/", "*.js", true)
appBundle.Orderer = new ApplicationOrderer();
bundles.Add(appBundle);
}

I think that the bundling code only happens once - during the initialisation of the website. If you are switching the appConfiguration.FeatureXEnabled flag without restarting/republishing the website then your change will be ignored.
One solution would be to create 2 separate bundles and then use Razor to display the correct bundle reference in your HTML.
e.g.
#{if (appConfiguration.FeatureXEnabled){
<script type="text/javascript" src="~/bundles/bundle-large.js"></script>
}else{
<script type="text/javascript" src="~/bundles/bundle-small.js"></script>
}

I upvoted Gent's answer above because it helped me accomplish what I wanted.
But then I expanded upon it and created a generic IBundleOrderer that uses lambdas to exclude whatever items you want to exclude very simply.
Note the excludes parameter is a params list so you are not limited to a single exclusion pattern. This doesn't do any ordering, but it could be easily added with another parameter.
public class ExcludeItemsOrderer : IBundleOrderer
{
private Func<BundleFile, bool>[] _excludes;
public ExcludeItemsOrderer(params Func<BundleFile, bool>[] excludes)
{
_excludes = excludes;
}
public IEnumerable<BundleFile> OrderFiles(BundleContext context, IEnumerable<BundleFile> files)
{
if(_excludes == null || _excludes.Length == 0)
{
return files;
}
foreach(var exclude in _excludes)
{
var exclusions = files.Where(exclude);
files = files.Except(exclusions);
}
return files;
}
}
And then I created an extension method to simplify using it:
public static class BundleExtensions
{
public static Bundle AsIsOrderExcluding(this Bundle bundle, params Func<BundleFile, bool>[] excludes)
{
bundle.Orderer = new ExcludeItemsOrderer(excludes);
return bundle;
}
}
So that it can be easily applied as you create your bundles, like this:
bundles.Add(
new ScriptBundle("~/Bundles/App/js")
.IncludeDirectory("~/App", "*.js", true)
.AsIsOrderExcluding(bf => bf.VirtualFile.VirtualPath.IndexOf("/skip_me/") >= 0)
);

Try defining the ignore list as follows (using wildcards instead of relative paths):
bundles.IgnoreList.Ignore("*/App/Organization/SomeCtrl.js", OptimizationMode.Always);
bundles.IgnoreList.Ignore("*/App/Filters/SomeFilter.js", OptimizationMode.Always);

Bundles.IgnoreList.Ignore(pattern) doesn't work like that. It compares just the Name part of files (i.e not the full (virtual)path ) with pattern in any bundle going to be returned.
For example you can ignore files starting with name old_style prefix. This filtering will be applied for files under any folder.
public IEnumerable<BundleFile> FilterIgnoredFiles(BundleContext context, IEnumerable<BundleFile> files) {
return files.Where(f => !ShouldIgnore(context, f.VirtualFile.Name));
}
See:
https://aspnetoptimization.codeplex.com/SourceControl/latest#src/System.Web.Optimization/IgnoreList.cs

Is there a way to programmatically export/save generated javascript and css bundles?

I am using System.Web.Optimization BundleTable for CSS and JavaScript file bundling. I was able to save bundle files manually by going to Chrome Developer Tools, Sources tab then clicking on bundle I would like to save and then clicking right mouse button on code and Save as (screenshot below)
I was wondering is there a way to save them programmatically (javascript or .NET I don't mind ether) as I need to test changes in bundle files by comparing them with old versions to see if they have changed or not. The end product will be selenium test most likely.

Sorted it out myself.
public static string GetBundleContents(string virtualPath)
{
OptimizationSettings config = new OptimizationSettings()
{
ApplicationPath = System.Web.Hosting.HostingEnvironment.ApplicationPhysicalPath,
BundleTable = BundleTable.Bundles
};
BundleResponse response = Optimizer.BuildBundle(virtualPath, config);
return response.Content;
}
public static void WriteBundlesToDisk(string path)
{
foreach (var bundle in BundleTable.Bundles)
{
var bundleContents = BundleConfig.GetBundleContents(bundle.Path);
File.WriteAllText(string.Format("{0}/{1}.{2}", path, bundle.Path.Split('/').Last(), BundleConfig.GetFileExtensionByBundleType(bundle)), bundleContents);
}
}
public static string GetFileExtensionByBundleType(Bundle bundle)
{
if (bundle is ScriptBundle)
return "js";
else if (bundle is StyleBundle)
return "css";
return "folderBundle";
}
usage:
BundleConfig.WriteBundlesToDisk("c://bundles");
sidenote: there's a type of bundle called System.Web.Optimization.DynamicFolderBundle that is not being handled properly by the solution above, it will save it as .folderBundle file type.

FTP Directory/SubDirectory listing using edtFTPnet,C#

On my FTP Server I have the following folder structure
- Parent Directory
-a.txt
-b.txt.old
-SubDirectory1
-c.txt
-NestedSubDirectory1
-d.txt
-SubDirectory2
-e.txt
-f.txt.old
The number of SDs are not fixed. I need a way to get all the files(can be any format) without the .old extension from the Parent Directory.
I'm currently using the 3rd party dll edtFTPnet.
ftpConnection.GetFileInfos()Where(f => !(f.Name.EndsWith(".old"))).ToList();
This helps me get the details of the files and folders at the current working directory level.
Can someone tell me a way to get all the files with the parentdirectory, subdirectories and nested subdirectories.
The solution may or may not use edtFTPnet.

FTPConnection.GetFileInfos() returns an array of FTPFile. The class FTPFile has a boolean property Dir which indicates whether its filename accesses a file (false) or directory (true).
Something like this should work:
void ReadSubDirectories(FTPConncetion connection, FTPFile[] files)
{
foreach (var file in files)
{
if (file.Dir)
{
// Save parent directory
var curDir = connection.ServerDirectory;
// Move into directory
connection.ChangeWorkingDirectory(file.Name)
// Read all files
ReadSubDirectories(connection, connection.GetFileInfos());
// Move back into parent directory
connection.ChangeWorkingDirectory(curDir)
}
else
{
// Do magic with your files
}
}
}
However you might be better off using just .NET's built-in FtpWebRequest class since its methods and naming conventions are clearer, it's better documented and it's easier to find references online.

Try to use extensions like this:
class Program
{
static void Main(string[] args)
{
using (var connection = new FTPConnection
{
ServerAddress = "127.0.0.1",
UserName = "Admin",
Password = "1",
})
{
connection.Connect();
connection.ServerDirectory = "/recursive_folder";
var resultRecursive =
connection.GetFileInfosRecursive().Where(f => !(f.Name.EndsWith(".old"))).ToList();
var resultDefault = connection.GetFileInfos().Where(f => !(f.Name.EndsWith(".old"))).ToList();
}
}
}
public static class FtpClientExtensions
{
public static FTPFile[] GetFileInfosRecursive(this FTPConnection connection)
{
var resultList = new List<FTPFile>();
var fileInfos = connection.GetFileInfos();
resultList.AddRange(fileInfos);
foreach (var fileInfo in fileInfos)
{
if (fileInfo.Dir)
{
connection.ServerDirectory = fileInfo.Path;
resultList.AddRange(connection.GetFileInfosRecursive());
}
}
return resultList.ToArray();
}
}

MVC Razor, Include JS / CSS files from another project

I've got a C# MVC project that uses Razor syntax.
To be able to reuse some code, I want to put some of my JavaScript and CSS files in a different project and include them somehow.
This is how my scripts are included at the moment:
<script src="#Url.Content("~/Scripts/bootstrap-typeahead.js")" type="text/javascript"></script>
<script src="#Url.Content("~/Scripts/bootstrap-dropdown.js")" type="text/javascript"></script>
At the moment, the scripts are in the same project as the cshtml file but they should be placed in the Common.Web project instead...
What I want to do is this (doesn't work though):
<script src="#Url.Content("Common.Web/Scripts/bootstrap-typeahead.js")" type="text/javascript"></script>
<script src="#Url.Content("Common.Web/Scripts/bootstrap-dropdown.js")" type="text/javascript"></script>

I do this very thing. However I embed the Javascript files and other content in another DLL and then call them from my razor syntax like so. Here is the code I use.
In the View:
Script example:
<script src=#Url.Action("GetEmbeddedResource", "Shared", new { resourceName = "Namespace.Scripts.jquery.qtip.min.js", pluginAssemblyName = #Url.Content("~/bin/Namespace.dll") }) type="text/javascript" ></script>
Image Example:
#Html.EmbeddedImage("corporate.gif", new { width = 150, height = 50})
Here is my helper methods:
public static MvcHtmlString EmbeddedImage(this HtmlHelper htmlHelper, string imageName, dynamic htmlAttributes)
{
UrlHelper url = new UrlHelper(HttpContext.Current.Request.RequestContext);
var anchor = new TagBuilder("img");
anchor.Attributes["src"] = url.Action("GetEmbeddedResource", "Shared",
new
{
resourceName = "Namespace.Content.Images." + imageName,
pluginAssemblyName = url.Content("~/bin/Namespace.dll")
});
if (htmlAttributes != null)
{
string width = "";
string height = "";
PropertyInfo pi = htmlAttributes.GetType().GetProperty("width");
if (pi != null)
width = pi.GetValue(htmlAttributes, null).ToString();
pi = htmlAttributes.GetType().GetProperty("height");
if (pi != null)
height = pi.GetValue(htmlAttributes, null).ToString();
if (!string.IsNullOrEmpty(height))
anchor.Attributes["height"] = height;
if (!string.IsNullOrEmpty(width))
anchor.Attributes["width"] = width;
}
return MvcHtmlString.Create(anchor.ToString());
}
Lastly my shared Controller:
[HttpGet]
public FileStreamResult GetEmbeddedResource(string pluginAssemblyName, string resourceName)
{
try
{
string physicalPath = Server.MapPath(pluginAssemblyName);
Stream stream = ResourceHelper.GetEmbeddedResource(physicalPath, resourceName);
return new FileStreamResult(stream, GetMediaType(resourceName));
//return new FileStreamResult(stream, GetMediaType(tempResourceName));
}
catch (Exception)
{
return new FileStreamResult(new MemoryStream(), GetMediaType(resourceName));
}
}
private string GetMediaType(string fileId)
{
if (fileId.EndsWith(".js"))
{
return "text/javascript";
}
else if (fileId.EndsWith(".css"))
{
return "text/css";
}
else if (fileId.EndsWith(".jpg"))
{
return "image/jpeg";
}
else if (fileId.EndsWith(".gif"))
{
return "image/gif";
}
else if (fileId.EndsWith(".png"))
{
return "image/png";
}
return "text";
}
Resource Helper:
public static class ResourceHelper
{
public static Stream GetEmbeddedResource(string physicalPath, string resourceName)
{
try
{
Assembly assembly = PluginHelper.LoadPluginByPathName<Assembly>(physicalPath);
if (assembly != null)
{
string tempResourceName = assembly.GetManifestResourceNames().ToList().FirstOrDefault(f => f.EndsWith(resourceName));
if (tempResourceName == null)
return null;
return assembly.GetManifestResourceStream(tempResourceName);
}
}
catch (Exception)
{
}
return null;
}
}
Plugin Helper
public static T LoadPluginByPathName<T>(string pathName)
{
string viewType = typeof(T).GUID.ToString();
if (HttpRuntime.Cache[viewType] != null)
return HttpRuntime.Cache[viewType] is T ? (T)HttpRuntime.Cache[viewType] : default(T);
object plugin = Assembly.LoadFrom(pathName);
if (plugin != null)
{
//Cache this object as we want to only load this assembly into memory once.
HttpRuntime.Cache.Insert(viewType, plugin);
return (T)plugin;
}
return default(T);
}
Remember that I am using these as embedded content!

To my knowledge you can't do this as the path would lie outside of the website.
You can however do the following:
1) Put all the scripts you want to share in Common.Web\Scripts
2) For each script file in your Web application 'Add as Link' to your Common.Web Scripts (you don't even need to do this step; it is however nice to see what scripts your web app uses in VS)
3) Add a post-build event to your Web application that copies the scripts from Common.Web\Scripts to your WebApp\Scripts folder:
copy $(ProjectDir)..\Common.Web\Scripts* $(ProjectDir)\Scripts
so from your perspective in Visual Studio you will only have a single place to update your .js files that can be used by multiple projects.

Instead of using Url helper use relative addressing. What you tried to do makes no sense, as helper is used to resolve paths relative to it's project.
Since you are trying to use resources of another project, it's ok to assume that you know upfront where you're going to deploy each project. Even though I don't like this practice, I can think of a pragmatic solution for this.
If your two applications are at urls:
http://www.mysite.com/app1
http://www.mysite.com/Common.Web
you could address like this:
<script src="#Url.Content("~")/../Common.Web/Scripts/bootstrap-typeahead.js")" type="text/javascript"></script>
meaning, resolve my app root folder, go up a level, and go down rest of the path.

I just found this question while looking for the same answer. Thanks to the previous posters, who made it clear that this cannot be done using Visual Studio alone: you will always end up with a copy of the original file, or an incomplete set of shipping files.
I do not wish to reinvent the wheel whenever I have simple common functionality, so I have a folder for common scripts on my hard drive:
/Common/Javascript
My web project is located in:
/ProjectName
so my common scripts lie outside my web project.
I solved this via source control. Most source control repositories will have the functionality to show a file in another directory. My steps were:
Create empty folder under my VS project: /ProjectName/Scripts/Common
Committed the empty folder to source control
Using Subversion (my source control), I set up an "extern" so that my common file was linked to the new folder. Other source control software may call this a "link" or some other such thing.
Updated the new folder, and my common Javascript files came in, and could now be added to my web project in Visual Studio.
Obviously, I tested this by changing the files within Visual Studio and committing them. The changes were indeed reflected in the original files, sitting in /Common/Javascript.
I can now simply add an extern to any other web project which needs to use the same functionality: though, of course, changing those files comes with additional risk now, as I may unexpectedly break other projects which use them. Though I can configure an extern to use a specific revision of a file, that's not what I want to do at this point; I shall await such complexity as and when it occurs.