I have a site that requires login before it lets you download files. Currently I am using the BrowserSession Class to login and do all the scraping required (at least for the most part).
BrowserSession Class source at bottom of post:
The download Links show up on the document nodes. But I don't know how to add download functionality to that class, and If I try to download them with a webclient it fails, I already had to heavily modify the BrowserSession class, (I should have Modified it as a Partial but didn't) So I don't really want to change from using the BrowserSession Class.
I believe its using htmlAgilityPack.HtmlWeb to download and load the webpages.
If there is no easy way to modify the BrowserSession, Is there someway to use it's CookieCollection With Webclient?
PS: I Need to be logged in to download the file, Otherwise the link redirects to the login screen. Which is why I am unable to simply use WebClient, and either need to modify the BrowserSession class to be able to download, or modify WebClient to use cookies before getting a page.
I will admit I do not understand cookies very well (I am not sure if they are used every time GET is used, or if its just on POST), but so far BrowserSession has taken care of all that.
PPS:The BrowserSession I Posted Is not the one that I added stuff too, however the core functions are all the same.
public class BrowserSession
{
private bool _isPost;
private HtmlDocument _htmlDoc;
/// <summary>
/// System.Net.CookieCollection. Provides a collection container for instances of Cookie class
/// </summary>
public CookieCollection Cookies { get; set; }
/// <summary>
/// Provide a key-value-pair collection of form elements
/// </summary>
public FormElementCollection FormElements { get; set; }
/// <summary>
/// Makes a HTTP GET request to the given URL
/// </summary>
public string Get(string url)
{
_isPost = false;
CreateWebRequestObject().Load(url);
return _htmlDoc.DocumentNode.InnerHtml;
}
/// <summary>
/// Makes a HTTP POST request to the given URL
/// </summary>
public string Post(string url)
{
_isPost = true;
CreateWebRequestObject().Load(url, "POST");
return _htmlDoc.DocumentNode.InnerHtml;
}
/// <summary>
/// Creates the HtmlWeb object and initializes all event handlers.
/// </summary>
private HtmlWeb CreateWebRequestObject()
{
HtmlWeb web = new HtmlWeb();
web.UseCookies = true;
web.PreRequest = new HtmlWeb.PreRequestHandler(OnPreRequest);
web.PostResponse = new HtmlWeb.PostResponseHandler(OnAfterResponse);
web.PreHandleDocument = new HtmlWeb.PreHandleDocumentHandler(OnPreHandleDocument);
return web;
}
/// <summary>
/// Event handler for HtmlWeb.PreRequestHandler. Occurs before an HTTP request is executed.
/// </summary>
protected bool OnPreRequest(HttpWebRequest request)
{
AddCookiesTo(request); // Add cookies that were saved from previous requests
if (_isPost) AddPostDataTo(request); // We only need to add post data on a POST request
return true;
}
/// <summary>
/// Event handler for HtmlWeb.PostResponseHandler. Occurs after a HTTP response is received
/// </summary>
protected void OnAfterResponse(HttpWebRequest request, HttpWebResponse response)
{
SaveCookiesFrom(response); // Save cookies for subsequent requests
}
/// <summary>
/// Event handler for HtmlWeb.PreHandleDocumentHandler. Occurs before a HTML document is handled
/// </summary>
protected void OnPreHandleDocument(HtmlDocument document)
{
SaveHtmlDocument(document);
}
/// <summary>
/// Assembles the Post data and attaches to the request object
/// </summary>
private void AddPostDataTo(HttpWebRequest request)
{
string payload = FormElements.AssemblePostPayload();
byte[] buff = Encoding.UTF8.GetBytes(payload.ToCharArray());
request.ContentLength = buff.Length;
request.ContentType = "application/x-www-form-urlencoded";
System.IO.Stream reqStream = request.GetRequestStream();
reqStream.Write(buff, 0, buff.Length);
}
/// <summary>
/// Add cookies to the request object
/// </summary>
private void AddCookiesTo(HttpWebRequest request)
{
if (Cookies != null && Cookies.Count > 0)
{
request.CookieContainer.Add(Cookies);
}
}
/// <summary>
/// Saves cookies from the response object to the local CookieCollection object
/// </summary>
private void SaveCookiesFrom(HttpWebResponse response)
{
if (response.Cookies.Count > 0)
{
if (Cookies == null) Cookies = new CookieCollection();
Cookies.Add(response.Cookies);
}
}
/// <summary>
/// Saves the form elements collection by parsing the HTML document
/// </summary>
private void SaveHtmlDocument(HtmlDocument document)
{
_htmlDoc = document;
FormElements = new FormElementCollection(_htmlDoc);
}
}
FormElementCollection Class:
/// <summary>
/// Represents a combined list and collection of Form Elements.
/// </summary>
public class FormElementCollection : Dictionary<string, string>
{
/// <summary>
/// Constructor. Parses the HtmlDocument to get all form input elements.
/// </summary>
public FormElementCollection(HtmlDocument htmlDoc)
{
var inputs = htmlDoc.DocumentNode.Descendants("input");
foreach (var element in inputs)
{
string name = element.GetAttributeValue("name", "undefined");
string value = element.GetAttributeValue("value", "");
if (!name.Equals("undefined")) Add(name, value);
}
}
/// <summary>
/// Assembles all form elements and values to POST. Also html encodes the values.
/// </summary>
public string AssemblePostPayload()
{
StringBuilder sb = new StringBuilder();
foreach (var element in this)
{
string value = System.Web.HttpUtility.UrlEncode(element.Value);
sb.Append("&" + element.Key + "=" + value);
}
return sb.ToString().Substring(1);
}
}
It is not easy to log in and download the WebPages. I have recently have had the same issue. If you find a solution aisde from this one, please provide it.
Now what I did was I used Selenium with PhantomJS. With Selenium I can interact with the webbrowser of my choice.
Also the Browser class does not use Html Agility Pack, which is a third party library available through nuget.
I want to refer you to this question, where I have created an entire example of how to use Selenium and how to download the HtmlDocument and filter out the necessary information using xpath.
I managed to get it working, using BrowserSession, and a modified webClient:
First off Change the _htmlDoc to Public to access the document Nodes:
public class BrowserSession
{
private bool _isPost;
public string previous_Response { get; private set; }
public HtmlDocument _htmlDoc { get; private set; }
}
Secondly Add this method to BrowserSession:
public void DownloadCookieProtectedFile(string url, string Filename)
{
using (CookieAwareWebClient wc = new CookieAwareWebClient())
{
wc.Cookies = Cookies;
wc.DownloadFile(url, Filename);
}
}
//rest of BrowserSession
Third Add this Class Somewhere, Which allows passing the cookies from BrowserSession to the WebClient.
public class CookieAwareWebClient : WebClient
{
public CookieCollection Cookies = new CookieCollection();
private void AddCookiesTo(HttpWebRequest request)
{
if (Cookies != null && Cookies.Count > 0)
{
request.CookieContainer.Add(Cookies);
}
}
protected override WebRequest GetWebRequest(Uri address)
{
WebRequest request = base.GetWebRequest(address);
HttpWebRequest webRequest = request as HttpWebRequest;
if (webRequest != null)
{
if (webRequest.CookieContainer == null) webRequest.CookieContainer = new CookieContainer();
AddCookiesTo(webRequest);
}
return request;
}
}
This should Give you the ability to use BrowserSession Like you normally would, And when you need to get a file that you can only access If your logged in, Simply Call BrowserSession.DownloadCookieProtectedFile() As if it were a WebClient, Only Set the Cookies like so:
Using(wc = new CookieAwareWebClient())
{
wc.Cookies = BrowserSession.Cookies
//Download with WebClient As normal
wc.DownloadFile();
}
Related
I couldn't find a satisfying answer to that question.
I'm trying to migrate my code from .NET Framework into .NET Core, and I ran into problems with my .ashx files.
Basically there are compile errors (Because I need to update my used NuGet Packages).
but even before that, I'm trying to figure out - Can I even use .ashx files inside .NET Core proejct?
Are there any changes I need to know about? Or should it be identical to .NET Framework (Besides packages).
Thanks!
Simply put, no.
Long answer:
All .ashx files have a background class that inherits either from System.Web.IHttpHandler or System.Web.IHttpAsyncHandler.
Now, if your handlers are programmed well, most of what you have to do is declare those interfaces in .NET Core:
namespace System.Web
{
[System.Runtime.InteropServices.ComVisible(true)]
public interface IAsyncResult
{
bool IsCompleted { get; }
System.Threading.WaitHandle AsyncWaitHandle { get; }
// Return value:
// A user-defined object that qualifies or contains information about an asynchronous
// operation.
object AsyncState { get; }
// Return value:
// true if the asynchronous operation completed synchronously; otherwise, false.
bool CompletedSynchronously { get; }
}
public interface IHttpAsyncHandler : IHttpHandler
{
// Return value:
// An System.IAsyncResult that contains information about the status of the process.
IAsyncResult BeginProcessRequest(HttpContext context, AsyncCallback cb, object extraData);
// An System.IAsyncResult that contains information about the status of the process.
void EndProcessRequest(IAsyncResult result);
}
public interface IHttpHandler
{
// true if the System.Web.IHttpHandler instance is reusable; otherwise, false.
bool IsReusable { get; }
void ProcessRequest(Microsoft.AspNetCore.Http.HttpContext context);
}
}
And automatically rename System.Web.HttpContext into Microsoft.AspNetCore.Http.HttpContext.
Then you need to remove all references to System.Web.HttpContext.Current (which you wouldn't need to do if you programmed your handlers properly).
If that is a lot of work, my answer here
https://stackoverflow.com/a/40029302/155077
suggest a dirty workaround (basically, spoofing System.Web.HttpContext.Current).
Then, all you need to do is search to root path recursively for the ashx-file-paths, parse those files with regex for the corresponding classes (or do it manually if you have few) and add them as endpoint in .NET Core. Either you can just instanciate the class (singleton IsReusable = true), or you can write a handler that creates a new instance of class, and then calls ProcessRequest in said class.
You'll need to fix all compilation problems, though.
You can also parse the paths in the source project once, and generate the middleware/endpoint-injection code programmatically once.
That's probably the best option you have for large projects.
Though, you are in luck if you have ashx-handlers.
But don't just start just yet.
If you have aspx-pages or ascx-controls, you are definitely out of luck, and don't need to start with the .ashx-handlers in the first place - unless your aspx/ascx usage is so tiny, you can replace them fairly quickly.
Still, if you just need to port a lot of plain raw .ashx handlers producing json/xml, this method can work excellently.
Oh, also note that you need to change context.Response.OutputStream with context.Response.Body.
You might also have to change everything to async (if it isn't already), or you need to allow synchronous methods in the .NET-Core application startup.
Here's how to extract data:
namespace FirstTestApp
{
public class EndpointData
{
public string Path;
public string Class;
public string Language;
}
public static class HandlerFinder
{
private static System.Collections.Generic.List<EndpointData> FindAllHandlers()
{
string searchPath = #"D:\username\Documents\Visual Studio 2017\TFS\COR-Basic-V4\Portal\Portal";
searchPath = #"D:\username\Documents\Visual Studio 2017\TFS\COR-Basic\COR-Basic\Basic\Basic";
string[] ashxFiles = System.IO.Directory.GetFiles(searchPath, "*.ashx", System.IO.SearchOption.AllDirectories);
int searchPathLength = searchPath.Length;
System.Collections.Generic.List<EndpointData> ls = new System.Collections.Generic.List<EndpointData>();
foreach (string ashxFile in ashxFiles)
{
string input = #"<%# WebHandler Language=""VB"" Class=""Portal.Web.Data"" %>";
input = System.IO.File.ReadAllText(ashxFile, System.Text.Encoding.UTF8);
// http://regexstorm.net/tester
// <%# WebHandler Language="VB" Class="AmCharts.JSON" %>
System.Text.RegularExpressions.Match mClass = System.Text.RegularExpressions.Regex.Match(input, #"\<\%\#\s*WebHandler.*Class=""(?<class>.*?)"".*?\%\>", System.Text.RegularExpressions.RegexOptions.Multiline | System.Text.RegularExpressions.RegexOptions.IgnoreCase);
System.Text.RegularExpressions.Match mLanguage = System.Text.RegularExpressions.Regex.Match(input, #"\<\%\#\s*WebHandler.*Language=""(?<language>.*?)"".*?\%\>", System.Text.RegularExpressions.RegexOptions.Multiline | System.Text.RegularExpressions.RegexOptions.IgnoreCase);
string endpointPath = ashxFile.Substring(searchPathLength).Replace(System.IO.Path.DirectorySeparatorChar, '/');
string classname = mClass.Groups["class"].Value;
string languagename = mLanguage.Groups["language"].Value;
ls.Add(new EndpointData() { Path = endpointPath, Class = classname, Language = languagename });
}
System.Console.WriteLine("Finished " + ls.Count.ToString());
string json = System.Text.Json.JsonSerializer.Serialize(ls, ls.GetType(),
new System.Text.Json.JsonSerializerOptions
{
IncludeFields = true,
WriteIndented = true
});
// System.IO.File.WriteAllText(#"D:\PortalHandlers.json", json);
System.IO.File.WriteAllText(#"D:\BasicHandlers.json", json);
return ls; // 137 Portal + 578 Basic
}
public static int Test(string[] args)
{
FindAllHandlers();
return 0;
} // End Sub Test
} // End Class HandlerFinder
} // End Namespace FirstTestApp
And here's the Middleware with example handler "Ping.ashx".
namespace FirstTestApp
{
using Microsoft.AspNetCore.Http;
public class AshxOptions
{ }
public interface IHttpHandler
{
// true if the System.Web.IHttpHandler instance is reusable; otherwise, false.
bool IsReusable { get; }
void ProcessRequest(Microsoft.AspNetCore.Http.HttpContext context);
}
public class Ping
: IHttpHandler
{
public void ProcessRequest(Microsoft.AspNetCore.Http.HttpContext context)
{
context.Response.StatusCode = 200;
context.Response.ContentType = "text/plain; charset=utf-8;";
context.Response.WriteAsync(System.Environment.MachineName).Wait();
}
public bool IsReusable
{
get
{
return false;
}
} // End Property IsReusable
}
/// <summary>
/// Some Middleware that exposes something
/// </summary>
public class AshxMiddleware<T>
where T:IHttpHandler, new()
{
private readonly Microsoft.AspNetCore.Http.RequestDelegate m_next;
private readonly AshxOptions m_options;
private readonly ConnectionFactory m_factory;
/// <summary>
/// Creates a new instance of <see cref="AshxMiddleware"/>.
/// </summary>
public AshxMiddleware(
Microsoft.AspNetCore.Http.RequestDelegate next
, Microsoft.Extensions.Options.IOptions<object> someOptions
, ConnectionFactory db_factory
)
{
if (next == null)
{
throw new System.ArgumentNullException(nameof(next));
}
if (someOptions == null)
{
throw new System.ArgumentNullException(nameof(someOptions));
}
if (db_factory == null)
{
throw new System.ArgumentNullException(nameof(db_factory));
}
m_next = next;
m_options = (AshxOptions)someOptions.Value;
m_factory = db_factory;
}
/// <summary>
/// Processes a request.
/// </summary>
/// <param name="httpContext"></param>
/// <returns></returns>
public async System.Threading.Tasks.Task InvokeAsync(Microsoft.AspNetCore.Http.HttpContext context)
{
if (context == null)
{
throw new System.ArgumentNullException(nameof(context));
}
// string foo = XML.NameValueCollectionHelper.ToXml(httpContext.Request);
// System.Console.WriteLine(foo);
// httpContext.Response.StatusCode = 200;
// The Cache-Control is per the HTTP 1.1 spec for clients and proxies
// If you don't care about IE6, then you could omit Cache-Control: no-cache.
// (some browsers observe no-store and some observe must-revalidate)
context.Response.Headers["Cache-Control"] = "no-cache, no-store, must-revalidate, max-age=0";
// Other Cache-Control parameters such as max-age are irrelevant
// if the abovementioned Cache-Control parameters (no-cache,no-store,must-revalidate) are specified.
// Expires is per the HTTP 1.0 and 1.1 specs for clients and proxies.
// In HTTP 1.1, the Cache-Control takes precedence over Expires, so it's after all for HTTP 1.0 proxies only.
// If you don't care about HTTP 1.0 proxies, then you could omit Expires.
context.Response.Headers["Expires"] = "-1, 0, Tue, 01 Jan 1980 1:00:00 GMT";
// The Pragma is per the HTTP 1.0 spec for prehistoric clients, such as Java WebClient
// If you don't care about IE6 nor HTTP 1.0 clients
// (HTTP 1.1 was introduced 1997), then you could omit Pragma.
context.Response.Headers["pragma"] = "no-cache";
// On the other hand, if the server auto-includes a valid Date header,
// then you could theoretically omit Cache-Control too and rely on Expires only.
// Date: Wed, 24 Aug 2016 18:32:02 GMT
// Expires: 0
context.Response.StatusCode = 200;
T foo = new T();
foo.ProcessRequest(context);
} // End Task AshxMiddleware
} // End Class AshxMiddleware
} // End Namespace
Then you still need a generic Endpoint Middleware, here:
namespace FirstTestApp
{
using FirstTestApp;
using Microsoft.AspNetCore.Builder;
// using Microsoft.AspNetCore.Diagnostics.HealthChecks;
// using Microsoft.Extensions.Diagnostics.HealthChecks;
// using Microsoft.AspNetCore.Routing;
// using Microsoft.Extensions.DependencyInjection;
/// <summary>
/// Provides extension methods for <see cref="IEndpointRouteBuilder"/> to add health checks.
/// </summary>
public static class GenericEndpointRouteBuilderExtensions
{
/// <summary>
/// Adds a health checks endpoint to the <see cref="IEndpointRouteBuilder"/> with the specified template.
/// </summary>
/// <param name="endpoints">The <see cref="IEndpointRouteBuilder"/> to add the health checks endpoint to.</param>
/// <param name="pattern">The URL pattern of the health checks endpoint.</param>
/// <returns>A convention routes for the health checks endpoint.</returns>
public static IEndpointConventionBuilder MapEndpoint<T>(
this Microsoft.AspNetCore.Routing.IEndpointRouteBuilder endpoints,
string pattern)
{
if (endpoints == null)
{
throw new System.ArgumentNullException(nameof(endpoints));
}
return MapEndpointCore<T>(endpoints, pattern, null);
}
/// <summary>
/// Adds a health checks endpoint to the <see cref="IEndpointRouteBuilder"/> with the specified template and options.
/// </summary>
/// <param name="endpoints">The <see cref="IEndpointRouteBuilder"/> to add the health checks endpoint to.</param>
/// <param name="pattern">The URL pattern of the health checks endpoint.</param>
/// <param name="options">A <see cref="SomeOptions"/> used to configure the health checks.</param>
/// <returns>A convention routes for the health checks endpoint.</returns>
public static IEndpointConventionBuilder MapEndpoint<T>(
this Microsoft.AspNetCore.Routing.IEndpointRouteBuilder endpoints,
string pattern,
object options)
{
if (endpoints == null)
{
throw new System.ArgumentNullException(nameof(endpoints));
}
if (options == null)
{
throw new System.ArgumentNullException(nameof(options));
}
return MapEndpointCore<T>(endpoints, pattern, options);
}
private static IEndpointConventionBuilder MapEndpointCore<T>(
Microsoft.AspNetCore.Routing.IEndpointRouteBuilder endpoints
, string pattern
, object? options)
{
if (ReferenceEquals(typeof(T), typeof(SQLMiddleware)))
{
if (endpoints.ServiceProvider.GetService(typeof(ConnectionFactory)) == null)
{
throw new System.InvalidOperationException("Dependency-check failed. Unable to find service " +
nameof(ConnectionFactory) + " in " +
nameof(GenericEndpointRouteBuilderExtensions.MapEndpointCore)
+ ", ConfigureServices(...)"
);
}
}
object[] args = options == null ? System.Array.Empty<object>() :
new object[]
{
Microsoft.Extensions.Options.Options.Create(options)
}
;
Microsoft.AspNetCore.Http.RequestDelegate pipeline = endpoints.CreateApplicationBuilder()
.UseMiddleware<T>(args)
.Build();
return endpoints.Map(pattern, pipeline).WithDisplayName(typeof(T).AssemblyQualifiedName!);
}
} // End Class
}
And then you can set up the Endpoint with:
app.MapEndpoint<AshxMiddleware<Ping>>("/ping.ashx", new AshxOptions() { });
Keep in mind, middleware is async, and your ASHX-Handlers probably aren't.
That's gonna be a source of work-intensive problems (or deadlocks, if you just use .Wait() like in my example).
I have a webservice client application. My webservice provider inform me to change the signature method from sha1 to sha256 which is in requests header part. Currently i have a CustomSendFilter class, and securing the outgoing messages with the function below. How can i convert to sha 256? I searched but havent found a definite solution yet.
public override void SecureMessage(SoapEnvelope envelope, Security security)
{
X509SecurityToken signatureToken;
signatureToken = new X509SecurityToken(CertificateManager.ClientCertificate);
security.Tokens.Add(signatureToken);
MessageSignature sig = new MessageSignature(signatureToken);
security.Elements.Add(sig);
security.Timestamp.TtlInSeconds = 60;
Logging.AddToLog(envelope.Envelope.InnerText);
}
You can set the signature algorithm to be used via MessageSignature.SignedInfo.SignatureMethod.
Unfortunately the .NET Framework may not have built-in support for http://www.w3.org/2001/04/xmldsig-more#rsa-sha256 at the time of this writing, but the following code can be used to fix that (credits go to https://gist.github.com/sneal/f35de432115b840c4c1f ):
/// <summary>
/// SignatureDescription impl for http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
/// </summary>
public class RSAPKCS1SHA256SignatureDescription : SignatureDescription
{
/// <summary>
/// Registers the http://www.w3.org/2001/04/xmldsig-more#rsa-sha256 algorithm
/// with the .NET CrytoConfig registry. This needs to be called once per
/// appdomain before attempting to validate SHA256 signatures.
/// </summary>
public static void Register()
{
CryptoConfig.AddAlgorithm(
typeof(RSAPKCS1SHA256SignatureDescription),
"http://www.w3.org/2001/04/xmldsig-more#rsa-sha256");
}
/// <summary>
/// .NET calls this parameterless ctor
/// </summary>
public RSAPKCS1SHA256SignatureDescription()
{
KeyAlgorithm = "System.Security.Cryptography.RSACryptoServiceProvider";
DigestAlgorithm = "System.Security.Cryptography.SHA256Managed";
FormatterAlgorithm = "System.Security.Cryptography.RSAPKCS1SignatureFormatter";
DeformatterAlgorithm = "System.Security.Cryptography.RSAPKCS1SignatureDeformatter";
}
public override AsymmetricSignatureDeformatter CreateDeformatter(AsymmetricAlgorithm key)
{
var asymmetricSignatureDeformatter =
(AsymmetricSignatureDeformatter)CryptoConfig.CreateFromName(DeformatterAlgorithm);
asymmetricSignatureDeformatter.SetKey(key);
asymmetricSignatureDeformatter.SetHashAlgorithm("SHA256");
return asymmetricSignatureDeformatter;
}
public override AsymmetricSignatureFormatter CreateFormatter(AsymmetricAlgorithm key)
{
var asymmetricSignatureFormatter =
(AsymmetricSignatureFormatter)CryptoConfig.CreateFromName(FormatterAlgorithm);
asymmetricSignatureFormatter.SetKey(key);
asymmetricSignatureFormatter.SetHashAlgorithm("SHA256");
return asymmetricSignatureFormatter;
}
}
Thank you for answering. I have a CustomSendFilter class for securing request as below. Where should i register the algorithm? I registered before calling the webservice function and inside the SecureMessage function below as well but didn't work.
public class CustomSendFilter : SendSecurityFilter
{
private string serviceDescription;
public CustomSendFilter(SecurityPolicyAssertion parentAssertion , string serviceDescription)
: base(parentAssertion.ServiceActor, true)
{
this.serviceDescription = serviceDescription;
}
public override SoapFilterResult ProcessMessage(SoapEnvelope envelope)
{
SoapFilterResult result = base.ProcessMessage(envelope);
Logging.SaveSoapEnvelope(envelope, SoapMessageDirection.Out , serviceDescription);
return result;
}
public override void SecureMessage(SoapEnvelope envelope, Security security)
{
X509SecurityToken signatureToken;
signatureToken = new X509SecurityToken(CertificateManager.ClientCertificate);
RSAPKCS1SHA256SignatureDescription.Register();
security.Tokens.Add(signatureToken);
MessageSignature sig = new MessageSignature(signatureToken);
security.Elements.Add(sig);
security.Timestamp.TtlInSeconds = 60;
Logging.AddToLog(envelope.Envelope.InnerText);
}
}
I am trying to Send Error reports with hockeyapp without having to let the whole app crash and burn. I dont think the HockeyApp.WPF library has this capability, so I started to mess around with implementing my own CrashHandler.
This quickly got confusing and very hackey. Does anyone have any code examples for this? At my current rate I will end up reproducing half of the HockeyApp Library, so I would appreciate some help.
I am not posting my code because I don't think it will help and its too much.
Edit: now I will post a shortened version of code that doesnt seem to work:
private static void HandleException(Exception e) {
try {
string crashID = Guid.NewGuid().ToString();
String filename = String.Format("{0}{1}.log", CrashFilePrefix, crashID);
CrashLogInformation logInfo = new CrashLogInformation() {
PackageName = Application.Current.GetType().Namespace,
Version = ((HockeyClient)HockeyClient.Current).VersionInfo,
OperatingSystem = Environment.OSVersion.Platform.ToString(),
Windows = Environment.OSVersion.Version.ToString() + Environment.OSVersion.ServicePack,
Manufacturer = "",
Model = ""
};
ICrashData crash = ((HockeyClient)HockeyClient.Current).CreateCrashData(e);
using (FileStream stream = File.Create(Path.Combine(GetPathToHockeyCrashes(), filename))) {
crash.Serialize(stream);
stream.Flush();
}
}
catch (Exception ex) {
((HockeyClient)HockeyClient.Current).HandleInternalUnhandledException(ex);
}
}
private static string GetPathToHockeyCrashes() {
string path = Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData);
if (!path.EndsWith("\\")) { path += "\\"; }
path += "HockeyCrashes\\";
if (!Directory.Exists(path)) { Directory.CreateDirectory(path); }
return path;
}
private struct CrashLogInformation {
/// <summary>
/// name of app package
/// </summary>
public string PackageName;
/// <summary>
/// version of app
/// </summary>
public string Version;
/// <summary>
/// os
/// </summary>
public string OperatingSystem;
/// <summary>
/// device manufacturer
/// </summary>
public string Manufacturer;
/// <summary>
/// device model
/// </summary>
public string Model;
/// <summary>
/// product id of app
/// </summary>
public string ProductID;
/// <summary>
/// windows phone version
/// </summary>
public string WindowsPhone;
/// <summary>
/// windows version
/// </summary>
public string Windows;
}
I was able to make a post after formatting the logs as described in the documentation for the crashes/upload endpoint(http://support.hockeyapp.net/kb/api/api-crashes#-u-post-api-2-apps-app_id-crashes-upload-u-).
Although I ended up hitting "crashes/", which from my understanding is different from crashes/upload(Therefore this is a solution that is hitting an undocumented endpoint).
private static readonly string HOCKEYUPLOADURL = #"https://rink.hockeyapp.net/api/2/apps/{0}/crashes/";
private static async Task SendDataAsync(String log, String userID, String contact, String description) {
string rawData = "";
rawData += "raw=" + Uri.EscapeDataString(log);
if (userID != null) {
rawData += "&userID=" + Uri.EscapeDataString(userID);
}
if (contact != null) {
rawData += "&contact=" + Uri.EscapeDataString(contact);
}
if (description != null) {
rawData += "&description=" + Uri.EscapeDataString(description);
}
WebRequest request = WebRequest.Create(new Uri(String.Format(HOCKEYUPLOADURL, HOCKEYAPPID)));
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
using (Stream stream = await request.GetRequestStreamAsync()) {
byte[] byteArray = Encoding.UTF8.GetBytes(rawData);
stream.Write(byteArray, 0, rawData.Length);
stream.Flush();
}
try {
using (WebResponse response = await request.GetResponseAsync()) { }
}
catch (WebException e) {
WriteLocalLog(e, "HockeyApp SendDataAsync failed");
}
}
I'm trying to find a way to wrap a a json response with a callback for jsonp. The problem I am running into though is I don't want to use the 'JsonResult' class to construct the response as it wraps it with its own object where as if I return the model directly it is properly serialized to json.
So far I have tried using the a 'ActionFilter' but couldn't find out how I could wrap the result after the action executed.
Any direction at all would be appreciated
I've been down this road, and can offer the following code which encapsulates JsonP calls into an ActionResult, adds a method to your Controller which allows you to prioritize the type of ActionResult you want, and a couple of extension methods to glue it all together. The only requirement is to consistently name your callback parameter, so it can be reliably culled from the Request.
First, the derived ActionResult:
using System;
using System.Text;
using System.Web;
using System.Web.Mvc;
using System.Web.Script.Serialization;
namespace CL.Enterprise.Components.Web.Mvc
{
/// <summary>
/// Extension of JsonResult to handle JsonP requests
/// </summary>
public class JsonPResult : ActionResult
{
private JavaScriptSerializer _jser = new JavaScriptSerializer();
public Encoding ContentEncoding { get; set; }
public string ContentType { get; set; }
public object Data { get; set; }
public string JsonCallback { get; set; }
public JsonPResult() { }
/// <summary>
/// Package and return the result
/// </summary>
public override void ExecuteResult(ControllerContext Context)
{
//Context.IsChildAction
if (Context == null)
{
throw new ArgumentNullException("Context");
}
JsonCallback = Context.HttpContext.Request["callback"];
if (string.IsNullOrEmpty(JsonCallback))
{
JsonCallback = Context.HttpContext.Request["jsoncallback"];
}
if (string.IsNullOrEmpty(JsonCallback))
{
throw new ArgumentNullException("JsonP callback parameter required for JsonP response.");
}
HttpResponseBase CurrentResponse = Context.HttpContext.Response;
if (!String.IsNullOrEmpty(ContentType))
{
CurrentResponse.ContentType = ContentType;
}
else
{
CurrentResponse.ContentType = "application/json";
}
if (ContentEncoding != null)
{
CurrentResponse.ContentEncoding = ContentEncoding;
}
if (Data != null)
{
CurrentResponse.Write(string.Format("{0}({1});", JsonCallback, _jser.Serialize(Data)));
}
}
}
}
Next, the Controller extension methods:
using System.IO;
using System.Web.Mvc;
namespace CL.Enterprise.Components.Web.Mvc
{
/// <summary>
/// Extension methods for System.Web.Mvc.Controller
/// </summary>
public static class ContollerExtensions
{
/// <summary>
/// Method to return a JsonPResult
/// </summary>
public static JsonPResult JsonP(this Controller controller, object data)
{
JsonPResult result = new JsonPResult();
result.Data = data;
//result.ExecuteResult(controller.ControllerContext);
return result;
}
/// <summary>
/// Get the currently named jsonp QS parameter value
/// </summary>
public static string GetJsonPCallback(this Controller controller)
{
return
controller.ControllerContext.RequestContext.HttpContext.Request.QueryString["callback"] ??
controller.ControllerContext.RequestContext.HttpContext.Request.QueryString["jsoncallback"] ??
string.Empty;
}
}
}
Finally, add this method to your Controller:
/// <summary>
/// Gets an ActionResult, either as a jsonified string, or rendered as normally
/// </summary>
/// <typeparam name="TModel">Type of the Model class</typeparam>
/// <param name="UsingJson">Do you want a JsonResult?</param>
/// <param name="UsingJsonP">Do you want a JsonPResult? (takes priority over UsingJson)</param>
/// <param name="Model">The model class instance</param>
/// <param name="RelativePathToView">Where in this webapp is the view being requested?</param>
/// <returns>An ActionResult</returns>
public ActionResult GetActionResult<T>(T Model, bool UsingJsonP, bool UsingJson, string RelativePathToView)
{
string ViewAsString =
this.RenderView<T>(
RelativePathToView,
Model
);
if (UsingJsonP) //takes priority
{
string Callback = this.GetJsonPCallback();
JsonPResult Result = this.JsonP(ViewAsString.Trim());
return Result;
}
if (UsingJson)//secondary
{
return Json(ViewAsString.Trim(), JsonRequestBehavior.AllowGet);
}
return View(Model); //tertiary
}
I'm building an app in .NET and C#, and I'd like to cache some of the results by using attributes/annotations instead of explicit code in the method.
I'd like a method signature that looks a bit like this:
[Cache, timeToLive=60]
String getName(string id, string location)
It should make a hash based on the inputs, and use that as the key for the result.
Naturally, there'd be some config file telling it how to actually put in memcached, local dictionary or something.
Do you know of such a framework?
I'd even be interested in one for Java as well
With CacheHandler in Microsoft Enterprise Library you can easily achieve this.
For instance:
[CacheHandler(0, 30, 0)]
public Object GetData(Object input)
{
}
would make all calls to that method cached for 30 minutes. All invocations gets a unique cache-key based on the input data and method name so if you call the method twice with different input it doesn't get cached but if you call it >1 times within the timout interval with the same input then the method only gets executed once.
I've added some extra features to Microsoft's code:
My modified version looks like this:
using System;
using System.Diagnostics;
using System.IO;
using System.Reflection;
using System.Runtime.Remoting.Contexts;
using System.Text;
using System.Web;
using System.Web.Caching;
using System.Web.UI;
using Microsoft.Practices.EnterpriseLibrary.Common.Configuration;
using Microsoft.Practices.Unity.InterceptionExtension;
namespace Middleware.Cache
{
/// <summary>
/// An <see cref="ICallHandler"/> that implements caching of the return values of
/// methods. This handler stores the return value in the ASP.NET cache or the Items object of the current request.
/// </summary>
[ConfigurationElementType(typeof (CacheHandler)), Synchronization]
public class CacheHandler : ICallHandler
{
/// <summary>
/// The default expiration time for the cached entries: 5 minutes
/// </summary>
public static readonly TimeSpan DefaultExpirationTime = new TimeSpan(0, 5, 0);
private readonly object cachedData;
private readonly DefaultCacheKeyGenerator keyGenerator;
private readonly bool storeOnlyForThisRequest = true;
private TimeSpan expirationTime;
private GetNextHandlerDelegate getNext;
private IMethodInvocation input;
public CacheHandler(TimeSpan expirationTime, bool storeOnlyForThisRequest)
{
keyGenerator = new DefaultCacheKeyGenerator();
this.expirationTime = expirationTime;
this.storeOnlyForThisRequest = storeOnlyForThisRequest;
}
/// <summary>
/// This constructor is used when we wrap cached data in a CacheHandler so that
/// we can reload the object after it has been removed from the cache.
/// </summary>
/// <param name="expirationTime"></param>
/// <param name="storeOnlyForThisRequest"></param>
/// <param name="input"></param>
/// <param name="getNext"></param>
/// <param name="cachedData"></param>
public CacheHandler(TimeSpan expirationTime, bool storeOnlyForThisRequest,
IMethodInvocation input, GetNextHandlerDelegate getNext,
object cachedData)
: this(expirationTime, storeOnlyForThisRequest)
{
this.input = input;
this.getNext = getNext;
this.cachedData = cachedData;
}
/// <summary>
/// Gets or sets the expiration time for cache data.
/// </summary>
/// <value>The expiration time.</value>
public TimeSpan ExpirationTime
{
get { return expirationTime; }
set { expirationTime = value; }
}
#region ICallHandler Members
/// <summary>
/// Implements the caching behavior of this handler.
/// </summary>
/// <param name="input"><see cref="IMethodInvocation"/> object describing the current call.</param>
/// <param name="getNext">delegate used to get the next handler in the current pipeline.</param>
/// <returns>Return value from target method, or cached result if previous inputs have been seen.</returns>
public IMethodReturn Invoke(IMethodInvocation input, GetNextHandlerDelegate getNext)
{
lock (input.MethodBase)
{
this.input = input;
this.getNext = getNext;
return loadUsingCache();
}
}
public int Order
{
get { return 0; }
set { }
}
#endregion
private IMethodReturn loadUsingCache()
{
//We need to synchronize calls to the CacheHandler on method level
//to prevent duplicate calls to methods that could be cached.
lock (input.MethodBase)
{
if (TargetMethodReturnsVoid(input) || HttpContext.Current == null)
{
return getNext()(input, getNext);
}
var inputs = new object[input.Inputs.Count];
for (int i = 0; i < inputs.Length; ++i)
{
inputs[i] = input.Inputs[i];
}
string cacheKey = keyGenerator.CreateCacheKey(input.MethodBase, inputs);
object cachedResult = getCachedResult(cacheKey);
if (cachedResult == null)
{
var stopWatch = Stopwatch.StartNew();
var realReturn = getNext()(input, getNext);
stopWatch.Stop();
if (realReturn.Exception == null && realReturn.ReturnValue != null)
{
AddToCache(cacheKey, realReturn.ReturnValue);
}
return realReturn;
}
var cachedReturn = input.CreateMethodReturn(cachedResult, input.Arguments);
return cachedReturn;
}
}
private object getCachedResult(string cacheKey)
{
//When the method uses input that is not serializable
//we cannot create a cache key and can therefore not
//cache the data.
if (cacheKey == null)
{
return null;
}
object cachedValue = !storeOnlyForThisRequest ? HttpRuntime.Cache.Get(cacheKey) : HttpContext.Current.Items[cacheKey];
var cachedValueCast = cachedValue as CacheHandler;
if (cachedValueCast != null)
{
//This is an object that is reloaded when it is being removed.
//It is therefore wrapped in a CacheHandler-object and we must
//unwrap it before returning it.
return cachedValueCast.cachedData;
}
return cachedValue;
}
private static bool TargetMethodReturnsVoid(IMethodInvocation input)
{
var targetMethod = input.MethodBase as MethodInfo;
return targetMethod != null && targetMethod.ReturnType == typeof (void);
}
private void AddToCache(string key, object valueToCache)
{
if (key == null)
{
//When the method uses input that is not serializable
//we cannot create a cache key and can therefore not
//cache the data.
return;
}
if (!storeOnlyForThisRequest)
{
HttpRuntime.Cache.Insert(
key,
valueToCache,
null,
System.Web.Caching.Cache.NoAbsoluteExpiration,
expirationTime,
CacheItemPriority.Normal, null);
}
else
{
HttpContext.Current.Items[key] = valueToCache;
}
}
}
/// <summary>
/// This interface describes classes that can be used to generate cache key strings
/// for the <see cref="CacheHandler"/>.
/// </summary>
public interface ICacheKeyGenerator
{
/// <summary>
/// Creates a cache key for the given method and set of input arguments.
/// </summary>
/// <param name="method">Method being called.</param>
/// <param name="inputs">Input arguments.</param>
/// <returns>A (hopefully) unique string to be used as a cache key.</returns>
string CreateCacheKey(MethodBase method, object[] inputs);
}
/// <summary>
/// The default <see cref="ICacheKeyGenerator"/> used by the <see cref="CacheHandler"/>.
/// </summary>
public class DefaultCacheKeyGenerator : ICacheKeyGenerator
{
private readonly LosFormatter serializer = new LosFormatter(false, "");
#region ICacheKeyGenerator Members
/// <summary>
/// Create a cache key for the given method and set of input arguments.
/// </summary>
/// <param name="method">Method being called.</param>
/// <param name="inputs">Input arguments.</param>
/// <returns>A (hopefully) unique string to be used as a cache key.</returns>
public string CreateCacheKey(MethodBase method, params object[] inputs)
{
try
{
var sb = new StringBuilder();
if (method.DeclaringType != null)
{
sb.Append(method.DeclaringType.FullName);
}
sb.Append(':');
sb.Append(method.Name);
TextWriter writer = new StringWriter(sb);
if (inputs != null)
{
foreach (var input in inputs)
{
sb.Append(':');
if (input != null)
{
//Diffrerent instances of DateTime which represents the same value
//sometimes serialize differently due to some internal variables which are different.
//We therefore serialize it using Ticks instead. instead.
var inputDateTime = input as DateTime?;
if (inputDateTime.HasValue)
{
sb.Append(inputDateTime.Value.Ticks);
}
else
{
//Serialize the input and write it to the key StringBuilder.
serializer.Serialize(writer, input);
}
}
}
}
return sb.ToString();
}
catch
{
//Something went wrong when generating the key (probably an input-value was not serializble.
//Return a null key.
return null;
}
}
#endregion
}
}
Microsoft deserves most credit for this code. We've only added stuff like caching at request level instead of across requests (more useful than you might think) and fixed some bugs (e.g. equal DateTime-objects serializing to different values).
To do exactly what you are describing, i.e. writing
public class MyClass {
[Cache, timeToLive=60]
string getName(string id, string location){
return ExpensiveCall(id, location);
}
}
// ...
MyClass c = new MyClass();
string name = c.getName("id", "location");
string name_again = c.getName("id", "location");
and having only one invocation of the expensive call and without needing to wrap the class with some other code (f.x. CacheHandler<MyClass> c = new CacheHandler<MyClass>(new MyClass());) you need to look into an Aspect Oriented Programming framework. Those usually work by rewriting the byte-code, so you need to add another step to your compilation process - but you gain a lot of power in the process. There are many AOP-frameworks, but PostSharp for .NET and AspectJ are among the most popular. You can easily Google how to use those to add the caching-aspect you want.