I'm trying to web-crawl a site that uses php sessions via cookies. It is a good-ol' Squirrelmail webmail server.
I saw a couple of posts like this one, but it's not working for me.
When reaching the part when the cookies are sent by the host, I tried to retrieve the cookies using:
HttpWebResponse rs = (HttpWebResponse)rq.GetResponse();
CookieCollection cc = new CookieCollection();
cc.Add(rs.Cookies);
But rs.Cookies comes empty. However, there are set-cookie headers on the response, which I try to use as a guide to build actual cookies, like this:
for (int i = 0; i < rs.Headers.Count; i++)
{
if (rs.Headers.Keys[i].ToLower().Contains("cookie"))
{
string val = rs.Headers[i];
string[] vv = val.Split(";=,".ToCharArray());
Cookie co = new Cookie(vv[0], vv[1]);
// I know this is not the cleanest way to do it
// I've tried to manually set different values for
// co.Domain, co.Path and co.HttpOnly, just to get a working
// example. I tried different alternatives, but it doesn't
// seem to change anything
cc.Add(co);
}
}
Next, I send the cookies to request the next page, which is nothing but a frameset. The fact that I reach the frameset means I've been successfully authenticated and the session cookie is working. However, when I request one of the frames, I get an authentication-error web page. I've done my research, and the cookies do not change in the meantime. What may be going wrong?
Some may wonder why I'm trying to access webmail when there is pop/smtp to do a cleaner job. The answer is this is just a first example to learn the basics, I don't really care what the site is as long as I can successfully manage sessions.
I don't think posting all the code is a good idea yet, since it is a bit messy, and long: I planned to clean it once it worked (I'll post it, if you think it's worth the confusion). Moreover, I think I may have a conceptual error related to the frames, that may be the key to solve the problem.
Related
I'm trying to implement punchout catalogs on our eComm site. Honestly, the documentation for cXML is a mess and all the code examples are in javascript and/or VB.Net (I use C# and would rather not have to try and translate). Does anyone out there have examples or samples of how to receive the PunchOutSetupRequest XML and then send out the PunchOutSetupResponse XML using C#? I've been unable to find anything on the interwebs (I've been looking for two days now)...
I'm hoping I can just do this inside an ActionResult (vs. a 'launch page' as suggested).
I'm a complete noob at punchouts and could really use some help here. The bosses are being pretty pushy, so any assistance would be greatly appreciated. Suggestions as to how to make this work would also be much appreciated.
I apologize to all for the vagueness of the question (request).
This isn't trivial, but this should get you started.
You'll need 3 generic handlers (.ashx): Setup, Start, and Order....
Setup and Order will receive HTTP Post with content-type of "text/xml". Look at HttpRequest.InputStream if needed to get the XML into a string. From there, look at LINQ-to-XML to dig out the data you want. Your HTTP Response to both of these will also be content-type "text/xml" and UTF8 encoded, returning the CXML as documented...use LINQ-to-XML to produce that.
The Setup handler will need to validate credentials and return a URL with a unique QueryString token pointing to the Start handler. Do not expect session persistence between Setup and Start, because they're not from the same caller. This handler will need to create an application object for the token and associated data you extracted from the cXML.
The Start handler will be called as a simple GET, and will need to match the token in the QueryString to the appropriate application object, copy that data to the session, and then do a response.redirect to whatever page in your site you want the buyer to land on.
Once they populate their cart with some things, and are ready to check out, you'll take them to a page that has an embedded form (not to be confused with an ASP.Net form that posts back to your server) and a submit button (again, not an ASP.Net button). From your Setup handler, you captured a URL to point this form's Post, and within the form you'll have a hidden input tag with the UTF8 encoded CXML Punchout Order injected as the value produced with LINQ-to-XML. Highly recommend Base64 encoding that value to avoid ASP.Net messing with the tags it contains during rendering, and naming the hidden input "cxml-base64" per the documentation. The result is the form is client-side POSTed to your customer's server instead of yours, and their server will extract the CXML Punchout Order and that ends your visitor's session.
The Order handler will receive a CXML OrderRequest and just like Setup, you'll dump that to a string and then use LINQ-to-XML to parse it and act upon it. Again you'll get credentials to verify, possibly a credit card to process, and the order items, ship-to, etc. Note that the OrderRequest may not contain all the items that were in the Punchout Order, because the system on your customer's side may remove items or even change item quantities before submitting the final OrderRequest to you. The OrderRequest could come back to you after the Punchout Order is posted to them in a matter of minutes, days, weeks, or never...don't bother storing the cart data in hopes of matching it to the order later.
Last note...the buyer may be experiencing your site in an iframe embedded in their web-based procurement UI, so design accordingly.
If you need more info, reply to this and I'll get back.
Update...Additional considerations:
Discuss with the buyer how they want fault handling to flow, particularly with orders, because you have a choice. 1) exhaustively evaluate everything in the CXML you receive and return response codes other than 200 if anything is wrong, or 2) always return a 200 Success and deal with any issues out of band or by generating a ConfirmationRequest that rejects the order. My experience is that a mix of the two works best. Certainly you should throw a non-200 if the credentials fail, but you may not want (or be able) to run a credit card or validate stock availability inline. Your buyer's system may not be able to cope with dozens of possible faults, and/or may not show your fault messages to the user for them to make corrections. I've seen systems that will flat-out discard any non-200 response code and just blindly retry the submission repeatedly on an interval for hours or days until it gives up on a sanity check, while others will handle response codes within certain ranges differently than others, for example a 4xx invokes a retry, while a 5xx is treated as fatal. Remember that Setup and Order are not coming directly from the user...their procurement system is generating those internally.
Update...answering the comment about how to test things...
You'd use the same method as you will for generating outbound ConfirmationRequest, ShipNoticeRequest, and InvoiceDetailRequest, all of which generally are produced on your side after receiving an OrderRequest from your customer's procurement system.
Start with Linq-To-XML for an example of crafting your outgoing cXML (Creating XML Trees section). Combine that example with this bit of code:
StringBuilder output = new StringBuilder();
XmlWriterSettings objXmlWriterSettings = new XmlWriterSettings();
objXmlWriterSettings.Indent = true;
objXmlWriterSettings.NewLineChars = Environment.NewLine;
objXmlWriterSettings.NewLineHandling = NewLineHandling.Replace;
objXmlWriterSettings.NewLineOnAttributes = false;
objXmlWriterSettings.Encoding = new UTF8Encoding();
using (XmlWriter objXmlWriter = XmlWriter.Create(output, objXmlWriterSettings)) {
XElement root = new XElement("Root",
new XElement("Child", "child content")
);
root.Save(objXmlWriter);
}
Console.WriteLine(output.ToString());
So at this point the StringBuilder (output) has your whole cXML, and you need to POST it someplace. Your Web Application project, started with F5 and a default.aspx page will be listening on localhost and some port (you'll see that in the URL it opens). Separately, perhaps using VS Express for Desktop, you have the above code in a console app that you can run to do the Post using something like this:
Net.HttpWebRequest objRequest = Net.WebRequest.Create("http://localhost:12345/handler.ashx");
objRequest.Method = "POST";
objRequest.UserAgent = "Some User Agent";
objRequest.ContentLength = output.Length;
objRequest.ContentType = "text/xml";
IO.StreamWriter objStreamWriter = new IO.StreamWriter(objRequest.GetRequestStream, System.Text.Encoding.ASCII);
objStreamWriter.Write(output);
objStreamWriter.Flush();
objStreamWriter.Close();
Net.WebResponse objWebResponse = objRequest.GetResponse();
XmlReaderSettings objXmlReaderSettings = new XmlReaderSettings();
objXmlReaderSettings.DtdProcessing = DtdProcessing.Ignore;
XmlReader objXmlReader = XmlReader.Create(objWebResponse.GetResponseStream, objXmlReaderSettings);
// Pipes the stream to a higher level stream reader with the required encoding format.
IO.MemoryStream objMemoryStream2 = new IO.MemoryStream();
XmlWriter objXmlWriter2 = XmlWriter.Create(objMemoryStream2, objXmlWriterSettings);
objXmlWriter2.WriteNode(objXmlReader, true);
objXmlWriter2.Flush();
objXmlWriter2.Close();
objWebResponse.Close();
// Reset current position to the beginning so we can read all below.
objMemoryStream2.Position = 0;
StreamReader objStreamReader = new StreamReader(objMemoryStream2, Encoding.UTF8);
Console.WriteLine(objStreamReader.ReadToEnd());
objStreamReader.Close();
Since your handler should be producing cXML you'll see that spat out in the console. If it pukes, you'll get a big blob of debug mess in the console, which of course will help you fix whatever is broken.
pardon the verbosity in the variable names, done to try to make things clear.
Why does the property SessionID on the Session-object in an ASP.NET-page change between requests?
I have a page like this:
...
<div>
SessionID: <%= SessionID %>
</div>
...
And the output keeps changing every time I hit F5, independent of browser.
This is the reason
When using cookie-based session state, ASP.NET does not allocate storage for session data until the Session object is used. As a result, a new session ID is generated for each page request until the session object is accessed. If your application requires a static session ID for the entire session, you can either implement the Session_Start method in the application's Global.asax file and store data in the Session object to fix the session ID, or you can use code in another part of your application to explicitly store data in the Session object.
http://msdn.microsoft.com/en-us/library/system.web.sessionstate.httpsessionstate.sessionid.aspx
So basically, unless you access your session object on the backend, a new sessionId will be generated with each request
EDIT
This code must be added on the file Global.asax. It adds an entry to the Session object so you fix the session until it expires.
protected void Session_Start(Object sender, EventArgs e)
{
Session["init"] = 0;
}
There is another, more insidious reason, why this may occur even when the Session object has been initialized as demonstrated by Cladudio.
In the Web.config, if there is an <httpCookies> entry that is set to requireSSL="true" but you are not actually using HTTPS: for a specific request, then the session cookie is not sent (or maybe not returned, I'm not sure which) which means that you end up with a brand new session for each request.
I found this one the hard way, spending several hours going back and forth between several commits in my source control, until I found what specific change had broken my application.
In my case I figured out that the session cookie had a domain that included www. prefix, while I was requesting page with no www..
Adding www. to the URL immediately fixed the problem. Later I changed cookie's domain to be set to .mysite.com instead of www.mysite.com.
my problem was that we had this set in web.config
<httpCookies httpOnlyCookies="true" requireSSL="true" />
this means that when debugging in non-SSL (the default), the auth cookie would not get sent back to the server. this would mean that the server would send a new auth cookie (with a new session) for every request back to the client.
the fix is to either set requiressl to false in web.config and true in web.release.config or turn on SSL while debugging:
Using Neville's answer (deleting requireSSL = true, in web.config) and slightly modifying Joel Etherton's code, here is the code that should handle a site that runs in both SSL mode and non SSL mode, depending on the user and the page (I am jumping back into code and haven't tested it on SSL yet, but expect it should work - will be too busy later to get back to this, so here it is:
if (HttpContext.Current.Response.Cookies.Count > 0)
{
foreach (string s in HttpContext.Current.Response.Cookies.AllKeys)
{
if (s == FormsAuthentication.FormsCookieName || s.ToLower() == "asp.net_sessionid")
{
HttpContext.Current.Response.Cookies[s].Secure = HttpContext.Current.Request.IsSecureConnection;
}
}
}
Another possibility that causes the SessionID to change between requests, even when Session_OnStart is defined and/or a Session has been initialized, is that the URL hostname contains an invalid character (such as an underscore). I believe this is IE specific (not verified), but if your URL is, say, http://server_name/app, then IE will block all cookies and your session information will not be accessible between requests.
In fact, each request will spin up a separate session on the server, so if your page contains multiple images, script tags, etc., then each of those GET requests will result in a different session on the server.
Further information: http://support.microsoft.com/kb/316112
My issue was with a Microsoft MediaRoom IPTV application. It turns out that MPF MRML applications don't support cookies; changing to use cookieless sessions in the web.config solved my issue
<sessionState cookieless="true" />
Here's a REALLY old article about it:
Cookieless ASP.NET
in my case it was because I was modifying session after redirecting from a gateway in an external application, so because I was using IP instead on localhost in that page url it was actually considered different website with different sessions.
In summary
pay more attention if you are debugging a hosted application on IIS instead of IIS express and mixing your machine http://Ip and http://localhost in various pages
In my case this was happening a lot in my development and test environments. After trying all of the above solutions without any success I found that I was able to fix this problem by deleting all session cookies. The web developer extension makes this very easy to do. I mostly use Firefox for testing and development, but this also happened while testing in Chrome. The fix also worked in Chrome.
I haven't had to do this yet in the production environment and have not received any reports of people not being able to log in. This also only seemed to happen after making the session cookies to be secure. It never happened in the past when they were not secure.
Update: this only started happening after we changed the session cookie to make it secure. I've determined that the exact issue was caused by there being two or more session cookies in the browser with the same path and domain. The one that was always the problem was the one that had an empty or null value. After deleting that particular cookie the issue was resolved. I've also added code in Global.asax.cs Sessin_Start method to check for this empty cookie and if so set it's expiration date to something in the past.
HttpCookieCollection cookies = Response.Cookies;
for (int i = 0; i < cookies.Count; i++)
{
HttpCookie cookie = cookies.Get(i);
if (cookie != null)
{
if ((cookie.Name == "ASP.NET_SessionId" || cookie.Name == "ASP.NET_SessionID") && String.IsNullOrEmpty(cookie.Value))
{
//Try resetting the expiration date of the session cookie to something in the past and/or deleting it.
//Reset the expiration time of the cookie to one hour, one minute and one second in the past
if (Response.Cookies[cookie.Name] != null)
Response.Cookies[cookie.Name].Expires = DateTime.Today.Subtract(new TimeSpan(1, 1, 1));
}
}
}
This was changing for me beginning with .NET 4.7.2 and it was due to the SameSite property on the session cookie. See here for more info: https://devblogs.microsoft.com/aspnet/upcoming-samesite-cookie-changes-in-asp-net-and-asp-net-core/
The default value changed to "Lax" and started breaking things. I changed it to "None" and things worked as expected.
Be sure that you do not have a session timeout that is very short, and also make sure that if you are using cookie based sessions that you are accepting the session.
The FireFox webDeveloperToolbar is helpful at times like this as you can see the cookies set for your application.
Session ID resetting may have many causes. However any mentioned above doesn't relate to my problem. So I'll describe it for future reference.
In my case a new session created on each request resulted in infinite redirect loop. The redirect action takes place in OnActionExecuting event.
Also I've been clearing all http headers (also in OnActionExecuting event using Response.ClearHeaders method) in order to prevent caching sites on client side. But that method clears all headers including informations about user's session, and consequently all data in Temp storage (which I was using later in program). So even setting new session in Session_Start event didn't help.
To resolve my problem I ensured not to remove the headers when a redirection occurs.
Hope it helps someone.
I ran into this issue a different way. The controllers that had this attribute [SessionState(SessionStateBehavior.ReadOnly)] were reading from a different session even though I had set a value in the original session upon app startup. I was adding the session value via the _layout.cshtml (maybe not the best idea?)
It was clearly the ReadOnly causing the issue because when I removed the attribute, the original session (and SessionId) would stay in tact. Using Claudio's/Microsoft's solution fixed it.
I'm on .NET Core 2.1 and I'm well aware that the question isn't about Core. Yet the internet is lacking and Google brought me here so hoping to save someone a few hours.
Startup.cs
services.AddCors(o => o.AddPolicy("AllowAll", builder =>
{
builder
.WithOrigins("http://localhost:3000") // important
.AllowCredentials() // important
.AllowAnyMethod()
.AllowAnyHeader(); // obviously just for testing
}));
client.js
const resp = await fetch("https://localhost:5001/api/user", {
method: 'POST',
credentials: 'include', // important
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify(data)
})
Controllers/LoginController.cs
namespace WebServer.Controllers
{
[Route("api/[controller]")]
[ApiController]
public class UserController : ControllerBase
{
[HttpPost]
public IEnumerable<string> Post([FromBody]LoginForm lf)
{
string prevUsername = HttpContext.Session.GetString("username");
Console.WriteLine("Previous username: " + prevUsername);
HttpContext.Session.SetString("username", lf.username);
return new string[] { lf.username, lf.password };
}
}
}
Notice that the session writing and reading works, yet no cookies seem to be passed to the browser. At least I couldn't find a "Set-Cookie" header anywhere.
I have this following code:
if (Request.UrlReferrer != null)
{
if (Request.UrlReferrer.PathAndQuery.ToLowerInvariant() == "/test/content.htm")
{
postbacklink = Request.UrlReferrer.AbsoluteUri.Replace("/TEST/Content.htm", "/Testing.aspx?") + Request.QueryString;
}
else
{
postbacklink = Request.UrlReferrer.AbsoluteUri;
}
}
ExtendedLoanView.PostbackLink = postbacklink;
Now this page can be accessed by two different locations. Which means this code:
postbacklink = Request.UrlReferrer.AbsoluteUri.Replace("/TEST/Content.htm", "/Test.aspx?") + Request.QueryString;
can only work with one page (Test.aspx) and is hard coded. So in IE7 Request.UrlReferrer shows me this:
Request.UrlReferrer = {http://Testing:12345/PPP/Content.htm}
Whereas in IE8+ I am getting this value:
Request.UrlReferrer = {http://Testing:12345/PPP/TestingPage.aspx?Name=Xyz&Address=123 YYY
How should I solve this issue? Its been bugging me for past month.
I would definitely advice not to base your logic on request information (not anymore than user entered values). The thing is that it will be different across browsers, and it is really hackable.
If you still need to pass information from client to server, make sure to have those validated. If you need those to stay in sync and have valid information, do not rely on what the browsers give you, but set it yourself and then take it from a place in the request you did set (for example, a hidden input, a control, a variable on the viewstate, or whatever allows the technology you're using).
Most sites handle the situation you're trying to solve by passing the destination URL in the URL itself, in a query parameter. For example:
http://www.example.com/Login.aspx?returnUrl=/TEST/content.htm
EDIT: I do realize that everything you send to the client is very hackable anyway, but if you set it yourself, it's easier for you to validate that it hasn't been tampered with. An example is the ViewState validation methods.
This is a follow-up to Choosing a Connection String based on kind of request for which I got no answer and what I thought worked doesn't.
I have a webservice that needs to choose a specific connection string based on the user calling it from a browser or from a client application.
I tried:
HttpContext.Current != null? ConnectionStrings["Website"].ConnectionString : ConnectionStrings["Client"].ConnectionString
but realized that at some point even if I'm using the client application, there is some HttpContext (if someone can explain why it'd be great) but the Browser field under Request is "Unknown". So, then I tried:
if ( HttpContext.Current != null )
{
if ( HttpContext.Current.Request.Browser != "Unknown" )
{
//browser connection string here
}
else
//client app connection string here
}
else
//client app connection string here
This worked wonders when debugging, but on testing environment it still points to Browser connection string even when calling from the client app, as if at some point the Browser isn't "Unknown" ...
Is there a MUCH easier/simpler way to do this? The way I'm doing it seems really ugly.
I'm quite desperate at the moment as I have no idea why this is happening..
Rather than detecting and switching on the browser type, consider these two suggestions:
Add Custom Request Headers
In your various callers, define a new custom header in your Http request.
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(url);
webRequest.Headers.Add("CallerType", "ClientApp"); // "Browser", etc.
Then you know exactly and reliably what type of client is calling. This would be hard to get wrong, and couldn't be spoofed/mistaken.
Include The Caller Type in the QueryString
myService.asmx?BrowserType=1
Add a simple new querystring parameter to your .asmx webmethod. This will work just the same in a controlled environment, but if other users/developers get it wrong, or malform the expected values, you'd have to take other measures to correct/handle.
Both allow you to easily determine the connString on the incoming value. Perhaps the absense of a modifier/header, you could assume a default. Your sample question has 2 basic outcomes, and either suggested solution will be easy to extend (browser, client app, iPhone, whathaveyou).
I set cookies based on the referral links and they all start with the same letters, lets say "google", but they end with _xxx, _yyy, _zzz or whatever is the reference.
Now, when I try to get the cookies later, I have the problem that I don't want to check for all of the different cookies, I would like to check for all cookies that start with "google" and based on that I will start a script that goes on with processing.
if (Request.Cookies("google"))
{
run other stuff
}
Any idea how I can add StartWith or something to it? I am a newbie, so not really that into C# yet.
Thanks in advance,
Pat
Well.. HttpRequest.Cookies is a collection. So use LINQ:
var qry = from cookieName in Request.Cookies.Keys
where cookieName.StartsWith("google")
select cookieName;
foreach(var item in qry)
{
// get the cookie and deal with it.
var cookie = Request.Cookies[item];
}
Bottom line: you can't get away from iterating over the entire cookie collection. But you can do it easily using LINQ.
You have to check all the cookies if you want to find ones with a certain suffix (Randolpho's answer will work).
It's not a particularly good idea to do it that way. The problem is that the more cookies you create, the more overhead you put on the server and connection. Say you have 10 cookies: google_aaa, google_bbb, etc. Each request will send all 10 cookies to your server (this includes requests for images, css, etc.
You're better off using a single cookie which is some sort of key to all the information stored on your server. Something like this:
var cookie = Cookies["google"];
if(cookie!=null)
{
// cookie.Value is a unique key for this user. Lookup this
// key in your database or other store to find out the
// information about this user.
}
If you prefer, use lambda expression, this way
var cookie = Request.Cookies.AllKeys.FirstOrDefault(s => s.Contains("yourName"));
Hope this help!