I am trying to set value of referer parameter of httpwebrequest header, but it is giving error-
Function that I am using (C#):
webRequest.Headers.Set(HttpRequestHeader.Referer, "http://www.microsoft.com");
Error:
Invalid parameters. A required parameter is not found or contains invalid value.
Try this method from MSDN If you post your code I can better help you out.
HttpWebRequest myHttpWebRequest=(HttpWebRequest)WebRequest.Create(myUri);
// Set referer property to http://www.microsoft.com .
myHttpWebRequest.Referer="http://www.microsoft.com";
// Assign the response object of 'HttpWebRequest' to a 'HttpWebResponse' variable.
HttpWebResponse myHttpWebResponse=(HttpWebResponse)myHttpWebRequest.GetResponse();
// Display the contents of the page to the console.
Stream streamResponse=myHttpWebResponse.GetResponseStream();
StreamReader streamRead = new StreamReader( streamResponse );
Char[] readBuffer = new Char[256];
int count = streamRead.Read( readBuffer, 0, 256 );
Console.WriteLine("\nThe contents of HTML page are.......");
while (count > 0)
{
String outputData = new String(readBuffer, 0, count);
Console.Write(outputData);
count = streamRead.Read(readBuffer, 0, 256);
}
Console.WriteLine("\nHTTP Request Headers :\n\n{0}",myHttpWebRequest.Headers);
Console.WriteLine("\nHTTP Response Headers :\n\n{0}",myHttpWebResponse.Headers);
streamRead.Close();
streamResponse.Close();
// Release the response object resources.
myHttpWebResponse.Close();
Console.WriteLine("Referer to the site is:{0}",myHttpWebRequest.Referer);
Here is typical use:
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("");
req.Referer = "http://www.google.com";
Show your code.
There are some reserved HTTP Headers which you can't set via webRequest.Headers.Set Referer is one of them.
For these there is always a special property to set.
In your case webRequest.Referer = "http://google.com" will do.
#kedar kamthe, you can't put HttpRequestHeader.Referer in there, because that is expecting a value, I believe, but not sure... I think you wanted to use this:
webRequest.Headers.Set("Referer", "http://www.microsoft.com");
But you should use it as Chris Ballance's example. That should work, but if that doesn't work, just try as I show above ;).
You should put your code here, so we can see what is wrong.
Related
Using the WebClient class I can get the title of a website easily enough:
WebClient x = new WebClient();
string source = x.DownloadString(s);
string title = Regex.Match(source,
#"\<title\b[^>]*\>\s*(?<Title>[\s\S]*?)\</title\>",
RegexOptions.IgnoreCase).Groups["Title"].Value;
I want to store the URL and the page title. However when following a link such as:
http://tinyurl.com/dbysxp
I'm clearly going to want to get the Url I'm redirected to.
QUESTIONS
Is there a way to do this using the WebClient class?
How would I do it using HttpResponse and HttpRequest?
If I understand the question, it's much easier than people are saying - if you want to let WebClient do all the nuts and bolts of the request (including the redirection), but then get the actual response URI at the end, you can subclass WebClient like this:
class MyWebClient : WebClient
{
Uri _responseUri;
public Uri ResponseUri
{
get { return _responseUri; }
}
protected override WebResponse GetWebResponse(WebRequest request)
{
WebResponse response = base.GetWebResponse(request);
_responseUri = response.ResponseUri;
return response;
}
}
Just use MyWebClient everywhere you would have used WebClient. After you've made whatever WebClient call you needed to do, then you can just use ResponseUri to get the actual redirected URI. You'd need to add a similar override for GetWebResponse(WebRequest request, IAsyncResult result) too, if you were using the async stuff.
I know this is already an answered question, but this works pretty to me:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://tinyurl.com/dbysxp");
request.AllowAutoRedirect = false;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
string redirUrl = response.Headers["Location"];
response.Close();
//Show the redirected url
MessageBox.Show("You're being redirected to: "+redirUrl);
Cheers.! ;)
With an HttpWebRequest, you would set the AllowAutoRedirect property to false. When this happens, any response with a status code between 300-399 will not be automatically redirected.
You can then get the new url from the response headers and then create a new HttpWebRequest instance to the new url.
With the WebClient class, I doubt you can change it out-of-the-box so that it does not allow redirects. What you could do is derive a class from the WebClient class and then override the GetWebRequest and the GetWebResponse methods to alter the WebRequest/WebResponse instances that the base implementation returns; if it is an HttpWebRequest, then set the AllowAutoRedirect property to false. On the response, if the status code is in the range of 300-399, then issue a new request.
However, I don't know that you can issue a new request from within the GetWebRequest/GetWebResponse methods, so it might be better to just have a loop that executes with HttpWebRequest/HttpWebResponse until all the redirects are followed.
I got the Uri for the redirected page and the page contents.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(strUrl);
request.AllowAutoRedirect = true;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream dataStream = response.GetResponseStream();
strLastRedirect = response.ResponseUri.ToString();
StreamReader reader = new StreamReader(dataStream);
string strResponse = reader.ReadToEnd();
response.Close();
In case you are only interested in the redirect URI you can use this code:
public static string GetRedirectUrl(string url)
{
HttpWebRequest request = (HttpWebRequest) HttpWebRequest.Create(url);
request.AllowAutoRedirect = false;
using (HttpWebResponse response = HttpWebResponse)request.GetResponse())
{
return response.Headers["Location"];
}
}
The method will return
null - in case of no redirect
a relative url - in case of a redirect
Please note: The using statement (or a final response.close()) is essential. See MSDN Library for details. Otherwise you may run out of connections or get a timeout when executing this code multiple times.
HttpWebRequest.AllowAutoRedirect can be set to false. Then you'd have to manually http status codes in the 300 range.
// Create a new HttpWebRequest Object to the mentioned URL.
HttpWebRequest myHttpWebRequest=(HttpWebRequest)WebRequest.Create("http://www.contoso.com");
myHttpWebRequest.MaximumAutomaticRedirections=1;
myHttpWebRequest.AllowAutoRedirect=true;
HttpWebResponse myHttpWebResponse=(HttpWebResponse)myHttpWebRequest.GetResponse();
The WebClient class has an option to follow redirects. Set that option and you should be fine.
Ok this is really hackish, but the key is to use the HttpWebRequest and then set the AllowAutoRedirect property to true.
Here's a VERY hacked together example
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("http://tinyurl.com/dbysxp");
req.Method = "GET";
req.AllowAutoRedirect = true;
WebResponse response = req.GetResponse();
response.GetResponseStream();
Stream responseStream = response.GetResponseStream();
// Content-Length header is not trustable, but makes a good hint.
// Responses longer than int size will throw an exception here!
int length = (int)response.ContentLength;
const int bufSizeMax = 65536; // max read buffer size conserves memory
const int bufSizeMin = 8192; // min size prevents numerous small reads
// Use Content-Length if between bufSizeMax and bufSizeMin
int bufSize = bufSizeMin;
if (length > bufSize)
bufSize = length > bufSizeMax ? bufSizeMax : length;
StringBuilder sb;
// Allocate buffer and StringBuilder for reading response
byte[] buf = new byte[bufSize];
sb = new StringBuilder(bufSize);
// Read response stream until end
while ((length = responseStream.Read(buf, 0, buf.Length)) != 0)
sb.Append(Encoding.UTF8.GetString(buf, 0, length));
string source = sb.ToString();string title = Regex.Match(source,
#"\<title\b[^>]*\>\s*(?<Title>[\s\S]*?)\</title\>",RegexOptions.IgnoreCase).Groups["Title"].Value;
enter code here
I'm using Corona SDK to post data to my C# server:
headers["Content-Type"] = "application/x-www-form-urlencoded"
headers["Accept-Language"] = "en-US"
local body = "color=red&size=small"
local params = {}
params.headers = headers
params.body = body
network.request( host .. "/UpdateHand", "POST", nwListener, params )
I receive a message on the server:
Post["/UpdateHand"] = x =>
{
Console.WriteLine("Received ...");
return "Ok";
};
But when I check the data (when I put a breakpoint on it) I don't see where my data is locaded (i.e. the params.body or params.headers). How can I extract this information?
I should POST it correctly according to the documentation on Corona: http://docs.coronalabs.com/daily/api/library/network/request.html
The post data is in
this.Request.Body
If you have suitable type you can deserialize your data to it using model binding:
var x = this.Bind<YourType>();
There is a Nancy extension for this. You will need to include the namespace for it.
using Nancy.Extensions;
var text = Context.Request.Body.AsString();
I like how concise this is, part of Nancy's super-duper easy path.
But a word of caution! This method leaves the stream at the end, so subsequent calls will return empty string. To fix this, always reset the stream immediately afterwards, like so:
Request.Body.Seek(0, SeekOrigin.Begin);
Nancy 2.0 is supposed to correct this so that the stream position is reset by default.
https://github.com/NancyFx/Nancy/pull/2158
This actually works great:
var body = this.Request.Body;
int length = (int) body.Length; // this is a dynamic variable
byte[] data = new byte[length];
body.Read(data, 0, length);
Console.WriteLine(System.Text.Encoding.Default.GetString(data));
For Nancy 2.0.0, Request.Body is a Stream rather than a RequestStream, so doesn't have an AsString method. However, this seems to work:
using (var reqStream = RequestStream.FromStream(Request.Body))
{
var body = reqStream.AsString();
// ... do stuff with body
}
Ideally getting your post data could be accomplished with a simple Bind() call. However, I've seen inconsistent results when using a Bind in a post call such that I've resorted to using the scheme outlined above.
I've seen various discussions about Nancy Bind() working and not working... I've seen both with Post but cannot explain the inconsistency. Where I saw it function properly was where I could guarantee the body of the request was managed as follows:
var data = Encoding.ASCII.GetBytes (postData);
request.Method = "POST";
request.ContentType = "application/json";
request.ContentLength = data.Length;
using (var stream = request.GetRequestStream ()) {
stream.Write (data, 0, data.Length);
}
However, when sending data that should have been similarly handled (though I couldn't confirm) through WSO2 infrastructure (data serialized as a JSON event dictionary sent to a service proxy), Bind failed while the method above succeeded.
Trying to access an XML online, using HttpWebRequest and the following code:
HttpWebRequest webRequest = HttpWebRequest.Create("http://example.com/example.xml") as HttpWebRequest;
webRequest.CookieContainer = new CookieContainer();
HttpWebResponse webResponse = webRequest.GetResponse() as HttpWebResponse;
StreamReader sr = new StreamReader(webResponse.GetResponseStream(), Encoding.ASCII);
string data = sr.ReadToEnd();
When I browse the URL, I can see the XML but variable data contains the following:
<html><body><script>document.cookie='lllllll=e0b70895lllllll_e0b70895; path=/';window.location.href=window.location.href;</script></body></html>
I've checked webResponse.Cookies but it's empty.
How can I get past this using the webrequest and get the xml into the data variable?
All what you wrote is correct. The problem is in your case (but it's a good solution against the robots), the cookie added by Javascript and not in the HTTP response.
document.cookie='lllllll=e0b70895lllllll_e0b70895; path=/'
This line of JavaScript code sets the cookie. So it's needs to set in the code after this response. You can do that easily with the CookieContainer.Add() method.
window.location.href=window.location.href
This line of code just refresh the page, but if in the browser the cookie is already set, that's why you can get the response.
To get this cookie, you need to use regex, because I think the name of the cookie is dynamic too.
So you need to add something like this:
// Catch the cookie name and value with using regex, than remove the
// characters what we only need for the regex match.
string cookieName = Regex.Match(data, "'[a-z]*").Value.Remove(0, 1);
string cookieValue = Regex.Match(data, "=[a-zA-Z0-9]*").Value.Remove(0, 1);
webRequest.CookieContainer.Add(new Cookie(cookieName,cookieValue));
webResponse = webRequest.GetResponse() as HttpWebResponse;
StreamReader sr2 = new StreamReader(webResponse.GetResponseStream(), Encoding.ASCII);
string data = sr2.ReadToEnd();
I am creating a HttpWebRequest object from another aspx page to save the response stream to my data store. The Url I am using to create the HttpWebRequest object has querystring to render the correct output. When I browse to the page using any old browser it renders correctly. When I try to retrieve the output stream using the HttpWebResponse.GetResponseStream() it renders my built in error check.
Why would it render correctly in the browser, but not using the HttpWebRequest and HttpWebResponse objects?
Here is the source code:
Code behind of target page:
protected void PageLoad(object sender, EventsArgs e)
{
string output = string.Empty;
if(Request.Querystring["a"] != null)
{
//generate output
output = "The query string value is " + Request.QueryString["a"].ToString();
}
else
{
//generate message indicating the query string variable is missing
output = "The query string value was not found";
}
Response.Write(output);
}
Code behind of page creating HttpWebRequest object
string url = "http://www.mysite.com/mypage.aspx?a=1";
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(url)
//this if statement was missing from original example
if(User.Length > 0)
{
request.Credentials = new NetworkCredentials("myaccount", "mypassword", "mydomain");
request.PreAuthenticate = true;
}
request.UserAgent = Request.UserAgent;
HttpWebResponse response = (HttpWebResponse) request.GetResponse();
Stream resStream = response.GetResponseStream();
Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
StreamReader readStream = new StreamReader(resStream, encode, true, 2000);
int count = readStream.Read(read, 0, read.Length);
string str = Server.HtmlEncode(" ");
while (count > 0)
{
// Dumps the 256 characters on a string and displays the string to the console.
string strRead = new string(read, 0, count);
str = str.Replace(str, str + Server.HtmlEncode(strRead.ToString()));
count = readStream.Read(read, 0, 256);
}
// return what was found
result = str.ToString();
resStream.Close();
readStream.Close();
Update
#David McEwing - I am creating the HttpWebRequest with the full page name. The page is still generating the error output. I updated the code sample of the target page to demonstrate exactly what I am doing.
#Chris Lively - I am not redirecting to an error page, I generate a message indicating the query string value was not found. I updated the source code example.
Update 1:
I tried using Fiddler to trace the HttpWebRequest and it did not show up in the Web Sessions history window. Am I missing something in my source code to get a complete web request and response.
Update 2:
I did not include the following section of code in my example and it was culprit causing the issue. I was setting the Credentials property of the HttpWebRequest with a sevice account instead of my AD account which was causing the issue.
I updated my source code example
What webserver are you using? I can remember at one point in my past when doing something with IIS there was an issue where the redirect between http://example.com/ and http://example.com/default.asp dropped the query string.
Perhaps run Fiddler (or a protocol sniffer) and see if there is something happening that you aren't expecting.
Also check if passing in the full page name works. If it does the above is almost certainly the problem.
Optionally, you can try to use the AllowAutoRedirect property of the HttpRequestObject.
I need to replace the following line of code:
request.Credentials = new NetworkCredentials("myaccount", "mypassword", "mydomain");
with:
request.Credentials = System.Net.CredentialCache.DefaultNetworkCredentials;
I am writting a winforms c# 2.0 application that needs to put an XML file into a document library on SharePoint.
I want to use a WebService instead of using the object model (no sharepoint.dll to reference here)
I am currently using the http://webserver/site/_vti_bin/copy.asmx webservice.
Here is some code:
byte[] xmlByteArray;
using (MemoryStream memoryStream = new MemoryStream())
{
xmlDocument.Save(memoryStream);
xmlBytes = memoryStream.ToArray();
}
string[] destinationUrlArray = new string[] {"http://webserver/site/Doclib/UploadedDocument.xml"};
FieldInformation fieldInfo = new FieldInformation();
FieldInformation[] fields = { fieldInfo };
CopyResult[] resultsArray;
using (Copy copyService = new Copy())
{
copyService.Credentials = CredentialCache.DefaultCredentials;
copyService.Url = "http://webserver/site/_vti_bin/copy.asmx";
copyService.Timeout = 600000;
uint documentId = copyService.CopyIntoItems("", destinationUrlArray, fields, xmlByteArray, out resultsArray);
}
When this code runs, I get a single result in the resultsArray out parameter:
DestinationURL: "http://webserver/site/Doclib/UploadedDocument.xml"
ErrorCode: UnKnown
ErrorMessage: "Object reference not set to an instance of an object."
From my searching, I have found a couple of possible helps.
Microsoft TechNet -- "The copy.asmx copyintoitems will only work if the source and destination urls are in the same SPWebApplication (Site Collection)."
Microsoft Social -- "Object reference not set to an instance of an object
error occurs because of SharePoint not able to identified that particular property."
This leads me to believe my source url should be set to something, but what? This is originating from a client workstation and does not have a source URL.
Any help would be appricated.
hank you,
Keith
I know this is an old thread but it kept coming up as I was searching for a solution to the same problem.
Check Steve Curran's answer on this thread http://social.msdn.microsoft.com/Forums/en-SG/sharepointdevelopment/thread/833e38a8-f13c-490d-8ba7-b889b6b25e38. Looks like Basically the request fails because the destination url can't be resolved.
(Limitations of a new stackflow user - can't post more than one link. See my comment for the rest)
pat
SharePoint responds to a plain old HTTP PUT
Here is what is currently working:
WebRequest request = WebRequest.Create(“http://webserver/site/Doclib/UploadedDocument.xml”);
request.Credentials = CredentialCache.DefaultCredentials;
request.Method = "PUT";
byte[] buffer = new byte[1024];
using (Stream stream = request.GetRequestStream())
{
using (MemoryStream memoryStream = new MemoryStream())
{
dataFile.MMRXmlData.Save(memoryStream);
memoryStream.Seek(0, SeekOrigin.Begin);
for (int i = memoryStream.Read(buffer, 0, buffer.Length); i > 0;
i = memoryStream.Read(buffer, 0, buffer.Length))
{
stream.Write(buffer, 0, i);
}
}
}
WebResponse response = request.GetResponse();
response.Close();
So... Does anyone have an opinion as to if this "PUT" method is better in the SharePoint environment than using a built-in webservice?
Right now I would have to say the "PUT" method is better since it works and I could not get the WebService to work.
Keith
your code is fine, just use the destination url instead of an empty string. See below:
byte[] xmlByteArray;
using (MemoryStream memoryStream = new MemoryStream())
{
xmlDocument.Save(memoryStream);
xmlBytes = memoryStream.ToArray();
}
string destinationUrl = “http://webserver/site/Doclib/UploadedDocument.xml”
string[] destinationUrlArray = new string[] { destinationUrl };
FieldInformation fieldInfo = new FieldInformation();
FieldInformation[] fields = { fieldInfo };
CopyResult[] resultsArray;
using (Copy copyService = new Copy())
{
copyService.Credentials = CredentialCache.DefaultCredentials;
copyService.Url = "http://webserver/site/_vti_bin/copy.asmx";
copyService.Timeout = 600000;
uint documentId = copyService.CopyIntoItems(destinationUrl , destinationUrlArray, fields, xmlByteArray, out resultsArray);
}
I get the same message when I use the default credentials.
Try replacing them with this:
copyWebService.Credentials
= new NetworkCredential("Administrator", "pass", "MyDomain");
Here's some code I wrote awhile (i apologize, i've had to piece meal it together, but hopefully you get the point of it)
// Create a request using a URL that can receive a post.
WebRequest request = WebRequest.Create("http://sharepointsite/somefile.txt");
// Set the Method property of the request to POST.
request.Method = "PUT"
Stream dataStream;
// Set the ContentType property of the WebRequest.
request.ContentType = "multipart/form-data; charset=ISO-8859-1";
byte[] byteArray = File.ReadAllBytes(#"c:\somefile.txt");
// Set the ContentLength property of the WebRequest.
request.ContentLength = byteArray.Length;
// Get the request stream.
dataStream = request.GetRequestStream();
// Write the data to the request stream.
dataStream.Write(byteArray, 0, byteArray.Length);
// Close the Stream object.
dataStream.Close();
// Get the response.
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
HttpStatusCode statCode = response.StatusCode;
// Get the stream containing content returned by the server.
dataStream = response.GetResponseStream();
// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader(dataStream);
// Read the content.
string responseFromServer = reader.ReadToEnd();
// Clean up the streams.
reader.Close();
dataStream.Close();
response.Close();
I'm not sure if it will solve your problem but, when you reference the webservice, don't use the [site] part of the URL.
Try instead: http://[server]/_vti_bin/[webservice].
I'm not an expert in SP but I'm pretty sure the webservices belongs to the main server, not to an especific site.
Hope it helps.
I had a similiar problem, it turned out that the the client was configured to use NTLM security, but no NTLM header was attached.
I my case, becuase of the fact that I was using this code on the server-side of an ASP.NET applicaton, was to enable Windows authentication and set
identity impersonate="true"
in the server.web section.
if your sharepoint server is built on a farm,
Check your "Alternate Access Mapping" see if there is an entry:
yourwebserverurl intranet yourwebserverurl
if not, add it.
for my case, after adding this, the Copy service start working.
It probably due to farm load balance address resolve related.
I don't get it, why are you using Copy rather then UpdateListItems. Perhaps UpdateListItems will be a better match?