Develop Reference

C# - blocking GetRequestStream()

I'm trying to get some data from Aerospike into SOLR, but I'm having a little problem with the web requests to SOLR. I'm using c#-s HttpWebRequest for the web requests on multiple threads. After a few batches the process gets blocked.
I found out that the code stops at the data = request.GetRequestStream(); line, no exceptions, no messages, no timeouts it just waits and does nothing.
Here is a part of my code:
public static ConcurrentBag<String> docs = new ConcurrentBag<String>();
public static int count { get; set; }
<< data extraction into the "docs" container >>
if (count == 50)
{
string dataStr = "{";
string temp;
lock (docs)
{
while (!docs.IsEmpty)
{
docs.TryTake(out temp);
dataStr += temp;
}
}
dataStr = dataStr.ToString().Remove(dataStr.ToString().Length - 1) + "}";
count = 0;
string SOLRInsert = "http://192.168.23.28:8985/solr/test/update";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(SOLRInsert);
request.ContentType = "application/json";
request.Method = "POST";
byte[] dataBytes = Encoding.ASCII.GetBytes(dataStr);
request.ContentLength = dataBytes.Length;
request.Proxy = null;
Stream data = null;
try
{
data = request.GetRequestStream();
sw.Start();
data.Write(dataBytes, 0, dataBytes.Length);
sw.Stop();
//data.Close();
data.Dispose();
}
catch (Exception ex)
{
Console.Write(key.userKey.ToString() + "->" + ex.ToString() + "\r\n");
File.AppendAllText(#"import.log", key.userKey.ToString() + "->" + ex.ToString() + Environment.NewLine);
}
finally
{
if (data != null)
{
//data.Close();
data.Dispose();
}
}
}
Any help would be appreciated.

How to call add account method for tempoplugin with post in C#

At this url there is a description how to call tempoplugin with usage of post. In order to achieve it, I created following string:
also created following code for posting data:
public static string HTTP_POST(string Url, string Data, string userName = "", string password = "")
{
string Out = String.Empty;
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(Url);
try
{
if (userName != null && password != null)
{
req.ContentType = "application/json";
req.Method = "POST";
req.ProtocolVersion = HttpVersion.Version11;
string base64Credentials = GetEncodedCredentials(userName, password);
req.Headers.Add("Authorization", "Basic " + base64Credentials);
}
req.Timeout = 100000;
byte[] sentData = Encoding.UTF8.GetBytes(Data);
req.ContentLength = sentData.Length;
using (System.IO.Stream sendStream = req.GetRequestStream())
{
sendStream.Write(sentData, 0, sentData.Length);
sendStream.Close();
}
HttpWebResponse res = (HttpWebResponse)req.GetResponse();
System.IO.Stream ReceiveStream = res.GetResponseStream();
using (System.IO.StreamReader sr = new System.IO.StreamReader(ReceiveStream, Encoding.UTF8))
{
Char[] read = new Char[256];
int count = sr.Read(read, 0, 256);
while (count > 0)
{
String str = new String(read, 0, count);
Out += str;
count = sr.Read(read, 0, 256);
}
}
}
catch (ArgumentException ex)
{
Out = string.Format("HTTP_ERROR :: The second HttpWebRequest object has raised an Argument Exception as 'Connection' Property is set to 'Close' :: {0}", ex.Message);
}
catch (WebException ex)
{
Out = string.Format("HTTP_ERROR :: WebException raised! :: {0}", ex.Message);
}
catch (Exception ex)
{
Out = string.Format("HTTP_ERROR :: Exception raised! :: {0}", ex.Message);
}
return Out;
}
But still I receive response "The remote server returned an error: (404) Not Found". Does anybody knows what I missed?

I fount the solution. The problem lied in permissions on Jira. After double checking with admins my code worked perfectly.

no http-connection possible after occurrence of server error 503

I build a windows-mobile 6.5 application (based on cf 2.0) and have a problem with a special test case of one method. So I hope someone can give me an advice or has a helpful idea what the reason for this behaviour is...
The method is called continuous every 30 seconds from inside a thread, looks for files to be transferred via a HTTP request to a web server (jboss) and brings them on their way. The server url itself is under my control.
Everything works fine ... until I stop the web server and force an 503 server error. So far so good. But after restarting the web server, I would expect, that the next call of the transfer method will end in success - but it does not. Every further try ends in a timeout exception and I have to restart the application to make it work again.
So my question is: where is the problem, when I want to connect to an uri after an earlier try has failed with error 503? It seems, that there is something cached, but what the hell should it be?
Many thanks for every hint you have.
Juergen
public static Boolean HttpUploadFile2(string url, string file)
{
HttpWebRequest requestToServer = null;
WebResponse response = null;
try
{
Logger.writeToLogFileCom(string.Format("Uploading {0} to {1}", file, url));
requestToServer = (HttpWebRequest)WebRequest.Create(url);
requestToServer. Timeout = 40000;
string boundaryString = "----SSLBlaBla";
requestToServer.AllowWriteStreamBuffering = false;
requestToServer.Method = "POST";
requestToServer.ContentType = "multipart/form-data;
boundary=" + boundaryString;
requestToServer.KeepAlive = false;
ASCIIEncoding ascii = new ASCIIEncoding();
string boundaryStringLine = "\r\n--" + boundaryString + "\r\n";
byte[] boundaryStringLineBytes = ascii.GetBytes(boundaryStringLine);
string lastBoundaryStringLine = "\r\n--" + boundaryString + "--\r\n";
byte[] lastBoundaryStringLineBytes = ascii.GetBytes(lastBoundaryStringLine);
// Get the byte array of the myFileDescription content disposition
string myFileDescriptionContentDisposition = String.Format(
"Content-Disposition: form-data; name=\"{0}\"\r\n\r\n{1}",
"myFileDescription",
"A sample file description");
byte[] myFileDescriptionContentDispositionBytes
= ascii.GetBytes(myFileDescriptionContentDisposition);
string fileUrl = file;
// Get the byte array of the string part of the myFile content
// disposition
string myFileContentDisposition = String.Format(
"Content-Disposition: form-data;name=\"{0}\"; "
+ "filename=\"{1}\"\r\nContent-Type: {2}\r\n\r\n",
"myFile", Path.GetFileName(fileUrl), Path.GetExtension(fileUrl));
byte[] myFileContentDispositionBytes =
ascii.GetBytes(myFileContentDisposition);
FileInfo fileInfo = new FileInfo(fileUrl);
// Calculate the total size of the HTTP request
long totalRequestBodySize = boundaryStringLineBytes.Length * 2
+ lastBoundaryStringLineBytes.Length
+ myFileDescriptionContentDispositionBytes.Length
+ myFileContentDispositionBytes.Length
+ fileInfo.Length;
// And indicate the value as the HTTP request content length
requestToServer.ContentLength = totalRequestBodySize;
// Write the http request body directly to the server
using (Stream s = requestToServer.GetRequestStream())
{
//TIMEOUT OCCURED WHEN CALLING GetRequestStream
// Send the file description content disposition over to the server
s.Write(boundaryStringLineBytes, 0, boundaryStringLineBytes.Length);
s.Write(myFileDescriptionContentDispositionBytes, 0,
myFileDescriptionContentDispositionBytes.Length);
// Send the file content disposition over to the server
s.Write(boundaryStringLineBytes, 0, boundaryStringLineBytes.Length);
s.Write(myFileContentDispositionBytes, 0,
myFileContentDispositionBytes.Length);
// Send the file binaries over to the server, in 1024 bytes chunk
FileStream fileStream = new FileStream(fileUrl, FileMode.Open,
FileAccess.Read);
byte[] buffer = new byte[1024];
int bytesRead = 0;
Logger.writeToLogFileCom("writing data...");
while ((bytesRead = fileStream.Read(buffer, 0, buffer.Length)) != 0)
{
s.Write(buffer, 0, bytesRead);
} // end while
fileStream.Close();
Logger.writeToLogFileCom("... finished, File closed");
// Send the last part of the HTTP request body
s.Write(lastBoundaryStringLineBytes, 0, lastBoundaryStringLineBytes.Length);
Logger.writeToLogFileCom("... finished, File closed");
} // end using
// Grab the response from the server. WebException will be thrown
// when a HTTP OK status is not returned
Logger.writeToLogFileCom("lese Response");
response = requestToServer.GetResponse();
StreamReader responseReader = new StreamReader(response.GetResponseStream());
string replyFromServer = responseReader.ReadToEnd();
response.Close();
if (Regex.Split(Regex.Split(replyFromServer, "content\\:RESPONSE\"\\>")[1], "\\</span\\>")[0].Equals("OK"))
{
return true;
}
else
{
return false;
}
}
catch (Exception ex)
{
Logger.writeToLogFileCom("Fehler im HTML Sender");
Logger.writeToLogFileCom(ex.Message);
Logger.writeToLogFileCom(ex.StackTrace);
}
finally
{
try
{
if (response != null)
{
response.Close();
}
}
catch (Exception ex) { }
}
return false;
}

I solved the problem.
I added an additional try / catch block inside the finally clause to call getResponse in every situation.
finally
{
try { response = requestToServer.GetResponse(); }
catch (Exception ex) { }
[...]

HttpWebRequest getRequestStream hangs on multiple runs

I've written some code to send and read text from a listener. This runs fine on the 1st and 2nd exchange, but on the 3rd send there's a long delay between calling GetRequestStream() and the actual writing of the data.
I've disposed the outstream on the send side, as well as the stream reader, and the input stream on the read side as recommended here: Does anyone know why I receive an HttpWebRequest Timeout?
And it still hangs on the 3rd attempt to send info. It definitely seems to be hanging at GetRequestStrean() in SendMessage():
public void SendMessage(string message)
{
HttpWebRequest request;
string sendUrl;
sendUrl = "http://" + termIPAddress + ":" + sendPort + "/";
Uri uri = new Uri(sendUrl);
Console.WriteLine("http://" + termIPAddress + ":" + sendPort + "/");
ServicePoint servicePoint = ServicePointManager.FindServicePoint(uri);
servicePoint.BindIPEndPointDelegate = new BindIPEndPoint(BindIPEndPointCallback);
servicePoint.ConnectionLeaseTimeout = 300;
request = (HttpWebRequest)WebRequest.Create(sendUrl);
request.KeepAlive = false;
request.Method = "POST";
request.ProtocolVersion = HttpVersion.Version11;
request.ContentType = "application/x-www-form-urlencoded";
request.Headers.Add("SourceIP", localIPAddress);
request.Headers.Add("MachineName", localName);
requestStarted = true;
byte[] buffer = System.Text.Encoding.UTF8.GetBytes(message);
request.ContentLength = buffer.Length;
try
{
using (Stream output = request.GetRequestStream())
{
output.Write(buffer, 0, buffer.Length);
output.Close();
request = null;
}
}
catch(WebException wE)
{
Console.WriteLine(wE.Message);
}
}
And this is the read portion :
public string GetMessage()
{
Console.WriteLine("Entering actual listener");
string s;
string sourceIP;
NameValueCollection headerList;
HttpListenerContext context = terminalListener.GetContext();
HttpListenerRequest request = context.Request;
headerList = request.Headers;
sourceIP = headerList.GetValues("SourceIP")[0];
termName = headerList.GetValues("MachineName")[0];
termIPAddress = sourceIP;
using (System.IO.Stream body = request.InputStream)
{
System.Text.Encoding encoding = request.ContentEncoding;
using (System.IO.StreamReader reader = new System.IO.StreamReader(body, encoding))
{
s = reader.ReadToEnd();
body.Close();
reader.Close();
}
}
return termName + " : " + s;
}
I also tried to add an IP End Point bind but have to be honest, I don't fully understand this piece of the code:
private IPEndPoint BindIPEndPointCallback(ServicePoint servicePoint, IPEndPoint remoteEndPoint, int retryCount)
{
int portNumber = Convert.ToInt32(sendPort);
IPEndPoint IEP = new IPEndPoint(IPAddress.Parse(localIPAddress), 0);
Console.WriteLine(Convert.ToString(IEP));
return IEP;
}

You just forgot to call HttpWebRequest.GetResponse and therefore run out of connection limit.
So, you should change your code as follows:
try
{
using (Stream output = request.GetRequestStream())
output.Write(buffer, 0, buffer.Length);
var response = request.GetResponse() as HttpWebResponse;
//TODO: check response.StatusCode, etc.
}
catch(WebException wE)
{
Console.WriteLine(wE.Message);
}
Also, in some cases you might want to adjust default connection limit:
ServicePointManager.DefaultConnectionLimit
Or use persistent connections:
HttpWebRequest.KeepAlive

KeepAliveException when using HttpWebRequest.GetResponse

I am trying to POST an attachment to CouchDB using the HttpWebRequest. However, when I attempt "response = (HttpWebResponse)httpWebRequest.GetResponse();" I receive a WebException with the message "The underlying connection was closed: A connection that was expected to be kept alive was closed by the server."
I have found some articles stating that setting the keepalive to false and httpversion to 1.0 resolves the situation. I am finding that it does not yeilding the exact same error, plus I do not want to take that approach as I do not want to use the 1.0 version due to how it handles the connection.
Any suggestions or ideas are welcome. I'll try them all until one works!
public ServerResponse PostAttachment(Server server, Database db, Attachment attachment)
{
Stream dataStream;
HttpWebResponse response = null;
StreamReader sr = null;
byte[] buffer;
string json;
string boundary = "----------------------------" + DateTime.Now.Ticks.ToString("x");
string headerTemplate = "Content-Disposition: form-data; name=\"_attachments\"; filename=\"" + attachment.Filename + "\"\r\n Content-Type: application/octet-stream\r\n\r\n";
byte[] headerbytes = System.Text.Encoding.UTF8.GetBytes(headerTemplate);
byte[] boundarybytes = System.Text.Encoding.ASCII.GetBytes("\r\n--" + boundary + "\r\n");
HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create("http://" + server.Host + ":" +
server.Port.ToString() + "/" + db.Name + "/" + attachment.Document.Id);
httpWebRequest.ContentType = "multipart/form-data; boundary=" + boundary;
httpWebRequest.Method = "POST";
httpWebRequest.KeepAlive = true;
httpWebRequest.ContentLength = attachment.Stream.Length + headerbytes.Length + boundarybytes.Length;
if (!string.IsNullOrEmpty(server.EncodedCredentials))
httpWebRequest.Headers.Add("Authorization", server.EncodedCredentials);
if (!attachment.Stream.CanRead)
throw new System.NotSupportedException("The stream cannot be read.");
// Get the request stream
try
{
dataStream = httpWebRequest.GetRequestStream();
}
catch (Exception e)
{
throw new WebException("Failed to get the request stream.", e);
}
buffer = new byte[server.BufferSize];
int bytesRead;
dataStream.Write(headerbytes,0,headerbytes.Length);
attachment.Stream.Position = 0;
while ((bytesRead = attachment.Stream.Read(buffer, 0, buffer.Length)) > 0)
{
dataStream.Write(buffer, 0, bytesRead);
}
dataStream.Write(boundarybytes, 0, boundarybytes.Length);
dataStream.Close();
// send the request and get the response
try
{
response = (HttpWebResponse)httpWebRequest.GetResponse();
}
catch (Exception e)
{
throw new WebException("Invalid response received from server.", e);
}
// get the server's response json
try
{
dataStream = response.GetResponseStream();
sr = new StreamReader(dataStream);
json = sr.ReadToEnd();
}
catch (Exception e)
{
throw new WebException("Failed to access the response stream.", e);
}
// close up all our streams and response
sr.Close();
dataStream.Close();
response.Close();
// Deserialize the server response
return ConvertTo.JsonToServerResponse(json);
}

After a considerable amount of research on the topic, I have decided to use PUT. While Futon uses the POST method, it is undocumented. For anyone reading this in the future, use the PUT method, it will make your life much easier.