Develop Reference

C# SQL Server CLR Request error on functions GET and POST

I followed the GitHub documentation to implement the http requests with the CURL extension, work in SQL Server 2008 R2, Visual Studio 2010 and .NET 3.5.
I managed to compile and sign correctly the .dll in visual studio, to then create the schemas and functions in SQL Server, since everything works correctly, I can perform GET and POST from SQL Server, however, when wanting to perform a GET or a POST at SABA API, it generates a series of errors.
A .NET Framework error occurred during execution of user-defined
routine or aggregate "XGET": System.Net.WebException: The underlying
connection was closed: An unexpected error occurred on a send. --->
System.IO.IOException: Received an unexpected EOF or 0 bytes from the
transport stream. System.IO.IOException: at
System.Net.FixedSizeReader.ReadPacket(Byte[] buffer, Int32 offset,
Int32 count) at System.Net.Security.SslState.StartReadFrame(Byte[]
buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest) at
System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer,
AsyncProtocolRequest asyncRequest) at
System.Net.Security.SslState.CheckCompletionBeforatextReceive(ProtocolTokat
message, AsyncProtocolRequest asyncRequest) at
System.Net.Security.SslState.StartSatdBlob(Byte[] incoming, Int32
count, AsyncProtocolRequest asyncRequest) at
System.Net.Security.SslState.ForceAuthattication(Boolean receiveFirst,
Byte[] buffer, AsyncProtocolRequest asyncRequest) at
System.Net.Security.SslState.ProcessAuthattication(LazyAsyncResult
lazyResult) at
System.Net.TlsStream.CallProcessAuthattication(Object state) at
System.Threading.ExecutionContext.runTryCode(Object userData) at
System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode
code, CleanupCode backoutCode, Object userData) at
System.Threading.ExecutionContext.RunInternal(ExecutionContext
executionContext, ContextCallback callback, Object state) at
System.Threading.ExecutionContext.Run(ExecutionContext
executionContext, ContextCallback callback, Object state) at
System.Net.TlsStream.ProcessAuthattication(LazyAsyncResult result)
at System.Net.TlsStream.Write(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.PooledStream.Write(Byte[] buffer, Int32 offset, Int32
size) at System.Net.ConnectStream.WriteHeaders(Boo ...
System.Net.WebException: at
System.Net.WebCliatt.DownloadDataInternal(Uri address, WebRequest&
request) at System.Net.WebCliatt.DownloadString(Uri address) ...
This is the code of the Assembly
using Microsoft.SqlServer.Server;
using System;
using System.Data.SqlTypes;
using System.Net;
using System.Threading;
public static class Curl
{
[SqlFunction]
[return: SqlFacet(MaxSize = -1)]
public static SqlChars Get(SqlChars H, SqlChars url)
{
ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3 | SecurityProtocolType.Tls;
var client = new WebClient();
AddHeader(H, client);
return new SqlChars(
client.DownloadString(
Uri.EscapeUriString(url.ToSqlString().Value)
).ToCharArray());
}
[SqlProcedure]
public static void Post(SqlChars H, SqlChars d, SqlChars url)
{
var client = new WebClient();
AddHeader(H, client);
if (d.IsNull)
throw new ArgumentException("You must specify data that will be sent to the endpoint", "#d");
var response =
client.UploadString(
Uri.EscapeUriString(url.ToSqlString().Value),
d.ToSqlString().Value
);
SqlContext.Pipe.Send("Request is executed. " + response);
}
[SqlProcedure]
public static void PostWithRetry(SqlChars H, SqlChars d, SqlChars url)
{
var client = new WebClient();
AddHeader(H, client);
if (d.IsNull)
throw new ArgumentException("You must specify data that will be sent to the endpoint", "#d");
int i = RETRY_COUNT;
string response = "";
do try
{
response =
client.UploadString(
Uri.EscapeUriString(url.ToSqlString().Value),
d.ToSqlString().Value
);
i = -1;
break;
}
catch (Exception ex)
{
SqlContext.Pipe.Send("Error:\t" + ex.Message + ". Waiting " + DELAY_ON_ERROR + "ms.");
i--;
Thread.Sleep(DELAY_ON_ERROR);
}
while (i > 0);
if (i == -1)
SqlContext.Pipe.Send("Request is executed." + response);
}
static readonly int RETRY_COUNT = 3;
static readonly int DELAY_ON_ERROR = 50;
public static bool IsNullOrWhiteSpace(this string theString)
{
if (theString == null)
{
return false;
}
if (theString.Trim() == string.Empty)
{
return false;
}
return true;
}
private static void AddHeader(SqlChars H, WebClient client)
{
if (!H.IsNull)
{
string header = H.ToString();
if (!IsNullOrWhiteSpace(header))
client.Headers.Add(HttpRequestHeader.UserAgent, header);
}
}
};
And this how to use in SQL Query
declare #hkey nvarchar(4000) = 'SabaCertificate: 31336132353061666330315E235E756F6E6555E6261536974655E235E656E5F55535E235E536162615E235E24414021463393C69358BE384802BA1BBEAD3B4661862F193021435F7E28A30F7540FE661B9C5F30FDB06C';
declare #endpoint nvarchar(1000) = 'https://libertad-api.sabacloud.com/v1/location?count=10&startPage=1';
select curl.xget(#hkey, #endpoint)
I already test it in PostMan, entering the Header of SabaCertificate, and if it throws a result at me, however, when the certificate is not correct it also throws a response and it is not shown.
Bad Request Example:
{"errorCode":123,"errorMessage":"Invalid or expired Certificate"}
But it also does not give me the answer of the certificate error, that I have to change in my WebClient for this to work.
Added to this I think the certificate is too big because sometimes I get this error:
The identifier that starts with 'SabaCertificate:
31336132353061666330315E235E756F6E6555E6261536974655E235E656E5F55535E235E536162615E235E24414021463393C69358BE384802BA1BBEAD3B4661862F193021435F7E28A30F7540FE661B9C5F30FDB06C'
is too long. Maximum length is 128.

One definite problem in the code is a slight change you made to the original code. In your AddHeader method you have the following line:
client.Headers.Add(HttpRequestHeader.UserAgent, header);
You need to remove the HttpRequestHeader.UserAgent because the code is now creating a "UserAgent" header with a value of whatever you pass in, which is "SabaCertificate: 31336132....".
You will also need to change the security protocols that you are setting as they are not correct. You should try:
ServicePointManager.SecurityProtocol |= (SecurityProtocolType)3072; // TLS 1.2
Since you are using .NET 3.5 via SQL Server 2008 R2, you cannot specify SecurityProtocolType.Tls12 since that value had not yet been added to the enum in Framework Version 3.5, so you have to use the numeric value as shown above. Please keep in mind that the actual ability to do the security protocol is a function of the underlying OS, so it is possible that an older version of Windows / Windows Server does not support TLS 1.2, or might need a registry setting changed in order to do so. You will have to play around with that if you continue to get similar errors from System.Net.TlsStream.
Also, the following error:
The identifier that starts with 'SabaCertificate: 31336...30FDB06C' is too long. Maximum length is 128.
is from user-error. An "identifier" is an item name within SQL Server (objects, Logins, variables, etc). This means that you are doing something different (and wrong) when that error happens, but I can't see how it could be coming from your code, at least not the Get method, as that has no internal interaction with the database.

VSTO Outlook Plugin - HttpClient.PostAsync fails without fiddler

I unfortunately had Fiddler running for the whole time I was developing this feature in the plugin and since deploying to clients I found that it will not work for anyone - unless they run fiddler as well! It also does not work on my development machine if I stop running Fiddler.
The main error is Error while copying content to a stream.. So I investigated the possibility of the data I'm POSTing being released by the GC before it finished hitting the server (That, to me, explained why running Fiddler solved it - as I believe the request gets sent to Fiddler first, and then Fiddler sends it to the server). However I couldn't find anything to support that this might be the problem. I have tried making sure it holds onto the data but I don't feel like I'm going down the right route.
The code is roughly like this:
HttpClientHandler httpHandler = new HttpClientHandler { UseDefaultCredentials = true };
var client = new HttpClient(httpHandler, false);
client.BaseAddress = new Uri(BaseUrl + "api/job/PostTest");
var content = new MultipartFormDataContent("Upload----" + DateTime.Now.ToString(System.Globalization.CultureInfo.InvariantCulture));
content.Add(new StringContent(mailItem.HTMLBody, Encoding.UTF8), "BodyHtml");
// content.Add()'s... Omitted for brevity
var response = client.PostAsync(BaseUrl + "api/job/PostTest", content);
response.ContinueWith(prevTask => {
if (prevTask.Result.IsSuccessStatusCode)
{
System.Diagnostics.Debug.WriteLine("Was success");
}
else
{
System.Diagnostics.Debug.WriteLine("Was Error");
}
}, System.Threading.Tasks.TaskContinuationOptions.OnlyOnRanToCompletion);
response.ContinueWith(prevTask =>{
MessageBox.Show(prevTask.Exception.ToString());
}, System.Threading.Tasks.TaskContinuationOptions.OnlyOnFaulted);
The full exception details are:
System.AggregateException: One or more errors occurred. ---> System.Net.Http.HttpRequestException: Error while copying content to a stream. ---> System.IO.IOException: The read operation failed, see inner exception. ---> System.Net.WebException: The request was aborted: The request was canceled.
at System.Net.ConnectStream.BeginRead(Byte[] buffer, Int32 offset, Int32 size, AsyncCallback callback, Object state)
at System.Net.Http.HttpClientHandler.WebExceptionWrapperStream.BeginRead(Byte[] buffer, Int32 offset, Int32 count, AsyncCallback callback, Object state)
--- End of inner exception stack trace ---
at System.Net.Http.HttpClientHandler.WebExceptionWrapperStream.BeginRead(Byte[] buffer, Int32 offset, Int32 count, AsyncCallback callback, Object state)
at System.Net.Http.StreamToStreamCopy.StartRead()
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
---> (Inner Exception #0) System.Net.Http.HttpRequestException: Error while copying content to a stream. ---> System.IO.IOException: The read operation failed, see inner exception. ---> System.Net.WebException: The request was aborted: The request was canceled.
at System.Net.ConnectStream.BeginRead(Byte[] buffer, Int32 offset, Int32 size, AsyncCallback callback, Object state)
at System.Net.Http.HttpClientHandler.WebExceptionWrapperStream.BeginRead(Byte[] buffer, Int32 offset, Int32 count, AsyncCallback callback, Object state)
--- End of inner exception stack trace ---
at System.Net.Http.HttpClientHandler.WebExceptionWrapperStream.BeginRead(Byte[] buffer, Int32 offset, Int32 count, AsyncCallback callback, Object state)
at System.Net.Http.StreamToStreamCopy.StartRead()
--- End of inner exception stack trace ---<---
If anyone could point me to some resources that help me or point out where I'm going wrong that would help a lot!

When developing our IronBox Outlook plugin we ran into this issue. What we found was that within the VSTO context, the ServicePointManager supported security protocols was only Tls and Ssl3 (which was not going to work with our API which supported only TLS 1.2 or better).
You can check this easily from within your VSTO code like this (here's an example from when we hooked into Application.ItemSend event):
private void ThisAddIn_Startup(object sender, System.EventArgs e)
{
// Handle event when item is sent
this.Application.ItemSend += Application_ItemSend;
}
private void Application_ItemSend(object Item, ref bool Cancel)
{
foreach (var c in (SecurityProtocolType[])Enum.GetValues(typeof(SecurityProtocolType)))
{
if (ServicePointManager.SecurityProtocol.HasFlag(c))
{
Debug.WriteLine(c.ToString());
}
}
Cancel = false;
}
We solved it by setting the ServicePointManager.SecurityProtocol property to support Tls12 like this:
ServicePointManager.SecurityProtocol |= SecurityProtocolType.Tls12;
Hope this helps someone someday,
Kevin

After much searching and much messing about I've not been able to solve this problem using HttpClient so instead what I've done is using WebClient. In case someone else has this problem in the future I'm posting what I ended up using:
System.Net.WebClient wc = new System.Net.WebClient();
wc.Headers.Add("Content-Type", String.Format("multipart/form-data; boundary=\"{0}\"", multipartFormBoundary));
wc.UseDefaultCredentials = true;
try
{
var wcResponse = wc.UploadData(BaseUrl + "api/job/PostJobEmailNote", byteArray);
}
catch(Exception e)
{
// response status code was not in 200's
}

I think problem may be with using anonymous function that returns void. They're a little bit problematic. Changing my lambda to one that returns bools fixed the issue for me.

Timeout exception when writing to a virtual COM

I am trying to write to COM port using .net sample
I have no problem to write to a real COM, but when I try to write to a virtual COM I get a timeout exception
Unhandled Exception: System.TimeoutException: The write timed out.
at System.IO.Ports.SerialStream.Write(Byte[] array, Int32 offset, Int32 count
, Int32 timeout)
at System.IO.Ports.SerialPort.Write(String text)
at System.IO.Ports.SerialPort.WriteLine(String text)
I googled the issue, and found this. according to what is said there this is the issue:
.net's write() function is using CreateFile and WriteFile Async like this :
CreateFile("\\\\.\\" + portName,
NativeMethods.GENERIC_READ | NativeMethods.GENERIC_WRITE,
0, // comm devices must be opened w/exclusive-access
IntPtr.Zero, // no security attributes
UnsafeNativeMethods.OPEN_EXISTING, // comm devices must use OPEN_EXISTING
FILE_FLAG_OVERLAPPED,
IntPtr.Zero // hTemplate must be NULL for comm devices
);
WriteFile(_handle, p + offset, count, IntPtr.Zero, overlapped);
when I used those I had NULL instead of the FILE_FLAG_OVERLAPPE
My question is, how can I overcome this issue? do I have to write my own code?

Invoke The stream does not support reading

I have a c# network application where alot of anonymous users connect to (game service).
Now I check the logs and occasionally I see this exception:
[10:30:18.21352] System.Int32 Read(Byte[], Int32, Int32): The stream does not support reading.
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at BusinessLayer.Listener.ListenerWorker.ProcessClient(Object obj) in File.cs:line 141
This error comes from a NetworkStream object, now I am trying to reproduce the problem, but how? How can I get this exception?
I tried disconnecting myself, but that just gives a timeout, tried other things, but cannot get it to work.
Maybe somebody has an idea?
Contents of the file is:
private static void ProcessClient(
Object obj)
{
ISession session = (ISession)obj;
NetworkStream networkStream = null;
try
{
DebugUtility.SetThreadName("Worker: {0}", session.Name);
networkStream = session.TcpClient.GetStream();
networkStream.ReadTimeout = Config.ReadTimeout;
// Loop received packets (blocks untill next packet)
Int32 packetSize;
Byte[] buffer = new Byte[session.PacketSize];
while ((packetSize = networkStream.Read(buffer, 0, buffer.Length)) != 0)
{
// Get String from packet bytes
String packet = Encoding.UTF8.GetString(buffer, 0, packetSize);
// Check if packet has data
if (String.IsNullOrEmpty(packet))
continue;
// Log biggest received package
DebugUtility.CheckMaxPacketSize(session.Name, packet.Length);
// Handle packet (in new thread)
Logger.DebugLog("Received: {0}", packet);
ThreadPool.QueueUserWorkItem(session.HandlePacket, packet);
}
}
catch (Exception ex)
{
Logger.LogException(ex);
}
finally
{
if (networkStream != null)
networkStream.Close();
if (session != null)
session.Disconnect();
}
}

What arguments are you passing in the
System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
method. Are you using any of NetworkStream.Length or NetworkStream.Position properties.
i.e is it somthing like (not exactly)
System.Net.Sockets.NetworkStream.Read(buffer, stream.Position, stream.Length)
then as explained in MSDN documentation use of NetworkStream.Length and NetworkStream.Position properties will always throw a NotSupportedException as its not currently Supported.

An established connection was aborted by the software in your host machine

Sorry if this is a bit long winded but I thought better to post more than less.
This is also my First post here, so please forgive.
I have been trying to figure this one out for some time. and to no avail, hoping there is a genius out there who has encountered this before.
This is an intermittent problem and has been hard to reproduce.
The code that I am running simply calls a web service
The Web Service call is in a loop (so we could be doing this a lot, 1500 times or more)
Here is the code that is causing the error:
HttpWebRequest groupRequest = null;
WebResponse groupResponse = null;
try
{
XmlDocument doc = new XmlDocument();
groupRequest = (HttpWebRequest)HttpWebRequest.Create(String.Format(Server.HtmlDecode(Util.GetConfigValue("ImpersonatedSearch.GroupLookupUrl")),userIntranetID));
groupRequest.Proxy = null;
groupRequest.KeepAlive = false;
groupResponse = groupRequest.GetResponse();
doc.Load(groupResponse.GetResponseStream());
foreach (XmlElement nameElement in doc.GetElementsByTagName(XML_GROUP_NAME))
{
foreach (string domain in _groupDomains )
{
try
{
string group = new System.Security.Principal.NTAccount(domain, nameElement.InnerText).Translate(typeof(System.Security.Principal.SecurityIdentifier)).Value;
impersonationChain.Append(";").Append(group);
break;
}
catch{}
} // loop through
}
}
catch (Exception groupLookupException)
{
throw new ApplicationException(String.Format(#"Impersonated Search ERROR: Could not find groups for user<{0}\{1}>", userNTDomain, userIntranetID), groupLookupException);
}
finally
{
if ( groupResponse != null )
{
groupResponse.Close();
}
}
Here is the error that happens sometimes:
Could not find groups for user<DOMAIN\auser> ---> System.IO.IOException: Unable to read
data from the transport connection: An established connection was aborted by the
software in your host machine. ---> System.Net.Sockets.SocketException: An established
connection was aborted by the software in your host machine at
System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags
socketFlags) at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32
size) --- End of inner exception stack trace --- at System.Net.ConnectStream.Read(Byte[]
buffer, Int32 offset, Int32 size) at System.Xml.XmlTextReaderImpl.ReadData() at
System.Xml.XmlTextReaderImpl.ParseDocumentContent() at
System.Xml.XmlLoader.LoadDocSequence
(XmlDocument parentDoc) at System.Xml.XmlDocument.Load(XmlReader reader) at
System.Xml.XmlDocument.Load(Stream inStream) at
MyWebServices.ImpersonatedSearch.PerformQuery(QueryParameters parameters,
String userIntranetID, String userNTDomain)--- End of inner exception stack trace
---at MyWebServices.ImpersonatedSearch.PerformQuery(QueryParameters parameters, String userIntranetID, String userNTDomain)
--- End of inner exception stack trace ---
at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message,
WebResponse response, Stream responseStream, Boolean asyncCall)
at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName,
Object[] parameters) at MyProgram. MyWebServices.ImpersonatedSearch.PerformQuery
(QueryParameters parameters, String userIntranetID, String userNTDomain)
at MyProgram.MyMethod()
Sorry that was alot of code to read through.
This happens about 30 times out of around 1700

You're probably hitting a timeout. First of all, turn the keepalive back on. Second, check the timestamps on the request and reply. If there is a firewall between the sender and receiver, make sure that it isn't closing the connection because of idle timeout. I've had both these problems in the past, although with generic socket programming, not DB stuff.