Develop Reference

No exception being thrown when opening MySqlConnection?

I'm just starting out with async and Task's and my code has stopped processing. It happens when I have an incoming network packet and I try and communicate with the database inside the packet handler.
public class ClientConnectedPacket : IClientPacket
{
private readonly EntityFactory _entityFactory;
public ClientConnectedPacket(EntityFactory entityFactory)
{
_entityFactory= entityFactory;
}
public async Task Handle(NetworkClient client, ClientPacketReader reader)
{
client.Entity = await _entityFactory.CreateInstanceAsync( reader.GetValueByKey("unique_device_id"));
// this Console.WriteLine never gets reached
Console.WriteLine($"Client [{reader.GetValueByKey("unique_device_id")}] has connected");
}
}
The Handle method gets called from an async task
if (_packetRepository.TryGetPacketByName(packetName, out var packet))
{
await packet.Handle(this, new ClientPacketReader(packetName, packetData));
}
else
{
Console.WriteLine("Unknown packet: " + packetName);
}
Here is the method which I think is causing the issue
public async Task<Entity> CreateInstanceAsync(string uniqueId)
{
await using (var dbConnection = _databaseProvider.GetConnection())
{
dbConnection.SetQuery("SELECT COUNT(NULL) FROM `entities` WHERE `unique_id` = #uniqueId");
dbConnection.AddParameter("uniqueId", uniqueId);
var row = await dbConnection.ExecuteRowAsync();
if (row != null)
{
return new Entity(uniqueId, false);
}
}
return new Entity(uniqueId,true);
}
DatabaseProvider's GetConnection method:
public DatabaseConnection GetConnection()
{
var connection = new MySqlConnection(_connectionString);
var command = connection.CreateCommand();
return new DatabaseConnection(_logFactory.GetLogger(), connection, command);
}
DatabaseConnection's constructor:
public DatabaseConnection(ILogger logger, MySqlConnection connection, MySqlCommand command)
{
_logger = logger;
_connection = connection;
_command = command;
_connection.Open();
}
When I comment out this line, it reaches the Console.WriteLine
_connection.Open();

I ran a POC project spinning 100 parallel tasks both with MySql.Data 8.0.19 and MySqlConnector 0.63.2 on .NET Core 3.1 console application. I create, open and dispose the connection into the context of every single task. Both providers runs to completion without errors.
The specifics are that MySql.Data queries run synchronously although the library provide async methods signature e.g. ExecuteReaderAsync() or ExecuteScalarAsync(), while MySqlConnector run truly asynchronously.
You may be running into:
a deadlock situation not specifically related to the mysql provider
not properly handling exceptions inside your tasks (you may inspect the task associated aggregate exception and also monitor mysql db logs)
you execution be still blocked (not returning result) when you assume it’s not working, if you running a high number of parallel tasks with MySql.Data as it executes synchronously

Multi-threading with MySQL must use independent connections. Given that, multithreading is not a MySQL question but an issue for the client language, C# in your question.
That is, build your threads without regard to MySQL, then create a connection in each thread that needs to do queries. It will be on your shoulders if you need to pass data between the threads.
I usually find that optimizing queries eliminates the temptation to multi-thread my applications.

SqlConnection.OpenAsync issue

I am faced with a peculiar async problem which I can reproduce easily but cannot understand.
My Current Setup
I have a WCF Service which exposes two API's - API1 and API2. Both the service contracts are synchronous. API1, looks up a dictionary in memory, then creates a task using Task.Factory.StartNew to create a new task which fetches data from a SQL server, compares it with the data from the dictionary and writes some logs. In case the SQl Server has connectivity issues, this re-tries SqlConnection.OpenAsync 3 more times. Note that the API call itself returns as soon as it has the data from the dictionary (does not wait for SQl operation to complete)
API2 is much simpler, it just calls a stored procedure on SQL server, gets the data and returns.
The code to open connection is as follows:
public static int OpenSqlConn(SqlConnection connection)
{
return OpenSqlConn(connection).Result;
}
public async static Task<int> OpenSqlConnAsync(SqlConnection connection)
{
return await OpenConnAsync(connection);
}
private static async Task<int> OpenConnAsync(SqlConnection connection)
{
int retryCounter = 0;
TimeSpan? waitTime = null;
while (true)
{
if (waitTime.HasValue)
{
await Task.Delay(waitTime.Value).ConfigureAwait(false);
}
try
{
startTime = DateTime.UtcNow;
await connection.OpenAsync().ConfigureAwait(false);
break;
}
catch (Exception e)
{
if (retryCounter >= 3)
{
SafeCloseConnection(connection);
return retryCounter;
}
retryCounter++;
waitTime = TimeSpan.FromSeconds(6);
}
}
return retryCounter;
}
The API1 code looks like below:
public API1Response API1 (API1Request request)
{
// look up in memory dictionary for the request
API1Response response = getDataFromDictionary(request);
// create a task to get some data from DB
Action action = () =>
{
GetDataFromDb(request);
}
Task.Factory.StartNew(action).ConfigureAwait(false);
// this is called immediately even if DB is not available and above task is retrying.
return API1Response;
}
public void GetDataFromDb(API1Request request)
{
using (var connection = new SqlConnection(...))
{
OpenSqlConn(connection);
/// hangs for long even if db is available
ReadDataFromDb(connection);
}
}
public API2Response API2(API2REquest request)
{
return GetDataFromDbForAPI2(request)
}
public API2Response GetDataFromDbForAPI2(API2Request request)
{
using (var connection = new SqlConnection(...))
{
OpenSqlConn(connection); /// hangs for long even if db is available
ReadDataFromDb(connection);
}
}
The Problem
The service runs into the following problem when the SQL Server is unavailable even for short periods of time, and some client makes just 100 calls to API1:
When my SQL server has connectivity issues, and I get around 100 calls of API1, even though API1 returns to the caller, it has created 100 tasks that will try to open a connection to the bad DB. Each of those tasks hangs in a retry look for some time (which is expected). In my experiments, I can simulate a DB unavailability by using a bad connection string for API1.
Now let's say the DB is back up again and a call to API2 is made to the service. What I find is that when API2 call reaches the OpenAsync portion above, it hangs. Just hangs :(
Some observations
1. When I look at the 'Parallel Stacks' from Visual Studio, I find that there are 100 threads with the API1 stack doing the following stack :
ManualResetEvenSlim.Wait()
Task.SpinThenBlockingWait
Task.InternalWait();
Task<>.GetREsultCore
OpenConn()
There is 1 thread with the API2 stack, which again is in a similar stack as above.
However, if I replace SqlConnection.OpenAsync with SqlConnection.Open(), API2 call returns immediately.
Need Help
What I would like to understand is why does the API2, which can open a DB connection (because DB is available at that time), also hang on OpenAsync. Is there any obvious synchronization issue that I am seeing? When i change SqlConnection.OpenAsync() to SqlConnection.Open() why does the behavior change?

Closing WCF Service from Async method?

I have a service layer project on an MVC 5 ASP.NET application I am creating on .NET 4.5.2 which calls out to an External 3rd Party WCF Service to Get Information asynchronously. An original method to call external service was as below (there are 3 of these all similar in total which I call in order from my GetInfoFromExternalService method (note it isnt actually called that - just naming it for illustration)
private async Task<string> GetTokenIdForCarsAsync(Car[] cars)
{
try
{
if (_externalpServiceClient == null)
{
_externalpServiceClient = new ExternalServiceClient("WSHttpBinding_IExternalService");
}
string tokenId= await _externalpServiceClient .GetInfoForCarsAsync(cars).ConfigureAwait(false);
return tokenId;
}
catch (Exception ex)
{
//TODO plug in log 4 net
throw new Exception("Failed" + ex.Message);
}
finally
{
CloseExternalServiceClient(_externalpServiceClient);
_externalpServiceClient= null;
}
}
So that meant that when each async call had completed the finally block ran - the WCF client was closed and set to null and then newed up when another request was made. This was working fine until a change needed to be made whereby if the number of cars passed in by User exceeds 1000 I create a Split Function and then call my GetInfoFromExternalService method in a WhenAll with each 1000 - as below:
if (cars.Count > 1000)
{
const int packageSize = 1000;
var packages = SplitCarss(cars, packageSize);
//kick off the number of split packages we got above in Parallel and await until they all complete
await Task.WhenAll(packages.Select(GetInfoFromExternalService));
}
However this now falls over as if I have 3000 cars the method call to GetTokenId news up the WCF service but the finally blocks closes it so the second batch of 1000 that is attempting to be run throws an exception. If I remove the finally block the code works ok - but it is obviously not good practice to not be closing this WCF client.
I had tried putting it after my if else block where the cars.count is evaluated - but if a User uploads for e.g 2000 cars and that completes and runs in say 1 min - in the meantime as the user had control in the Webpage they could upload another 2000 or another User could upload and again it falls over with an Exception.
Is there a good way anyone can see to correctly close the External Service Client?

Based on the related question of yours, your "split" logic doesn't seem to give you what you're trying to achieve. WhenAll still executes requests in parallel, so you may end up running more than 1000 requests at any given moment of time. Use SemaphoreSlim to throttle the number of simultaneously active requests and limit that number to 1000. This way, you don't need to do any splits.
Another issue might be in how you handle the creation/disposal of ExternalServiceClient client. I suspect there might a race condition there.
Lastly, when you re-throw from the catch block, you should at least include a reference to the original exception.
Here's how to address these issues (untested, but should give you the idea):
const int MAX_PARALLEL = 1000;
SemaphoreSlim _semaphoreSlim = new SemaphoreSlim(MAX_PARALLEL);
volatile int _activeClients = 0;
readonly object _lock = new Object();
ExternalServiceClient _externalpServiceClient = null;
ExternalServiceClient GetClient()
{
lock (_lock)
{
if (_activeClients == 0)
_externalpServiceClient = new ExternalServiceClient("WSHttpBinding_IExternalService");
_activeClients++;
return _externalpServiceClient;
}
}
void ReleaseClient()
{
lock (_lock)
{
_activeClients--;
if (_activeClients == 0)
{
_externalpServiceClient.Close();
_externalpServiceClient = null;
}
}
}
private async Task<string> GetTokenIdForCarsAsync(Car[] cars)
{
var client = GetClient();
try
{
await _semaphoreSlim.WaitAsync().ConfigureAwait(false);
try
{
string tokenId = await client.GetInfoForCarsAsync(cars).ConfigureAwait(false);
return tokenId;
}
catch (Exception ex)
{
//TODO plug in log 4 net
throw new Exception("Failed" + ex.Message, ex);
}
finally
{
_semaphoreSlim.Release();
}
}
finally
{
ReleaseClient();
}
}
Updated based on the comment:
the External WebService company can accept me passing up to 5000 car
objects in one call - though they recommend splitting into batches of
1000 and run up to 5 in parallel at one time - so when I mention 7000
- I dont mean GetTokenIdForCarAsync would be called 7000 times - with my code currently it should be called 7 times - i.e giving me back 7
token ids - I am wondering can I use your semaphore slim to run first
5 in parallel and then 2
The changes are minimal (but untested). First:
const int MAX_PARALLEL = 5;
Then, using Marc Gravell's ChunkExtension.Chunkify, we introduce GetAllTokenIdForCarsAsync, which in turn will be calling GetTokenIdForCarsAsync from above:
private async Task<string[]> GetAllTokenIdForCarsAsync(Car[] cars)
{
var results = new List<string>();
var chunks = cars.Chunkify(1000);
var tasks = chunks.Select(chunk => GetTokenIdForCarsAsync(chunk)).ToArray();
await Task.WhenAll(tasks);
return tasks.Select(task => task.Result).ToArray();
}
Now you can pass all 7000 cars into GetAllTokenIdForCarsAsync. This is a skeleton, it can be improved with some retry logic if any of the batch requests has failed (I'm leaving that up to you).

How to call a WCF method asynchronously from MVC4 WebAPI

I am new to the WebApi, .Net world and am totally confused with all the information available as to what approach I should take. I have created a WebService using MVC4 WebApi that Twilio calls when a text message is received. I need to respond to this text message. I am consuming a WCF method which is currently being called synchronously. Since it is possible that my process can take longer than 3-5 seconds to process a reply to the text message the connection to Twilio gets disconnected due to timeout. So I am looking for ways to call this WCF method asynchronously.
My question is to call the WCF method (I am calling the WCF using a Object Factory and using)
do I need to update the contract to say Async? I am little confused on that.
BTW my Web Service is in IIS7 and am using .Net4.5 framework and MVC4 WebApi .
My code is somewhat like this: So I would like to call the SendSms part asynchronously. How do I do that? Can I simply use Task.Run Async and Await?
using Twilio.Mvc;
using Twilio.TwiML.Mvc;
using Twilio.TwiML;
public class SmsController : ApiController
{
[HttpPost]
public HttpResponseMessage Post([FromBody]SmsRequest smsReq)
{
var response = new Twilio.TwiML.TwilioResponse();
//validation checks..
try
{
-- call to WCF to get the List of sms to be sent
if ((txtMessageResponse != null) && (txtMessageResponse.SmsMessageInfo.Count > 0))
{
_smsStagingList = txtMessageResponse.SmsMessageInfo;
foreach (TextMessageStaging prepareTextMessageResponse in _smsStagingList)
{
smsDTO textMessageItems = new smsDTO();
textMessageItems.PhoneNumber = prepareTextMessageResponse.PhoneNumber;
textMessageItems.SmsMessage = prepareTextMessageResponse.SmsMessageBody;
isTxtMessageSent = SendSms(textMessageItems);
//If the messages were sent then no need to set the flag to be updated
if (isTxtMessageSent)
{
txtMessageStatusToBeUpdated = false;
}
}
return Request.CreateResponse(HttpStatusCode.OK, twilioResponse.Element);
}
else
{
//send error response
}
catch (Exception msgProcessingError)
{
//send error response again as processing error
}
finally
{
//set the outbound flag in the table
}
}
private bool SendSms(smsDTO textMessageItems)
{
bool isTxtMessageSent = false;
PushMessageRequest txtMessageRequest = new PushMessageRequest();
PushMessageResponse txtMessageResponse = null;
txtMessageRequest.SmsMessageInfo = new SendTextMessage(); //instantiate the dto
txtMessageRequest.SmsMessageInfo.ToPhone = textMessageItems.PhoneNumber;
txtMessageRequest.SmsMessageInfo.TextMessage = textMessageItems.SmsMessage;
try
{
using (ITextService textService = ObjectFactory.SendSmsMessage())
{
txtMessageResponse = textService.SendSmsMessage(txtMessageRequest);
}
isTxtMessageSent = txtMessageResponse.IsSuccessful;
}
catch (Exception ex)
{
isTxtMessageSent = false;
}
return isTxtMessageSent;
}

Twilio evangelist here.
OK, so you have a Web API method in which you call a WCF method that is potentially long running. There are two problems to solve here:
How do you call the WCF method in a way that does not block the Web API method from returning a response
How do you get Twilio to wait until the WCF method has finished
I wrote a blog post a while ago that shows you how to create an indefinite wait loop in an IVR by leveraging .NET's Task Parallel library and the loop attribute on Twilios <Play> verb.
The gist of the post is that you can use the TPL's StartNew method to start the long running WCF method on a new thread. This lets ASP.NET continue and lets you return some TwiML so Twilio does not end the call. Then you pair that with a continuation which lets you know when the WCF service request is done and you can signal back to Twilio using the REST API to redirect the in-progress call to a new set of TwiML instructions.
Hope that helps.

start Windows Service

i am writing a windows service that checks for a particular service and check it. if it is stop it will start it...
protected override void OnStart(string[] args)
{
Thread thread = new Thread(new ThreadStart(ServiceThreadFunction));
thread.Start();
}
public void ServiceThreadFunction()
{
try
{
ServiceController dc = new ServiceController("WebClient");
//ServiceController[] services = ServiceController.GetServices();
while (true)
{
if ((int)dc.Status == 1)
{
dc.Start();
WriteLog(dc.Status.ToString);
if ((int)dc.Status == 0)
{
//heartbeat
}
}
else
{
//service started
}
//Thread.Sleep(1000);
}
}
catch (Exception ex)
{
// log errors
}
}
i want the service to check for the another service and start... plz help me how can i do that

First of all, why are you casting the ServiceController's Status property from the convenient ServiceControllerStatus enum to an int? Best to leave it as an enum. Especially since your Heartbeat code, which compares it to 0, will never be run because ServiceControllerStatus doesn't have 0 as a possible value.
Secondly, you shouldn't use a while(true) loop. Even with the Thread.Sleep you have commented out there, it's a needless drain on resources. You can just use the WaitForStatus method to wait for the service to start:
ServiceController sc = new ServiceController("WebClient");
if (sc.Status == ServiceControllerStatus.Stopped)
{
sc.Start();
sc.WaitForStatus (ServiceControllerStatus.Running, TimeSpan.FromSeconds(30));
}
This will wait up to 30 seconds (or whatever) for the service to reach the Running state.
UPDATE: I re-read the original question, and I think what you're trying to do here shouldn't even be done with code. If I understood correctly, you want to set a dependency for your service on the WebClient service when you're installing it. Then, when the user starts your service in the Service Manager, it will automatically try to start the dependent service.