Below is a section of code that is part of a static class that is giving me trouble refactoring. I've spent a few weeks thinking about ways of approaching this but have been unsuccessful thus far.
Some context - this is part of a TCP Server I'm working on. The TCP Server behaves similar to the websocket specification - when a new client connects, all the management of the client socket is abstracted away, and instead simple callbacks are exposed which can be registered to subscribers.
ie. OnReceived, OnConnect, OnDisconnect, OnSend
This way I can have another class subscribe to the behaviors required from the socket client and keep the responsibilities of managing the socket and business logic separate. Whenever the client socket receives data, it reads the entire data block and then raises the 'OnReceived' callback method, passing in the received data.
The purpose of all of this is to read various requests from a Lua client and perform the appropriate action. Some actions will go to a MySQL Database (via calling stored procedure and passing the request data as arguments to the sproc). There is also the added flexibility of being able to send more than a single dataset in the request, meaning fewer request/responses need to be sent between client and server.
This has been great for many months and has made it very easy to maintain and change the business code without needing to worry about breaking the socket behavior. If I needed to change a stored procedure in a database, it only required updating the database and the Lua client code, as the TCP Server does not care about these implementation details.
However, now there is a new requirement which is to support both MySQL and Redis, and for the TCP Server to be able to direct the request to either of these sources in a flexible way.
The first issue of duplication is the ProtocolResponseSingleData and ProtocolResponseMultiData structs - these structures are almost identical, except for the 'Data' property. I need to be able to return either a single data set, or a collection of data sets. Both of these classes are serialized into a JSON string. This has caused duplication with SendToDBSingleData and SendToDBMultiData methods.
struct ProtocolResponseMultiData
{
public string Action;
public bool Result;
public string Error;
public List<List<object>> Data;
}
struct ProtocolResponseSingleData
{
public string Action;
public bool Result;
public string Error;
public List<object> Data;
}
The second issue is there are 4 methods which are very similar to each other in ways - SendToMySQLDBSingleData, SendToMySQLDBMultiData, SendToRedisDBSingleData, SendToRedisDBMultiData. I just can't seem to figure out how to collapse these down to 1 or 2 methods at most.
The third issue is this class is all static methods - the socket client exposes callbacks, but it is not aware of any particular instance of an object, hence the callbacks need to be declared static. This makes it difficult to apply things like Strategy and Factory patterns to simplify the design.
Is there any hope for me?
public static void OnReceived(object sender, IPCReceivedEventArgs e)
{
try
{
LogToFile(e.data, ((SocketClient)(sender)).LogFile);
Console.WriteLine("Received data from client (" + ((SocketClient)(sender)).Address + ")");
dynamic j = Newtonsoft.Json.JsonConvert.DeserializeObject(e.data);
// Verify the request format is valid
if (j["Action"] != null && j["BulkQuery"] != null && j["Data"] != null && j["Destination"] != null)
{
ProtocolRequest request = new ProtocolRequest
{
Action = j["Action"],
Destination = j["Destination"],
IPAddress = ((SocketClient)(sender)).Address,
LogPath = ((SocketClient)(sender)).LogFile
};
bool isBulkQuery = j["BulkQuery"];
string jsonResp = "";
if (isBulkQuery)
{
ProtocolResponseMultiData resp = SendToDBMultiData(request, ref j);
jsonResp = JsonConvert.SerializeObject(resp);
}
else
{
ProtocolResponseSingleData resp = SendToDBSingleData(request, ref j);
jsonResp = JsonConvert.SerializeObject(resp);
}
((SocketClient)(sender)).Write(jsonResp);
}
else
{
// send malformed request response
string jsonResponse = "{ \"Action\" : " + j["Action"] + ", \"Result\" : false, \"Error\" : \"Malformed Request Received\", \"Data\" : null }";
((SocketClient)(sender)).Write(jsonResponse);
}
}
catch (Exception ex)
{
Console.WriteLine("Exception encountered during OnReceived handler: " + ex.Message);
LogToFile("Exception encountered during OnReceived handler: " + ex.Message, ((SocketClient)(sender)).LogFile);
string jsonResponse = "{ \"Action\" : \"UNKNOWN\", \"Result\" : false, \"Error\" : \"Malformed JSON Request Received\", \"Data\" : null }";
((SocketClient)(sender)).Write(jsonResponse);
}
finally
{
}
}
public static ProtocolResponseSingleData SendToDBSingleData(ProtocolRequest request, ref dynamic j)
{
if (request.Destination == "MYSQL")
{
return SendToMySQLDBSingleData(request, ref j);
}
else if (request.Destination == "REDIS")
{
return SendToRedisDBSingleData(request, ref j);
}
else
{
ProtocolResponseSingleData response = new ProtocolResponseSingleData
{
Action = request.Action,
Error = "Invalid Destination specified - must be either 'REDIS' or 'MYSQL'",
Data = null,
Result = false
};
return response;
}
}
public static ProtocolResponseMultiData SendToDBMultiData(ProtocolRequest request, ref dynamic j)
{
if (request.Destination == "MYSQL")
{
return SendToMySQLDBMultiData(request, ref j);
}
else if (request.Destination == "REDIS")
{
return SendToRedisDBMultiData(request, ref j);
}
else
{
ProtocolResponseMultiData response = new ProtocolResponseMultiData
{
Action = request.Action,
Error = "Invalid Destination specified - must be either 'REDIS' or 'MYSQL'",
Data = null,
Result = false
};
return response;
}
}
private static ProtocolResponseSingleData SendToMySQLDBSingleData(ProtocolRequest request, ref dynamic j)
{
// serialize a new json string for just the data by itself
string jdataString = Newtonsoft.Json.JsonConvert.SerializeObject(j["Data"]);
// now deserialize this string into a list of dictionaries for parsing
Dictionary<string, object> dataDictionary = null;
if (((JToken)j["Data"]).Type == JTokenType.Object)
dataDictionary = Newtonsoft.Json.JsonConvert.DeserializeObject<Dictionary<string, object>>(jdataString);
else
dataDictionary = new Dictionary<string, object>();
ProtocolResponseSingleData result = new ProtocolResponseSingleData
{
Action = request.Action,
Error = "",
Data = new List<object>(),
Result = false
};
// special scenario - because we cant get the ip address of the game server from DCS, we'll get it from the socket sender object
// and specially insert it as a parameter into the data dictionary
// the other special scenario is the server description request can supply - this can contain harmful html, so we must sanitize the input
if (request.Action == ACTION_GET_SERVERID)
{
dataDictionary.Add("IP", request.IPAddress);
if (dataDictionary.ContainsKey("Description"))
{
try
{
string html = Convert.ToString(dataDictionary["Description"]);
html = System.Web.HttpUtility.HtmlEncode(html);
dataDictionary["Description"] = SanitizeHTML(html);
}
catch (Exception ex)
{
LogToFile("Error sanitizing ServerDescription html string (Action: " + request.Action + ") - " + ex.Message, request.LogPath);
result.Error = "Error sanitizing ServerDescription html string (Action: " + request.Action + ") - " + ex.Message;
return result;
}
}
}
MySql.Data.MySqlClient.MySqlConnection _conn = null;
MySql.Data.MySqlClient.MySqlDataReader rdr = null;
try
{
_conn = new MySql.Data.MySqlClient.MySqlConnection(Config.MySQLDBConnect);
_conn.Open();
MySql.Data.MySqlClient.MySqlCommand cmd = new MySql.Data.MySqlClient.MySqlCommand(request.Action)
{
Connection = _conn,
CommandType = System.Data.CommandType.StoredProcedure
};
foreach (var d in dataDictionary)
{
if (d.Value.GetType() == typeof(Int64) && (Int64)d.Value == LUANULL)
cmd.Parameters.AddWithValue(d.Key, null);
else
cmd.Parameters.AddWithValue(d.Key, d.Value);
}
rdr = cmd.ExecuteReader();
if (rdr.Read())
{
for (int i = 0; i < rdr.FieldCount; i++)
{
result.Data.Add(rdr[i]);
}
}
rdr.Close();
_conn.Close();
result.Result = true;
}
catch (Exception ex)
{
LogToFile("Error executing query against MySQL (Action: " + request.Action + ") - " + ex.Message, request.LogPath);
result.Error = "Error executing query against MySQL (Action: " + request.Action + ") - " + ex.Message;
}
finally
{
if (_conn != null)
if (_conn.State == System.Data.ConnectionState.Open || _conn.State == System.Data.ConnectionState.Connecting)
_conn.Close();
if (rdr != null)
if (!rdr.IsClosed)
rdr.Close();
}
return result;
}
private static ProtocolResponseMultiData SendToMySQLDBMultiData(ProtocolRequest request, ref dynamic j)
{
// serialize a new json string for just the data by itself
string jdataString = Newtonsoft.Json.JsonConvert.SerializeObject(j["Data"]);
// now deserialize this string into a list of dictionaries for parsing
List<Dictionary<string, object>> dataDictionary =
Newtonsoft.Json.JsonConvert.DeserializeObject<List<Dictionary<string, object>>>(jdataString);
ProtocolResponseMultiData result = new ProtocolResponseMultiData
{
Action = request.Action,
Error = "",
Data = new List<List<object>>(),
Result = false
};
MySql.Data.MySqlClient.MySqlConnection _conn = null;
MySql.Data.MySqlClient.MySqlDataReader rdr = null;
try
{
foreach (var d in dataDictionary)
{
_conn = new MySql.Data.MySqlClient.MySqlConnection(Config.MySQLDBConnect);
_conn.Open();
MySql.Data.MySqlClient.MySqlCommand cmd = new MySql.Data.MySqlClient.MySqlCommand(request.Action)
{
Connection = _conn,
CommandType = System.Data.CommandType.StoredProcedure
};
foreach (var kv in d)
{
if (kv.Value.GetType() == typeof(Int64) && (Int64)kv.Value == LUANULL)
cmd.Parameters.AddWithValue(kv.Key, null);
else
cmd.Parameters.AddWithValue(kv.Key, kv.Value);
}
rdr = cmd.ExecuteReader();
if (rdr.Read())
{
List<object> result_set = new List<object>();
for (int i = 0; i < rdr.FieldCount; i++)
{
result_set.Add(rdr[i]);
}
result.Data.Add(result_set);
}
else
{
result.Error += "No Results Returned\n";
}
rdr.Close();
_conn.Close();
}
result.Result = true;
}
catch (Exception ex)
{
LogToFile("Error executing query against MySQL (Action: " + request.Action + ") - " + ex.Message, request.LogPath);
result.Error = "Error executing query against MySQL (Action: " + request.Action + ") - " + ex.Message;
}
finally
{
if (_conn != null)
if (_conn.State == System.Data.ConnectionState.Open || _conn.State == System.Data.ConnectionState.Connecting)
_conn.Close();
if (rdr != null)
if (!rdr.IsClosed)
rdr.Close();
}
return result;
}
private static ProtocolResponseSingleData SendToRedisDBSingleData(ProtocolRequest request, ref dynamic j)
{
// Serialize the JSON Data property into its own JSON String
string jdataString = Newtonsoft.Json.JsonConvert.SerializeObject(j["Data"]);
// now deserialize this string into a list of dictionaries for parsing
Dictionary<string, object> dataDictionary = null;
if (((JToken)j["Data"]).Type == JTokenType.Object)
dataDictionary = Newtonsoft.Json.JsonConvert.DeserializeObject<Dictionary<string, object>>(jdataString);
else
dataDictionary = new Dictionary<string, object>();
ProtocolResponseSingleData result = new ProtocolResponseSingleData
{
Action = request.Action,
Error = "",
Data = new List<object>(),
Result = false
};
if (!RedisConnection.IsConnected)
{
LogToFile("Connection to Redis Closed - Attempting to reopen...", request.LogPath);
try
{
RedisConnection = ConnectionMultiplexer.Connect(Config.RedisDBConnect);
}
catch (Exception ex)
{
LogToFile("Error connecting to Redis - lost connection (" + ex.Message + ")", request.LogPath);
result.Error = "Error connecting to Redis - lost connection (" + ex.Message + ")";
return result;
}
}
try
{
string serverid = Convert.ToString(dataDictionary["ServerID"]);
string rediskey = Config.RedisActionKeys[request.Action];
if (serverid == null)
{
result.Error = "Error executing query against Redis (Action: " + request.Action + ") - " + "'ServerID' not found in Data request";
return result;
}
if (rediskey == null)
{
result.Error = "Error executing query against Redis - Action: '" + request.Action + "' not found in server configuration - please check action message or server configuration.";
return result;
}
IDatabase db = RedisConnection.GetDatabase();
string k = rediskey + ":" + serverid;
if (!db.StringSet(k, jdataString))
{
result.Error = "Failed to Set Key in Redis (Key: '" + k + "')";
}
else
{
result.Data.Add(1);
result.Result = true;
}
}
catch (Exception ex)
{
LogToFile("Error executing query against Redis (Action: " + request.Action + ") - " + ex.Message, request.LogPath);
result.Error = "Error executing query against Redis (Action: " + request.Action + ") - " + ex.Message;
}
return result;
}
private static ProtocolResponseMultiData SendToRedisDBMultiData(ProtocolRequest request, ref dynamic j)
{
// serialize a new json string for just the data by itself
string jdataString = Newtonsoft.Json.JsonConvert.SerializeObject(j["Data"]);
// now deserialize this string into a list of dictionaries for parsing
List<Dictionary<string, object>> dataDictionary =
Newtonsoft.Json.JsonConvert.DeserializeObject<List<Dictionary<string, object>>>(jdataString);
ProtocolResponseMultiData result = new ProtocolResponseMultiData
{
Action = request.Action,
Error = "",
Data = new List<List<object>>(),
Result = false
};
if (!RedisConnection.IsConnected)
{
LogToFile("Connection to Redis Closed - Attempting to reopen...", request.LogPath);
try
{
RedisConnection = ConnectionMultiplexer.Connect(Config.RedisDBConnect);
}
catch (Exception ex)
{
LogToFile("Error connecting to Redis - lost connection (" + ex.Message + ")", request.LogPath);
result.Error = "Error connecting to Redis - lost connection (" + ex.Message + ")";
return result;
}
}
try
{
int id = 0;
foreach (Dictionary<string, object> x in dataDictionary)
{
id += 1;
string serverid = Convert.ToString(x["ServerID"]);
string rediskey = Config.RedisActionKeys[request.Action];
if (serverid == null)
{
result.Error = "Error executing query against Redis (Action: " + request.Action + ") - " + "'ServerID' not found in Data request";
return result;
}
if (rediskey == null)
{
result.Error = "Error executing query against Redis - Action: '" + request.Action + "' not found in server configuration - please check action message or server configuration.";
return result;
}
IDatabase db = RedisConnection.GetDatabase();
string k = rediskey + ":" + serverid + ":" + id;
string jdatastring = Newtonsoft.Json.JsonConvert.SerializeObject(x);
if (!db.StringSet(k, jdatastring))
{
result.Error = "Failed to Set Key in Redis (Key: '" + k + "')";
result.Result = false;
}
else
{
List<object> res = new List<object>
{
k
};
result.Data.Add(res);
result.Result = true;
}
}
}
catch (Exception ex)
{
LogToFile("Error executing query against Redis (Action: " + request.Action + ") - " + ex.Message, request.LogPath);
result.Error = "Error executing query against Redis (Action: " + request.Action + ") - " + ex.Message;
}
return result;
}
Related
This question already has answers here:
Creating manual threads - but getting duplicate threads
(2 answers)
Closed 9 years ago.
ISSUE: Getting duplicate items, i.e more threads are getting created than the array size...
Hi Folks, I am creating thread in the loop for each element of array. The real use is that the of sending a batch of messages using amazon ses. the messages are stored in the messageamazonRequestBatch and the loop runs through the batch and sends the messages.
HERE IS THE CODE:
Thread thrdSendEmail;
try
{
string amazonMessageID = string.Empty;
List<Thread> lstThread = new List<Thread>();
foreach (int n in arrMessageid)
{
thrdSendEmail = new Thread(() =>
{
try
{
amazonMessageID = SendSimpleEmail_Part2(messageAmazonRequestBatch.ElementAt(n).req);
messageAmazonRequestBatch.ElementAt(n).msg.AmazonMessageID = amazonMessageID;
logManager_MessageLogwithAmazonmsgID.LogMessage(",\t" + n , true);
//logManager_MessageLogwithAmazonmsgID.LogMessage(",\t" + n + ",\t" + messageAmazonRequestBatch.ElementAt(n).msg.QueueMessageId + ",\t" + amazonMessageID, true);
}
catch (Exception ex) { logManager_RunSummary.LogMessage(ex.Message, true); }
});
thrdSendEmail.Name = n.ToString();
lstThread.Add(thrdSendEmail);
thrdSendEmail.Start();
//logManager_MessageLogwithAmazonmsgID.LogMessage(",\t" + n, true);
}
foreach (Thread t in lstThread)
{
t.Join();
//logManager_MessageLogwithAmazonmsgID.LogMessage(",\t" + t.Name, true);
}
}
catch (Exception ex)
{
logManager_RunSummary.LogMessage(ex.Message, true);
}
I have also tried parallel.foreach and asynch and await options... they also give the duplicates. I know that the lock will solve the problem but in my case the lock degrades the performance by a factor of 10.. that is my performance drops 10 times... coz putting the sendemail login in lock is blocking untill i get a return amazonmessageid from amazon...
Any help on this will be greatly appreciated. I am not a novice programmer but new to threading...my contact email is shabbirbohra#gmail.com
ALSO TRIED MANY VERSION OF PARALLEL.foreach
private int SendEmailTask_Ver9_23Jan()//tried to create manual threads in parallel foreach and called SendSimpleEmail_Part3 but still duplicates
{
activeThreadCount++; threadCount++;
IList<Airmail.Core.Message> messageBatch = null;
lock (dbLocker)
{
if (activeThreadCount > maxNoofTaskCount)//targetThreadCount
{
return 0;
}
if (abort)
{
sendComplete = true;
return 0;
}
try
{
messageBatch = messageRepository.ash_GetNextBatch_AirmailVer2(maxBatchSize, this.senderTrackingHost);//messageBatch = messageRepository.ash_GetNextBatch(maxBatchSize);
}
catch (Exception ex)
{
logManager_RunSummary.LogException(ex);
messageBatch = new List<Airmail.Core.Message>();
}
Console.WriteLine(this.currentStatus);
}
while (messageBatch != null && messageBatch.Count != 0)
{
IDictionary<Airmail.Core.Message, MessageHistory> toUpdate = new Dictionary<Airmail.Core.Message, MessageHistory>();
batchSize = messageBatch.Count;
sendComplete = false;
//foreach (Airmail.Core.Message message in messageBatch)
logManager_CollectionLog.LogMessage("\tBatch\t-\t" + messageBatch.Count + "\t-\t" + System.Threading.Thread.CurrentThread.ManagedThreadId, true);//ASH-TEST 11Jan14
int intCounter = 0;//ash-teset 11han14
System.Collections.Concurrent.ConcurrentBag<Airmail.Core.Message> messageBatchConcurrent = new System.Collections.Concurrent.ConcurrentBag<Airmail.Core.Message>(messageBatch);
//All public and protected members of ConcurrentBag<T> are thread-safe and may be used concurrently from multiple threads.
//foreach (Airmail.Core.Message message in messageBatchConcurrent)
Parallel.ForEach(messageBatchConcurrent, message =>
{
//messageBatchConcurrent.Where(x => x == message).Take(1);//ash12Jan14
lock (statLocker)
{
//messageBatchConcurrent.TryTake(out message);
totalProcessed++;
intCounter += 1;//ASH-TEST 10Jan14
message.ash_BatchLoopCounter = intCounter.ToString();
//message.ash_BatchSizeCount = messageBatchConcurrent.Count.ToString();
}
if (message.ExpiryDate < DateTime.UtcNow)
{
toUpdate.Add(message, message.UpdateStatus(MessageStatus.Expired, "", null, null, true));
//message.continue(); //continue;//continue will just skip the current iteration.
return; //using return instead of continue as --> (the body is just a function called for each item)
}
lock (statLocker)
{
StatisticKey key = new StatisticKey(Convert.ToInt32(message.ash_campaignHistoryID), Convert.ToInt32(message.ash_campaignTemplateID), message.Status);//ASH25,OCT13//Airmail 2.0 changes
if (!statistics.ContainsKey(key)) statistics.Add(key, 0);
statistics[key]--;
}
try
{
string amazonMessageID = string.Empty;
if (message.Attachments == null || message.Attachments == "")//ASH25,OCT13//Airmail 2.0 changes
{
//test//if (intCounter > 1000) { Debugger.Break(); }
SendEmailResponse response = null;
if (message.ash_isSent == "YES") { return; }
//if (message.ash_isSent == null) { response = SendSimpleEmail(ref message, message.QueueMessageId, message.ash_BatchLoopCounter + "-" + message.ash_BatchSizeCount, message.ash_isSent); }//ASH-TEST 11Jan14
/// Start - this is parallel.invoke testing on 23Jan14
try
{
//Parallel.Invoke(
// delegate() // Param #2 - in-line delegate
// {
//mReq.msg.AmazonMessageID = SendSimpleEmail_Part2((SendEmailRequest)mReq.req);
//logManager_MessageLog.LogMessage(",\t" + mReq.msg.QueueMessageId, true);
// }
//);
//intthreadCount++;
//logManager_MessageLog.LogMessage(",\t creating new thread", true);
Thread thrdSendEmail = new Thread(() =>
{
if (message.ash_isSent == null) { response = SendSimpleEmail_Part3(message, message.QueueMessageId, message.ash_BatchLoopCounter + "-" + message.ash_BatchSizeCount, message.ash_isSent); }
});
lock (statLocker)
{
thrdSendEmail.Start();
}
thrdSendEmail.Join();
//logManager_MessageLog.LogMessage(",\t finishing new thread", true);
}
// No exception is expected in this example, but if one is still thrown from a task,
// it will be wrapped in AggregateException and propagated to the main thread.
catch (AggregateException e)
{
Console.WriteLine("An action has thrown an exception. THIS WAS UNEXPECTED.\n{0}", e.InnerException.ToString());
}
/// End - this is parallel.invoke testing on 23Jan14
//SendRawEmailResponse response = SendRawEmail(message);
//cSH12Jan14-test//sqlLogSentMessage += "EXEC ash_Log_SentMessageids " + "#MessageID = " + message.QueueMessageId + ", " + "#Identifier = '" + message.Identifier.ToString() + "', " + "#AmazonMessageID = '" + message.AmazonMessageID + "', " + "#Status = " + ((int)message.Status).ToString() + ", " + "#ToEmailAddress = '" + message.To.Address + "', " + "#CreatedDate = '" + DateTime.UtcNow.ToString() + "'\n";
//logManager_MessageLog.LogMessage( ",\t" + message.ash_BatchLoopCounter + "-" + message.ash_BatchSizeCount + ",\t" + response.SendEmailResult.MessageId + ",\t" + message.QueueMessageId, true);//ASH-TEST 11Jan14
lock (statLocker)
{
if (response != null) amazonMessageID = response.SendEmailResult.MessageId;
if (message.ash_isSent == "DUPLICATE") { return; }
}
//logManager_CollectionLog.LogMessage("\tSendSimpleEmail\t-\t" + message.ash_BatchSizeCount + "-" + message.ash_BatchLoopCounter + "\t-\t" + message.QueueMessageId, true);//ASH-TEST 10Jan14
}
else
{
SendRawEmailResponse response = SendRawEmail(message);
lock (statLocker)
{
if (response != null) amazonMessageID = response.SendRawEmailResult.MessageId;
intCounter += 1;//ASH-TEST 10Jan14
logManager_MessageLog.LogMessage("\tSendRawEmail-1" + intCounter + "\t-\t" + amazonMessageID + "\t-\t" + message.QueueMessageId, true);//ASH-TEST 10Jan14
}
}
lock (statLocker)
{
message.AmazonMessageID = amazonMessageID;
toUpdate.Add(message, message.UpdateStatus(amazonMessageID == string.Empty ? MessageStatus.Tested : MessageStatus.Sent,
"", null, null, true));
messageCount++;
}
}
catch (Exception ex)
{
if (ex.Message.ToLower().Contains("blacklist")
|| ex.Message.ToLower().Contains("rejected")
|| ex.Message.ToLower().Contains("not verified")
|| ex.Message.ToLower().Contains("illegal")
//|| message.OldStatus == MessageStatus.Failed)
|| message.Status == MessageStatus.Failed)
{
toUpdate.Add(message, message.UpdateStatus(MessageStatus.Undeliverable, ex.Message, null, null, true));
}
else
{
toUpdate.Add(message, message.UpdateStatus(MessageStatus.Failed, ex.Message, null, null, true));
}
Console.WriteLine(ex.Message.ToLower());//ASH22Nov
}
lock (statLocker)
{
StatisticKey key = new StatisticKey(Convert.ToInt32(message.ash_campaignHistoryID), Convert.ToInt32(message.ash_campaignTemplateID), message.Status);//ASH25,OCT13//Airmail 2.0 changes
if (!statistics.ContainsKey(key)) statistics.Add(key, 0);
statistics[key]++;
}
});
lock (dbLocker)
{
//cSH12Jan14-test//messageRepository.ash_Log_SentMessageids(sqlLogSentMessage);//ASH12Jan14
try
{
Task UpdateMessages_Task = Task.Factory.StartNew(() => messageRepository.ash_UpdateMessages(toUpdate), TaskCreationOptions.AttachedToParent);
UpdateMessages_Task.Wait();//ASH18Sep2013 - This task added for updating message asynchronously
}
catch (Exception ex)
{
logManager_RunSummary.LogException(ex);
}
if (activeThreadCount > maxNoofTaskCount)//targetThreadCount
{
return 0;
}
if (abort)
{
sendComplete = true;
return 1;
}
try
{
if (messageBatch == null && messageBatch.Count == 0)
{
messageBatch = messageRepository.ash_GetNextBatch_AirmailVer2(maxBatchSize, this.senderTrackingHost);//messageBatch = messageRepository.ash_GetNextBatch(maxBatchSize);
}
else { messageBatch = null; }
}
catch (Exception ex)
{
logManager_RunSummary.LogException(ex);
messageBatch = new List<Airmail.Core.Message>();
}
Console.WriteLine(this.currentStatus);
}
}
return 1;
}
It's an "Access to modified closure" problem. Look into that for more details. There's a lot of examples of what it is.
Simple fix is to store your n variable in a temp variable everywhere you use it in your delegate.
foreach (int n in arrMessageid)
{
int tempN = n;
Thread thrdSendEmail = new Thread(() =>
{
try
{
amazonMessageID = SendSimpleEmail_Part2(messageAmazonRequestBatch.ElementAt(tempN ).req);
messageAmazonRequestBatch.ElementAt(tempN ).msg.AmazonMessageID = amazonMessageID;
logManager_MessageLogwithAmazonmsgID.LogMessage(",\t" + tempN , true);
//logManager_MessageLogwithAmazonmsgID.LogMessage(",\t" + tempN + ",\t" + messageAmazonRequestBatch.ElementAt(tempN ).msg.QueueMessageId + ",\t" + amazonMessageID, true);
}
catch (Exception ex) { logManager_RunSummary.LogMessage(ex.Message, true); }
});
thrdSendEmail.Name = n.ToString();
lstThread.Add(thrdSendEmail);
thrdSendEmail.Start();
//logManager_MessageLogwithAmazonmsgID.LogMessage(",\t" + n, true);
}
This question already has an answer here:
Creating manual threads, also tried using parallel.foreach and async await - but getting duplicate [duplicate]
(1 answer)
Closed 8 years ago.
ISSUE: Getting duplicate items, i.e more threads are getting created than the array size...
Hi Folks, I am creating thread in the loop for each element of array. The real use is that the of sending a batch of messages using amazon ses. the messages are stored in the messageamazonRequestBatch and the loop runs through the batch and sends the messages.
HERE IS THE CODE:
Thread thrdSendEmail;
try
{
string amazonMessageID = string.Empty;
List<Thread> lstThread = new List<Thread>();
foreach (int n in arrMessageid)
{
thrdSendEmail = new Thread(() =>
{
try
{
amazonMessageID = SendSimpleEmail_Part2(messageAmazonRequestBatch.ElementAt(n).req);
messageAmazonRequestBatch.ElementAt(n).msg.AmazonMessageID = amazonMessageID;
logManager_MessageLogwithAmazonmsgID.LogMessage(",\t" + n , true);
//logManager_MessageLogwithAmazonmsgID.LogMessage(",\t" + n + ",\t" + messageAmazonRequestBatch.ElementAt(n).msg.QueueMessageId + ",\t" + amazonMessageID, true);
}
catch (Exception ex) { logManager_RunSummary.LogMessage(ex.Message, true); }
});
thrdSendEmail.Name = n.ToString();
lstThread.Add(thrdSendEmail);
thrdSendEmail.Start();
//logManager_MessageLogwithAmazonmsgID.LogMessage(",\t" + n, true);
}
foreach (Thread t in lstThread)
{
t.Join();
//logManager_MessageLogwithAmazonmsgID.LogMessage(",\t" + t.Name, true);
}
}
catch (Exception ex)
{
logManager_RunSummary.LogMessage(ex.Message, true);
}
I have also tried parallel.foreach and asynch and await options... they also give the duplicates. I know that the lock will solve the problem but in my case the lock degrades the performance by a factor of 10.. that is my performance drops 10 times... coz putting the sendemail login in lock is blocking untill i get a return amazonmessageid from amazon...
Any help on this will be greatly appreciated. I am not a novice programmer but new to threading.
ALSO TRIED MANY VERSION OF PARALLEL.foreach
private int SendEmailTask_Ver9_23Jan()//tried to create manual threads in parallel foreach and called SendSimpleEmail_Part3 but still duplicates
{
activeThreadCount++; threadCount++;
IList<Airmail.Core.Message> messageBatch = null;
lock (dbLocker)
{
if (activeThreadCount > maxNoofTaskCount)//targetThreadCount
{
return 0;
}
if (abort)
{
sendComplete = true;
return 0;
}
try
{
messageBatch = messageRepository.ash_GetNextBatch_AirmailVer2(maxBatchSize, this.senderTrackingHost);//messageBatch = messageRepository.ash_GetNextBatch(maxBatchSize);
}
catch (Exception ex)
{
logManager_RunSummary.LogException(ex);
messageBatch = new List<Airmail.Core.Message>();
}
Console.WriteLine(this.currentStatus);
}
while (messageBatch != null && messageBatch.Count != 0)
{
IDictionary<Airmail.Core.Message, MessageHistory> toUpdate = new Dictionary<Airmail.Core.Message, MessageHistory>();
batchSize = messageBatch.Count;
sendComplete = false;
//foreach (Airmail.Core.Message message in messageBatch)
logManager_CollectionLog.LogMessage("\tBatch\t-\t" + messageBatch.Count + "\t-\t" + System.Threading.Thread.CurrentThread.ManagedThreadId, true);//ASH-TEST 11Jan14
int intCounter = 0;//ash-teset 11han14
System.Collections.Concurrent.ConcurrentBag<Airmail.Core.Message> messageBatchConcurrent = new System.Collections.Concurrent.ConcurrentBag<Airmail.Core.Message>(messageBatch);
//All public and protected members of ConcurrentBag<T> are thread-safe and may be used concurrently from multiple threads.
//foreach (Airmail.Core.Message message in messageBatchConcurrent)
Parallel.ForEach(messageBatchConcurrent, message =>
{
//messageBatchConcurrent.Where(x => x == message).Take(1);//ash12Jan14
lock (statLocker)
{
//messageBatchConcurrent.TryTake(out message);
totalProcessed++;
intCounter += 1;//ASH-TEST 10Jan14
message.ash_BatchLoopCounter = intCounter.ToString();
//message.ash_BatchSizeCount = messageBatchConcurrent.Count.ToString();
}
if (message.ExpiryDate < DateTime.UtcNow)
{
toUpdate.Add(message, message.UpdateStatus(MessageStatus.Expired, "", null, null, true));
//message.continue(); //continue;//continue will just skip the current iteration.
return; //using return instead of continue as --> (the body is just a function called for each item)
}
lock (statLocker)
{
StatisticKey key = new StatisticKey(Convert.ToInt32(message.ash_campaignHistoryID), Convert.ToInt32(message.ash_campaignTemplateID), message.Status);//ASH25,OCT13//Airmail 2.0 changes
if (!statistics.ContainsKey(key)) statistics.Add(key, 0);
statistics[key]--;
}
try
{
string amazonMessageID = string.Empty;
if (message.Attachments == null || message.Attachments == "")//ASH25,OCT13//Airmail 2.0 changes
{
//test//if (intCounter > 1000) { Debugger.Break(); }
SendEmailResponse response = null;
if (message.ash_isSent == "YES") { return; }
//if (message.ash_isSent == null) { response = SendSimpleEmail(ref message, message.QueueMessageId, message.ash_BatchLoopCounter + "-" + message.ash_BatchSizeCount, message.ash_isSent); }//ASH-TEST 11Jan14
/// Start - this is parallel.invoke testing on 23Jan14
try
{
//Parallel.Invoke(
// delegate() // Param #2 - in-line delegate
// {
//mReq.msg.AmazonMessageID = SendSimpleEmail_Part2((SendEmailRequest)mReq.req);
//logManager_MessageLog.LogMessage(",\t" + mReq.msg.QueueMessageId, true);
// }
//);
//intthreadCount++;
//logManager_MessageLog.LogMessage(",\t creating new thread", true);
Thread thrdSendEmail = new Thread(() =>
{
if (message.ash_isSent == null) { response = SendSimpleEmail_Part3(message, message.QueueMessageId, message.ash_BatchLoopCounter + "-" + message.ash_BatchSizeCount, message.ash_isSent); }
});
lock (statLocker)
{
thrdSendEmail.Start();
}
thrdSendEmail.Join();
//logManager_MessageLog.LogMessage(",\t finishing new thread", true);
}
// No exception is expected in this example, but if one is still thrown from a task,
// it will be wrapped in AggregateException and propagated to the main thread.
catch (AggregateException e)
{
Console.WriteLine("An action has thrown an exception. THIS WAS UNEXPECTED.\n{0}", e.InnerException.ToString());
}
/// End - this is parallel.invoke testing on 23Jan14
//SendRawEmailResponse response = SendRawEmail(message);
//cSH12Jan14-test//sqlLogSentMessage += "EXEC ash_Log_SentMessageids " + "#MessageID = " + message.QueueMessageId + ", " + "#Identifier = '" + message.Identifier.ToString() + "', " + "#AmazonMessageID = '" + message.AmazonMessageID + "', " + "#Status = " + ((int)message.Status).ToString() + ", " + "#ToEmailAddress = '" + message.To.Address + "', " + "#CreatedDate = '" + DateTime.UtcNow.ToString() + "'\n";
//logManager_MessageLog.LogMessage( ",\t" + message.ash_BatchLoopCounter + "-" + message.ash_BatchSizeCount + ",\t" + response.SendEmailResult.MessageId + ",\t" + message.QueueMessageId, true);//ASH-TEST 11Jan14
lock (statLocker)
{
if (response != null) amazonMessageID = response.SendEmailResult.MessageId;
if (message.ash_isSent == "DUPLICATE") { return; }
}
//logManager_CollectionLog.LogMessage("\tSendSimpleEmail\t-\t" + message.ash_BatchSizeCount + "-" + message.ash_BatchLoopCounter + "\t-\t" + message.QueueMessageId, true);//ASH-TEST 10Jan14
}
else
{
SendRawEmailResponse response = SendRawEmail(message);
lock (statLocker)
{
if (response != null) amazonMessageID = response.SendRawEmailResult.MessageId;
intCounter += 1;//ASH-TEST 10Jan14
logManager_MessageLog.LogMessage("\tSendRawEmail-1" + intCounter + "\t-\t" + amazonMessageID + "\t-\t" + message.QueueMessageId, true);//ASH-TEST 10Jan14
}
}
lock (statLocker)
{
message.AmazonMessageID = amazonMessageID;
toUpdate.Add(message, message.UpdateStatus(amazonMessageID == string.Empty ? MessageStatus.Tested : MessageStatus.Sent,
"", null, null, true));
messageCount++;
}
}
catch (Exception ex)
{
if (ex.Message.ToLower().Contains("blacklist")
|| ex.Message.ToLower().Contains("rejected")
|| ex.Message.ToLower().Contains("not verified")
|| ex.Message.ToLower().Contains("illegal")
//|| message.OldStatus == MessageStatus.Failed)
|| message.Status == MessageStatus.Failed)
{
toUpdate.Add(message, message.UpdateStatus(MessageStatus.Undeliverable, ex.Message, null, null, true));
}
else
{
toUpdate.Add(message, message.UpdateStatus(MessageStatus.Failed, ex.Message, null, null, true));
}
Console.WriteLine(ex.Message.ToLower());//ASH22Nov
}
lock (statLocker)
{
StatisticKey key = new StatisticKey(Convert.ToInt32(message.ash_campaignHistoryID), Convert.ToInt32(message.ash_campaignTemplateID), message.Status);//ASH25,OCT13//Airmail 2.0 changes
if (!statistics.ContainsKey(key)) statistics.Add(key, 0);
statistics[key]++;
}
});
lock (dbLocker)
{
//cSH12Jan14-test//messageRepository.ash_Log_SentMessageids(sqlLogSentMessage);//ASH12Jan14
try
{
Task UpdateMessages_Task = Task.Factory.StartNew(() => messageRepository.ash_UpdateMessages(toUpdate), TaskCreationOptions.AttachedToParent);
UpdateMessages_Task.Wait();//ASH18Sep2013 - This task added for updating message asynchronously
}
catch (Exception ex)
{
logManager_RunSummary.LogException(ex);
}
if (activeThreadCount > maxNoofTaskCount)//targetThreadCount
{
return 0;
}
if (abort)
{
sendComplete = true;
return 1;
}
try
{
if (messageBatch == null && messageBatch.Count == 0)
{
messageBatch = messageRepository.ash_GetNextBatch_AirmailVer2(maxBatchSize, this.senderTrackingHost);//messageBatch = messageRepository.ash_GetNextBatch(maxBatchSize);
}
else { messageBatch = null; }
}
catch (Exception ex)
{
logManager_RunSummary.LogException(ex);
messageBatch = new List<Airmail.Core.Message>();
}
Console.WriteLine(this.currentStatus);
}
}
return 1;
}
You are probably having issue that happens when you combine loop constructs and lambda expressions. Read more about it here. Cleanest way to fix it is to create an object, that represents the thread set it's parameters and start it. Or you can just create local copies of all variables you are using as suggested in the previous link.
I have make C# console application which uses timer which connects to MSMQ every 10 seconds get data insert into Oracle database. But the issue is that it log in and log off to domain and create high CPU also create security audit log very much which waste my resources.
My console application runs with task schedule. Code is below
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Messaging;
using System.Xml;
using System.IO;
using System.Timers;
using Oracle.DataAccess.Client;
using System.Data;
namespace MSMQ_News
{
class Program
{
private static System.Timers.Timer aTimer;
static void Main(string[] args)
{
try
{
// Create a timer with a ten second interval.
aTimer = new System.Timers.Timer(60000);//10000
// Hook up the Elapsed event for the timer.
aTimer.Elapsed += new ElapsedEventHandler(OnTimedEvent);
// Set the Interval to 2 seconds (2000 milliseconds).
//aTimer.Interval = 10000;
aTimer.Enabled = true;
aTimer.Start();
Console.WriteLine("Press the Enter key to exit the program.");
Console.ReadLine();
}
catch (Exception ex)
{
Log(" From Main -- " + ex.Message);
}
}
private static void OnTimedEvent(object source, ElapsedEventArgs e)
{
// Just in case someone wants to inherit your class and lock it as well ...
object _padlock = new object();
try
{
aTimer.Stop();
lock (_padlock)
{
Console.WriteLine("The Elapsed event was raised at {0}", e.SignalTime);
ProcessQueueMsgs();
}
}
catch (Exception ex)
{
Log(" From OnTimedEvent -- " + ex.Message);
}
finally
{
aTimer.Start();
}
}
private static void ProcessQueueMsgs()
{
try
{
while ((DateTime.Now.Hour >= 06)
&& (DateTime.Now.Hour <= 16))
{
DateTime dt = DateTime.Now;
ReceiveNewsDetail(dt);
ReceiveNewsHeader(dt);
}
CloseApp();
}
catch (Exception ex)
{
Log(" From ProcessQueueMsgs -- " + ex.Message);
}
}
static bool QueueExist(string QueueName)
{
try
{
if (MessageQueue.Exists(QueueName))
return true;
else
return false;
}
catch (Exception ex)
{
Log(" From QueueExist -- " + ex.Message);
return false;
}
}
private static void ReceiveNewsHeader(DateTime dt)
{
try
{
MessageQueue mqNewsHeader = null;
string value = "", _tmp = "";
_tmp = "<newsHeader></newsHeader> ";
/*if (QueueExist(#".\q_ws_ampnewsheaderrep"))*/
mqNewsHeader = new MessageQueue(#".\q_ws_ampnewsheaderrep");
int MsgCount = GetMessageCount(mqNewsHeader, #".\q_ws_ampnewsheaderrep");
for (int i = 0; i < MsgCount; i++)
{
Message Msg = mqNewsHeader.Receive();
Msg.Formatter = new ActiveXMessageFormatter();
//need to do this to avoid ??? for arabic characters
using (StreamReader strdr = new StreamReader(Msg.BodyStream, System.Text.Encoding.Default))
{
value = strdr.ReadToEnd();
}
value = value.Replace("\0", String.Empty);
if (value != _tmp)
{
LoadNewsHeader(value, dt);
}
}
}
catch (Exception ex)
{
Log("From ReceiveNewsHeader -- " + ex.Message);
}
}
private static void ReceiveNewsDetail(DateTime dt)
{
try
{
MessageQueue mqNewsDetails = null;
string value = "", _tmp = "";
_tmp = "<news></news> ";
/*if (QueueExist(#".\q_ws_ampnewsrep"))*/
mqNewsDetails = new MessageQueue(#".\q_ws_ampnewsrep");
int MsgCount = GetMessageCount(mqNewsDetails, #".\q_ws_ampnewsrep");
for (int i = 0; i < MsgCount; i++)
{
Message Msg = mqNewsDetails.Receive();
Msg.Formatter = new ActiveXMessageFormatter();
//need to do this to avoid ??? for arabic characters
using (StreamReader strdr = new StreamReader(Msg.BodyStream, System.Text.Encoding.Default))
{
value = strdr.ReadToEnd();
}
value = value.Replace("\0", String.Empty);
if (value != _tmp)
{
LoadNewsDetail(value, dt);
}
}
}
catch (Exception ex)
{
Log("From ReceiveNewsDetail -- " + ex.Message);
}
}
private static void LoadNewsHeader(string text , DateTime dt)
{
try
{
//text = ReplaceSpecialCharacters(text);
//text = Clean(text);
//XmlDocument _xmlDoc = new XmlDocument();
//_xmlDoc.LoadXml(text);
//string fileName = "NewsHeader.xml";
text = text.Replace("<arabicFields>", "<arabicFields>\n\t\t");
//createXMLFile(fileName, text);
XmlDocument _xmlDoc = LoadXMLDoc(text);
string SQL = "";
XmlNodeList newsHeaderList = _xmlDoc.SelectNodes("newsHeader/newsHeaderRep");
if (newsHeaderList.Count > 0)
{
OracleParameter pTRUNCATE = new OracleParameter("P_TABLE_NAME", OracleDbType.Varchar2);
pTRUNCATE.Value = "COMPANIES_NEWS";
DatabaseOperation(CommandType.StoredProcedure, "TRUNCATE_TABLE", pTRUNCATE);
}
foreach (XmlNode news in newsHeaderList)
{
XmlNodeList newsIdList = news.SelectNodes("newsId");
SQL = "Insert into COMPANIES_NEWS(NewsID, NewsID_SEQNO, NEWSSTATUS, LANGUAGE_CD, SEC_CD, RELEASEDATE, RELEASETIME, TITLE, STG_TIME) Values(";
foreach (XmlNode newsId in newsIdList)
{
SQL += "'" + newsId["id"].InnerText + "',";
SQL += "" + newsId["seqNo"].InnerText + ",";
}
SQL += "'" + news["newsStatus"].InnerText + "',";
XmlNodeList newsItemList = news.SelectNodes("newsItem");
foreach (XmlNode newsItem in newsItemList)
{
SQL += "'" + newsItem["languageId"].InnerText + "',";
if (newsItem["reSecCode"] != null)
SQL += "'" + newsItem["reSecCode"].InnerText + "',";
else
SQL += "' ',";
XmlNodeList releaseTimeList = newsItem.SelectNodes("releaseTime");
foreach (XmlNode releaseTime in releaseTimeList)
{
SQL += "TO_DATE('" + releaseTime["date"].InnerText + "','YYYYMMDD'),";
SQL += "" + releaseTime["time"].InnerText + ",";
}
}
XmlNodeList arabicFieldsList = news.SelectNodes("arabicFields");
foreach (XmlNode arabicFields in arabicFieldsList)
{
SQL += "'" + RevertSpecialCharacters(arabicFields["title_AR"].InnerText) + "',";
}
SQL += "TO_DATE('" + dt.ToString() + "','MM/DD/YYYY HH12:MI:SS PM'))";
DatabaseOperation(CommandType.Text, SQL, null);
Console.WriteLine("Header : " + DateTime.Now.ToString());
}
if (SQL != "") //RecordCount("Select Count(*) from COMPANIES_NEWS_DETAILS") > 0
{
OracleParameter pREFRESH = new OracleParameter("P_TABLE_NAMEs", OracleDbType.Varchar2);
pREFRESH.Value = "COMPANIES_NEWS";
DatabaseOperation(CommandType.StoredProcedure, "REFRESH_VW_ALL", pREFRESH);
}
}
catch (Exception ex)
{
Log("From LoadNewsHeader -- " + ex.Message);
}
}
private static void LoadNewsDetail(string text, DateTime dt)
{
try
{
//string fileName = "NewsDetail.xml";
text = text.Replace("<arabicFields>", "<arabicFields>\n\t\t");
//text = createXMLFile(fileName);
//text = text.Replace("<arabicFields>", "<arabicFields>\n\t\t");
XmlDocument _xmlDoc = LoadXMLDoc(text);
string SQL = "";
XmlNodeList newsList = _xmlDoc.SelectNodes("news/newsRep");
if (newsList.Count > 0)
{
OracleParameter pTRUNCATE = new OracleParameter("P_TABLE_NAME", OracleDbType.Varchar2);
pTRUNCATE.Value = "COMPANIES_NEWS_DETAILS";
DatabaseOperation(CommandType.StoredProcedure, "TRUNCATE_TABLE", pTRUNCATE);
}
foreach (XmlNode news in newsList)
{
XmlNodeList newsIdList = news.SelectNodes("newsId");
SQL = "Insert into Companies_news_details(NewsID_ID, NewsID_SEQNO, NewsText_1,NewsText_2,STG_TIME) Values(";
foreach (XmlNode newsId in newsIdList)
{
SQL += "" + newsId["id"].InnerText + ",";
SQL += "" + newsId["seqNo"].InnerText + ",";
}
XmlNodeList arabicFieldsList = news.SelectNodes("arabicFields");
foreach (XmlNode arabicFields in arabicFieldsList)
{
// Log(" Before Arabic Text Data -- :" + arabicFields["newsText_AR"].InnerText);
if (arabicFields["newsText_AR"].InnerText.Length > 4000)
{
SQL += "'" + RevertSpecialCharacters(arabicFields["newsText_AR"].InnerText.Substring(0, 3999)).Replace("\n",Environment.NewLine) + "',";
SQL += "'" + RevertSpecialCharacters(arabicFields["newsText_AR"].InnerText.Substring(3999, arabicFields["newsText_AR"].InnerText.Length)).Replace("\n", Environment.NewLine) + "',";
SQL += "TO_DATE('" + dt.ToString() + "','MM/DD/YYYY HH12:MI:SS PM')";
}
else
{
SQL += "'" + RevertSpecialCharacters(arabicFields["newsText_AR"].InnerText).Replace("\n", Environment.NewLine) + "','',";
SQL += "TO_DATE('" + dt.ToString() + "','MM/DD/YYYY HH12:MI:SS PM')";
}
SQL += ")";
DatabaseOperation(CommandType.Text, SQL, null);
Console.WriteLine("Detail : " + DateTime.Now.ToString());
}
}
if (SQL != "") //RecordCount("Select Count(*) from COMPANIES_NEWS_DETAILS") > 0
{
OracleParameter pREFRESH = new OracleParameter("P_TABLE_NAMEs", OracleDbType.Varchar2);
pREFRESH.Value = "COMPANIES_NEWS_DETAILS";
DatabaseOperation(CommandType.StoredProcedure, "REFRESH_VW_ALL", pREFRESH);
}
}
catch (Exception ex)
{
Log("From LoadNewsDetail -- " + ex.Message);
}
}
private static void CloseApp()
{
System.Environment.Exit(0);
}
protected static int GetMessageCount(MessageQueue q, string queueName)
{
var _messageQueue = new MessageQueue(queueName, QueueAccessMode.Peek);
_messageQueue.Refresh(); //done to get the correct count as sometimes it sends 0
var x = _messageQueue.GetMessageEnumerator2();
int iCount = 0;
while (x.MoveNext())
{
iCount++;
}
return iCount;
}
private static void DatabaseOperation(CommandType cmdType, string SQL, OracleParameter param)
{
string oracleConnectionString = System.Configuration.ConfigurationSettings.AppSettings["OracleConnectionString"];
using (OracleConnection con = new OracleConnection())
{
con.ConnectionString = oracleConnectionString;
con.Open();
OracleCommand command = con.CreateCommand();
command.CommandType = cmdType;
command.CommandText = SQL;
if (param != null)
command.Parameters.Add(param);
command.ExecuteNonQuery();
command.Dispose();
con.Close();
}
}
private static String RevertSpecialCharacters(string pValue)
{
string _retVal = String.Empty;
_retVal = pValue.Replace("'", "''");
return _retVal;
}
public static void Log(string Message)
{
// Create a writer and open the file:
StreamWriter log;
//C:\Software\MSMQ_New_News_Fix
if (!File.Exists(#"C:\MSMQ_New_News_Fix\log.txt"))
{
log = new StreamWriter(#"C:\MSMQ_New_News_Fix\log.txt");
}
else
{
log = File.AppendText(#"C:\MSMQ_New_News_Fix\log.txt");
}
// Write to the file:
log.WriteLine(DateTime.Now.ToString() + " : " + Message);
// Close the stream:
log.Close();
}
public static XmlDocument LoadXMLDoc(string xmlText)
{
XmlDocument doc = new XmlDocument();
try
{
string xmlToLoad = ParseXMLFile(xmlText);
doc.LoadXml(xmlToLoad);
}
catch (Exception ex)
{
Log("From LoadXMLDoc -- " + ex.Message);
}
return doc;
}
private static string ParseXMLFile(string xmlText)
{
StringBuilder formatedXML = new StringBuilder();
try
{
StringReader xmlReader = new StringReader(xmlText);
while (xmlReader.Peek() >= 0)
formatedXML.Append(ReplaceSpecialChars(xmlReader.ReadLine()) + "\n");
}
catch (Exception ex)
{
Log("From ParseXMLFile -- " + ex.Message);
}
return formatedXML.ToString();
}
private static string ReplaceSpecialChars(string xmlData)
{
try
{
//if (xmlData.Contains("objectRef")) return "<objectRef></objectRef>";
int grtrPosAt = xmlData.IndexOf(">");
int closePosAt = xmlData.IndexOf("</");
int lenthToReplace = 0;
if (grtrPosAt > closePosAt) return xmlData;
lenthToReplace = (closePosAt <= 0 && grtrPosAt <= 0) ? xmlData.Length : (closePosAt - grtrPosAt) - 1;
//get the string between xml element. e.g. <ContactName>Hanna Moos</ContactName>,
//you will get 'Hanna Moos'
string data = xmlData.Substring(grtrPosAt + 1, lenthToReplace);
string formattedData = data.Replace("&", "&").Replace("<", "<")
.Replace(">", ">").Replace("'", "'");
if (lenthToReplace > 0) xmlData = xmlData.Replace(data, formattedData);
return xmlData;
}
catch (Exception ex)
{
Log("From ReplaceSpecialChars -- " + ex.Message);
return "";
}
}
}
}
How can i solve above issue
Why not host your queue reader process in a windows service. This will continually poll the queue each 10 seconds.
Then use the windows scheduler to start/stop the service at relevant times to create your service window.
This means you won't need to do anything complicated in your scheduled task, and you won't be loading and unloading all the time.
Well from logic you are very correct that I should make windows service not timer service and Task schedule.
But my question was why It is login / log out frequently, which waste the resource of my Domain server. After intense investigation, I found that calling QueueExits is resource critical. Another thing what I found is that when you connect MSMQ queue you are login to share resource, which will login to Domain. As my code was running every 10-20 seconds it was wasting my Domain server resources.
For resolution, I make my MessageQueue object globally in following way
private static MessageQueue mqNewsHeader = new MessageQueue(#".\q_ws_ampnewsheaderrep");
private static MessageQueue mqNewsDetails = new MessageQueue(#".\q_ws_ampnewsrep");
So it will create Once in the life of the Application and we will log in and log out only once. Then I will pass this object to the function as parameter. I see also that my MessageQueue count function was also resource critical, So I change it to following
protected static int GetMessageCount(MessageQueue q)
{
//var _messageQueue = new MessageQueue(queueName, QueueAccessMode.Peek);
//_messageQueue.Refresh(); //done to get the correct count as sometimes it sends 0
// var x = _messageQueue.GetMessageEnumerator2();
int iCount = q.GetAllMessages().Count();
// while (x.MoveNext())
// {
// iCount++;
// }
return iCount;
}
Hope this clear my answer and will help others also.
Friends,I need a help regarding showing error message or any other message in web method. I tried with writing some function to show message if the return value is "0" or the return string variable's length is "0" but no result. Can you give some solution to this?
Yes, sure. Here is my code:-
[WebMethod]
public string[] GetCompletionCompany(string prefixText, int count)
{
if (count == 0)
{
count = 10;
}
if (prefixText != "")
{
strSQLQuery = "SELECT ID," +
" stallno," +
" company " +
"FROM IESS2012_IND_PartDetails " +
"WHERE company LIKE '%" + prefixText + "%' " +
"ORDER BY company";
}
DataTable dt = objDBHelper.gReturnDataSet(System.Data.CommandType.Text, strSQLQuery).Tables[0];
List<string> items = new List<string>(count);
for (int i = 0; i < dt.Rows.Count; i++)
{
string strName = dt.Rows[i][2].ToString();
items.Add(strName);
}
if (items.ToArray().Length == 0)
{
ShowMessage();
return items.ToArray();
}
else
{
return items.ToArray();
}
}
private void ShowMessage()
{
string msg= gUserMessage.NoRecords;
return msg;
}
How will I show the message that there is no record?
If you want to show the Error Message on Client Side you can just throw an appropriate Exception - SOAP will encapsulate it and you get a SOAP-Error on the client side containing the exception message.
So on the Server side:
if (items.ToArray().Length == 0)
{
throw new NoRecordsException("Your Message...");
}
and on the client side (assuming you are using C# too):
string[] result;
try
{
result = service.GetCompletionCompany(prefixText, count);
}
catch(SoapException exp)
{
string message = exp.Message; //or exp.InnerException.Message
}
I am totally new to C#.
I have one service method
//actual method
public DataTable LoadDownLoadTablesData(string strBusinessUnit, string strExecutive, string strTableName, ref string strDate, string strTerritoryCode, string strUField1, string strUField2, string strUFeild3)
{
DataTable ds = new DataTable();
try
{
XontPDAServiceDAL vu = new XontPDAServiceDAL();
if (vu.validateExecutive(strBusinessUnit, strExecutive) == true)
{
DownloadFetchBLL wmd = new DownloadFetchBLL();
strDate = DateTime.Now.ToString();
ds = wmd.LoadDownLoadTableData(strBusinessUnit, strExecutive, strTableName, strTerritoryCode, strUField1, strUField2, strUFeild3);
}
else
{
throw new FaultException("Executive Not Active in the system.");
}
}
catch (FaultException) { }
catch (Exception ex)
{
throw new FaultException("Database Server is Not Responding." + ex.Message);
}
return ds;
}
//Converting datatable to String type
public string LoadDownLoadTablesDataJson(string strBusinessUnit, string strExecutive, string strTableName, ref string strDate, string strTerritoryCode, string strUField1, string strUField2, string strUFeild3)
{
DataTable dtDownloadJson = new DataTable();
dtDownloadJson = this.LoadDownLoadTablesData(strBusinessUnit, strExecutive, strTableName, ref strDate, strTerritoryCode, strUField1, strUField2, strUFeild3);
return this.ConverTableToJson(dtDownloadJson);
}
//Converting table to json
public String ConverTableToJson(DataTable dtDownloadJson)
{
string[] StrDc = new string[dtDownloadJson.Columns.Count];
string HeadStr = string.Empty;
// if (dtDownloadJson.Columns.Count > 0)
// {
for (int i = 0; i < dtDownloadJson.Columns.Count; i++)
{
StrDc[i] = dtDownloadJson.Columns[i].Caption;
HeadStr += "\"" + StrDc[i] + "\" : \"" + StrDc[i] + i.ToString() + "¾" + "\",";
}
if (HeadStr.Length > 0)
{
HeadStr = HeadStr.Substring(0, HeadStr.Length - 1);
StringBuilder Sb = new StringBuilder();
Sb.Append("{\"" + dtDownloadJson.TableName + "\" : [");
for (int i = 0; i < dtDownloadJson.Rows.Count; i++)
{
string TempStr = HeadStr;
Sb.Append("{");
for (int j = 0; j < dtDownloadJson.Columns.Count; j++)
{
TempStr = TempStr.Replace(dtDownloadJson.Columns[j] + j.ToString() + "¾", dtDownloadJson.Rows[i][j].ToString());
}
Sb.Append(TempStr + "},");
}
Sb = new StringBuilder(Sb.ToString().Substring(0, Sb.ToString().Length - 1));
Sb.Append("]}");
return Sb.ToString();
}else
{
return "0";
}
// }else{
// return "0";
// }
}
This LoadDownLoadTablesData() reference variable is there.
I have to pass call this LoadDownLoadTablesDataJson(....) from Android,
I called like this way;
// ksoap2 calling wcf
public SoapPrimitive soapPrimitiveData(String method_name1, String soap_action1, String NAMESPACE, String APPURL ,String tablename ) throws IOException,XmlPullParserException {
SoapPrimitive responses = null;
SoapObject request = new SoapObject(NAMESPACE, method_name1); // set up
request.addProperty("strBusinessUnit", "HEMA");
request.addProperty("strExec", "4515");
request.addProperty("strTableName", "RD.AlternativeProductHeader");
// after login we will get these fields value
request.addProperty("strDate", "2000-04-29");
request.addProperty("TerritoryCode", "KAND");
request.addProperty("strUField1", "");
request.addProperty("strUField2", "");
request.addProperty("strUField3", "");
SoapSerializationEnvelope envelope = new SoapSerializationEnvelope(SoapEnvelope.VER11); // put all required data into a soap// envelope
envelope.dotNet = true;
envelope.setOutputSoapObject(request);
AndroidHttpTransport httpTransport = new AndroidHttpTransport(APPURL);
httpTransport.debug = true;
try {
httpTransport.call(soap_action1, envelope);
responses = (SoapPrimitive) envelope.getResponse();
Log.w("log_tag", "#### 218 ####" + responses);
} catch (Exception e) {
e.printStackTrace();
}
return responses;
}
This always return "0". But when I run through dummy testing C# site, It return some result.
See
String da1 = "2000-04-29";
String s = client.LoadDownLoadTablesDataJson("HEMA", "4515", "RD.AlternativeProductHeader", ref da1, "KAND", "", "", "");
Label1.Text = s;
output is :
{"Table1" : [{"TableName" : "LoadDistributor","Description" : "Distributor ","MandatoryFlag" : "1","Status" : "","Priority" : "0"},{"TableName" : "LoadPrice","Description" : "Price ","MandatoryFlag" : "1","Status" : "","Priority" : "0"}]}
I have confuse how we want to pass reference type using Android?
Please help me..
Thanks in advance.
Should have give like this request.addProperty("strExecutive", "4515");
not request.addProperty("strExe", "4515");
I shoube be service argument name.Can't give different name
It looks like the property name in your andriod request doesn't match up with the parameter name on your .NET web service method: strExe != strExecutive.