I have implemented asp.net caching. But I am getting weird results
Unlike most caching where you are trying to avoid the amount of hits to the DB. I am trying to avoid any hits to the DB by the user. This is b/c the amount of time the fist page takes to load. It is basically a dashboard with a lot of charts and long running queries
I tried several techniques
1) Have the cache time very long and have a schedule process expire and get new cache.
2) and on RemoveCallback
In the second option I have all the cache go through a static class I created. The purpose is as it expires to refresh the data. Here is what I am calling.
Cache.Insert(dbCacheString, dtNetwork, null, DateTime.Now.AddHours(2),
System.Web.Caching.Cache.NoSlidingExpiration, CacheItemPriority.High,
new CacheItemRemovedCallback(CacheManager.CacheRemovedCallback));
public static class CacheManager
{
private static Hashtable times = new Hashtable();
private static bool isRefreshingCache = false;
public static void CacheRemovedCallback(String key, object value,
CacheItemRemovedReason removedReason)
{
RefreshCache(key);
}
public static void StartCache()
{
string lcUrl = "http://localhost/ratingspage/";
// *** Establish the request
try
{
WebClient client = new WebClient();
client.Credentials = new NetworkCredential("xxx", "xxx",
"xxx");
byte[] myDataBuffer = client.DownloadData(lcUrl);
}
catch (Exception ex)
{
ErrHandler.WriteError(ex.Message + "\n" +
ex.StackTrace.ToString());
LogUtil.LogDebugMessages(ex.Message + ":" +
ex.StackTrace.ToString());
}
}
public static void RefreshCache(string key)
{
string controlname = "";
if ( key.ToLower().StartsWith("control:") )
{
string[] tmp = key.Split(':');
if (tmp.Length > 1)
controlname = tmp[1];
else
return;
}
else
return;
string lcUrl = "http://localhost/ratingspage/Admin/" + "
"LoadControl.aspx?CachingSpider=true&Control=" + controlname;
string lcHtml = isRefreshingCache.ToString();
// *** Establish the request
if (!isRefreshingCache)
{
isRefreshingCache = true;
lcHtml = isRefreshingCache.ToString();
try
{
WebClient client = new WebClient();
client.Credentials = new NetworkCredential("xxx",
"xxx", "xxx");
byte[] myDataBuffer = client.DownloadData(lcUrl);
lcHtml = Encoding.ASCII.GetString(myDataBuffer);
}
catch (Exception ex)
{
lcHtml = ex.Message;
isRefreshingCache = false;
ErrHandler.WriteError(ex.Message + "\n" +
ex.StackTrace.ToString());
LogUtil.LogDebugMessages(ex.Message + ":" +
ex.StackTrace.ToString());
}
isRefreshingCache = false;
}
MailMessage mail = new MailMessage(
new MailAddress("jgiblin#univision.net"),
new MailAddress("jgiblin#univision.net"));
mail.Subject = "Cache Expire: " + key;
mail.Body = string.Format("The Key {0} has expired at {1}",
key, DateTime.Now.ToShortDateString() + " " +
DateTime.Now.ToShortTimeString()) + "\nRefreshing Cache: " +
lcHtml;
SmtpClient smtp = new SmtpClient("mercury.utg.uvn.net");
mail.IsBodyHtml = false;
try
{
smtp.Send(mail);
}
catch (Exception ex)
{
ErrHandler.WriteError(ex.Message + "\n" +
ex.StackTrace.ToString());
LogUtil.LogDebugMessages(ex.Message + ":" +
ex.StackTrace.ToString());
}
}
}
for some reason, when I go to the page. Someone times the data is cached and sometimes it is not. Is there something wrong here
I tried app fabric but since the server does not have iis 7 I am not able to use that
Looks to me like you may have a race condition in your RefreshCache call. Check out this great answer for suggestions on how to handle synchronization:
C# version of java's synchronized keyword?
If I were you, I'd simplify my code. You only really need:
when the application starts up, load the cache
when the cache expires, reload it
if a user tries to access the cached object and misses, sleep their thread for a second and check again
The amount of data you can get into the cache may be the issue.
If the cache is full then the data will obviously not be cached.
You can check your memory allocation using the process monitor, have a look in "Cache Total Entries."
Related
First and foremost I am very new to C# and am sure most of my code could be cleaned up so please don't suggest it unless you are also offering help with my issue.
I am sending an email via SmtpClient(). I am trying to build the body of the email using strings which are returned from functions in loops. My issue is that the string for the body isn't building how I thought it should.
Currently, I am creating a new StringBuilder() with some default text in it. I am then running some functions and trying to add the results to the StringBuilder() object via StringBuilder.AppendLine().
Here is (some of) my code:
// Setup SMTP Client for sending mail updates
//-----------------------------------
String from_addr_text = "<removed>";
String to_addr_text = "<removed>";
String msg_subject = "Updates to USPTO searches";
StringBuilder msg_body = new StringBuilder("The following searches have received updated results:" + Environment.NewLine);
SmtpClient AlertMail = new SmtpClient
{
Port = 587,
Host = "<removed>",
EnableSsl = true,
Timeout = 10000,
DeliveryMethod = SmtpDeliveryMethod.Network,
UseDefaultCredentials = false,
Credentials = new System.Net.NetworkCredential("<removed>", "<removed>")
};
MailMessage update = new MailMessage(from_addr_text, to_addr_text, msg_subject, msg_body.ToString())
{
BodyEncoding = UTF8Encoding.UTF8,
IsBodyHtml = false,
DeliveryNotificationOptions = DeliveryNotificationOptions.OnFailure
};
...
// Process data from api for Assignor
//-----------------------------------
bool isUpdated = false;
foreach (String url in searchesByAssignorUSPTO)
{
try
{
String longName = <removed>);
String name = <removed>;
String thisHash = await GetSearchData(url, name, "Assignor");
if (DoesHashExist(thisHash))
{
Debug.WriteLine(thisHash + " already exists. No update to " + name + " search.");
}
else
{
Debug.WriteLine(thisHash + " does not exist. There is an update to " + name + " search.");
isUpdated = true;
msg_body.AppendLine(name + " as " + "Assignor" + Environment.NewLine);
}
}
catch
{
Console.WriteLine("something is broken with ASSIGNOR search dummy!");
}
}
// Process data from api for Assignee
foreach (String url in searchesByAssigneeUSPTO)
{
try
{
String longName = <removed>;
String name = <removed>;
String thisHash = await GetSearchData(url, name, "Assignee");
if (DoesHashExist(thisHash))
{
Debug.WriteLine(thisHash + " already exists. No update to " + name + " search.");
}
else
{
Debug.WriteLine(thisHash + " does not exist. There is an update to " + name + " search.");
isUpdated = true;
msg_body.AppendLine(name + " as " + "Assignee" + Environment.NewLine);
}
}
catch
{
Console.WriteLine("something is broken with ASSIGNEE search dummy!");
}
}
// Send email is search results are updated
if (isUpdated)
{
AlertMail.Send(update);
Debug.WriteLine(msg_body.ToString());
}
When the program runs and there are results returned from the loops, msg_body is printed to the output window correctly but, when the email is received the body is only: "The following searches have received updated results:".
I have tried:
changing the value of isBodyHtml to true and used <br />
instead of Environment.NewLine.
adding \n to end of stringing and removing Environment.NewLine.
changing msg_body to type String and concatenating results to msg_body using =+.
using the Append() method instead of AppendLine().
Here is a snip of the output window:
Be sure to watch the assignment of variables in your code. When you assign msg_body to the update MailMessage object, it's only inputting the one line mentioned that is being returned in the email and doesn't include the information generated by the API.
Try moving the intialization of your SmtpClient and MailMessage variables to right before the if (isUpdated) block and you should be good to go.
per #tym32167, I needed to move the instantiation of MailMessage() to AFTER my loops and functions were completed. I was creating the object before the AppendLines() methods were called and therefore they weren't being included.
I'm building a c# console application which read messages from MSMQ(Microsoft Message Queuing), I expected it to run forever but after a day running it stop reading message from MSMQ until I right click on its Console, my problem seems to be alittle bit similar to this: "Console Application "pausing" c#". Bellow is the function that I'm using:
private static void downloadHandler(object args)
{
while (true)
{
while (CURRENT_ACTIVE_THREAD < MAX_THREAD)
{
DownloadController.WriteLog("Current active Thread = " + CURRENT_ACTIVE_THREAD);
Console.WriteLine("Current active Thread = " + CURRENT_ACTIVE_THREAD);
Thread.Sleep(1000);
MessageQueue messageQueue;
if (MessageQueue.Exists(messageQueuePath))
{
messageQueue = new MessageQueue(messageQueuePath);
Message requestMessage = messageQueue.Receive();
try
{
requestMessage.Formatter = new BinaryMessageFormatter();
string msg = requestMessage.Body.ToString();
if (!string.IsNullOrEmpty(msg))
{
DownloadController.WriteLog("received message with message = " + msg);
CURRENT_ACTIVE_THREAD += 1;
RequestDownload request = new RequestDownload();
request = JsonConvert.DeserializeObject<RequestDownload>(msg);
DownloadController.WriteLog("received message with contentId = " + request.contentId + "from message queue | title= " + request.contentTitle + " | url = " + request.baseURL);
DownloadController downloader = new DownloadController();
Thread t = new Thread(new ParameterizedThreadStart(downloader.findURLandDownload));
object[] objs = new object[2];
objs[0] = request;
objs[1] = "1";
t.Start(objs);
Console.WriteLine("received message with contentId = " + request.contentId);
}
}
catch (Exception ex)
{
CURRENT_ACTIVE_THREAD -= 1;
Console.WriteLine("Exception: " + ex.Message);
DownloadController.WriteLog("There is exception while trying to read message from message queue, Exception = " + ex.Message);
}
}
}
}
}
So,could anyone please tell me what the problem is? Why this happening?
It might be you're while loop. I had while loops freeze or break my applications before. Might i suggest using timers instead ? I have some links you could use for reference:
c# wait for a while without blocking
https://msdn.microsoft.com/en-us/library/system.timers.timer(v=vs.110).aspx
I do hope this fixes the problem you're having !
greatings,
Ramon.
The following method calls Ping.Send(). When I pass an invalid URL, Send() dies and an unhandled exception happens. What is the cause of this?
private void ping()
{
comboBox3.Visible = false;
listBox2.Items.Clear();
// check the url if it is null
if (string.IsNullOrEmpty(textBox1.Text) || textBox1.Text == "")
{
listBox2.Items.Add("Please use valid IP or web address!!");
comboBox3.Visible = false;
coloring_red_tab4();
}
else
{
// do the ping
coloring_green_tab4();
for (int i = 0; i < numericUpDown1.Value; i++)
{
string s;
s = textBox1.Text;
string data = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
byte[] buffer = Encoding.ASCII.GetBytes(data);
int timeout = 120;
Ping p = new Ping();
PingOptions options = new PingOptions();
options.DontFragment = true;
//pingexception was unhalded (if the url wrong here is the error)
PingReply r = p.Send(s, timeout, buffer, options);
// if it's true url
if (r.Status == IPStatus.Success)
{
listBox2.Items.Add("Ping to " + s.ToString() + "[" + r.Address.ToString() + "]" + " (Successful) "
+ "Bytes =" + r.Buffer.Length + " TTL=" + r.Options.Ttl + " Response delay = " + r.RoundtripTime.ToString() + " ms " + "\n");
label91.Text = r.Address.ToString();
}
else
{
// just to know the ip for the website if they block the icmp protocol
listBox2.Items.Add(r.Status);
IPAddress[] ips;
ips = Dns.GetHostAddresses(textBox1.Text);
foreach (IPAddress ip in ips)
{
label91.Text = ip.ToString();
}
}
}
}
}
The exception is unhandled because you do not handle it. Whenever you call a .Net library method, you need to check its documentation to see what exceptions it throws, and decide which, if any, you want to handle at that level of code. Here is the relevant portion of the documentation for Ping.Send(), which I am including as an image so you will be able to recognize these sections going forward:
Notice that the documentation states that a PingException can occur if
An exception was thrown while sending or receiving the ICMP messages. See the inner exception for the exact exception that was thrown.
Thus it's clear from the documentation that many errors from Ping() will be reported as thrown exceptions rather than reported by setting PingReply.Status != IPStatus.Success. So you need to modify your code to be something like the following:
public static bool TryPing(string hostNameOrAddress, out string pingStatusMessage, out string pingAddressMessage)
{
if (String.IsNullOrWhiteSpace(hostNameOrAddress))
{
pingStatusMessage = "Missing host name";
pingAddressMessage = "";
return false;
}
var data = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
var buffer = Encoding.ASCII.GetBytes(data);
var timeout = 120;
using (var p = new Ping())
{
var options = new PingOptions();
options.DontFragment = true;
try
{
var r = p.Send(hostNameOrAddress, timeout, buffer, options);
if (r.Status == IPStatus.Success)
{
pingStatusMessage = "Ping to " + hostNameOrAddress.ToString() + "[" + r.Address.ToString() + "]" + " (Successful) "
+ "Bytes =" + r.Buffer.Length + " TTL=" + r.Options.Ttl + " Response delay = " + r.RoundtripTime.ToString() + " ms " + "\n";
pingAddressMessage = r.Address.ToString();
return true;
}
else
{
// just to know the ip for the website if they block the icmp protocol
pingStatusMessage = r.Status.ToString();
var ips = Dns.GetHostAddresses(hostNameOrAddress);
pingAddressMessage = String.Join(",", ips.Select(ip => ip.ToString()));
return false;
}
}
catch (PingException ex)
{
pingStatusMessage = string.Format("Error pinging {0}: {1}", hostNameOrAddress, (ex.InnerException ?? ex).Message);
pingAddressMessage = hostNameOrAddress;
return false;
}
}
}
Here I have extracted a utility method from the user interface code and also properly disposed of the Ping instance after it is no longer needed.
Then
TryPing(#"www.google.com", out pingStatusMessage, out pingAddressMessage);
Gives
Ping to www.google.com[146.115.8.83] (Successful) Bytes =32 TTL=62 Response delay = 8 ms
While
TryPing(#"www.kdjf98rglkfgjldkfjgdl;fge8org.com", out pingStatusMessage, out pingAddressMessage);
Gives
Error pinging www.kdjf98rglkfgjldkfjgdl;fge8org.com: No such host is known
I have an odd problem with setting time range for oneshot search. I set time range for my oneshot search, but results are just first found on server matching query. It seems like oneshot is just ignoring time range.
I've read:
How to run searches and jobs using the Splunk SDK for C# -
http://dev.splunk.com/view/csharp-sdk/SP-CAAAEQG
Service.SearchOneShotAsync Method -
http://docs.splunk.com/DocumentationStatic/CshrpSDK/2.1.1/Splunk.Client/html/a5323948-7506-ad15-6f04-7a95b70e616d.htm
JobArgs Class -
http://docs.splunk.com/DocumentationStatic/CshrpSDK/2.1.1/Splunk.Client/html/7dc4e71d-1ed7-4eb1-5a10-183d7663da26.htm
Time modifiers for search -
http://docs.splunk.com/Documentation/Splunk/6.0.3/SearchReference/SearchTimeModifiers
but after hours of tests and experiments - nothing.
private void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e)
{
var connectArgs = new ServiceArgs
{
Host = "myip",
Port = 8089,
Scheme = "https"
};
Splunk.Service service = new Splunk.Service(connectArgs);
service.Login("login", "password");
var oneshotSearchArgs = new Splunk.Client.JobArgs();
oneshotSearchArgs.EarliestTime = "2015-08-23 13:00";//textBoxOD.Text + "T" + textBoxODG.Text + ":00.000";
oneshotSearchArgs.LatestTime = "2015-08-23 14:00";//textBoxDO.Text + "T" + textBoxDOG.Text + ":00.000";
String oneshotSearchQuery = "search query *" + textBox1.text + "* | head 500";
var outArgs = new JobResultsArgs
{
OutputMode = JobResultsArgs.OutputModeEnum.Xml,
Count = 0,
};
try
{
using (var stream = service.Oneshot(oneshotSearchQuery, outArgs))
{
using (var rr = new ResultsReaderXml(stream))
{
string raw = "_raw";
foreach (var #event in rr)
{
wynik += "EVENT:" + Environment.NewLine;
foreach (string key in #event.Keys)
{
if (key.Contains(raw))
{
wynik += " " + key + " -> " + #event[key] + Environment.NewLine + Environment.NewLine;
}
}
}
}
}
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
}
}
Sorry for being so late with an answer. I landed into the same issue.
The reason that your OneShot search ignores the time range is because it does not take one. (I did not come across any documentation to do so)
To overcome this issue, I tried the 2.X SDK for C#. It fixed the issue.
You can use the 2.X version in 3 steps:
Setting up Service parameters. (host, port etc.)
Authenticating the Service.
Setting up parameters for the OneShot job and executing it.
You can find the NuGet package "Splunk PCL Client for .Net" in NuGet library here.
Here is a sample code:
// Setting up Service parameters
Service _splunkService = new Service(Scheme.Https, "your-api-or-ip", 8089);
// Authenticating
await _splunkService.LogOnAsync("username", "password");
// Setting up parameters for the OneShot job and executing it
var query = "your search query";
var oneShot = new JobArgs();
oneShot.EarliestTime = DateTime.Now.AddMinutes(-2).Date.ToString("yyyy-MM-dd") + "T" + DateTime.Now.AddMinutes(-2).TimeOfDay; //"2015-09-12T12:00:00.000-07:00";
oneShot.LatestTime = "your latest time";
using (var stream = await _splunkService.SearchOneShotAsync(query, 0, oneShot))
{
try
{
foreach (var result in stream)
{
var rawValue = Convert.ToString(result.GetValue("_raw"));
if (rawValue != null)
{
// do something.
}
}
}
}
Make sure the 'await' parts go inside an async method.
I'm working with RabbitMQ on someone else's project and having trouble with dequeuing and missing data.
The data is all there as a string when I publish, and it's also on there correctly on the RabbitMQ queue. When I pull the data off, buts of the data is there like the User ID, but the rest of it is gone. I've looked throughout the code and I'm fairly positive that its something going on with RabbitMQ, and its happening when I dequeue. Any help would be greatly appreciated. Thanks.
Here is the code right before the publish.
private bool sendJobToMQ(EncodeJobModel job, string p_correlation_id, string p_request_routing_key)
{
JavaScriptSerializer ser = new JavaScriptSerializer();
StringBuilder sb_job = new StringBuilder();
ser.Serialize(job, sb_job);
string rpc_reply_queue;
ConnectionFactory factory = new ConnectionFactory();
factory.HostName = HOST_NAME;
factory.VirtualHost = VHOST_NAME;
factory.UserName = USERNAME;
factory.Password = PASSWORD;
IConnection rabconn = factory.CreateConnection();
IModel sender_channel = rabconn.CreateModel();
try
{
sender_channel.ExchangeDeclare(EXCHANGE_NAME, ExchangeType.Direct, true, false, null);
}
catch (Exception err)
{
logger.Error("Error Declaring Exchange " + EXCHANGE_NAME + ": " + err.ToString());
return false;
}
try
{
sender_channel.QueueDeclare(REQUEST_QUEUE, true, false, false, null);
}
catch (Exception err)
{
logger.Error("Error QueueDeclare (" + REQUEST_QUEUE + " true, false, false, null): " + err.ToString());
return false;
}
try
{
sender_channel.QueueBind(REQUEST_QUEUE, EXCHANGE_NAME, REQUEST_ROUTING_KEY, null);
}
catch (Exception err)
{
logger.Error("Error QueueBind (" + REQUEST_QUEUE + " -> " + EXCHANGE_NAME + " " + REQUEST_ROUTING_KEY + ", null): " + err.ToString());
return false;
}
//rpc_reply_queue = sender_channel.QueueDeclare("rq_" + job.encodejob_id.ToString(), false, false, true, null);
//////bind the rpc reply queue to the exchange via a routing key (I appended _routingkey to signify this)
//sender_channel.QueueBind(rpc_reply_queue, EXCHANGE_NAME, rpc_reply_queue + "_routingkey");
//// Not sure what the props object is for yet but you can try to pass null in the mean time - Steve "Apeshit" Han
BasicProperties props = new BasicProperties();
props.CorrelationId = p_correlation_id;
//props.ReplyTo = rpc_reply_queue;
try
{
sender_channel.BasicPublish(EXCHANGE_NAME, REQUEST_ROUTING_KEY, props, Encoding.UTF8.GetBytes(sb_job.ToString()));
}
And the code for the dequeue.
QueueingBasicConsumer consumer = new QueueingBasicConsumer(p_channel);
string consumerTag = p_channel.BasicConsume(p_queue, false, consumer);
if (_is_console && Environment.UserInteractive)
Console.WriteLine("Listening...");
while (m_Listen)
{
try
{
//get the properties of the message, including the ReplyTo queue, to which we can append '_routingkey' (designated by me), to reply with messages
BasicDeliverEventArgs e;
Object message;
if (!consumer.Queue.Dequeue(4000, out message)) {
// we do not wait to indefinitely block on waiting for the queue
// if nothing in queue continue loop iteration and wait again
continue;
}
// cast as necessary back to BasicDeliverEventArgs
e = (BasicDeliverEventArgs)message;
IBasicProperties props = e.BasicProperties;
//get the Correlation ID sent by the client to track the job
string client_correlation_id = props.CorrelationId;
// I left out the reply_to field in the wizard, it can be set back in ApiEncodeServiceDefault - Steve "Smurfing Smurf" Han
//string reply_to = props.ReplyTo;
//get the body of the request
byte[] body = e.Body;
string body_result = Encoding.UTF8.GetString(body);
bool redelivered = e.Redelivered;
The e.Body string is missing data.
why continue if you don't have any message
it is better to block until you receive a message otherwise the process is not interesting (work with no data?)
try like this
QueueingBasicConsumer consumer = new QueueingBasicConsumer(channel);
channel.BasicConsume(queueName, null, consumer);
while (m_Listen) {
try {
RabbitMQ.Client.Events.BasicDeliverEventArgs e =
(RabbitMQ.Client.Events.BasicDeliverEventArgs)
consumer.Queue.Dequeue();
IBasicProperties props = e.BasicProperties;
byte[] body = e.Body;
// ... process the message
channel.BasicAck(e.DeliveryTag, false);
} catch (OperationInterruptedException ex) {
// The consumer was removed, either through
// channel or connection closure, or through the
// action of IModel.BasicCancel().
break;
}
}