Develop Reference

How to get the TTL of multiple Redis keys from C# StackExchange.Redis

Trying to make up a function that takes a list of redis keys, and outputs a list of timespans for the time to live (TTL) of the keys. I'm not familiar with Lua Scripting though. I've tried following other guides online for this, but am unable to get something to work for this situation. Any insight would be super helpful and very reusable.
public async Task<TimeSpan?[]> GetTTLManyAsync(string[] keySet)
{
try
{
var db = await this.GetDatabaseAsync();
string script = #"...";
...
var keysCasted = keySet.Select(k => new RedisKey(k)).ToArray();
var result = await db.ScriptEvaluateAsync(script, keysCasted);
...
return ...[array of timespans];
}
catch (Exception e)
{
WriteLogException(e);
return null;
}
}`

Here is a rough solution I worked out.
public async Task<List<TimeSpan?>> GetTTLManyAsync(IEnumerable<string> keys)
{
List<string> returnList = new List<string>();
try
{
var db = await this.GetDatabaseAsync();
string script = #"local result = {}
for i,key in ipairs(KEYS) do
local ttl = tonumber(redis.call('ttl', key))
result[i] = ttl
end
return result";
var result = await db.ScriptEvaluateAsync(script, keys.Select(k => new RedisKey(k)).ToArray());
var arrResult = ((double[])result).Select(ttl =>
{
if (ttl == -2) return default(TimeSpan?); // Key does not exist
else if (ttl == -1) return default(TimeSpan?); // No expiration set.
else return TimeSpan.FromSeconds(ttl);
}).ToList();
return arrResult;
}
catch (Exception e)
{
WriteLogException(e);
}
// Fill w/ empty values
return new List<TimeSpan?>(new TimeSpan?[keys.Count()]);
}

Flurl PostUrlEncoded does GET instead of POST

I must be missing something very obvious, but I can't tell what. I have a DoLoginAsync like so:
private async Task DoLoginAsync(bool force = false)
{
try
{
if (client.Cookies.ContainsKey("user_credentials") && !force)
{
return;
}
var html = client.Request("login").GetStringAsync().Result;
var doc = new HtmlDocument();
doc.LoadHtml(html);
var csrf_token = doc.DocumentNode.SelectNodes("//meta[#name='csrf-token']").First().GetAttributeValue("content", string.Empty);
var values = new Dictionary<string, string>
{
{ "user_session[email]", user },
{ "user_session[password]", password },
{ "authenticity_token", csrf_token }
};
var result = await client.Request("user_session").PostUrlEncodedAsync(values);
}
catch (Exception e)
{
}
When I run this code in a test with a breakpoint in the catch clause I get an exception
Call failed with status code 404 (Not Found): GET http://www.whatever.com/user_session
WTF? I'm expecting PostUrlEncodedAsync to do a POST, not a GET. Anybody have an idea why this can happen?
The Flurl client is instantiated as client = new FlurlClient(BASE_URL).EnableCookies();
UPDATE
Tried the following test which fails with the same exception
[TestMethod]
public async Task TheTest()
{
var message = "";
try
{
var client = new FlurlClient("http://www.slimmemeterportal.nl/").EnableCookies();
var html = await client.Request("login").GetStringAsync();
var doc = new HtmlDocument();
doc.LoadHtml(html);
var csrf_token = doc.DocumentNode.SelectNodes("//meta[#name='csrf-token']").First().GetAttributeValue("content", string.Empty);
var values = new Dictionary<string, string>
{
{ "user_session[email]", "******" },
{ "user_session[password]", "******" },
{ "commit", "inloggen" }, // Not sure if this is actually needed, but it is in the website's request parameters.
{ "authenticity_token", csrf_token }
};
var result = await client.Request("user_session").PostUrlEncodedAsync(values);
}
catch (FlurlHttpException ex)
{
message = ex.Message;
}
Assert.AreEqual("Call failed with status code 404 (Not Found): POST http://www.slimmemeterportal.nl/user_session", message);
}

Mystery solved: As it turns out after some debugging with Wireshark, the website was returning HTTP status code 301. As explained here the default action is to follow the URI in the response's location header using a GET even if the original request was a POST.

ReliableCollections Service Fabric: Statemanager's GetOrAddAsync() returns invalid queue with different names

I have two reliable queues and they are being accessed by two guest executables and each of them access their own. Sometimes the function I use to access them doesn't update the reliable queue object in the function and the wrong request is sent to the wrong guest executable.
What happens is that the clientId is passed by the guest executable to this function in the Get request. Let us say that there are two clientId(s) called T1 and T2.
What happens is that the guest executable (client) T2 at times gets the request that was meant for T1. Even though I tried line by line debugging the parameters passed to this function are correct.
Here is my API's POST that is passed a json to be added to the queue for the clients to receive from the GET
[HttpPost("MarketInfo")]
public JObject GetMarketInfo([FromBody] JObject jObject)
{
List<JToken> clients = jObject.GetValue("clients").ToList();
string json;
JObject response = new JObject();
JArray jsonArray = new JArray();
try
{
foreach (JToken client in clients)
{
var id = Guid.NewGuid();
json = "{'name':'MarketInfo','id':'" + id.ToString() + "','mtClientId':'" + terminal["name"].ToString() + "','parameters':{'symbol':'" + terminal["symbol"].ToString() + "','property':24}}";
bool result = _requestsCollectionHandler.CreateRequestForClient(JObject.Parse(json));
JObject clientResponse = new JObject();
if (result==true)
{
clientResponse["name"] = client["name"].ToString();
clientResponse["guid"] = id.ToString();
jsonArray.Add(clientResponse);
}
else
{
clientResponse["name"] = terminal.Children()["name"].ToString();
clientResponse["guid"] = "ERROR";
jsonArray.Add(terminalResponse);
}
}
response["clients"] = jsonArray;
return response;
}
catch (Exception e)
{
Debug.Write(e.Message);
return null;
}
}
This is the json that we pass to this API
{"clients":[{"name":"T1","symbol":"SomeInfo"},{"name":"T2","symbol":"SomeInfo"}]}
The problem is always with the clients object that is passed first.
Before I explain further let me also share the code for the client's HttpGet
[HttpGet("{clientId}")]
public string Get([FromRoute] string clientId)
{
try
{
string request = _requestsCollectionHandler.GetRequestJsonFromQueue(clientId);
return request;
}
catch(Exception e)
{
return e.Message;
}
}
This is the function that creates an object that is to be added by another function in the reliable queue
public bool CreateRequestForClient(JObject jObject)
{
try
{
this._jObject = new JObject(jObject);
CreateKey();
AddToRequestToQueueAsync();
return true;
}
catch (Exception e)
{
Debug.Write(e.Message);
_exceptionMessage = e.Message;
return false;
}
}
private void CreateKey()
{
dynamic data = JObject.Parse(_jObject.ToString(Newtonsoft.Json.Formatting.None));
string name = data.name;
string id = data.id;
string clientId = data.clientId;
_key.id = id;
_key.name = name;
_key.clientId = clientId;
//key.timestamp = GetTimestamp();
_key.timestamp = GetTimestamp();
_key.requestJson = _jObject.ToString(Newtonsoft.Json.Formatting.None);
}
_key is a private variable in class a custom class
This is the function in my class of request handler that adds the requests to the queue
private void AddToRequestToQueueAsync()
{
var transaction = this._stateManager.CreateTransaction();
CancellationToken cancellationToken
= new CancellationToken(false);
try
{
string queue = _key.clientId;
IReliableConcurrentQueue<TerminalResponseKey> reliableQueue =
_stateManager.GetOrAddAsync<IReliableConcurrentQueue<TerminalResponseKey>>(queue).Result;
transaction = this._stateManager.CreateTransaction();
if (reliableQueue!=null)
{
long count = reliableQueue.Count;
reliableQueue.EnqueueAsync(transaction, _key);
count = reliableQueue.Count;
transaction.CommitAsync().Wait();
}
else
{
transaction.Abort();
}
}
catch
{
transaction.Abort();
throw;
}
}
This is function that is used by the client
public string GetRequestJsonFromQueue(string clientId)
{
string queue = clientId;
try
{
IReliableConcurrentQueue<TerminalResponseKey> reliableQueue =
this._stateManager.GetOrAddAsync<IReliableConcurrentQueue<TerminalResponseKey>>(queue).Result;
if(reliableQueue != null)
{
ConditionalValue<TerminalResponseKey> key =
reliableQueue.TryDequeueAsync(transaction).Result;
if(key.HasValue)
{
string request = key.Value.requestJson;
transaction.CommitAsync().Wait();
return request;
}
}
else
{
transaction.Abort();
}
return "NO QUEUE";
}
catch (Exception e)
{
Debug.WriteLine(e);
transaction.Abort();
return e.InnerException.Message;
}
}
As far as I have found out I think my problem is in this function above. Because I don't know how the client T2 or client T1 gets another client's queue because the parameters determining the queue are their IDs and are totally unique.
These Ids are also passed correctly to this:
IReliableConcurrentQueue<TerminalResponseKey> reliableQueue =
this._stateManager.GetOrAddAsync<IReliableConcurrentQueue<TerminalResponseKey>>(queue).Result;
As you can see that we have queue=clientId
I have tried adding proper timespans but it was of no use as there is no exception thrown for OperationTimedOut. Furthermore since I am new to ServiceFabric I maybe totally doing anything wrong.
PS: Sorry for maybe a lot of jumbled up and confused code and question AND SOME OF THE INFORMATION IS OBFUSCATED DUE TO CONFIDENTIALITY BUT NOTHING OBSTRUCTING THE UNDERSTANDING OF THIS IS HIDDEN (I Hope not an issue)
I hope this is not an issue maybe an error I am overlooking at my side

When you put the request in the queue, in AddToRequestToQueueAsync(), the name of the queue is set from _key.terminalId (and I don't see where you assign it), but when you read from it, in GetRequestJsonFromQueue(), the clientId
is used as the queue name.

'ArrayList' does not contain a definition for 'GetAwaiter'

I'm getting multiple errors. As I'm new to this async/await process. So with little research I've done this bit:-
I've a function like:-
public async Task<JsonResult> GetMultipleTblResult(AdminBundle aBundleFetch)
{
if (!string.IsNullOrEmpty(aBundleFetch.ListType) && aBundleFetch.ListType.Equals(Constants.Board))
{
ArrayList MainList = new ArrayList();
aBundleFetch.ListType = Constants.Board;
Func<ArrayList> functionBoard = new Func<ArrayList>(() => FetchTableDataAsync(aBundleFetch)); // Getting Error (Cannot implicitly convert type 'System.Threading.Tasks.Task<System.Collections.Generic.IEnumerable<System.Collections.ArrayList>>' to 'System.Collections.ArrayList')
ArrayList resBoard = await Task.Factory.StartNew<ArrayList>(functionBoard);
aBundleFetch.ListType = Constants.Classes;
Func<ArrayList> functionClass = new Func<ArrayList>(() => FetchTableDataAsync(aBundleFetch)); // Getting Error (Cannot implicitly convert type 'System.Threading.Tasks.Task<System.Collections.Generic.IEnumerable<System.Collections.ArrayList>>' to 'System.Collections.ArrayList')
ArrayList resClass = await Task.Factory.StartNew<ArrayList>(functionClass);
aBundleFetch.ListType = Constants.ClassSubject;
Func<ArrayList> functionClassSubject = new Func<ArrayList>(() => FetchTableDataAsync(aBundleFetch)); // Getting Error (Cannot implicitly convert type 'System.Threading.Tasks.Task<System.Collections.Generic.IEnumerable<System.Collections.ArrayList>>' to 'System.Collections.ArrayList')
ArrayList resClassSubject = await Task.Factory.StartNew<ArrayList>(functionClassSubject);
aBundleFetch.ListType = Constants.ClassMaterial;
Func<ArrayList> functionClassMaterial = new Func<ArrayList>(() => FetchTableDataAsync(aBundleFetch)); // Getting Error (Cannot implicitly convert type 'System.Threading.Tasks.Task<System.Collections.Generic.IEnumerable<System.Collections.ArrayList>>' to 'System.Collections.ArrayList')
ArrayList resClassMaterial = await Task.Factory.StartNew<ArrayList>(functionClassMaterial);
MainList.Add(resBoard);
MainList.Add(resClass);
MainList.Add(resClassSubject);
MainList.Add(resClassMaterial);
var jsonSerialiser = new JavaScriptSerializer();
var json = jsonSerialiser.Serialize(MainList);
return new JsonResult { Data = json, JsonRequestBehavior = JsonRequestBehavior.AllowGet };
}
else
return new JsonResult { Data = "", JsonRequestBehavior = JsonRequestBehavior.AllowGet };
}
From my FetchTableDataAsync function I want to return a List of arraylists and send them over to GetMultipleTblResult :-
public async Task<IEnumerable<ArrayList>> FetchTableDataAsync(AdminBundle abundleList)
{
AdminBundle abundle = new AdminBundle();
string innerMesage = string.Empty;
if (Session["AdminBundle"] != null)
abundle = (AdminBundle)Session["AdminBundle"];
ArrayList BulkList = null;
abundle.ListType = abundleList.ListType;
if (!string.IsNullOrEmpty(abundleList.ListType))
{
using (SMContext db = new SMContext())
{
switch (abundleList.ListType)
{
case "Category":
List<Category> CategoryList = null;
CategoryList = db.CatObj.Where(x => x.Status_Info == Constants.StatusInfoOne).ToList();
BulkList.Add(CategoryList);
break;
//Class Starts
case "Board":
List<Board> BoardList = null;
BoardList = db.BoardObj.Where(x => x.Status_Info == Constants.StatusInfoOne).ToList();
BulkList.Add(BoardList);
break;
default:
break;
//Main default Ends
}
}
}
return await BulkList; //Getting Error 'ArrayList' does not contain a definition for 'GetAwaiter' and no extension method 'GetAwaiter' accepting a first argument of type 'ArrayList' could be found (are you missing a using directive or an assembly reference?)
}
Basically I want to return set of multiple lists asynchronously from later function(FetchTableDataAsync) to previous function(GetMultipleTblResult) and then pass it to my angular.js file in JSON format.
EDIT:
So with help of #JohnWu I've done this bit:-
[HttpPost]
[LogInFilter]
public JsonResult GetMultipleTblResult(AdminBundle aBundleFetch)
{
if (!string.IsNullOrEmpty(aBundleFetch.ListType) && aBundleFetch.ListType.Equals(Constants.Board))
{
Task<AllTblListClass> AllTblObj = GetTableDataAsync(aBundleFetch);
//var jsonSerialiser = new JavaScriptSerializer();
//var json = jsonSerialiser.Serialize(AllTblObj);
return new JsonResult { Data = "", JsonRequestBehavior = JsonRequestBehavior.AllowGet };
}
else
return new JsonResult { Data = "", JsonRequestBehavior = JsonRequestBehavior.AllowGet };
}
public async Task<AllTblListClass> GetTableDataAsync(AdminBundle abundleList)
{
if (!string.IsNullOrEmpty(abundleList.ListType) && abundleList.ListType.Equals(Constants.Board))
{
return new AllTblListClass
{
BoardObj = await FetchBoardsAsync(),
ClsObj = await FetchClassAsync(),
ClsSubObj = await FetchClassSubAsync(),
MatTypeObj = await FetchMaterialTAsync(),
ClassSubMatRelationObj = await FetchClassSubMatRelAsync()
};
}
else
{
return new AllTblListClass { };
}
}
public async Task<List<ClassSubMatRelation>> FetchClassSubMatRelAsync()
{
using (SMContext db = new SMContext())
{
return await Task<List<ClassSubMatRelation>>.Run(() => db.ClassSubMatRelationObj.Where(x => x.Status_Info == Constants.StatusInfoOne).ToList()); // It executes untill here and then sleeps for endless time.
}
} //I'm not writing all functions as it will create a long question
But on this line of code:-
return await Task<List<ClassSubMatRelation>>.Run(() => db.ClassSubMatRelationObj.Where(x => x.Status_Info == Constants.StatusInfoOne).ToList());
The execution sleeps and nothing happens. There isn't any Error or Exception generating.

From the end of your second method:
return await BulkList;
Here, BulkList is declared as ArrayList. There is no need for this method to be async or involve Task<T> in any way, so the most appropriate option is simply to remove all the async and Task from that method. If you need to expose it as a Task<T> - Task.FromResult may be of use, but it is sub-optimal.

It seems you want a single function to return either a list of categories or a list of boards. If boards and categories are not related (e.g. they do not share a common interface), then this is a questionable design. How is the caller going to call it? The caller at some point has to know the difference, and the developer has to know in order to put the list into something type-specific so the objects can be read. If the caller knows the difference anyway, why not use two separate functions? For example
public async Task<IEnumerable<Category>> FetchCategoriesAsync(AdminBundle abundleList)
{
if (abundleList.ListType != "Category") throw new ArgumentException("abundleList");
AdminBundle abundle = Session["AdminBundle"] as AdminBundle;
abundle.ListType = abundleList.ListType;
using (SMContext db = new SMContext())
{
return await Task<List<Category>>.Run( () => db.CatObj.Where(x => x.Status_Info == Constants.StatusInfoOne).ToList());
}
}
Notice in this example the db call is wrapped in a task and awaited. This will give you the asynchronicity that you are looking for (a method can't act async unless there is an await inside it somewhere).
If you want to be able to get categories and boards at the same time, you can implement a wrapper function on top of it, like this:
class TableData
{
public List<Catgeory> Categories { get; set; }
public List<Boards> Boards { get; set; }
}
public async Task<TableData> GetTableDataAsync(AdminBundle abundleList)
{
return new TableData
{
Categories = await FetchCategoriesAsync(abundleList),
Boards = await FetchBoardsAsync(abundleList);
};
}

How to check if connection to Elasticsearch is established?

I want to check if the connection to Elasticsearch database is ok. In other words I want to ping Elasticsearch. When I execute the code below, an exception is thrown.
public async Task<HealthCheckResult> Execute()
{
if (_configuration.Nodes?.Length == 0)
{
await Task.Delay(1);
return new HealthCheckResult("Connection Failed - Missing elasticsearch connection string")
{
ChildResults = new List<HealthCheckResult>() {new HealthCheckResult()}
};
}
var node = new Uri(_configuration.Nodes.First());
try
{
var connectionPool = new SniffingConnectionPool(new[] {node});
var settings = new ConnectionConfiguration(connectionPool);
var client = new ElasticLowLevelClient(settings);
client.IndicesExists<string>("applications");
}
catch (Exception exception)
{
return new HealthCheckResult(exception.Message)
{
ChildResults = new List<HealthCheckResult>() { new HealthCheckResult() }
};
}
return new HealthCheckResult("Connection Passed")
{
ChildResults = new List<HealthCheckResult>() { new HealthCheckResult() }
};
}
When I execute method above, exception is thrown and I get this message:
Failed sniffing cluster state.
What can I do to check if the connection to Elasticsearch is established?

The Nest IElasticClient interface provides a Ping method for this purpose

I was having the same problem and I managed to fix this by changing the SniffingConnectionPool to a SingleNodeConnectionPool.