Since we were forced to upgrade our mongo installation, we're receiving an error during some aggregation function calls:
MongoDB.Driver.MongoCommandException: "Command 'aggregate' failed: The
'cursor' option is required, except for aggregate with the explain
argument (response: { "ok" : 0.0, "errmsg" : "The 'cursor' option is
required, except for aggregate with the explain argument", "code" : 9,
"codeName" : "FailedToParse" })"
BsonArray arr = BsonSerializer.Deserialize<BsonArray>("[{ \"$match\" : { \"Param1\" : \"VAL\" } }, { \"$unwind\" : \"$Entries\" }, { \"$match\" : { \"PARAM\" : \"VALUE\" } }]");
var pipeline = arr.Select(x => x.AsBsonDocument).ToList();
// AggregateArgs aArgs = new AggregateArgs { Pipeline = bsonList };
var cursor = collection.Aggregate(pipeline).ResultDocuments;
I already figured out, that we have to manually add cursor configuration to the BsonDocument - but we weren't able to figure out, how the query should be configured.
Is there any work around for this exception (without changing drivers)?
give this a shot:
var cursor = collection.Aggregate<BsonDocument>(pipeline);
var results = cursor.ToList(); //just get a list of documents and be done with it
while (cursor.MoveNext()) // or iterate over cursor
{
foreach (var doc in cursor.Current.ToArray())
{
//access your documents here
}
}
You have extra brace in the end of query string
was finally able to fix it, by building the command by myself:
var cmd = new CommandDocument()
{
{"aggregate", "collection_name" },
{"pipeline", arr},
{"cursor", BsonDocument.Parse("{}") }
};
var res = db.RunCommand(cmd);
This is what worked in my situation (mongocshardriver v1.9.0-rc0, mongodb server 4.4.0); OutputMode = AggregateOutputMode.Cursor in the AggregateArgs.
public IEnumerable<BsonDocument> Run(MongoCollection<Item> items)
{
var priceRange = new BsonDocument(
"$subtract",
new BsonArray
{
"$Price",
new BsonDocument(
"$mod",
new BsonArray{"$Price", 100})
});
var grouping = new BsonDocument(
"$group",
new BsonDocument
{
{"_id", priceRange},
{"count", new BsonDocument("$sum", 1)}
});
var sort = new BsonDocument(
"$sort",
new BsonDocument("_id", 1)
);
var args = new AggregateArgs
{
Pipeline = new[] { grouping, sort },
OutputMode = AggregateOutputMode.Cursor,
};
return items.Aggregate(args);
}
Related
I have a function in which i am getting data(array of objects) from db and then adding those objects of array one by one into a lit of type ExpandoObject
public async Task<List<ExpandoObject>> GetGroupWithMaxTickets(){
List<ExpandoObject> topGroupsWithMaxTickets = new List<ExpandoObject>();
dynamic ticketDetails = new ExpandoObject();
var pipeline_tickets = new BsonDocument[]{
new BsonDocument("$match",
new BsonDocument
{
{ "nsp", "/sbtjapan.com" },
{ "datetime",
new BsonDocument
{
{ "$gte", "2019-12-03T00:00:34.417Z" },
{ "$lte", "2019-12-03T24:00:34.417Z" }
} }
}),
new BsonDocument("$group",
new BsonDocument
{
{ "_id", "$group" },
{ "totalTIckets",
new BsonDocument("$sum", 1) }
}),
new BsonDocument("$project",
new BsonDocument
{
{ "_id", 0 },
{ "group", "$_id" },
{ "totalTIckets", 1 }
}),
new BsonDocument("$sort",
new BsonDocument("totalTIckets", -1)),
new BsonDocument("$limit", 5)
};
var collection = await DbService.tickets.AggregateAsync<RawBsonDocument>(pipeline_tickets, new AggregateOptions {UseCursor = true, BatchSize = 500});
await collection.MoveNextAsync();
if(collection.Current.ToList().Count > 0){
// ticketDetails = JsonConvert.DeserializeObject(collection.Current.ToJson());
// ticketDetails.group = collection.Current.ToList()[0]["group"];
// ticketDetails.totalTickets = collection.Current.ToList()[0]["totalTIckets"];
Parallel.ForEach(collection.Current.ToList(), (ticket) => {
Console.WriteLine("Ticket----"+ticket);
dynamic groupWithTickets = new ExpandoObject();
groupWithTickets = ticket;
topGroupsWithMaxTickets.Add(groupWithTickets);
});
}
return topGroupsWithMaxTickets;
}
But it throws an error like this
System.AggregateException: One or more errors occurred. (The best overloaded method match for 'System.Collections.Generic.List<System.Dynamic.ExpandoObject>.Add(System.Dynamic.ExpandoObject)' has some invalid arguments)
I want that my function must return array of objects of type List<ExpandoObject>
How can i do this in c#?
Since you have changed the question, following is the answer that should resolve your matters.
How to NOT work with ExpandoObjects
I tested this on my system and got it to reproduce the same results as you are getting. Following is the failed try:
dynamic employee = new ExpandoObject();
List<ExpandoObject> listOfEmployees = new List<ExpandoObject>();
employee = "someStrangeString";
listOfEmployees.Add(employee); // ERROR !!!!
and just as expected, i get the following error on Add.
Microsoft.CSharp.RuntimeBinder.RuntimeBinderException
HResult=0x80131500
Message=The best overloaded method match for 'System.Collections.Generic.List.Add(System.Dynamic.ExpandoObject)' has some invalid arguments
Source=
StackTrace:
Corrected way of ExpandoObject use
Following is the method that will take care of the issues with Adding it to the list.
Parallel.ForEach(collection.Current.ToList(), (ticket) =>
{
Console.WriteLine("Ticket----" + ticket);
dynamic groupWithTickets = new ExpandoObject();
groupWithTickets.users = ticket; //<---- Assign ticket to users element.
topGroupsWithMaxTickets.Add(groupWithTickets);
});
What was done to fix it?
When you are working with ExpandoObjects, you have to think of dictionary type of a deal. When you declare ExpandoObject, you have to dynamically assign the value to an element (that you define).
Example from MS site: shows the proper use of ExpandoObject
dynamic employee, manager;
employee = new ExpandoObject();
employee.Name = "John Smith";
employee.Age = 33;
manager = new ExpandoObject();
manager.Name = "Allison Brown";
manager.Age = 42;
manager.TeamSize = 10;
Hopefully this resolves your issue.
It should be like this;
dynamic ticketDetails = new ExpandoObject();
ticketDetails.user = collection;
string json = Newtonsoft.Json.JsonConvert.SerializeObject(ticketDetails);
You can simply do this:
dynamic ticketDetails = new ExpandoObject();
ticketDetails = Json(new users = JsonConvert.DeserializeObject(collection.Current.ToJson()));
For testing purposes, i used an array arr that holds one of the elements. If you need that array to be part of ExtendoObject with first element being users, you can create a Json object and set the array as value to the "users" element.
dynamic ticketDetails = new ExpandoObject();
JArray arr = new JArray();
arr.Add(#"[{""name"": ""Alex"", ""age"": 21}]");
JObject o = new JObject();
o["users"] = arr.ToString();
ticketDetails = o;
// output: { "users" : [{"name" : "Alex", "age" : 21}]}
I have an api based on asp.net core 2.2 in which i am building up an array of ips(strings type) like this
[HttpGet ("{nsp}/geolocation")]
[ResponseCache (Duration = 0, Location = ResponseCacheLocation.None, NoStore = true)]
public async Task<dynamic> getLocation (string nsp) {
nsp = "/"+nsp;
// string ipInfoBaseUrl = "http://ip-api.com/json/";
string baseUrl = "http://ip-api.com/batch";
// string userIpAddress = "197.157.194.90";
// string ipUrl = ipInfoBaseUrl + userIpAddress;
// var client = _httpClientFactory.CreateClient();
// var result = await client.PostAsync(baseUrl,new StringContent(JsonConvert.SerializeObject(finals), System.Text.Encoding.UTF8, "application/json"));
// // var final = Newtonsoft.Json.JsonConvert.DeserializeObject<UserLocation>(result);
// Console.WriteLine(finals+" --result-----");
// return Ok(result);
var match = new BsonDocument ();
var group = new BsonDocument ();
var project = new BsonDocument ();
var sort = new BsonDocument ();
var addFields = new BsonDocument ();
var pipeline = new [] { new BsonDocument () };
/* #Aggregation : Stage-1 */
match = new BsonDocument ("$match",
new BsonDocument {
{
"nsp" , nsp
}
});
/* #Aggregation : Stage-2 */
group = new BsonDocument("$group",
new BsonDocument
{ {
"_id", "null"
},
{ "geoLocations",
new BsonDocument("$addToSet", "$visitor.ip")
}
});
/* #Aggregation : Stage-3 */
project = new BsonDocument ("$project", new BsonDocument { { "_id", 0 }});
pipeline = new [] { match, group,project};
var list = await DbService.tickets.AggregateAsync<BsonDocument> (pipeline, new AggregateOptions { UseCursor = true, BatchSize = batchCount });
while (await list.MoveNextAsync ()) {
var list_real = new List<BsonValue> ();
foreach (var data in list.Current.ToArray ()) {
list_real.Add (data);
}
return list_real.ToJson ();
}
return new BsonArray ().ToJson ();
}
It is returning result like this
[
{
" geoLocations": [
"122.8.208.9",
"196.62.107.243",
"182.188.38.219",
"39.50.244.198",
"39.51.40.251",
"103.20.134.56",
"103.228.156.83",
"202.143.125.21",
"196.62.151.47",
"45.116.232.50",
"39.57.128.75",
"103.18.8.60",
"202.143.125.20",
"182.190.252.96",
"119.153.56.2",
"46.101.89.227",
"196.194.172.211",
"192.168.20.186",
"64.233.173.146",
"104.236.195.147",
"39.50.156.242",
"103.255.5.58"
]
}
]
How can i get comma separated string from this result like
"111.92.158.82","202.142.168.162","122.8.157.172",.....
From very first i am getting all ips from all documents from my collection and forming an array of ips.But my ultimate goal is to form a comma separated string from that array because i have to pass that comma separated string of ips into an api to get ips locations.
I am using asp.net core and c#. How can i achieve this?
Assuming you want a single comma-separated result string containing all IP addresses, replace the method signature with IEnumerable<string>, and replace the bottom of the method with. Just use whatever you need for your result and get rid of the rest.
var list = await DbService.tickets.AggregateAsync<BsonDocument> (pipeline,
new AggregateOptions
{
UseCursor = true,
BatchSize = batchCount
});
var result = new List<string>();
while (await list.MoveNextAsync())
result.AddRange(list.Current.Cast<string>());
return string.Join(',', result);
I'm not sure why you're doing everything with BsonDocuments, you can just iterate over the data directly and return strings.
Also, consider upgrading to .NET Core 3, which you can then use C# 8's async enumerable. You'll also be able to use the new JSON functionality built-in .NET Core 3.
This will return:
"122.8.208.9,196.62.107.243,182.188.38.219,<all the rest>"
I have a DynamoDB query that works fine on the AWS Console but it doesn't on code.
Here is my query on the console:
Now here is my c# code to query it:
var query = new QueryOperationConfig
{
KeyExpression = new Expression
{
ExpressionStatement = "#pkey = :v_pkey and #skey >= :v_skey",
ExpressionAttributeNames = {
{ "#pkey", "MailingId" },
{ "#skey", "RegistroCarteiraId" },
},
ExpressionAttributeValues = new Dictionary<string, DynamoDBEntry>()
{
{ ":v_pkey", new Primitive("62", true) },
{ ":v_skey", new Primitive("00e0bbfc-aed0-4f0e-acef-a3623a9f9694") },
},
},
BackwardSearch = false,
ConsistentRead = true,
Limit = 1,
FilterExpression = new Expression
{
ExpressionStatement = "#psituacao = :v_psituacao and attribute_not_exists(#pdisponibilidade)",
ExpressionAttributeNames =
{
{ "#psituacao", "Situacao" },
{ "#pdisponibilidade", "Disponibilidade" }
},
ExpressionAttributeValues =
{
{ ":v_psituacao", new Primitive("1", true) },
}
}
};
var search = table.Query(query);
var docs = await search.GetNextSetAsync();
I get no errors, only an empty array as the result. If I change the sort key to different values, it works, but for this particular value it does not...
I've been at it all day and couldn't figure it out what is wrong.
Any help will be much appreciated.
Thanks
The problem was the LIMIT 1.
As I found out, the filter only happens on the fetched items and, since I was only fetching 1 item, when the filter occurred, the result had no records that matched the criteria.
Removing the Limit 1 solved the problem.
I am trying to retrieve index statistics using the MongoDB .NET Driver.
I have tried the following variations of my pipeline
var statsPipeline = new[] { new BsonDocument(new BsonElement("$indexStats", BsonNull.Value)) };
var statsPipeline = new[] { new BsonDocument { {"$indexStats", "" } } };
var statsPipeline = new[] { new BsonDocument { {"$indexStats", null } } };
var statsPipeline = new[] { new BsonDocument { {"$indexStats", BsonNull.Value } } };
var statsPipeline = new[] { new BsonDocument { {"$indexStats", "{ }"} } };
which is passed to the query
var stats = await db
.GetCollection<BsonDocument>("CollectionName")
.AggregateAsync<BsonDocument>(statsPipeline);
With the exception of the one containing null, which resulted in an ArgumentNullException, I have received the exception
MongoDB.Driver.MongoCommandException: Command aggregate failed: The $indexStats stage specification must be an empty object.
How do I change my query such that the $indexStats stage specification is indeed an empty object?
Ok, this one worked:
var statsPipeline = new[] { new BsonDocument(new BsonElement("$indexStats", new BsonDocument())) };
Given this class:
class SomeClass
{
public int SomeValue { get; set; }
}
The following list:
var queryableData = new List<SomeClass>() {
new SomeClass{SomeValue=1 },
new SomeClass{SomeValue=2 },
new SomeClass{SomeValue=3 },
new SomeClass{SomeValue=4 },
new SomeClass{SomeValue=5 },
new SomeClass{SomeValue=6 },
new SomeClass{SomeValue=7 },
}.AsQueryable();
And this code where i try to dinamically query the list (note that the string query could contain anything such as Take,Select,OrderBy, etc).
var externals = new Dictionary<string, object>();
externals.Add("SomeClass", queryableData);
string query = "SomeClass.Where(o => o.SomeValue >= 3)"; // or any query
// here i use the code from System.Linq.Dynamic
var expression = DynamicExpression.Parse(typeof(IQueryable<SomeClass>), query, new[] { externals });
// result will have five values
var result = queryableData.Provider.CreateQuery<SomeClass>(expression);
// here i use the code from System.Linq.Dynamic.Core
var expression2 = DynamicExpressionParser.ParseLambda(typeof(IQueryable<SomeClass>), query, new[] { externals });
// will throw exception here: Argument expression is not valid
var result2 = queryableData.Provider.CreateQuery<SomeClass>(expression2).ToDynamicArray();
I want to know what am i doing wrong that the Provider.CreateQuery is thowing the "Argument expression is not valid" exception?
The difference with DynamicExpression.Parse (hence the cause of the problem) is that the DynamicExpressionParser.ParseLambda method returns LambdaExpression (basically Expression<Func<TResult>>) which is not a valid query expression.
But the Body of it is, so the simplest fix is to use
var result2 = queryableData.Provider.CreateQuery<SomeClass>(expression2.Body);
Alternatively you could use System.Linq.Dynamic.Core.Parser.ExpressionParser class directly:
var expression2 = new ExpressionParser(null, query, new[] { externals }, null)
.Parse(typeof(IQueryable<SomeClass>));
var result2 = queryableData.Provider.CreateQuery<SomeClass>(expression2);