Develop Reference

How to build mongo aggregation pipeline with nested and/or conditions?

I'm trying to build an aggregation pipeline that that filters out documents using a AND(OR(AND) condition.
What I need is something like
if (A == "A" && B == "B" && (C == NULL || (C != NULL && C == "C")))
My query looks like below.
The inner part:
var x = new BsonDocument
{
{
"$or",
new BsonArray
{
new BsonDocument {{"C", BsonNull.Value}},
new BsonDocument
{
{
"$and",
new BsonArray
{
new BsonDocument
{
{
"C",
new BsonDocument {{"$ne", BsonNull.Value}}
}
},
new BsonDocument {{"C", C}}
}
}
}
}
}
};
Then the full filter is
var filter = new BsonDocument
{
{
"$and",
new BsonArray
{
new BsonDocument {{"A", A}},
new BsonDocument {{"B", B}},
x
}
}
};
Then I call this like:
var y = await collection.Aggregate().Match(filter).ToListAsync();
However, it not returning any results though it should.
In my debugger, final Bson looks like:
{
{
"$and": [
{
"A": "A"
},
{
"B": "B"
},
{
"$or": [
{
"C": null
},
{
"$and": [
{
"C": {
"$ne": null
}
},
{
"C": "C"
}
]
}
]
}
]
}
}

Managed to figure it out. I changed the query to build it as a match as well since it's easier to test with mongo shell:
var match = new BsonDocument
{
{
"$match",
new BsonDocument
{
{
"$and",
new BsonArray
{
new BsonDocument {{"A", A}},
new BsonDocument {{"B", B}},
new BsonDocument
{
{
"$or",
new BsonArray
{
new BsonDocument {{C, BsonNull.Value}},
new BsonDocument
{
{
"$and",
new BsonArray
{
new BsonDocument
{
{
"C",
new BsonDocument {{"$ne", BsonNull.Value}}
}
},
new BsonDocument {{"C", C}}
}
}
}
}
}
}
}
}
}
}
};
Hope this helps anyone running into a problem with similar query.

Aggregating By match date $or with other field in Mongodb in C#

I am using mongo aggregation with match/group. Below is my mongo query & similar i am trying to convert to c#. Able to get date part but struggling for $or part for "ev" field. Need to apply below match/or/group/sort on one collection & lookup on other collection.
db.getCollection("customer").aggregate(
[
{
"$match" : {
"eventTs" : {
"$gte" : ISODate("2020-02-27T00:00:00.000-0500"),
"$lte" : ISODate("2020-02-28T00:00:00.000-0500")
},
$or: [
{ "ev": "JournalAdded" },
{ "ev": "JournalProcessed" },
{ "ev": "JournalModified" },
{ "ev": "JournalVoided" },
{ "ev": "JournalApproved" },
{ "ev": "JournalCancelled" }
]
}
},
{
"$sort" : {
"eid" : 1.0,
"eseq" : 1.0
}
},
{
"$group" : {
"_id" : "$eid",
lastSeq: {$last: "$eseq"},
eventNames:{$push: "$ev"}
}
},
{
"$lookup":
{
from: "xyz",
localField: "_id",
foreignField: "_eid",
as: "Replicated"
}
}
]
);
working c# code:
var match = new BsonDocument
{
{
"$match",
new BsonDocument
{
{ "eventTs", new BsonDocument
{
{ "$gte", startTs },
{ "$lte", endTs }
}
},
{ "ev", "JournalAdded" },
//{ "ev", "JournalProcessed" },
//{ "ev", "JournalModified" },
//{ "ev", "JournalVoided" },
//{ "ev", "JournalApproved" },
//{ "ev", "JournalCancelled" }
}
}
};
var pipeline = new[] { match };
var result = instructionEventsDocs.Aggregate<BsonDocument>(pipeline);
List<BsonDocument> list = result.ToList();
Need c# equivalent of above mongo query. Can anyone help me with this ?

You can also add the BsonArray after the BsonDocument for $or part for "ev" field,look into the answers in this link Adding BSON array to BsonDocument in MongoDB

var match = new BsonDocument
{
{
"$match",
new BsonDocument
{
{ "eventTs", new BsonDocument
{
{ "$gte", startTs },
{ "$lte", endTs }
}
},
{
"$or", new BsonArray {
new BsonDocument
{
{ "ev", "JournalAdded" }
},
new BsonDocument
{
{ "ev", "JournalProcessed" }
}
}
},
}
}
};
Equivalent mongo query
{{ "$match" : { "eventTs" : { "$gte" : ISODate("2020-02-27T05:00:00Z"), "$lte" : ISODate("2020-02-28T05:00:00Z") }, "$or" : [{ "ev" : "JournalAdded" }, { "ev" : "JournalProcessed" }] } }}

Need help writing data to json in c#

I am webscraping some data and trying to write the scraped data to a json file using c# newtonsoft.Json.
I get stuck when writing the data to the Json file in my controller.
The multidimensional arrays in c# confuse me.
Thanks in advance.
This is an example of the Json file I am trying to create:
[
{
"testmodule1": {
"name": {
"required": true,
"options": [
"option1",
"option2"
]
},
"admin": {
"required": true,
"options": [
"option1",
"option2"
]
},
"path": {
"required": true,
"options": [
"option1",
"option2"
]
}
}
},
{
"testmodule2": {
"name": {
"required": true,
"options": [
"option1",
"option2"
]
},
"server": {
"required": false,
"options": [
]
},
"port": {
"required": true,
"options": [
"option1",
"option2"
]
}
}
}
]
These are my classes:
public class JsonData
{
public Dictionary<string, JsonParameters> modulename { get; set; }
}
public class JsonParameters
{
public JsonParametersData parameter { get; set; }
}
public class JsonParametersData
{
public bool required { get; set; }
public List<string> options { get; set; }
}
This is my controller, here is where I get stuck. the name modulename does not exist in the current context:
public class WebscrapeController : Controller
{
// GET: Webscrape
public ActionResult Index()
{
List<JsonData> data = new List<JsonData>();
data.Add(new JsonData()
{
modulename = new Dictionary<string, JsonParameters>()
{
modulename.Add("testmodule1", new JsonParameters()
{
parameter = new JsonParametersData()
{
required = true,
options = new List<string>()
{
"option1",
"option2"
}
}
})
}
});
string json = JsonConvert.SerializeObject(data.ToArray());
//write string to file
System.IO.File.WriteAllText(
#"C:mypath",
json);
}
}
Note that the property names "testmodule1" and "testmodule2" as well as "name", "admin", "path", "server" are arbitrary; they differ for each array.

Since the property names "testmodule1" and "testmodule2" as well as "name", "admin", "path", "server" and "port" are arbitrary and not known in advance, you need to model your results array as a List<Dictionary<string, Dictionary<string, JsonParametersData>>>. That is because, when serializing a dictionary to JSON using Json.NET, the dictionary keys become JSON property names.
Thus the JSON above can be created as follows:
// Allocate results using collection initializer syntax for the lists and for the dictionaries.
// https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/object-and-collection-initializers#collection-initializers
// https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/how-to-initialize-a-dictionary-with-a-collection-initializer
var results = new List<Dictionary<string, Dictionary<string, JsonParametersData>>>()
{
new Dictionary<string, Dictionary<string, JsonParametersData>>()
{
{
"testmodule1",
new Dictionary<string, JsonParametersData>()
{
{
"name",
new JsonParametersData
{
required = true,
options = new List<string>() { "option1", "option2" },
}
},
{
"admin",
new JsonParametersData
{
required = true,
options = new List<string>() { "option1", "option2" },
}
},
{
"path",
new JsonParametersData
{
required = true,
options = new List<string>() { "option1", "option2" },
}
}
}
},
}
};
var moduleName = "testmodule2";
var moduleParameters = new [] { "name", "server", "port" };
// Now add another result, allocating the dictionary with collection initializer syntax also
results.Add(new Dictionary<string, Dictionary<string, JsonParametersData>>()
{
{
moduleName,
// Loop through the parameters and convert them to a dictionary,
// where the parameter name is the key and the corresponding JsonParametersData is the value
moduleParameters
.ToDictionary(n => n,
n => new JsonParametersData
{
required = true,
options = new List<string>() { "option1", "option2" },
})
}
});
var json = JsonConvert.SerializeObject(results, Formatting.Indented);
Notes:
For documentation on how dictionaries are serialized to JSON, see Serialize a Dictionary and Serialization Guide: Dictionaries and Hashtables.
I am initializing the outermost List<T> using collection initializer syntax.
I am also initializing the dictionaries using collection initializer syntax as shown in How to: Initialize a Dictionary with a Collection Initializer (C# Programming Guide).
Given a collection of parameter names and a way to get the JsonParametersData for each one (not shown in the question), the LINQ extension method Enumerable.ToDictionary<TSource, TKey, TElement>() can be used to construct a Dictionary<string, JsonParametersData> from the parameter collection.
Working sample .Net fiddle here.

Here is a different approach using Newtonsoft JObject.
https://dotnetfiddle.net/TdFDQc
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using System;
namespace StackOverflow
{
public class Program
{
public static void Main(string[] args)
{
JArray array = new JArray();
// Module 1
JObject parameter = new JObject();
AddParameter(parameter, "name", true, new[] { "option1", "option2" });
AddParameter(parameter, "admin", true, new[] { "option1", "option2" });
AddParameter(parameter, "path", false, new[] { "option1", "option2", "option3" });
JObject module = new JObject();
module.Add("testmodule1", parameter);
array.Add(module);
// Module 2
parameter = new JObject();
AddParameter(parameter, "name", true, new[] { "option1", "option2" });
AddParameter(parameter, "server", false, Array.Empty<string>());
AddParameter(parameter, "port", true, new[] { "option1", "option2", "option3" });
module = new JObject();
module.Add("testmodule2", parameter);
array.Add(module);
// Display result
var json = array.ToString();
Console.WriteLine(json);
}
static void AddParameter(JObject jObject, string name, bool required, string[] options)
{
JObject parameterProperties = new JObject();
parameterProperties.Add("required", JToken.FromObject(required));
parameterProperties.Add("options", JToken.FromObject(options));
jObject.Add(name, parameterProperties);
}
}
}

C# JToken.SelectTokens Method - what JPath expression?

I have json text like this:
{
"course_editions": {
"2014/SL": [
{
"grades": {
"course_units_grades": {
"159715": {
"1": {
"value_symbol": "4",
"exam_session_number": 1,
"exam_id": 198172,
"value_description": {
"en": "good",
}
}
}
},
"course_grades": {}
}
},
{
"grades": {
"course_units_grades": {
"159796": {
"1": {
"value_symbol": "5",
"exam_session_number": 1,
"exam_id": 198259,
"value_description": {
"en": "very good",
}
}
}
},
"course_grades": {}
}
},
I would like to use JToken.SelectTokens Method from Namespace: Newtonsoft.Json.Linq
I've tried like this:
string json_response = GetResponse(sign(url_courses));
var courses_tokens = JObject.Parse(json_response).SelectTokens("['course_editions'].['2014/SL'].[*].['grades'].*")
It doesn't work. I would like to get only these numbers after course_unit_grades and before "1". So in this example only: "159715" and "159796" to be able to use all of them, one by one in
foreach(var lp in courses_tokens) {
}

This is one possible way :
var jobj = JObject.Parse(json);
var coursesTokens = jobj.SelectTokens("course_editions.2014/SL[*].grades.course_units_grades")
.Select(o => o.First) //get the first child of `course_units_grades`
.Cast<JProperty>() //cast to JProperty
.Select(o => o.Name); //get the name of the property
foreach (string coursesToken in coursesTokens)
{
Console.WriteLine(coursesToken);
}
Dotnetfiddle Demo
given json sample at the bottom, the output is as follow :
159715
159796
json sample :
var json = #"{
'course_editions': {
'2014/SL': [
{
'grades': {
'course_units_grades': {
'159715': {
'1': {
'value_symbol': '4',
'exam_session_number': 1,
'exam_id': 198172,
'value_description': {
'en': 'good'
}
}
}
},
'course_grades': {}
}
},
{
'grades': {
'course_units_grades': {
'159796': {
'1': {
'value_symbol': '5',
'exam_session_number': 1,
'exam_id': 198259,
'value_description': {
'en': 'very good'
}
}
}
},
'course_grades': {}
}
}
]
}
}";

Adding BSON array to BsonDocument in MongoDB

How can I add BsonArray to BsonDocument in MongoDB using a C# driver? I want a result something like this
{
author: 'joe',
title : 'Yet another blog post',
text : 'Here is the text...',
tags : [ 'example', 'joe' ],
comments : [ { author: 'jim', comment: 'I disagree' },
{ author: 'nancy', comment: 'Good post' }
]
}

You can create the above document in C# with the following statement:
var document = new BsonDocument {
{ "author", "joe" },
{ "title", "yet another blog post" },
{ "text", "here is the text..." },
{ "tags", new BsonArray { "example", "joe" } },
{ "comments", new BsonArray {
new BsonDocument { { "author", "jim" }, { "comment", "I disagree" } },
new BsonDocument { { "author", "nancy" }, { "comment", "Good post" } }
}}
};
You can test whether you produced the correct result with:
var json = document.ToJson();

you can also add the array after the BsonDocument already exists, like this:
BsonDocument doc = new BsonDocument {
{ "author", "joe" },
{ "title", "yet another blog post" },
{ "text", "here is the text..." }
};
BsonArray array1 = new BsonArray {
"example", "joe"
};
BsonArray array2 = new BsonArray {
new BsonDocument { { "author", "jim" }, { "comment", "I disagree" } },
new BsonDocument { { "author", "nancy" }, { "comment", "Good post" } }
};
doc.Add("tags", array1);
doc.Add("comments", array2);