Develop Reference

MongoDB grouping by month using aggregation timezone issue

I have the following aggregation structure where it groups the data by month and sums it up on a monthly basis starting from the beginning of the year. At the moment my documents are starting at month 9 and nothing before that.When I run the below aggregation, result shows that data starts from the month 8. I guess it is related to the timezone. Because only 3 hours of data (my timezone is +3) is listed in the month 8.
db.collectedData.aggregate(
// Pipeline
[
// Stage 1
{
$match: {
_Timestamp: {
$gte: ISODate("2017-01-01T21:00:00.000Z"),
$lte: ISODate("2019-04-19T01:30:00.000Z")
},
"Registers.totalconsumedactiveenergy" : {"$exists" : true}
}
},
// Stage 2
{
$project: {
"Datem" : "$_Timestamp",
"month" : {"$month" : "$_Timestamp" },
"device" : "$_DeviceID",
"datam" : "$$ROOT.Registers.totalconsumedactiveenergy"
}
},
// Stage 3
{
$group: {
"_id": {
"Month": "$month",
"Device" : "$device"
},
first:{$first:"$$ROOT.datam"},
last: { $last: "$$ROOT.datam" }
}
},
// Stage 4
{
$project: {
totalConsumption: { $sum: { $subtract: [ "$last", "$first" ] } },
month: "$_id.Month"
}
},
// Stage 5
{
$group: {
"_id": "$month",
"total": { "$sum": "$totalConsumption" }
}
},
// Stage 6
{
$sort: {
"_id" : 1
}
},
]
);
This works and gives me results like this
From my application I am sending localtimezone to the aggregation but still no success.
Earliest date in my documents is
2017-08-31T21:00:00.000Z (UTC)
How can I solve this?

Nested array $pull query using C# MongoDB driver

I have this following query working on mongo shell as expected.
db.getCollection('personnels').update(
{
_id: ObjectId("55f6728b9d73a15807885de8"),
"Devices._id":ObjectId("55fa5f7ac9e7863a3836e331")
},
{
$pull:{ "Devices.$.DeviceCloudFolders": { "CloudFolderId": ObjectId("5615124b06275f072040c4f1")}}
}
);
And here is my document structure:
{
"_id" : ObjectId("55f6728b9d73a15807885de8"),
"FirstName" : "Tolga",
"Devices" : [
{
"_id" : ObjectId("55fa5f7ac9e7863a3836e331"),
"Name" : "tolga-laptop",
"DeviceCloudFolders" : [{
"AuthorityType" : 1,
"CloudFolderId" : ObjectId("55f96db5c9e7863a3836e310"),
"Status" : 1
}],
"Status" : 1
}
],
"Status" : 1
}
I need to use it in C# and couldn't figure out how.
I started with these lines:
var filter = Builders<Personnel>.Filter.And(
Builders<Personnel>.Filter.Eq("_id", ownerPersonnelId),
Builders<Personnel>.Filter.Eq("Devices._id", _id));
var update = Builders<Personnel>.Update.PullFilter("Devices.$.DeviceCloudFolders", /*couldn't figure out what goes here*/))
Personnels.FindOneAndUpdateAsync(filter, update);

I'm not sure, but you can try using this:
var update = Builders<Personnel>.Update.PullFilter(
"Devices.$.DeviceCloudFolders",
Builders<DeviceCloudFolder>.Filter.Eq("CloudFolderId", _cloudFolderId));

Exact Match Array in Mongo Query

Using Linq syntax, how would you exact match an array inside of a document?
This query works great except that it matches documents who have other Children with ages not 3 or 4.
var query = collection.AsQueryable<Parent>().Where(p => p.Children.Any(c => c.Age == 3) && p.Children.Any(c => c.Age == 4));
For example, it should NOT return this document:
{
"_id" : ObjectId("5514c620923a9b55e22f0adf"),
"Name" : "Bob",
"Children" : [
{
"Name" : "Kid1",
"Age" : 5
},
{
"Name" : "Kid2",
"Age" : 4
},
{
"Name" : "Kid3",
"Age" : 3
}
]
}
And should match this document:
{
"_id" : ObjectId("5514c620923a9b55e22f0adf"),
"Name" : "Bob",
"Children" : [
{
"Name" : "Kid2",
"Age" : 4
},
{
"Name" : "Kid3",
"Age" : 3
}
]
}
This is basically a duplicate of this question, but using linq c# syntax.
Matching an array field which contains any combination of the provided array in MongoDB
Bonus points if you can also show it using c# QueryBuilder instead of linq.

You could create another array, which includes a distinct list of the ages. I.e.:
{
ages: [3,4],
children: [
{ name: "A", age: 3 },
{ name: "A", age: 4 },
{ name: "A", age: 3 }
]
}
Now you can use the $all query: http://docs.mongodb.org/manual/reference/operator/query/all/
Combined with the $size query: http://docs.mongodb.org/manual/reference/operator/query/size/
{ ages: { $all: [3,4], $size: 2 } }

Mongodb performance issue with simple search query

This is a sample structure of a collection i am having in mongodb, likewise some 60+ lakh similar records are there in one collection. We moved to mongodb seeing the fast accessing performance,but now its bit hard to say evean simple searching is taking huge time. I have given index for the search params. For name searching (search params MNG_FIRST_NAME, MNG_MIDDLE_NAME, MNG_LAST_NAME, MNG_ALIASIST.AKAFIRST_NAME, MNG_ALIASIST.AKALAST_NAME).
Please consider is the method i'm following the correct way with mongodb.
Please find the code sample and collection structure below i'm using.
{
"_id" : ObjectId("5308e2e1f525bc0f0472cbf6"),
"MNG_UID" : "1389120",
"MNG_CATEGORY" : "OFFICER",
"MNG_FIRST_NAME" : "JOHN",
"MNG_LAST_NAME" : "DOE",
"MNG_MIDDLE_NAME" : "MARK",
"MNG_CREATEDON" : "2010-11-19",
"MNG_UPDATEDON" : "2014-01-01",
"MNG_TITLE" : "Dr",
"MNG_REMARKS" : "",
"MNG_TYPE" : "WORLDCHECK",
"MNG_PROGRAMLIST" : [],
"MNG_IDLIST" : [],
"MNG_ALIASIST" : [
{
"AKAUID" : "101",
"AKATYPE" : "ALIAS",
"AKACATEGORY" : "",
"AKAFIRST_NAME" : "JOHNNY TURA, DOE",
"AKALAST_NAME" : ""
},
{
"AKAUID" : "102",
"AKATYPE" : "ALIAS",
"AKACATEGORY" : "",
"AKAFIRST_NAME" : "MARK TURA, DOE",
"AKALAST_NAME" : ""
}
],
"MNG_ADDRESSLIST" : [
{
"ADDRESSUID" : "",
"ADDRESS1" : "Dakar",
"ADDRESS2" : "Dakar",
"ADDRESS3" : "SENEGAL",
"ADDRESSCITY" : "Dakar",
"ADDRESSCOUNTRY" : "SENEGAL",
"ADDRESSPOCODE" : "",
"ADDRESSSTATE" : "Dakar"
}
],
"MNG_NATIONALITYLIST" : [
{
"NATUID" : "",
"NATCOUNTRY" : "ISRAEL",
"NATMAINENTRY" : ""
},
{
"NATUID" : "",
"NATCOUNTRY" : "AMERICIAN",
"NATMAINENTRY" : ""
}
],
"MNG_CITIZENSHIPLIST" : [],
"MNG_DOBLIST" : [],
"MNG_PLACEOBLIST" : [],
"MNG_ADDINFOLIST" : []
}
code used to obtain collection using mongodb c# driver
string MNG_REMTNAME = "JOHN";
var collection = mongoDB.GetCollection<EMPLOYEES>("EMPLOYEES");
if (!string.IsNullOrEmpty(MNG_REMTNAME))
{
IMongoQuery MFName = new QueryDocument();
MFName = Query.Or(Query.Matches("MNG_FIRST_NAME", MNG_REMTNAME.ToUpper()),
Query.Matches("MNG_MIDDLE_NAME", MNG_REMTNAME.ToUpper()),
Query.Matches("MNG_LAST_NAME", MNG_REMTNAME.ToUpper()),
Query.ElemMatch("MNG_ALIASIST", Query.Or(Query.Matches("AKAFIRST_NAME",
MNG_REMTNAME.ToUpper())))
);
}
var colln = collection.Find(MFName).ToList();
Please help with some solution to increase the performance.

Change the line
Query.ElemMatch("MNG_ALIASIST", Query.Or(Query.Matches("AKAFIRST_NAME",
MNG_REMTNAME.ToUpper())))
for this one
Query.ElemMatch("MNG_ALIASIST", Query.EQ("AKAFIRST_NAME", MNG_REMTNAME.ToUpper()))
Also have in mind that searching in MongoDB Arrays is very expensive unless you have the proper Indexes. So make sure that you have an index with at least the field MNG_ALIASIST.
Consider creating a Multikey index for this field http://docs.mongodb.org/manual/core/index-multikey/

Searching an elasticsearch index with NEST yields no results

I'm playing around with NEST, and I'm adding stuff to the ES index without any problems. Getting any results back is another matter, however.
Here's the output in my browser for /nest-testing/_search/?pretty
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 1.0,
"hits" : [ {
"_index" : "nest-testing",
"_type" : "plain_text",
"_id" : "\\\\domain.net\\path\\to\\lorem.txt",
"_score" : 1.0, "_source" : {
"id": "\\\\domain.net\\path\\to\\lorem.txt",
"full_unc_path": "\\\\domain.net\\path\\to\\lorem.txt",
"file_name": "lorem.txt",
"parent_directory": "\\\\domain.net\\path\\to\\lorem",
"date_created": "2014-01-28T15:47:30.4302806Z",
"date_modified": "2014-01-29T15:47:30.4302806Z",
"fulltext": "Also, I like spaghetti and Meatballs.",
"original_filesize": 37,
"extracted_text_size": 37
}
} ]
}
}
Here's my NEST call:
var result = client.Search(s => s
.Index(TEST_INDEX)
.MatchAll()
);
I get an empty collection back when I run it. (QueryString searches don't work, either.) Making the query less specific ( .AllIndices() ) doesn't change the result.
Any ideas?

Since you are not explicitly saying what the return type should be NEST will assume Search<dynamic>
If you do so you need to explicitly tell it which index or type you want to search through at or .AllIndices() and/or .AllTypes()