I have the following aggregation structure where it groups the data by month and sums it up on a monthly basis starting from the beginning of the year. At the moment my documents are starting at month 9 and nothing before that.When I run the below aggregation, result shows that data starts from the month 8. I guess it is related to the timezone. Because only 3 hours of data (my timezone is +3) is listed in the month 8.
db.collectedData.aggregate(
// Pipeline
[
// Stage 1
{
$match: {
_Timestamp: {
$gte: ISODate("2017-01-01T21:00:00.000Z"),
$lte: ISODate("2019-04-19T01:30:00.000Z")
},
"Registers.totalconsumedactiveenergy" : {"$exists" : true}
}
},
// Stage 2
{
$project: {
"Datem" : "$_Timestamp",
"month" : {"$month" : "$_Timestamp" },
"device" : "$_DeviceID",
"datam" : "$$ROOT.Registers.totalconsumedactiveenergy"
}
},
// Stage 3
{
$group: {
"_id": {
"Month": "$month",
"Device" : "$device"
},
first:{$first:"$$ROOT.datam"},
last: { $last: "$$ROOT.datam" }
}
},
// Stage 4
{
$project: {
totalConsumption: { $sum: { $subtract: [ "$last", "$first" ] } },
month: "$_id.Month"
}
},
// Stage 5
{
$group: {
"_id": "$month",
"total": { "$sum": "$totalConsumption" }
}
},
// Stage 6
{
$sort: {
"_id" : 1
}
},
]
);
This works and gives me results like this
From my application I am sending localtimezone to the aggregation but still no success.
Earliest date in my documents is
2017-08-31T21:00:00.000Z (UTC)
How can I solve this?
Related
I have following json object that will come from database before send the object to UI need to sort or order by the weekday of empWeek object in the following order using linq.Currently we can see in the list of empWeek getting "Thursday" object first but it should always start with sunday, monday...and saturday . Need sample code in C#
Sunday,Monday,Tuesday,Wednesday, Thursday,Friday and Saturday
{
"empId":1
"empName":test
"empWeek":[
{
"id":100
"weekDay":"Thursday",
},
{
"id":102
"weekDay":"Friday",
},
{
"id":103
"weekDay":"Saturday",
},
{
"id":105
"weekDay":"Sunday",
},
{
"id":100
"weekDay":"Mondday",
},
{
"id":100
"weekDay":"Tuesday",
},
{
"id":100
"weekDay":"Thursday",
}
]
}
I have the following db config:
db={
"order": [
{
"id": 1,
"version": 1
},
{
"id": 1,
"version": 2
},
{
"id": 2,
"version": 1
},
{
"id": 2,
"version": 2
}
],
"orderDetail": [
{
"orderId": 1,
"orderDate": new Date("2020-01-18T16:00:00Z")
},
{
"orderId": 1,
"orderDate": new Date("2020-01-11T16:00:00Z")
},
{
"orderId": 1,
"orderDate": new Date("2020-01-12T16:00:00Z")
}
]
}
I'm using the fluent interface to perform a Lookup joining the orderDetails to the order collection (as shown in this post). Now that I have the join in place what's the best method to:
Sort the joined array such that the details are sorted by orderDate
Group the Orders (by OrderID) and sort by version to select the latest (largest Version #)
The workaround I implemented for #1 involves sorting the list after performing the lookup, but that's only because I wasn't able to apply a sort to the "as" of collection as part of the Lookup.
If anyone has any ideas, I'd appreciate it. Thanks!
If you are using MongoDB v3.6 or higher, you can use the $lookup with uncorrelated subqueries to use the inner pipelines to archive what you want.
Join Conditions and Uncorrelated Sub-queries
Since you didn't provide what collections or fields you are using, I will give a generic example:
db.customers.aggregate([
{
$lookup: {
from: "orders",
let: { customer_id: "$_id" },
pipeline: [
{ $match: { $expr: { $eq: [ "$customer_id", "$$customer_id" ] } } },
{ $sort: { orderDate: -1 } }
],
as: "orders"
}
}
]);
I hope that gives you a way to get where you want. =]
I am querying a CosmosDB in such a way that I am getting a string in and ned to return some data out through a C# WEB API, the query that works for me is as below
SELECT *
FROM c IN jongel.OriginalData.base.sales.variants
WHERE c.globalTradeItemNumber.globalTradeItemNumberType[0].GTIN = '1111111111111'
The problem is that I have to know the ARRAY INDEX for the globalTradeItemNumberType ARRAY, [0] in this example, for it to work but it is not always 0, it could be any number from 0-9 basically and I cannot figure out how to rewrite the query so that it works regardless of the index where the matching data is found?
How can I rewrite this query so that I do not need to know the ARRAY INDEX beforehand?
--- EDIT ---
A sample document shortened to only include the needed parts
{
"id": "635af816-8db7-49c6-8284-ab85116b499b",
"brand": "XXX",
"IntegrationSource": "XXX",
"DocumentType": "Item",
"ItemInformationType": "",
"ItemLevel": "Article",
"ItemNo": "0562788040",
"UpdatedDate": "1/1/2020 4:00:01 AM",
"UpdatedDateUtc": "2020-01-01T04:00:01.82Z",
"UpdatedBy": "XXX",
"OriginalData": {
"corporateBrandId": "2",
"productId": "0562788",
"articleId": "0562788040",
"season": "201910",
"base": {
"sales": {
"SAPArticleNumber": "562788040190",
"simpleColour": {
"simpleColourId": "99",
"simpleColourDescription": "Green",
"translatedColourDescription": [
{
"languageCode": "sr",
"simpleColourDescription": "Zeleno"
},
{
"languageCode": "zh-Hans",
"simpleColourDescription": "绿色"
},
{
"languageCode": "vi-VN",
"simpleColourDescription": "Xanh la cay"
}
]
},
"variants": [
{
"variantId": "0562788040001",
"variantNumber": "562788040190001",
"variantDescription": "YYYYYYYYY, XXS",
"sizeScaleAndCode": "176-001",
"netWeight": 0.491,
"unitsOfMeasure": {
"unitsOfMeasureType": [
{
"alternativeUOM_ISO": "PCE",
"length": 320,
"width": 290,
"height": 31,
"unitOfDimension": "MM",
"volume": 2876.8,
"volumeUnit": "CCM",
"weightUnit": "KG"
}
]
},
"globalTradeItemNumber": {
"globalTradeItemNumberType": [
{
"GTIN": "1111111111111",
"GTINCategory": "Z3"
},
{
"GTIN": "2222222222222",
"GTINCategory": "Z3"
},
{
"GTIN": "3333333333333",
"GTINCategory": "IE"
}
]
}
}
]
}
}
}
}
I tried the following query based on suggested answer below but it did not work
SELECT *
FROM c
WHERE ARRAY_CONTAINS(c.OriginalData.base.sales.variants.globalTradeItemNumber.globalTradeItemNumberType, {GTIN:"1111111111111"}, true)
I guess the above fails because variants part of the tree is also an array?
NOTE: the variants array can hold several objects so its not always index[0]
You could try using the ARRAY_CONTAINS function.
SELECT *
FROM c IN jongel.OriginalData.base.sales.variants
WHERE ARRAY_CONTAINS(c.globalTradeItemNumber.globalTradeItemNumberType, {GTIN:"1111111111111"}, true)
This will allow the query to search all items in the array for a matching GTIN value.
https://learn.microsoft.com/en-us/azure/cosmos-db/sql-query-array-contains
To contextualize:
A Batch file has BachItems
Each BatchItem is a row and has a line number.
Each row is processed in order.
I'm new to NoSQL and mongo and I'd like to know how to query the last processing step executed (most recent EventType column) for each BatchItem (line number), filtering by BatchId?
For example, it should return the following result for BatchId "102030":
I believe I can achieve this using Aggregate and Group functions but don't know how.
Thanks.
You can do it as below:
db.batch.aggregate([
{
$match: {
"BatchId": 102030
}
},
{ $sort: { "Date": -1 } },
{
$group: {
_id: "$BatchItemId",
"doc": { "$push": { lastEventName: "$EventType" } },
}
},
{
$replaceRoot: {
newRoot: { $arrayElemAt: ["$doc", 0] }
}}
])
I'm trying to rollup some of my 'other' results using Elasticsearch. Ideally, I'd like my query to return the top N hits and then roll the rest of the data up into an N+1 hit titled "Other".
So for example, if I'm trying to aggregate "Institutions by Total Value", I'd get back 10 Institutions with the most value and then the total aggregated value of the other institutions as another record. The purpose is that I'd like to see the total value aggregated across all institutions but not have to list thousands.
An example search I've been using is:
GET my_index/institution/_search?pretty=true
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
... terms queries ...
]
}
}
}
},
"aggs": {
"dimension_type_name_agg": {
"terms": {
"field": "institution_name",
"order": {
"metric_sum_total_value_agg": "desc"
},
"size": 0
},
"aggs": {
"metric_sum_total_value_agg": {
"sum": {
"field": "total_value"
}
},
"metric_count_account_id_agg": {
"value_count": {
"field": "institution_id"
}
}
}
}
}
}
I'm curious as to if this can be done by modifying a query like the one given above. Also, I'm using C# and Nest/Elasticsearch.NET so any tips on how this translates to that side is appreciated as well.