Develop Reference

Mongo error when trying to update nested arrays: No array filter found for identifier

I am trying to update a document in Mongo that represents a community with the following scenario.
A community has a collection of blocks
A block has a collection of floors
A floor has a collection of doors
A door has a collection of label names
Given a document Id and information about the labels that must be placed into each door, I want to use the MongoDb C# driver v2.10.4 and mongo:latest to update nested lists (several levels).
I've reading the documentation, about array filters, but I can't have it working.
I've created a repository from scratch to reproduce the problem, with instructions on the Readme on how to run the integration test and a local MongoDB with docker.
But as a summary, my method groupds the labels so that I can bulk place names on the desired door and then it iterates over these groups and updates on Mongo the specific document setting the desired value inside some levels deep nested object. I couldn't think of a more efficient way.
All the code in the above repo.
The DB document:
public class Community
{
public Guid Id { get; set; }
public IEnumerable<Block> Blocks { get; set; } = Enumerable.Empty<Block>();
}
public class Block
{
public string Name { get; set; } = string.Empty;
public IEnumerable<Floor> Floors { get; set; } = Enumerable.Empty<Floor>();
}
public class Floor
{
public string Name { get; set; } = string.Empty;
public IEnumerable<Door> Doors { get; set; } = Enumerable.Empty<Door>();
}
public class Door
{
public string Name { get; set; } = string.Empty;
public IEnumerable<string> LabelNames = Enumerable.Empty<string>();
}
The problematic method with array filters:
public async Task UpdateDoorNames(Guid id, IEnumerable<Label> labels)
{
var labelsGroupedByHouse =
labels
.ToList()
.GroupBy(x => new { x.BlockId, x.FloorId, x.DoorId })
.ToList();
var filter =
Builders<Community>
.Filter
.Where(x => x.Id == id);
foreach (var house in labelsGroupedByHouse)
{
var houseBlockName = house.Key.BlockId;
var houseFloorName = house.Key.FloorId;
var houseDoorName = house.Key.DoorId;
var names = house.Select(x => x.Name).ToList();
var update =
Builders<Community>
.Update
.Set($"Blocks.$[{houseBlockName}].Floors.$[{houseFloorName}].Doors.$[{houseDoorName}].LabelNames", names);
await _communities.UpdateOneAsync(filter, update);
}
}
The exception is
MongoDB.Driver.MongoWriteException with the message "A write operation resulted in an error.
No array filter found for identifier 'Block 1' in path 'Blocks.$[Block 1].Floors.$[Ground Floor].Doors.$[A].LabelNames'"
Here's a more visual sample on how the nested structure looks like in the database. Notice the value I want to update is the LabelNames, which is an array of string.
I appreciate any help to have this working and suggestions on whether it's the right approach assuming that I cannot change the repository's method signature.
SOLUTION RESULT:
Thanks for the quick answer #mickl, it works perfectly.
Result at this repo's specific point of history exactly as suggested.

The $[{houseBlockName}] expects an identifier which acts as a placeholder and has a corresponding filter defined within arrayfilters (positional filtered). It seems like you're trying to pass the filter value directly which is incorrect.
Your C# code can look like this:
var houseBlockName = house.Key.BlockId;
var houseFloorName = house.Key.FloorId;
var houseDoorName = house.Key.DoorId;
var names = house.Select(x => x.Name).ToList();
var update = Builders<Community>.Update.Set("Blocks.$[block].Floors.$[floor].Doors.$[door].LabelNames", names);
var arrayFilters = new List<ArrayFilterDefinition>();
ArrayFilterDefinition<BsonDocument> blockFilter = new BsonDocument("block.Name", new BsonDocument("$eq", houseBlockName));
ArrayFilterDefinition<BsonDocument> floorFilter = new BsonDocument("floor.Name", new BsonDocument("$eq", houseFloorName));
ArrayFilterDefinition<BsonDocument> doorFilter = new BsonDocument("door.Name", new BsonDocument("$eq", houseDoorName));
arrayFilters.Add(blockFilter);
arrayFilters.Add(floorFilter);
arrayFilters.Add(doorFilter);
var updateOptions = new UpdateOptions { ArrayFilters = arrayFilters };
var result = _communities.UpdateOne(filter, update, updateOptions);

{
var filterCompany = Builders<CompanyInfo>.Filter.Eq(x => x.Id, Timekeepping.CompanyID);
var update = Builders<CompanyInfo>.Update.Set("LstPersonnel.$[i].Timekeeping.$[j].CheckOutDate", DateTime.UtcNow);
var arrayFilters = new List<ArrayFilterDefinition>
{
new BsonDocumentArrayFilterDefinition<BsonDocument>(new BsonDocument("i.MacAddress",new BsonDocument("$eq", Timekeepping.MacAddress) )),
new BsonDocumentArrayFilterDefinition<BsonDocument>(new BsonDocument("j.Id", new BsonDocument("$eq", timeKeeping.Id)))
};
var updateOptions = new UpdateOptions { ArrayFilters = arrayFilters};
var updateResult = await _companys.UpdateOneAsync(filterCompany, update, updateOptions);
return updateResult.ModifiedCount != 0;
}

In Mongo DB, how can I update a nested property of a property that can be null

I have a MongoDB document with a settings field, itself with two nested fields, selectedSectionIds and sectionColors. settings is not an array. I need to update only the selectedSectionIds field.
My update builder looks like this:
Builders<Account>.Update.Set(
"settings.selectedSectionIds",
sectionIds)
And I'm calling UpdateOneAsync with nothing more special.
When settings is not present in the original document or already contains something, all works well, but when settings is null (and it can), I get the following MongoWriteException:
A write operation resulted in an error.
cannot use the part (settings of settings.selectedSectionIds) to traverse the element ({settings: null})
How can I update my builder (or class maps/serializers?) to support all scenarios?
(MongoDB C# drivers 2.8)

you cannot update a property of a null. if it was an empty object {} it would work.
so my suggestion is to do a bulk update command with 2 steps. where in the first step you check for the null and change it, and in the second step you set the sub property value as needed.
here's an example using MongoDB.Entities for brevity:
using MongoDB.Entities;
namespace StackOverflow
{
public class Program
{
public class Account : Entity
{
public string Name { get; set; }
public Settings Settings { get; set; }
}
public class Settings
{
public string[] SelectedSectionIDs { get; set; }
public string[] SectionColors { get; set; }
}
private static void Main(string[] args)
{
new DB("test", "127.0.0.1");
var acc1 = new Account
{
Name = "Account One",
Settings = new Settings
{
SectionColors = new[] { "green", "red" },
SelectedSectionIDs = new[] { "xxx", "yyy" }
}
}; acc1.Save();
var acc2 = new Account
{
Name = "Account Two",
Settings = null
}; acc2.Save();
DB.Update<Account>()
.Match(a => a.Settings == null)
.Modify(a => a.Settings, new Settings())
.AddToQueue()
.Match(_ => true)
.Modify(a => a.Settings.SelectedSectionIDs, new[] { "aaa", "bbb" })
.AddToQueue()
.Execute();
}
}
}

Add custom column to IDataView in ML.NET

I'd like to add a custom column after loading my IDataView from file.
In each row, the column value should be the sum of previous 2 values. A sort of Fibonacci series.
I was wondering to create a custom transformer but I wasn't able to find something that could help me to understand how to proceed.
I also tried to clone ML.Net Git repository in order to see how other transformers were implemented but I saw many classes are marked as internal so I cannot re-use them in my project.

There is a way to create a custom transform with CustomMapping
Here's an example I used for this answer.
The input and output classes:
class InputData
{
public int Age { get; set; }
}
class CustomMappingOutput
{
public string AgeName { get; set; }
}
class TransformedData
{
public int Age { get; set; }
public string AgeName { get; set; }
}
Then, in the ML.NET program:
MLContext mlContext = new MLContext();
var samples = new List<InputData>
{
new InputData { Age = 16 },
new InputData { Age = 35 },
new InputData { Age = 60 },
new InputData { Age = 28 },
};
var data = mlContext.Data.LoadFromEnumerable(samples);
Action<InputData, CustomMappingOutput> mapping =
(input, output) =>
{
if (input.Age < 18)
{
output.AgeName = "Child";
}
else if (input.Age < 55)
{
output.AgeName = "Man";
}
else
{
output.AgeName = "Grandpa";
}
};
var pipeline = mlContext.Transforms.CustomMapping(mapping, contractName: null);
var transformer = pipeline.Fit(data);
var transformedData = transformer.Transform(data);
var dataEnumerable = mlContext.Data.CreateEnumerable<TransformedData>(transformedData, reuseRowObject: true);
foreach (var row in dataEnumerable)
{
Console.WriteLine($"{row.Age}\t {row.AgeName}");
}

Easy thing. I am assuming, you know how to use pipelines.
This is a part of my project, where I merge two columns together:
IEstimator<ITransformer> pipeline = mlContext.Transforms.CustomMapping(mapping, contractName: null)
.Append(mlContext.Transforms.Text.FeaturizeText(inputColumnName: "question1", outputColumnName: "question1Featurized"))
.Append(mlContext.Transforms.Text.FeaturizeText(inputColumnName: "question2", outputColumnName: "question2Featurized"))
.Append(mlContext.Transforms.Concatenate("Features", "question1Featurized", "question2Featurized"))
//.Append(mlContext.Transforms.NormalizeMinMax("Features"))
//.AppendCacheCheckpoint(mlContext)
.Append(mlContext.BinaryClassification.Trainers.SdcaLogisticRegression(labelColumnName: nameof(customTransform.Label), featureColumnName: "Features"));
As you can see the two columns question1Featurized and question2Featurized are combined into Features which will be created and can be used as any other column of IDataView. The Features column does not need to be declared in a separate class.
So in your case you should transform the columns firs in their data type, if strings you can do what I did and in case of numeric values use a custom Transformer/customMapping.
The documentation of the Concatenate function might help as well!

How to cast from a list with objects into another list.

I have an Interface [BindControls] which takes data from GUI and store it into a list „ieis”.
After that, Into another class, which sends this data through WebServices, I want to take this data from „ieis” and put it into required by WS Class fields (bottom is a snippet of code)
This is the interface:
void BindControls(ValidationFrameBindModel<A.B> model)
{
model.Bind(this.mtbxTax, (obj, value) =>
{
var taxa = TConvertor.Convert<double>((string)value, -1);
if (taxa > 0)
{
var ieis = new List<X>();
var iei = new X
{
service = new ServiceInfo
{
id = Constants.SERVICE_TAX
},
amount = tax,
currency = new CurrencyInfo
{
id = Constants.DEFAULT_CURRENCY_ID
}
};
ieis.Add(iei);
}
},"Tax");
}
This is the intermediate property:
//**********
class A
{
public B BasicInfo
{
get;
set;
}
class B
{
public X Tax
{
get;
set;
}
}
}
//***********
This is the class which sends through WS:
void WebServiceExecute(SomeType someParam)
{
//into ‚iai’ i store the data which comes from interface
var iai = base.Params.FetchOrDefault<A>( INFO, null);
var convertedObj = new IWEI();
//...
var lx = new List<X>();
//1st WAY: I tried to put all data from ‚Tax’into my local list ‚lx’
//lx.Add(iai.BasicInfo.Tax); - this way is not working
//2nd WAY: I tried to put data separately into ‚lx’
var iei = new X
{
service = new ServiceInfo
{
id = iai.BasicInfo.Tax.service.id
},
amount = iai.BasicInfo.Tax.amount,
currency = new CurrencyInfo
{
id = iai.BasicInfo.Tax.currency.id
}
};
lx.Add(iei);
// but also is not working
Can you help me please to suggest how to implement a way that will fine do the work (take data from ‚ieis’ and put her into ‚lx’).
Thank you so much

As noted in my comment, it looks like iai.BasicInfo.Tax is null, once you find out why that is null your original Add() (#1) will work.

Dictionary of Objects doesn't work as JSON

I have spent WAY too much time trying to figure out how to pull all the values I need to from my C# application using JS and JSON. It works fine when I just use simple structures, such as an array, but I need to be able to grow the list at runtime.
Right now, the best I could figure out was doing a Dictionary with an incrementing key value, and the other 3 values as a class object. However, this seems to crash out my C# application.
What would be the best way to do this?
Relevant C# Code:
public class ChatData
{
string userName;
string message;
System.DateTime timestamp;
public ChatData(string name, string msg)
{
userName = name;
message = msg;
timestamp = System.DateTime.Now;
}
}
else if (string.Equals(request, "getchat"))
{
//string since = Request.Query.since;
Dictionary<int, ChatData> data = new Dictionary<int, ChatData>();
data.Add(1, new ChatData("bob", "hey guys"));
data.Add(2, new ChatData("david", "hey you"));
data.Add(3, new ChatData("jill", "wait what"));
return Response.AsJson(data);
}
Relevant Javascript:
function getChatData()
{
$.getJSON(dataSource + "?req=getchat", "", function (data)
{
//$.each(data, function(key, val)
//{
//addChatEntry(key, val);
//})
});
}

You haven't explained what Response.AsJson is and how it is implemented but if it uses JavaScriptSerializer you will get the following exception:
Unhandled Exception: System.ArgumentException: Type
'System.Collections.Generic. Dictionary`2[[System.Int32, mscorlib,
Version=4.0.0.0, Culture=neutral,
PublicKeyToken=b77a5c561934e089],[ChatData, Test, Version=1.0.0.0,
Culture=neutral, PublicKeyToken=null]]' is not supported for
serialization/deserialization of a dictionary, keys must be strings or
objects.
which is pretty self-explanatory. You cannot use integers as keys if you intend to JSON serialize this structure. Also because your ChatData class no longer has a default/parameterless constructor you won't be able to deserialize a JSON string back to this class (but I guess you don't need this yet).
So one possible solution to your problem would be to use:
Dictionary<string, ChatData> data = new Dictionary<string, ChatData>();
data.Add("1", new ChatData("bob", "hey guys"));
data.Add("2", new ChatData("david", "hey you"));
data.Add("3", new ChatData("jill", "wait what"));
Now of course this being said and looking at the javascript you commented out and what you intend to do, as I already explained you in your previous question, dictionaries are not serialized as javascript arrays, so you cannot loop over them.
Long story short, define a class:
public class ChatData
{
public string Username { get; set; }
public string Message { get; set; }
public DateTime TimeStamp { get; set; }
}
and then fill an array of this class:
var data = new[]
{
new ChatData { Username = "bob", Message = "hey guys" },
new ChatData { Username = "david", Message = "hey you" },
new ChatData { Username = "jill", Message = "wait what" },
};
return Response.AsJson(data);
and finally consume:
$.getJSON(dataSource, { req: 'getchat' }, function (data) {
$.each(data, function(index, element) {
// do something with element.Username and element.Message here, like
$('body').append(
$('<div/>', {
html: 'user: ' + element.Username + ', message:' + element.Message
})
);
});
});

Why not simply use a typed list? Also, you'll need a default constructor to serialize/deserialize it. Note how I've modified your class to use properties
as well. Note, as #rudolf_franek mentions, you can add an ID property to the ChatData class if you need to be able to link to it.
public class ChatData
{
public ChatData()
{
TimeStamp = DateTime.Now;
}
public int ID { get; set; }
public string Who { get; set; }
public string Said { get; set; }
public DateTime TimeStamp { get; set; }
}
...
var data = new List<ChatData>
{
new ChatData { ID = 1, Who = "bob", Said = "hey guys" },
...
};
return Response.AsJson( data );