We are looking to switch from a relational database to elastic search and I am trying to get some basic code up and running with Nest. We have existing objects which use guids for ids that I would like to save into an elastic search index.
I don't want to add any specific attributes as the class is used in different applications and I don't want to add unnecessary dependencies to Nest.
Right now my code looks like this:
var node = new Uri("http://localhost:9200");
var settings = new ConnectionSettings(node)
settings.DefaultIndex = "test";
var client = new ElasticClient(settings);
var testItem = new TestType { Id = Guid.NewGuid(), Name = "Test", Value = "10" };
var response = client.Index(testItem);
With TestType as:
public class TestType
{
public Guid Id { get; set; }
public string Name { get; set; }
public decimal Value { get; set; }
}
However I get an error like:
ServerError: 400Type: mapper_parsing_exception Reason: "failed to
parse [id]" CausedBy: "Type: number_format_exception Reason: "For
input string: "c9c0ed42-86cd-4a94-bc86-a6112f4c9188""
I think I need to specify a mapping that tells the server the Id is a string, but I can't find any examples or documentation on how I do this without using the attributes.
Assuming you're using Elasticsearch 2.x and NEST 2.x (e.g. latest of both at time of writing is Elasticsearch 2.3.5 and NEST 2.4.3), then NEST will automatically infer the id of a POCO by default from the Id property. In the case of a GUID id, this will be saved as a string in Elasticsearch.
Here's an example to get you going
void Main()
{
var node = new Uri("http://localhost:9200");
var settings = new ConnectionSettings(node)
// default index to use if one is not specified on the request
// or is not set up to be inferred from the POCO type
.DefaultIndex("tests");
var client = new ElasticClient(settings);
// create the index, and explicitly provide a mapping for TestType
client.CreateIndex("tests", c => c
.Mappings(m => m
.Map<TestType>(t => t
.AutoMap()
.Properties(p => p
// don't analyze ids when indexing,
// so they are indexed verbatim
.String(s => s
.Name(n => n.Id)
.NotAnalyzed()
)
)
)
)
);
var testItem = new TestType { Id = Guid.NewGuid(), Name = "Test", Value = "10" };
// now index our TestType instance
var response = client.Index(testItem);
}
public class TestType
{
public Guid Id { get; set; }
public string Name { get; set; }
public decimal Value { get; set; }
}
Take a look at the Automapping documentation for more examples of how to explicitly map a POCO for controlling norms, analyzers, multi_fields, etc.
What I normally do is to have a separate class that is only specific to Elasticsearch. And use Automapper to map that into a DTO or ViewModel, or Model into the Elasticsearch Document.
That way, you won't have to expose an object that have a dependency in NEST and attributes that might be specific only to Elasticsearch.
Another good reason is that normally, documents in ES are flat, so you would normally flatten your objects before you index them to ES.
Related
I am using .net5 and EntityFrameworkCore 5.
I have a many to many relationship between Questions and Categories.
I am using Code first generation.
public class Question
{
public int Id { get; set; }
public string Title { get; set; }
public ICollection<Category> Categories { get; set; }
}
public class Category
{
public int Id { get; set; }
public string Name { get; set; }
public ICollection<Question> Questions { get; set; }
}
I want to know how to add a Question with Categories.
I tried this :
[HttpPost]
public async Task<ActionResult<Question>> PostQuestion(Question question)
{
question.Categories.Add(new Category() { Id = 1 });
_context.Questions.Add(question);
await _context.SaveChangesAsync();
return CreatedAtAction("GetQuestion", new { id = question.Id }, question);
}
And I have a Category with the Id : 1 in my database.
However I get this exception
SqlException: Cannot insert explicit value for identity column in table 'Categories' when IDENTITY_INSERT is set to OFF.
What is the correct way to do many to many entity relation insert ?
The truly correct and intended way is to load the related entities into context before adding them to the "links" collection.
Let say you have a list of existing related entity keys:
var categoryIds = new[] { 1, 3, 4 };
Then you can use the Find method to load the corresponding entities into context and obtain their instances:
question.Categories = categoryIds
.Select(id => _context.Categories.Find(id))
.ToList();
The drawback is that it makes N database roundtrips to load the data you might not really need.
It can be made with just one additional database roundtrip by issuing Contains based query instead:
question.Categories = await _context.Categories
.Where(e => categoryIds.Contains(e.Id))
.ToListAsync();
If you really don't want the related entities, following are some other ways.
If the context lifetime is scoped to just that call, then you can use fake (stub) entities as with your attempt, but you have to Attach them to the context to let EF Core treat them as existing rather than as new if you don't do that:
question.Categories = categoryIds
.Select(id => _context.Attach(new Category { Id = id }))
.ToList();
Another way is to insert directly entries in the shadow join dictionary type entity set. But it requires knowing the conventional names of the join entity type and its shadow FKs, so this is type unsafe.
Also you need to first to Add the entity in order to have its temporary key available:
var entry = _context.Questions.Add(question);
Then for the shown model you have
var joinEntityName = "CategoryQuestion";
var fromPK = nameof(Question.Id);
var fromFK = "QuestionsId";
var toFK = "CategoriesId";
Actually these can be obtained from the EF Core metadata, which would make it safer:
var navInfo = entry.Collection(e => e.Courses).Metadata as Microsoft.EntityFrameworkCore.Metadata.ISkipNavigation;
var joinEntityName = navInfo.JoinEntityType.Name;
var fromPK = navInfo.ForeignKey.PrincipalKey.Properties.Single().Name;
var fromFK = navInfo.ForeignKey.Properties.Single().Name;
var toFK = navInfo.Inverse.ForeignKey.Properties.Single().Name;
Then the insert code is:
var fromId = entry.CurrentValues[fromPK]; // the temp PK
db.Set<Dictionary<string, object>>(joinEntityName).AddRange(
categoryIds.Select(toId => new Dictionary<string, object>
{
{ fromFK, fromId },
{ toFK, toId },
}));
I have a class inheriting from another class
I am doing a query from the database
How do I fill in the static List without loop using linq lambda
If he finds a lot of data. this will not be fast
I want to escape from loop
public class Currencys
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public long Curr_Id { get; set; }
[StringLength(50)]
public string Curr_Name { get; set; }
[StringLength(50)]
public string CentName { get; set; }
[StringLength(50)]
public string curr_abbrivation { get; set; }
[StringLength(50)]
public string en_curr_name { get; set; }
[StringLength(50)]
public string en_centname { get; set; }
}
public class test1 : Currencys
{
static List<test1> _currenciesList;
public static void Fill()
{
if (_currenciesList != null)
{
_currenciesList.Clear();
}
_currenciesList = new List<test1>();
using (var context = new ContextFormeDb())
{
var list = context.Currencies.ToList();
list.ForEach(o=>
{
test1 _test1 = new test1();
_test1.Curr_Id = o.Curr_Id;
_test1.Curr_Name = o.Curr_Name;
_test1.CentName = o.CentName;
_test1.curr_abbrivation = o.curr_abbrivation;
_test1.en_curr_name = o.en_curr_name;
_test1.en_centname = o.en_centname;
_currenciesList.Add(_test1);
});
}
}
}
Is there anything better than this? without loop
list.ForEach(o=>
{
test1 _test1 = new test1();
_test1.Curr_Id = o.Curr_Id;
_test1.Curr_Name = o.Curr_Name;
_test1.CentName = o.CentName;
_test1.curr_abbrivation = o.curr_abbrivation;
_test1.en_curr_name = o.en_curr_name;
_test1.en_centname = o.en_centname;
_currenciesList.Add(_test1);
});
Is there anything better than this? without loop?
It depends on what you would call better. Faster? Probably not. Not much anyway. Easier to read and understand, easier to test, to debug, to change, to reuse? Probably.
Without Loop? there must be a loop somewhere, but it can be hidden inside a LINQ statement.
Whenever you want to fetch items from a database using entity framework, and you don't want to update the fetched items, always use Select, and select only the properties that you plan to use. Don't fetch the complete items, nor use Include. This will cost you overhead that you will only use if you update the fetched data.
So instead of:
var result = dbContext.Schools
.Where(school => school.Name == "Hogwarts")
.Include(school => school.Students)
.ToList();
consider to use:
var result = dbContext.Schools
.Where(school => school.Name == "Hogwarts")
.Select(school => new
{
// Select only the properties that you plan to use
Id = school.Id,
Name = school.Name,
...
students = dbContext.Students
.Where(student => student.SchoolId == school.Id)
.Select(student => new
{
Id = student.Id,
Name = student.Name,
...
// not needed, you know the value
// SchoolId = student.SchoolId,
})
.ToList(),
})
.ToList();
It will prevent the transfer of properties that you won't use
It will prevent that the fetched data will be copied to DbContext.ChangeTracker.
If you don't put data that won't be changed in the ChangeTracker, then SaveChanges will be faster.
So in your case, your code would be easier to understand, easier to reuse, easier to test and debug, and without "for each" if you use Select:
var fetchedData = dbContext.Currencies
.Where(currency => ...) // if you don't want all currencies
.Select(currency => new
{
// Select only the properties that you plan to use:
Id = currency.Id,
Name = currency.Name,
...
})
.ToList();
I used an anonymous type (new without specifying a class). This way you won't have to create a "dummy" class. The advantage is that you just write the properties and you'll have the object, you even have an "equality by value". If in future you need to add or remove a property, just do it, without any problem, no need to change your dummy class.
Disadvantage: you can't use it outside the current block, and certainly not as a return value of a procedure.
So if you need it outside your procedure:
.Select(currency => new Test1
{
// Select only the properties that you plan to use:
Id = currency.Id,
Name = currency.Name,
...
})
.ToList(),
If two lists are of the same type , you can use AddRange.
if not and for any reason you need to map properties or its diffrent object type, i would suggest configure AutoMapper in your app and like this you can easily convert you List from Type A to Type B and after that use AddRange
I have a DynamoDB table with GSI and I need to use AWS .NET SDK to retrieve two specific attributes, preferably using the persistence model. I've created a separate class for this projection like this
[DynamoDBTable("MyTable")]
public class MyItem
{
public long Number{ get; set; }
public string Name{ get; set; }
}
but even when I specify SelectValues.SpecificAttributes to get only these attributes, it seems that DynamoDB client tries to retrieve primary key attributes (which I would like to avoid) and gives me an exception Unable to locate property for key attribute <PK hash key attribute name>. Here is the code I use to query
var keyExpression = new Expression
{
ExpressionStatement = $"GSI_PK = :pkValue",
ExpressionAttributeValues =
{
[":pkValue"] = 1
}
};
db.FromQueryAsync<MyItem>(new QueryOperationConfig
{
IndexName = "MyIndex",
KeyExpression = keyExpression,
Select = SelectValues.SpecificAttributes,
AttributesToGet = new List<string> { "Number", "Name" }
}
Is there a way to stop DynamoDB client from mapping non-requested items?
I am creating a CRUD Web API and I am just following this Microsoft tutorial. In my case, I have two models:
First_Model.cs:
// i removed other unnecessary data
public string Id { get; set; }
public IList<Second_Model> Second_Models {get; set;} = new List<Second_Model>();
Second_Model.cs:
// i removed other unnecessary data
public string Id { get; set; }
Just like in the tutorial, it will statically add a default value of the collection, so I want to add List<Second_Model> into Second_Models
Here's what I tried in my First_ModelController.cs
_context.First_Models.Add(
new First_Model {
Id = "default_id",
Second_Models = new List<Second_Model>() {
new List<Second_Model>().Find(p => p.Id == "default_id")
// given that Second_Model also has default_id
};
}
);
However, this set of codes will return this error:
ArgumentNullException: Value cannot be null. Parameter name: key
Statically creating a list of objects would look more like this:
....
Id = "default_id",
Second_Models = new List<Second_Model>() {
new Second_Model() { Id = "<your second model ID 1>", OtherProperty = <value> },
new Second_Model() { Id = "<your second model ID 2>", OtherProperty = <value> },
}
However, if you are want to add them directly to the context, you would do something similar to:
_context.Second_Models.Add(
new Second_Model() {
Id = "<your second model ID 1>",
OtherProperty = <value>,
}
);
... repeat ...
That does not associate the second_model with the first_model though.
You might need to read more about defining foreign keys on EF models so make this all join together, because I don't think the tutorial includes anything related to that.
I got an .net core application that is pretty straight forward it is using REST to add and download objects to and from mongo db. Adding items works really well. Getting a list that contains all items aswell, but when I try to access one using id then everytime I get null. What should i change to make this piece of code work. It means get a Tool object from database using it unique ID when there's one matching in database.
Here's a object in database
Here's my repository class
private IMongoCollection<Tool> Tools => _database.GetCollection<Tool>("Tools");
public async Task<Tool> GetAsync(Guid id) =>
await Tools.AsQueryable().FirstOrDefaultAsync(tool => tool.Id == id);
Argument looks like that when I check it out in debugger "{ee1aa9fa-5d17-464c-a8ba-f685203b911f}"
Edit
Tool Class Properties
public Guid Id { get; protected set; }
public string Model { get; protected set; }
public string Brand { get; protected set; }
public string Type { get; protected set; }
public uint Box { get; protected set; }
Fixed check comments
Project on github
The easiest way to do this in C# MongoDB Driver is to set a global GuidRepresentation setting which can be found on the BsonDefaults object. This is a global setting and will effect all serialization/deserialization of GUIDs in to Bson Binary Objects.
BsonDefaults.GuidRepresentation = GuidRepresentation.PythonLegacy;
var collection = new MongoClient().GetDatabase("test").GetCollection<ClassA>("test");
var item = collection.Find(x => x.MyId == new Guid("ee1aa9fa-5d17-464c-a8ba-f685203b911f"))
.FirstOrDefault();
The second option is to convert the GUID manually from a LUUID to a CSUUID, for this there is a helper class within the MongoDB driver of GuidConverter, with this it converts the GUID in to byte[] which is normally used for storage but we can use it for our query.
BsonDefaults.GuidRepresentation = GuidRepresentation.CSharpLegacy;
var collection = new MongoClient().GetDatabase("test").GetCollection<ClassA>("test");
var luuid = new Guid("0798f048-d8bb-7048-bb92-7518ea4272cb");
var bytes = GuidConverter.ToBytes(luuid, GuidRepresentation.PythonLegacy);
var csuuid = new Guid(bytes);
var item = collection.Find(x => x.MyId == csuuid)
.FirstOrDefault();
I also noticed that you're using Robo 3T (formerly Robomongo), within this application you can set how GUIDs are displayed by going to Options -> Legacy UUID Encodings