I'm indexing a type that has percolate query but Nest/elasticsearch choose to ignore the query property.
public class MyQueryModel
{
public string Id { get; set; }
public string UserId { get; set;}
public string Email { get ; set;}
public string Name { get; set; }
public string State { get; set; }
public QueryContainer PercolatedQuery { get; set; }
}
public class DocModel
{
public string Id { get; set; }
public string Title { get; set; }
public string State { get; set; }
public string Category { get; set;}
public string Email { get; set; }
}
EDIT: some of property names between the 2 are same by coincidence. They totally mean different things on either of the 2 models and maybe mapped differently.
my mappings:
on queries index:
client.CreateIndex("on_my_queries", c => c
.Mappings(m => m
.Map<MyQueryModel>(mq => mq
.AutoMap()
.Properties(props => props
.Percolator(perc => perc
.Name(m => m.PercolatedQuery)
)
)
)
)
)
on doc index
client.CreateIndex("on_my_docs", c => c
.Mappings(m => m
.Map<MyDocModel>(md => md
.AutoMap()
)
)
)
Indexing my query model:
var queryModel = new MyQueryModel
{
Id = "some-id",
UserId = "some-user-id",
Email = "some-valid-email",
State = "some-valid-state",
PercolatedQuery = new TermQuery
{
Field = "category",
Value = "some-valid-cat-on-my-doc-models"
}
}
var request = new IndexRequest<QueryModel>(DocumentPath<MyQueryModel>.Id(queryModel));
var result = client.Index(request);
Everything gets indexed except the PercolatedQuery field. After scratching a lot of my head, I find out that client is not even serializing it. I ran the following only to see that PercolatedQuery was not serialized:
var jsonString = client.Serializer.SerializeToString(request);
jsonString:
{
"id" : "some-id",
"userId" : "some-user-id",
"email: : "some-valid-email",
"state" : "some-valid-state"
}
What client see as percolated query:
var queryString = client.Serializer.SerializeToString(queryModel.PercolatedQuery);
queryString:
{
"term": {
"category": {
"value": "some-valid-cat-on-my-doc-models"
}
}
}
Related
This is what my classes look like:
public class TestA : MongoDocument
{
public string ProjectID { get; set; }
public TestB Content { get; set; }
}
public class TestB
{
public string Id { get; set; }
public List<TestC> CustInfo { get; set; }
}
public class TestC
{
public string Id { get; set; }
public CustomerComment CustomerComment { get; set; }
}
public class CustomerComment
{
public string Id { get; set; }
public string notes { get; set; }
}
I would like to return only the CustomerComment based on TestA.ProjectID && TestC.Id
var projection = Builders<TestA>.Projection
.Include(x => x.Content.CustInfo);
var options = new FindOptions<TestA>
{
Projection = projection
};
var find = await Collection.FindAsync(p => p.ProjectID == "555" &&
p.Content.CustInfo.Any(l => l.Id == "123"), options).ConfigureAwait(false);
var result = await find.ToListAsync().ConfigureAwait(false);
This works but it will return everything. In this case, I would only want to return CustomerComments.
So for me to only retrieve the customerComments, I do another query based on the results from mongo. The code below gives me the correct data but I would rather do the filtering through the database.
var test = result.SelectMany(x => x.Content.CustInfo.Where(l => l.Id == "123").Select(y => y.CustomerComment)).ToList();
I think Aggregation Query meets your requirements for querying the data in the database and returning desired output.
$match - Filtering data.
$unwind - Deconstruct array fields to multiple documents.
$match - Filtering data for Content.CustInfo.
$replaceWith - Replace the current document with the new document for output.
db.collection.aggregate([
{
$match: {
"ProjectID": "555",
"Content.CustInfo.Id": "123"
}
},
{
$unwind: "$Content.CustInfo"
},
{
$match: {
"Content.CustInfo.Id": {
$eq: "123"
}
}
},
{
"$replaceWith": "$Content.CustInfo.CustomerComment"
}
])
Sample Mongo Playground
Solution 1: With AggregateFluent
Pre-requisite: Create unwind classes.
public class UnwindTestA
{
public ObjectId _id { get; set; }
public string ProjectID { get; set; }
public UnwindTestB Content { get; set; }
}
public class UnwindTestB
{
public string Id { get; set; }
public TestC CustInfo { get; set; }
}
var result = await Collection.Aggregate()
.Match(
p => p.ProjectID == "555"
&& p.Content.CustInfo.Any(l => l.Id == "123")
)
.Unwind<TestA, UnwindTestA>(x => x.Content.CustInfo)
.Match<UnwindTestA>(x => x.Content.CustInfo.Id == "123")
.ReplaceWith<CustomerComment>("$Content.CustInfo.CustomerComment")
.ToListAsync();
Solution 2: With BsonDocument
Sometimes, writing a query with AggregateFluent is quite complex. You can use the MongoDB query which is converted to BsonDocument.
BsonDocument[] aggregate = new BsonDocument[]
{
new BsonDocument("$match",
new BsonDocument
{
{ "ProjectID", "555" },
{ "Content.CustInfo.Id", "123" }
}),
new BsonDocument("$unwind", "$Content.CustInfo"),
new BsonDocument("$match",
new BsonDocument("Content.CustInfo.Id",
new BsonDocument("$eq", "123"))),
new BsonDocument("$replaceWith", "$Content.CustInfo.CustomerComment")
};
var result = await Collection.Aggregate<CustomerComment>(aggregate)
.ToListAsync();
Output
How can I achieve the projection on the last select? I need the property defined by the string prop.Name to be selected into the SeriesProjection object.
public override IQueryable<SeriesProjection> FilterOn(string column)
{
//Get metadata class property by defined Attributes and parameter column
var prop = typeof(CommunicationMetaData)
.GetProperties()
.Single(p => p.GetCustomAttribute<FilterableAttribute>().ReferenceProperty == column);
var attr = ((FilterableAttribute)prop.GetCustomAttribute(typeof(FilterableAttribute)));
var param = Expression.Parameter(typeof(Communication));
Expression conversion = Expression.Convert(Expression.Property(param, attr.ReferenceProperty), typeof(int));
var condition = Expression.Lambda<Func<Communication, int>>(conversion, param); // for LINQ to SQl/Entities skip Compile() call
var result = DbQuery.Include(prop.Name)
//.GroupBy(c => c.GetType().GetProperty(attr.ReferenceProperty))
.GroupBy(condition)
.OrderByDescending(g => g.Count())
.Select(group => new SeriesProjection()
{
Count = group.Count(),
Id = group.Key,
//set this navigation property dynamically
Name = group.FirstOrDefault().GetType().GetProperty(prop.Name)
});
return result;
}
For the GroupBy I used the fk property name that's always an int on the Communication entity, but for the select I can't figure out the expression.
[EDIT]
System.Data.Entity.Infrastructure.DbQuery<Communication> DbQuery;
---
[MetadataType(typeof(CommunicationMetaData))]
public partial class Communication
{
public int CommunicationId { get; set; }
public Nullable<int> TopicId { get; set; }
public int CreateById { get; set; }
public virtual Employee CreateByEmployee { get; set; }
public virtual Topic Topic { get; set; }
}
---
public class CommunicationMetaData
{
[Filterable("By Employee", nameof(Communication.CreateById))]
public Employee CreateByEmployee { get; set; }
[Filterable("By Topic", nameof(Communication.TopicId))]
public Topic Topic { get; set; }
}
---
[AttributeUsage(AttributeTargets.Property)]
public class FilterableAttribute : System.Attribute
{
public FilterableAttribute(string friendlyName, string referenceProperty)
{
FriendlyName = friendlyName;
ReferenceProperty = referenceProperty;
}
public string FriendlyName { get; set; }
public string ReferenceProperty { get; set; }
}
---
public class SeriesProjection
{
public int Count { get; set; }
public int Id { get; set; }
public object Name { get; set; }
}
Without some expression helper library, you have to build the whole selector expression manually.
The input of the selector will be a parameter of type IGrouping<int, Communication>, the result type - SeriesProjection, and the body will be MemberInit expression:
var projectionParameter = Expression.Parameter(typeof(IGrouping<int, Communication>), "group");
var projectionType = typeof(SeriesProjection);
var projectionBody = Expression.MemberInit(
// new SeriesProjection
Expression.New(projectionType),
// {
// Count = group.Count(),
Expression.Bind(
projectionType.GetProperty(nameof(SeriesProjection.Count)),
Expression.Call(typeof(Enumerable), "Count", new[] { typeof(Communication) }, projectionParameter)),
// Id = group.Key
Expression.Bind(
projectionType.GetProperty(nameof(SeriesProjection.Id)),
Expression.Property(projectionParameter, "Key")),
// Name = group.FirstOrDefault().Property
Expression.Bind(
projectionType.GetProperty(nameof(SeriesProjection.Name)),
Expression.Property(
Expression.Call(typeof(Enumerable), "FirstOrDefault", new[] { typeof(Communication) }, projectionParameter),
prop.Name))
// }
);
var projectionSelector = Expression.Lambda<Func<IGrouping<int, Communication>, SeriesProjection>>(projectionBody, projectionParameter);
and then of course use simply:
var result = DbQuery.Include(prop.Name)
.GroupBy(condition)
.OrderByDescending(g => g.Count())
.Select(projectionSelector);
I'm currently experiencing a funny issue here, each time I insert a new item with EntityFramework, I get to see a new Item submitted first before the real time but with the same entries(properties) and a String[] Array in the last column. I actually have a model defined thus
public class SupervisionStageSetupModel
{
public int Id { get; set; }
[Required]
public string StageLevel { get; set; }
[Required]
public string Name { get; set; }
[Required]
public string Description { get; set; }
public IList<SupervisionStageSetupChecklistModel> SupervisionStageSetupChecklists { get; set; }
public string GetAuditDetails(string action)
{
return string.Format("{0} SupervisionStageSetup Setup: {1}", action,Name);
}
}
public class SupervisionStageSetupChecklistModel
{
public int Id { get; set; }
public int SupervisionStageSetupId { get; set; }
public string Description { get; set; }
public string[] Options { get; set; }
}
And in my service, I have a method that does the insert operation as thus
private void ProcessCheckListItems(List<SupervisionStageSetupChecklist> entities,
IList<SupervisionStageSetupChecklistModel> checkListItems, int stageSetupId)
{
var modelIds = checkListItems.Select(x => x.Id).ToList();
var supervisionStageSetupChecklistRepository = _repository.GetRepository<SupervisionStageSetupChecklist>();
var entitiesToDelete =
entities.Where(x => !modelIds.Contains(x.Id));
entitiesToDelete.ForEach(x => supervisionStageSetupChecklistRepository.Delete(x));
var entitiesToAdd = checkListItems.Where(x => x.Id == 0).Select(x =>
new SupervisionStageSetupChecklist()
{
Description = x.Description,
SupervisionStageSetupId = stageSetupId,
Options = String.Join(",",x.Options)
}).ToList();
entitiesToAdd.ForEach(x =>
{
supervisionStageSetupChecklistRepository.Insert(x);
}
);
var entityToUpdate =
entities.Where(x => modelIds.Contains(x.Id));
entityToUpdate.ForEach(source =>
{
var checklistItem = checkListItems.Single(x => x.Id == source.Id);
source.Description = checklistItem.Description;
source.Options = String.Join(",",checklistItem.Options);
supervisionStageSetupChecklistRepository.Update(source);
});
When I put my VS2013 in debug mode to see what's happening, I saw that it was just only the number of items that came from the view that was inserted, but going back to my MSSQL2014 , I found two records inserted as
Id| SupervisionStageId | Description | Options
2 | 3 | Description | String[] Array
3 | 3 | Description | Yes,No
And of course, after setting breakpoint while inserting, I hovered on the entitiesToAdd object which returned 1 as the count, and this not enough, retrieving the item with the following ...
public SupervisionStageSetupItem GetDetails(int id)
{
var entity = _repository.Find(id);
var item = Mapper.Map<SupervisionStageSetup, SupervisionStageSetupItem>(entity);
item.SupervisionStageSetupChecklists =
entity.SupervisionStageSetupChecklists.Where(x => x.SupervisionStageSetupId == entity.Id)
.Select(x => new SupervisionStageSetupChecklistModel()
{
Description = x.Description,Options = x.Options.Split(',')
}).ToList();
return item;
}
I got the one Options property value as "Y","e","s",",","N","o".
I'm perplexed!, Don't know what I'm doing wrong
If I have an entity:
class Post
{
[ Key ]
public int PostID { get; set; }
public string Title { get; set; }
public string Body { get; set; }
public string Comment { get; set; }
public virtual Project Project { get; set; }
...
public bool LikedByUser; // Append this property
}
public class PostLike
{
public int PostLikeID { get; set; }
public virtual Post Post { get; set; }
}
Is there a way I can do a join on a call to the database and just append this last property (mentioned in the code) as an extra... for example, this code is stupid (and probably doesn't work):
this.context.Posts
.Join(
this.context.PostLikes,
p => p.PostID,
pl => pl.Post.PostID,
// This is the bit that can't be right!
( p, pl ) => new Post()
{
PostID = p.PostID,
Title = p.Title,
Body = p.Body,
Comment = p.Comment
Project = p.Project,
LikedByUser = pl.Count > 0 ? true : false
}
);
I would like something like this:
this.context.Posts
.Join(
this.context.PostLikes,
p => p.PostID,
pl => pl.Post.PostID,
( p, pl ) => {
p.LikedByUser = pl.Count > 0 ? true : false;
}
);
So - one way you could do this, is via a sub-query:
var posts = this.context.Posts
.Select(p =>
new Post()
{
PostID = p.PostID,
Title = p.Title,
Body = p.Body,
Comment = p.Comment
Project = p.Project,
LikedByUser = this.context.PostLikes.Any(pl => pl.PostId = p.PostID)
});
If that property isn't part of the existing class - you could project into an anonymous type, for use:
var posts = this.context.Posts
.Select(p =>
new {
Post = p,
LikedByUser = this.context.PostLikes.Any(pl => pl.PostId = p.PostID)
});
I hope it's more clear what I want to do from the code than the title. Basically I am grouping by 2 fields and want to reduce the results into a collection all the ProductKey's constructed in the Map phase.
public class BlockResult
{
public Client.Names ClientName;
public string Block;
public IEnumerable<ProductKey> ProductKeys;
}
public Block()
{
Map = products =>
from product in products
where product.Details.Block != null
select new
{
product.ClientName,
product.Details.Block,
ProductKeys = new List<ProductKey>(new ProductKey[]{
new ProductKey{
Id = product.Id,
Url = product.Url
}
})
};
Reduce = results =>
from result in results
group result by new {result.ClientName, result.Block} into g
select new BlockResult
{
ClientName = g.Key.ClientName,
Block = g.Key.Block,
ProductKeys = g.SelectMany(x=> x.ProductKeys)
};
}
I get some weird System.InvalidOperationException and a source code dump where basically it is trying to initialize the list with an int (?).
If I try replacing the ProductKey with just IEnumerable ProductIds (and make appropriate changes in the code). Then the code runs but I don't get any results in the reduce.
You probably don't want to do this. Are you really going to need to query in this manner? If you know the context, then you should probably just do this:
var q = session.Query<Product>()
.Where(x => x.ClientName == "Joe" && x.Details.Block == "A");
But, to answer your original question, the following index will work:
public class Products_GroupedByClientNameAndBlock : AbstractIndexCreationTask<Product, Products_GroupedByClientNameAndBlock.Result>
{
public class Result
{
public string ClientName { get; set; }
public string Block { get; set; }
public IList<ProductKey> ProductKeys { get; set; }
}
public class ProductKey
{
public string Id { get; set; }
public string Url { get; set; }
}
public Products_GroupedByClientNameAndBlock()
{
Map = products =>
from product in products
where product.Details.Block != null
select new {
product.ClientName,
product.Details.Block,
ProductKeys = new[] { new { product.Id, product.Url } }
};
Reduce = results =>
from result in results
group result by new { result.ClientName, result.Block }
into g
select new {
g.Key.ClientName,
g.Key.Block,
ProductKeys = g.SelectMany(x => x.ProductKeys)
};
}
}
When replicating I get the same InvalidOperationException, stating that it doesn't understand the index definition (stack trace omitted for brevity).
Url: "/indexes/Keys/ByNameAndBlock"
System.InvalidOperationException: Could not understand query:
I'm still not entirely sure what you're attempting here, so this may not be quite what you're after, but I managed to get the following working. In short, Map/Reduce deals in anonymous objects, so strongly typing to your custom types makes no sense to Raven.
public class Keys_ByNameAndBlock : AbstractIndexCreationTask<Product, BlockResult>
{
public Keys_ByNameAndBlock()
{
Map = products =>
from product in products
where product.Block != null
select new
{
product.Name,
product.Block,
ProductIds = product.ProductKeys.Select(x => x.Id)
};
Reduce = results =>
from result in results
group result by new {result.Name, result.Block}
into g
select new
{
g.Key.Name,
g.Key.Block,
ProductIds = g.SelectMany(x => x.ProductIds)
};
}
}
public class Product
{
public Product()
{
ProductKeys = new List<ProductKey>();
}
public int ProductId { get; set; }
public string Url { get; set; }
public string Name { get; set; }
public string Block { get; set; }
public IEnumerable<ProductKey> ProductKeys { get; set; }
}
public class ProductKey
{
public int Id { get; set; }
public string Url { get; set; }
}
public class BlockResult
{
public string Name { get; set; }
public string Block { get; set; }
public int[] ProductIds { get; set; }
}