Develop Reference

How do I define custom/dynamic columns/fields/properties in OData (v8 in .NET 6)?

There is a sample here for creating 100% dynamic OData models in Microsoft.AspNetCore.OData 8.x. However, in our case we have an existing model that we are happy with, but we want to add custom fields to it.
In other words, we want an OData model with entities that have some fixed columns/properties and some dynamically-generated columns/properties that come from the database, like this:
public class ODataEntity
{
[Key]
public int Id { get; set; }
public string Name { get; set; } = "";
// From the perspective of clients like Power BI, this should produce
// a series of additional columns (the columns are the same on all
// instances, but the schema can change at any time)
public Dictionary<string, object> CustomFields { get; set; }
}
To my tremendous surprise, the key-value pairs in CustomFields become properties in the JSON output (i.e. there is no CustomFields column; its contents are inserted into the parent object). However, the custom fields are not recognized by Power BI:
I assume that this is because there is no metadata for the custom fields in https://.../odata/$metadata. So my question is:
How can I modify the following code so that the custom columns are included in the IEdmModel?
static IEdmModel GetEdmModel(params CustomFieldDef[] customFields)
{
var builder = new ODataConventionModelBuilder() {
Namespace = "Namespace",
ContainerName = "Container", // no idea what this is for
};
builder.EntitySet<ODataEntity>("objects");
return builder.GetEdmModel();
}
public class CustomFieldDef
{
public string FieldName;
public Type Type;
}
How can I modify the following startup code so that the IEdmModel is regenerated every time https://.../odata/$metadata is accessed?
IMvcBuilder mvc = builder.Services.AddControllers();
mvc.AddOData(opt => opt.AddRouteComponents("odata", GetEdmModel())
.Select().Filter().OrderBy().Count().Expand().SkipToken());

Regarding the first question, there are two basic approaches that one could take:
1. Dynamic Everything
Use the 100% dynamic approach that is used in the ODataDynamicModel sample. This approach is difficult, especially if you already have a working model, because (i) the way the sample code works is difficult to understand, and (ii) you have to completely rewrite your code, or use reflection to help generate both the schema ($metadata) and the output data.
2. Modify the EdmModel
This is the approach I'm taking in this answer.
Step 1: change the model
The IEdmModel returned by ODataConventionModelBuilder.GetEdmModel is mutable (and actually has type EdmModel), so you can add custom fields to it.
static IEdmModel GetEdmModel(params CustomFieldDef[] customFields)
{
var builder = new ODataConventionModelBuilder() {
Namespace = "Namespace",
ContainerName = "Container", // no idea what this is for
};
builder.EntitySet<ODataEntity>("objects");
var model = (EdmModel) builder.GetEdmModel();
IODataTypeMapper mapper = model.GetTypeMapper();
foreach (var edmType in model.SchemaElements.OfType<EdmEntityType>()) {
if (edmType.Name == nameof(ODataEntity)) {
foreach (var field in customFields) {
var typeRef = mapper.GetEdmTypeReference(model, field.Type);
edmType.AddStructuralProperty(field.FieldName, typeRef);
}
}
}
return model;
}
Step 2: Obtain the custom fields
Normally you call mvc.AddOData in ConfigureServices in Startup.cs, passing it a lambda that will create the IEdmModel. But wait, are your custom field definitions stored in your database? If so, how can you access the database from inside ConfigureServices? Don't worry: the lambda passed to AddOData is called after Configure, so it is possible to access the database or any other required service with some slightly hacky code. This code also installs a class called ODataResourceSerializerForCustomFields which is the subject of step 3:
public void ConfigureServices(IServiceCollection services)
{
...
IMvcBuilder mvc = services.AddControllers(...);
// OData Configuration
mvc.AddOData(opt => {
var scopeProvider = _serviceProvider!.CreateScope().ServiceProvider;
var cfdm = scopeProvider.GetRequiredService<CustomFieldDefManager>();
var edmModel = GetEdmModel(cfdm.GetAll());
opt.AddRouteComponents("odata", edmModel, services => {
services.AddScoped<ODataResourceSerializer,
ODataResourceSerializerForCustomFields>();
}).Select().Filter().OrderBy().Count().Expand().SkipToken();
});
...
}
IServiceProvider? _serviceProvider;
public void Configure(IApplicationBuilder app, ...)
{
_serviceProvider = app.ApplicationServices;
...
}
Of course, this code is not dynamic: it generates the EdmModel only once on startup. I will figure out later how to make this dynamic.
Step 3: Stop it from crashing
Microsoft.AspNetCore.OData.dll isn't designed to support custom fields in the EdmModel. It's a young product, you understand, only version 8.0. As soon as you add a custom field, you'll get an InvalidOperationException like "The EDM instance of type '[ODataEntity Nullable=True]' is missing the property 'ExampleCustomField'., because the library assumes that all properties in the EdmModel are real CLR properties.
I found a way around this problem by overriding a few methods of ODataResourceSerializer. But first, define an IHasCustomFields interface and make sure that any OData entity with custom fields implements this interface:
public interface IHasCustomFields
{
public Dictionary<string, object?> CustomFields { get; set; }
}
Now let's add ODataResourceSerializerForCustomFields, which uses special behavior when this interface is present.
/// <summary>
/// This class modifies the behavior of ODataResourceSerializer to
/// stop it from crashing when the EdmModel contains custom fields.
/// Note: these modifications are designed for simple custom fields
/// (e.g. string, bool, DateTime).
/// </summary>
public class ODataResourceSerializerForCustomFields : ODataResourceSerializer
{
public ODataResourceSerializerForCustomFields(IODataSerializerProvider serializerProvider)
: base(serializerProvider) { }
IHasCustomFields? _hasCustomFields;
HashSet<string>? _realProps;
public override Task WriteObjectInlineAsync(
object graph, IEdmTypeReference expectedType,
ODataWriter writer, ODataSerializerContext writeContext)
{
_hasCustomFields = null;
if (graph is IHasCustomFields hasCustomFields) {
_hasCustomFields = hasCustomFields;
var BF = BindingFlags.Public | BindingFlags.Instance;
_realProps = graph.GetType().GetProperties(BF).Select(p => p.Name).ToHashSet();
}
return base.WriteObjectInlineAsync(graph, expectedType, writer, writeContext);
}
public override ODataResource CreateResource(
SelectExpandNode selectExpandNode, ResourceContext resourceContext)
{
return base.CreateResource(selectExpandNode, resourceContext);
}
public override ODataProperty CreateStructuralProperty(
IEdmStructuralProperty structuralProperty, ResourceContext resourceContext)
{
// Bypass tne base class if the current property doesn't physically exist
if (_hasCustomFields != null && !_realProps!.Contains(structuralProperty.Name)) {
_hasCustomFields.CustomFields.TryGetValue(structuralProperty.Name, out object? value);
return new ODataProperty {
Name = structuralProperty.Name,
Value = ToODataValue(value)
};
}
return base.CreateStructuralProperty(structuralProperty, resourceContext);
}
public static ODataValue ToODataValue(object? value)
{
if (value == null)
return new ODataNullValue();
if (value is DateTime date)
value = (DateTimeOffset)date;
return new ODataPrimitiveValue(value);
}
// The original implementation of this method can't be prevented from
// crashing, so replace it with a modified version based on the original
// source code. This version is simplified to avoid calling `internal`
// methods that are inaccessible, but as a result I'm not sure that it
// behaves quite the same way. If properties that aren't in the EdmModel
// aren't needed in the output, the method body is optional and deletable.
public override void AppendDynamicProperties(ODataResource resource,
SelectExpandNode selectExpandNode, ResourceContext resourceContext)
{
if (_hasCustomFields == null) {
base.AppendDynamicProperties(resource, selectExpandNode, resourceContext);
return;
}
if (!resourceContext.StructuredType.IsOpen || // non-open type
(!selectExpandNode.SelectAllDynamicProperties && selectExpandNode.SelectedDynamicProperties == null)) {
return;
}
IEdmStructuredType structuredType = resourceContext.StructuredType;
IEdmStructuredObject structuredObject = resourceContext.EdmObject;
object value;
if (structuredObject is IDelta delta) {
value = ((EdmStructuredObject)structuredObject).TryGetDynamicProperties();
} else {
PropertyInfo dynamicPropertyInfo = resourceContext.EdmModel.GetDynamicPropertyDictionary(structuredType);
if (dynamicPropertyInfo == null || structuredObject == null ||
!structuredObject.TryGetPropertyValue(dynamicPropertyInfo.Name, out value) || value == null) {
return;
}
}
IDictionary<string, object> dynamicPropertyDictionary = (IDictionary<string, object>)value;
// Build a HashSet to store the declared property names.
// It is used to make sure the dynamic property name is different from all declared property names.
HashSet<string> declaredPropertyNameSet = new HashSet<string>(resource.Properties.Select(p => p.Name));
List<ODataProperty> dynamicProperties = new List<ODataProperty>();
// To test SelectedDynamicProperties == null is enough to filter the dynamic properties.
// Because if SelectAllDynamicProperties == true, SelectedDynamicProperties should be null always.
// So `selectExpandNode.SelectedDynamicProperties == null` covers `SelectAllDynamicProperties == true` scenario.
// If `selectExpandNode.SelectedDynamicProperties != null`, then we should test whether the property is selected or not using "Contains(...)".
IEnumerable<KeyValuePair<string, object>> dynamicPropertiesToSelect =
dynamicPropertyDictionary.Where(x => selectExpandNode.SelectedDynamicProperties == null || selectExpandNode.SelectedDynamicProperties.Contains(x.Key));
foreach (KeyValuePair<string, object> dynamicProperty in dynamicPropertiesToSelect) {
if (string.IsNullOrEmpty(dynamicProperty.Key))
continue;
if (declaredPropertyNameSet.Contains(dynamicProperty.Key))
continue;
dynamicProperties.Add(new ODataProperty {
Name = dynamicProperty.Key,
Value = ToODataValue(dynamicProperty.Value)
});
}
if (dynamicProperties.Count != 0)
resource.Properties = resource.Properties.Concat(dynamicProperties);
}
}

How to get names of all required fields in new row with EF?

I'm using EF(db first) and trying to add new row in table using the next code:
var user = new User();
//Some logic to fill the properties
context.Users.AddObject(user);
context.SaveChanges();
Before saving changes on EF i want to verify that all required (not null and with no default value) properties are filled. How can i get all such fields?
I've tried few ways, but can't achieve needed result. The last try was like that:
var resList = new List<PropertyInfo>();
var properties = type.GetProperties(BindingFlags.DeclaredOnly |
BindingFlags.Public |
BindingFlags.Instance).Where(p => !p.PropertyType.IsGenericType);
foreach (var propertyInfo in properties)
{
var edmScalarProperty =
propertyInfo.CustomAttributes.FirstOrDefault(
x => x.AttributeType == typeof (EdmScalarPropertyAttribute));
var isNullable = true;
if (edmScalarProperty != null)
{
var arg = edmScalarProperty.NamedArguments.FirstOrDefault(x => x.MemberName == "IsNullable");
if (arg != null)
{
isNullable = (bool) arg.TypedValue.Value;
}
}
if (!isNullable)
{
resList.Add(propertyInfo);
}
}
return resList;

Create a constructor with the required fields as parameters.
I always separate my domain objects from my EF objects (DTO objects). The domain object has only one constructor with the required fields. When I want to save these objects I convert them to DTO objects.

Have you looked at all into DataAnnotations for your model classes? Utilizing these (and using a separate object from the one EF creates for you) you can get pretty significant validation built into your models from the model level. Additionally, as L01NL pointed out, you can have your constructor take in parameters that require data.
Lots of information on Model and Validation can be found, one such example is:
http://msdn.microsoft.com/en-us/library/dd410405(v=vs.100).aspx
(look through this main section and its subsections)
using System.ComponentModel.DataAnnotations
public class Foo
{
public Guid Id { get; private set; }
[StringLength(50),Required]
public string FooName { get; private set; }
[Required]
public int Age { get; private set; }
// etc props
public Foo(string fooName, int age)
{
if (string.IsNullOrEmpty(fooName))
throw new ArgumentException("FooName cannot be null or empty"); // note there is also a "minimum length" data annotation to avoid doing something like this, was just using this as an example.
this.Id = Guid.NewGuid();
this.FooName = fooName;
this.Age = age;
}
}
public class YourController
{
[HttpPost]
public ActionResult Add(Foo foo)
{
if (!ModelState.IsValid)
// return - validation warnings, etc
// Add information to persistence
// return successful add?
}
}

asp.net mvc web api partial update with OData Patch

I am using HttpPatch to partially update an object. To get that working I am using Delta and Patch method from OData (mentioned here: What's the currently recommended way of performing partial updates with Web API?). Everything seems to be working fine but noticed that mapper is case sensitive; when the following object is passed the properties are getting updated values:
{
"Title" : "New title goes here",
"ShortDescription" : "New text goes here"
}
But when I pass the same object with lower or camel-case properties, Patch doesn't work - new value is not going through, so it looks like there is a problem with deserialisation and properties mapping, ie: "shortDescription" to "ShortDescription".
Is there a config section that will ignore case sensitivity using Patch?
FYI:
On output I have camel-case properties (following REST best practices) using the following formatter:
//formatting
JsonSerializerSettings jss = new JsonSerializerSettings();
jss.ContractResolver = new CamelCasePropertyNamesContractResolver();
config.Formatters.JsonFormatter.SerializerSettings = jss;
//sample output
{
"title" : "First",
"shortDescription" : "First post!"
}
My model classes however are follwing C#/.NET formatting conventions:
public class Entry {
public string Title { get; set;}
public string ShortDescription { get; set;}
//rest of the code omitted
}

Short answer, No there is no config option to undo the case sensitiveness (as far as i know)
Long answer: I had the same problem as you today, and this is how i worked around it.
I found it incredibly annoying that it had to be case sensitive, thus i decided to do away with the whole oData part, since it is a huge library that we are abusing....
An example of this implementation can be found at my github github
I decided to implement my own patch method, since that is the muscle that we are actually lacking. I created the following abstract class:
public abstract class MyModel
{
public void Patch(Object u)
{
var props = from p in this.GetType().GetProperties()
let attr = p.GetCustomAttribute(typeof(NotPatchableAttribute))
where attr == null
select p;
foreach (var prop in props)
{
var val = prop.GetValue(this, null);
if (val != null)
prop.SetValue(u, val);
}
}
}
Then i make all my model classes inherit from *MyModel*. note the line where i use *let*, i will excplain that later. So now you can remove the Delta from you controller action, and just make it Entry again, as with the put method. e.g.
public IHttpActionResult PatchUser(int id, Entry newEntry)
You can still use the patch method the way you used to:
var entry = dbContext.Entries.SingleOrDefault(p => p.ID == id);
newEntry.Patch(entry);
dbContext.SaveChanges();
Now, let's get back to the line
let attr = p.GetCustomAttribute(typeof(NotPatchableAttribute))
I found it a security risk that just any property would be able to be updated with a patch request. For example, you might now want the an ID to be changeble by the patch. I created a custom attribute to decorate my properties with. the NotPatchable attribute:
public class NotPatchableAttribute : Attribute {}
You can use it just like any other attribute:
public class User : MyModel
{
[NotPatchable]
public int ID { get; set; }
[NotPatchable]
public bool Deleted { get; set; }
public string FirstName { get; set; }
}
This in this call the Deleted and ID properties cannot be changed though the patch method.
I hope this solve it for you as well. Do not hesitate to leave a comment if you have any questions.
I added a screenshot of me inspecting the props in a new mvc 5 project. As you can see the Result view is populated with the Title and ShortDescription.

It can be done quite easily with a custom contract resolver that inherits CamelCasePropertyNamesContractResolver and implementing CreateContract method that look at concrete type for delta and gets the actual property name instead of using the one that comes from json. Abstract is below:
public class DeltaContractResolver : CamelCasePropertyNamesContractResolver
{
protected override JsonContract CreateContract(Type objectType)
{
// This class special cases the JsonContract for just the Delta<T> class. All other types should function
// as usual.
if (objectType.IsGenericType &&
objectType.GetGenericTypeDefinition() == typeof(Delta<>) &&
objectType.GetGenericArguments().Length == 1)
{
var contract = CreateDynamicContract(objectType);
contract.Properties.Clear();
var underlyingContract = CreateObjectContract(objectType.GetGenericArguments()[0]);
var underlyingProperties =
underlyingContract.CreatedType.GetProperties(BindingFlags.Public | BindingFlags.Instance);
foreach (var property in underlyingContract.Properties)
{
property.DeclaringType = objectType;
property.ValueProvider = new DynamicObjectValueProvider()
{
PropertyName = this.ResolveName(underlyingProperties, property.PropertyName),
};
contract.Properties.Add(property);
}
return contract;
}
return base.CreateContract(objectType);
}
private string ResolveName(PropertyInfo[] properties, string propertyName)
{
var prop = properties.SingleOrDefault(p => p.Name.Equals(propertyName, StringComparison.OrdinalIgnoreCase));
if (prop != null)
{
return prop.Name;
}
return propertyName;
}
}

SqlTableProfileProvider, custom ProfileBase class, add extra attributes to properties

I am using SqlTableProfileProvider as my profile provider and a custom class called 'ProfileCommon' inheriting from System.Web.Profile.ProfileBase. My ProfileCommon class properties that represent the columns in my profile table each get an attribute of [CustomProviderData("columnName;dbType")]. I am trying to add a new attribute to specific column and then am going to pull that info from the SqlTableProfileProvider class.
The team and I are looking to associate a foreign key (table) with our Profile table. Right now, our Profile table stores, basically, key value pairs, FirstName, LastName, Age, etc; however, we are planning to store bookmarks, links to favorite articles and what not, that will be presented in a list on our dashboard page. We like using the SqlTableProfileProvider and the ProfileCommon object I created. All our asp.net pages inherit from a BasePage and a property called Profile gets the profile common object.
It would be nice to just be able to do:
Profile.Bookmarks.Count; // to know if there are bookmarks
// to also just be able to foreach through them
foreach (Bookmark bk in Profile.Bookmarks) { ... }
Ex:
public class ProfileCommon : System.Web.Profile.ProfileBase
{
public static ProfileCommon GetProfile() { .... }
[CustomProviderData("FirstName;varchar")]
public virtual string FirstName
{
get
{
return ((string)(this.GetPropertyValue("FirstName")));
}
set
{
this.SetPropertyValue("FirstName", value);
}
}
[CustomProviderData("LastName;varchar")]
public virtual string LastName
{
get
{
return ((string)(this.GetPropertyValue("LastName")));
}
set
{
this.SetPropertyValue("LastName", value);
}
}
[CustomProviderData("OtherColumn;int")]
[TableNameData("OtherTable")]
public virtual int OtherColumn
{
get ...
set ...
}
}
// My new attribute
[AttributeUsage(AttributeTargets.Property)]
public class TableNameData : Attribute
{
private string _tableName;
public TableNameData(string tableName)
{
_tableName = tableName;
}
public string TableName
{
get
{
return _tableName;
}
}
}
// Not my implementation, but looking to enhance it.
public class SqlTableProfileProvider : ProfileProvider
{
public override SettingsPropertyValueCollection GetPropertyValues(SettingsContext context, SettingsPropertyValueCollection svc, string username, SqlConnection conn)
{
...
foreach (SettingsProperty prop in properties)
{
...
// in here, gets CustomProviderData
string persistenceData = prop.Attributes["CustomProviderData"] as string.
// how do i get to mine?
}
}
}
The SqlTableProfileProvider was implemented by Hao Kung. It inherits from ProfileProvider.
One of the methods GetPropertyValues returns a SettingsPropertyValueCollection. There is a private method called GetProfileDataFromTable. In there, I wish to access my custom attribute that I created.
Question: How do I access my attribute that I have specified on my property?
UPDATE: 07162011:1517, 7 days after question asked,
I did find a way to do this. The following is how I did it:
// In the default constructor add the following
public ProfileCommon()
{
// Get all properties for this class 'ProfileCommon'
PropertyInfo[] propertyInfos = typeof(ProfileCommon).GetProperties();
// The ProfileBase, base class, has a property called Properties and
// one can get to all attributes on that property but there are only
// a few attributes that ProfileBase looks for. If the developer wishes
// to use custom attributes on a property, it wont appear in the
// ProfileCommon.Properties.Attributes list.
//
// So, what are we going to do, well, we are going to come up with a hack and solution to this problem
//
foreach (SettingsProperty settingsProperty in ProfileCommon.Properties)
{
foreach (PropertyInfo propertyInfo in propertyInfos)
{
if (settingsProperty.Name == propertyInfo.Name)
{
// get all attributes from the associated property, but we are getting it from the propertyInfo variable
// which was retrieved through reflection and will list ALL attributes.
object[] attributes = propertyInfo.GetCustomAttributes(false);
for (int i = 0; i < attributes.Count(); i++)
{
Type type = attributes[i].GetType();
PropertyInfo[] attributeClassProperities = type.GetProperties();
foreach (PropertyInfo attributeClassProperty in attributeClassProperities)
{
// not intested in the TypeId property for the object
if (!attributeClassProperty.Name.Equals("TypeId"))
{
// if the settingsProperty.Attributes does not contain our key value pair, then add it.
if (settingsProperty.Attributes[attributeClassProperty.Name] == null)
{
settingsProperty.Attributes.Add(attributeClassProperty.Name, attributes[i]);
}
}
}
}
}
}
}
}

You only need to take care of implementing your ProfileCommon as you have in the example and make sure the attributes are correct.
For the specialized profile property Bookmark you would also write your own profile class just as you have in the example. There you have to implement a Count property that would query the database for the number of bookmarks of that user.
In the Profile table in the database you would have a Bookmarks column with a suitable FK type such as Guid, Int or BigInt that would be nothing more than a Foreign key to another table you could name [dbo].[Bookmarks] that would actually contain the bookmarks for each user. This Bookmarks table should have a column like UserId that would be a FK to the ASP.NET UserId.

DSL Custom Constructor - only calling when created not loading

Info: VS2010, DSL Toolkit, C#
I have a custom constructor on one of my domain classes which adds some child elements. I have an issue as I only want this to run when the domain class element is created , not every time the diagram is opened (which calls the construtors)
public Entity(Partition partition, params PropertyAssignment[] propertyAssignments)
: base(partition, propertyAssignments)
{
if (SOMETHING_TO_STOP_IT_RUNNING_EACH_TIME)
{
using (Transaction tx = Store.TransactionManager.BeginTransaction("Add Property"))
{
Property property = new Property(partition);
property.Name = "Class";
property.Type = "System.String";
this.Properties.Add(property);
this.Version = "1.0.0.0"; // TODO: Implement Correctly
tx.Commit();
}
}
}

It looks like you are initializing some domain class properties from within the constructor. This is best done by creating an AddRule. AddRules are invoked when an instance of the domain class to which they are attached is added to the model. For example :
[RuleOn(typeof(Entity), FireTime = TimeToFire.TopLevelCommit)]
internal sealed partial class EntityAddRule : AddRule
{
public override void ElementAdded(ElementAddedEventArgs e)
{
if (e.ModelElement.Store.InUndoRedoOrRollback)
return;
if (e.ModelElement.Store.TransactionManager.CurrentTransaction.IsSerializing)
return;
var entity = e.ModelElement as Entity;
if (entity == null)
return;
// InitializeProperties contains the code that used to be in the constructor
entity.InitializeProperties();
}
}
The AddRule then needs to be registered by overriding a function in your domain model class:
public partial class XXXDomainModel
{
protected override Type[] GetCustomDomainModelTypes()
{
return new Type[] {
typeof(EntityAddRule),
}
}
}
For more information about rules, have a look at the "How to: Create Custom Rules" topic in the VS SDK documentation.
Note: the solution is based on the VS 2008 DSL Tools. YMMV.

Although not the correct approach (Paul Lalonde answer is the best),
here's how you may know, at any given time, if the model is being serialized (= loading):
this.Store.TransactionManager.CurrentTransaction!= null &&
this.Store.TransactionManager.CurrentTransaction.IsSerializing