Develop Reference

Ignoring / Not Mapping certain POCO properties in NEST 2.0

I am having a hard time finding good detail on NEST 2.0, the wrapper for Elasticsearch 2.2 I am using. My question is this: Can I do a bulk insert on an object (class with public data members) and map that to Elasticsearch where only the mapped fields between my C# class and the ES server mapping will save? And it will not add the additional fields in my class I do not want?
Right now I have a class of with strings and doubles and lists of other classes in it. I want to save the strings and doubles, but NOT include the Lists on my bulk inserts to Elasticsearch. It wants to save every piece of data in every field of my class. Is there a class member attribute or some other way to say "do not add this field if it has no mapping" that I have missed? I hope so.

You can ignore properties of your POCO in a number of ways with NEST 2.x. Let's use the following POCO as an example
using Nest;
using Newtonsoft.Json;
[ElasticsearchType(Name = "company")]
public class Company
{
public string Name { get; set; }
[String(Ignore = true)]
public string IgnoreViaAttribute { get; set; }
public string IgnoreViaSettings { get;set; }
[JsonIgnore]
public string IgnoreViaSerializerSpecificAttribute { get; set; }
}
1.Using the Ignore property on a derived ElasticsearchPropertyAttribute type (in our example, StringAttribute on IgnoreViaAttribute property) applied to the property that should be ignored on the POCO
2.Using the .InferMappingFor<TDocument>(Func<ClrTypeMappingDescriptor<TDocument>, IClrTypeMapping<TDocument>> selector) on the connection settings
var settings = new ConnectionSettings(new Uri("http://localhost:9200"))
.InferMappingFor<Company>(i => i
.Ignore(p => p.IgnoreViaSettings)
);
var client = new ElasticClient(settings);
These settings are cached per instance of ConnectionSettings.
3.Using an ignore attribute applied to the POCO property that is understood by the IElasticsearchSerializer used, and inspected inside of the CreatePropertyMapping() on the serializer. In the case of the default JsonNetSerializer, this is the Json.NET JsonIgnoreAttribute. In our example, this is demonstrated by the attribute applied to the IgnoreViaSerializerSpecificAttribute property.

What I found by digging around a bit and testing a small class is that the following structure did indeed hide the attributes when you post a class with NEST 2.0 C#. If you do this just above the class member you wish to ignore when doing a bulk add that covers it.
[String(Ignore = true)]

Converting JSON data into an object

I am very new to this.Pardon me if I make any mistakes.
I have data in JSON form.Can I read this data directly and use it in C# code ?
From what I understood from reading up on the internet,I think I have to convert it into an object form to use the data.Am I right ?
If yes,Then I saw this method to convert as below :
string data = JsonConvert.DeserializeObject<string>(getmyissue());
getmyissue is the function which returns a string which has data in json format.
This gives me an exception saying
"Error reading string.Unexpected Token."
Can someone guide me where am I going wrong ?
EDIT
MyIssue.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace Example
{
public class MyIssue
{
public string name{get;set;}
public string description { get; set; }
public string created { get;set; }
public string updated{get;set;}
public string displayName { get; set; }
}
}
Program.cs
MyIssue obj=null;
try
{
obj = JsonConvert.DeserializeObject<MyIssue>(manager.getmyissue());
}
catch(Exception e)
{
Console.WriteLine(e.Message);
}
Console.WriteLine("Name: "+obj.name);

The easiest way to de-serialize Json in a C#/.NET program is to use the brilliant NewtonSoft JSON library.
There are numerous ways to do it, but I have to admit that the NS libs just get on with the task (and no I'm not a member of the team etc, just an Avid User :-) ).
To do what you want for example, if you had:
{'Name': 'A person', 'AllowAccess': true,'Areas': ['Sales','Admin']}
You would first build an object to represent it as follows:
public MyObject
{
public string Name { get; set; }
public bool AllowAccess { get; set; }
public List<string> Areas { get; set; }
}
Once you've done this, it's a simple case of just doing the following:
string jsonString = "// Your json formated string data goes here";
MyObject myObject = JsonConvert.DeserializeObject<MyObject>(jsonString);
The properties in your object should at that point, reflect the properties in the JSON data you sent to it.
You will of course need to add the NS JSON Libs to your project, either via NuGet or Manually, which ever is easier for you, everything you need to know about that is here:
How to install JSON.NET using NuGet?
The really good thing about NS JSON however is not the ease of use, but the fact that it can also do dynamic de-serialization.
This comes in handy if you've no idea what to expect in the JSON you receive, and so don't know ahead of time how to construct an object to hold the results.
Rather than repeat what others have said however, you can find more information of doing things dynamically in this stack overflow post:
Deserializing JSON using JSon.NET with dynamic data
Update
Looking at your JSON data you have way more fields/properties in there than your trying to parse, and none of the libraries in common use (To the best of my knowledge) will pick and choose the fields to copy, you either have an object that represents them all, or not at all, the later of which I believe is the problem your facing.
I have a rather neat "JSON" plug in for chrome, than when given a chunk of JSON data formats the output for me nicely and makes it easy to read, it also helps massively when defining objects, allowing you to see the full nested structure of your data, here are a series of images showing your JSON data formatted using this plugin:
I'm not going to paste anymore images in, but that goes on for another 4 pages!!
Now, some extra information that may help you.
I know from experience (I had to write a parser in PHP for these Jira webhooks) that within the Jira control panel, you can configure your webhooks to ONLY return the information your interested in.
Right now, it looks like you've just told the system to dump everything, for every event that you've hooked too (Which looks like - all of them), it's been a while since I did any work with these, but as well as a global webhook, you also have individual webhooks, which only fire on specific events and produce JSON data that's very much smaller than what your dealing with here.
I'd therefore advise you, to take a look in your Jira control panel (Or ask your Admin/Lead Dev/etc to take a look) and seriously trim down as much of that data as you can.
Further more, if memory serves me right, you can also make various web API calls to the Jira service to get this info too, and in that case you can tell the API exactly what your interested in, meaning it will only return the fields you need.
Right now, your main problem is the sheer volume of data your trying to deal with, if you tackle that problem, you'll find the issues surrounding the code your trying to get working will be very much easier to deal with.
Update 2
Just to make it clearer what I mean by using a "dynamic" type to get at your data, you would use something like the following code:
string jsonString = "// Your json formated string data goes here";
var result = JsonConvert.DeserializeObject<dynamic>(jsonString);
The difference here is that your using the C# dynamic type rather than a strongly typed object of your own design.
"dynamic" is useful, because it's kind of like having an empty object, and then having the properties added for you, without you having to define it.
What this essentially means is that, if you pass in the following JSON:
{'Name': 'A person', 'AllowAccess': true,'Areas': ['Sales','Admin']}
You'll end up with a dynamic object that looks like:
result = dynamic
{
public string Name { get; set; }
public bool AllowAccess { get; set; }
public List<string> Areas { get; set; }
}
thus:
result.Name
will get you access to the contents of the Name field and so on.
If your JSON was then changed to become:
{'Name': 'A person', 'AllowAccess': true,'Areas': ['Sales','Admin'], 'Location': 'The World' }
Your object would magically have a property called 'Location' containing the value 'The World' which you could access using:
result.Location
In your case, this would allow you to define your concrete object EG:
public MyObject
{
public string Name { get; set; }
public string Email { get; set; }
}
and then do something like the following (Assuming that your inbound JSON had properties in called Name & Email):
string jsonString = "// Your json formated string data goes here";
var result = JsonConvert.DeserializeObject<dynamic>(jsonString);
MyObject myObject = new MyObject
{
Name = result.Name,
Email = result.Email
}
You'd then discard the dynamic object as you'd not need it anymore.
The BIG problem your going to have with this approach is maintaining your models. Manual property assignment is all fine and dandy for a small handful of properties and objects, but it soon becomes a huge maintenance nightmare as your software grows.
I'm sure it doesn't take much to imagine what kind of task you'd be facing if you had to do this for 100 different JSON requests and 50 different types of objects.
For this reason, using this approach you should really consider using some kind of mapping technology such as "AutoMapper", however for now I'm going to advise you leave that until later before you start researching it, as it'll not help you to be clear about dealing with this dynamic approach.

The JSON you get is already a string, so converting it to string doesn't make much sense. You need to create classes that reflect the structure represented by the JSON string.
For example to convert the following JSON into objects, you'd have to create a class for the users:
{"user":{"name":"asdf","teamname":"b","email":"c","players":["1","2"]}}
public class User
{
public string name { get; set; }
public string teamname { get; set; }
public string email { get; set; }
public Array players { get; set; }
}
Then you should be able to use this:
JavaScriptSerializer jss= new JavaScriptSerializer();
List<User> users = jss.Deserialize<List<User>>(jsonResponse);

c# how to use system reserved names as properties [duplicate]

I have never used Web API before, but I need a web service that will accept/return JSON objects and using this seemed like a reasonable thing. It looked pretty simple (if not a bit of overkill for my purposes), but a data structure I need to deal with looks something like:
{
"values":["foo", "bar"],
"default":"bar"
}
And so I went to make a Model object:
class DropDownValues {
public string[] values { get; set; }
public string default { get; set; }
}
Problem is that default seems to be a protected keyword. There must be some way to get around that, right?

You can use keywords in C# as identifiers by prepending # in front of them.

I would suggest to go different way. Keep your C# object model as much standard as possible (I wouldn't use # sign and C# keywords as property name).
We can separate the serialized (JSON) world and C# objects - just by using the Json.NET features.
One of the simpliest to use is decoration with Attribute:
[JsonProperty(PropertyName = "default")]
public string DefaultValue { get; set; }
In this case we have to reference Newtonsoft.Json in the project. If it must be POCO, we can introduce CustomResolver derrived from DefaultContractResolver and define these conversions there...
But separation of concern in this case is a bit more pure solution, I would say
EDIT: JSON Contract Resolver draft (see comments)
Important NOTE: Newtonsoft.Json is part of the Web API. Not only it is an open source, but even MS team bet on that as a core JSON serializer.
1) Newtonsoft.Json (as a part of the Web.API) is already installed in your solution. So you do not have to downloaded (nuget) separately. It would always be in your packages folder. So, to use the attribute is just adding the reference. It is there...
2) There is a small draft how to do the attribute stuff, while keeping the POCO. As I've tried explain here: POCO's, behavior and Peristance Igorance, to keep POCO (e.g. we do profit from layered Architecture with NHibernate on a data layer), we can replace attributes with a Contract Resolver. Our POCO library does not have to reference anything
We just have to do extend the service layer:
public class MyResolver : DefaultContractResolver
{
protected override JsonProperty CreateProperty(
MemberInfo member,
MemberSerialization memberSerialization)
{
var jProperty = base.CreateProperty(member, memberSerialization);
var propertyInfo = member as PropertyInfo;
if (propertyInfo == null)
{
return jProperty;
}
// just adjust in case if Property name is DefaultValue
var isDefaultValueProeprty =
propertyInfo.Name.Equals("DefaultValue");
if(isDefaultValueProeprty)
{
jProperty.PropertyName = "default";
}
return jProperty;
}
...
This way we've provided the same information to serailizer as with the [JsonPropertyAttribute].
Now, we just have to use it. There are many ways (e.g. global) but we can do it for a controller only:
protected override void Initialize(HttpControllerContext context)
{
base.Initialize(context);
var jSettings = context.Configuration.Formatters.JsonFormatter.SerializerSettings;
jSettings.ContractResolver = MyResolver;
}

The class DropDownValues using camel convention:
class DropDownValues {
public string[] values { get; set; }
public string default { get; set; }
}
You can use prefix # to passby but it is still not following C# coding convention.
The better solution which you can both avoid reserved keyword and still use C# coding convention is using CamelCasePropertyNamesContractResolver:
class DropDownValues {
public string[] Values { get; set; }
public string Default { get; set; }
}
And customize JsonFormatter to avoid convention mismatch between C# and json object as below:
var jsonFormatter = configuration.Formatters.JsonFormatter;
jsonFormatter.SerializerSettings = new JsonSerializerSettings()
{
ContractResolver = new CamelCasePropertyNamesContractResolver()
};

How can I use a reserved keyword as an identifier in my JSON model class?

I have never used Web API before, but I need a web service that will accept/return JSON objects and using this seemed like a reasonable thing. It looked pretty simple (if not a bit of overkill for my purposes), but a data structure I need to deal with looks something like:
{
"values":["foo", "bar"],
"default":"bar"
}
And so I went to make a Model object:
class DropDownValues {
public string[] values { get; set; }
public string default { get; set; }
}
Problem is that default seems to be a protected keyword. There must be some way to get around that, right?

You can use keywords in C# as identifiers by prepending # in front of them.

I would suggest to go different way. Keep your C# object model as much standard as possible (I wouldn't use # sign and C# keywords as property name).
We can separate the serialized (JSON) world and C# objects - just by using the Json.NET features.
One of the simpliest to use is decoration with Attribute:
[JsonProperty(PropertyName = "default")]
public string DefaultValue { get; set; }
In this case we have to reference Newtonsoft.Json in the project. If it must be POCO, we can introduce CustomResolver derrived from DefaultContractResolver and define these conversions there...
But separation of concern in this case is a bit more pure solution, I would say
EDIT: JSON Contract Resolver draft (see comments)
Important NOTE: Newtonsoft.Json is part of the Web API. Not only it is an open source, but even MS team bet on that as a core JSON serializer.
1) Newtonsoft.Json (as a part of the Web.API) is already installed in your solution. So you do not have to downloaded (nuget) separately. It would always be in your packages folder. So, to use the attribute is just adding the reference. It is there...
2) There is a small draft how to do the attribute stuff, while keeping the POCO. As I've tried explain here: POCO's, behavior and Peristance Igorance, to keep POCO (e.g. we do profit from layered Architecture with NHibernate on a data layer), we can replace attributes with a Contract Resolver. Our POCO library does not have to reference anything
We just have to do extend the service layer:
public class MyResolver : DefaultContractResolver
{
protected override JsonProperty CreateProperty(
MemberInfo member,
MemberSerialization memberSerialization)
{
var jProperty = base.CreateProperty(member, memberSerialization);
var propertyInfo = member as PropertyInfo;
if (propertyInfo == null)
{
return jProperty;
}
// just adjust in case if Property name is DefaultValue
var isDefaultValueProeprty =
propertyInfo.Name.Equals("DefaultValue");
if(isDefaultValueProeprty)
{
jProperty.PropertyName = "default";
}
return jProperty;
}
...
This way we've provided the same information to serailizer as with the [JsonPropertyAttribute].
Now, we just have to use it. There are many ways (e.g. global) but we can do it for a controller only:
protected override void Initialize(HttpControllerContext context)
{
base.Initialize(context);
var jSettings = context.Configuration.Formatters.JsonFormatter.SerializerSettings;
jSettings.ContractResolver = MyResolver;
}

The class DropDownValues using camel convention:
class DropDownValues {
public string[] values { get; set; }
public string default { get; set; }
}
You can use prefix # to passby but it is still not following C# coding convention.
The better solution which you can both avoid reserved keyword and still use C# coding convention is using CamelCasePropertyNamesContractResolver:
class DropDownValues {
public string[] Values { get; set; }
public string Default { get; set; }
}
And customize JsonFormatter to avoid convention mismatch between C# and json object as below:
var jsonFormatter = configuration.Formatters.JsonFormatter;
jsonFormatter.SerializerSettings = new JsonSerializerSettings()
{
ContractResolver = new CamelCasePropertyNamesContractResolver()
};

InvalidDataContractException on WP7 application on class that is serializable

I'm having an strange error when trying to save an object into isolated storage. I have a class that has some properties, here's the code :
[DataContract]
public class ExerciseStatistic
{
[XmlIgnore]
public int CorrectAnswers
{
get
{
return Attempts.Where(a => a.AttemptAnswerIsCorrect).Count();
}
}
[XmlIgnore]
public int IncorrectAnswers
{
get
{
return Attempts.Where(a => !a.AttemptAnswerIsCorrect).Count();
}
}
[XmlIgnore]
public int AnswerAttempts
{
get { return Attempts.Count; }
}
public List<AnswerAttempt> Attempts { get; set; }
public ExerciseStatistic()
{
Attempts = new List<AnswerAttempt>();
}
}
public class AnswerAttempt
{
public DateTime AttemptDate { get; set; }
public string AttemptTargetName { get; set; }
public string AttemptName { get; set; }
public bool AttemptAnswerIsCorrect { get; set; }
}
However, when trying to save it with this sentence :
IsolatedStorageSettings.ApplicationSettings["a"] = new ExerciseStatistic()
{
Attempts = new List<AnswerAttempt>()
{
new AnswerAttempt()
{
AttemptAnswerIsCorrect = true,
AttemptDate = DateTime.Now,
AttemptName = "lala",
AttemptTargetName = "lala2"
},
new AnswerAttempt()
{
AttemptAnswerIsCorrect = false,
AttemptDate = DateTime.Now,
AttemptName = "lalab",
AttemptTargetName = "lalab2"
}
}
};
I'm getting an exception like this one (i changed a bit the signature of the code with fake names, but for the example it serves its purpose) :
Type 'XX.Model.FirstClass.SecondClass' cannot be serialized. Consider
marking it with the DataContractAttribute attribute, and marking all
of its members you want serialized with the DataMemberAttribute
attribute.
I don't understand why the serializer is trying to serialize an object of my model (which is not serializable) when the class that I'm giving it doesn't have any references to that kind of type... what am i missing? -> nope, i don't want to add datacontract attributes to classes that i don't need and am not planning to serialize, so please don't answer with this :)

You might experience this problem if you work through the reference procedure in "Walkthrough: Consuming OData with MVVM for Windows Phone" at http://msdn.microsoft.com/en-us/library/hh394007(v=VS.92).aspx
When you get to the point where you call :
Return DataServiceState.Serialize(_context, collections);
You might get an InvalidDataContractException with the message:
Type 'DataBoundApp1.Northwind.NorthwindEntities' cannot be serialized. Consider marking it with the DataContractAttribute attribute, and marking all of its members you want serialized with the DataMemberAttribute attribute.
Thanks to the answer by Daniel Perez, I was able to resolve this problem and I am documenting my steps to clarify the solution for others:
Show hidden files in Solution Explorer
Open the file "Reference.cs" (under your Service Reference, expand
Reference.datasvcmap)
If your Data Service Context class is missing the [DataContract]
attribute, add it as shown here:
.
namespace OCC.WindowsPhone.OrlandoCodeCampService
{
[DataContract] <--- I ADDED THIS
public partial class OrlandoCodeCampEntities : global::System.Data.Services.Client.DataServiceContext
{..}
Once I added the DataContract attribute, the problem went away!

It seems to me you try to exclude properties from serialization by using XmlIgnore.
From the documentation:
You can opt out members from serialization by using the IgnoreDataMemberAttribute.
so try using IgnoreDataMemberAttribute instead of XmlIgnore to opt out members from serialization.
I also had some troubles with DataContract in the very same situation as you, therefore I reverted to plain old XML serialization to strings, which i then stored in isolated storage. This also eases debugging.

I don't think that this is a proper answer, but it's what i had to do in order to fix it.
After changing some more the code, i realised that this was failing EVEN if I wasn't saving anything to the isolated storage. Just declaring a DataContract attribute on the type made the error arise. I must think that WP7's framework at some point is parsing all classes that have this attribute, and for some strange and obscure reason (which i can't find) it's looking for them in other classes as well. I added the DataContract attributes in the classes that the framework is complaning about, and also some KnownType attributes as well, and everything started to run smoothly... weird weird... if someone can shed some light into this, i'd be happy (i hate it when i solve a problem but without knowing the exact cause)