Develop Reference

Transforming Microsoft Graph ListItem Output to a corresponding C# type

The work of transforming JSON data into a typed data model through seems to be made much more complex by the "help" the combination of SharePoint and MS Graph offer. :-)
I have a SharePoint List in Microsoft 365 that I'm accessing through the Graph API in C#, where the query destination is a typed class with properties identical to the SharePoint List Column Properties.
The ListItem class Graph API returns the results in the a Fields.AdditionalData of type Dictionary<string,object{System.Text.Json.JsonElement}> It needs to become an IEnumerable<DataItem>, which I can do by taking the List from the query result through a Serialize/Deserialize round trip, as below:
var backToJSON = ListItems.Select(o => System.Text.Json.JsonSerializer.Serialize(o.Fields.AdditionalData));
var stronglyTypedItems = backToJSON.Select(jsonO => System.Text.Json.JsonSerializer.Deserialize<DataItem>(jsonO));
Is there a way to do this, either with smarter OData or something in Graph API I haven't seen, without taking what used to be JSON and sending it back through JSON Serializers twice?
More details below:
Sample output JSON from Graph Explorer, where value contains an array of :
"value" : [
{ "id": "1001,
"fields": {
"Column" : "true",
"Column2" : "value2",
"Column3" : "65"
}
},
{ "id": "1002,
"fields": {
<and so forth until the array terminates>
]
}
Corresponding C# Class (literally built using "Paste JSON as class"):
Public class DataItem {
public bool Column {get; set;}
public string Column2 {get; set;}
public int Column3 {get; set;}
}
The "Helper" classes in the C# Graph API deliver mostly transformed into the array of fields I actually need:
private static GraphServiceClient graphClient;
public static IListItemsCollectionRequest LicenseExpirationsList => graphClient
.Sites["<guid>"]
.Lists["<nameOfList>"].Items
.Request()
.Header("Accept", "application/json;odata.metadata=none")
.Select("fields,id")
.Expand("fields");
var ListItems = (await GraphHelper.LicenseExpirationsList.GetAsync()).CurrentPage;
// JSON round tripping through JSONSerializer to get the strong type...
// But why? ListItems.Fields.AdditionalData is a Dictionary of JSON elements in the first place!
var backToJSON = ListItems.Select(o => System.Text.Json.JsonSerializer.Serialize(o.Fields.AdditionalData));
var stronglyTypedItems = backToJSON.Select(jsonO => System.Text.Json.JsonSerializer.Deserialize<DataItem>(jsonO));
return stronglyTypedItems;

You could customize the client's JSON serialization to return a derived type of default FieldValueSet.
First, define your own extended FieldValueSet:
public class FieldValueSetWithDataItem : FieldValueSet
{
public bool Column { get; set; }
public string Column2 { get; set; }
public int Column3 { get; set; }
}
Second, implement your own JSON converter:
class CustomFieldValueSetJsonConverter : JsonConverter<FieldValueSet>
{
private static readonly JsonEncodedText ODataTypeProperty
= JsonEncodedText.Encode("#odata.type");
private static readonly JsonEncodedText IdProperty
= JsonEncodedText.Encode("id");
private static readonly JsonEncodedText ColumnProperty
= JsonEncodedText.Encode("Column");
private static readonly JsonEncodedText Column2Property
= JsonEncodedText.Encode("Column2");
private static readonly JsonEncodedText Column3Property
= JsonEncodedText.Encode("Column3");
public override FieldValueSet Read(ref Utf8JsonReader reader,
Type typeToConvert, JsonSerializerOptions options)
{
var result = new FieldValueSetWithDataItem();
using var doc = JsonDocument.ParseValue(ref reader);
var root = doc.RootElement;
foreach (var element in root.EnumerateObject())
{
if (element.NameEquals(ODataTypeProperty.EncodedUtf8Bytes))
{
result.ODataType = element.Value.GetString();
}
else if (element.NameEquals(IdProperty.EncodedUtf8Bytes))
{
result.Id = element.Value.GetString();
}
else if (element.NameEquals(ColumnProperty.EncodedUtf8Bytes))
{
result.Column = element.Value.GetBoolean();
}
else if (element.NameEquals(Column2Property.EncodedUtf8Bytes))
{
result.Column2 = element.Value.GetString();
}
else if (element.NameEquals(Column3Property.EncodedUtf8Bytes))
{
result.Column3 = element.Value.GetInt32();
}
else
{
// Capture unknown property in AdditionalData
if (result.AdditionalData is null)
{
result.AdditionalData = new Dictionary<string, object>();
}
result.AdditionalData.Add(element.Name, element.Value.Clone());
}
}
return result;
}
public override void Write(Utf8JsonWriter writer,
FieldValueSet value, JsonSerializerOptions options)
{
// To support roundtrip serialization:
writer.WriteStartObject();
writer.WriteString(ODataTypeProperty, value.ODataType);
writer.WriteString(IdProperty, value.Id);
if (value is FieldValueSetWithDataItem dataItem)
{
writer.WriteBoolean(ColumnProperty, dataItem.Column);
writer.WriteString(Column2Property, dataItem.Column2);
writer.WriteNumber(Column3Property, dataItem.Column3);
}
if (value.AdditionalData is not null)
{
foreach (var kvp in value.AdditionalData)
{
writer.WritePropertyName(kvp.Key);
((JsonElement)kvp.Value).WriteTo(writer);
}
}
writer.WriteEndObject();
}
}
Last, use the JSON converter when making your request:
// Use custom JSON converter when deserializing response
var serializerOptions = new JsonSerializerOptions();
serializerOptions.Converters.Add(new CustomFieldValueSetJsonConverter());
var responseSerializer = new Serializer(serializerOptions);
var responseHandler = new ResponseHandler(responseSerializer);
var request = (ListItemsCollectionRequest)client.Sites[""].Lists[""].Items.Request();
var listItems = await request
.WithResponseHandler(responseHandler)
.GetAsync();
To access your column values:
var col3 = ((FieldValueSetWithDataItem)listItem.Fields).Column3;

You may find the HttpProvider of the GraphServiceClient helpful in this scenario:
var listItemsCollectionRequest = graphServiceClient
.Sites["<guid>"]
.Lists["<nameOfList>"]
.Items
.Request()
.Header("Accept", "application/json;odata.metadata=none")
.Select("fields,id")
.Expand("fields");
using (var requestMessage = listItemsCollectionRequest.GetHttpRequestMessage())
{
using var responseMessage = await graphServiceClient.HttpProvider.SendAsync(requestMessage);
//deserialize the response body into DataItem
}
By using the HttpProvider you can directly work with the response from the Graph API and deserialize the response body into your custom class.

JSON object does not serialize for huge data

I am trying to serialize data in JSON. But i face below exception.
OutOfMemoryException was unhandled by user code.
An exception of type 'System.OutOfMemoryException' occurred in Newtonsoft.Json.dll but was not handled in user code
Below i defined my code:
Main controller:
public class TrackingController : BaseAngularController
{
var lstDetails = _services.GetTrackingDetailsByAWBIds(awbids, awbType);
if (lstDetails != null)
{
return AngularJson(lstDetails);
}
}
Base Controller:
public abstract class BaseAngularController : Controller
{
public AngularJsonResult<T> AngularJson<T>(T model)
{
return new AngularJsonResult<T>() { Data = model };
}
}
Angular JSON Result class:
public class AngularJsonResult<T> :AngularJsonResult
{
public new T Data
{
get { return (T)base.Data; }
set { base.Data = value; }
}
}
JSON Result class:
public class AngularJsonResult : JsonResult
{
public override void ExecuteResult(ControllerContext context)
{
DoUninterestingBaseClassStuff(context);
SerializeData(context.HttpContext.Response);
}
private void DoUninterestingBaseClassStuff(ControllerContext context)
{
if (context == null)
{
throw new ArgumentNullException("context");
}
var response = context.HttpContext.Response;
response.ContentType = string.IsNullOrEmpty(ContentType) ? "application/json" : ContentType;
if (ContentEncoding != null)
{
response.ContentEncoding = ContentEncoding;
}
}
protected virtual void SerializeData(HttpResponseBase response)
{
if (ErrorMessages.Any())
{
Data = new
{
ErrorMessage = string.Join("\n", ErrorMessages),
ErrorMessages = ErrorMessages.ToArray()
};
response.StatusCode = 400;
}
if (Data == null) return;
response.Write(Data.ToJson());
}
}
Serializing Object to JSON:
public static class JsonExtensions
{
public static string ToJson<T>(this T obj, bool includeNull = true)
{
var settings = new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver(),
Converters = new JsonConverter[] { new StringEnumConverter() },
ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore,//newly added
//PreserveReferencesHandling =Newtonsoft.Json.PreserveReferencesHandling.Objects,
NullValueHandling = includeNull ? NullValueHandling.Include : NullValueHandling.Ignore
};
return JsonConvert.SerializeObject(obj, settings);
}
}
Here i defined AngularJson method for passing object and override ExecuteResult method for converting Object to JSON.
So my SerializeData method was passing Response and converting to Objet in JSON, like Data.ToJson()
Please let me know your suggestions.

Your problem is that you are serializing your huge data to a string in memory on the server, then writing the entire string to the HttpResponseBase (which also buffers everything by default), and running out of memory somewhere in the process, possibly by exceeding the maximum c# string length.
One way to reduce memory use is to serialize directly to HttpResponseBase.OutputStream using JsonSerializer.Serialize(). This avoids the intermediate string representation.
You may also need to set HttpResponseBase.Buffer = false, and if so, follow the advice given in Unbuffered Output Very Slow and wrap the output stream in a BufferedStream.
The following extension method can be used for this:
public static class HttpResponseBaseExtensions
{
public static void WriteJson<T>(this HttpResponseBase response, T obj, bool useResponseBuffering = true, bool includeNull = true)
{
var contentEncoding = response.ContentEncoding ?? Encoding.UTF8;
if (!useResponseBuffering)
{
response.Buffer = false;
// use a BufferedStream as suggested in //https://stackoverflow.com/questions/26010915/unbuffered-output-very-slow
var bufferedStream = new BufferedStream(response.OutputStream, 256 * 1024);
bufferedStream.WriteJson(obj, contentEncoding, includeNull);
bufferedStream.Flush();
}
else
{
response.OutputStream.WriteJson(obj, contentEncoding, includeNull);
}
}
static void WriteJson<T>(this Stream stream, T obj, Encoding contentEncoding, bool includeNull)
{
var settings = new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver(),
Converters = new JsonConverter[] { new StringEnumConverter() },
ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore,//newly added
//PreserveReferencesHandling =Newtonsoft.Json.PreserveReferencesHandling.Objects,
NullValueHandling = includeNull ? NullValueHandling.Include : NullValueHandling.Ignore
};
var serializer = JsonSerializer.CreateDefault(settings);
var textWriter = new StreamWriter(stream, contentEncoding);
serializer.Serialize(textWriter, obj);
textWriter.Flush();
}
}
Then use the extension method in place of response.Write(Data.ToJson());

Return typed object from method

Is there a way to make a function return the type of object I pass in? I would like to call the one method below to return the type I pass in. Is that possible? Should I even be trying to do this? Is there a better way...short of having two different methods?
Currently, I tried the first two calls and I get back (with the first call) what looks like a dictionary with a system.object[] in the value part of the dictionary. Screen shot below might show it better than my explanation. I ask this as I might have more types that I need to deserialize and don't want to have a different method for each.
var firstTry = this.Deserialize(path, typeof(ObservableCollection<ListItemPair>();
var secondTry = this.Deserialize(path, typeof(ListItemPair));
var thirdTry = this.Deserialize(path, typeof(SomeOtherObject));
public static object Deserialize(string jsonFile, object type)
{
var myObject = new object();
try
{
using (StreamReader r = new StreamReader(jsonFile))
{
var serializer = new JavaScriptSerializer();
string json = r.ReadToEnd();
myObject = serializer.Deserialize<object>(json);
}
}
catch (Exception ex)
{
}
return myObject ;
}
public class ListItemPair
{
public string Name
{
get;
set;
}
public object Value
{
get;
set;
}
}
object created:

Yes, you can create a generic method. Your Deserialize() method would look something like this:
public static T Deserialize<T>(string jsonFile)
{
T myObject = default(T);
try
{
using (var r = new StreamReader(jsonFile))
{
var serializer = new JavaScriptSerializer();
string json = r.ReadToEnd();
myObject = serializer.Deserialize<T>(json);
}
}
catch (Exception ex)
{
}
return myObject;
}
In this example T is a type parameter. When invoking this method, you can pass the type like this:
var firstTry = Deserialize<ObservableCollection<ListItemPair>>(path);
var secondTry = Deserialize<ListItemPair>(path);
var thirdTry = Deserialize<SomeOtherObject>(path);
One side note: I wouldn't recommend silently swallowing an exception. In this case, it is expected that the deserialization can fail. Therefore, I would change it to a TryDeserialize() method:
public static bool TryDeserialize<T>(string jsonFile, out T myObject)
{
try
{
using (var r = new StreamReader(jsonFile))
{
var serializer = new JavaScriptSerializer();
string json = r.ReadToEnd();
myObject = serializer.Deserialize<T>(json);
}
}
catch (Exception ex)
{
myObject = default(T);
return false;
}
return true;
}

JSON .Net Unity Deserialize dictionary of anonymous objects

I have a Dictionary that I serialize onto a binary file, and deserialize back again using JSON .net from https://json.codeplex.com/
The dictionary may contain abritrary objects (string, classes, even List). Each class is [System.Serializable]
At serialize time, I add serializer.TypeNameHandling = TypeNameHandling.All; to make sure the deserializer has the type info required to deserialize the dictionary.
I am unable to deserialize it correctly to the identical list of object, I only get JObjects in my container and not the original type. Can anyone help me accomplish this?
Thanks in advance;
Laurent
Update:
To get data in / out I use those two methods:
public static byte[] SerializeToByteArray<T>(T data)
{
byte[] serializedData = new byte[]{};
using(var stream = new MemoryStream())
{
using (BsonWriter writer = new BsonWriter(stream))
{
JsonSerializer serializer = new JsonSerializer();
serializer.TypeNameHandling = TypeNameHandling.All;
serializer.Serialize(writer, data);
}
return stream.ToArray();
}
}
public static T DeserializeFromByteArray<T>(byte[] serializedData )
{
using (var stream = new MemoryStream(serializedData))
{
using (BsonReader reader = new BsonReader(stream))
{
JsonSerializer serializer = new JsonSerializer();
return (T)serializer.Deserialize<T>( reader );
}
}
}
[System.Serializable]
public class FavoriteLevel
{
public FavoriteLevel(string ID, int TYPE) { id = ID; type = TYPE;}
public string id;
public int type;
}
Dictionary<string,object> dict = new Dictionary<string,object>(1);
List<FavoriteLevel> levels = new List<FavoriteLevel>(1);
levels.Add (new FavoriteLevel("123",FavoriteType.Favorite) );
dict.Add ( "123", levels );
byte[] data = SerializeToByteArray( dict );
Dictionary<string,object> incomingDict = DeserializeFromByteArray<Dictionary<string,object>>( data );
object listBack = incomingDict["123"];
// ERROR: listBack is a Json object and not a List<FavoriteLevel> object

You need to set serializer.TypeNameHandling = TypeNameHandling.All when deserializing as well as serializing. Otherwise, the "$type" property will be ignored.
Thus:
public static class JsonExtensions
{
public static byte[] SerializeToByteArray<T>(T data, JsonSerializerSettings settings)
{
using (var stream = new MemoryStream())
{
using (var writer = new BsonWriter(stream))
{
JsonSerializer serializer = JsonSerializer.Create(settings);
serializer.Serialize(writer, data);
}
return stream.ToArray();
}
}
public static T DeserializeFromByteArray<T>(byte[] serializedData, JsonSerializerSettings settings)
{
using (var stream = new MemoryStream(serializedData))
{
using (var reader = new BsonReader(stream))
{
JsonSerializer serializer = JsonSerializer.Create(settings);
return (T)serializer.Deserialize<T>(reader);
}
}
}
}
public static class TestClass
{
public static void Test()
{
Dictionary<string, object> dict = new Dictionary<string, object>(1);
List<FavoriteLevel> levels = new List<FavoriteLevel>(1);
levels.Add(new FavoriteLevel("123", 0));
dict.Add("123", levels);
var settings = new JsonSerializerSettings();
settings.TypeNameHandling = TypeNameHandling.All;
byte[] data = JsonExtensions.SerializeToByteArray(dict, settings);
Dictionary<string, object> incomingDict = JsonExtensions.DeserializeFromByteArray<Dictionary<string, object>>(data, settings);
object listBack = incomingDict["123"];
Debug.Assert(listBack.GetType() == levels.GetType()); // No assert.
}
}

RestSharp: Deserialize Xml to c# Object return null

i've an xml from some api :
<auditypes>
<auditype code="a" description="aaa"/>
<auditype code="b" description="bbb"/>
<auditype code="c" description="ccc"/>
<auditype code="d" description="ddd"/>
<auditype code="e" description="eee"/>
</auditypes>
and mapping as object in C# class :
public class auditypes
{
public List<auditype> auditype { get; set; }
}
public class auditype
{
public string code { get; set; }
public string description { get; set; }
}
I call it with this function :
public List<auditypes> Execute<auditypes>(RestRequest request) where auditypes : new()
{
var client = new RestClient();
client.BaseUrl = "https://www.myurl.com/auditypes";
var response = client.Execute<auditypes>(request);
return response.Data as List<auditypes>;
}
public List<auditypes> GetCall()
{
var request = new RestRequest();
request.RequestFormat = DataFormat.Xml;
request.RootElement = "auditype";
return Execute<auditypes>(request);
}
but it always return null, does anyone knows why is this happening?

The generic parameter passed to Execute<T> is the type that should be deserialized by the RestSharp library. That means that your response.Data property is already of type T, which in your case is auditypes. But when you return you try to cast it to a List<auditypes> where no such cast exists.
Also, there is no need for the type constraint, as your method is not generic as it accepts an explicit type.
Refactor your method:
public auditypes Execute<auditypes>(RestRequest request)
{
var client = new RestClient();
client.BaseUrl = "https://www.myurl.com/auditypes";
var response = client.Execute<auditypes>(request);
return response.Data;
}

Finally, it work for me :)
public auditypes Execute<auditypes>(RestRequest request) where auditypes : new()
{
var client = new RestClient();
client.BaseUrl = "https://www.myurl.com/auditypes";
var response = client.Execute<auditypes>(request).Data;
return response;
}
public auditypes GetCall()
{
var request = new RestRequest();
request.RequestFormat = DataFormat.Xml;
return Execute<auditypes>(request);
}