I need to retrieve children from a notion block (page). I use C# with Notion.Net library. There's a database, with other pages at rows. I'm trying to retrieve such page's children with
await notion.Blocks.RetrieveChildrenAsync(id);
It causes an exception:
Newtonsoft.Json.JsonSerializationException: Could not create an instance of type Notion.Client.IBlock. Type is an interface or abstract class and cannot be instantiated. Path 'object', line 1, position 68129.
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateNewObject(JsonReader reader, JsonObjectContract objectContract, JsonProperty containerMember, JsonProperty containerProperty, String id, Boolean& createdFromNonDefaultCreator)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
at Newtonsoft.Json.Serialization.JsonSerializerProxy.DeserializeInternal(JsonReader reader, Type objectType)
at Newtonsoft.Json.JsonSerializer.Deserialize(JsonReader reader, Type objectType)
at JsonSubTypes.JsonSubtypes.ThreadStaticReadObject(JsonReader reader, JsonSerializer serializer, JToken jToken, Type targetType)
at JsonSubTypes.JsonSubtypes.ReadObject(JsonReader reader, Type objectType, JsonSerializer serializer)
at JsonSubTypes.JsonSubtypes.ReadJson(JsonReader reader, Type objectType, JsonSerializer serializer)
at JsonSubTypes.JsonSubtypes.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateList(IList list, JsonReader reader, JsonArrayContract contract, JsonProperty containerProperty, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, Object existingValue, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
at Newtonsoft.Json.JsonSerializer.Deserialize(JsonReader reader, Type objectType)
at Newtonsoft.Json.JsonSerializer.Deserialize[T](JsonReader reader)
at Notion.Client.Extensions.HttpResponseMessageExtensions.ParseStreamAsync[T](HttpResponseMessage response, JsonSerializerSettings serializerSettings)
at Notion.Client.RestClient.GetAsync[T](String uri, IDictionary`2 queryParams, IDictionary`2 headers, JsonSerializerSettings serializerSettings, CancellationToken cancellationToken)
at Notion.Client.BlocksClient.RetrieveChildrenAsync(String blockId, BlocksRetrieveChildrenParameters parameters)
I already used other notion library with python. Even now It successfully retrieves block children with the same Id. I don't know, how to solve It(
I expected to get all page block children. It seems, that some serialization errors occured. I can't find difference between cases.
I have successfully set up, configured, and run the MassTransit (v8.0.3) JobConsumers sample and am now trying to customize to my project. After modifying, I'm getting deserialization issues when submitting my job:
Newtonsoft.Json.JsonSerializationException: Cannot deserialize the current JSON array (e.g. [1,2,3]) into type 'MyNamespace.Foo' because the type requires a JSON object (e.g. {"name":"value"}) to deserialize correctly.
To fix this error either change the JSON to a JSON object (e.g. {"name":"value"}) or change the deserialized type to an array or a type that implements a collection interface (e.g. ICollection, IList) like List<T> that can be deserialized from a JSON array. JsonArrayAttribute can also be added to the type to force it to deserialize from a JSON array.
Path 'foos[0]'.
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.EnsureArrayContract(JsonReader reader, Type objectType, JsonContract contract)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, Object existingValue, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateList(IList list, JsonReader reader, JsonArrayContract contract, JsonProperty containerProperty, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, Object existingValue, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
at Newtonsoft.Json.Serialization.JsonSerializerProxy.DeserializeInternal(JsonReader reader, Type objectType)
at MassTransit.Serialization.JsonConverters.InterfaceProxyConverter.CachedConverter`1.MassTransit.Serialization.JsonConverters.BaseJsonConverter.IConverter.Deserialize(JsonReader reader, Type objectType, JsonSerializer serializer) in /_/src/MassTransit.Newtonsoft/Serialization/JsonConverters/InterfaceProxyConverter.cs:line 31
at MassTransit.Serialization.JsonConverters.BaseJsonConverter.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer) in /_/src/MassTransit.Newtonsoft/Serialization/JsonConverters/BaseJsonConverter.cs:line 15
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
at Newtonsoft.Json.JsonSerializer.Deserialize[T](JsonReader reader)
at MassTransit.Serialization.NewtonsoftObjectDeserializer.DeserializeObject[T](Object value, T defaultValue) in /_/src/MassTransit.Newtonsoft/Serialization/NewtonsoftObjectDeserializer.cs:line 34
at MassTransit.Serialization.BaseSerializerContext.DeserializeObject[T](Object value, T defaultValue) in /_/src/MassTransit/Serialization/EnvelopeSerializerContext.cs:line 52
at MassTransit.JobServiceEventExtensions.GetJob[TJob](ConsumeContext`1 context) in /_/src/MassTransit/JobServiceEventExtensions.cs:line 24
at MassTransit.JobService.FinalizeJobConsumer`1.Consume(ConsumeContext`1 context) in /_/src/MassTransit/JobService/JobService/FinalizeJobConsumer.cs:line 45
at MassTransit.Middleware.MethodConsumerMessageFilter`2.MassTransit.IFilter<MassTransit.ConsumerConsumeContext<TConsumer,TMessage>>.Send(ConsumerConsumeContext`2 context, IPipe`1 next) in /_/src/MassTransit/Middleware/MethodConsumerMessageFilter.cs:line 27
at MassTransit.Middleware.LastPipe`1.Send(TContext context) in /_/src/MassTransit.Abstractions/Middleware/Middleware/LastPipe.cs:line 30
at MassTransit.Consumer.DelegateConsumerFactory`1.Send[TMessage](ConsumeContext`1 context, IPipe`1 next) in /_/src/MassTransit/Consumers/Consumer/DelegateConsumerFactory.cs:line 29
at MassTransit.Consumer.DelegateConsumerFactory`1.Send[TMessage](ConsumeContext`1 context, IPipe`1 next) in /_/src/MassTransit/Consumers/Consumer/DelegateConsumerFactory.cs:line 39
at MassTransit.Middleware.ConsumerMessageFilter`2.MassTransit.IFilter<MassTransit.ConsumeContext<TMessage>>.Send(ConsumeContext`1 context, IPipe`1 next) in /_/src/MassTransit/Middleware/ConsumerMessageFilter.cs:line 46
I am using MassTransit.Newtonsoft for JSON support because I use it in my project and due to issues encountered when upgrading to MassTransit v8. It is configured here:
x.UsingRabbitMq((context, cfg) =>
{
cfg.UseNewtonsoftJsonSerializer();
cfg.UseNewtonsoftJsonDeserializer();
...
}
Here is my message definition (that replaces the sample's ConvertVideo), stripped down to the smallest failing case:
public interface ProcessFoos
{
List<Foo> Foos { get; }
}
where Foo is defined as:
public class Foo
{
public string FooID { get; set; }
}
I'm initiating a new job from a different consumer by calling:
var foos = new List<Foo>
{
new Foo { FooID = "foo" }
};
Response<JobSubmissionAccepted> response = await _processFoosClient.GetResponse<JobSubmissionAccepted>(new
{
Foos = foos
});
My job consumer looks like this:
public class ProcessFoosJobConsumer : IJobConsumer<ProcessFoos>
{
public async Task Run(JobContext<ProcessFoos> context)
{
// Never gets here due to the deserialization issue
var processFoos = context.Job;
...
}
}
Based on the MassTransit docs and everything I've found while researching, it seems like this should be supported and I'm probably just missing something simple.
When using Entity Framework with the built-in JobServiceDbContext, the Job property in the JobSaga table is serialized to JSON using System.Text.Json – which is not compatible with the serialization of Newtonsoft when it comes to nested object (the job message type is converted to a Dictionary<string,object>).
I don't know of a current fix at this point, since there isn't a way to dynamically specify the serializer based upon the incoming message types for the DbContext.
Or you could create your own mappings, and your own DbContext, with the Newtonsoft serializer methods instead of the ones used in v8.
Your best bet when using job consumers and the job service sagas with Cosmos or EF is to use System.Text.Json.
I'm trying to read data from a node with elastic search but I'm getting this error whenever I try to. Is anyone familiar with this problem? The times when it passes, the response result is empty and contains no data. Have been trying to many possible ways and none of them seem to work, and this method has always been working until now.
Elasticsearch.Net.Exceptions.MaxRetryException
HResult=0x80131500
Message=Failed after retrying 2 times: 'POST dev-news/newsarticle/_search'.
InnerException: JsonSerializationException, InnerMessage: Cannot create and populate list type Microsoft.FSharp.Collections.FSharpList`1[Montel.Databus.Messages.News.Entities.Journalist]. Path 'hits.hits[0]._source.journalists', line 7, position 19., InnerStackTrace: at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateNewList(JsonReader reader, JsonArrayContract contract, Boolean& createdFromNonDefaultCreator)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, Object existingValue, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.ResolvePropertyAndCreatorValues(JsonObjectContract contract, JsonProperty containerProperty, JsonReader reader, Type objectType, IDictionary`2& extensionData)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObjectUsingCreatorWithParameters(JsonReader reader, JsonObjectContract contract, JsonProperty containerProperty, ObjectConstructor`1 creator, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateNewObject(JsonReader reader, JsonObjectContract objectContract, JsonProperty containerMember, JsonProperty containerProperty, String id, Boolean& createdFromNonDefaultCreator)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Populate(JsonReader reader, Object target)
at Newtonsoft.Json.Serialization.JsonSerializerProxy.PopulateInternal(JsonReader reader, Object target)
at Nest.ConcreteTypeConverter`1.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer) in c:\code\elasticsearch-net\src\Nest\Resolvers\Converters\ConcreteTypeConverter.cs:line 90
at Nest.DefaultHitConverter.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer) in c:\code\elasticsearch-net\src\Nest\Resolvers\Converters\ConcreteTypeConverter.cs:line 42
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateList(IList list, JsonReader reader, JsonArrayContract contract, JsonProperty containerProperty, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, Object existingValue, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
at Nest.NestSerializer.DeserializeUsingSettings[T](Stream stream, JsonSerializerSettings settings) in c:\code\elasticsearch-net\src\Nest\ExposedInternals\NestSerializer.cs:line 105
at Nest.NestSerializer.Deserialize[T](Stream stream) in c:\code\elasticsearch-net\src\Nest\ExposedInternals\NestSerializer.cs:line 85
at Nest.NestSerializer.DeserializeInternal[T](Stream stream, JsonConverter converter) in c:\code\elasticsearch-net\src\Nest\ExposedInternals\NestSerializer.cs:line 94
at Nest.ElasticClient.FieldsSearchDeserializer[T,TResult](IElasticsearchResponse response, Stream stream, ISearchRequest d) in c:\code\elasticsearch-net\src\Nest\ElasticClient-Search.cs:line 124
at Nest.ElasticClient.<>c__DisplayClass513_0`2.<CreateSearchDeserializer>b__0(IElasticsearchResponse r, Stream s) in c:\code\elasticsearch-net\src\Nest\ElasticClient-Search.cs:line 136
at Elasticsearch.Net.Connection.RequestHandlers.RequestHandler.StreamToTypedResponse[T](ElasticsearchResponse`1 streamResponse, ITransportRequestState requestState, Byte[] readBytes) in c:\code\elasticsearch-net\src\Elasticsearch.Net\Connection\RequestHandlers\RequestHandler.cs:line 248
at Elasticsearch.Net.Connection.RequestHandlers.RequestHandler.ReturnTypedResponse[T](TransportRequestState`1 requestState, ElasticsearchResponse`1 streamResponse, ElasticsearchServerError& error) in c:\code\elasticsearch-net\src\Elasticsearch.Net\Connection\RequestHandlers\RequestHandler.cs:line 109
at Elasticsearch.Net.Connection.RequestHandlers.RequestHandler.CoordinateRequest[T](TransportRequestState`1 requestState, Int32 maxRetries, Int32 retried, Boolean& aliveResponse) in c:\code\elasticsearch-net\src\Elasticsearch.Net\Connection\RequestHandlers\RequestHandler.cs:line 134
at Elasticsearch.Net.Connection.RequestHandlers.RequestHandler.DoRequest[T](TransportRequestState`1 requestState) in c:\code\elasticsearch-net\src\Elasticsearch.Net\Connection\RequestHandlers\RequestHandler.cs:line 155
InnerException: JsonSerializationException, InnerMessage: Cannot create and populate list type Microsoft.FSharp.Collections.FSharpList`1[Montel.Databus.Messages.News.Entities.Journalist]. Path 'hits.hits[0]._source.journalists', line 7, position 19., InnerStackTrace: at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateNewList(JsonReader reader, JsonArrayContract contract, Boolean& createdFromNonDefaultCreator)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, Object existingValue, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.ResolvePropertyAndCreatorValues(JsonObjectContract contract, JsonProperty containerProperty, JsonReader reader, Type objectType, IDictionary`2& extensionData)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObjectUsingCreatorWithParameters(JsonReader reader, JsonObjectContract contract, JsonProperty containerProperty, ObjectConstructor`1 creator, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateNewObject(JsonReader reader, JsonObjectContract objectContract, JsonProperty containerMember, JsonProperty containerProperty, String id, Boolean& createdFromNonDefaultCreator)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Populate(JsonReader reader, Object target)
at Newtonsoft.Json.Serialization.JsonSerializerProxy.PopulateInternal(JsonReader reader, Object target)
at Nest.ConcreteTypeConverter`1.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer) in c:\code\elasticsearch-net\src\Nest\Resolvers\Converters\ConcreteTypeConverter.cs:line 90
at Nest.DefaultHitConverter.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer) in c:\code\elasticsearch-net\src\Nest\Resolvers\Converters\ConcreteTypeConverter.cs:line 36
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateList(IList list, JsonReader reader, JsonArrayContract contract, JsonProperty containerProperty, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, Object existingValue, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
at Nest.NestSerializer.DeserializeUsingSettings[T](Stream stream, JsonSerializerSettings settings) in c:\code\elasticsearch-net\src\Nest\ExposedInternals\NestSerializer.cs:line 105
at Nest.NestSerializer.Deserialize[T](Stream stream) in c:\code\elasticsearch-net\src\Nest\ExposedInternals\NestSerializer.cs:line 85
at Nest.NestSerializer.DeserializeInternal[T](Stream stream, JsonConverter converter) in c:\code\elasticsearch-net\src\Nest\ExposedInternals\NestSerializer.cs:line 94
at Nest.ElasticClient.FieldsSearchDeserializer[T,TResult](IElasticsearchResponse response, Stream stream, ISearchRequest d) in c:\code\elasticsearch-net\src\Nest\ElasticClient-Search.cs:line 124
at Nest.ElasticClient.<>c__DisplayClass513_0`2.<CreateSearchDeserializer>b__0(IElasticsearchResponse r, Stream s) in c:\code\elasticsearch-net\src\Nest\ElasticClient-Search.cs:line 136
at Elasticsearch.Net.Connection.RequestHandlers.RequestHandler.StreamToTypedResponse[T](ElasticsearchResponse`1 streamResponse, ITransportRequestState requestState, Byte[] readBytes) in c:\code\elasticsearch-net\src\Elasticsearch.Net\Connection\RequestHandlers\RequestHandler.cs:line 248
at Elasticsearch.Net.Connection.RequestHandlers.RequestHandler.ReturnTypedResponse[T](TransportRequestState`1 requestState, ElasticsearchResponse`1 streamResponse, ElasticsearchServerError& error) in c:\code\elasticsearch-net\src\Elasticsearch.Net\Connection\RequestHandlers\RequestHandler.cs:line 109
at Elasticsearch.Net.Connection.RequestHandlers.RequestHandler.CoordinateRequest[T](TransportRequestState`1 requestState, Int32 maxRetries, Int32 retried, Boolean& aliveResponse) in c:\code\elasticsearch-net\src\Elasticsearch.Net\Connection\RequestHandlers\RequestHandler.cs:line 134
at Elasticsearch.Net.Connection.RequestHandlers.RequestHandler.DoRequest[T](TransportRequestState`1 requestState) in c:\code\elasticsearch-net\src\Elasticsearch.Net\Connection\RequestHandlers\RequestHandler.cs:line 155
InnerException: JsonSerializationException, InnerMessage: Cannot create and populate list type Microsoft.FSharp.Collections.FSharpList`1[Montel.Databus.Messages.News.Entities.Journalist]. Path 'hits.hits[0]._source.journalists', line 7, position 19., InnerStackTrace: at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateNewList(JsonReader reader, JsonArrayContract contract, Boolean& createdFromNonDefaultCreator)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, Object existingValue, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.ResolvePropertyAndCreatorValues(JsonObjectContract contract, JsonProperty containerProperty, JsonReader reader, Type objectType, IDictionary`2& extensionData)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObjectUsingCreatorWithParameters(JsonReader reader, JsonObjectContract contract, JsonProperty containerProperty, ObjectConstructor`1 creator, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateNewObject(JsonReader reader, JsonObjectContract objectContract, JsonProperty containerMember, JsonProperty containerProperty, String id, Boolean& createdFromNonDefaultCreator)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Populate(JsonReader reader, Object target)
at Newtonsoft.Json.Serialization.JsonSerializerProxy.PopulateInternal(JsonReader reader, Object target)
at Nest.ConcreteTypeConverter`1.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer) in c:\code\elasticsearch-net\src\Nest\Resolvers\Converters\ConcreteTypeConverter.cs:line 90
at Nest.DefaultHitConverter.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer) in c:\code\elasticsearch-net\src\Nest\Resolvers\Converters\ConcreteTypeConverter.cs:line 36
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateList(IList list, JsonReader reader, JsonArrayContract contract, JsonProperty containerProperty, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, Object existingValue, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
at Nest.NestSerializer.DeserializeUsingSettings[T](Stream stream, JsonSerializerSettings settings) in c:\code\elasticsearch-net\src\Nest\ExposedInternals\NestSerializer.cs:line 105
at Nest.NestSerializer.Deserialize[T](Stream stream) in c:\code\elasticsearch-net\src\Nest\ExposedInternals\NestSerializer.cs:line 85
at Nest.NestSerializer.DeserializeInternal[T](Stream stream, JsonConverter converter) in c:\code\elasticsearch-net\src\Nest\ExposedInternals\NestSerializer.cs:line 94
at Nest.ElasticClient.FieldsSearchDeserializer[T,TResult](IElasticsearchResponse response, Stream stream, ISearchRequest d) in c:\code\elasticsearch-net\src\Nest\ElasticClient-Search.cs:line 124
at Nest.ElasticClient.<>c__DisplayClass513_0`2.<CreateSearchDeserializer>b__0(IElasticsearchResponse r, Stream s) in c:\code\elasticsearch-net\src\Nest\ElasticClient-Search.cs:line 136
at Elasticsearch.Net.Connection.RequestHandlers.RequestHandler.StreamToTypedResponse[T](ElasticsearchResponse`1 streamResponse, ITransportRequestState requestState, Byte[] readBytes) in c:\code\elasticsearch-net\src\Elasticsearch.Net\Connection\RequestHandlers\RequestHandler.cs:line 248
at Elasticsearch.Net.Connection.RequestHandlers.RequestHandler.ReturnTypedResponse[T](TransportRequestState`1 requestState, ElasticsearchResponse`1 streamResponse, ElasticsearchServerError& error) in c:\code\elasticsearch-net\src\Elasticsearch.Net\Connection\RequestHandlers\RequestHandler.cs:line 109
at Elasticsearch.Net.Connection.RequestHandlers.RequestHandler.CoordinateRequest[T](TransportRequestState`1 requestState, Int32 maxRetries, Int32 retried, Boolean& aliveResponse) in c:\code\elasticsearch-net\src\Elasticsearch.Net\Connection\RequestHandlers\RequestHandler.cs:line 134
at Elasticsearch.Net.Connection.RequestHandlers.RequestHandler.DoRequest[T](TransportRequestState`1 requestState) in c:\code\elasticsearch-net\src\Elasticsearch.Net\Connection\RequestHandlers\RequestHandler.cs:line 155
Source=Elasticsearch.Net
StackTrace:
at Elasticsearch.Net.Connection.RequestHandlers.RequestHandlerBase.ThrowMaxRetryExceptionWhenNeeded[T](TransportRequestState`1 requestState, Int32 maxRetries)
at Elasticsearch.Net.Connection.RequestHandlers.RequestHandler.RetryRequest[T](TransportRequestState`1 requestState)
at Elasticsearch.Net.Connection.RequestHandlers.RequestHandler.DoRequest[T](TransportRequestState`1 requestState)
at Elasticsearch.Net.Connection.RequestHandlers.RequestHandler.RetryRequest[T](TransportRequestState`1 requestState)
at Elasticsearch.Net.Connection.RequestHandlers.RequestHandler.DoRequest[T](TransportRequestState`1 requestState)
at Elasticsearch.Net.Connection.RequestHandlers.RequestHandler.RetryRequest[T](TransportRequestState`1 requestState)
at Elasticsearch.Net.Connection.RequestHandlers.RequestHandler.DoRequest[T](TransportRequestState`1 requestState)
at Elasticsearch.Net.Connection.RequestHandlers.RequestHandler.Request[T](TransportRequestState`1 requestState, Object data)
at Elasticsearch.Net.Connection.Transport.DoRequest[T](String method, String path, Object data, IRequestParameters requestParameters)
at Elasticsearch.Net.ElasticsearchClient.DoRequest[T](String method, String path, Object data, IRequestParameters requestParameters)
at Elasticsearch.Net.ElasticsearchClient.Search[T](String index, String type, Object body, Func`2 requestParameters)
at Nest.RawDispatch.SearchDispatch[T](ElasticsearchPathInfo`1 pathInfo, Object body)
at Nest.ElasticClient.Search[T,TResult](Func`2 searchSelector)
at Nest.ElasticClient.Search[T](Func`2 searchSelector)
at Insider2008.WebApi.NewsController.SearchNewsArchive(NewsSearchDto search) in C:\Projects\MontelOnline\Insider2008\WebApi\NewsController.cs:line 125
at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.<>c__DisplayClass10.<GetExecutor>b__9(Object instance, Object[] methodParameters)
at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.Execute(Object instance, Object[] arguments)
at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ExecuteAsync(HttpControllerContext controllerContext, IDictionary`2 arguments, CancellationToken cancellationToken)
Inner Exception 1:
AggregateException: One or more errors occurred.
Inner Exception 2:
JsonSerializationException: Cannot create and populate list type Microsoft.FSharp.Collections.FSharpList`1[Montel.Databus.Messages.News.Entities.Journalist]. Path 'hits.hits[0]._source.journalists', line 7, position 19.
Check your results from hits.hits[0]._source.journalists. I think they are incorrect JSON format
All of this is origianlly from https://github.com/JamesNK/Newtonsoft.Json/issues/469
Posted here because I first looked on SO and didn't see anything, so I posted on the project's GitHub page.
We're currently using JsonConvert.SerializeObject and JsonConvert.DeserializeObject<T> to send data between a client and server.
I created a tool that creates 10 clients, sends a command to 10 different servers, the servers serialize a response and send it back over the network, then the 10 clients deserialize the object on the local machine.
I'm running these 10 tasks concurrently in the thread pool, and about 20% of the time all of the JsonConvert.DeserializeObject calls fail with the following stack trace:
Error: Index was outside the bounds of the array.
at System.Collections.Generic.List1.Add(T item)
at System.Collections.Generic.List1.System.Collections.IList.Add(Object item)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateList(IList list, JsonReader reader, JsonArrayContract contract, JsonProperty containerProperty, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, Object existingValue, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings)
at Newtonsoft.Json.JsonConvert.DeserializeObject[T](String value, JsonSerializerSettings settings)
at MyClientCode()
MyClientCode() is using DeserializeObject like this:
string json = GetServerResponse();
return JsonConvert.DeserializeObject<ResponseObject>(json);
The ResponseObject is pretty big, and includes several composite objects. However, I can save the json and deserialize it properly with DeserializeObject, so I don't think the object structure is the problem.
Doing some research of the List error indicates that this occurs when trying to modify a List object concurrently.
From James Newton-King:
A new JsonSerializerInternalReader is created each time you
deserialize an object. Each deserialization happens in its own state.
A high volume server that it deserializing incoming JSON will be
deserializing many many things at the same time without issue.
My guess is you have multiple deserializers working over the same
list.
Thanks James. After digging deeper I found that you're right, we were using the same list object for multiple instances of the deserialized type. Specifically, the object looked something like this:
class Obj {
static List<string> _validSelections = new List<string>() { "One", "Two", "Three", "Four" };
public IEnumerable<string> ValidSelections { get { return _validSelections; } }
... more ...
}
The exception was being thrown on line 1261 of JsonSerializerInternalReader.cs when trying to add objects to the list concurrently.
After seeing how this is implemented in our code, I am going to get rid of the static backing since it wasn't providing us anything anyway.