BsonSerializationException occurs if element name ends in a period - c#

I am writing a C# application that reads from an XML file, converts that to JSON, and uploads to MongoDB. Some of our tags are structured with a period at the end, like so:
<BatteryTest.>GOOD</BatteryTest.>
Using the Newtonsoft library I am able to convert the XML to JSON without a problem. It is when I go to deserialize it to a BsonDocument that I have trouble:
var document = MongoDB.Bson.Serialization.BsonSerializer.Deserialize<BsonDocument>(jsonText);
I get the following error message:
An exception of type 'MongoDB.Bson.BsonSerializationException' occurred in mscorlib.dll but was not handled in user code
Additional information: Element name 'BatteryTest.' is not valid'.
I have looked at the documentation but I haven't found anything that would explain how I can change the formatting properties of the deserializer. This is valid XML so I am not sure why the deserializer would choke on it, either.
Is this invalid JSON? If so, is there a way to still insert it into MongoDB without dropping that period?


As the the dot can be used in MongoDB queries, it can not be used in field names. You will have to preprocess the JSON before converting it into a BSONDocument.

Related

AWS Athena query on parquet data to return JSON output

I have follwoing setup.
Application sends serialized JSON data into Firehose.
Firehose is configured with Data conversion to praquet using a glue table definition for efficient query execution.
I am able to run query in Athena and see the results.
Now what I need is to create another application which can query Athena using AWSSDK (C#) and read the data back in JSON format.
Is it possible to somehow use the table's input/output format and serde to read the data back in JSON format using Athena SDK? Or I need to implement custom logic to convert the data back to JSON?

Question is old, but might help someone who trying to export results from Athena table into different output format. AWS CTAS can be used to export data into different formats (ORC, PARQUET, AVRO, JSON, or TEXTFILE) using simple Athena CTAS (https://docs.aws.amazon.com/athena/latest/ug/create-table-as.html) statement. You can also specify compression format for saving output data.
Below example would export data into JSON format on s3.
output_location='s3://s3_bucket/output.json'
CREATE TABLE output
WITH (
format = 'JSON',
external_location = output_location) AS SELECT * FROM target_table
Downside of this approach : Output is always compressed.

Athena bases on Presto version 0.172, so you will find the answer in Presto documentation.For instance, this chapter can be useful to you:
https://prestodb.io/docs/current/functions/json.html

So I get something like this from Athena:
SELECT output FROM "test"."prod" limit 10;
{score=41, avg_time_solving={double=17439.333333333332, long=null}
If I use cast for this field I get:
SELECT CAST(output AS JSON) FROM "test"."prod" limit 10;
[51, [8205.57142857143,null], [null,5159], 7, [0.8571428571428571,null], null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null]

Create XElement from invalid or partial XML String

I am working with a vendor supplied application that stores XML data as byte array in a SQL database table. I have found if the XML data is "too long" (meaning by a possible predetermined length in black box code provided by the vendor) the XML is truncated and a second record, containing the remainder of the XML data, is created.
My task is to take these "linked" records and merge them into one valid XML string. These linked records can be broken off anywhere, in the middle of an element, node, etc. There is no rhyme or reason to where the XML string is broken.
Taking the invalid XML data and loading it into an XElement causes an error "Tag has no closing tag".
I've also tried using an XmlReader and reading through each Node, based on this article as well as this msdn article. They also result in the above missing tag error.
Is there a way to take these partial xml strings and merge them? Or am I simply stuck?
The vendor application we use does perform this merge, but that code is hidden from me.
Thank you

How to read an xaml file and get the data I want in C#

I want to read through a xaml file, and find all the lines with 'Annotation.AnnotationText' and get specific data from that line.For example, this line:
<prwab:Branch Condition="{x:Null}" sap2010:Annotation.AnnotationText="testing information " ContinuouslyExecute="False" CreatedBy="System Administrator" CreatedOn="2013-02-23T14:51:28.1555955-05:00" DisplayName="Failure" EnableValidationRule="False" sap:VirtualizedContainerService.HintSize="160,234" ID="ab91dec8-1976-491e-91eb-58e073a69d16" IsReportable="False" LastModifiedBy="System Administrator" LastModifiedOn="2013-02-23T14:51:28.1555955-05:00" MediaRecord="[MediaRecord]" SystemName="CollectDigitsActivity1 Failure6" Timeout="10000" Type="Voice">
I want find all the lines with 'AnnotationText' in my xaml file, and get information like text = 'testing information', id = 'ab91dec8-1976-491e-91eb-58e073a69d16' , created date and lastmodified date.
I have 0 knowledge in this area and I don't know where to start and which method should I use. Thanks for helping!

XAML is just a specific flavour of XML. You will need to use XML parsing to read the file into an object that you can process in this manner. I recommend Linq to XML for this (look at XDocument class to get started), specifically as finding values by XName using a specific namespace as you will need to for the "sap2010" namespace is very easy.
You can then easily parse and extract the information you are looking for using those classes.

problem with conversion of JSOn string to Mongo document

I am trying to convert a JSOn string into a Mongo document and there is not much help availabel online.
the only helpful thing that I found was:
1:
2:
MongoDB.Bson.BsonDocument doc4=MongoDB.Bson.Serialization
.BsonSerializer.Deserialize(genericjson);
Toggle HighlightingOpen in New WindowSelect All
as described in the post #
Convert string into MongoDB BsonDocument
It creates the document in the databse but it's not the simulation of what it is here in C#. When I click on the nodes they don't show me the data inside. this is what Mongo does when the conversion has corrupted the file.
is there any other way to solve it?

Try the BsonDocument.Parse() method, e.g.
var bsonDoc = BsonDocument.Parse(jsonString);

C# Yaml Processing

I am using the C# Yaml Parser mentioned on Code Project Site
If my Yaml looks like the following
- id: tagid
tag:
- name: tagname
value: tagvalue
After it has been successfully parsed, how do I access the Data Items so that I can do some further processing. For example if I need to get the value of "name" what code would I need to write, the document says use doc.Root but can't find any examples on how to use it.

I figured out how to use doc.Root
The doc.Root returns a Mapping after it has parsed the YAML input. I recursively parse that Mapping to check if any of the Data Items are Mapping or Sequence and then act accordingly.
It's a bit naive method but so long as it works.
Many Thanks

Categories

Resources