I've got a collection of around 20,000 objects that need to get persisted to my database. Now, instead of doing 20,000 insert statements, I want to pass all the records in using an XML parameter.
As far as serializing the object and passing it into the procedure goes, I'm all set. However, I'm wondering if anyone has an elegant way to do the following:
In our C# code base; we have some static values that represent a NULL when saved to the database. For example, if an integer equals -1, or a DateTime equals DateTime.MinValue; save NULL. We have our own little custom implementation that handles this for us when saving objects.
Is there any way I can do something similar to this when performing the XML serialization? Right now it's outputting -1 and DateTime.MinValue in the XML. I do have an extension method (IsNull()) that will return true/false if the value being saved is the null default value.
Any suggestions? Tips/Tricks?
The XmlSerializer understands a number of different attributes; one of them is DefaultValueAttribute.
When included, the XmlSerializer will only serialize the value check if the actual value differs from the default, so all you should need is:
[DefaultValue(-1)]
public int SomeProperty
{get;set;}
Also, if you haven't considered it, take a look at the SqlBulkCopy class, which is a highly-performant approach to sending a large number of records to SQL Server.
You can implement IXmlSerializable to control an object's XML serialization. In particular, implement WriteXml to substitute blank or xsi:null values (however you want to handle this) for those properties/fields that contain your null signifier values.
Related
I want to order a list of my 'SortObject' class. This class is meant to imitate a DataGrid Row by holding arbitrary data organized in a dictionary (named 'Cells'), where the key is analogous to a DataGrid Column. It is enforced that any given key is associated with only one data type, for example the key "Name" will only have values of the String type.
My code (below) actually works for a majority of the cases I've used:
// public Dictionary<string, object> Cells { get; set; } <-- relevant field from 'SortObject'
List<SortObject> sortObjects = GetSortObjects(); // This is simplified, the data has a few different sources
IOrderedEnumerable<SortObject> orderedList = sortObjects.OrderBy(p => p.Cells["ArbitraryKey"]);
SortObject firstObject = sortedList.First();
// other work with 'orderedList' follows
The problem occurs when I'm trying to OrderBy objects of the DateTime type, and some of those objects are not set and default to 'System.DBNull'. In this case an exception is thrown when calling 'sortedList.First()' or in any of the later references to 'sortedList'. The exception is simple: "Object must be of type DateTime", which seems to be a consequence of OrderBy trying to compare the type DateTime to the type DBNull.
I've tried two solutions that haven't worked so far:
Attempt One: Set DBNull to new DateTime. In theory this should work, but I would need to create not simply DateTime type objects, but objects of any arbitrary type on the fly. (I'd also need to take note of these SortObjects and set their data back to DBNull once I had the order correct; I can't be actually changing data after all).
Attempt Two: Organize just DBNull, then just DateTime, then slap them together. Again this might work in theory, but the "other work" mentioned in the code snippet is extensive, including reordering using ThenBy() an arbitrary number of times on any key(s). Doubling its complexity is not an elegant solution and I consider it a backup.
What is the best way to resolve this?
PS: For OrderBy and DateTime I'm using the Microsoft .NET Framework v4.6.2
Change the OrderBy statement to OrderBy(v => v is DBNull ? null : v)
OrderBy can handle nulls, but not dbNulls.
That code should work for all the data types
I'm trying to deserialize an unstructured JSON object (there are multiple schema possibilities) to a BsonDocument, and I'm trying to specify the correct types for some properties (which I know in advance).
Say for example (very simplified example):
{ "Id": "039665be-a1a8-4062-97d6-e44fea2affff", "Foo":"Bar", "Baz":30 }
So I know everytime I find an "Id" property (which may or may not be there) on the root of the object, this is to be converted to an UUID type (bson binary type 4).
I've made a simple JsonReader descendant, and I'm overriding ReadBsonType, and both returning a new CurrentBsonType and providing a converted value there, then overriding all methods for every possible type (ReadDateTime(), ReadInt32(), ReadInt64(), ReadBinaryData(), etc.) and providing a parsed value.
This works fine (albeit I find it a bit unconfortable) when the JSON object is plain, but if it has nested objects with properties of the same name (which I do not want to parse), then problems arise.
I've tried overriding ReadStartArray(), ReadStartDocument(), etc., and tried building a "path" which I can query to, but the actual order of the calling of the methods of the JsonReader are baffling to me (it seems to check for the type before checking for the name of the property, so the CurrentName property when checking for the type refers to the previous property, etc.).
I've somewhat circumvented it with very ugly code... and I'm sure there must be a better way to do this without class mapping, although finding documentation is proving hard (since mongo often calls "json" the actual "json extended", so documentation gets mixed from here and there).
Has anyone ever found themselves in such situation?
PS: before anyone asks, I'm storing data returned as json strings from a third party server on a mongo database (that will be mined later on), and while there are some schemas for the datatypes available (and I could classmap them), there might be new schemas in the future so I can't just classmap everything. Some properties (if they exist) are always the same though, so instead of storing everything on mongo as a string I'd rather give them the correct possible types from a start.
I've encountered a problem where a small number of data objects stored using a BinaryFormatter are coming back with parameters missing (null/default).
I'd like to know if the missing items were saved as null, or if the objects that were serialized were changed from the versions in source control and then reverted before a code commit (eg int numDogs vs unsigned int dogCount).
The former would represent a serious bug in the data validation code ran before the serialization was done; while the latter is just junk data in a test DB and ignorable.
Since the BinaryFormatter is able to get everything else out when a member is changed, added, or removed I assume it's writing objects in a form similar to a key value store. Is there any way to get a human readable representation of it, without having to try and guess the exact details of the object that was serialized?
If you implement ISerializable on your objects, you can have a look at what's been serialized by trying to deserialize.
You will need to add a constructor with the same signature as ISerializable.GetObjectData - this is where deserialization occurs.
I want to store list of parameters (that will define how document is going to be generated on the web page) in data base.
There is a number of item (or document) types, each type has a different set of parameters that vary (each type has it's own parameters).
Is it a good idea to store all parameters (key-value) as JSON in table's column?
Otherwise I would have to create Parameter Table for every Type and column for every parameter (10-30 params for every type).
A note: I am not going to search by parameters or something like that.
I will load the the JSON string (if I'll choose JSON), serialize it to Object and apply them on document as usual.
Since you have no requirement of searching by parameters, To me Json seems to be more robust because you will have Object ready with information when you Deserialize it. where as if you store it in columns and table you will have to initialize class members yourself. It will also have performance benifit as there will be only one column to fetch based on your document type.
Conclusion go with Json data in Database
Sounds like you should have a look at http://sisodb.com. However, it does support querying, but that is something you could turn-off and only rely on GetById.
First off, I am new to programming (especially with C#) and thanks for your help.
I have a static web form with about 150 form objects (most checkboxes). I decided to go 1 record per form submission in the sql db. So, for example, question X has a choice of 5 checkboxes. Each of these 5 checkboxes has a column in the db.
I have the post page complete(working) and am building an edit page where I load the record and then populate the form.
How I am doing this is by passing a stored proc the id and then putting all the returned column values into the according object properties, then setting the asp control object to them.
An example of setting the asp controls to the selected value:
questionX.Items[0].Selected = selectedForm.questionX0
questionX.Items[1].Selected = selectedForm.questionX1
questionX.Items[2].Selected = selectedForm.questionX2
As you see, this is very tiresome since there are over 150 of these to do. Also, I just found out if the response is NULL then I get the error that it cant be converted to a string. So, I have added this line of code to get past it:
This is the part where I am populating the returned column values into the object properties (entity is the object):
if (!String.IsNullOrEmpty((string)reader["questionX0"].ToString()))
{entity.patientUnderMdTreatment = (string)reader["questionX0"];}
So, instead of having to add this if then statement 150+ times. There must be a way to do this more efficiently.
First of all, it seems that you are using string.IsNullOrEmpty(value), but this won’t check for the special DBNull value that is returned from databases when the data is null. You should use something more akin to value is DBNull.
The rest of your problem sounds complex, so please don’t be put off if my answer is complex too. Personally I would use custom attributes:
Declare a custom attribute
The following is a skeleton to give you the idea. You may want to use the “Attribute” code snippet in Visual Studio to find out more about how to declare these.
[AttributeUsage(AttributeTargets.Field, AllowMultiple = false)]
public sealed class QuestionColumnAttribute : Attribute
{
public string ColumnName { get; private set; }
public QuestionColumnAttribute(string columnName)
{
ColumnName = columnName;
}
}
Use the custom attribute in the entity class
Where you declare your entity class, add this custom attribute to every field, for example where patientUnderMdTreatment is declared:
[QuestionColumn("questionX0")]
public string patientUnderMdTreatment;
Iterate over the fields
Instead of iterating over the columns in the reader, iterate over the fields. For each field that has a QuestionColumnAttribute on it, get the relevant column from the reader:
foreach (var field in entity.GetType().GetFields())
{
var attributes = field.GetCustomAttributes(typeof(QuestionColumnAttribute), true);
if (attributes.Length == 0)
continue;
object value = reader[attributes[0].ColumnName];
if (!(value is DBNull))
field.SetValue(entity, value.ToString());
}
For the first part of your question where you set the ASP controls, you can use a similar strategy iterating over the fields of selectedForm, and this is probably simpler because you don’t need a custom attribute — just take only the fields whose name starts with “questionX”.
this is a quick & easy way of doing it.. there are some suggestions to investigate LINQ, and I'd go with those first.
for (int i = 0; i < 150; i++)
{
if (!String.IsNullOrEmpty((string)reader["questionX" + i.ToString()].ToString()))
{entity.patientUnderMdTreatment = (string)reader["questionX" + i.ToString()];}
}
... though this wouldn't be any good with the
questionX.Items[0].Selected = selectedForm.questionX0
questionX.Items[1].Selected = selectedForm.questionX1
questionX.Items[2].Selected = selectedForm.questionX2
lines
so I hear two questions:
- how to deal with null coming from IDataReader?
- how to deal with multiple fields?
Lets start with simple one. Define yourself a helper method:
public static T IsDbNull<T>(object value, T defaultValue)
{
return (T)(value is DBNull ? defaultValue : value);
}
then use it:
entity.patientUnderMdTreatment = IsDbNull<string>(reader["question"], null);
Now how to map entity fields to the form? Well that really is up to you. You can either hardcode it or use reflection. The difference of runtime mapping vs compile-time is likely to be completely irrelevant for your case.
It helps if your form fields have identical names to ones in the DB, so you don't have to do name mapping on top of that (as in Timwi's post), but in the end you'll likely find out that you have to do validation/normalization on many of them anyway at which point hardcoding is really what you need, since there isn't a way to dynamically generate logic according to the changing spec. It doesn't matter if you'll have to rename 150 db fields or attach 150 attributes - in the end it is always a O(n) solution where n is number of fields.
I am still a little unsure why do you need to read data back. If you need to preserve user's input on form reload (due to validation error?) wouldn't it be easier/better to reload them from the request? Also are entity and selectedForm the same object type? I assume its not a db entity (otherwise why use reader at all?).
Its possible that there are some shortcuts you may take, but I am having hard time following what are you reading and writing and when.
I recommend using the NullableDataReader. It eliminates the issue.