At work, we have multiple telegrams to communicate with a customer. Those telegrams are always different with each customer, but the logic, how we work with them is always the same. The data is sent to a table on our schema with other data for example like sent time and telegram type. We split it into a view to simplify it for development and testing purpose.
Example JSON:
{"Prop1": "Test"}
Example view script (using Oracle Enterprise):
CREATE OR REPLACE VIEW test
AS
SELECT prop1
FROM telegram
, JSON_TABLE (
telegram.json,
'$' COLUMNS (
prop1 PATH '$.Prop1'
)
)
My problem is now, that I do not know the correct or good way to create those view scripts with C#. Currently, I use a constant string with placeholders ({0}, {1}, ...) but they are getting really hard to maintain. I also read about T4, but it is only useable with Visual Studio. AFAIK we plan to move to .NET 5 and it looks like T4 is not available anymore and/or you cannot call the created C# classes in the code without some errors. I heard about Roslyn, too. Is it possible to create .txt/.sql instead of .cs files?
I hope I got all the information needed to answer this question. If I forgot some information, tell me. Thanks in advance!
Is it possible to create .txt/.sql instead of .cs files?
Sure, you can add the .txt/.sql file to your project and set its build action to "Embedded Resource".
In your code, you can access the content of that resource with Assembly.GetManifestResourceStream or by linking it to a resource file. Afterwards, you can apply String.Replace or String.Format for placeholders, if required.
That having been said, you might want to consider storing your data as relational data in your database, instead of JSON text that needs to be parsed by a for Every. Single. Query. Not only will this allow you to get rid of your view, but it will improve performance and allow you to use indexes on relevant fields.
Related
Been thinking about this some time: Let's say I have a application where you can add and use reminders.
What is the best way to store this? In the past I've always used a textfile but it can get problematic if I later want to add another "field" to each reminder in the textfile. Let's say I add an feature for recurring reminders.
What is the most volatile way? Text? Xml? Json? SQLite?
Use a database. Adding another field is as simple as adding another column to a table.
MySQL is a solid query language and easy to pick up for beginners. When I started out, I watched (and really enjoyed) this tutorial series:
https://www.youtube.com/watch?v=6pbxQQG25Jw
If you ever make something that needs lots of scalability, you might want to look into PostgreSQL.
SQLite becomes a better option as your data model becomes more complex. The upgrade process (changing, adding, and removing tables) is a bit of work, and is required for your code to even refer to a new field in a query.
XML And JSON have the advantage of having parsers built into the standard libraries for most platforms these days, so you don't have to fix your parser every time you change your data model (as you would with plain text). XML can validate your model and let you know if the file does not comply with your model. JSON is really just a serialization protocol and doesn't provide anything in terms of model validation, which makes it the most flexible of the plain text options (IMO).
In terms of updating your model, your code should read in the file and allow for the new field to be missing or empty. If the field is mandatory, you should provide a default value and then write your model back out to the file so it's good to go the next time. This process is roughly the same for SQLite, but is just a bit more involved in terms of what you have to do to upgrade your model.
I'm a novice programmer. I am full of theoretical knowledge, but I'm behind with the practice. OK. I am trying to make a program for adding categories and descriptions to files. The language is C#, it should run on Windows 7...
1.The categories can contain sub-categories.
I don't want to call them "tags", because these are different. A category can be fx "favorites". But it can also be: "favorites->music->2013". You can create sub-categories, I will use a TreeView on a WinForm for all the operations a user can do with them.
QUESTION: Should I use XML file for the categories?
2.Every file CAN have a description and one or many categories. However:
Even if the file is deleted, I want to keep its description, so that it can be available for later usage.
Folders themselves will be omitted. The folders themselves cannot have nor categories, nor description. But the contained files YES.
I made a very simple SQL Server database containing one table: !http://img832.imageshack.us/img832/3931/finalprojectdb.png
QUESTION: Is this a good idea? Maybe the categories column is better to be of type XML ?
Any advice on what should the best approach in this situation be, is welcomed. Thanks in advance !
SQL is not great for getting nested data at once. You can store things in XML which gives you a lot of flexibility, but you also have to write a parser or deserializer for it. Nowadays people also just write a little Javascript class and use something like Newtonsoft to deserialize it automatically.
If you want a DB solution, you can use something like SQLite embedded in your application if you don't want to install a database separately.
XML is a great design for an app that needs to communicate cross platform (say c# to java), or cross internet, or cross network. But as a way to store data as a subset in a table, not really.
A normalized database is a terrific tool. It can be indexed (xml can not) this allows for rapid querying of data. If you de-normalize your data by embedding xml in a column querying it will be slow and updating / maintaining a pain.
I personally prefer foreign key tables.
At the moment I'm working on a quite tricky transferring from a .csv file to DB. I have to develop a package/solution/xxxyyy that handles a flow of data from this .csv file to my SQL Server DB (the .csv is updated with new data everyday).
The approach that my boss "suggested" I should use is through SSIS (normally I would have wrote some kind of "parser" to easily convoy the data from the .csv). The fact is that I have quite a bit of transformation to do.
i.e.
An employee has this fields:
name;surname;id;roles
The field "roles" is formatted like this:
role1,role2,role3
This relationship in my db is mapped in 3 different tables:
tblEmployee
PK_Emp | name | surname
tblRoles
PK_Role | roleName
tblEmployeeRole
PK_Emp | PK_Role
So, from the .csv I have to extract the roles of a single employee, insert those in tblRoles (checking that there's no duplicate). Then I have to manage the relationship in tblEmployeeRole.
Considering that this is just an example of one of the different transformations that I have to manage I was wondering if SSIS is the best tool to achieve my goal (loads if script components). When I explained my perplexities to my boss he came up with this "idea":
Use SSIS to transfer the data, as they are, in a temporary table then handle the different transformations through stored procedures.
From the very little I know about stored procedure, I'm not sure that I should follow this idea.
Now, considering that my actual superior isn't that enlightened project manager (he usually mess up our work with bizarre ideas) and considering the fact that I'm not such an expert neither in SSIS nor in stored procedure, I've decided to write here and see if anyone can explain me if one of the previous approaches is the right one or if I have to consider some other (better) solution.
Sorry for my poor English, ty for any help =)
I would insert the data from the CSV file as-is.
Then do any parsing in the database end. If this is something that has to be done often I would then take any scripts you have made to do this and create procedures/functions from that. This question is a bit grand-scheme so this is only a general solution. If you need help doing the parsing of the roles into the look up tables then that would be more specific and of better use.
In general when I work with massive flat-file data sets that need to be parsed into a SQL structure:
Import the data as-is
Find the commonalities among the look up codes
Create the base look up tables (in your case it would be tblRoles)
Create a script to insert into both tblEmployee and tblEmployee role
Once my test scenarios work then I worry about combining each component step into one monolithic SSIS or stored procedure.
I suggest something similar here. Break this import task into small pieces and worry about the grand design later. SSIS, procs, compiled code...any of these might work for you. You just need to know what you need it to do.
Depending upon your transformations they can all be done within SSIS. If you don't need to store the raw .csv data, I would stay away from stored procedures and temporary tables as you are bypassing a large portion of SSIS's strengths.
As an example, you can do look-ups on your incoming data to determine proper relationships and insert those results into multiple tables (your 3 in the example).
Looks like the task is very suitable for bcp utility or BULK INSERT command
I'm new to windows app and I would like to know what the best way to save a small amount of data, like 1 value a day.
I'm going for the text file because it's easy, but I know i could use MS Access.
Do you have other option ? Faster or better ?
Since you are already considering using a MS Access database, I would recommend using SQLite. Here's a quote from their site (SQLite Home Page):
SQLite is a software library that implements a self-contained, serverless, zero-configuration, transactional SQL database engine.
It is really very easy to use - no installations required, you simply need to reference a DLL.
If you need to read it then use a plain text file.
If you need to read the values back into the application then serialize to an XML or binary file by making your user data serializable possibly by having a List of values in your object.
How do you want to use the data? Do you just want to look at it once in awhile? Do you plan to analyze it in a spreadsheet? Etc. Based upon what you say so far, I would just use a text file, one value per line. Even if later you wanted to do more with it, it's easy to import into spreadsheets, etc. If the daily data is a little more complicated (maybe a couple of different values for things each day), you might consider something like YAML.
Why stray from the path? XML gives you the ability to expand on it later without having to rethink everything.
Its mainly dependent upon the complexity of the data that you want to store. If its just DateTime some other simple built in type you would be able to recreate that object as a strongly typed one easily. But in case if its more complicated I would suggest you to create a serializable class (link on how to create such class is here) and then use one of Binary or SOAP serializations based on the size, security and other such needs. I am suggesting this as it would be best to be able to recreate objects as strongly typed ones from a flat file rather than just trying to parse what's there in the flat file.
Please let me know in case you need more clarity.
Thanks,
Sai Pavan
does anyone know how of a tool where I can point to my sql server database and it reads the schema and generates c# interface classes from the tables?
for example - say I have a table called 'Customers' with a "Name" column, "Address" Column and a "Phone" column it would generate a ICustomer.cs file with string Name {get;set;} string Address {get;set;} and int Phone {get;set;}
I am using a 'incomplete' code generator and it does not generate these interfaces.
How about the Linq to Sql O/R Designer?
I don't know of a tool doing it, but I know for sure that you can do it yourself quite easily!
Set up a string containing the header of the class, and another with the footer.
Then, create a new text file named as your table.
Write the header into the file.
For the body, just write a loop reading your table, which extracts the names and types of the fields, and writes an interface with that info.
At the end, write the footer to the file.
There you go with your brand new interface!
(As interfaces are juste plain text files, it's really easy...)
You could use SQLMetal.exe to get part of the job done. IT sounds like you want an interface, but this will create concrete classes. It'd be a small task to find/replace class with interface, and modify the names.
C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\SqlMetal.exe or open a Visual Studio Command Prompt.
Usage: sqlmetal /server:myserver /database:myDB /user:myUser /pwd:myPwd /language:csharp /code:myDB.cs
More options over at the MSDN page for SQLMetal.
You need a tool that allows you to customize your code generation templates. Have you considered Enterprise Architect, or CodeSmith?
There are numerous others - you may even want to go for a Model Driven Architecture. Design your solution in UML and generate the database and the code from the UML model. You can use a combination of tools for this, for example MagicDraw and Maven.
I think I can use Resharpers 'Extract Interface' refactor as I have generated the class already.
Thanks for everyones input though
My SqlSharpener project lets you parse SQL files at design-time to create a meta-model which you can then use to generate any type of code you like in a T4 template. For example, you could create Entity Framework Code First entities.
You can also use MyGeneration