I have an object in my SQL Server database that is called dbo.stringlist
CREATE TYPE [dbo].[StringList] AS TABLE
(
[Item1] [varchar](2000) NULL
)
I have a stored procedure that has syntax in SSMS like
exec [dbo].[Getdata] #Fields
(which is the dbo.stringlist) which effectively just joins on a table in the stored procedure and acts as a long list of criteria. It's how I want to deal with the request on the database side, so I won't be changing that.
All I want to do is execute this in C# (hopefully) using Dapper, because the rest of the goal has been built out using nothing but Dapper. I can scrap it if need be. The goal is to effectively write an ETL of sorts, but instead of endpoint-to-endpoint SSIS-like packages, I am just automating deploy scripts for new builds.
If there is a way to do this outside of Dapper that is easier, then I am all for scrapping this at this point. Like I said, the documentation for this is sparse, at least the updated documentation for this specific task is sparse.
I tried this method - https://gist.github.com/taylorkj/9012616
And also tried this method - https://bornsql.ca/blog/table-valued-parameters-and-dapper-in-net-core/
The second one didn't work, because I just want to pass the list in, but that's not an option. It needs to be an IEnumerable, which I don't feel like I can pass values into like a list. But maybe I just don't know what I am doing when it comes to that interface.
So if anyone has an UP-TO-Date link to help me out that would be stellar. I have a feeling I am going to have to scrap everything I have done, which is fine, I guess. I just want the tool to work. I know this post doesn't have a lot of information, that's why I am just asking for ideas to accomplish what I am trying to do in C# and hopefully Dapper, but like I said, I am not married to it.
I got it to work with the second link.
private static IEnumerable<SqlDataRecord> CreateSqlDataRecord(IEnumerable<string> list)
{
var metaData = new SqlMetaData("Item1", SqlDbType.VarChar, 2000);
var record = new SqlDataRecord(metaData);
foreach (var item in list)
{
record.SetSqlString(0, item);
yield return record;
}
}
public static SqlMapper.ICustomQueryParameter GetTableValuedParameter()
{
string[] list = new string[] { };
return CreateSqlDataRecord(list).AsTableValuedParameter("dbo.StringList");
}
I wasn't updating the parameter nor was I including a varchar data length of 255.
This is a contrived example however I have simplified it for ease of explanation.
Please see my update at the bottom before investing too much of your
time!
Background
I have some (a lot of) code that ordinarily queries my DB as follows:
SELECT name FROM sites where IsLive=1;
My challenge is to, under certain conditions, return the full list of sites, essentially
SELECT name from sites;
I do not wish to modify the actual C# code issuing the SQL (although I can do if I have to in order to achieve my goal which is purely for demonstration purposes).
Therefore in order to leave as much untouched as possible my thoughts are to insert a database-proxy-view called site that returns the data dependent on a control variable
Method
Rename existing site table to site_table
Create a new view named site that the C# code now unknowingly targets and which returns the (possibly filtered) details from site_table according to the control variable value (Note a limitation on variables in views meant I had to create a function in order to demonstrate this - see http://dev.mysql.com/doc/refman/5.7/en/create-view.html wrt error 1351)
Changes made
ALTER TABLE site RENAME TO site_table;
CREATE FUNCTION controlVariableFn() RETURNS VARCHAR(16) RETURN #controlVariable;
CREATE OR REPLACE VIEW site AS SELECT * from site_table WHERE (IsLive = 1 OR controlVariableFn() = 'SHOWALL');
The above statements are ugly but achieve the result I want, however my problem is to dynamically pass through controlVariable without changing my main SQL queries being sent.
My Question
Is there a way to (ideally as I am creating my connection object) define the controlVariable outside the actual SQL to be executed but which the View can still access similar to the above as though it had been supplied as a regular user variable parameter to the query?
so the code would look something like
var connectionString = "Server=localhost;User ID=un;Password=pw;Database=dbname;....";
DbConnection db = new MySql.Data.MySqlClient.MySqlConnection
(connectionString, "controlVariable=SHOWALL");
var results = db.Query<Site>("SELECT * FROM site;");
(I understand that this would not be a smart permanent solution)
Update
My preferred solution as outlined above will not work for me as once I get into my data access layer as the results set will
essentially be filtered again back to the original set. There are some circumstances where it
could work; it would depend on the SQL issued (e.g. when collapsing a
results set down instead of trying to expand a results set as I was
trying to do here).
In that regard I am no longer looking for an answer here but will leave it for posterity as a preferred option and as per the guidelines - thanks anyway.
If you do not want to edit the c# code then the variable will have to be stored in the database although i am not sure how you will not edit the code.
If you are willing to edit the code then you can access a secondary configuration table which will have the settings that you would like the user to pass to the view. take this and allow the user to select which they want and then pass it to the view through the application.
I have a regular C# class called "vehicle" with properties like Name, NumberPlate, MaxSpeed, etc.
All the data for the class is stored in a SQLite Database where I have a Table "Car" and "Boat". The tables colums have the same names as the class properties (however, there are more columns than class properties - vehicle is a more generic abstraction). At the moment, I have to assign the result of the query individually one by one like this:
while (await statement.StepAsync())
{
myVehicle.Name = statement.Columns["Name"];
//[...]
myVehicle.MaxSpeed = decimal.TryParse(statement.Columns["MaxSpeed"]);
}
Additionally, I have to check if some columns exist ("Car" and "Boat" have a different set of columns) which is more code than I'd like it to be.
I read about EntityFramework to map my db table to my class - but that seems overkill. My requirement is to map properties and columns that have the same name and ignore everything else.
Is there a "easy" (dev time, lines of code) way to map my table columns to my class?
Thanks for reading!
The restrictions in phone 8 mean that a lot of the standard answers to this ("just use {some ORM / micro-ORM}") won't apply, since they don't work on phone 8. You can probably use reflection for a lot of this, but: reflection can be (relatively) slow, so it depends on how much data you will be processing. If it is occasional and light: fine, reflect away.
Runtime meta-programming (the tricks used by libraries like "dapper" in full .NET to make these things really fast) is not available on restricted runtimes, so if you want to avoid lots of boiler-plate that leaves build-time meta-programming. At the simplest, I wonder if you could use something like T4 to automate creating these methods for you as C#. There are also ways to use the reflection-emit API to construct assemblies (at build-time) for phone 8, but that is a pretty hard-core route.
My thoughts:
if the amount of types here isn't huge, just write the code
if you have a lot of types, or you just feel like it, consider a build-time code-generation meta-programming step; you might even think "hmm, is this something I could make available to the community?"
of course, the first thing to do is to check that such a thing doesn't already exist
There is a little helper which might fit your case. Basically, it will take a dictionary and try it's best to populate a objects properties using reflection. I didn't try it by myself though.
You'd simply do something like:
while (await statement.StepAsync())
{
myVehicle = DictionaryToObject<Car>(statement.Columns);
}
It might need some further work to get it running but maybe a good start.
I am trying to create something to hold global site-wide settings in our ASP.NET website - things such as site name, google analytics account number, facebook url etc... The site can have multiple ‘brands’ or sub-sites associated with it, hence the sitguid column, we would also like to have the option to put them into groups, hence the group column – e.g. TEST and PRODUCTION (set via web.config appsetting).
I do not want any of the KEY’s to be hardcoded anywhere, but I would like to be able to reference them as simply as possible in code, e.g. SiteSetting.getSetting(“SiteName”) (these may be used in templates (masterpages and such) that more junior devs will create)
I would also like to be able to administer the existing settings in our admin console and to be able to create new settings.
The datatype column is for the edit form so that the correct input element can be used, e.g. checkbox for bit types, text box for varchar etc...
SiteSettings database table currently:
[sts_sitGuid] [uniqueidentifier] NOT NULL, -- tells us which site the setting is for
[sts_group] [nvarchar](50) NOT NULL, -- used to group settings e.g. test/live
[sts_name] [nvarchar](max) NULL, -- the display name of the setting, for edit forms
[sts_alias] [nvarchar](50) NOT NULL, -- the name for the setting
[sts_value] [nvarchar](max) NOT NULL, -- the settings value
[sts_dataType] [nvarchar](50) NULL, -- indicates the control to render on edit form
[sts_ord] [tinyint] NULL, -- The order they will appear in on the admin form
I am part way through having this working at the moment, but I am not happy with the way I have done it and would like any advice people here have that might help find the ‘right’ solution! I'm sure people have done this before me. (I would share what I have so far, but do not want to skew the answers in any particular way) All i'm looking for is an overview of how this might be best done, not looking for anyone to write it for me ;)
I’ve done quite a bit of searching both here and Google and have not really found what I’m looking for, especially the ability to add new setting ‘definitions’ as well as editing the settings that exist.
The system runs on ASP.NET using webforms, it's all written in c# and uses MSSQL 2008.
As always, any help is very much appreciated!
EDIT: To clarify I am going to explain what I have built so far. I am dead set on storing all of this in SQL as we don't want web.config or other xml files or another database floating around since it'll give us more to do when we rollout the app to other customers.
So far I have a SiteSettings class, this has a method GetSetting which i can call with GetSetting("SettingAlias") to get the value for "SettingAlias" in the DB. This class's constructor fetches all the settings for the current site from the database and stores those in a dictionary, GetSetting reads from that dictionary. All of that part I am happy with so far.
The part I am struggling with is generating the edit form. The previous version of this used a webservice to get/set the settings and I am trying to continue using something similar to save work, but they were all defined in the code, such as GoogleAnalyticsID, Sitename etc... and each had a column in the database, the change I am making is to store these settings as ROWS instead (since then it's easier to add more, no need to change the schema & all of the sitesettings class) Currently my SiteSettings class has a SiteSettingsEditForm method which grabs all the info from the db, creates a bunch of controls for the form elements, puts that in a temporary page and executes that, then passes the HTML generated to our management system via ajax. This feels wrong and is a bit clunky, and is the reason for posting it here, I am having trouble figuring out how to save this stuff back via the webservice, but more importantly generating a bunch of HTML by executing a page containing a load of form controls just feels like the wrong way to do it.
So in summary I (think i) want to write a class to be able to cache & read a handful of rows from a database table, and also give me an edit form (or give data to something else to generate the form) that is dynamic based on the contents of the same database table (e.g. where my type column is 'bit' I want a checkbox, where it is 'text' I want a text input)
Sometimes this kind of problem is easier to visualize if you start off with the data model. If you want a setting per row, then two tables would probably be the best way to store this:
Site:
SiteId SiteKey SiteName
1 XYGZ4345 Client Site 1
2 AZT43752 Client Site 2
This would define the list of sites you have config for. I'd use a SiteKey as you'd put this in your web.config and it's better to abstract this away into a random string or GUID (to make it harder to accidentally load someone else's config), the client can change their name and you don't get confused in the future as you didn't use their old name as a key etc etc.
The config table itself is also simple, if we treat every setting as a string:
SiteSetting:
SettingId SiteId SettingName SettingValue
1 1 Brand KrustyBrand
2 1 GoogleId MSFTSUX0R
3 2 Brand ConfigMasters(TM)
You can then load all the config quite simply:
SELECT * FROM SiteSetting INNER JOIN Site ON (SiteSetting.SiteId = Site.SiteId) WHERE Site.SiteKey = 'XYGZ4345'
Now we have a list of key value pairs you could then store in a class like:
public class SiteSetting
{
public Site Site {
get; set; //Site would just be a simple class consisiting of Id, Key and Name to match the database table
}
protected Dictionary<String, String> Settings { get; set; } //Simple key value pairs
}
So this is a very simple solution. However, we can take it further - things to consider:
1) Can we add an environment to this somewhere?
We could either add a site per environment
OR
Add an environment to the SiteSetting table. The advantage of this is that you could define enironment = 'ALL' to avoid duplication.
OR
The database the configuration is loaded from defines the environment; so you change the config connection string in the app config. Of course, to connect to a different environment you have to change app.config, but you would potentially have to do that anyway to change the client key and/or environment.
2) Add the concept of user defineable settings - some settings you are going to want to change, some you are going to want to lock. A bit column containing "UserDefinable" would allow you to sort this out
3) Typing of settings.
This is slightly more difficult. You might have something like:
PropertyId PropertyName PropertyType Format UserDefined
1 Brand String NULL 1
2 DatePresentation DateTime "yyyy-MM-dd" 1
The Settings table then only defines a value, and a PropertyId. The advantage of this is that you can then start to increase the information about each setting you are storing, and reuse this information as the design is more normalized. The Settings class then changes like so:
public List<PropertyValue> { get; set; } //replacing the dictionary
PropertyValue then looks something like:
public class PropertyValue
{
int Id { get; set; }
public string Name { get; set; }
public string Value { get; set; }
public string PVType { get; set; } //Could be an enum
public string DisplayFormat { get; set;
private string _RawValue;
public string Value{
get{
switch(PVType){
case "DateTime":
return Convert.ToDateTime(_RawValue).ToString(DisplayFormat);
break;
case "Double":
return Convert.ToDouble(_RawValue).ToString(DisplayFormat);
break;
default:
return _RawValue;
}
}
set{
_RawValue = value;
}
}
}
Things like the Value method need to be improved to support robust error handling (you could also investigate using Convert.ChangeType to simplify the switch block)
This topic is as simple or as complicated as you choose to make it ;-)
Editing
As regards maintaining them; a very simple GUI would allow the user to see all of their properties in a tabular format. You might consider having rows where UserDefinable = 0 as readonly, otherwise the user can edit and add rows. You'd need to validate, especially for duplicate Setting Names for example.
The easiest way to do this is to use the DataGrid; a simple mockup might look something like this:
And a more sophisticated approach might look something like this
Generating the form is therefore as simple as databinding a collection of PropertyValue objects to your chosen grid solution.
As you probably found, there are a variety of ways that you can do this, ranging from Microsoft-recommended to 100% roll-your-own. You're currently on the roll-your-own end of the spectrum, which I think is usually better anyway.
Anyway, basically what you're looking for is a StateBag. Something loosely typed and highly configurable, which doesn't really lend itself very well to an RDBMS, but if you already have the SQL Server in place, it's probably going to be easiest to just use that.
Option 1: Redis
However, another option is to run something like Redis for site configurations. You can store it as a collection of key/value pairs in a Redis object, pull it in a JSON object, and then either convert that to a Hashtable, where lookups will be very fast, or some other hashable collection. A simple wrapper class around the collection would suffice.
Redis is very light and in your case wouldn't require a lot of configuration or maintenance.
Option 2: SQL Server
The way you've mentioned is actually a pretty good way of doing it, but rather than query the database every time, put a strongly typed collection or Hashtable in the cache and let it expire every few hours.
If you go with essentially what you have now, you could store it like this:
Setting
SettingId Int PK,
Key NVarchar(50),
Name NVarchar(50),
Description NVarchar(1000),
Type NVarchar(50),
Priority Int
Configuration
SiteId Int,
SettingId Int FK(Setting),
SettingValue NVarchar(1000)
Don't store things like live/test/dev in the database. You need to have completely separate databases and servers to distinguish between live/test/dev/stage/QA etc.
im currenctly doing this by creating a class which have all the settings as propertiese like
class Setting
{
GUID siteGuid{get; set;}
//other settings
}
then i created a static class SettingManager like this
public static class SettingManager
{
private ConcurrentDictionary<GUID,Setting> settings= new ConcurrentDictionary<GUID,Setting>;
GetSetting(Guid siteGUID)
{
settings.TryGet(siteGuid);
Lastrefreshed = DateTime.Now;
//other code
}
Private DateTime LastRefreshedOn = DateTime.Now;
public void PopulateSetingsDic()
{
//populate the settings dictionary by getting the values from the database
}
}
now anywhere in your code just include the namespace and use the settings.
u can populate the settings once or on every interval in application_start using lastRefreshedOn variable
it will be fast because u have all the settings inside the memory.
also if u want that the u should be able to add the settings dynamically then u can use an ExpandoObject and add settings dynamically using the column names from the database or just a mapping
then ull be able to use the settings by casting the settings ExpandoObject to IDictionary<string,object> here string can be the string converted GUID
Edit:- http://zippedtech.blogspot.in/2012/04/dynaminism-in-net-40.html check the link.. i have added a new post for solution to problems like this.
I would use xml, make a class that can take xpath as your "key"
Ex.
MySett.get("//level1/mysetting")
or even
MySett.get("//mysetting")
where each one can return a collection, just the first one, or what ever you want.
You could even overload.
I like xml because of it's great flexability, and to reduce code elsewher, just write a class.
Downside, You need to load your document at application startup and save at shutdown.
Here is an example class in vb code. (c code would still be very similar, I just used vb because it was up at thetime
Imports System.Xml
Public Class XSett
Public xml As XmlDocument
Public Overloads Function gett(ByVal xp As String)
Return CType(xml.SelectSingleNode(xp), XmlElement).InnerXml
'by using inner xml, you can have either text setting
'or more markup that you might need for another function
'your choice. you could even cast it into another instance
'of this class
End Function
Public Overloads Function gett(ByVal xp As String, ByVal sel As Integer)
Return CType(xml.SelectNodes(xp)(sel), XmlElement).InnerXml
'here, you can have multiple and choose the one you want
End Function
Public Overloads Sub gett(ByVal xp As String, ByRef col As Collection)
Dim i As Integer
Dim nds = xml.SelectNodes(xp)
For i = 0 To nds.Count - 1
col.Add(CType(nds(i), XmlElement).InnerXml)
Next
'Creted an entire collection of elemens.
'i used vb's "collection" object, but any collection would do
End Sub
Public Overloads Sub sett(ByVal ap As String, ByVal name As String, ByVal data As String)
'assume add here.
'ap asks for existing parent element. eg: //guids
'name ask for name of setting element
Dim ts = xml.CreateElement(name)
ts.InnerXml = data
If ap = "" Then 'we assume document element
xml.DocumentElement.AppendChild(ts)
Else
Dim p = CType(xml.SelectSingleNode(ap), XmlElement)
p.AppendChild(ts)
End If
End Sub
Public Overloads Sub sett(ByVal xp As String, ByVal sel As Integer, ByVal data As String)
'just change existing setting
CType(xml.SelectNodes(xp)(sel), XmlElement).InnerXml = data
End Sub
'naturally you can expand infinitely if needed
End Class
If I understand your question correctly you are looking for a centralized configuration framework. For configuration & server management I would normally recommend Chef or Puppet however for ASP.NET I did some quick googling and it seems like the WCF based Configuration Service might do the trick for you. The document I linked to is a step by step tutorial for the configuration service used in the .NET StockTrader 5 Sample Application.
I have a database table that essentially contains different types of things. I'll use animals as an example. I have a table called AnimalTypes:
AnimalTypes
{
ID:int,
Name:string
}
I then populate it with:
1:Dog,
2:Cat,
3:Fish
I would like to then have some sort of C# object created that functions similar to this enum be entirely read from the database:
enum AnimalTypes
{
Dog = 1,
Cat = 2,
Fish = 3
}
Is there a way to create an enum/class from a database table as described? I basically want to be able to reference things in the AnimalTypes table using intellisense and AnimalTypes.Dog as an example; I don't actually need an enum, just something that kind of functions like one. Is this possible?
Edit: I'm not really that thrilled about generating a DLL as I've seen in other related problems. I feel like this should be possible with reflection.
Lets suppose I don't need intellisense.
You will have to generate an assembly if you want to be able to use the enumeration or class at compilation time. Reflection happens at execution time so that won't give you intellisense.
This is a common problem - there are a set of distinct values in a database table and those values don't change often so they are modeled as an enum in the source code. This allows these somewhat static values to be easily used in a very readable way. The problem is that when the values do change, it would be nice if the enum changed as well.
The problem with trying to keep the enum and database in sync is that an automatic process doesn't change the fact that if you are changing the database it would be very unlikely that you would do so without having to roll new code to leverage the changed value. It is better to model these values as an enum and still store them in the database. Just manually sync them as the need arises.
Try this solution:
"Creating Enums from database lookup tables"
using T4 code generation for lookup tables.
There's always code generation: http://www.mygenerationsoftware.com/ if you don't want to go the reflection route.