Is there a way to cache the results of a MySQL query, specifically a data-reader?
I have some mundane queries that really only need to be performed once when the application is loaded, then each form can use the cache values instead of querying the remote server.
At present my method is to use MySqlDataReader to retrieve the data and store it in another MySqlDataReader so I can retrieve it at a later time (example below)
if (parentfrm.prioritylist != null)
{
if (parentfrm.prioritylist.HasRows)
{
using (var rows = parentfrm.prioritylist)
{
while (rows.Read())
{
cboxitem cbi = new cboxitem(int.Parse(rows["priorityid"].ToString()), rows["label"].ToString());
cb.Items.Add(cbi);
}
}
}
else
{
query = #"SELECT priorityid, label FROM prioritylist WHERE active = 'Y' ORDER BY theorder ASC";
parentfrm.prioritylist = db.localfetchrows(query);
if (cb != null)
{
using (var rows = db.localfetchrows(query))
{
while (rows.Read())
{
cboxitem cbi = new cboxitem(int.Parse(rows["priorityid"].ToString()), rows["label"].ToString());
cb.Items.Add(cbi);
}
}
}
}
}
else
{
query = #"SELECT priorityid, label FROM prioritylist WHERE active = 'Y' ORDER BY theorder ASC";
parentfrm.prioritylist = db.localfetchrows(query);
if (cb != null)
{
using (var rows = db.localfetchrows(query))
{
while (rows.Read())
{
cboxitem cbi = new cboxitem(int.Parse(rows["priorityid"].ToString()), rows["label"].ToString());
cb.Items.Add(cbi);
}
}
}
}
public class Priority{
public int PriorityId {get;set;}
public string Label {get;set;}
}
public class CachedItems {
private static List<Priority> _priorityList=new List<Priority>();
public static List<Priority> GetPriorityList() {
if (_priorityList==null){
// Load DB Items to the _priorityList,
// if the app is multithreaded, you might wanna add some locks here
}
}
}
Then anywhere in the code, just use CachedItems.GetPriorityList() to access the cached list.
Short answer: populate an object, and reuse it
Longer answer: design your application to use some kind of decoupling. The simplest way is to work with the dual DataSet/DataAdapter classes + some databindings.
Not only your code will be simpler to read (and write), but you will have far more flexibility (like firing requests only as needed).
A traditional approach is to create :
a Data Abstraction layer (DAL) that wrap the DB operations to return Dataset (or custom classes), save changes (DataSet has a builtin change tracking feature)
A Business Logic Layer (BLL) that encapsulate the logic. The Bll can work with DALs
a UI layer, that contains the application itself. The Ui should only call BLL's method.
Start by reading this article : http://www.codeproject.com/Articles/36847/Three-Layer-Architecture-in-C-NET
A quick way is to use the dynamic Type/anonymous classes (.NET 4) which is on par with what you are already doing (building on Mennans code a bit)
public class CachedItems
{
private static List<dynamic> _priorityList = new List<dynamic>();
public static List<dynamic> GetPriorityList()
{
if (_priorityList == null)
{
// Load DB Items to the _priorityList,
// if the app is multithreaded, you might wanna add some locks here
query = #"SELECT priorityid, label FROM prioritylist WHERE active = 'Y' ORDER BY theorder ASC";
parentfrm.prioritylist = db.localfetchrows(query);
if (cb != null)
{
using (var rows = db.localfetchrows(query))
{
while (rows.Read())
{
_priorityList.Add(new {
PriorityId = int.Parse(rows["priorityid"].ToString()),
Label = rows["label"].ToString()
});
}
}
}
}
return _priorityList;
}
}
Using the cached values:
foreach (var item in CachedItems.GetPriorityList())
{
if (item.Priority == 1)
{
}
}
What you don't get is type safety i.e. the item.Priority migth cause you runtime errors if Priority does not contain a value or a value which is not an int, you will have to cover this when loading the cache, but that is basically the situation you are in right now with the datareader.
You might consider a small application toolkit class, PriorityCheckboxes, and make it a singleton. Then when you need those checkbox values, first reference of the GetInstance() method on PriorityCheckboxes, should load it up. If you have a race condition with this, the load logic could go in a static initializer.
public class PriorityCheckboxes {
private static CheckBox _CBItems = null;
private static PriorityCheckboxes _instance - null;
public CheckBox CheckBoxes {
get() { return _CBItems; }
}
private PriorytyCheckboxes() {
this.LoadCBItems();
_instance = new PriorityCheckboxes();
}
public static PriorityCheckboxes GetInstance() {
if(_instance == null ) _instance = new PriorityCheckboxes();
return _instance;
}
private void LoadCBItems() { }
}
Related
i have transform a php/js code to js/c#, but i stuck for update the new value.
The php code is :
`if (isset($_POST['update'])) {
foreach($_POST['positions'] as $position) {
$index = $position[0];
$newPosition = $position[1];
$conn->query("UPDATE country SET position = '$newPosition' WHERE id='$index'");
}
exit('success');
}`
My "empty" c# code
[HttpPost]
public ActionResult Index (userTable index)
{
picturesEntities MyDb = new picturesEntities();
homeViewModel HVM = new homeViewModel();
HVM.userTables = MyDb.userTables.ToList();
if (Request["update"] != null)
{
foreach (Request["positions"])
{
MyDb.SaveChanges();
}
return View(HVM);
}
}
If someone could help me for it that would be great, i'm stuck on it for days and i didn't find a workning solution yet.
Thanks to everyone who read my message.
Most ASP.NET will bind a custom class which will be compatible to your request.
public class UserPositionsRequest
{
public bool Update { get; set; }
// For orderly, this actually be a list of a custom class
public List<int[]> Positions { get; set; }
}
This by any means is not a complete and working solution, the following code was never been tested and can be consider as pseudo-like code.
Also, the .Id and .Position should be the same sensitivity as in Db.
// Binding our UserPositionsRequest class
public void Index(UserPositionsRequest request) {
// Checking if we should update, if you will change the request to boolean type: "true"
// ..on the client side, then you could actually change the condition to be: if (request.Update)
if (request.Update == 1) {
// Creating database connection using (I assume) EntityFramework
using (var picturesEntities = new picturesEntities()) {
// Building a dictionary for fast lookup. Key, Value as the 0, 1 arg respectfully
var usersDataToUpdate = request.Positions.ToDictionary(p => p[0], p => p[1]);
// Finding the entries that needs to be updated
var usersEntitiesToUpdate = picturesEntities.userTables.Where(cntry => usersDataToUpdate.ContainsKey(cntry.Id));
// Iterating over the entities
foreach (var userEntity in usersEntitiesToUpdate) {
// Updating their position.
userEntity.Position = usersDataToUpdate[userEntity.Id];
}
picturesEntities.SaveChanges();
}
}
// Probably you wanted to return something here, but it's probably an ajax and you can skip that.
}
I'm learning my way around EF, and I know caching is faster than a round trip to the DB for things like state, country, etc. But I'm not sure how to implement it. I was looking at this post (entities from local cache) that mentioned an extension, but is there something built in I should leverage?
I'd like to have a function like this that wouldn't have to go to the db every time:
public static int GetCountryId(string countryCode = "US")
{
if (countryCode == string.Empty)
countryCode = "US"; // Assume US
return db.Country.Where
(
p => p.CountryCode == countryCode
).Select(p => p.CountryId).First();
}
Updated:
I'm now thinking about a GetCached function that would use generics to hold some lookup lists in memory. Somthing like this:
public class Lookups
{
private static MkpContext db = new MkpContext();
public static IEnumerable GetCached(CachedLists list)
{
ObjectCache cache = MemoryCache.Default;
var listOut = cache[list.ToString()] as IEnumerable;
if (listOut != null) return listOut;
switch (list)
{
case CachedLists.Countries:
cache.Set
(
list.ToString(),
db.Country.ToList(),
new DateTimeOffset(DateTime.Now,new TimeSpan(1,0,0,0))
);
break;
default:
return null;
}
listOut = cache[list.ToString()] as IEnumerable;
return listOut;
}
}
public enum CachedLists
{
Countries,
}
But the above solution would leave me with an un-typed Enumerable. I'd love to be able to specify the types somehow, or better yet, do some sort of extension.
There are a lot of options, but here's one approach that will work well if users are mostly querying the same few country codes:
Create a MemoryCache instance to use as a static, private, readonly field on your class. In your method, try to get a cached item from this cache if there is one with the given countryCode as its key. If it's not there, do your database query and Add the result into the cache before returning it.
By the way, the approach in the article you linked probably isn't a very good approach: it will only help if you've already got data in the specific database context instance that you're dealing with, and usually it's best for your contexts to be short-lived. It's also really CPU-intensive to compile an expression into a function, and then run that function against every entity that the context has cached, just to find out whether the item is there or not. If you then find out that it's not there, and you have to go back to the database anyway, you just wasted time and resources.
The method you have proposed is usually what I do to return cached data.
However data is cached as an object and therefor you have to return it as the object you expect.
Lets assume that your country class is represented by:
public class Country
{
#region Constructors
public Country(string code, string name)
{
this.Code = code;
this.Name = name;
}
#endregion
#region Properties
public string Name { get; set; }
public string Code { get; set; }
#endregion
}
What we need to do is look up our cache by a given key, if exists return the cache, otherwise get from database - same logic as you did.
Difference is here var listOut = cache[list.ToString()] as IEnumerable;
You want to check whether the value for that key is of type Country
If we define a GetCache method as following:
static object GetCache(string cacheKey)
{
if (Cache[cacheKey] is object cachedResult)
{
return cachedResult;
}
return null;
}
What we need to do in order to return a List<Country> is
if (GetCache(cacheKey) is List<Country> cachedData)
{
return cachedData;
}
Now instead of a list of object we have a List<Country>
I have made a simple console app to show the result - hope it helps:
namespace ConsoleApp3
{
#region Usings
using System;
using System.Collections.Generic;
using System.Runtime.Caching;
#endregion
class Program
{
#region Fields
private static readonly ObjectCache Cache = MemoryCache.Default;
#endregion
static void Main(string[] args)
{
//simulates app life span
for (int i = 0; i < 5; i++)
{
var countries = GetData();
foreach (var country in countries)
{
Console.WriteLine(country.Name);
}
Console.ReadLine();
}
}
static List<Country> GetData()
{
string cacheKey = "Country-Lookup";
if (GetCache(cacheKey) is List<Country> cachedData)
{
return cachedData;
}
// otherwise do some logic stuff and get from DB
//db data simulation
List<Country> coutries = new List<Country>
{
new Country("IT", "Italy"),
new Country("UK", "United Kindom"),
new Country("US", "United States")
};
//add to cache
AddToCache(cacheKey, coutries);
return coutries;
}
static object GetCache(string cacheKey)
{
if (Cache[cacheKey] is object cachedResult)
{
return cachedResult;
}
return null;
}
static void AddToCache(string cacheKey, object dataToCache)
{
if (dataToCache == null)
{
return;
}
Cache.Set(cacheKey, dataToCache, DateTimeOffset.Now.AddMinutes(1));
}
}
}
You can use EntityFramework Plus.
public static int GetCountryId(string countryCode = "US")
{
return db.Country.Where
(
p => p.CountryCode == countryCode
).Select(p => p.CountryId).First();
}
Would probably become...
public static int GetCountryId(string countryCode = "US")
{
using var context = new Context();
var allCountries = context.Country
.FromCache({cachePolicy})
.ToDictionary(x => x.CountryCode);
return allCountries[countryCode];
}
For something basic like mapping a country code to a countryId I'd suggest keeping it simple and using a dictionary. If this is happening inside a web application you'll want to use a ConcurrentDictionary to handle multiple threads hitting the code, otherwise a normal dictionary will be fine.
You could also have some code that populates the dictionary when the app starts up to make the experience even snappier for users.
static ConcurrentDictionary<string, int> countryLookup = new ConcurrentDictionary<string, int>();
public static int GetCountryId(string countryCode = "US")
{
if (countryCode == string.Empty)
countryCode = "US"; // Assume US
if (countryLookup.TryGetValue(countryCode, out int countryId))
return countryId;
var countryId = db.Country
.First(p => p.CountryCode == countryCode)
.CountryId;
countryLookup.TryAdd(countryCode, countryId);
return countryId;
}
I have a WizardInfo class which as several TLists as properties, this then populates as the user goes through the wizard on the last screen I query the Tlists and make them into Lists and private fields
I then create Lists of DefaultItems from these lists. This is my own class and as name and Id as its property.
He is some code
public class DefaultItem
{
public int ID {get;set;}
public string Name {get;set;}
}
private List<DefaultItem> _defaultList = null;
_defaultList = new List<DefaultItem>();
defaultValue = PopulateDefaultList(_asmgps, defaultList);
private int PopulateDefaultList(
List<ASGMP> asmgps,
ref List<DefaultItem> defaultList)
{
int isdefault = -1;
foreach (ASGMP asgmp in asgmps)
{
if (asgmp.IsChecked)
{
if (asgmp.IsDefault)
{
isdefault = asgmp.ID;
}
DefaultItem defaultItem = new DefaultItem();
defaultItem.ID = asgmp.ID;
defaultItem.Name = GetMPTName(asgmp.ID);
defaultList.Add(defaultItem);
}
}
return isdefault;
}
private string GetMPTName(int ID)
{
try
{
SGMP sgmp = DataRepository.SGMPProvider.GetByASGMPID(ID)
if (serviceGroupMailPresentation != null)
{
MPT mpt DataRepository.MPTProvider.GetByMPTID(SGMP.MPTID);
if (mailPresentationType != null)
{
return mpt.Name;
}
}
return string.Empty;
}
catch (Exception ex)
{
WindowsEventLog.Write(ex);
throw;
}
}
The problem i am having is when i remove a item from the defaultList it affects asgmp.
I have found the answer. When I get the mpt name I get asgmp from the database this is where Codesmith does a strange thing and connects the usage of the List and the DefaultList. By querying the original List instead of going to the database it now works fine.
It is being removed because List<T> is derived from object, and is a Reference type. Reference types are passed by reference, i.e. when you pass your list, you are passing a pointer to its location in memory. So any changed you make on the copied reference, will also be reflected on the original object.
In order to make a copy you can change this like:
defaultValue = PopulateDefaultList(_asmgps, defaultList);
to this:
defaultValue = PopulateDefaultList(_asmgps.ToList(), defaultList);
This will enumerate the collection as IEnumerable<T> and return is as a list. This will effectivlly create a copy.
erm, instead of PopulateDefaultList why not just do,
var defaultList = asgmps
.Where(asgmp => asgmp.IsChecked)
.Select(asgmp => new
{
IsDefault = asgmp.IsDefault,
Item = new DefaultItem
{
ID = asgmp.ID,
Name = GetMPTName(asgmp.ID)
}
}).ToList();
of course, naming a collection defaultList that contains non-defaults seems counter intuitive.
I found out that this is because of ntiers instead of using the database the to get the ID I should of used the in List of T in
I would like to know if there is a better way to solve this problem that I am overlooking. (I'm looking for a second opinion)
I want to create a generic and easy way to bind objects to database reader queries using "Oracle.DataAccess.Client".
In order to do this I initially wanted to create an object which inherited from OracleCommand; however, OracleCommand is a sealed object.
To deal with this I decided to create an extension method which attempts to map objects to generic columns in the database for each row.
EDIT : In my scenario, I know what the database will look like; however, I will not know where the database is before run time. i.e. The database may have been transferred ahead of time and the end user will specify the credentials for the database at run time.
Here is the implementation:
public static T[] Bind<T>(this OracleCommand oe, Binding binding, CommandBehavior Behavior = CommandBehavior.Default)
{
List<T> ret = new List<T>();
using (var reader = oe.ExecuteReader(Behavior))
{
while (reader.Read())
{
T unknownObj = (T)Activator.CreateInstance(typeof(T));
for (int i = 0; i < binding.GetBindCount(); i++)
{
var propinfo = unknownObj.GetType().GetProperties().ToList();
var prop = propinfo.Find((p) => p.Name == binding.GetBindValue(i, true));
prop.SetValue(unknownObj, reader[binding.GetBindValue(i, false)]);
}
ret.Add(unknownObj);
}
}
return ret.ToArray();
}
}
public class Binding
{
List<BindingMap> _map = new List<BindingMap>();
public void AddBind(String VariableName, String ColumnName)
{
_map.Add(new BindingMap(VariableName, ColumnName));
}
public String GetBindValue(int index, bool IsVariable = true)
{
var a = _map.ToArray();
return (IsVariable) ? a[index].Variable : a[index].Column;
}
public int GetBindCount()
{
return _map.Count;
}
}
public class BindingMap
{
public String Column;
public String Variable;
public BindingMap(String v, String c)
{
Variable = v;
Column = c;
}
}
Is there a better way to do this that I've overlooked, or is this a sound?
The way it would be used in real code is like this :
static void Main()
{
Binding b = new Binding();
b.AddBind("CreatedBy", "Create_by");
using (var Conn = new OracleConnection())
{
Conn.ConnectionString = od.Options.GetConnectionString();
using (var Command = new OracleCommand())
{
Command.Connection = Conn;
Command.CommandText = "Select * From Accounts";
Conn.Open();
var a = Command.Bind<Account>(b);
foreach (Account e in a)
{
Console.WriteLine(e.CreatedBy);
}
}
}
Console.Read();
}
public class Account
{
public String CreatedBy
{
get;
set;
}
}
As a slightly better way, you could designate the bound property like Telerik does: with a Linq expression. Here is the usage. Instead of :
AddBind("CreatedBy", "Created_by");
You would write
AddBind( x => x.CreatedBy, "Created_by");
You get a slightly stronger typing opportunity. The signature of AddBind would be:
public void AddBind<T>(Expression<Func<Account, T>> variable, string columnName) {
// ...
}
But I would not go into the way of generic functions. I'd rather overload a non-generic function :
public void AddBind(Expression<Func<Account, double>> variable, string columnName) {
// Add binding for double
}
public void AddBind(Expression<Func<Account, DateTime>> variable, string columnName) {
// Add binding for DateTime
}
// ...
The type of binding would then be selected according to the type of your mapped object. This prevents you from misnaming your properties, so you keep the possibility of performing name changes in the Account class without breaking your bindings.
The column name has still to be a string, sorry.
Of course, the way then to generalize is to make your BindingMap generic. (Taking your business class as a type parameter)
class BindingMap<BusinessClass> {
// ....
public void AddBind(Expression<Func<BusinessClass, double>> variable, string columnName) {
// Add binding for double
}
public void AddBind(Expression<Func<BusinessClass, DateTime>> variable, string columnName) {
// Add binding for DateTime
}
// ...
};
I leave as an exercice to you the problem of digging the property descriptor out of the expression :)
I'm following this guide and I am getting an error. Can anyone help me?
The code for my datamodel is below
namespace Datalayer {
public class DataModel {
public DataModel()
{
using (btWholesaleDataContext db = new btWholesaleDataContext()) {
//! requires auth
var MACRequestList = from r in db.btRequests
select new Models.BT.Request {
ID = r.ID,
Date = r.DateTime,
StatusCode = 3,
Status = r.Status
};
MACRequests = MACRequestList.AsQueryable();
}
}
public IQueryable<Models.BT.Request> MACRequests { get; private set; }
}
}
The web service gives the error
Cannot access a disposed
object.Object name: 'DataContext
accessed after Dispose.'
When I access MACRequests
I have only posted the code I think is broken. If you want to see more just let me know.
Your data context is being disposed at the end of your constructor, at the end of the using { } block. However when you use the IQueryable MACRequests property, it needs that underlying context, which has since been disposed.
One possible way to handle this is to make your class IDisposable and dispose the context that way:
public class DataModel : IDisposable {
private btWholesaleDataContext wholesaleDataContext;
public DataModel()
{
wholesaleDataContext = new btWholesaleDataContext();
//! requires auth
var MACRequestList = ... ;
MACRequests = MACRequestList.AsQueryable();
}
public IQueryable<Models.BT.Request> MACRequests { get; private set; }
public void Dispose() {
if(wholesaleDataContext != null)
wholesaleDataContext.Dispose();
}
}
Then you have to make sure that DataModel is properly disposed by whatever uses it.
Another alternative is to make MACRequests the actual list of items instead of the IQueryable:
public class DataModel {
public DataModel()
{
using (btWholesaleDataContext db = new btWholesaleDataContext()) {
//! requires auth
var MACRequestList = ... ;
MACRequests = MACRequestList.ToList(); // ToList reads the records now, instead of later.
}
}
public List<Models.BT.Request> MACRequests { get; private set; }
}
I think its because you are using an IQueryable<>. Its lazily queries the service.
Use List<> instead so that it queries immediately
Or make "btWholesaleDataContext db" into a member variable
Queries to MACRequests are deferred - once you're out of the using block and your DataContext is disposed you're not going to be able to make the query you want.
You're creating the data context in a using block in the constructor of your DataModel...so by the time you access the MACRequests, the data context has been disposed.
Consider the following:
public class DataModel : IDisposable {
btWholesaleDataContext db = new btWholesaleDataContext();
public void Dispose()
{
btWholesaleDataContext.Dipose();
}
public IQueryable<Models.BT.Request> MACRequests {
get {
return from r in db.btRequests
select new Models.BT.Request {
ID = r.ID,
Date = r.DateTime,
StatusCode = 3,
Status = r.Status
};
}
}
}
Note that this usage will work:
using (var dm = new DataModel())
{
dm.MACRequests.ToArray();
}
but this will fail for the same reason as the original:
IQueryable<Models.BT.Request> requests = null;
using (var dm = new DataModel())
{
requests = dm.MACRequests;
}
// this will fail because the context is disposed by the time we force enumeration of the query
requests.ToArray();
...Alternatively, since WCF data services can't filter on projections, and thus all you can really do with the query
from r in db.btRequests
select new Models.BT.Request {
ID = r.ID,
Date = r.DateTime,
StatusCode = 3,
Status = r.Status
};
is execute it...
just consider changing your original code to return an array or a list instead of leaving it as a queryable.