I'm currently trying out the new MemoryCache in .Net 4 to cache a few bits of data in one of our apps. The trouble I have is the objects are updated and the cache appears to be persisting the changes e.g.
public IEnumerable<SomeObject> GetFromDatabase(){
const string _cacheKeyGetDisplayTree = "SomeKey";
ObjectCache _cache = MemoryCache.Default;
var objectInCache = _cache.Get(_cacheKeyGetDisplayTree) as IEnumerable<SomeObject>;
if (objectInCache != null)
return objectInCache.ToList();
// Do something to get the items
_cache.Add(_cacheKeyGetDisplayTree, categories, new DateTimeOffset(DateTime.UtcNow.AddHours(1)));
return categories.ToList();
}
public IEnumerable<SomeObject> GetWithIndentation(){
var categories = GetFromDatabase();
foreach (var c in categories)
{
c.Name = "-" + c.Name;
}
return categories;
}
If I were calling GetWithIndentation() first and then later calling GetFromDatabase() I would expect it to return the original list of SomeObject but instead it returns the modified items (with "-" prefixed on the name).
I thought ToList() destroyed the reference but it still seems to persist the changes. I'm sure it's obvious but can anyone spot where I'm going wrong?
I created a ReadonlyMemoryCache class to solve this problem. It inherits from the .NET 4.0 MemoryCache, but objects are stored readonly (by-value) and cannot be modified. I deep copy the objects before storing using binary serialization.
using System;
using System.Collections.Generic;
using System.Collections.Specialized;
using System.IO;
using System.Runtime.Caching;
using System.Runtime.Serialization.Formatters.Binary;
using System.Threading.Tasks;
namespace ReadOnlyCache
{
class Program
{
static void Main()
{
Start();
Console.ReadLine();
}
private static async void Start() {
while (true)
{
TestMemoryCache();
await Task.Delay(TimeSpan.FromSeconds(1));
}
}
private static void TestMemoryCache() {
List<Item> items = null;
string cacheIdentifier = "items";
var cache = ReadonlyMemoryCache.Default;
//change to MemoryCache to understand the problem
//var cache = MemoryCache.Default;
if (cache.Contains(cacheIdentifier))
{
items = cache.Get(cacheIdentifier) as List<Item>;
Console.WriteLine("Got {0} items from cache: {1}", items.Count, string.Join(", ", items));
//modify after getting from cache, cached items will remain unchanged
items[0].Value = DateTime.Now.Millisecond.ToString();
}
if (items == null)
{
items = new List<Item>() { new Item() { Value = "Steve" }, new Item() { Value = "Lisa" }, new Item() { Value = "Bob" } };
Console.WriteLine("Reading {0} items from disk and caching", items.Count);
//cache for x seconds
var policy = new CacheItemPolicy() { AbsoluteExpiration = new DateTimeOffset(DateTime.Now.AddSeconds(5)) };
cache.Add(cacheIdentifier, items, policy);
//modify after writing to cache, cached items will remain unchanged
items[1].Value = DateTime.Now.Millisecond.ToString();
}
}
}
//cached items must be serializable
[Serializable]
class Item {
public string Value { get; set; }
public override string ToString() { return Value; }
}
/// <summary>
/// Readonly version of MemoryCache. Objects will always be returned in-value, via a deep copy.
/// Objects requrements: [Serializable] and sometimes have a deserialization constructor (see http://stackoverflow.com/a/5017346/2440)
/// </summary>
public class ReadonlyMemoryCache : MemoryCache
{
public ReadonlyMemoryCache(string name, NameValueCollection config = null) : base(name, config) {
}
private static ReadonlyMemoryCache def = new ReadonlyMemoryCache("readonlydefault");
public new static ReadonlyMemoryCache Default {
get
{
if (def == null)
def = new ReadonlyMemoryCache("readonlydefault");
return def;
}
}
//we must run deepcopy when adding, otherwise items can be changed after the add() but before the get()
public new bool Add(CacheItem item, CacheItemPolicy policy)
{
return base.Add(item.DeepCopy(), policy);
}
public new object AddOrGetExisting(string key, object value, DateTimeOffset absoluteExpiration, string regionName = null)
{
return base.AddOrGetExisting(key, value.DeepCopy(), absoluteExpiration, regionName);
}
public new CacheItem AddOrGetExisting(CacheItem item, CacheItemPolicy policy)
{
return base.AddOrGetExisting(item.DeepCopy(), policy);
}
public new object AddOrGetExisting(string key, object value, CacheItemPolicy policy, string regionName = null)
{
return base.AddOrGetExisting(key, value.DeepCopy(), policy, regionName);
}
//methods from ObjectCache
public new bool Add(string key, object value, DateTimeOffset absoluteExpiration, string regionName = null)
{
return base.Add(key, value.DeepCopy(), absoluteExpiration, regionName);
}
public new bool Add(string key, object value, CacheItemPolicy policy, string regionName = null)
{
return base.Add(key, value.DeepCopy(), policy, regionName);
}
//for unknown reasons, we also need deepcopy when GETTING values, even though we run deepcopy on all (??) set methods.
public new object Get(string key, string regionName = null)
{
var item = base.Get(key, regionName);
return item.DeepCopy();
}
public new CacheItem GetCacheItem(string key, string regionName = null)
{
var item = base.GetCacheItem(key, regionName);
return item.DeepCopy();
}
}
public static class DeepCopyExtentionMethods
{
/// <summary>
/// Creates a deep copy of an object. Must be [Serializable] and sometimes have a deserialization constructor (see http://stackoverflow.com/a/5017346/2440)
/// </summary>
public static T DeepCopy<T>(this T obj)
{
using (var ms = new MemoryStream())
{
var formatter = new BinaryFormatter();
formatter.Serialize(ms, obj);
ms.Position = 0;
return (T)formatter.Deserialize(ms);
}
}
}
}
In memory cached objects are stored within the same process space as the cache client process. When a cache client requests a cached object, the client receives a reference to the locally cached object rather than a copy.
The only way to get a clean copy of the object is to implement a custom clone mechanism (ICloneable, Serialization, Automapping, ...). With that copy you will be able to alter the new object without altering the parent object.
Depending on your use case, it is generally not recommanded to update an object in the cache.
You can do it easier if you deserialize and serialize again and get your cache object "By Value".
You can do it like this with Newtonsoft lib (just get it from NuGet)
var cacheObj = HttpRuntime.Cache.Get(CACHEKEY);
var json = JsonConvert.SerializeObject(cacheObj);
var byValueObj = JsonConvert.DeserializeObject<List<string>>(json);
return byValueObj;
Why not just store as json or a string? These are not passed by reference and when you get out of the cache you will get a new copy :) I am here to be challenged as thats what I am doing atm!
Serialization/Deserialization will solve the problem but at the same time it defeats the porpose of having objects in memory. The role of cache is to provide fast access to the stored object and we are adding the deserialization overhead here. Since deserialization is required I would suggest cache as service , something like redis cache, it will be centralized so you don't have to have copy of memory object per worker process and deserialization is anyways done.
The key thing in this case that you chose a fast serialization/deserialization option.
Related
I'm working on a Window Forms application in Visual Studio, and I'm using a custom settings object to keep track of some application settings.
The user can change these settings through the PropertyGrid widget.
This works great for string and integer values, but now I also want to add a List<string> variable, so the user can enter a list of keywords.
I've added the List<string> variable to the settings object and I've added a TypeConverter to show it as a comma separated string representation in the PropertyGrid. Without the TypeConverter the value would display as just (Collection). It is displayed correctly and I can edit it, see screenshot below
this._MyProps = new PropsClass();
this._MyProps.ReadFromIniFile("mysettings.ini");
propertyGrid1.SelectedObject = this._MyProps;
Now I also want to write and read these setting to a settings.ini file, so I've added SaveToIniFile and ReadFromIniFile methods to the object. This works for string and integer values, except the List<string> is not saved and loaded to and from the .ini file correctly. When I call SaveToIniFile the content mysettings.ini is for example this, still using the "(Collection)" representation and not the values entered by the user:
[DataConvert]
KeyWordNull=NaN
ReplaceItemsList=(Collection)
YearMaximum=2030
So my question is, how can I save/load a List<string> setting to an ini file while also allowing the user to edit it in a PropertyGrid?
I know it'd have to convert from a string to a List somehow, maybe using quotes around the string to inclkude the line breaks, or maybe just comma-separated back to a list of values? But anyway I thought that is what the TypeConverter was for. So why is it showing correctly in he PropertyGrid but not in the ini file? See code below
The custom settings properties object:
// MyProps.cs
public class PropsClass
{
[Description("Maximum year value."), Category("DataConvert"), DefaultValue(2050)]
public int YearMaximum { get; set; }
[Description("Null keyword, for example NaN or NULL, case sensitive."), Category("DataConvert"), DefaultValue("NULL")]
public string KeyWordNull { get; set; }
private List<string> _replaceItems = new List<string>();
[Description("List of items to replace."), Category("DataConvert"), DefaultValue("enter keywords here")]
[Editor("System.Windows.Forms.Design.StringCollectionEditor, System.Design, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a", typeof(UITypeEditor))]
[DesignerSerializationVisibility(DesignerSerializationVisibility.Content)]
[TypeConverter(typeof(StringListConverter))]
public List<string> ReplaceItemsList
{
get
{
return _replaceItems;
}
set
{
_replaceItems = value;
}
}
and in the same PropsClass class, the write and read methods to save/load from a settings.ini file
[DllImport("kernel32.dll")]
public static extern int GetPrivateProfileSection(string lpAppName, byte[] lpszReturnBuffer, int nSize, string lpFileName);
public void SaveToIniFile(string filename)
{
// write to ini file
using (var fp = new StreamWriter(filename, false, Encoding.UTF8))
{
// for each different section
foreach (var section in GetType()
.GetProperties()
.GroupBy(x => ((CategoryAttribute)x.GetCustomAttributes(typeof(CategoryAttribute), false)
.FirstOrDefault())?.Category ?? "General"))
{
fp.WriteLine(Environment.NewLine + "[{0}]", section.Key);
foreach (var propertyInfo in section.OrderBy(x => x.Name))
{
var converter = TypeDescriptor.GetConverter(propertyInfo.PropertyType);
fp.WriteLine("{0}={1}", propertyInfo.Name, converter.ConvertToInvariantString(propertyInfo.GetValue(this, null)));
}
}
}
}
public void ReadFromIniFile(string filename)
{
// Load all sections from file
var loaded = GetType().GetProperties()
.Select(x => ((CategoryAttribute)x.GetCustomAttributes(typeof(CategoryAttribute), false).FirstOrDefault())?.Category ?? "General")
.Distinct()
.ToDictionary(section => section, section => GetKeys(filename, section));
//var loaded = GetKeys(filename, "General");
foreach (var propertyInfo in GetType().GetProperties())
{
var category = ((CategoryAttribute)propertyInfo.GetCustomAttributes(typeof(CategoryAttribute), false).FirstOrDefault())?.Category ?? "General";
var name = propertyInfo.Name;
if (loaded.ContainsKey(category) && loaded[category].ContainsKey(name) && !string.IsNullOrEmpty(loaded[category][name]))
{
var rawString = loaded[category][name];
var converter = TypeDescriptor.GetConverter(propertyInfo.PropertyType);
if (converter.IsValid(rawString))
{
propertyInfo.SetValue(this, converter.ConvertFromString(rawString), null);
}
}
}
}
// helper function
private Dictionary<string, string> GetKeys(string iniFile, string category)
{
var buffer = new byte[8 * 1024];
GetPrivateProfileSection(category, buffer, buffer.Length, iniFile);
var tmp = Encoding.UTF8.GetString(buffer).Trim('\0').Split('\0');
return tmp.Select(x => x.Split(new[] { '=' }, 2))
.Where(x => x.Length == 2)
.ToDictionary(x => x[0], x => x[1]);
}
}
and the TypeConverter class for the ReplaceItemsList property
public class StringListConverter : TypeConverter
{
public override object ConvertTo(ITypeDescriptorContext context, CultureInfo culture, object value, Type destinationType)
{
if (value is List<string>)
{
return string.Join(",", ((List<string>)value).Select(x => x));
}
return base.ConvertTo(context, culture, value, destinationType);
}
}
The reason your type converter is not used is because of this line:
var converter = TypeDescriptor.GetConverter(propertyInfo.PropertyType);
You are getting the TypeConverter that is defined on the type of the property. So for ReplaceItemsList that would be the TypeConverter for List<T>. You need to get the TypeConverter for the property since that is where you added the TypeConverter attribute. So either you do something like you did for the category attribute in the read method where you use the PropertyInfo's GetCustomAttributes or you do what the PropertyGrid does which is use the PropertyDescriptors to get to the properties and their state. The latter would be better since if the object implemented ICustomTypeDescriptor or some other type augmentation like TypeDescriptionProvider then you would get that automatically.
So something like the following for the Save using PropertyDescriptors would be:
public void SaveToIniFile(string filename)
{
// write to ini file
using (var fp = new StreamWriter(filename, false, Encoding.UTF8))
{
// for each different section
foreach (var section in TypeDescriptor.GetProperties(this)
.Cast<PropertyDescriptor>()
.GroupBy(x => x.Attributes.Cast<Attribute>().OfType<CategoryAttribute>()
.FirstOrDefault()?.Category ?? "General"))
{
fp.WriteLine(Environment.NewLine + "[{0}]", section.Key);
foreach (var propertyInfo in section.OrderBy(x => x.Name))
{
var converter = propertyInfo.Converter;
fp.WriteLine("{0}={1}", propertyInfo.Name, converter.ConvertToInvariantString(propertyInfo.GetValue(this)));
}
}
}
}
If you are going to use a custom TypeConverter, you'll have to register it as a provider to TypeDescriptionProvider:
TypeDescriptor.AddProvider(new CustumTypeDescriptorProvider(), typeof(List<string>));
And in your implementation you could just do this in the constructor of PropsClass (instead of using the attribute). I created some custom code below that would do the split.
public class PropsClass
{
[DllImport("kernel32.dll")]
public static extern int GetPrivateProfileSection(string lpAppName, byte[] lpszReturnBuffer, int nSize, string lpFileName);
[Description("Maximum year value."), Category("DataConvert"), DefaultValue(2050)]
public int YearMaximum { get; set; }
[Description("Null keyword, for example NaN or NULL, case sensitive."), Category("DataConvert"), DefaultValue("NULL")]
public string KeyWordNull { get; set; }
private List<string> _replaceItems = new List<string>();
[Description("List of items to replace."), Category("DataConvert"), DefaultValue("enter keywords here")]
[Editor("System.Windows.Forms.Design.StringCollectionEditor, System.Design, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a", typeof(UITypeEditor))]
[DesignerSerializationVisibility(DesignerSerializationVisibility.Content)]
public List<string> ReplaceItemsList
{
get
{
return _replaceItems;
}
set
{
_replaceItems = value;
}
}
public PropsClass()
{
TypeDescriptor.AddProvider(new CustumTypeDescriptorProvider(), typeof(List<string>));
}
public void SaveToIniFile(string filename)
{
// write to ini file
using (var fp = new StreamWriter(filename, false, Encoding.UTF8))
{
// for each different section
foreach (var section in GetType()
.GetProperties()
.GroupBy(x => ((CategoryAttribute)x.GetCustomAttributes(typeof(CategoryAttribute), false)
.FirstOrDefault())?.Category ?? "General"))
{
fp.WriteLine(Environment.NewLine + "[{0}]", section.Key);
foreach (var propertyInfo in section.OrderBy(x => x.Name))
{
var converter = TypeDescriptor.GetConverter(propertyInfo.PropertyType);
fp.WriteLine("{0}={1}", propertyInfo.Name, converter.ConvertToInvariantString(propertyInfo.GetValue(this, null)));
}
}
}
}
public void ReadFromIniFile(string filename)
{
// Load all sections from file
var loaded = GetType().GetProperties()
.Select(x => ((CategoryAttribute)x.GetCustomAttributes(typeof(CategoryAttribute), false).FirstOrDefault())?.Category ?? "General")
.Distinct()
.ToDictionary(section => section, section => GetKeys(filename, section));
//var loaded = GetKeys(filename, "General");
foreach (var propertyInfo in GetType().GetProperties())
{
var category = ((CategoryAttribute)propertyInfo.GetCustomAttributes(typeof(CategoryAttribute), false).FirstOrDefault())?.Category ?? "General";
var name = propertyInfo.Name;
if (loaded.ContainsKey(category) && loaded[category].ContainsKey(name) && !string.IsNullOrEmpty(loaded[category][name]))
{
var rawString = loaded[category][name];
var converter = TypeDescriptor.GetConverter(propertyInfo.PropertyType);
if (converter.IsValid(rawString))
{
propertyInfo.SetValue(this, converter.ConvertFromString(rawString), null);
}
}
}
}
// helper function
private Dictionary<string, string> GetKeys(string iniFile, string category)
{
var buffer = new byte[8 * 1024];
GetPrivateProfileSection(category, buffer, buffer.Length, iniFile);
var tmp = Encoding.UTF8.GetString(buffer).Trim('\0').Split('\0');
return tmp.Select(x => x.Split(new[] { '=' }, 2))
.Where(x => x.Length == 2)
.ToDictionary(x => x[0], x => x[1]);
}
}
public class CustumTypeDescriptorProvider : TypeDescriptionProvider
{
public override ICustomTypeDescriptor GetTypeDescriptor(System.Type objectType, object instance)
{
if (objectType.Name == "List`1") return new StringListDescriptor();
return base.GetTypeDescriptor(objectType, instance);
}
}
public class StringListDescriptor : CustomTypeDescriptor
{
public override TypeConverter GetConverter()
{
return new StringListConverter();
}
}
public class StringListConverter : TypeConverter
{
public override bool CanConvertFrom(ITypeDescriptorContext context, Type sourceType)
{
if (sourceType == typeof(string))
{
return true;
}
return base.CanConvertFrom(context, sourceType);
}
public override object ConvertFrom(ITypeDescriptorContext context, CultureInfo culture, object value)
{
string s = value as string;
if (!string.IsNullOrEmpty(s))
{
return ((string)value).Split(',').ToList();
}
return base.ConvertFrom(context, culture, value);
}
public override object? ConvertTo(ITypeDescriptorContext? context, CultureInfo? culture, object? value, Type destinationType)
{
if (destinationType == typeof(string))
{
return string.Join(",", (List<string>)value);
}
return base.ConvertTo(context, culture, value, destinationType);
}
}
*NOTE: In my testing, the method ConvertFrom is called twice, once from converter.IsValid and once from propertyInfo.SetValue.
*NOTE2: You are using streamwriter to update the ini file. Since you are using GetPrivateProfileSection, it seems you should be using WritePrivateProfileSection to update the ini file.
*NOTE3: please consider the original comments in question about whether you should be using this method to read/write to an ini file. These methods have been around a long time. (https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-getprivateprofilesection)
*NOTE4: A lot of this answer came from: Convert string to Array of string using TypeConverter
Leaving apart the PropertyGrid part which is just a way to put the info in a cartesian way on a Windows Form, the tricky part as I understand it, is to force a ini file (which is typically a 1:1 correlation between strings and strings) to house a collection of right hand side (rhs) values. Obviously, they cannot reside of a left hand side (lhs) of the ini line. You could, but then likely the choice of a ini file to persist your data is wrong at the roots.
So, why not avoiding to reinvent the wheel and go straight to custom parsing? This way you can also choose the separator to use (you may like to mimic windows, or apache setting files, etc).
Look at this implementation of one old code of mine. Grossly, it is an ini file telling my app which web exchange to contact for crypto trading. Before you get into a big headache with names: Monday is the name of the library holding the common code, and Friday is an app using Monday for trading online with a specific algo. Nothing important for the case being.
The ini file looks like this:
[Friday]
; if true the permutations will be only defined starting from the currency provided with index 0 in [Friday-Currencies]
MaxChainTransactions = 7
; if UseLastTradeValues is true, it will override UseConservativeBidAskValues
UseLastTradeValues = true
; if true the matrix values are (Ask+Bid)/2. This flag overrides UseConservativeBidAskValues but gets overridden by UseLastTradeValues
UseMidPriceValues = false
; if true the most conservative value between bid and ask will be used to calculate gains. Used only if UseLastTradeValues is false
UseConservativeBidAskValues = true
; if true, the solver will also solve with a matrix that will contain the best trade values from the available exchanges in parallel to other exchanges individually
MergeExchanges = false
; If true, Friday will issue one solution (if any) for each exchange, comprising one for the merged market if calculated.
DeliverOneSolutionPerExchange = true
; use values from the provided exchanges only
FridayExchanges = Bittrex| Binance | Kraken
; the minimum allowed gain remaining at the closure of the transaction chain to approve the full chain for ordering
MimimumAllowedGain = 1.05
; this is the target gain of a transaction. it is not used in case AllowPriceCorrectionBelowTargetGain is false
TargetGain = 1.09
; if true the prices in the transaction will be modified (increased towards the aggressive trade) of a factor equal to MaxPriceCorrectionFactor
AllowPriceCorrectionBelowTargetGain = true
; the frequency of each solution cycle (provided that the previous cycle has already completed and new data is arrived)
SolveEveryNSeconds = 15
Look at the line:
FridayExchanges = Bittrex| Binance | Kraken
it receives more than one exchange as a string separated by a pipe (|). You may choose other separators.
I suppose you parse the ini file with the support of some third party code. I used ini-parser (https://github.com/rickyah/ini-parser) that helped me a lot.
Parsing is done this way:
private static (int, string[]) ReadFridayExchanges()
{
string? configEntry = MondayConfiguration.MondaySettings["Friday"]["FridayExchanges"];
List<string> fridayExchanges = configEntry.ParseExchangeKey();
return (fridayExchanges.Count, fridayExchanges.ToArray());
}
Leave apart all the variables and names, the interesting one is MondaySettings. It is defined like this:
using IniParser;
using IniParser.Model;
/// <summary>
/// Holds the dictionary of Settings in IniData format.
/// </summary>
public static IniData MondaySettings { get; private set; }
and initialized like this:
[MemberNotNull(nameof(MondaySettings))]
private static void ReadSettingsFile(string settingsFile)
{
var iniParser = new FileIniDataParser();
MondaySettings = iniParser.ReadFile(settingsFile);
if (!DateTime.TryParse(MondaySettings["General"]["LastShutDown"], out _))
{
CorrectLastShutDownTime();
}
}
you find all nice calls in the ini-parser package to read and write the ini files automatically with almost one-liners.
When saving the file before closing and exiting:
private static bool SaveInternalSettings(string settingsFilename)
{
string settingsFile = Path.Combine(SettingsLocationFullPath, settingsFilename);
MondaySettings["General"]["LastShutDown"] = DateTime.UtcNow.ToString(CultureInfo.InvariantCulture);
File.Delete(settingsFile);
try
{
var parser = new FileIniDataParser();
parser.WriteFile(settingsFile, MondaySettings);
return true;
}
catch
{
return false;
}
}
Now the interesting part: parsing multiple rhs values within ini. My solution was to do it manually, which is also one of the fastest ways.
/// <summary>
/// Reading Settings.ini for multi value lines, this routine parses the right hand side
/// of each records to define the list of values coupling it with the required
/// exchanges. Returns an array of exchanges that may be contained in a complex key to
/// resolve the All case and the presence of the pipe symbol (|).
/// NOTE: it makes use of the indexes for markets, therefore those structures should be
/// ready before this call to the method.
/// </summary>
/// <param name="rhrMarket">
/// The original key as written in the .ini file.
/// </param>
/// <returns>
/// A list of strings representing all the exchanges in a single record of
/// InterestingMarkets section the settings. If only one is contained in the original
/// key, a single element is contained.
/// </returns>
public static List<string> ParseExchangeKey(this string rhrMarket)
{
List<string> answer = new();
if (string.Equals(rhrMarket, "all", StringComparison.OrdinalIgnoreCase))
{
// producing all known exchanges as answer
foreach (KeyValuePair<string, int> ex in Tags.Exchanges.ActiveExchanges)
{
if (ex.Value != 0) // to exclude Monday which is id=0
{
answer.Add(ex.Key);
}
}
}
else if (rhrMarket.Contains('|'))
{
// sorting multiple cases
string[] split = rhrMarket.Split('|');
foreach (string? subElement in split)
{
answer.Add(subElement.ToLower().Trim());
}
}
else
{
answer.Add(rhrMarket.ToLower());
}
return answer;
}
There is also some XML to help you get to the point. As you can see, ParseExchangeKey returns a list of strings.
Just to make sure you have all the elements to get the Ienumerables, this is the definition of Tags.Exchanges.ActiveExchanges:
/// <summary>
/// Contains the set of active exchanges included into [ActiveExchanges] section in
/// settings.ini and having proper credentials to activate the web account. The
/// Values of the dictionary are the ID in _allExchanges. Monday is not included.
/// </summary>
[PublicAPI]
public static Dictionary<string, int> ActiveExchanges => _b_activeExchanges_;
/// <summary>
/// IMPL: This is internal because it needs to be set by <see cref="MondayConfiguration"/>.
/// it does not contain Monday's entry.
/// </summary>
private static readonly Dictionary<string, int> _b_activeExchanges_ = LoadExchangesDictionary();
private static Dictionary<string, int> LoadExchangesDictionary()
{
Dictionary<string, int> answer = new();
// set it internally without using the property which blocks the caller up the
// closure of the dictionary.
var counter = 0;
foreach (KeyValuePair<string, int> item in _allExchanges_)
{
if (item.Value != 0 && MondayConfiguration.Credentials.Any(c => c.Exchange == item.Key))
{
answer.Add(item.Key.ToLower(), item.Value);
counter++;
}
}
Saving is the same, however ini-parser leaves the original string with separators (pipes in this case) in its memory, if your used modified the set of multi-values, you just need to provide a simple ToString() version to concatenate it back before saving the file using the accustomed separator.
Then coupling with the controls of the Windows Forms is easy when you can get the List<string> object to move around.
This way you may skip all the custom classes for converting the type around which will likely slow down maintenance when you need to add or remove records from the ini file.
I'm creating a new feature here in my application to cache using AOP.
When my method is called using RedisCacheableResultAttribute I will cache / retrieve the data.
I'm successfully storing, don't know it's the correct way serializing and storing as string, but when I return the object from Redis I need convert to the type I stored at the first time.
I imagine that I can do it with reflection but no success, i'm not very experienced with reflection.
public sealed override void OnInvoke(MethodInterceptionArgs args)
{
var cache = RedisConnectorHelper.Connection.GetDatabase();
var result = cache.StringGet(args.Method.Name); // retrieving from Redis
if (result.HasValue)
{
args.ReturnValue = result; // here I need to convert back to the type I received when I stored.
return;
}
base.OnInvoke(args);
cache.StringSet(args.Method.Name, Serialize(args.ReturnValue)); //storing the data OK.
}
private string Serialize(object obj)
{
return JsonConvert.SerializeObject(obj);
}
No need for reflection here, just use JsonSerializerSettings to save your type in data blob:
private static readonly JsonSerializerSettings _settings = new JsonSerializerSettings { TypeHandling = TypeHandling.Object };
Then put it into serialization and deserialization methods:
private static string Serialize(object obj)
{
return JsonConvert.SerializeObject(obj, _settings);
}
private static object Deserialize(string data)
{
return JsonConvert.DeserializeObject(data, _settings);
}
Then in code:
public sealed override void OnInvoke(MethodInterceptionArgs args)
{
var cache = RedisConnectorHelper.Connection.GetDatabase();
var result = cache.StringGet(args.Method.Name); // retrieving from Redis
if (result.HasValue)
{
args.ReturnValue = Deserialize(result);
return;
}
base.OnInvoke(args);
cache.StringSet(args.Method.Name, Serialize(args.ReturnValue));
}
For faster solution, I recommend you to use another Newtonsoft format called BSON (binary json), it will convert your object into byte array considerably faster than to string. So as long as you don't care about readability, it will be wise to use it.
I did not test it, so can be compilation errors, but Newtonsoft has their ways to save type, so you should not bother about it.
Fixed the problem with the following solution:
In the input method where I am using the Apsect I changed the types to Dynamic. Using object works too.
[RedisCacheableResult]
public List<dynamic> ReturnCustomer()
{
var lstCustomer = new List<dynamic>();
var customer = new Customer
{
Id = 1,
Name = "Acme Inc",
Email = "acme#email.com"
};
var customer1 = new Customer
{
Id = 2,
Name = "Marvel Inc",
Email = "Marvel#email.com"
};
lstCustomer.Add(customer);
lstCustomer.Add(customer1);
return lstCustomer;
}
On Deserialize:
private static dynamic Deserialize(string data)
{
return JsonConvert.DeserializeObject<List<dynamic>>(data, Settings);
}
Segue a solução publicada no GitHub: https://github.com/thiagoloureiro/TestaCache
Issues, pullrequests, stars and forks are welcome :)
I'm learning my way around EF, and I know caching is faster than a round trip to the DB for things like state, country, etc. But I'm not sure how to implement it. I was looking at this post (entities from local cache) that mentioned an extension, but is there something built in I should leverage?
I'd like to have a function like this that wouldn't have to go to the db every time:
public static int GetCountryId(string countryCode = "US")
{
if (countryCode == string.Empty)
countryCode = "US"; // Assume US
return db.Country.Where
(
p => p.CountryCode == countryCode
).Select(p => p.CountryId).First();
}
Updated:
I'm now thinking about a GetCached function that would use generics to hold some lookup lists in memory. Somthing like this:
public class Lookups
{
private static MkpContext db = new MkpContext();
public static IEnumerable GetCached(CachedLists list)
{
ObjectCache cache = MemoryCache.Default;
var listOut = cache[list.ToString()] as IEnumerable;
if (listOut != null) return listOut;
switch (list)
{
case CachedLists.Countries:
cache.Set
(
list.ToString(),
db.Country.ToList(),
new DateTimeOffset(DateTime.Now,new TimeSpan(1,0,0,0))
);
break;
default:
return null;
}
listOut = cache[list.ToString()] as IEnumerable;
return listOut;
}
}
public enum CachedLists
{
Countries,
}
But the above solution would leave me with an un-typed Enumerable. I'd love to be able to specify the types somehow, or better yet, do some sort of extension.
There are a lot of options, but here's one approach that will work well if users are mostly querying the same few country codes:
Create a MemoryCache instance to use as a static, private, readonly field on your class. In your method, try to get a cached item from this cache if there is one with the given countryCode as its key. If it's not there, do your database query and Add the result into the cache before returning it.
By the way, the approach in the article you linked probably isn't a very good approach: it will only help if you've already got data in the specific database context instance that you're dealing with, and usually it's best for your contexts to be short-lived. It's also really CPU-intensive to compile an expression into a function, and then run that function against every entity that the context has cached, just to find out whether the item is there or not. If you then find out that it's not there, and you have to go back to the database anyway, you just wasted time and resources.
The method you have proposed is usually what I do to return cached data.
However data is cached as an object and therefor you have to return it as the object you expect.
Lets assume that your country class is represented by:
public class Country
{
#region Constructors
public Country(string code, string name)
{
this.Code = code;
this.Name = name;
}
#endregion
#region Properties
public string Name { get; set; }
public string Code { get; set; }
#endregion
}
What we need to do is look up our cache by a given key, if exists return the cache, otherwise get from database - same logic as you did.
Difference is here var listOut = cache[list.ToString()] as IEnumerable;
You want to check whether the value for that key is of type Country
If we define a GetCache method as following:
static object GetCache(string cacheKey)
{
if (Cache[cacheKey] is object cachedResult)
{
return cachedResult;
}
return null;
}
What we need to do in order to return a List<Country> is
if (GetCache(cacheKey) is List<Country> cachedData)
{
return cachedData;
}
Now instead of a list of object we have a List<Country>
I have made a simple console app to show the result - hope it helps:
namespace ConsoleApp3
{
#region Usings
using System;
using System.Collections.Generic;
using System.Runtime.Caching;
#endregion
class Program
{
#region Fields
private static readonly ObjectCache Cache = MemoryCache.Default;
#endregion
static void Main(string[] args)
{
//simulates app life span
for (int i = 0; i < 5; i++)
{
var countries = GetData();
foreach (var country in countries)
{
Console.WriteLine(country.Name);
}
Console.ReadLine();
}
}
static List<Country> GetData()
{
string cacheKey = "Country-Lookup";
if (GetCache(cacheKey) is List<Country> cachedData)
{
return cachedData;
}
// otherwise do some logic stuff and get from DB
//db data simulation
List<Country> coutries = new List<Country>
{
new Country("IT", "Italy"),
new Country("UK", "United Kindom"),
new Country("US", "United States")
};
//add to cache
AddToCache(cacheKey, coutries);
return coutries;
}
static object GetCache(string cacheKey)
{
if (Cache[cacheKey] is object cachedResult)
{
return cachedResult;
}
return null;
}
static void AddToCache(string cacheKey, object dataToCache)
{
if (dataToCache == null)
{
return;
}
Cache.Set(cacheKey, dataToCache, DateTimeOffset.Now.AddMinutes(1));
}
}
}
You can use EntityFramework Plus.
public static int GetCountryId(string countryCode = "US")
{
return db.Country.Where
(
p => p.CountryCode == countryCode
).Select(p => p.CountryId).First();
}
Would probably become...
public static int GetCountryId(string countryCode = "US")
{
using var context = new Context();
var allCountries = context.Country
.FromCache({cachePolicy})
.ToDictionary(x => x.CountryCode);
return allCountries[countryCode];
}
For something basic like mapping a country code to a countryId I'd suggest keeping it simple and using a dictionary. If this is happening inside a web application you'll want to use a ConcurrentDictionary to handle multiple threads hitting the code, otherwise a normal dictionary will be fine.
You could also have some code that populates the dictionary when the app starts up to make the experience even snappier for users.
static ConcurrentDictionary<string, int> countryLookup = new ConcurrentDictionary<string, int>();
public static int GetCountryId(string countryCode = "US")
{
if (countryCode == string.Empty)
countryCode = "US"; // Assume US
if (countryLookup.TryGetValue(countryCode, out int countryId))
return countryId;
var countryId = db.Country
.First(p => p.CountryCode == countryCode)
.CountryId;
countryLookup.TryAdd(countryCode, countryId);
return countryId;
}
I have this (simplified) class:
public class StarBuildParams
{
public int BaseNo { get; set; }
public int Width { get; set; }
}
And I have to transform instances of it to a querystring like this:
"BaseNo=5&Width=100"
Additionally I have to transform such a querystring back in an object of that class.
I know that this is pretty much what a modelbinder does, but I don't have the controller context in my situation (some deep buried class running in a thread).
So, is there a simple way to convert a object in a query string and back without having a controller context?
It would be great to use the modelbinding but I don't know how.
A solution with Newtonsoft Json serializer and linq:
string responseString = "BaseNo=5&Width=100";
var dict = HttpUtility.ParseQueryString(responseString);
string json = JsonConvert.SerializeObject(dict.Cast<string>().ToDictionary(k => k, v => dict[v]));
StarBuildParams respObj = JsonConvert.DeserializeObject<StarBuildParams>(json);
You can use reflection, something like this:
public T GetFromQueryString<T>() where T : new(){
var obj = new T();
var properties = typeof(T).GetProperties();
foreach(var property in properties){
var valueAsString = HttpContext.Current.Request.QueryString[property.PropertyName];
var value = Parse( valueAsString, property.PropertyType);
if(value == null)
continue;
property.SetValue(obj, value, null);
}
return obj;
}
You'll need to implement the Parse method, just using int.Parse, decimal.Parse, DateTime.Parse, etc.
Use this Parse method with the ivowiblo's solution (accepted answer):
public object Parse(string valueToConvert, Type dataType)
{
TypeConverter obj = TypeDescriptor.GetConverter(dataType);
object value = obj.ConvertFromString(null, CultureInfo.InvariantCulture, valueToConvert);
return value;
}
You can set the properties of this object in its constructor by retrieving the relevant values from the querystring
public StarBuildParams()
{
this.BaseNo = Int32.Parse(Request.QueryString["BaseNo"].ToString());
this.Width = Int32.Parse(Request.QueryString["Width"].ToString());
}
and you can ensure that the object is converted to the correct querystring format by overriding the ToString method.
public override string ToString()
{
return String.Format("BaseNo={0}&Width={1}", this.BaseNo, this.Width);
}
You'll still need to construct and call ToString in the appropriate places, but this should help.
You can just use .NET's HttpUtility.ParseQueryString() method:
HttpUtility.ParseQueryString("a=b&c=d") produces a NameValueCollection as such:
[0] Key = "a", Value = "b"
[1] Key = "c", Value = "d"
This should work so long as none of the properties match any other route parameters like controller, action, id, etc.
new RouteValueDictionary(Model)
http://msdn.microsoft.com/en-us/library/cc680272.aspx
Initializes a new instance of the RouteValueDictionary class and adds
values that are based on properties from the specified object.
To parse back from the query string you can use the model class as an action parameter and let the ModelBinder do it's job.
Serialize query string and deserialize to your class object
JObject json;
Request.RequestUri.TryReadQueryAsJson(out json);
string sjson = JsonConvert.SerializeObject(json);
StarBuildParams query = JsonConvert.DeserializeObject<StarBuildParams>(sjson);
Building off of Ivo and Anupam Singh's great solutions above, here is the code that I used to turn this into a base class for POST requests (in the event that you may only have the raw query string like in a Web API setup). This code works for lists of objects, but could easily be modified to parse a single object.
public class PostOBjectBase
{
/// <summary>
/// Returns a List of List<string> - one for each object that is going to be parsed.
/// </summary>
/// <param name="entryListString">Raw query string</param>
/// <param name="firstPropertyNameOfObjectToParseTo">The first property name of the object that is sent in the list (unless otherwise specified). Used as a key to start a new object string list. Ex: "id", etc.</param>
/// <returns></returns>
public List<List<string>> GetQueryObjectsAsStringLists(string entryListString, string firstPropertyNameOfObjectToParseTo = null)
{
// Decode the query string (if necessary)
string raw = System.Net.WebUtility.UrlDecode(entryListString);
// Split the raw query string into it's data types and values
string[] entriesRaw = raw.Split('&');
// Set the first property name if it is not provided
if (firstPropertyNameOfObjectToParseTo == null)
firstPropertyNameOfObjectToParseTo = entriesRaw[0].Split("=").First();
// Create a list from the raw query array (more easily manipulable) for me at least
List<string> rawList = new List<string>(entriesRaw);
// Initialize List of string lists to return - one list = one object
List<List<string>> entriesList = new List<List<string>>();
// Initialize List for current item to be added to in foreach loop
bool isFirstItem = false;
List<string> currentItem = new List<string>();
// Iterate through each item keying off of the firstPropertyName of the object we will ultimately parse to
foreach (string entry in rawList)
{
if (entry.Contains(firstPropertyNameOfObjectToParseTo + "="))
{
// The first item needs to be noted in the beginning and not added to the list since it is not complete
if (isFirstItem == false)
{
isFirstItem = true;
}
// Finished getting the first object - we're on the next ones in the list
else
{
entriesList.Add(currentItem);
currentItem = new List<string>();
}
}
currentItem.Add(entry);
}
// Add the last current item since we could not in the foreach loop
entriesList.Add(currentItem);
return entriesList;
}
public T GetFromQueryString<T>(List<string> queryObject) where T : new()
{
var obj = new T();
var properties = typeof(T).GetProperties();
foreach (string entry in queryObject)
{
string[] entryData = entry.Split("=");
foreach (var property in properties)
{
if (entryData[0].Contains(property.Name))
{
var value = Parse(entryData[1], property.PropertyType);
if (value == null)
continue;
property.SetValue(obj, value, null);
}
}
}
return obj;
}
public object Parse(string valueToConvert, Type dataType)
{
if (valueToConvert == "undefined" || valueToConvert == "null")
valueToConvert = null;
TypeConverter obj = TypeDescriptor.GetConverter(dataType);
object value = obj.ConvertFromString(null, CultureInfo.InvariantCulture, valueToConvert);
return value;
}
}
Then you can inherit from this class in wrapper classes for POST requests and parse to whichever objects you need. In this case, the code parses a list of objects passed as a query string to a list of wrapper class objects.
For example:
public class SampleWrapperClass : PostOBjectBase
{
public string rawQueryString { get; set; }
public List<ObjectToParseTo> entryList
{
get
{
List<List<string>> entriesList = GetQueryObjectsAsStringLists(rawQueryString);
List<ObjectToParseTo> entriesFormatted = new List<ObjectToParseTo>();
foreach (List<string> currentObject in entriesList)
{
ObjectToParseToentryPost = GetFromQueryString<ObjectToParseTo>(currentObject);
entriesFormatted.Add(entryPost);
}
return entriesFormatted;
}
}
}
I need to persist in Session some data.
I wrote many properties like that:
public List<string> FillOrder
{
get { return Session[SessionKeys.QueryFillOrder] as List<string> ?? new List<string>(); }
set { Session[SessionKeys.QueryFillOrder] = value; }
}
When I have to consume this data I have to write code like that:
List<string> fillOrder = FillOrder;
fillOrder.Add(accordion.ID);
FillOrder = fillOrder;
that seems to me so ugly, because I would prefer to do that:
FillOrder.Add(accordion.ID);
but this way my value would not be saved back in Session.
Can you think of any better way to achieve the same result?
Thank you very much!
I always use a wrapper class around the ASP.NET session to simplify access to session variables:
public class MySession
{
// private constructor
private MySession()
{
FillOrder = new List<string>();
}
// Gets the current session.
public static MySession Current
{
get
{
var session = (MySession)HttpContext.Current.Session["__MySession__"];
if (session == null)
{
session = new MySession();
HttpContext.Current.Session["__MySession__"] = session;
}
return session;
}
}
// **** add your session properties here, e.g like this:
public List<string> FillOrder {get; set; }
public string Property1 { get; set; }
public DateTime MyDate { get; set; }
public int LoginId { get; set; }
}
This class stores one instance of itself in the ASP.NET session and allows you to access your session properties in a type-safe way from any class, e.g like this:
MySession.Current.FillOrder.Add(accordion.ID);
int loginId = MySession.Current.LoginId;
string property1 = MySession.Current.Property1;
MySession.Current.Property1 = newValue;
DateTime myDate = MySession.Current.MyDate;
MySession.Current.MyDate = DateTime.Now;
This approach has several advantages:
you can initialize your session variables in the constructor (i.e. new List<string>)
it saves you from a lot of type-casting
you don't have to use hard-coded session keys throughout your application (e.g. Session["loginId"]
you can document your session items by adding XML doc comments on the properties of MySession
You can use an extension method as well, but I do think the example by M4N might be better:
EDIT made it a generic type
public static class Extensions
{
public static void AddWithSession<T>(this List<T> list, T value, string key)
{
list.Add(value);
HttpContext.Current.Session[key] = list;
}
}
str.AddWithSession(accordion.ID,SessionKeys.QueryFillOrder);
You could write an own class that implements ICollection or IList, there you would implement Add as Session[...] = ...
Using a single class for all Session variables as suggested by #M4N is a good idea, though it risks becoming a "God" class (in which case you could partition into several classes implemented in this way).
However you could just change your property implemetation as follows:
public List<string> FillOrder
{
get
{
List<string> result = Session[SessionKeys.QueryFillOrder] as List<string>;
if (result == null)
{
result = new List<string>();
Session[SessionKeys.QueryFillOrder] = result;
}
return result;
}
set { Session[SessionKeys.QueryFillOrder] = value; }
}
In this example, you probably don't want a setter.