Develop Reference

Strange Issue With GUID Search using C#

I'm trying to learn to use ABP with Blazor and MongoDB. I've got a set of objects in a database that I display as a list but I also want to be able to retrieve objects from the database by the object's GUID and this is where I'm running into problems.
This is the task I wrote to retrieve objects based on a filter from the database:
private async Task GetObjectAsync(string filter)
{
var result = await ObjectAppService.GetListAsync(
new GetObjectListDto
{
MaxResultCount = PageSize,
SkipCount = CurrentPage * PageSize,
Sorting = CurrentSorting,
Filter = filter
}
);
ObjectList = result.Items;
TotalCount = (int)result.TotalCount;
errorVal = TotalCount.ToString();
This is my GetListAsync() method where I retrieve objects from the database according to the filter string:
public async Task<PagedResultDto<ObjectDto>> GetListAsync(GetObjectListDto input)
{
if (input.Sorting.IsNullOrWhiteSpace())
{
input.Sorting = nameof(Object.Payload);
}
var objects = await _objectRepository.GetListAsync(
input.SkipCount,
input.MaxResultCount,
input.Sorting,
input.Filter
);
var totalCount = input.Filter == null
? await _objectRepository.CountAsync()
: await _objectRepository.CountAsync(
** SEARCH METHODS GO HERE **
return new PagedResultDto<ObjectDto>(
totalCount,
ObjectMapper.Map<List<Object>, List<ObjectDto>>(objects)
);
}
These are the search methods I've been trying:
object => object.Name.Contains(input.Filter));
object => object.Id.ToString().Contains(input.Filter));
object => object.Id.Equals(Guid.Parse(input.Filter)));
The first search method (search by name and return any DB results which contain it) works perfectly except for the fact I want to be able to search by GUID. The second search method would be ideal except it throws an exception when I use it:
(System.ArgumentException: Unsupported filter: {document}{_id}.ToString().Contains("70f35018-a504-89f1-51c0-3a04cb49cb97").).
The third search method definitely retrieves something from the database (I have verified) but it doesn't display this result on the page for some reason and I obviously have to enter a complete GUID to return the object (not ideal).
I'm sure this is possible and I'm doing something wrong here but I'm not sure what that is. Can anyone tell me the correct way to search by GUID here? I expected Id.ToString().Contains(Filter) to work.
UPDATE:
I've sort of solved this problem - I needed to look at the MongoObjectRepository.cs file:
public async Task<List<Object>> GetListAsync(
int skipCount,
int maxResultCount,
string sorting,
string filter = null)
{
var queryable = await GetMongoQueryableAsync();
return await queryable
.WhereIf<Object, IMongoQueryable<Object>>(
!filter.IsNullOrWhiteSpace(),
obj => obj.Id.Equals(filter)
)
.OrderBy(sorting)
.As<IMongoQueryable<Object>>()
.Skip(skipCount)
.Take(maxResultCount)
.ToListAsync();
}
For some reason, I was filtering based on the Payload property rather than Id.
I'm still confused, however, why obj => obj.Id.ToString().Contains((filter)) doesn't work - is this not allowed when retrieving items from a database? I've tried this on DateTimeOffset fields as well (e.g. obj => obj.InsertTimestamp.ToString().Contains((filter))) and get similar problems. When I try to add the .FirstOrDefault() to the end of the .WhereIf<Object, IMongoQueryable<Object>>() statement it invalidates the function (has to be chained with all the other methods - maybe that's the problem there?)

Dapper provide default name for dynamic result sets with QueryMultiple

TLDR; Is there a way (using a type map or some other solution) to give dynamic result sets a default name, such as "(No Column Name)" in Dapper when no column name is supplied?
I am writing a query editor that allows users to write and run user-supplied queries against MS SQL Server databases. I've been using Dapper for all our querying and it has been working beautifully for 99% of what we need. I've hit a snag though and I'm hoping someone has a solution.
The query editor is similar to SSMS. I don't know ahead of time what the script will look like, what the shape or type the result set(s) will be, or even how many result sets will be returned. For this reason, I've been batching the scripts and using Dapper's QueryMultiple to read dynamic results from the GridReader. The results are then sent to a third party UI data grid (WPF). The data grid knows how to consume dynamic data and the only thing it requires to display a given row is at least one key value pair with a non-null, but not necessarily unique key and a nullable value. So far, so good.
The simplified version of the Dapper call looks something like this:
public async Task<IEnumerable<IEnumerable<T>>> QueryMultipleAsync<T>(string sql,
object parameters,
string connectionString,
CommandType commandType = CommandType.Text,
CancellationTokenSource cancellationTokenSource = null)
{
using (IDbConnection con = _dbConnectionFactory.GetConnection(connectionString))
{
con.Open();
var transaction = con.BeginTransaction();
var sqlBatches = sql
.ToUpperInvariant()
.Split(new[] { " GO ", "\r\nGO ", "\n\nGO ", "\nGO\n", "\tGO ", "\rGO "}, StringSplitOptions.RemoveEmptyEntries);
var batches = new List<CommandDefinition>();
foreach(var batch in sqlBatches)
{
batches.Add(new CommandDefinition(batch, parameters, transaction, null, commandType, CommandFlags.Buffered, cancellationTokenSource.Token));
}
var resultSet = new List<List<T>>();
foreach (var commandDefinition in batches)
{
using (GridReader reader = await con.QueryMultipleAsync(commandDefinition))
{
while (!reader.IsConsumed)
{
try
{
var result = (await reader.ReadAsync<T>()).AsList();
if (result.FirstOrDefault() is IDynamicMetaObjectProvider)
{
(result as List<dynamic>).ConvertNullKeysToNoColumnName();
}
resultSet.Add(result);
}
catch(Exception e)
{
if(e.Message.Equals("No columns were selected"))
{
break;
}
else
{
throw;
}
}
}
}
}
try
{
transaction.Commit();
}
catch (Exception ex)
{
Trace.WriteLine(ex.ToString());
if (transaction != null)
{
transaction.Rollback();
}
}
return resultSet;
}
}
public static IEnumerable<dynamic> ConvertNullKeysToNoColumnName<dynamic>(this IEnumerable<dynamic> rows)
{
foreach (var row in rows)
{
if (row is IDictionary<string, object> rowDictionary)
{
if (rowDictionary == null) continue;
rowDictionary.Where(x => string.IsNullOrEmpty(x.Key)).ToList().ForEach(x =>
{
var val = rowDictionary[x.Key];
if (x.Value == val)
{
rowDictionary.Remove(x);
rowDictionary.Add("(No Column Name)", val);
}
else
{
Trace.WriteLine("Something went wrong");
}
});
}
}
return rows;
}
This works with most queries (and for queries with only one unnamed result column), but the problem manifests when the user writes a query with more than one unnamed column like this:
select COUNT(*), MAX(create_date) from sys.databases.
In this case, Dapper returns a DapperRow that looks something like this:
{DapperRow, = '9', = '2/14/2020 9:51:54 AM'}
So the result set is exactly what the user asks for (i.e., values with no names or aliases) but I need to supply (non-unique) keys for all data in the grid...
My first thought was to simply change the null keys in the DapperRow object to a default value (like '(No Column Name)'), as it appears to be optimized for storage so table keys are only stored once in the object (which is nice and would provide a nice performance bonus for queries with huge result sets). The DapperRow type is private though. After searching around, I found that I could cast the DapperRow to an IDictionary<string, object> to access keys and values for the object, and even set and remove values. That's where the ConvertNullKeysToNoColumnName extension method comes from. And it works... But only once.
Why? Well, it appears that when you have multiple null or empty keys in a DapperRow that gets cast to an IDictionary<string,object> and you call the Remove(x) function (where x is the entire item OR just the key for any single item with a null or empty key), all subsequent attempts to resolve other values with a null or empty key via the indexer item[key] fail to retrieve a value--even if the additional key value pairs still exist in the object.
In other words, I can't remove or replace subsequent empty keys after the first one is removed.
Am I missing something obvious? Do I just need to alter the DapperRow via reflection and hope it doesn't have any weird side affects or that the underlying data structure doesn't change later? Or do I take the performance/memory hit and just copy/map the entire potentially large result set into a new sequence to give empty keys a default value at runtime?

I suspect this is because the dynamic DapperRow object is actually not a 'normal' dictionary. It can have several entries with the same key. You can see this if you inspect the object in the debugger.
When you reference rowDictionary[x.Key], I suspect you will always get the first unnamed column.
If you call rowDictionary.Remove(""); rowDictionary.Remove("");, you actually only remove the first entry - the second is still present, even though rowDictionary.ContainsKey("") returns false.
You can Clear() and rebuild the entire dictionary.
At that point, you're really not gaining much by using a dynamic object.
if (row is IDictionary<string, object>)
{
var rowDictionary = row as IDictionary<string, object>;
if (rowDictionary.ContainsKey(""))
{
var kvs = rowDictionary.ToList();
rowDictionary.Clear();
for (var i = 0; i < kvs.Count; ++i)
{
var kv = kvs[i];
var key = kv.Key == ""? $"(No Column <{i + 1}>)" : kv.Key;
rowDictionary.Add(key, kv.Value);
}
}
}
Since you're working with unknown result structure, and just want to pass it to a grid view, I would consider using a DataTable instead.
You can still keep Dapper for parameter handling:
foreach (var commandDefinition in batches)
{
using(var reader = await con.ExecuteReaderAsync(commandDefinition)) {
while(!reader.IsClosed) {
var table = new DataTable();
table.Load(reader);
resultSet.Add(table);
}
}
}

Extension method with dynamic named parameters

I'm currently writing an extension to replace the normal string.Format with my FormatNamed-function.
So far I have got this code, but I want to change the way to input the parameters
void Main()
{
string sql = "SELECT {fields} FROM {table} WHERE {query}"
.FormatNamed(new { fields = "test", table = "testTable", query = "1 = 1" });
Console.WriteLine(sql);
}
public static class StringExtensions
{
public static string FormatNamed(this string formatString, dynamic parameters)
{
var t = parameters.GetType();
var tmpVal = formatString;
foreach(var p in t.GetProperties())
{
tmpVal = tmpVal.Replace("{" + p.Name + "}", p.GetValue(parameters));
}
return tmpVal;
}
}
Not the prettiest of replaces, but it does the job.
Anyway. I want to change so I can execute it with
.FormatName(field: "test", table: "testTable", query: "1 = 1");
Is there any way I can do this? I have tried googling for dynamic named parameters with no good results.

You won't be able to specify an arbitrary number of dynamic, named parameters. that's just not something that C# supports. Your existing code seems okay to me, although I don't see the need for the dynamic parameter. This will work just as well:
public static string FormatNamed(this string formatString, object parameters)
{
var t = parameters.GetType();
var tmpVal = formatString;
foreach(var p in t.GetProperties())
{
tmpVal = tmpVal.Replace("{" + p.Name + "}", p.GetValue(parameters));
}
return tmpVal;
}
And then calling it as:
string sql = "SELECT {fields} FROM {table} WHERE {query}"
.FormatNamed(new { fields = "test", table = "testTable", query = "1 = 1" });
Although I really wouldn't advise using this sort of method for constructing SQL (it won't save you from SQL injection attacks at all), the method itself is sound.

I have tried googling for dynamic named parameters with no good results
That's because the capability does not exist. Think about it - how would the function know what to do if the parameters and their names were not known at compile time? The closest thing I can think of is using params which gives you an array of values, but they must all be the same type, and you can still access them by a given name (and index value).
I'd stick with the method you're currently using:
.FormatName(new {field = "test", table = "testTable", query = "1 = 1"});
That creates an anonymous type with the properties specified, which should work fine with your existing code. Plus it's only a few extra characters to type.
Also note that dynamic doesn't buy you anything here since it's used to access properties directly without using reflection. Since you're using reflection to get the propeties you can just use object.

How to create a custom property in a Linq-to-SQL entity class?

I have two tables Studies and Series. Series are FK'd back to Studies so one Study contains a variable number of Series.
Each Series item has a Deleted column indicating it has been logically deleted from the database.
I am trying to implement a Deleted property in the Study class that returns true only if all the contained Series are deleted.
I am using O/R Designer generated classes, so I added the following to the user modifiable partial class for the Study type:
public bool Deleted
{
get
{
var nonDeletedSeries = from s in Series
where !s.Deleted
select s;
return nonDeletedSeries.Count() == 0;
}
set
{
foreach (var series in Series)
{
series.Deleted = value;
}
}
}
This gives an exception "The member 'PiccoloDatabase.Study.Deleted' has no supported translation to SQL." when this simple query is executed that invokes get:
IQueryable<Study> dataQuery = dbCtxt.Studies;
dataQuery = dataQuery.Where((s) => !s.Deleted);
foreach (var study in dataQuery)
{
...
}
Based on this http://www.foliotek.com/devblog/using-custom-properties-inside-linq-to-sql-queries/, I tried the following approach:
static Expression<Func<Study, bool>> DeletedExpr = t => false;
public bool Deleted
{
get
{
var nameFunc = DeletedExpr.Compile();
return nameFunc(this);
}
set
{ ... same as before
}
}
I get the same exception when a query is run that there is no supported translation to SQL. (
The logic of the lambda expression is irrelevant yet - just trying to get past the exception.)
Am I missing some fundamental property or something to allow translation to SQL? I've read most of the posts on SO about this exception, but nothing seems to fit my case exactly.

I believe the point of LINQ-to-SQL is that your entities are mapped for you and must have correlations in the database. It appears that you are trying to mix the LINQ-to-Objects and LINQ-to-SQL.
If the Series table has a Deleted field in the database, and the Study table does not but you would like to translate logical Study.Deleted into SQL, then extension would be a way to go.
public static class StudyExtensions
{
public static IQueryable<study> AllDeleted(this IQueryable<study> studies)
{
return studies.Where(study => !study.series.Any(series => !series.deleted));
}
}
class Program
{
public static void Main()
{
DBDataContext db = new DBDataContext();
db.Log = Console.Out;
var deletedStudies =
from study in db.studies.AllDeleted()
select study;
foreach (var study in deletedStudies)
{
Console.WriteLine(study.name);
}
}
}
This maps your "deleted study" expression into SQL:
SELECT t0.study_id, t0.name
FROM study AS t0
WHERE NOT EXISTS(
SELECT NULL AS EMPTY
FROM series AS t1
WHERE (NOT (t1.deleted = 1)) AND (t1.fk_study_id = t0.study_id)
)
Alternatively you could build actual expressions and inject them into your query, but that is an overkill.
If however, neither Series nor Study has the Deleted field in the database, but only in memory, then you need to first convert your query to IEnumerable and only then access the Deleted property. However doing so would transfer records into memory before applying the predicate and could potentially be expensive. I.e.
var deletedStudies =
from study in db.studies.ToList()
where study.Deleted
select study;
foreach (var study in deletedStudies)
{
Console.WriteLine(study.name);
}

When you make your query, you will want to use the statically defined Expression, not the property.
Effectively, instead of:
dataQuery = dataQuery.Where((s) => !s.Deleted);
Whenever you are making a Linq to SQL query, you will instead want to use:
dataQuery = dataQuery.Where(DeletedExpr);
Note that this will require that you can see DeletedExpr from dataQuery, so you will either need to move it out of your class, or expose it (i.e. make it public, in which case you would access it via the class definition: Series.DeletedExpr).
Also, an Expression is limited in that it cannot have a function body. So, DeletedExpr might look something like:
public static Expression<Func<Study, bool>> DeletedExpr = s => s.Series.Any(se => se.Deleted);
The property is added simply for convenience, so that you can also use it as a part of your code objects without needing to duplicate the code, i.e.
var s = new Study();
if (s.Deleted)
...

Linq-to-sql logic pain

I've been trying to get the following method cleaned up using more sensible and lean syntax, but I'm striking serious headaches when it comes to aggregate clauses and filtering using L2S. Particularly, I feel I should be able to use a .Contains() method to filter out objects whose tags fit the string parameter passed in the method, but it hasn't worked.
public TagListViewModel GetTagModel(string Name)
{
var model = new TagListViewModel();
var repo = new SimpleRepository("Wishlist");
var ideas = repo.All<Idea>();
List<Idea> ideaList = new List<Idea>();
foreach (Idea i in ideas)
{
var query = from tag in repo.All<Tag>()
join ideatag in repo.All<IdeaTag>()
on tag.ID equals ideatag.TagId
where ideatag.IdeaId == i.ID
select tag;
i.Tags = query.ToList<Tag>();
ideaList.Add(i);
}
foreach (Idea i in ideaList)
{
var query = from vote in repo.All<IdeaVotes>()
where vote.IdeaId == i.ID
select vote;
i.Votes = query.ToList<IdeaVotes>();
}
// Here begins the problem area. I should be able to get a tag from the repo
// whose name matches the "Name" parameter and then call a .Contains() method to
// filter this list, shouldn't I?
List<Idea> filteredTagList = new List<Idea>();
foreach (Idea item in ideaList){
foreach(Tag t in item.Tags)
{
if (t.Name == Name)
filteredTagList.Add(item);
}
}
model.Ideas = filteredTagList;
return model;
}
It's ugly. I know it's ugly but after over 2 hours of playing with several preferred variations I still can't get it to filter the way it's supposed to. Where am I going wrong?

This should be equivalent assuming there are no duplicate tags on a single Idea.
model.Ideas = ideaList.Where(
idea => idea.Tags.Any(
tag => tag.Name == Name)).ToList();