Develop Reference

CREATE FUNCTION for "CleanFile" failed because T-SQL and CLR types for return value do not match

Been scratching my head with this one for a few hours. I've got a handful of CLR DLLs which work ok, but this most recent is giving me a few headaches. Any insight will be helpful.
I've tried several things, such as various string lengths (including nvarchar(MAX)), plus also checked mapping docs
Error Message:
CREATE FUNCTION for "CleanFile" failed because T-SQL and CLR types for return value do not match.
SQL Command I'm running:
CREATE FUNCTION [dbo].[CleanFile](#filename nvarchar(500), #runmode int)
RETURNS nvarchar(500) WITH EXECUTE AS CALLER
AS
EXTERNAL NAME [sqlFileClean].[sqlFileClean].[CleanFile]
C# DLL:
//usings removed for visibility
public class sqlFileClean
{
[Microsoft.SqlServer.Server.SqlFunction]
public static SqlString CleanFile(
SqlString FileName,
SqlInt32 RunMode)
{
try
{
//Code removed here for visibility
}
catch (Exception ex)
{
// return any unhandled error message
return ex.Message;
}
}
};

I made copy/paste your code into new project in VS 2019, then registered assembly (permission_set=SAFE) on MSSQL 2019 - it works fine (in try block just returned some string).
Ensure that your assembly is created from latest and successfully compiled DLL.
If no success - give more details about what you are doing in try { }
If you need to use NVARCHAR(MAX) - you have to use in C# - SqlChars type instead of SqlString.

For those reading this with same problem, I solved the issue by coping the code into a brand new project.

The type initializer for 'System.Data.SqlClient.TdsParser' threw an exception error while calling Stored Procedure from Azure function

I have an Azure Event hub with readings from my smart electricity meter. I am trying to use an Azure Function to write the meter readings to an Azure SQL DB. I have created a target table in the Azure SQL DB and a Stored Procedure to parse a JSON and store the contents in the table. I have successfully tested the stored procedure.
When I call it from my Azure Function however I am getting an error: The type initializer for 'System.Data.SqlClient.TdsParser' threw an exception. For testing purposes, I have tried to execute a simple SQL select statement from my Azure Function, but that gives the same error. I am lost at the moment as I have tried many options without any luck. Here is the Azure function code:
#r "Microsoft.Azure.EventHubs"
using System;
using System.Text;
using System.Data;
using Microsoft.Azure.EventHubs;
using System.Data.SqlClient;
using System.Configuration;
using Dapper;
public static async Task Run(string events, ILogger log)
{
var exceptions = new List<Exception>();
try
{
if(String.IsNullOrWhiteSpace(events))
return;
try{
string ConnString = Environment.GetEnvironmentVariable("SQLAZURECONNSTR_azure-db-connection-meterreadevents", EnvironmentVariableTarget.Process);
using(SqlConnection conn = new SqlConnection(ConnString))
{
conn.Execute("dbo.ImportEvents", new { Events = events }, commandType: CommandType.StoredProcedure);
}
} catch (Exception ex) {
log.LogInformation($"C# Event Hub trigger function exception: {ex.Message}");
}
}
catch (Exception e)
{
// We need to keep processing the rest of the batch - capture this exception and continue.
// Also, consider capturing details of the message that failed to process so it can be processed again later.
exceptions.Add(e);
}
// Once processing of the batch is complete if any messages in the batch failed process throw an exception so that there is a record of the failure.
if (exceptions.Count > 1)
throw new AggregateException(exceptions);
if (exceptions.Count == 1)
throw exceptions.Single();
}
The events coming in are in JSON form as follows
{
"current_consumption":450,
"back_low":0.004,
"current_back":0,
"total_high":13466.338,
"gas":8063.749,
"current_rate":"001",
"total_low":12074.859,
"back_high":0.011,
"timestamp":"2020-02-29 22:21:14.087210"
}
The stored procedure is as follows:
CREATE PROCEDURE [dbo].[ImportEvents]
#Events NVARCHAR(MAX)
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON
-- Insert statements for procedure here
INSERT INTO dbo.MeterReadEvents
SELECT * FROM OPENJSON(#Events) WITH (timestamp datetime2, current_consumption int, current_rate nchar(3), current_back int, total_low numeric(8, 3), back_high numeric(8, 3), total_high numeric(8, 3), gas numeric(7, 3), back_low numeric(8, 3))
END
I have added a connection string of type SQL AZURE and changed {your password} by the actual password in the string. Any thoughts on how to fix this issue or maybe how to get more logging as the error is very general?.

I managed to fix this exception by re-installing Microsoft.Data.SqlClient.SNI. Then clean and rebuild your project.

I managed to fix the issue by changing the Runtime version to ~2 in the Function App Settings.
Does this mean this is some bug in runtime version ~3 or should there be another way of fixing it in runtime version ~3?

I might be late to the party, in my case the cause of the error was "Target runtime" when publishing, I developed on windows machine but was transferring the file to linux, the solution was to change target runtime to the correct one, initial it was win-x64(merely because I started off by deploying locally), see screenshot below

Try to connect to a local SQL, use SQL profiler, and check what you are sending, and what precisely SQL is trying to do with the command being executed.
It's very hard to replicate your code, because, I obviously do not have your Azure SQL :)
So I would suggest, try to execute each step in the Stored procedure, as direct queries.
See if that works, then try to wrap the statements into store-procedures called back-to-back, and get that to work.
Then combine the commands to a single command, and fiddle with it till you get it to work ;)
Get the most simple query to execute towards the Azure SQL, so you are sure your connection is valid. (Like just a simple select on something)
Because without more information, it is very difficult to assist you.

Pretty silly but I got this after installing the EntityFrameworkCore Nuget package but not EntityFrameworkCore.SqlServer Nuget package. I had the SqlServer version for EntityFramework 6 installed.

I had the same error with a VSTO-application that was installed with a double click in the file explorer. Windows copied not all files to such an automatic location somewhere into ProgramData, so the application was simply not complete!
The solution was to register the VSTO-application manualy in HKEY_CURRENT_USER and pointed the "Manifest" to the complete directory with all the files. (like Microsoft.Data.SqlClient.dll, Microsoft.Data.SqlClient.SNI.x64.dll etc)
Those automatically by Windows chosen installations/directories will give unexpected behaviour. :(

Some ws methods ran fine and other would fail with this same error. I ran ws method that was failing in the browser on the box that the ws was being served from and got a lengthy, and helpful error message. One item was an InnerException that said
<ExceptionMessage>Failed to load C:\sites\TXStockChecker.xxxxxx.com\bin\x64\SNI.dll</ExceptionMessage>
I noticed that that file was right were it was expected in my development environment so I copied it to the matching directory on the prod ws and now all the methods run as expected.

Fortify cleanse rule doesn't cleanse everything

My team has recently started using Fortify Static Code Analyzer (version 17.10.0 156) on our .NET code base (C# 6 and VB.NET), and are experiencing some pain with the amount of false positives it reports. For any given issue we can't know if it is a false positive without looking at it, and we don't want any actual problems to get lost in the clutter.
We have a utilities library with a method ReadEmbeddedSql which extracts sql from resources embedded in the assembly to execute. Fortify flags any OracleCommand (from Oracle.ManagedDataAccess.Client) which execute the sql returned from this method with a Sql Injection vulnerability.
This vulnerability is reported at the point the sql is set on the command, be it via constructor, or via the CommandText property.
It does not do this if the ReadEmbeddedSql method is defined in the local assembly.
A pared down listing of the source code which produces this result follows below. In the example code, ExecuteSomeSql() and ExecuteSomeSqlDifferently() are flagged with a vulnerability where ExecuteSomeLocalSql() is not.
For Analysis Evidence it only lists the line the OracleCommand is created:
TestDao.cs:27 - OracleCommand()
RuleID: 31D4607A-A3FF-447C-908A-CA2BBE4CE4B7
in the details for the issue it provides:
On line 27 of TestDao.cs, the method ExecuteSomeSql() invokes a SQL query
built using input coming from an untrusted source. This call could
allow an attacker to modify the statement's meaning or to execute
arbitrary SQL commands.
A sample diagram presented by Fortify for this issue:
After much searching, I came across this post describing a similar problem and proposed solution: Can I register a trusted source for SQL statements
After following the instructions there, and verifying the instructions independently in a different version of the user guide (page 90)
The result is unchanged. I added an additional 'SQL Injection Validation Rule' rule which is specifically described as "... identifies a function that properly validates data before using them in a SQL query."
Still no avail.
EDIT:
I have played around with custom rules more, and have been able to determine that the CustomCleanseRules are actually being applied (they do remove other types of taint), but not removing some trust specific flag Fortify applies to our in-house library.
Every value returned from any method of my libraries is distrusted, and none of the rules I've created seem to be able to remove this distrust.
Is there something I am doing wrong, or does Fortify just not work?
Is there a different type of rule needed to cleanse this general distrust?
Example Source code:
In library:
namespace Our.Utilities.Database
{
public abstract class BaseDao
{
protected string ReadEmbeddedSql(string key)
{
//... extract sql from assembly
return sql;
}
}
}
In application:
namespace Our.Application.DataAccess
{
public class TestDao: Our.Utilities.Database.BaseDao
{
public void ExecuteSomeSql()
{
//... connection is created
// Fortify Does not trust sqlText returned from library method.
var sqlText = ReadEmbeddedSql("sql.for.ExecuteSomeSql");
using(var someSqlCommand = new OracleCommand(sqlText, connection)) // Fortify flags creation of OracleCommand as SqlInjection vulnerability.
{
someSqlCommand.ExecuteNonQuery();
}
}
public void ExecuteSomeSqlDifferently()
{
//... connection is created
// Fortify Does not trust sqlText returned from library method.
var sqlText = ReadEmbeddedSql("sql.for.ExecuteSomeSql");
using(var someSqlCommand = connection.CreateCommand())
{
someSqlCommand.CommandText = sqlText; //Fortify flags setting CommandText as SqlInjection vulnerability.
someSqlCommand.ExecuteNonQuery();
}
}
public void ExecuteSomeLocalSql()
{
//... connection is created
var sqlText = ReadEmbeddedSqlLocallyDefined("sql.for.ExecuteSomeSql");
using(var someSqlCommand = new OracleCommand(sqlText, connection))
{
someSqlCommand.ExecuteNonQuery();
}
}
protected string ReadEmbeddedSqlLocallyDefined(string key)
{
//... extract sql from assembly
return sql;
}
}
}
XML of custom rules:
<?xml version="1.0" encoding="UTF-8"?>
<RulePack xmlns="xmlns://www.fortifysoftware.com/schema/rules">
<RulePackID>5A78FC44-4EEB-49C7-91DA-6564805C3F23</RulePackID>
<SKU>SKU-C:\local\path\to\custom\rules\Our-Utilities</SKU>
<Name><![CDATA[C:\local\path\to\custom\rules\Our-Utilities]]></Name>
<Version>1.0</Version>
<Description><![CDATA[]]></Description>
<Rules version="17.10">
<RuleDefinitions>
<DataflowCleanseRule formatVersion="17.10" language="dotnet">
<RuleID>7C49FEDA-AA67-490D-8820-684F3BDD58B7</RuleID>
<FunctionIdentifier>
<NamespaceName>
<Pattern>Our.Utilities.Database</Pattern>
</NamespaceName>
<ClassName>
<Pattern>BaseDao</Pattern>
</ClassName>
<FunctionName>
<Pattern>ReadSqlTemplate</Pattern>
</FunctionName>
<ApplyTo implements="true" overrides="true" extends="true"/>
</FunctionIdentifier>
<OutArguments>return</OutArguments>
</DataflowCleanseRule>
<DataflowCleanseRule formatVersion="17.10" language="dotnet">
<RuleID>14C423ED-5A51-4BA1-BAE1-075E566BE58D</RuleID>
<TaintFlags>+VALIDATED_SQL_INJECTION</TaintFlags>
<FunctionIdentifier>
<NamespaceName>
<Pattern>Our.Utilities.Database</Pattern>
</NamespaceName>
<ClassName>
<Pattern>BaseDao</Pattern>
</ClassName>
<FunctionName>
<Pattern>ReadSqlTemplate</Pattern>
</FunctionName>
<ApplyTo implements="true" overrides="true" extends="true"/>
</FunctionIdentifier>
<OutArguments>return</OutArguments>
</DataflowCleanseRule>
</RuleDefinitions>
</Rules>
</RulePack>

When I run your sample code (I do have to modify it since it will not compile as is). When I run it with SCA 17.10 with 2017Q3 (I also did this on 2017Q2) I did not get a the same SQL Injection Rule ID as you.
Looking at your Analysis Evidence, I assume that the analyzer that found this was not Dataflow or Control flow but maybe the Semantic or Structural?
You can see the type of analyzer that found the finding by looking at the summary tab:
Either way, I don't think a Custom Rule is what I would do here.
An option you can do is to use a Filter file.
This a file that can contain
RuleIDs
InstanceIDs
Categories
When this file is passed into the scan command, any finding that matches any of the fields in the filter file will be filtered out from the results.
You can find examples of using the filter file in <SCA Install Dir>\Samples\Advanced\filter
or you can look in the Fortify SCA Users Guide in Appendix B: Filtering the Analysis
*Note: Your analysis of using filters (in comment) are spot on.

C# - Load existing system environment variables when the current process don't have them loaded

On Windows, I have a C# assembly that is COM visible. It references other assemblies to control an application in the machine. It works fine.
However, under Apache Web Server and using CGI, it doesn't work. After doing some debuging, I found out that the problem is that, while running under Apache's CGI, the environment variables SYSTEMROOT and SYSTEMDRIVE, which aparently are needed by the referenced assemblies, are not loaded.
I can configure Apache to pass those environemtn variables too, but before doing so, I'd really like to know if there's some command I can put on my C# COM visible assembly to make it load environment variables as if it was, let's say, the SYSTEM user or something like that, so it doesn't have to relay on the environment passed by the starting application.
How do you force loading an existent system environment variable in C#, when IT IS NOT SET in the current process (or it was process-deleted by the launching process)?
Thanks in advance for any suggestions!
EDIT 1 - ADDED INFO: Just to make it more clear (as I see in the current answers it's not so clear): Apache intendedly deletes a lot of environment variables for CGI processes. It's not that Apache cannot see them, it can, but it won't pass them to CGI processes.

This should do the trick:
Environment.GetEnvironmentVariable("variable", EnvironmentVariableTarget.Machine);
I did a small test and it is working:
//has the value
string a = Environment.GetEnvironmentVariable("TMP");
Environment.SetEnvironmentVariable("TMP", null);
//does not have has the value
a = Environment.GetEnvironmentVariable("TMP");
//has the value
a = Environment.GetEnvironmentVariable("TMP", EnvironmentVariableTarget.Machine);

SOLUTION: Marco's answer was great and technically answered my question - except that I found out that the environment variables SYSTEMROOT and SYSTEMDRIVE are not really set in the registry where all environment variables are set, so, the chosen answer works for all variables except those two, which I specified in the OP.
SYSTEMROOT is defined on the registry in HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\SystemRoot, and apparently (after more research), SYSTEMDRIVE is generated as a substring of SYSTEMDRIVE.
So, to get SYSTEMDRIVE and SYSTEMROOT from registry and load them into the environment:
using Microsoft.Win32;
namespace MySpace
{
public class Setup
{
public Setup()
{
SetUpEnvironment();
}
private void SetUpEnvironment()
{
string test_a = Environment.GetEnvironmentVariable("SYSTEMDRIVE", EnvironmentVariableTarget.Process);
string test_b = Environment.GetEnvironmentVariable("SYSTEMROOT", EnvironmentVariableTarget.Process);
if (test_a == null || test_a.Length == 0 || test_b == null || test_b.Length == 0)
{
string RegistryPath = "HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion";
string SYSTEMROOT = (string) Registry.GetValue(RegistryPath, "SystemRoot", null);
if (SYSTEMROOT == null)
{
throw new System.ApplicationException("Cannot access registry key " + RegistryPath);
}
string SYSTEMDRIVE = SYSTEMROOT.Substring(0, SYSTEMROOT.IndexOf(':') + 1);
Environment.SetEnvironmentVariable("SYSTEMROOT", SYSTEMROOT, EnvironmentVariableTarget.Process);
Environment.SetEnvironmentVariable("SYSTEMDRIVE", SYSTEMDRIVE, EnvironmentVariableTarget.Process);
}
}
}
}
Then you can just call Setup setup = new Setup(); from other classes. And that's it. :-)

Environment.GetEnvironmentVariable
see reference here.
e.g.
Environment.CurrentDirectory = Environment.GetEnvironmentVariable("windir");
DirectoryInfo info = new DirectoryInfo(".");
lock(info)
{
Console.WriteLine("Directory Info: "+info.FullName);
}

Are the variables set as system wide?
If they are not, that is what you need to do, otherwise create user variables for the user the COM is running under.

Thank you. I cannot state with any certainty that this has once and for all driven a stake through the heart of the vampire, but amazingly enough, the error has disappeared (for now). The odd thing is that access to the statement
Environment.GetEnvironmentVariable("variable", EnvironmentVariableTarget.Machine);
is a real oddity in the debugger. It does not show up in Intellisense and does not even appear to fire, which leads me to suspect, which you all knew already, that this is some sort of magic runtime object Environment that has no instantiation in the debugger but also can be benignly jumped over. Oh well.
Oh and I should mention that after you see that error, you will note oddities in your Windows OS, which is worrisome. In particular, you will see, if you try to use the Control Panel /System/Advanced Properties (whatever) that it cannot load the dialog for the environment variables any more, indicating that %windir% has been seriously hosed (compromised) across all applications. Bad bad bad....

Querying custom LDAP property via System.DirectoryServices in C#?

I have a custom LDAP schema installed on my OpenLDAP server which is as follows:
attributeType ( 999.0.01
NAME 'picturePath'
EQUALITY caseIgnoreMatch
SUBSTR caseIgnoreSubstringsMatch
SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{1024}
)
objectClass ( 999.1.01
NAME 'indieStackTeam'
DESC 'Team definition for IndieStack'
SUP groupOfUniqueNames
STRUCTURAL
MAY ( picturePath )
)
In my ASP.NET MVC 2 application, I'm querying for the picturePath property like so (and it is confirmed that picturePath exists in the list of keys):
this.Picture = properties["picturePath"].Value as string;
When I attempt to do this under .NET 3.5 I get the following exception:
[COMException (0x8000500c): Unknown error (0x8000500c)]
System.DirectoryServices.PropertyValueCollection.PopulateList() +347013
System.DirectoryServices.PropertyValueCollection..ctor(DirectoryEntry entry, String propertyName) +49
System.DirectoryServices.PropertyCollection.get_Item(String propertyName) +150
However, when the same code runs under Mono (on the same server as OpenLDAP) it works perfectly fine. Clients such as LDAPAdmin can also read the picturePath property correctly.
More so, it's only when I go to read the value that it fails; I can see the property is there in the keys list, I just can't access it.
Unfortunately unknown error doesn't tell me a lot about what's going wrong, but I'm finding the .NET implementation of System.DirectoryServices is very flaky (you get the same unknown error if you connect to the LDAP server using lowercase in 'DC=').
Has anyone had this problem before and if so, how is it solved?

Two things you should check:
1) does that particular user object indeed have a value in picturePath? You might want to check for existance of the property before accessing it:
if(properties.Contains("picturePath") && properties["picturePath"].Count > 0)
{
....
}
2) If I remember correctly, to get access to custom attributes, you should explicitly refresh the cache for a user object before doing anything:
DirectoryEntry de = ......; // find / assign that DirectoryEntry somehow
de.RefreshCache(); // to load all properties from the directory
or:
de.RefreshCache(new string[] { "picturePath" }); // to just load the "picturePath" attribute
Also: the classes in System.DirectoryServices are really mostly geared towards being used against Active Directory - there might be "surprises" or subtle incompatibilities when used against some other LDAP server - like OpenLDAP.

It seems that the .NET LDAP client expects a correctly formed OID for attribute types and object classes.
You'll note that I was using OIDs of the form 999.X.YY, which while they might be syntactically correct, aren't usually encountered in the real world. My guess is the LDAP client parses OIDs and since these don't conform to what is expected, it throws an error.
I changed the OIDs to 1.3.6.1.4.1.40000.1.3.1 and 1.3.6.1.4.1.40000.1.4.1 respectively (I've also applied for a PEN, which will give me an assigned number instead of '40000'), refreshed the schema in the server and recreated the entries and the LDAP client now correctly reads the custom attributes.