TSQL MD5 generation with UTF8 - c#

I have a .NET function MD5 that when run on "146.185.59.178acu-cell.com" it returns f36674ed3dbcb151e1c0dfe4acdbb9f5
public static String MD5(String s)
{
using (var provider = System.Security.Cryptography.MD5.Create())
{
StringBuilder builder = new StringBuilder();
foreach (Byte b in provider.ComputeHash(Encoding.UTF8.GetBytes(s)))
builder.Append(b.ToString("x2").ToLower());
return builder.ToString();
}
}
I wrote the same code in TSQL, but for some reason only the varchar returns the expected result. The nvarchar returns a different md5 : f04b83328560f1bd1c08104b83bc30ea
declare #v varchar(150) = '146.185.59.178acu-cell.com'
declare #nv nvarchar(150) = '146.185.59.178acu-cell.com'
select LOWER(CONVERT(VARCHAR(32), HashBytes('MD5', #v), 2))
--f36674ed3dbcb151e1c0dfe4acdbb9f5
select LOWER(CONVERT(VARCHAR(32), HashBytes('MD5',#nv), 2))
--f04b83328560f1bd1c08104b83bc30ea
Not sure what is going on here because I do expect for the nvarchar to return f36674ed3dbcb151e1c0dfe4acdbb9f5 as it does in .NET

You get different hashes because the binary representation of the text is different. The following query demonstrates this:
declare #v varchar(150) = '146.185.59.178acu-cell.com'
declare #nv nvarchar(150) = '146.185.59.178acu-cell.com'
select convert(varbinary(max), #v) -- 0x3134362E3138352E35392E3137386163752D63656C6C2E636F6D
select convert(varbinary(max), #nv) -- 0x3100340036002E003100380035002E00350039002E003100370038006100630075002D00630065006C006C002E0063006F006D00
The extra 0 bytes for the nvarchar are due to the fact that it's a 2-byte Unicode datatype. Refer to MSDN for more information on Unicode in SQL Server.

Turns out I need to explicitly convert NVarChar to UTF8
Found this code on the net:
CREATE FUNCTION [dbo].[fnUTF8] (
#String NVarChar(max)
) RETURNS VarChar(max) AS BEGIN
DECLARE #Result VarChar(max)
,#Counter Int
,#Len Int
SELECT #Result = ''
,#Counter = 1
,#Len = Len(#String)
WHILE (##RowCount > 0)
SELECT #Result = #Result
+ CASE WHEN Code < 128 THEN ''
WHEN Code < 2048 THEN Char(192 + Code / 64)
ELSE Char(224 + Code / 4096)
END
+ CASE WHEN Code < 128 THEN Char(Code)
WHEN Code < 2048 THEN Char(128 + Code % 64)
ELSE Char(128 + Code / 64 % 64)
END
,#Counter = #Counter + 1
FROM (SELECT UniCode(SubString(#String,#Counter,1)) AS Code) C
WHERE #Counter <= #Len
RETURN #Result
END
GO
And now I use it like this:
select LOWER(CONVERT(VARCHAR(32), HashBytes('MD5', [dbo].[fnUTF8](#nv)), 2))

Related

C# passing multiple oracle PL\SQL blocks gets "Encountered the symbol / "

I'm trying to pass the oracle a PL\SQL script consists of a multiple blocks (begin/end).
The following example code works on SqlDev, but not with C#.
BEGIN
DBMS_OUTPUT.PUT_LINE(TO_CHAR(SYSDATE()) || ' Block1');
END;
/
BEGIN
DBMS_OUTPUT.put_line(TO_CHAR(SYSDATE()) || ' Block2');
END;
C# code:
var str = #"
begin
DBMS_OUTPUT.put_line(TO_CHAR(SYSDATE()) || ' Block1');
end;
/
begin
DBMS_OUTPUT.put_line(TO_CHAR(SYSDATE()) || ' Block2');
end;
";
str = str.Replace(Environment.NewLine, "\n");
ExecuteNonQueryThis(str2)
The error I get:
ORA-06550: line 5, column 1: PLS-00103: Encountered the symbol "/"
Is it even possible to execute a multi-block-script of Oracle via the C#?
just wrap it into one begin end
var str = #"
begin
begin
DBMS_OUTPUT.put_line(TO_CHAR(SYSDATE()) || ' Block1');
end;
begin
DBMS_OUTPUT.put_line(TO_CHAR(SYSDATE()) || ' Block2');
end;
end;";

How to calculate hashbyte SHA1 using C#?

In a table I have a column URL which I am using to save urls. I am calculating the hash in another column by using formula (CONVERT([varbinary](20),hashbytes('SHA1',[URL]))). It's working fine.
Now I need to get similar function in C# to get hash so that I can compare and check that similar row doesnt exist before I insert a new row. I tried few links but no luck.
Here are the links:
http://weblogs.sqlteam.com/mladenp/archive/2009/04/28/Comparing-SQL-Server-HASHBYTES-function-and-.Net-hashing.aspx
How do I calculate the equivalent to SQL Server (hashbytes('SHA1',[ColumnName])) in C#?
** I found this link working. All I need to do is change formula in the db. but is it possible to make it in one line
**
http://forums.asp.net/t/1782626.aspx
DECLARE #HashThis nvarchar(4000);
DECLARE #BinHash varbinary(4000);
SELECT #HashThis = CONVERT(nvarchar(4000),'Password#Test');
SELECT #BinHash = HashBytes('SHA1', #HashThis);
SELECT cast(N'' as xml).value('xs:base64Binary(xs:hexBinary(sql:variable("#BinHash")))', 'nvarchar(4000)');
in c#
string pwd = "Password#Test";
var sha1Provider = HashAlgorithm.Create("SHA1");
var binHash = sha1Provider.ComputeHash(Encoding.Unicode.GetBytes(pwd));
Console.WriteLine(Convert.ToBase64String(binHash));
I am using sql server 2012. collation for the database is SQL_Latin1_General_CP1_CI_AS
Thanks
Paraminder
It's an encoding issue:
C#/.Net/CLR strings are, internally, UTF-16 encoded strings. That means each character is at least two bytes.
Sql Server is different:
char and varchar represent each character as a single byte using the code page tied to the collation used by that column
nchar and nvarchar represent each character as 2 bytes using the [old and obsolete] UCS-2 encoding for Unicode — something which was deprecated in 1996 with the release of Unicode 2.0 and UTF-16.
The big difference between UTF-16 and UCS-2 is that UCS-2 can only represent characters within the Unicode BMP (Basic Multilingual Plane); UTF-16 can represent any Unicode character. Within the BMP, as I understand it, UCS-2 and UTF-16 representations are identical.
That means that to compute a hash that is identical to the one that SQL Server computes, you're going to have to get a byte representation that is identical to the one that SQL Server has. Since it sounds like you're using char or varchar with the collation SQL_Latin1_General_CP1_CI_AS, per the documentation, the CP1 part means code page 1252 and the rest means case-insensitive, accent-sensitive. So...
You can get the encoding for code page 1252 by:
Encoding enc = Encoding.GetEncoding(1252);
Using that information, and given this table:
create table dbo.hash_test
(
id int not null identity(1,1) primary key clustered ,
source_text varchar(2000) collate SQL_Latin1_General_CP1_CI_AS not null ,
hash as
( hashbytes( 'SHA1' , source_text ) ) ,
)
go
insert dbo.hash_test ( source_text ) values ( 'the quick brown fox jumped over the lazy dog.' )
insert dbo.hash_test ( source_text ) values ( 'She looked like something that might have occured to Ibsen in one of his less frivolous moments.' )
go
You'll get this output
1: the quick brown fox jumped over the lazy dog.
sql: 6039D100 3323D483 47DDFDB5 CE2842DF 758FAB5F
c#: 6039D100 3323D483 47DDFDB5 CE2842DF 758FAB5F
2: She looked like something that might have occured to Ibsen in one of his less frivolous moments.
sql: D92501ED C462E331 B0E129BF 5B4A854E 8DBC490C
c#: D92501ED C462E331 B0E129BF 5B4A854E 8DBC490C
from this program
class Program
{
static byte[] Sha1Hash( string s )
{
SHA1 sha1 = SHA1.Create() ;
Encoding windows1252 = Encoding.GetEncoding(1252) ;
byte[] octets = windows1252.GetBytes(s) ;
byte[] hash = sha1.ComputeHash( octets ) ;
return hash ;
}
static string HashToString( byte[] bytes )
{
StringBuilder sb = new StringBuilder() ;
for ( int i = 0 ; i < bytes.Length ; ++i )
{
byte b = bytes[i] ;
if ( i > 0 && 0 == i % 4 ) sb.Append( ' ' ) ;
sb.AppendFormat( b.ToString("X2") ) ;
}
string s = sb.ToString() ;
return s ;
}
private static DataTable ReadDataFromSqlServer()
{
DataTable dt = new DataTable();
using ( SqlConnection conn = new SqlConnection( "Server=localhost;Database=sandbox;Trusted_Connection=True;"))
using ( SqlCommand cmd = conn.CreateCommand() )
using ( SqlDataAdapter sda = new SqlDataAdapter(cmd) )
{
cmd.CommandText = "select * from dbo.hash_test" ;
cmd.CommandType = CommandType.Text;
conn.Open();
sda.Fill( dt ) ;
conn.Close() ;
}
return dt ;
}
static void Main()
{
DataTable dt = ReadDataFromSqlServer() ;
foreach ( DataRow row in dt.Rows )
{
int id = (int) row[ "id" ] ;
string sourceText = (string) row[ "source_text" ] ;
byte[] sqlServerHash = (byte[]) row[ "hash" ] ;
byte[] myHash = Sha1Hash( sourceText ) ;
Console.WriteLine();
Console.WriteLine( "{0:##0}: {1}" , id , sourceText ) ;
Console.WriteLine( " sql: {0}" , HashToString( sqlServerHash ) ) ;
Console.WriteLine( " c#: {0}" , HashToString( myHash ) ) ;
Debug.Assert( sqlServerHash.SequenceEqual(myHash) ) ;
}
return ;
}
}
Easy!
I would suggest that that anytime a hash is created that it be done in a single place. Either in code or on the database. It will make your life easier in the long run. That would mean either changing you C# code to create the hash before inserting the record or doing the duplication check within a stored procedure instead.
Regardless though, the duplication check and insert should be synchronized such that no other inserts could occur between the time you check for any duplicates and when the record is actually inserted. Easiest way to do that would be to perform them both within the same transaction.
If you insist on leaving the logic as it stands I would then suggest that you create the hash in the database but expose it via a stored procedure or user defined function that could be called from your C# code.

Need to convert oracle function into c# code

I have one function in oracle i need to convert it into c# code
please help or tell me any links to do it.
create or replace
FUNCTION "GETSEPARATEDSTRING"
( pString IN VARCHAR2
,pSeparator IN VARCHAR2
,pReturnNumber IN PLS_INTEGER
)
RETURN VARCHAR2
IS
l_SearchString_FinPos PLS_INTEGER :=0;
l_SearchString_StartPos PLS_INTEGER :=0;
l_SearchString_Length PLS_INTEGER :=0;
l_SearchString_CurrentPos PLS_INTEGER :=0;
l_Return VARCHAR2(4000);
BEGIN
-- expecting values as String Seperator String Seperator
-- so if pReturnNumber = 2 then where are
-- looking for seperators 2 and 1. If there is no seperator
-- at the end of the string it is added before comparison,
-- Will return a null if:
-- The length of pString is > 4000
-- The pSeparator has not been specified
-- The pReturnNumber IS <= 0
-- The pReturnNumber IS greater than the number of pSeparator + 1 and therefore we can't pick up a string
-- There was an empty string at the position requested
-- Strings are returned without pSeparator
IF LENGTH( pString || pSeparator ) <= 4000
AND pSeparator IS NOT NULL
AND pReturnNumber > 0
THEN
l_SearchString_FinPos := pReturnNumber;
l_SearchString_StartPos := pReturnNumber - 1;
-- Concat a seperator at the end of the string so at least we
-- know there is one
IF INSTR( pString, pSeparator, -1, 1) != ( LENGTH( RTRIM( pString )) - LENGTH( pSeparator ) + 1 )
THEN
-- There isn't one at the end so add it
l_Return := pString || pSeparator;
--DBMS_OUTPUT.PUT_LINE('Did not find seperator - ADDING');
ELSE
l_Return := pString;
--DBMS_OUTPUT.PUT_LINE('FOUND seperator');
END IF;
-- Set the start position of where we will check to the
-- the last position we found a pSeparator value.
l_SearchString_CurrentPos := l_SearchString_FinPos;
-- Search for the next pSeparator position
l_SearchString_FinPos := INSTR( l_Return, pSeparator, 1, l_SearchString_CurrentPos );
IF l_SearchString_FinPos != 0
THEN
IF l_SearchString_StartPos != 0
THEN
l_SearchString_CurrentPos := l_SearchString_StartPos;
l_SearchString_StartPos := INSTR( l_Return, pSeparator, 1, l_SearchString_CurrentPos ) + 1;
ELSE
-- If we are looking for the first value then StartPos will = 0
-- and cause INSTR to fail
l_SearchString_CurrentPos := 1;
END IF;
l_SearchString_Length := l_SearchString_FinPos - l_SearchString_StartPos;
l_Return := RTRIM( SUBSTR( l_Return, l_SearchString_StartPos, l_SearchString_Length ), pSeparator );
ELSE
l_Return := NULL;
END IF;
END IF;
RETURN l_Return;
EXCEPTION
WHEN OTHERS
THEN
DBMS_OUTPUT.PUT_LINE( 'FUNCTION GetSeperatedString Captured Error: ' || SQLERRM );
RETURN NULL;
END;
I don't see why you can't convert it yourself? What is the problem? You even got a comment describing exactly what the function is doing.
Do something like this:
Convert the method signature and all local variables to C# ones.
Lookup the documentation of INSTR to see if uses a zero-based index as string.IndexOf
Do a straight conversion by looking at one line at a time
Test the function
Rename all variables to have C# names
Refactor to take advantage of C#

LINQ-to-SQL orderby question

I have a LINQ-to-SQL query, and I order on an nvarchar field called CustomerReference. The problem is, reference's that start with a capital letter seem to be after ones without capitals, when I need this the other way around. For example, if I have the following rows:
d93838
D98484
It is currently ordered in that sequence right now, however I need it reversed - so it'd be like this
D98484
d93838
Any ideas guys? Thanks
This assumes the Format [A-Za-z]\d+ and will put b3432 before C1234 but after B9999
list.OrderBy (l => l.CustomerReference.Substring(0,1).ToLower())
.ThenByDescending(l =>l.CustomerReference.Substring(0,1).ToUpper()==l.CustomerReference.Substring(0,1))
.ThenBy (l =>l.CustomerReference )
EDIT: I was asked for the SQL too so this is what LINQPad does
-- Region Parameters
DECLARE #p0 Int SET #p0 = 0
DECLARE #p1 Int SET #p1 = 1
DECLARE #p2 Int SET #p2 = 0
DECLARE #p3 Int SET #p3 = 1
DECLARE #p4 Int SET #p4 = 0
DECLARE #p5 Int SET #p5 = 1
-- EndRegion
SELECT [T0].CustomerReference FROM [dbo].[test] AS [t0]
ORDER BY LOWER(SUBSTRING([t0].[CustomerReference], #p0 + 1, #p1)),
(CASE
WHEN UPPER(SUBSTRING([t0].[CustomerReference], #p2 + 1, #p3)) = SUBSTRING([t0].[CustomerReference], #p4 + 1, #p5) THEN 1
WHEN NOT (UPPER(SUBSTRING([t0].[CustomerReference], #p2 + 1, #p3)) = SUBSTRING([t0].[CustomerReference], #p4 + 1, #p5)) THEN 0
ELSE NULL
END) DESC, [t0].[CustomerReference]
In most implementations, lower-case comes first (not least, that is how code-points are arranged ordinally). You won't be able to get SQL server to change that, so the next best thing is to bring it back unsorted, and write a custom comparer. Note that the inbuilt .NET comparers will also treat lower-case as either first or equal (compared to their upper-case equivalent), depending on the comparer.
However! Unless you limit yourself to very simple examples (ASCII etc), ordering "alike" characters is a very non-trivial exercise. Even if we ignore the Turkish I / İ / ı / i, accented characters are going to cause you problems).

c# ip address generator to SQL

I have the following python code I found on the internet, I would like to make a table in a SQL database with every ipv4 address that there is. I dont code in python but its what I found.
My question is
1: Is there T-SQL code I can use to generate the table ? (one column ie 0.0.0.0-255.255.255.255)
2: Is how would I make this in c#? using the fastest method possible ? I know showing the results slows the console application down by 400 %
#!/usr/bin/env python
def generate_every_ip_address():
for octet_1 in range( 256 ):
for octet_2 in range( 256 ):
for octet_3 in range( 256 ):
for octet_4 in range( 256 ):
yield "%d.%d.%d.%d" % (octet_1, octet_2, octet_3, octet_4)
for ip_address in generate_every_ip_address():
print ip_address
Would this work?
DECLARE #a INTEGER
DECLARE #b INTEGER
DECLARE #c INTEGER
DECLARE #d INTEGER
DECLARE #IPADDRESS nvarchar(50)
set #a = 0
WHILE #a < 256
BEGIN
SET #b = 0
WHILE #b < 256
BEGIN
SET #c = 0
WHILE #c < 256
BEGIN
SET #d = 0
WHILE #d < 256
BEGIN
SET #IPADDRESS = CAST(#a AS nvarchar(3)) + '.' + CAST(#b AS nvarchar(3)) + '.' + CAST(#c AS nvarchar(3)) + '.' + CAST(#d AS nvarchar(3))
PRINT #IPADDRESS
SET #d = #d + 1
END
SET #c = #c + 1
END
SET #b = #b + 1
END
SET #a = #a + 1
END
To insert in batches of 16,581,375 rows would be quite straightforward using the following TSQL.
DECLARE #Counter INT
SET #Counter = 0
SET NOCOUNT ON ;
WHILE ( #Counter <= 255 )
BEGIN
RAISERROR('Procesing %d' ,0,1,#Counter) WITH NOWAIT ;
WITH Numbers ( N )
AS ( SELECT CAST(number AS VARCHAR(3))
FROM master.dbo.spt_values
WHERE type = 'P'
AND number BETWEEN 0 AND 255
)
INSERT INTO YourTable
( IPAddress
)
SELECT #Counter + '.' + N1.N + '.' + N2.N + '.' + N3.N
FROM Numbers N1 ,
Numbers N2 ,
Numbers N3
SET #Counter = #Counter + 1
END
Please just use an int IDENTITY column to store each IP address. They're only 32 bits. Fill your table up with whatever else you're storing.

Categories

Resources