When testing the performance in SQL clr assembly it seems that call to unmaged code is extrem slow
I did the test as console app and as clr assembly with the following result
Console App:
decimal.Parse("-49823174320.9293800") 304.7 ns 855.7 cycles
PInvoke.Empty()...........................................11.9 ns 33.6 cycles
Clr assembly:
decimal.Parse("-49823174320.9293800") 304.7 ns 855.7 cycles
PInvoke.Empty()......................................5095.4 ns 14308.1 cycles
The PInvoke.Empty() is an asm function with only ret statement. This function is created with virtualAlloc.
public static class Pinvoke
{
[SuppressUnmanagedCodeSecurity]
[UnmanagedFunctionPointer(CallingConvention.StdCall)]
public delegate ulong FuncUInt64();
private const uint PAGE_EXECUTE = 0x10;
private const uint PAGE_EXECUTE_READWRITE = 0x40;
private const uint MEM_COMMIT = 0x1000;
private const uint MEM_RELEASE = 0x8000;
public static readonly FuncUInt64 Empty;
private static readonly byte[] ReturnOnlyAsm = { 0xC3 };
static Pinvoke()
{
var buf = IntPtr.Zero;
try
{
// We pad the functions to 64 bytes (the length of a cacheline on the Intel processors)
var rdtscpLength = (ReturnOnlyAsm.Length & 63) != 0 ? (ReturnOnlyAsm.Length | 63) + 1 : ReturnOnlyAsm.Length;
buf = VirtualAlloc(IntPtr.Zero, (IntPtr) rdtscpLength, MEM_COMMIT, PAGE_EXECUTE_READWRITE);
if (buf == IntPtr.Zero) throw new Win32Exception();
Marshal.Copy(ReturnOnlyAsm, 0, buf, ReturnOnlyAsm.Length);
for (var i = ReturnOnlyAsm.Length; i < rdtscpLength; i++) Marshal.WriteByte(buf, i, 0x90); // nop
// Change the access of the allocated memory from R/W to Execute
var result = VirtualProtect(buf, (IntPtr) rdtscpLength, PAGE_EXECUTE, out var oldProtection);
if (!result) throw new Win32Exception();
// Create a delegate to the "function"
Empty = (FuncUInt64) Marshal.GetDelegateForFunctionPointer(buf, typeof(FuncUInt64));
buf = IntPtr.Zero;
}
finally
{
if (buf != IntPtr.Zero)
VirtualFree(buf, IntPtr.Zero, MEM_RELEASE);
}
}
[DllImport("kernel32.dll", ExactSpelling = true, SetLastError = true)]
private static extern IntPtr VirtualAlloc(IntPtr lpAddress, IntPtr dwSize, uint flAllocationType, uint flProtect);
[DllImport("kernel32.dll", ExactSpelling = true, SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
private static extern bool VirtualProtect(IntPtr lpAddress, IntPtr dwSize, uint flAllocationType, out uint lpflOldProtect);
[DllImport("kernel32.dll", ExactSpelling = true, SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
private static extern bool VirtualFree(IntPtr lpAddress, IntPtr dwSize, uint dwFreeType);
}
The decimal parse has no unmanged code but the GetTimestamp has. Calls to unmanged code seems to be more than 250 times slower
For code used to test see https://github.com/Anderman/SqlClrPerformance.
This code used asm code trick to call the rdtscp timer and will only run on 64 bit.
Used sql server 2017 (13.0.1742.0) and tested with release build for both sql and console app.
Also tested with a the Stopwatch which is not so accurate, but reveals the same issue
private void swGetCurrentProcess()
{
var i = 0;
var sw = Stopwatch.StartNew();
do
{
i++;
Process.GetCurrentProcess();
} while (sw.ElapsedMilliseconds < 5000);
var nanoSecodsPerIteration = 5_000_000_000 / i;
_reporter($"GetCurrentProcess with StopWatch {nanoSecodsPerIteration,5:0.0} ns ");
}
Console:
GetCurrentProcess with StopWatch 426.0 ns
SqlClr:
GetCurrentProcess with StopWatch 12422.0 ns
Related
I have an application that monitors file and folder changes. I get an error when a new file is created, deleted or modified. I have the following code and have no idea why the Marshal.PtrToStringAuto throw "Attempted to read or write protected memory. This is often an indication that other memory is corrupt.".
Why am I getting this error when I try to get the filename?
Can someone please help me on what to look for to resolve this error...
error image
[DllImport("kernel32.dll", EntryPoint = "FindFirstChangeNotification")]
static extern System.IntPtr FindFirstChangeNotification(string lpPathName, bool bWatchSubtree, uint dwNotifyFilter);
[DllImport("kernel32.dll", EntryPoint = "FindNextChangeNotification")]
static extern bool FindNextChangeNotification(IntPtr hChangedHandle);
[DllImport("kernel32.dll", EntryPoint = "FindCloseChangeNotification")]
static extern bool FindCloseChangeNotification(IntPtr hChangedHandle);
[DllImport("kernel32.dll", EntryPoint = "WaitForSingleObject")]
static extern uint WaitForSingleObject(IntPtr handle, uint dwMilliseconds);
[DllImport("kernel32.dll", EntryPoint = "ReadDirectoryChangesW")]
static extern bool ReadDirectoryChangesW(IntPtr hDirectory, IntPtr lpBuffer, uint nBufferLength, bool bWatchSubtree, uint dwNotifyFilter, out uint lpBytesReturned, uint lpOverlapped, uint lpCompletionRoutine);
[DllImport("kernel32.dll", CharSet = CharSet.Unicode)]
public static extern int ReadDirectoryChangesW(uint hDirectory, out FILE_NOTIFY_INFORMATION finfo, uint nBufferLength,uint bWatchSubtree, uint dwNotifyFilter, out uint lpbytesReturned,uint PassZero1, uint PassZero2);
[DllImport("kernel32.dll", EntryPoint = "CreateFile")]
public static extern IntPtr CreateFile(string lpFileName, uint dwDesiredAccess, uint dwShareMode, IntPtr SecurityAttributes, uint dwCreationDisposition, uint dwFlagsAndAttributes, IntPtr hTemplateFile);
private const int MaxChanges = 4096;
public async Task ReadChangesAsyncNew()
{
var token = _cancellationTokenSource.Token;
await Task.Run(() =>
{
unsafe
{
var directoryHandle = CreateFile(_path, 0x80000000, 0x00000007, IntPtr.Zero, 3, 0x02000000, IntPtr.Zero);
var fileCreatedDeletedOrUpdated = FileSystemNotifications.FileNameChanged | FileSystemNotifications.FileModified;
var waitable = FindFirstChangeNotification(_path, true, (uint)fileCreatedDeletedOrUpdated);
var notifySize = Marshal.SizeOf(typeof(FileNotifyInformation));
do
{
var changes = new FileNotifyInformation[MaxChanges];
var pinnedArray = GCHandle.Alloc(changes, GCHandleType.Pinned);
var buffer = pinnedArray.AddrOfPinnedObject();
uint bytesReturned = 0;
if (!ReadDirectoryChangesW(directoryHandle, buffer, (uint)(notifySize * MaxChanges), true, (uint)fileCreatedDeletedOrUpdated, out bytesReturned, 0, 0))
throw Win32Error.GetLastError().GetException();
var result = new List<FileEvent>();
for (var i = 0; i < bytesReturned / notifySize; i += 1)
{
var change = Marshal.PtrToStructure<FileNotifyInformation>(new IntPtr(buffer.ToInt64() + i * notifySize));
if (((int)change.Action) == (int)FileActions.FileAdded)
{
result.Add(new FileEvent(Marshal.PtrToStringAuto(change.FileName, change.FileNameLength), FileEventType.FileAdded));
}
else if (((int)change.Action) == (int)FileActions.FileRemoved)
{
result.Add(new FileEvent(Marshal.PtrToStringAuto(change.FileName, change.FileNameLength), FileEventType.FileDeleted));
}
else if (((int)change.Action) == (int)FileActions.FileModified)
{
result.Add(new FileEvent(Marshal.PtrToStringAuto(change.FileName, change.FileNameLength), FileEventType.FileChanged));
}
else if (((int)change.Action) == (int)FileActions.FileRenamedNew)
{
result.Add(new FileEvent(Marshal.PtrToStringAuto(change.FileName, change.FileNameLength), FileEventType.FileRenamed));
}
}
pinnedArray.Free();
} while (FindNextChangeNotification(waitable));
FindCloseChangeNotification(waitable);
}
}, token);
}
My problem is in calling shellcode in C#. I alloc memory in the process, write there my bytes and try to call it with Marshal.GetDelegateFromFunctionPointer(). It seems working, but, only if my function doesn't have parameters. If there's any parameter there then it's value equals {Unable to read memory}. In C++ I just need to typedef func, cast it, and call. How to do this properly in C#? (Shellcode performs jump to another C# func)
First of all in C# you have to declare delegate with all required parameters and calling convension. When you use GetDelegateForFunctionPointer ptr is converted to a delegate that invokes the unmanaged method using the __stdcall calling convention on Windows, or the __cdecl calling convention on Linux and macOS. You can set the calling convention by applying the UnmanagedFunctionPointerAttribute to the delegate.
Also you need indicate how to marshal parameters between managed and unmanaged code to achieve this you should use MarshalAsAttribute
See remarks section of Marshal.GetDelegateForFunctionPointer
and UnmanagedFunctionPointerAttribute
and MarshalAsAttribute
Example:
[DllImport("kernel32.dll", CharSet = CharSet.Unicode, ExactSpelling = true)]
private static extern IntPtr LoadLibraryW([MarshalAs(UnmanagedType.LPWStr)] string libFileName);
[DllImport("kernel32.dll", CharSet = CharSet.Ansi, ExactSpelling = true)]
private static extern IntPtr GetProcAddress(IntPtr hModule, [MarshalAs(UnmanagedType.LPStr)] string lpProcName);
const int MEM_COMMIT = 0x00001000;
const int PAGE_READWRITE = 0x04;
const int PAGE_EXECUTE_READ = 0x20;
[DllImport("kernel32.dll", ExactSpelling = true)]
private static extern IntPtr VirtualAlloc(IntPtr address, IntPtr size, int allocationType, int protect);
[DllImport("kernel32.dll", ExactSpelling = true)]
[return: MarshalAs(UnmanagedType.Bool)]
private static extern bool VirtualProtect(IntPtr address, IntPtr size, int newProtect, out int oldProtect);
private delegate int GetTempPath(int nBufferLength, [MarshalAs(UnmanagedType.LPStr)] StringBuilder buffer);
static void Main(string[] args)
{
IntPtr hModule = LoadLibraryW("c:\\windows\\system32\\kernel32.dll");
IntPtr procAddress = GetProcAddress(hModule, "GetTempPathA");
// allocate page with read/write access
IntPtr page = VirtualAlloc(IntPtr.Zero, new IntPtr(4096), MEM_COMMIT, PAGE_READWRITE);
// write to allocated memory
// mov r11, procAddress
Marshal.WriteByte(page, 0, 0x49);
Marshal.WriteByte(page, 1, 0xBB);
Marshal.WriteInt64(page, 2, procAddress.ToInt64());
// jmp r11
Marshal.WriteByte(page, 0xA, 0x41);
Marshal.WriteByte(page, 0xB, 0xFF);
Marshal.WriteByte(page, 0xC, 0xE3);
// protect memory to allow execute code
bool result = VirtualProtect(page, new IntPtr(4096), PAGE_EXECUTE_READ, out int oldProtect);
var getTempPathTrampoline = Marshal.GetDelegateForFunctionPointer<GetTempPath>(page);
StringBuilder builder = new StringBuilder(1024);
getTempPathTrampoline(1024, builder);
Console.WriteLine(builder.ToString());
}
Example (call managed code)
static void Main(string[] args)
{
GetTempPath delegateToManaged = ManagedMethod;
IntPtr managedPtr = Marshal.GetFunctionPointerForDelegate(delegateToManaged);
// allocate page with read/write access
IntPtr page = VirtualAlloc(IntPtr.Zero, new IntPtr(4096), MEM_COMMIT, PAGE_READWRITE);
// write to allocated memory
// mov r11, procAddress
Marshal.WriteByte(page, 0, 0x49);
Marshal.WriteByte(page, 1, 0xBB);
Marshal.WriteInt64(page, 2, managedPtr.ToInt64());
// jmp r11
Marshal.WriteByte(page, 0xA, 0x41);
Marshal.WriteByte(page, 0xB, 0xFF);
Marshal.WriteByte(page, 0xC, 0xE3);
// protect memory to allow execute code
bool result = VirtualProtect(page, new IntPtr(4096), PAGE_EXECUTE_READ, out int oldProtect);
var getTempPathTrampoline = Marshal.GetDelegateForFunctionPointer<GetTempPath>(page);
StringBuilder builder = new StringBuilder(1024);
getTempPathTrampoline(1024, builder);
Console.WriteLine(builder.ToString());
}
public static int ManagedMethod(int nBufferLength, StringBuilder buffer)
{
buffer.Append("Hello World");
return 0;
}
I am using in my program "Kernel32.dll" functionality to access raw disk sectors on WinXP SP3 OS (external HDD).
Everything works fine till the program reaches sector number 8388607 - which means the bytes offset in SetFilePointer exceeds 32 bit (uint!).
But my code, as below, uses all variables as "long". What I am doing wrong?
The code (on "Dump" button click):
int drive = DRV.SelectedIndex; // DRV is the drive combo box
long bps = BytesPerSector(drive), spt = GetTotalSectors(drive);
string dr = DRV.SelectedItem.ToString();
int moveToHigh, read = 0;
uint GENERIC_READ = 0x80000000;
uint OPEN_EXISTING = 3;
SafeFileHandle handleValue = CreateFile(dr, GENERIC_READ, 0, IntPtr.Zero, OPEN_EXISTING, 0, IntPtr.Zero);
if (handleValue.IsInvalid)
Marshal.ThrowExceptionForHR(Marshal.GetHRForLastWin32Error());
// idx = Loop starting index
// FS = The starting sector index
// TS = The final sector index
long idx = (FS == -1) ? 0 : FS, tot = (TS == -1) ? spt : TS;
for ( ; idx < tot; idx++)
{
byte[] b = new byte[bps];
// HERE IS THE ISSUE!!!
SetFilePointer(handleValue, idx*bps), out moveToHigh, EMoveMethod.Current);
if (ReadFile(handleValue, b, bps, out read, IntPtr.Zero) == 0)
Marshal.ThrowExceptionForHR(Marshal.GetHRForLastWin32Error());
if (this.IsDisposed == true) { handleValue.Close(); break; }
Application.DoEvents();
}
handleValue.Close();
The kernel32.dll external functions:
[DllImport("Kernel32.dll", SetLastError = true, CharSet = CharSet.Auto)]
static extern uint SetFilePointer(
[In] SafeFileHandle hFile,
[In] long lDistanceToMove,
[Out] out int lpDistanceToMoveHigh,
[In] EMoveMethod dwMoveMethod);
[DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
static extern SafeFileHandle CreateFile(string lpFileName, uint dwDesiredAccess,
uint dwShareMode, IntPtr lpSecurityAttributes, uint dwCreationDisposition,
uint dwFlagsAndAttributes, IntPtr hTemplateFile);
[DllImport("kernel32", SetLastError = true)]
internal extern static int ReadFile(SafeFileHandle handle, byte[] bytes,
int numBytesToRead, out int numBytesRead, IntPtr overlapped_MustBeZero);
I have tried many things, but no idea what is wrong, application just ending up with a fatal exception asking to send bug report
Thanks a lot
Your P/Invoke definition is wrong. The function takes a 32bit value in but you defined it as a 64bit value. It won't work properly and definitely not past the value range of 32bit variables.
See the definition and example on how to use at pinvoke.net:
[DllImport("Kernel32.dll", SetLastError = true, CharSet = CharSet.Auto)]
private static extern int SetFilePointer(IntPtr handle, int lDistanceToMove, out int lpDistanceToMoveHigh, uint dwMoveMethod);
int lo = (int)(offset & 0xffffffff);
int hi = (int)(offset >> 32);
lo = SetFilePointer(handle, lo, out hi, moveMethod);
So you need to split the 64bit value in two and provide both parts for the function.
Also don't use doubles for integers. You will get into trouble when the accuracy ends and there is no reason to use them.
My processor (Intel i7) supports the POPCNT instruction and I would like to call it from my C# application. Is this possible?
I believe I read somewhere that it isn't, but the JIT will invoke it if it finds it available but what function would I have to call that may be substituted with such an instruction?
Popcount is being called millions of times in a loop so I'd like to be able to have this CPU optimization if possible.
You want to play with fire, and here we like to play with fire...
class Program
{
const uint PAGE_EXECUTE_READWRITE = 0x40;
const uint MEM_COMMIT = 0x1000;
[DllImport("kernel32.dll", SetLastError = true)]
static extern IntPtr VirtualAlloc(IntPtr lpAddress, IntPtr dwSize, uint flAllocationType, uint flProtect);
private delegate int IntReturner();
static void Main(string[] args)
{
List<byte> bodyBuilder = new List<byte>();
bodyBuilder.Add(0xb8); // MOV EAX,
bodyBuilder.AddRange(BitConverter.GetBytes(42)); // 42
bodyBuilder.Add(0xc3); // RET
byte[] body = bodyBuilder.ToArray();
IntPtr buf = VirtualAlloc(IntPtr.Zero, (IntPtr)body.Length, MEM_COMMIT, PAGE_EXECUTE_READWRITE);
Marshal.Copy(body, 0, buf, body.Length);
IntReturner ptr = (IntReturner)Marshal.GetDelegateForFunctionPointer(buf, typeof(IntReturner));
Console.WriteLine(ptr());
}
}
(this small example of assembly will simply return 42... I think it's the perfect number for this answer :-) )
In the end the trick is that:
A) You must know the opcodes corresponding to the asm you want to write
B) You use VirtualAlloc to make a page of memory executable
C) In some way you copy your opcodes there
(the code was taken from http://www.cnblogs.com/netact/archive/2013/01/10/2855448.html)
Ok... the other one was as written on the site (minus an error on the uint -> IntPtr dwSize), this one is how it should be written (or at least it's a +1 compared to the original... I would encapsulate everything in a IDisposable class instead of using try... finally)
class Program
{
const uint PAGE_READWRITE = 0x04;
const uint PAGE_EXECUTE = 0x10;
const uint MEM_COMMIT = 0x1000;
const uint MEM_RELEASE = 0x8000;
[DllImport("kernel32.dll", SetLastError = true)]
static extern IntPtr VirtualAlloc(IntPtr lpAddress, IntPtr dwSize, uint flAllocationType, uint flProtect);
[DllImport("kernel32.dll", SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
static extern bool VirtualProtect(IntPtr lpAddress, IntPtr dwSize, uint flAllocationType, out uint lpflOldProtect);
[DllImport("kernel32.dll", SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
static extern bool VirtualFree(IntPtr lpAddress, IntPtr dwSize, uint dwFreeType);
private delegate int IntReturner();
static void Main(string[] args)
{
List<byte> bodyBuilder = new List<byte>();
bodyBuilder.Add(0xb8); // MOV EAX,
bodyBuilder.AddRange(BitConverter.GetBytes(42)); // 42
bodyBuilder.Add(0xc3); // RET
byte[] body = bodyBuilder.ToArray();
IntPtr buf = IntPtr.Zero;
try
{
// We VirtualAlloc body.Length bytes, with R/W access
// Note that from what I've read, MEM_RESERVE is useless
// if the first parameter is IntPtr.Zero
buf = VirtualAlloc(IntPtr.Zero, (IntPtr)body.Length, MEM_COMMIT, PAGE_READWRITE);
if (buf == IntPtr.Zero)
{
throw new Win32Exception();
}
// Copy our instructions in the buf
Marshal.Copy(body, 0, buf, body.Length);
// Change the access of the allocated memory from R/W to Execute
uint oldProtection;
bool result = VirtualProtect(buf, (IntPtr)body.Length, PAGE_EXECUTE, out oldProtection);
if (!result)
{
throw new Win32Exception();
}
// Create a delegate to the "function"
// Sadly we can't use Funct<int>
var fun = (IntReturner)Marshal.GetDelegateForFunctionPointer(buf, typeof(IntReturner));
Console.WriteLine(fun());
}
finally
{
if (buf != IntPtr.Zero)
{
// Free the allocated memory
bool result = VirtualFree(buf, IntPtr.Zero, MEM_RELEASE);
if (!result)
{
throw new Win32Exception();
}
}
}
}
}
I have an unmanaged C++ Windows console app that works fine. I want it in C#. I have done DllImport statements for the necessary Kernel32.dll symbols:
[StructLayout(LayoutKind.Sequential)]
internal struct DiskGeometry
{
public long Cylinders;
public int MediaType;
public int TracksPerCylinder;
public int SectorsPerTrack;
public int BytesPerSector;
}
internal static class NativeMethods
{
internal const uint FileAccessGenericRead = 0x80000000;
internal const uint FileShareWrite = 0x2;
internal const uint FileShareRead = 0x1;
internal const uint CreationDispositionOpenExisting = 0x3;
internal const uint IoCtlDiskGetDriveGeometry = 0x70000;
[DllImport("Kernel32.dll", SetLastError = true, CharSet = CharSet.Auto)]
public static extern SafeFileHandle CreateFile(
string fileName,
uint fileAccess,
uint fileShare,
IntPtr securityAttributes,
uint creationDisposition,
uint flags,
IntPtr template);
[DllImport("Kernel32.dll", SetLastError = false, CharSet = CharSet.Auto)]
public static extern int DeviceIoControl(
SafeFileHandle device,
uint controlCode,
IntPtr inBuffer,
uint inBufferSize,
IntPtr outBuffer,
uint outBufferSize,
ref uint bytesReturned,
IntPtr overlapped);
}
I then have the following application code:
public static void Main()
{
SafeFileHandle diskHandle = NativeMethods.CreateFile(
"\\\\.\\PhysicalDrive0",
NativeMethods.FileAccessGenericRead,
NativeMethods.FileShareWrite | NativeMethods.FileShareRead,
IntPtr.Zero,
NativeMethods.CreationDispositionOpenExisting,
0,
IntPtr.Zero);
if (diskHandle.IsInvalid)
{
Console.WriteLine("CreateFile failed with error: {0}", Marshal.GetLastWin32Error());
return;
}
int geometrySize = Marshal.SizeOf(typeof(DiskGeometry));
Console.WriteLine("geometry size = {0}", geometrySize);
IntPtr geometryBlob = Marshal.AllocHGlobal(geometrySize);
uint numBytesRead = 0;
if (0 == NativeMethods.DeviceIoControl(
diskHandle,
NativeMethods.IoCtlDiskGetDriveGeometry,
IntPtr.Zero,
0,
geometryBlob,
(uint)geometrySize,
ref numBytesRead,
IntPtr.Zero))
{
Console.WriteLine("DeviceIoControl failed with error: {0}", Marshal.GetLastWin32Error());
return;
}
Console.WriteLine("Bytes read = {0}", numBytesRead);
DiskGeometry geometry = (DiskGeometry)Marshal.PtrToStructure(geometryBlob, typeof(DiskGeometry));
Marshal.FreeHGlobal(geometryBlob);
long bytesPerCylinder = (long)geometry.TracksPerCylinder * (long)geometry.SectorsPerTrack * (long)geometry.BytesPerSector;
long totalSize = geometry.Cylinders * bytesPerCylinder;
Console.WriteLine("Media Type: {0}", geometry.MediaType);
Console.WriteLine("Cylinders: {0}", geometry.Cylinders);
Console.WriteLine("Tracks per Cylinder: {0}", geometry.TracksPerCylinder);
Console.WriteLine("Sectors per Track: {0}", geometry.SectorsPerTrack);
Console.WriteLine("Bytes per Sector: {0}", geometry.BytesPerSector);
Console.WriteLine("Bytes per Cylinder: {0}", bytesPerCylinder);
Console.WriteLine("Total disk space: {0}", totalSize);
}
My C# app prints "Bytes read = 0" and the geometry member values are garbage. I am certainly no expert on DllImport and marshaling. Please help me understand what I am doing wrong. If I change the fifth parameter to DeviceIoControl to be "ref DiskGeometry" and just pass in one created right before the call (instead of IntPtr and alloc), all printed geometry member values are 0.
You have mistype, try this:
internal const uint IoCtlDiskGetDriveGeometry = 0x70000;
Try using one of those signatures for the DllImport: http://pinvoke.net/default.aspx/kernel32.DeviceIoControl