Develop Reference

Memory Violation Dynamically Appending to Methods at runtime

Disclaimer: I'm doing this for learning purposes. This is not going to be used in code.
I'm trying to understand how method table are structure for generics, I want to dynamically appending to methods at runtime. I found a very useful stack overflow question reference for getting me started.
I have a simple controller which I'm using as a test to verify my methods are swapping:
public class ValuesController : ControllerBase
{
static ValuesController() {
var methodToReplace = typeof(ValuesController).GetMethod(nameof(ValuesController.Seven),
BindingFlags.Instance | BindingFlags.Static | BindingFlags.NonPublic | BindingFlags.Public);
var methodToAppend = typeof(ValuesController).GetMethod(nameof(ValuesController.Eight),
BindingFlags.Instance | BindingFlags.Static | BindingFlags.NonPublic | BindingFlags.Public);
new Initializer(methodToReplace, methodToAppend);
}
[HttpGet("Seven")]
public int Seven(string id)
{
return 7;
}
[HttpGet("Eight")]
public int Eight(string id)
{
return 8;
}
}
I have a class Initializer which is in charge of handling appending to the method.
public class Initializer
{
public Initializer(MethodInfo methodToReplace, MethodInfo methodToAppend)
{
var dummyMethod = typeof(Initializer).GetMethod(nameof(Dummy),
BindingFlags.Instance | BindingFlags.Static | BindingFlags.NonPublic | BindingFlags.Public);
var proxyMethod = typeof(Initializer).GetMethod(nameof(Proxy),
BindingFlags.Instance | BindingFlags.Static | BindingFlags.NonPublic | BindingFlags.Public);
var appendedMethod = typeof(Initializer).GetMethod(nameof(Appended),
BindingFlags.Instance | BindingFlags.Static | BindingFlags.NonPublic | BindingFlags.Public);
dummyMethod.OneWayReplace(methodToReplace);
methodToReplace.OneWayReplace(proxyMethod);
appendedMethod.OneWayReplace(methodToAppend);
}
public int Proxy(string id)
{
Dummy(id);
return Appended(id);
}
public int Dummy(string id)
{
return 0;
}
public int Appended(string id)
{
return 0;
}
}
And then I have the Extensions which I've obtained from the original stackoverflow question:
public static class InjectionExtensions
{
// Note: This method replaces methodToReplace with methodToInject
// Note: methodToInject will still remain pointing to the same location
public static unsafe MethodReplacementState OneWayReplace(this MethodInfo methodToReplace, MethodInfo methodToInject)
{
//#if DEBUG
RuntimeHelpers.PrepareMethod(methodToReplace.MethodHandle);
RuntimeHelpers.PrepareMethod(methodToInject.MethodHandle);
//#endif
MethodReplacementState state;
IntPtr tar = methodToReplace.MethodHandle.Value;
var inj = methodToInject.MethodHandle.Value + 8;
if (!methodToReplace.IsVirtual)
tar += 8;
else
{
var index = (int)(((*(long*)tar) >> 32) & 0xFF);
var classStart = *(IntPtr*)(methodToReplace.DeclaringType.TypeHandle.Value + (IntPtr.Size == 4 ? 40 : 64));
tar = classStart + IntPtr.Size * index;
}
#if DEBUG
tar = *(IntPtr*)tar + 1;
inj = *(IntPtr*)inj + 1;
state.Location = tar;
state.OriginalValue = new IntPtr(*(int*)tar);
*(int*)tar = *(int*)inj + (int)(long)inj - (int)(long)tar;
return state;
#else
state.Location = tar;
state.OriginalValue = *(IntPtr*)tar;
* (IntPtr*)tar = *(IntPtr*)inj;
return state;
#endif
}
}
Note: Using the current setup everything works fine. However, the second I change the Initializer class to be a generic class Initializer<T> I get a memory violation:
System.AccessViolationException: 'Attempted to read or write protected memory. This is often an indication that other memory is corrupt.'
My guess is that either the methodToReplace.DeclaringType.TypeHandle.Value calculation differs for generics, Or since the compiler is the one who generates the generic class it written to protected memory?
Edit
I've found more information I need to prepare the method properly when using generic parameters e.g:
RuntimeHelpers.PrepareMethod(methodToInject.MethodHandle, new[] { typeof(T).TypeHandle });
However there are still a few more pieces to the puzzle to get this working.
Edit
There are a few open source project's such as harmony that do similar things, However it looks like their emitting their own assemblies. While I've considered the option, I would still prefer to understand how I method tables work with generics
How can I append to methods that reside in generic classes?

I suppose you have already seen: Dynamically replace the contents of a C# method?
I have adapted some of those methods in my own project # https://github.com/juliusfriedman/net7mma_core/blob/master/Concepts/Classes/MethodHelper.cs
I think the problem is that if your are running with the Debugger Attached then you need to also handle the portion of the logic which is currently defined by IFDEF at compilation time and replace that with an System.Diagnostics.Debugger.IsAttached although the offsets calculations (to jump over the debugger injected code) will probably have to change depending on various things like the version of the framework in use.
See https://github.com/juliusfriedman/net7mma_core/blob/master/Concepts/Classes/MethodHelper.cs#L35
This works for me in .Net Core 3.1 when the debugger IS NOT attached and I am running in Release mode, when running in Debug mode with or without the debugger attached or in Release mode with the debugger attached I receive different exceptions. (In debug I receive Arithmetic Overflow, while in release I receive Execution Engine Exception).
Furthermore this only works until the JIT Tiering kicks in, if I run the method a 2nd time without the debugger attached I am getting a Internal CLR Error.
I believe this has to do with the code injected by the debugger when attached and to be honest I am not up to do date on exactly what the debugger is injecting when attached.
I would make a simplified repo of the problem and ask a question # https://github.com/dotnet/runtime if you need this to work with the debugger attached and I am sure someone there will guide you in the right direction.

Selenium with .net core: performance impact, multiple threads in IWebElement?

I've been using Selenium for writing an end to end tests using the .net framework 4.6.
For a new project I'm using Selenium to do some web crawling but I'm experiencing enormous lag on very simple Selenium tasks.
Please look at the screenshot below:
The code itself is here:
public IList<Category> ListAllParentCategories()
{
var categories = new List<Category>();
var liItems = _driver.FindElements(By.CssSelector(".nav__branch.branch ul.tree li"));
foreach(var liItem in liItems)
{
var innerLink = liItem.FindElement(By.TagName("a"));
var c = new Category();
c.Id = long.Parse(liItem.GetAttribute("id"));
c.Link = innerLink.GetAttribute("href");
c.Name = innerLink.Text;
c.Parent = null;
categories.Add(c);
}
return categories;
}
From my experience, this should not take minutes to complete. Basically, we're just looking for a list of LI elements and then we query for some more info related to that element, which does not require interaction from the browser as it's already contained in the WebElement.
Yet this procedure (for 22 categories) takes at least an entire minute. Fetching the innerlink.GetAttribute("id") takes an entire second.
Now while trying to figure out why this is happening, I've been removing all thread related stuff and right now this code runs from a Console app. There's no improvement at all.
A curious thing is that when I open the innerLink in the debugger, I can see that most properties are using another thread?! I assume this is what is causing the long delays?
My setup is as follows:
- Console app running .NET Core 2.0
Dependencies:
- Selenium.Chrome.WebDriver 2.32.00
- Selenium.WebDriver 3.6.0
Update: This problem is definitely related to the .NET core variant of Selenium Webdriver. After converting the project to a regular .NET Framework 4.6 Console app, everything is 10 times faster.

There is a dirty fix for the problem for now.
//this is in my Dependency Register method
string driverPath = $#"{AppDomain.CurrentDomain.BaseDirectory}";
var service = FirefoxDriverService.CreateDefaultService(driverPath);
container.RegisterInstance<IWebDriver>(new FirefoxDriver(service), new PerThreadLifetimeManager());
FixDriverCommandExecutionDelay(container.Resolve<IWebDriver>());
and the method looks something like this:
//to be deleted when https://github.com/dotnet/corefx/issues/24104 is fixed
//in .net core 2.0
public static void FixDriverCommandExecutionDelay(IWebDriver driver)
{
PropertyInfo commandExecutorProperty = typeof(RemoteWebDriver).GetProperty("CommandExecutor", BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.GetProperty);
ICommandExecutor commandExecutor = (ICommandExecutor)commandExecutorProperty.GetValue(driver);
FieldInfo remoteServerUriField = commandExecutor.GetType().GetField("remoteServerUri", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.GetField | BindingFlags.SetField);
if (remoteServerUriField == null)
{
FieldInfo internalExecutorField = commandExecutor.GetType().GetField("internalExecutor", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.GetField);
commandExecutor = (ICommandExecutor)internalExecutorField.GetValue(commandExecutor);
remoteServerUriField = commandExecutor.GetType().GetField("remoteServerUri", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.GetField | BindingFlags.SetField);
}
if (remoteServerUriField != null)
{
string remoteServerUri = remoteServerUriField.GetValue(commandExecutor).ToString();
string localhostUriPrefix = "http://localhost";
if (remoteServerUri.StartsWith(localhostUriPrefix))
{
remoteServerUri = remoteServerUri.Replace(localhostUriPrefix, "http://127.0.0.1");
remoteServerUriField.SetValue(commandExecutor, new Uri(remoteServerUri));
}
}
the answer to the problem is explained fully in this thread: https://github.com/dotnet/corefx/issues/24104
Answer taken from there also, credits given to the good folks!
It works for me on FF driver v0.19.1, Selenium v3.8

How to switch on/off statistics panel in editor while playing?

I want to switch this panel on/off by C# script while playing.
Is this possible? Haven't found any Editor API functions for this.

You can do it with reflection. Modified similar answer I made long ago. Below is a working set/get stats function. Tested with Unity 5.4.0f1. I put the Unity version so that people won't complain when it stops working. Unity's update can break this anytime if they rename any of the variables.
GameView = A class that is used to represent Unity GameView tab in
the Editor.
GetMainGameView = static function that returns current GameView
instance.
m_Stats = a boolean variable that is used to determine if stats
should be displayed or not.
Code:
//Show/Hide stats
void showStats(bool enableStats)
{
Assembly asm = Assembly.GetAssembly(typeof(Editor));
Type type = asm.GetType("UnityEditor.GameView");
if (type != null)
{
MethodInfo gameViewFunction = type.GetMethod("GetMainGameView", BindingFlags.Static |
BindingFlags.NonPublic);
object gameViewInstance = gameViewFunction.Invoke(null, null);
FieldInfo getFieldInfo = type.GetField("m_Stats", BindingFlags.Instance |
BindingFlags.NonPublic | BindingFlags.Public);
getFieldInfo.SetValue(gameViewInstance, enableStats);
}
}
//Returns true if stats is enabled
bool statsIsEnabled()
{
Assembly asm = Assembly.GetAssembly(typeof(Editor));
Type type = asm.GetType("UnityEditor.GameView");
if (type != null)
{
MethodInfo gameViewFunction = type.GetMethod("GetMainGameView", BindingFlags.Static |
BindingFlags.NonPublic);
object gameViewInstance = gameViewFunction.Invoke(null, null);
FieldInfo getFieldInfo = type.GetField("m_Stats", BindingFlags.Instance |
BindingFlags.NonPublic | BindingFlags.Public);
return (bool)getFieldInfo.GetValue(gameViewInstance);
}
return false;
}
Usage:
//Show stats
showStats(true);
//Hide stats
showStats(false);
//Read stats
bool stats = statsIsEnabled();

No it's not possible, unless you are a persistent hacker. GameView is an internal class, not accessible for editor scripting. But hey, there always is an option for good ol' reflection. This question will set you on the right track:
http://answers.unity3d.com/questions/179775/game-window-size-from-editor-window-in-editor-mode.html

Where to find the System.Net.Mail.MailWriter class? [duplicate]

So, the below code used to work in .NET 4 to get a System.Net.Mail.MailMessage object as a MemoryStream, however with the release of .NET 4.5 beta a runtime exception occurs.
Assembly assembly = typeof(SmtpClient).Assembly;
Type mailWriterType = assembly.GetType("System.Net.Mail.MailWriter");
using (MemoryStream stream = new MemoryStream())
{
ConstructorInfo mailWriterContructor = mailWriterType.GetConstructor(BindingFlags.Instance | BindingFlags.NonPublic, null, new[] { typeof(Stream) }, null);
object mailWriter = mailWriterContructor.Invoke(new object[] { stream });
MethodInfo sendMethod = typeof(MailMessage).GetMethod("Send", BindingFlags.Instance | BindingFlags.NonPublic);
sendMethod.Invoke(message, BindingFlags.Instance | BindingFlags.NonPublic, null, new[] { mailWriter, true }, null);
.....
}
Runtime exception occurs on sendMethod.Invoke().

Managed to figure out how to get this working again in .NET 4.5 beta. The private API Send() method in MailMessage has changed to: internal void Send(BaseWriter writer, bool sendEnvelope, bool allowUnicode)
Please find updated code below.
Assembly assembly = typeof(SmtpClient).Assembly;
Type mailWriterType = assembly.GetType("System.Net.Mail.MailWriter");
using (MemoryStream stream = new MemoryStream())
{
ConstructorInfo mailWriterContructor = mailWriterType.GetConstructor(BindingFlags.Instance | BindingFlags.NonPublic, null, new[] { typeof(Stream) }, null);
object mailWriter = mailWriterContructor.Invoke(new object[] { stream });
MethodInfo sendMethod = typeof(MailMessage).GetMethod("Send", BindingFlags.Instance | BindingFlags.NonPublic);
sendMethod.Invoke(message, BindingFlags.Instance | BindingFlags.NonPublic, null, new[] { mailWriter, true, true }, null);
.....
}

This might be usable if you don't want to go with unsupported hacks and don't mind extra performance hit.
public static class MailMessageExtensions
{
public static string RawMessage(this MailMessage m)
{
var smtpClient = new SmtpClient { DeliveryMethod = SmtpDeliveryMethod.SpecifiedPickupDirectory };
using (var tempDir = new TemporaryDirectory())
{
smtpClient.PickupDirectoryLocation = tempDir.DirectoryPath;
smtpClient.Send( m );
var emlFile = Directory.GetFiles( smtpClient.PickupDirectoryLocation ).FirstOrDefault();
if ( emlFile != null )
{
return File.ReadAllText( emlFile );
}
else
return null;
}
return null;
}
}
class TemporaryDirectory : IDisposable
{
public TemporaryDirectory()
{
DirectoryPath = Path.Combine(Path.GetTempPath(), Path.GetRandomFileName());
Directory.CreateDirectory( DirectoryPath );
}
public string DirectoryPath { get; private set; }
public void Dispose()
{
if ( Directory.Exists( DirectoryPath ) )
Directory.Delete( DirectoryPath, true );
}
}

for checking if extra boolean i use :
If _sendMethod.GetParameters.Length = 2 Then
_sendMethod.Invoke(Message, BindingFlags.Instance Or BindingFlags.NonPublic, Nothing, New Object() {_mailWriter, True}, Nothing)
Else
_sendMethod.Invoke(Message, BindingFlags.Instance Or BindingFlags.NonPublic, Nothing, New Object() {_mailWriter, True, True}, Nothing)
End If

The proposed solution with the extra TRUE works beautifully.
I started to getting the error while running my project in VS2012 even though I am not using .net 4.5 but 4.0 in all my libraries.
The error only happens on the machine where you have installed VS2012, looks like VS2012 makes reference to .net 4.5 while you are debugging. When you deploy and run the application in clients running .net 4.0 everything works fine.
Thus : If you run 4.0 - do not add the extra TRUE, if you run 4.5 add it.

We fought with the mail message conversion for a long time. Ultimately the solution was to use MimeKit.
var memoryStream = new MemoryStream();
var mimeMessage = MimeMessage.CreateFromMailMessage(message);
mimeMessage.WriteTo(memoryStream);
If you use the methods above you will get really close and it will work in most cultures but eventually the subject encoding will defeat you.

For those, who are struggling with mailWriterContructor being null in .NET 5 or facing Parameter count mismatch exception, take a closer look on my solution usable for any stream. Link here

HttpModule to add headers to request

This seems like a simple operation.
We have a need in our development environment (running on XP/IIS 5) to add some headers into each HttpRequest arriving at our application. (This is to simulate a production environment that we don't have available in dev). At first blush, this seemed like a simple HttpModule, along the lines of:
public class Dev_Sim: IHttpModule
{
public void Init(HttpApplication app)
{
app.BeginRequest += delegate { app.Context.Request.Headers.Add("UserName", "XYZZY"); };
}
public void Dispose(){}
}
But on trying to do that, I find that the Headers collection of the Request is read-only, and the Add method fails with an OperationNotSupported exception.
Spending a couple hours researching this on Google, I've come up with no easy answer to what should be a relatively straight-forward problem.
Does anyone have any pointers?

Okay, with the assistance of a co-worker and some experimentation, I found that this can be done with the assistance of some protected properties and methods accessed through reflection:
var headers = app.Context.Request.Headers;
Type hdr = headers.GetType();
PropertyInfo ro = hdr.GetProperty("IsReadOnly",
BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.IgnoreCase | BindingFlags.FlattenHierarchy);
// Remove the ReadOnly property
ro.SetValue(headers, false, null);
// Invoke the protected InvalidateCachedArrays method
hdr.InvokeMember("InvalidateCachedArrays",
BindingFlags.InvokeMethod | BindingFlags.NonPublic | BindingFlags.Instance,
null, headers, null);
// Now invoke the protected "BaseAdd" method of the base class to add the
// headers you need. The header content needs to be an ArrayList or the
// the web application will choke on it.
hdr.InvokeMember("BaseAdd",
BindingFlags.InvokeMethod | BindingFlags.NonPublic | BindingFlags.Instance,
null, headers,
new object[] { "CustomHeaderKey", new ArrayList {"CustomHeaderContent"}} );
// repeat BaseAdd invocation for any other headers to be added
// Then set the collection back to ReadOnly
ro.SetValue(headers, true, null);
This works for me, at least.

You can add to the Header this way. This is a way to add credential information to the request before it enter the authentication sequence.
string cred = "UN:PW";
System.Web.HttpContext.Current.Request.Headers.Add("Authorization", "Basic " +Convert.ToBase64String(Encoding.ASCII.GetBytes(cred)));