Develop Reference

can I use static classes in Nlua? [closed]

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 2 years ago.
Improve this question
First and formost, I am not a native english speaker, sorry for any communication errors this question might emit. Also, couldn't really understand some of your guidelines for asking a question, so followed what I understood of it. As a consequence, please tell me if I violate any custom.
background
I have been making an app that trys to implement something like xaml hot reload for code. As I can't directly make it with compiled languages like c#, I decided I will rely on a third-party scripting language, thinking I could write a bit of my app logic in it, then deploy it faster to the device because I would only need to press a button to send the code across. So, one can think of it like the expo client in the react-native world.
I decided to use lua as the scripting language, not only because it's very simple to write code in, but also because it has a nice binding for .net, I'm talking about Nlua, of course. And, as ironpython3 is not ready for production yet, what choices do I have honestly?
the problem I am facing
So, as I have to bind some xamarin.essentials stuff to make it easier for me to write the lua side of the app, I have to interface with some objects that have, among many other things, static methods. I thought it would be very intuitive and straight-forward to use static methods in Nlua, though it seemns I was mistaken so far.
how to reproduce the issue
So, in visual studio, create a xamarin.forms project, name it however you like, it doesn't matter.
Next, add the Nlua nuget package to all your projects. It needs to be integrated in all of them because it, like some xamarin.forms plugins, relies on native libraries to work, so they need to be included in the proper places for the OS to find them. This is automatically done by the package when installed in a supported project type, so do that.
Add the Nlua namespace to the top of the codebehind of your MainPage content page, like this:
using NLua;
For easy use going forward, add a class level property to the MainPage class, like this:
Lua LuaState{get; set;}
Now, initialise it in the page constructor, for simplicity's sake:
LuaState = new Lua();
In the xaml of the page, add an edit field in which to input the text and a button to be able to run it:
<StackLayout>
<Editor
x:Name="edCode"
Placeholder="type your code here"
/>
<Button
x:Name="btnStartCode"
Text="compile"
HorizontalOptions="End"
VerticalOptions="EndAndExpand"
Clicked="btnStartCode_Clicked"
/>
</StackLayout>
In the button clicked handler, add code to load the things in the edit box, then make nlua execute it, something like this:
async void btnStartCode_Clicked(object sender, EventArgs e)
{
try
{
LuaState.DoString(edCode.Text);
}
catch (Exception exc)
{
await DisplayAlert("error", $"A fatal error was incountered while running your code\nException details:\n\tException type: {exc.GetType().ToString()}\n\terror message from interpreter:{exc.Message}.", "OK");
}
}
Now, add a new class, call it however you want, I called mine tts. Delete everything inside the file vs generated, then add the following content in it:
using System;
using System.Collections.Generic;
using System.Text;
using Xamarin.Essentials;
namespace autoaccess.scriptables
{
public class tts
{
public async void speak(params object[] messages)
{
foreach(var message in messages)
{
if (message == null)
{
await TextToSpeech.SpeakAsync("null value");
return;
}
await TextToSpeech.SpeakAsync(message.ToString());
}
}
}
}
Back in the main content class, either in the constructor or in a dedicated method called from the constructor, register the lua type, like this:
LuaState["text_to_speech"=typeof(TextToSpeech);
expected behaviour
It should speak the params given to the function if using . notation, e.g tts.speak("hello world")
actual behaviour
When a call with . notation is executed, the lua interpreter throws the following error:
[string "chunk"]:1: attempt to call a nil value (field 'speak').
what I tryed so far
First, I tried to get the type of the static object with the typeof keyword, then registering it directly as a lua accessible value, like I have shown above. This threw an error, as you could see.
If, for example, I try to invoke the type like a function to get a new object like shown in the documentation, it fails again.
If I use the State.UseClrPackage(); method, then importing the stuff manually, it doesn't work either, can't access any of the methods or variables, lua says they're all "nill".
Note: Upon closer inspection, I see the variable lua sees is actually the real clr type, since the str function of lua shows the full type qualifier, as present in .net. However, the type function returns userdata.
Instead of making the entire type be visible to lua as it is, I tried to make a lua table with the name, then add key-value pairs where the key is the method name, and the value is a static object's method, as returned by typeof(thing).GetMethod(MethodName) cast to the LuaFunction type.
This worked to a certain point, though it quickly grue too complicated and overwhelmingfor me, so I stopped using that approach, as my codebase grue very intangled.
Another thing I tryed is to make proxi objects for static ones in which nothing is actually static, even though the methods return void and are acting on static properties.
That, again, kind of worked, but I then necessarely had to use the object:thing, when the norm for static methods is object.thing. If I tryed to use the object.thing notation, I got a pretty long error, though I'm sure it's content is not relevant to the problem at hand, so not going to include it here.
conclusion
So, officially, I tryed everything I could, even did a search on here to see if I would be able to find anything to enlighten me, though no luck, unfortunately. Now, what do you recommend I use? Is there something I overlooked somewhere?

C# Reflection inconsistency with COM objects

Having spent the last few days reading everything I can find about C# reflection on COM objects, trying many experiments in code, and analyzing sample code to try and improve my understanding, I am now forced to admit that I just don't know enough, so I am requesting help from the Community.
I need to be able to access and update the properties of a late bound COM object that is wrapped as System._COM Object.
I tried all the standard refection stuff without success and I looked through using IDispatch, but I'm not comfortable with using the pointers involved, so I'm hoping I have missed something pretty simple in the normal interface. I found papers on MSDN that DO show how to do what I need, but all the examples are in C++ and it is over my head.
It would be really helpful if someone could explain why the following simple C# code just doesn't work as expected:
try
{
// late binding:
// localCB is a COM object (System._COMObject) created by Activator.CreateInstance() from
// the ProgID of a registered COM .DLL.
//
// The original .DLL has a string PROPERTY called
// "TESTEXTERNAL1". localCB has an IDispatch Interface. The original COM .DLL has a separate Typelib,
// and, although I did not register the typelib separately, I can see from OLEView on the COM object
// that the GUID for the typelib is included in it.
// Here's the code that is puzzling me...
var vv = localCB.GetType().InvokeMember("TESTEXTERNAL1", BindingFlags.GetProperty,
null, localCB, null);
string rt = vv.ToString();
// works exactly as expected and returns the value of TESTEXTERNAL1 - OK.
// now try and update the SAME PROPERTY, on the SAME COM object...
Parameters = new Object[1];
Parameters[0] = "Hello, World!";
localCB.GetType().InvokeMember("TESTEXTERNAL1", BindingFlags.SetProperty,
null, localCB, Parameters);
// throws an (inner) exception: HRESULT 0x8002003 DISP_E_MEMBERNOTFOUND !!!
}
catch (Exception xa)
{
string xam = xa.Message;
}
Is it unreasonable to expect an object that has already found and provided a property, to be able to update the same property? Is there some "alternative update" strategy that I am not aware of?
Many thanks for any help,
Pete.
UPDATE:
in response to Jon's request, here are snippets of the OleView:
(I had to use images because Oleview would not let me cut & paste, sorry...)
OleView of the COM .DLL
OLEView typelib view
Jon, I think you have correctly identified that the problem is with a setter method. The DLL is written in Fujitsu COBOL and provides an "under the covers" GET and SET for fields identified as PROPERTY. Accessing the COM component from C# or COBOL, it works fine, but, as you can see, it doesn't work when I try and access it for SET with reflection. Because I am unfamiliar with using reflection I was doubtful whether I had the syntax right, so I tried to make the SET as close as possible to the GET. I think I will need to generate my own SET methods (for each PROPERTY) into the COBOL and then change my "BindingFlags.SetProperty" to be "BindingFlags.InvokeMember". (I did the homework on BindingFlags and found that if you specify "SetProperty" it automatically implies the other 2 flags you mentioned.)
I think the key to it all is in recognizing that the problem is with the Fujitsu *COM Class SET, and it took your experienced eye to see that. Many thanks. If you have any other comments after seeing the OLEView, or can suggest any alternative approach in order to get the properties set, I'd be very interested. (I'm not looking forward to having to generate SETter methods for every property; it smacks of brute force... :-))
Thanks again,
Pete.

Hans was correct. The problem was with the setter method. I have written code to generate a setter for each of the properties, back in the original COBOL COM component. It wasn't as tedious or ugly as I thought it would be (about 7 lines of COBOL for each PROPERTY) and it is all working very well now. Many thanks to the community and particularly Hans Passant for support.

How do I invoke .getmethodinfo?

I have recently started to read how a open source encryption program works.
The problem is, I'm stuck here on these 2 lines of code which I can't understand at all, I have looked at MSDN but it doesn't help me at all to understand what these 2 lines do.
MethodInfo run = Assembly.Load(InjRes).GetType("Resource.reflect").GetMethod("Run");
bool inj = (bool)run.Invoke(null, new object[] { Assembly.GetExecutingAssembly().Location, "", PayloadRes, false });
I'm also trying to figure out another way to accomplish the same thing, is there a way to invoke the .GetMethod?

This is fairly basic "reflection" in .NET.
Basically what happens here is:
Loads a .NET assembly as indicated by InjRes
Inside that assembly, get the type definition for Resource.reflect (which is probably a class)
Inside the Resource.reflect type, get the Run method and save it to variable run
Invoke (in other works, "execute") the Run() function on a null instance (which means Run() is probably a static method) with the parameters (Assembly.GetExecutingAssembly().Location, "", PayloadRes, false)
More simply, these lines would translate to "regular old C#" that looks like:
bool inj = Resource.reflect.Run(Assembly.GetExecutingAssembly().Location, "", PayloadRes, false);
but the original author probably couldn't have just written the simple 1 line above because either:
The assembly wasn't referenced at build-time, so the compiler doesn't know about the Resouce namespace so would have thrown a compile error
or the Resouce.reflect class or the Run method isn't public so couldn't have been called normally. Reflection can bypass this limitation.

Why is JIT_MethodAccessAllowedBySecurity taking so much time?

I'm working on a C# application that allows users to basically import tables of data, and then enter their own formulas in a mini-language to compute new columns from the underlying data.
These formulas are compiled into LINQ expression trees in the engine, which the .NET 4.0 expression tree library then presumably compiles into IL so they can be executed.
We've started using our engine for some high-volume ticking data recently, and we're finding the speed of these compiled expression trees to be a real bottleneck - the speed is pretty slow when re-calculating all these columns on the fly. Hitting it with the built-in Visual Studio 2010 profiler reveals half of all our execution time is being spent in clr.dll, in a method called JIT_MethodAccessAllowedBySecurity.
Cursory googling of this string hasn't yielded anything, so I'm wondering if there's anyone out there who can tell me what this method is, and whether there's a way to keep it from eating up all my cycles? Maybe there's a way to compile this code and explicitly give it permission to do whatever it wants so the clr can stop these checks? Perhaps the temporary assemblies being generated by the expression tree engine do not have full trust?
Anyhow, I'm pretty much at a loss and I'm very interested to hear if any other StackOverflow'ers have come across this issue in the past. Thanks in advance!

The solution is to use LambdaExpression.CompileToMethod(MethodBuilder method) instead of LambdaExpression.Compile().
I think Jethro was on the right track when he posited that CAS was involved. In testing, the profiler only started showing calls to JIT_MethodAccessAllowedBySecurity when I used expression trees to call functions that weren't dynamically defined in the generated assembly (i.e. using Expression.Call to invoke a library method rather than a piece of generated code.) This suggests that the slowdown was caused by CAS checking that my generated code had access to the methods it was invoking. It seems to follow that by applying declarative security modifications to the functions I wished to call, I could avoid this overhead.
Unfortunately, I wasn't able to get rid of the JIT_MethodAccessAllowedBySecurity overhead through any use of declarative security (PermissionSet, SecurityAction.LinkDemand, and the like). At one point I had literally every method in my project marked with [PermissionSet(SecurityAction.LinkDemand, Unrestricted = true)], with no results.
Luckily, when looking for ways to add attributes to the generated delegates, I stumbled upon the solution - using a MethodBuilder to compile the expression tree rather than the built-in LambdaExpression.Compile method.
I've included the bit of code that replaced .Compile() and led to elimination of the JIT_MethodAccessAllowedBySecurity calls and a >2x speedup in our calculation engine:
// T must be of delegate type (Func<T>, Func<T1, T2>, etc.)
public static T GetCompiledDelegate<T>(Expression<T> expr)
{
var assemblyName = new AssemblyName("DelegateHostAssembly") { Version = new Version("1.0.0.0") };
var assemblyBuilder =
AppDomain.CurrentDomain.DefineDynamicAssembly(
assemblyName,
AssemblyBuilderAccess.RunAndSave);
var moduleBuilder = assemblyBuilder.DefineDynamicModule("DelegateHostAssembly", "DelegateHostAssembly.dll");
var typeBuilder = moduleBuilder.DefineType("DelegateHostAssembly." + "foo", TypeAttributes.Public);
var methBldr = typeBuilder.DefineMethod("Execute", MethodAttributes.Public | MethodAttributes.Static);
expr.CompileToMethod(methBldr);
Type myType = typeBuilder.CreateType();
var mi = myType.GetMethod("Execute");
// have to box to object because .NET doesn't allow Delegates as generic constraints,
// nor does it allow casting of Delegates to generic type variables like "T"
object foo = Delegate.CreateDelegate(typeof(T), mi);
return (T)foo;
}
This code is consistently >2x faster when using any code that uses Expression trees to call functions that are not themselves defined by Expression trees. Thanks for everyone's help, and I hope this saves someone else some cycles.

I think this has something to do with CAS (Code access security).
CAS is assembly based. When you code
calls a protected method, the .NET
framework runtimes checks your
assembly to see if it has been granted
on or more permissions necessary for
the method. The .NET Framework rutime
then walks the stack to check every
assembly in the stack for these
ermissions. If one assembly does not
have all of the required permissions,
a securityexception is raised and the
code is run.
The below is what I think is happening to your code.
... a stack walk occurs and policy
check is preformed every time that the
method is called. This is a particular
problem for components in a class
library, which may be called many
times. In this situation, you can use
link demand to indicate that the
permission set check is performed on
at link time as part of the JIT
compliction process. To do this, you
decorate the method by using a
permission attribute that has a
parameter of the value
SecurityAction.LinkDemand.
I hope this helps, it looks like all you need to do is set the SecurityAction.LinkDemand attribute. The quoted text comes from Advanced foundations of Microsoft .NET 2.0 Development.
Regards

Passing a lambda to a secondary AppDomain as a stream of IL and assembling it back using DynamicMethod

Is it possible to pass a lambda expression to a secondary AppDomain as a stream of IL bytes and then assemble it back there using DynamicMethod so it can be called?
I'm not too sure this is the right way to go in the first place, so here's the (detailed) reason I ask this question...
In my applications, there are a lot of cases when I need to load a couple of assemblies for reflection, so I can determine what to do with them next. The problem part is I need to be able to unload the assemblies after I'm finished reflecting over them. This means I need to load them using another AppDomain.
Now, most of my cases are sort of similar, except not quite. For example, sometimes I need to return a simple confirmation, other times I need to serialize a resource stream from the assembly, and again other times I need to make a callback or two.
So I end up writing the same semi-complicated temporary AppDomain creation code over and over again and implementing custom MarshalByRefObject proxies to communicate between the new domain and the original one.
As this is not really acceptable anymore, I decided to code me an AssemblyReflector class that could be used this way:
using (var reflector = new AssemblyReflector(#"C:\MyAssembly.dll"))
{
bool isMyAssembly = reflector.Execute(assembly =>
{
return assembly.GetType("MyAssembly.MyType") != null;
});
}
AssemblyReflector would automize the AppDomain unloading by virtue of IDisposable, and allow me to execute a Func<Assembly,object>-type lambda holding the reflection code in another AppDomain transparently.
The problem is, lambdas cannot be passed to other domains so simply. So after searching around, I found what looks like a way to do just that: pass the lambda to the new AppDomain as an IL stream - and that brings me to the original question.
Here's what I tried, but didn't work (the problem was BadImageFormatException being thrown when trying to call the new delegate):
public delegate object AssemblyReflectorDelegate(Assembly reflectedAssembly);
public class AssemblyReflector : IDisposable
{
private AppDomain _domain;
private string _assemblyFile;
public AssemblyReflector(string fileName) { ... }
public void Dispose() { ... }
public object Execute(AssemblyReflectorDelegate reflector)
{
var body = reflector.Method.GetMethodBody();
_domain.SetData("IL", body.GetILAsByteArray());
_domain.SetData("MaxStackSize", body.MaxStackSize);
_domain.SetData("FileName", _assemblyFile);
_domain.DoCallBack(() =>
{
var il = (byte[])AppDomain.CurrentDomain.GetData("IL");
var stack = (int)AppDomain.CurrentDomain.GetData("MaxStackSize");
var fileName = (string)AppDomain.CurrentDomain.GetData("FileName");
var args = Assembly.ReflectionOnlyLoadFrom(fileName);
var pars = new Type[] { typeof(Assembly) };
var dm = new DynamicMethod("", typeof(object), pars,
typeof(string).Module);
dm.GetDynamicILInfo().SetCode(il, stack);
var clone = (AssemblyReflectorDelegate)dm.CreateDelegate(
typeof(AssemblyReflectorDelegate));
var result = clone(args); // <-- BadImageFormatException thrown.
AppDomain.CurrentDomain.SetData("Result", result);
});
// Result obviously needs to be serializable for this to work.
return _domain.GetData("Result");
}
}
Am I even close (what's missing?), or is this a pointless excercise all in all?
NOTE: I realize that if this worked, I'd still have to be carefull about what I put into lambda in regard to references. That's not a problem, though.
UPDATE: I managed to get a little further. It seems that simply calling SetCode(...) is not nearly enough to reconstruct the method. Here's what's needed:
// Build a method signature. Since we know which delegate this is, this simply
// means adding its argument types together.
var builder = SignatureHelper.GetLocalVarSigHelper();
builder.AddArgument(typeof(Assembly), false);
var signature = builder.GetSignature();
// This is the tricky part... See explanation below.
di.SetCode(ILTokenResolver.Resolve(il, di, module), stack);
dm.InitLocals = initLocals; // Value gotten from original method's MethodInfo.
di.SetLocalSignature(signature);
The trick is as follows. Original IL contains certain metadata tokens which are valid only in the context of the original method. I needed to parse the IL and replace those tokens with ones that are valid in the new context. I did this by using a special class, ILTokenResolver, which I adapted from these two sources: Drew Wilson and Haibo Luo.
There is still a small problem with this - the new IL doesn't seem to be exactly valid. Depending on the exact contents of the lambda, it may or may not throw an InvalidProgramException at runtime.
As a simple example, this works:
reflector.Execute(a => { return 5; });
while this doesn't:
reflector.Execute(a => { int a = 5; return a; });
There are also more complex examples that are either working or not, depending on some yet-to-be-determined difference. It could be I missed some small but important detail. But I'm reasonably confident I'll find it after a more detailed comparison of the ildasm outputs. I'll post my findings here, when I do.
EDIT: Oh, man. I completely forgot this question was still open. But as it probably became obvious in itself, I gave up on solving this. I'm not happy about it, that's for sure. It's really a shame, but I guess I'll wait for better support from the framework and/or CLR before I attempt this again. There're just to many hacks one has to do to make this work, and even then it's not reliable. Apologies to everyone interested.

I didn't get exactly what is the problem you are trying to solve, but I made a component in the past that may solve it.
Basically, its purpose was to generate a Lambda Expression from a string. It uses a separate AppDomain to run the CodeDOM compiler. The IL of a compiled method is serialized to the original AppDomain, and then rebuild to a delegate with DynamicMethod. Then, the delegate is called and an lambda expression is returned.
I posted a full explanation of it on my blog. Naturally, it's open source. So, if you get to use it, please send me any feedback you think is reasonable.

Probably not, because a lambda is more than just an expression in source code. lambda expressions also create closures which capture/hoist variables into their own hidden classes. The program is modified by the compiler so everywhere you use those variables you're actually talking to the class. So you'd have to not only pass the code for the lambda, but also any changes to closure variables over time.