Develop Reference

ScriptBuffer throws NullReferenceException when called from an Event as part of a loop - SSIS Script Component [duplicate]

This question already has answers here:
What is a NullReferenceException, and how do I fix it?
(27 answers)
Closed 1 year ago.
I have an SSIS package which calls a Data Flow Task as part of a loop which iterates different end-point addresses (out of scope).
The Data Flow Task has a source Script Component responsible for calling a REST API and creating a row for each result.
There are 3 output buffers;
1. actual data row
2. error row
3. monitoring
The monitoring buffer used for telemetry and is populated through an event (EventHander) that is fired every time the API makes a request.
During the first iteration of the ForEach int the Control Flow loop, everything runs as expected, all the buffers produce the correct rows.
However, during the next iterations, the monitoring buffer which is populated within the event throws;
System.NullReferenceException: Object reference not set to an instance of an object.
at Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.HandleUserException(Exception e)
at Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.PrimeOutput(Int32 outputs, Int32[] outputIDs, PipelineBuffer[] buffers)
at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostPrimeOutput(IDTSManagedComponentWrapper100 wrapper, Int32 outputs, Int32[] outputIDs, IDTSBuffer100[] buffers, IntPtr ppBufferWirePacket)
I don't understand why the MonitoringBuffer is not initialised in the proceeding iterations.
The exception occurs while calling MonitoringBuffer.AddRow();.
Here's the whole Script Component simplified for readability:
[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]
public class ScriptMain : UserComponent
{
private string ClientCode { get { return Variables.ErplyClientCode; } }
private string Username { get { return Variables.ErplyUsername; } }
private string Password { get { return Variables.ErplyPassword; } }
private bool IsTest { get { return Variables.IsTest; } }
private int ErplyRecordsPerPage { get { return Variables.ErplyRecordsPerPage; } }
private string ErplyDebugOutputPath { get { return Variables.ErplyDebugOutputPath; } }
private DateTime ChangeSince { get { return Variables.ChangeSince; } }
private int records { get; set; }
private int errors { get; set; }
private string rawFolder { get; set; }
public override void PreExecute()
{
base.PreExecute();
}
public override void PostExecute()
{
base.PostExecute();
}
public override void CreateNewOutputRows()
{
ErplyAPI.OnPreRequestEvent += new EventHandler<EAPIEvent>(ErplyAPI_OnPreRequestEvent);
var staff = ErplyAPI.getStaff(ClientCode, Username, Password, ChangeSince, ErplyRecordsPerPage, IsTest);
foreach (var p in staff.List)
{
try
{
if (!p.IsError)
{
EmployeeBuffer.AddRow();
EmployeeBuffer.employeeID = p.employeeID;
}
else
{
ErrorBuffer.AddRow();
ErrorBuffer.employeeID = p.employeeID;
ErrorBuffer.Error = p.Error.Message.Trim() + "\n" + p.Error.StackTrace;
errors++;
}
records++;
}
catch (Exception ex)
{
this.ComponentMetaData.FireWarning(0, "Script", ex.Message + "\n" + ex.StackTrace, string.Empty, 0);
}
}
EmployeeBuffer.SetEndOfRowset();
ErrorBuffer.SetEndOfRowset();
}
private void ErplyAPI_OnPreRequestEvent(object sender, EAPIEvent e)
{
var request = string.Empty;
var sessionKey = string.Empty;
bool fireAgain = true;
if (e == null)
{
ComponentMetaData.FireWarning(0, "SC_ERPLY_API", string.Format("EAPIEvent is NULL in ErplyAPI_OnPreRequestEvent. Amonit did not log the Erply request."), string.Empty, 0);
return;
}
if (e.eAPI == null)
{
ComponentMetaData.FireWarning(0, "SC_ERPLY_API", string.Format("EAPIEvent.eAPI is NULL in ErplyAPI_OnPreRequestEvent. Amonit did not log the Erply request."), string.Empty, 0);
return;
}
try
{
if (e.Parameters != null && e.Parameters.ContainsKey("request"))
request = e.Parameters["request"].ToString();
if (request != "verifyUser" && e.Parameters != null && e.Parameters.ContainsKey("sessionKey"))
sessionKey = e.Parameters["sessionKey"].ToString();
}
catch (Exception ex)
{
ComponentMetaData.FireWarning(0, "SC_ERPLY_API", string.Format("Error occurred assigning variables from EAPIEvent parameters in ErplyAPI_OnPreRequestEvent. {0} {1}", ex.Message, ex.StackTrace), string.Empty, 0);
}
try
{
MonitoringBuffer.AddRow(); // Exception occurs here
MonitoringBuffer.Request = ResizeString(request, 255);
MonitoringBuffer.SessionKey = ResizeString(sessionKey, 128);
}
catch (Exception ex)
{
var message = string.Format("Error occurred outputting Erply request in ErplyAPI_OnPreRequestEvent. {0} {1}", ex.Message, ex.StackTrace);
MonitoringBuffer.ErrorMessage = ResizeString(message, 8000);
ComponentMetaData.FireWarning(0, "SC_ERPLY_API", message, string.Empty, 0);
}
finally
{
MonitoringBuffer.EndOfRowset();
}
}
}

I sorted the problem out.
The exception was being raised when the variable dispenser was being accessed from the Event. For some reason the GetValueWithContext(ScriptComponent.EvaluatorContext) is being dropped during the second call. Why this happens is beyond me.
The solution is simple, assign the variables from the variables dispenser to a local property or variable in the OnPreExecute function.
It's also good practice to not call the variable dispenser in the CreateNewOutputRows as it cause variable locking.

I ran into this issue too, but my solution was a little different -- moving the variable assignments into PreExecute() didn't help.
Instead, what I'd done is that I wanted to parse three different files, and read each of them with a Script Component. Their columns were kinda similar, so I created one Data Flow task, made sure it worked, then copied it and modified each copy to reflect the differences in the files. Running each individual Data Flow task was successful, but when I tried to run two of them, one after the other in a loop, I got a NullReferenceException from HostPrimeOutput() after calling the OutputBuffer.AddRow() method in my Script Component.
It turns out that when I copied each Data Flow task, the Script Components all kept the same namespace, and I guess it doesn't like that. So, I created brand new Script Components, set up all the output columns again (ugh!), copied the body of the script over, and it's happy.

cannot evaluate expression because a native frame is on top of the call stack and system.accessviolationexception

I have this code using c#.
public partial class MainForm : Form
{
private CvCapture VideoCapture;
private IplImage frame;
private IplImage imgMain;
public MainForm()
{
InitializeComponent();
}
private void btnVideo_Click(object sender, EventArgs e)
{
double vidWidth, vidHeight;
try
{
VideoCapture = highgui.CvCreateCameraCapture(0);
}
catch (Exception except)
{
MessageBox.Show(except.Message);
}
if (btnVideo.Text.CompareTo("Start Video") == 0)
{
if (VideoCapture.ptr == IntPtr.Zero)
{
MessageBox.Show("badtrip ah!!!");
return;
}
btnVideo.Text = "Stop Video";
highgui.CvSetCaptureProperty(ref VideoCapture, highgui.CV_CAP_PROP_FRAME_WIDTH, 640);
highgui.CvSetCaptureProperty(ref VideoCapture, highgui.CV_CAP_PROP_FRAME_HEIGHT, 480);
highgui.CvQueryFrame(ref VideoCapture);
vidWidth = highgui.cvGetCaptureProperty(VideoCapture, highgui.CV_CAP_PROP_FRAME_WIDTH);
vidHeight = highgui.cvGetCaptureProperty(VideoCapture, highgui.CV_CAP_PROP_FRAME_HEIGHT);
picBoxMain.Width = (int)vidWidth;
picBoxMain.Height = (int)vidHeight;
timerGrab.Enabled = true;
timerGrab.Interval = 42;
timerGrab.Start();
}
else
{
btnVideo.Text = "Start Video";
timerGrab.Enabled = false;
if (VideoCapture.ptr == IntPtr.Zero)
{
highgui.CvReleaseCapture(ref VideoCapture);
VideoCapture.ptr = IntPtr.Zero;
}
}
}
private void timerGrab_Tick(object sender, EventArgs e)
{
try
{
frame = highgui.CvQueryFrame(ref VideoCapture);
if (frame.ptr == IntPtr.Zero)
{
timerGrab.Stop();
MessageBox.Show("??");
return;
}
imgMain = cxcore.CvCreateImage(cxcore.CvGetSize(ref frame), 8, 3);
picBoxMain.Image = highgui.ToBitmap(imgMain, false);
cxcore.CvReleaseImage(ref imgMain);
//cxcore.CvReleaseImage(ref frame);
}
catch (Exception excpt)
{
MessageBox.Show(excpt.Message);
}
}
}
The problem is after i break all and step through the debugger the program stops at a certain code.
the code where it stops is here: frame = highgui.CvQueryFrame(ref VideoCapture);
the error is that it says that cannot evaluate expression because a native frame is on top of the call stack.
and then when i try to shift+F11 it. there is another error saying that system.accessviolationexception.
the stack trace says that: at System.Runtime.InteropServices.Marshal.CopyToManaged(IntPtr source, Object destination, Int32 startIndex, Int32 length)
at CxCore.IplImage.get_ImageDataDb()
Here is the error i get when i changed the c# opencv wrapper i used
from the cvlib to cvwrapcs230d.
Also the DLL's used in the project is included in the folder.
But when i changed the wrapper from cvwrapcs230d to cvwrap230 here is
the error i get.
The first Note is the cvwrapcs230d wrapper used while the second is
the cvwrapcs230 wrapper.
1st Note: cvwrapcs230d
-Thrown: "Unable to load DLL 'cvwrapcpp230': The specified module could not be found. (Exception from HRESULT: 0x8007007E)"
(System.DllNotFoundException) Exception Message = "Unable to load DLL
'cvwrapcpp230': The specified module could not be found. (Exception
from HRESULT: 0x8007007E)", Exception Type =
"System.DllNotFoundException" Exception Message "Unable to load DLL
'cvwrapcpp230': The specified module could not be found. (Exception
from HRESULT: 0x8007007E)" string Exception
Type "System.DllNotFoundException" string
-Thrown: "The type initializer for 'cvlib' threw an exception." (System.TypeInitializationException) Exception Message = "The type
initializer for 'cvlib' threw an exception.", Exception Type =
"System.TypeInitializationException" Exception Message "The type
initializer for 'cvlib' threw an exception." string Exception
Type "System.TypeInitializationException" string Stopped at
Exception
2nd Note: cvwrapcs230
-Thrown: "Unable to load DLL 'cvwrapcpp230': The specified module could not be found. (Exception from HRESULT: 0x8007007E)"
(System.DllNotFoundException) Exception Message = "Unable to load DLL
'cvwrapcpp230': The specified module could not be found. (Exception
from HRESULT: 0x8007007E)", Exception Type =
"System.DllNotFoundException" Exception Message "Unable to load DLL
'cvwrapcpp230': The specified module could not be found. (Exception
from HRESULT: 0x8007007E)" string Exception
Type "System.DllNotFoundException" string
-cvCreateCameraCapture cvlib.cvCreateCameraCapture(index = {unknown}) index {unknown} int

invoke with multiple arguments resulting in FormatException

I'm trying to invoke a Form method from a different thread. In the form class, I have:
delegate int ReplaceMessageCallback(string msg, int key);
public int ReplaceMessage(string msg, int key)
{
if (this.InvokeRequired)
{
ReplaceMessageCallback amc = new ReplaceMessageCallback(ReplaceMessage);
object[] o = new object[] { msg, key };
return (int)this.Invoke(amc, o);
}
bool found = false;
int rv;
lock (this)
{
if (key != 0)
{
found = RemoveMessage(key);
}
if (found)
{
rv = AddMessage(msg, key);
}
else
{
rv = AddMessage(msg);
}
}
MainForm.EventLogInstance.WriteEntry((found)
? EventLogEntryType.Information
: EventLogEntryType.Warning,
IntEventLogIdent.MessageFormReplace1,
String.Format("MessageForm::ReplaceMessage(({2},{0}) returns {1}.\n\n(The message {3} exist to be replaced.)",
key,
rv,
msg,
(found)
? "did"
: "did not"));
return rv;
}
When I run this, I get an exception "FormatException was unhandled" "Index (zero based) must be greater than or equal to zero and less than the size of the argument list." on the call to Invoke.
Essentially this same code fragment works fine on class methods that only take a single parameter, so I assume I'm doing something wrong with the object array but I have no idea what.

An easier way to handle this is:
if (this.InvokeRequired)
{
int rslt;
this.Invoke((MethodInvoker) delegate
{
rslt = ReplaceMessage(msg, key);
}
return rslt;
}

It turns out that the invoke call will pass along exceptions within the function it calls, and you can't step (F11 in debugger) into it. I assumed that it would step into the called code, so when it failed I thought it was the actual Invoke call.
I messed up a String.Format in the body of the function, and Invoke passed that exception to me with no indication of where in the code the problem actually happened.

How to marshal function with out parameter to native code?

I have delegate that is declared as:
internal delegate bool readDelegate(out byte data);
At some point i want to get a function pointer to an instance of the delegate:
readDelegate reader = (out byte data) =>
{
Console.WriteLine("Managed output! opening");
if (stream.CanRead)
{
data = (byte)stream.ReadByte();
return true;
}
else
{
data = 0;
return false;
}
};
which reads a singy byte from stream, if able.
But when i try to get a function pointer, like this:
IntPtr foo = Marshal.GetFunctionPointerForDelegate(reader);
I get an exception:
System.NotSupportedException was unhandled
Message="0x80131515"
StackTrace:
в System.Runtime.InteropServices.Marshal.GetFunctionPointerForDelegate(Delegate d)
...
What is the reason? How should i change my code?

What's the rest of the message of the exception? There should be a explanation of the error.
Anyway, I think this might work:
internal unsafe delegate bool readDelegate(byte* data);

C# pass int and string by reference to C++ ActiveX Control: type mismatch

I have a problem passing by reference int or string variables to C++ ActiveX Control.
Also I pass these variables by reference to C++ DLL and everything works fine.
C++ DLL:
__declspec (dllexport) void
Execute (LPCTSTR cmd, int& resultCode, LPCTSTR& message, long& receiptNumber)
{
message = _T("ReplyProblem");
resultCode = 100;
receiptNumber = -1;
}
C#:
[DllImport("MyCOM.dll", CharSet = CharSet.Unicode)]
public static extern void Execute (string cmd, out int resultCode, out string message, out int receiptNumber);
...
int resultCode = 0;
string message = "";
int receiptNumber = 0;
Execute ("cmd", out resultCode, out message, out receiptNumber); // OK
How to get this done in ActiveX Control? I tried to define methods using & reference symbol, but MIDL compiler did not allow that.
MyCOM.idl:
[id(1025315)] void Execute (LPCTSTR cmd, [out]long& returnCode); // MIDL2025: syntax error
I modified the methods to use pointers *.
MyCOM.idl:
[id(1025315)] void Execute (LPCTSTR cmd, [out]long* returnCode);
MyCOMCtrl.h:
// Dispatch maps
afx_msg void Execute (LPCTSTR cmd, long* resultCode);
MyCOMCtrl.cpp
// Dispatch map
...
DISP_FUNCTION_ID(MyCOMCtrl, "Execute", DISPID_EXECUTE_METHOD, Execute, VT_EMPTY, VTS_PI4)
...
void MyCOMCtrl::Execute (LPCTSTR cmd, long* resultCode)
{
*resultCode = 111;
}
C#:
using MyCOMLib;
...
MyCOM client = new MyCOM();
int resultCode = 0;
// COMException: Type mismatch. (Exception from HRESULT: 0x80020005 (DISP_E_TYPEMISMATCH))
client.Execute ("Test command", out resultCode);
The same exception occurs using string type in C# and LPCTSTR* in C++ ActiveX instead.
Any tips or suggestions will be appreciated.

SOLVED:
In MyCOMCtrl.cpp:
// Dispatch map
...
DISP_FUNCTION_ID(MyCOMCtrl, "Execute", DISPID_EXECUTE_METHOD, Execute, VT_EMPTY, VTS_PI4)
...
Must be:
DISP_FUNCTION_ID(MyCOMCtrl, "Execute", DISPID_EXECUTE_METHOD, Execute, VT_EMPTY, VTS_BSTR VTS_PI4) // two VTS arguments

This is just a long shot (no pun intended), but try using the "long" datatype for you .net resultCode variable.