Develop Reference

Using AddRow() in Output Buffer when C# transform in SSIS needs synchronous?

First off I'm quite new to both SSIS and C#, so apologies for any rookie mistakes. I am trying to muddle my way through splitting one column by a specific delimiter from an input file that will have a variable length header, and a footer.
For example, Input0Buffer has one column. The actual data is always preceded by a row starting with the phrase "STARTDATA", and is bracketed with a row starting with "ENDDATA".
The one input column contains 5 bits of data separated by | . Two of these columns I don't care about.
Basically the input file looks like this:
junkrow
headerstuff
morejunk
STARTDATA
ID1|rubbish|stuff|apple|cheese
ID2|badger|junk|pear|yoghurt
So far I have tried to get some row-by-row logic going in the C# transformer, which I think I am happy with - but I can't work out how to get it to output my split data. Code is below.
bool passedSOD;
bool passedEOD;
public void ProcessRow(Input0Buffer data)
{
string Col1, Col2, Col3;
if (data.Column0.StartsWith("ENDDATA"))
{
passedEOD = true;
}
if (passedSOD && !passedEOD)
{
var SplitData = data.Column0.Split('|');
Col1 = SplitData[0];
Col2 = SplitData[3];
Col3 = SplitData[4];
//error about Output0Buffer not existing in context
Output0Buffer.Addrow();
Output0Buffer.prodid = Col1;
Output0Buffer.fruit = Col2;
Output0Buffer.dairy = Col3;
}
if (data.Column0.StartsWith("STARTDATA"))
{
passedSOD = true;
}
}
If I change the output to asynchronous it stops the error about Output0Buffer not existing in the current context, and it runs, but gives me 0 rows output - presumably because I need it to be synchronous to work through each row as I've set this up?
Any help much appreciated.

you can shorten your code by just checking if the row contains a '|'
if(Row.Column0.Contains("|")
{
string[] cols = Row.Column0.Split('|');
Output0Buffer.AddRow();
Output0Buffer.prodid = cols[0];
Output0Buffer.fruit = cols[3];
Output0Buffer.dairy = cols[4];
}
Like Bill said. Make sure this is a transformation component and not a destination. Your options are source, transformation, and destination.
You also might want this as a different output as well. Otherwise, you will need to conditionally split out the "extra" rows.

Thanks both for for answering - it is a transformation, and thank you for the shorter way, however the header and footer are not well formatted and may contain junk characters also, so I daren't risk looking for | in rows. But I will definitely store that away for processing a better formatted file next time.
I got a reply outside this forum so I thought I should answer my own question in case any one else has a similar problem.
Note that:
it's a transform
the Output is be set to SynchronousInputID = None in the Inputs and Outputs section of the Script Transformation Editor
my input is just called Input, and contains one column called RawData
my output is called GenOutput, and has three columns
although the input file only really has 5 fields, there is a trailing | at the end of each row so this counts as 6
Setting the synchronous to None means that Output0Buffer is now recognised in context.
The code that works for me is:
bool passedSOD;
bool passedEOD;
public override void_InputProcessInputRow(InputBuffer Row)
{
if (Row.RawData.Contains("ENDDATA"))
{
passedEOD = true;
GenOutputBuffer.SetEndOfRowset();
}
//IF WE HAVE NOT PASSED THE END OF DATA, BUT HAVE PASSED THE START OF DATA, SPLIT THE ROW
if (passedSOD && !passedEOD)
{
var SplitData = Row.RawData.Split('|');
//ONLY PROCESS IF THE ROW CONTAINS THE RIGHT NUMBER OF ELEMENTS I.E. EXPECTED NUMBER OF DELIMITERS
if (SplitData.Length == 6)
{
GenOutputBuffer.AddRow();
GenOutputBuffer.prodid = SplitData[0];
GenOutputBuffer.fruit = SplitData[3];
GenOutputBuffer.dairy = SplitData[4];
}
//SILENTLY DROPPING ROWS THAT DO NOT HAVE RIGHT NUMBER OF ELEMENTS FOR NOW - COULD IMPROVE THIS LATER
}
if (Row.RawData.Contains("STARTDATA"))
{
passedSOD = true;
}
}
Now I've just got to work out how to convert one of the other fields from string to decimal, but decimal null and allow it to output a null if someone has dumped "N.A" in that field :D

How can I exactly match a string inside of a larger string?

I have the following enum declared:
public enum EtcMethod
{
ACCORD,
COROLLA,
COROLLA_S,
CAMRY,
CIVIC
}
On my form, I have a handful of controls with their Tag property set:
myControl1.Tag = "ACCORD";
myControl2.Tag = "COROLLA";
myControl3.Tag = "CIVIC COROLLA_S CAMRY";
Then I'm checking the controls' tags in a loop to see if any of the values are found:
private void HideControls(EtcMethod etcMethod, LayoutControlGroup lcg)
{
foreach (BaseLayoutItem ctl in lcg.Items)
{
if (ctl.GetType() == typeof (LayoutControlItem))
{
LayoutControlItem item = (LayoutControlItem)ctl;
if (item.Tag.ToString().IndexOf(etcMethod.ToString()) >= 0)
item.Visibility = LayoutVisibility.Always;
else
item.Visibility = LayoutVisibility.Never;
}
}
}
But the problem with this is, for example, if etcMethod is COROLLA and item.Tag.ToString() is "COROLLA_S" that'll erroneously pass the check.
How can I make sure that it'll find an exact match instead of a "partial" match? In other words, I would like it to behave as if you checked off the "Match whole word" option using Visual Studio's Find feature.
The only solution I could think of would be to check the value of the character at etcMethod.Lenght+1 and see if it's a space (indicating the beginning of another enum value) or if that position even exists (indicating the end of the tag), but that seems particularly sloppy.

Why don't you Split it and use Contains ?
if (item.Tag.ToString().Split().Contains(etcMethod.ToString()))
This will first split your Tag on space, if it hasn't space it just turn it into a string array, then using Contains on array will look for exact match.

This may be one solution:
if (item.Tag.ToString() + " ").IndexOf(etcMethod.ToString() + " ") >= 0)

Does Console.Write ever fail?

I am testing my run() function to make sure it has all the correct things populated by the end of the method. I populate 4 fields from our databases, two strings, and two IEnumerable(string)'s. The idea is to print them all out for all the people I am pulling from the database.
When I only print the string fields, everything works fine. However, when I try to add the Enumerables as well, nothing prints at all. I am also printing a counter so I know the program is still running and working. Does Console ever decide to not print anything at all due to space or something else?
Here's the code that does it:
int i = 0;
foreach (Contact contact in Contact.LoadWithPredicate(getAll))
{
-------other code to populate fields---------
i ++;
Console.WriteLine(i);
//Turn the Enumerables into strings for printing
string firstEnumAsString = String.Join(", ", firstEnumerable.ToArray());
string secondEnumAsString = String.Join(", ", secondEnumerable.ToArray());
Console.WriteLine("Email: " + firstString+ ", correspondance: " + secondString+ ", PAM addresses: " + firstEnumAsString+ ", famousPeople: " + secondEnumAsString);
}
So, as my output, whenever I run the code above, I get an output like this:
1
2
Email: foo#bar.com, correspondance: John, PAM addresses: bar#foo.com, famousPeople: foo, bar, foo
3
4
etc
Problem is, only two people out of the total (~425) show up, the rest of the lines are just numbers. Shouldn't I at least get the strings "Email", and ", correspondence"? It seems like Console is just deciding not to do anything. And I know that the code must reach it because it prints out the integer i just before I call that Console.WriteLine().
On the other hand, whenever I just ask for the two string fields to be printed, the Console displays both fields for all 425 users, right after their corresponding integer. Does anyone know what's happening here? TIA

Based on your comments, I think either your firstEnumerable or secondEnumerable objects are null. But since you did not post how you obtain these objects, I can't comment on why they are null or how to fix it.
It's okay for them to contain null entries, or even be fully empty, but they themselves cannot be null or it will throw an ArgumentNullException when you call .ToArray() on them. This corresponds with the "value cannot be null" exception message you're seeing.
The reason why it's not crashing and burning hard is because you are swallowing (and logging?) the exceptions within your iteration loop with a try/catch block that was not posted in your code sample.
I'm guessing your actual code is something like this:
foreach (Contact contact in Contact.LoadWithPredicate(getAll))
{
try
{
...
object[] firstEnumerable = null;
object[] secondEnumerable = null;
//some logic gates here which under some circumstances do not
//assign a valid instance to firstEnumerable or secondEnumerable
...
Console.WriteLine(i); //this prints every time
//Turn the Enumerables into strings for printing
string firstEnumAsString = String.Join(", ", firstEnumerable.ToArray()); //exception here
string secondEnumAsString = String.Join(", ", secondEnumerable.ToArray()); //or exception here
Console.WriteLine("Email: " + firstString+ ", correspondance: " + secondString+ ", PAM addresses: " + firstEnumAsString+ ", famousPeople: " + secondEnumAsString);
...
}
catch
{
//maybe some logging? maybe not?
}
}
This will print out the i value each time. But when it attempts to create firstEnumAsString or secondEnumAsString it throws an exception and never hits your second Console.WriteLine("Email: " + ...); thus producing the output you're seeing. It's not that the Console.WriteLine is failing, it's that you're never calling it in the first place.

If value is null, only the line terminator is written to the standard output stream.
For more information about the line terminator, see the Remarks section of the WriteLine() method. - Source
On top of that, if one of the values you're trying to write to the console throw an Exception, like in the following code:
private static string SomeVal { get { throw new Exception(); } }
Console.WriteLine("Foo");
Console.WriteLine("This throws an exception: " + SomeVal);
"This throws an exception:" won't be written to the console either; the entire string is just considered null. To me it looks like that's what's actually happening in your code; one of the values you're trying to write to the console is throwing an Exception, and it's handled somewhere silently. Are you running this code in a supressive try/catch clause?
You can easily figure out if that's the case by putting a breakpoint on Console.WriteLine or the trailing } and check the values of firstString, secondString and see if they're null. You can also tell whether one of the values you're trying to write threw an exception.
Also, you should consider using string.Format(string format, params object[] args) rather than concatenating with +. Makes debugging your code a lot easier, too.

Saving settings based on strings

I need to save to a different setting based on the input string. Why will this line not work?
Properties.Settings.Default + colorOptionNametoSave = selectedIndexString;
Properties.Settings.Default.Save();
Where the colorOptionNametoSave is a different color setting and the selectedIndexString is the value to save. However I get the message:
Error 2: The left-hand side of an assignment must be a variable, property or indexer.
The only work around I can think of is a switch statement, but I have a lot of colors so that would be really long. Any ideas on a more efficient solution?

if/else or a switch is what you are looking for.
You cannot concatenate a variable name!
So a solution for you would be something like this ( if colorOptionNametoSave is a string ):
if(colorOptionNametoSave == "Blue")
{
Properties.Settings.Default.Blue = selectedIndexString;
}
else if(colorOptionNametoSave == "Red")
{
Properties.Settings.Default.Red = selectedIndexString;
}

C#/SQL Checking for duplicate entries

I am working on a project, and I have hit a brick wall. My code adds dates with a type to a database, but I need to create an error when there is already a similar entry within the code. However, I cannot get my loop to check for duplicates, and instead it adds duplicates! I am not very good at loops so I'm a bit stuck at this. Any help to check for duplicate entries and to stop it from creating too many would be a great help! Changed my code within this text area so it's not exactly the same variable names.
Here is my code: -
if (DT != null && DT.Length > 0 || DF != null && DF.Length > 0)
{
for (int t = 0; t < Type.Length; t++)
{
DateTime checkDate;
if (Type.IsSelectionValid(8, out typeError) && DateTime.TryParse(DF, out typeError) && DateTime.TryParse(DT, out checkDate))
{
TypeValid = true;
error.Error = false;
}
else
{
error.Errors = "Type-Invalid";
absenceTypeValid = false;
break;
}
}
else
{
error.Errors = "Type-Duplicate";
TypeValid = false;
break;
}
}
}

I'm 'fairly' sure you are going out of your way to make a problem more difficult than it is here, but I can't say for sure since I'm not entirely sure what this is doing.
But here are the conditions that need to be met to get to your Type-Duplicate Error line:
1) Either DT or DF have to not be empty to get past the first if statement
2) Either IsSelectionValid() has to return false or either DT or DF have to be an invalid DateTime.
None of those things constitute a duplicate.
Let me try to explain what I see here:
I first see variables called DT, DF. I can see these are dates, but that's all I know about them. I see 'Type' which I understand even less about than DT and DF. I see that you are doing a loop for Type.Length number of iterations... but what does this mean to me if I don't have a clue what Type is?
If you had comments explaining what things are I 'might' be able to help you, but there's just really not enough information to know what's happening here.
If you simply want to know how to avoid adding duplicates to a database, then I would suggest adding a constraint or index to the column in the database and then you can just catch the exceptions that are thrown when you try to insert a duplicate and deal with it that way. Alternatively, account for it in your insert statement.