I have this code to execute a package. What the package does is just export very large table data into excel file.
[HttpPost]
public ActionResult TableToFile()
{
try
{
var connection = new SqlConnection(System.Configuration.ConfigurationManager.ConnectionStrings["SSIS"].ConnectionString);
var integrationServices = new IntegrationServices(connection);
var package = integrationServices
.Catalogs["SSISDB"]
.Folders["MyFolder"]
.Projects["MyProject"]
.Packages["MyPackage.dtsx"];
long executionIdentifier = package.Execute(true, null);
ExecutionOperation eo = integrationServices.Catalogs["SSISDB"].Executions[executionIdentifier];
while (!eo.Completed)
{
eo.Refresh();
System.Threading.Thread.Sleep(5000);
}
return Json(new { result = "ok" }, JsonRequestBehavior.AllowGet);
}
catch (Exception e)
{
return Json(new { result = "error" }, JsonRequestBehavior.AllowGet);
}
}
However, the package has hard-coded connections like sql server source and excel file destination filename. That we can see in connection manager in the package.
I have this code in VB.Net and using a different library it sets also the connection of the package
Dim source As String = System.Windows.Forms.Application.StartupPath() & "\export.dtsx"
Dim p As Microsoft.SqlServer.Dts.Runtime.Package = app.LoadPackage(source, Nothing)
Dim exec As Executables = p.Executables
For Each config As ConnectionManager In p.Connections
Select Case config.Name
Case "SourceConnectionOLEDB"
config.ConnectionString = "..... some connection string here"
Case "DestinationConnectionFlatFile"
config.ConnectionString = file
Dim inner As Microsoft.SqlServer.Dts.Runtime.Wrapper.IDTSConnectionManagerFlatFile90 = CType(config.InnerObject, Microsoft.SqlServer.Dts.Runtime.Wrapper.IDTSConnectionManagerFlatFile90)
For i As Integer = 0 To inner.Columns.Count - 1
Dim innerColumn As Microsoft.SqlServer.Dts.Runtime.Wrapper.IDTSConnectionManagerFlatFileColumn90 = inner.Columns(i)
If i < inner.Columns.Count - 1 Then
innerColumn.ColumnDelimiter = delimiter
Else
innerColumn.ColumnDelimiter = vbNewLine
End If
Next
End Select
Next
For Each taskhost As TaskHost In exec
taskhost.Execute(Nothing, Nothing, Nothing, Nothing, Nothing)
Next
My problem is how do I change connection string or some settings before I execute the package in my C# code above like my VB.Net does?
since you seem to be using the SSIS catalog, you should also be able to use project environment variables. Basically your package would reference these variables and you would set the values before execution.
There is an explanation on this at http://www.sqlchick.com/entries/2015/1/4/parameterizing-connections-and-values-at-runtime-using-ssis-environment-variables
Related
I am running the latest Office 365 Excel version, 1901. I have updated to the latest OpenXml SDK but am unable to figure out how to programmatically read information about threaded comments, as all I'm seeing is a full summary comment. Even using the latest OpenXml Nuget package.
If I convert the Excel document to a .zip file, I can see "threadedComments.xml" files which has what I need, but do not know how to go about it programmatically in C# .NET.
I know, you didn't watch for VBA, but there the new CommentThreaded object now works at least (Excel version 1906, tested June 2019).
I actually tested it in Visual Studio C#, but it still seems to be not supported.
As of May 15th 2019 the new object CommentThreaded is described by Microsoft.
In my Excel version 1906, it's fully supported in VBA.
Here's some VBA-code to explain the handling a little:
Private Sub ExcelsNewCommentThreaded()
Dim AllCommentsThreaded As Excel.CommentsThreaded
Dim OneCommentThreaded As Excel.CommentThreaded
Dim AllReplies As Excel.CommentsThreaded
Dim OneReply As Excel.CommentThreaded
Dim r As Range
Set AllCommentsThreaded = ActiveSheet.CommentsThreaded
' loop over all threaded comments of a worksheet and get their info
For Each OneCommentThreaded In AllCommentsThreaded
With OneCommentThreaded
Debug.Print .Author.Name, .Date, .Text
For Each OneReply In .Replies
With OneReply
Debug.Print .Author.Name, .Date, OneReply.Text
End With
Next OneReply
End With
Next OneCommentThreaded
Set r = Selection.Cells(1)
' check if the selected cell already contains a threaded comment
If r.CommentThreaded Is Nothing Then
r.AddCommentThreaded ("my new comment")
End If
With r.CommentThreaded
' get text of comment
Debug.Print .Text
' add some replies
.AddReply ("my reply 1")
.AddReply ("my reply 2")
' change text of comment
Debug.Print .Text(Text:="text of comment changed")
Debug.Print .Text
' change text of a reply
.Replies(1).Text Text:="text of reply 1 changed"
Debug.Print .Replies(1).Text
' delete second reply
.Replies(2).Delete
' delete whole comment including its replies
.Delete
End With
End Sub
You can access to content programmatically, if you know exact location in .zip archieve:
static class Program
{
static void Main(string[] args)
{
using (var archive = ZipFile.OpenRead(args[0]))
{
var entry = archive.Entries.Where(_ => _.FullName.Equals("xl/comments1.xml", StringComparison.OrdinalIgnoreCase)).FirstOrDefault();
if (entry != null)
{
var stopwatch = new Stopwatch();
stopwatch.Start();
var data = new List<string>(Decompress(entry.Open()));
var graph = new Graph(data);
stopwatch.Watch();
Console.ReadLine();
}
}
}
public static IEnumerable<string> Decompress(Stream stream)
{
using (var reader = new StreamReader(stream, Encoding.ASCII))
{
string line;
while ((line = reader.ReadLine()) != null)
{
yield return line;
}
}
}
}
Purpose: I have an SSIS Solution with various packages, one set of these packages is currently created with a package per table due to them having different structures. The aim is to create a template package (done), update variables/table names (done), reinitialise and re-map columns, execute package for each table.
Problem: So, as you can tell, I'm up to the point that i need to reinitialise but I am unable to get to the data flow task and keep getting this error:
No such interface supported (Exception from HRESULT: 0x80004002 (E_NOINTERFACE))
When I run this code:
//// Get Data Flow task
TaskHost tHost = pkgOut.Executables[0] as TaskHost;
MainPipe dataFlowTask = (MainPipe)tHost.InnerObject;
The code that I am using is below:
public void Main()
{
// Set Project Variables
List<string> pkgVars = new List<string>
{
#"pPR_SSIS_Catalog_Server",
#"pPR_SSIS_Catalog_Project",
#"pPR_SSIS_Catalog_Folder"
};
// Get Package
String pkgLocation = #"C:\PATH\TO\TEMPLATE\PACKAGE\a_Load_SAP_Template.dtsx";
Application app = new Application();
Package pkgIn = app.LoadPackage(pkgLocation, null);
String pkgName = "DynamicPackage";
// Add Connections (cos they're project connections and aren't stored in package)
ConnectionEnumerator allConns = Dts.Connections.GetEnumerator();
while ((allConns.MoveNext()) && (allConns.Current != null))
pkgIn.Connections.Join(allConns.Current);
// Convert new package to XML so we can cheat and just do a find and replace
XmlDocument pkgXML = new XmlDocument();
pkgIn.SaveToXML(ref pkgXML, null, null);
// Replace strings
// Set SAP table
String pkgStr = pkgXML.OuterXml.ToString();
pkgStr = pkgStr.Replace("#SAP_SRC_TABLE#", "MyF-ingSAPTables"); // Replace SAP Table Name
// Set Project Variables references == values -- REMEMBER TO CHECK FOR INT PARAMS zzz
foreach (string var in pkgVars)
pkgStr = pkgStr.Replace(#"#[$Project::" + var + #"]", #"""" + Convert.ToString(Dts.Variables[var].Value) + #"""");
// Convert back to XML
XmlDocument newXML = new XmlDocument();
newXML.LoadXml(pkgStr);
Package pkgOut = new Package();
pkgOut.LoadFromXML(newXML, null);
//// Get Data Flow task
TaskHost tHost = pkgOut.Executables[0] as TaskHost;
MainPipe dataFlowTask = (MainPipe)tHost.InnerObject; // THIS IS WHERE THE CODE ERRORS
new Application().SaveToXml(String.Format(#"D:\PATH\TO\SAVE\LOCATION\{0}.dtsx", pkgName), pkgOut, null);
Dts.TaskResult = (int)ScriptResults.Success;
}
I have tried:
Removing everything but the data flow
Recreated a blank data flow
Recreated the package
Made a package in the script and created the data flow (this works but when i open the package, the data flow does not appear...might have added it wrong but it can see it when I run the TaskHost section)
I am not sure what else I can try, I have seen information that I might need to re-register some .dlls but I do not have admin access and I'd prefer not to go through the hassle for something where I do not know whether it will work.
Any help would be appreciated.
Thanks,
Fixed, was due to version difference in the .DLL that was being loaded. Loaded an older version and all went through fine.
I have 130 batch files stored in one central location, I want to create VB.NET program to execute those batch files in parallel NOT in sequence. and then return the output of these 130 batch files to Textbox control.
How can I cater the above requirements? or what's the recommended way of doing it, keeping in mind the program will run on a high Specs Server so performance is not an issue here.
What I have done is I placed this for loop inside Button_Click Event to get my .bat names & then to get them executed in parallel:
Dim Directory As New IO.DirectoryInfo("C:\Batch\")
Dim fileName As IO.FileInfo() = Directory.GetFiles("*.bat")
For Each file As IO.FileInfo In fileName
RichTextBox1.AppendText(file.ToString() & vbNewLine)
Next
But now here after i got the names of .dat, how to get them executed in parallel?
This is as close as I could get, combining the comments and the updated description
Dim Directory As New IO.DirectoryInfo("C:\Batch\")
Dim fileName As IO.FileInfo() = Directory.GetFiles("*.bat")
Dim list As New List(Of String)
For Each file As IO.FileInfo In fileName
list.Add(file.ToString())
Next
Dim processTasks As New List(Of Task(Of String))()
For Each bat As String In list
Dim processTask As Task(Of String) = Task.Factory.StartNew((Function()
Dim filePath As String = IO.Path.Combine(myDirectory.FullName, bat)
Dim info As New System.Diagnostics.ProcessStartInfo()
info.FileName = filePath
info.CreateNoWindow = True
info.WindowStyle = ProcessWindowStyle.Hidden
System.Diagnostics.Process.Start(info)
Return "hello"
End Function))
processTasks.Add(processTask)
Next
' if you are using a synchronous method
' Dim results() As String = Task.WhenAll(processTasks).GetAwaiter().GetResult()
' if you are using an async method
Dim results() As String = await Task.WhenAll(processTasks)
' Either use the string array as a whole, or
For Each result As String In results
' Do something with every individual result
Next
Note I've answered the question using C# because OP tagged the question with c#.
You should simply use Task:
List<Task<string>> processTasks = new List<Task<string>>();
for(int i = 0; i < 130; i ++)
{
Task<string> processTask = Task<string>
(
() =>
{
// System.Diagnostics.Process stuff goes here, you should return a string
// since it represents the output of the whole process
}
);
processTasks.Add(processTask);
Task.Run(processTask);
}
// Asynchronously wait for all tasks to finish
await Task.WhenAll(processTasks);
// Now after all tasks have finished, each stored task in "processTasks"
// will contain a task where their "Result" property is the string representing
// the whole process output
I run the following code but nothing shows up in ALM:
AttachmentFactory attachmentFactory = (AttachmentFactory)tsTest.Attachments;
TDAPIOLELib.Attachment attachment = (TDAPIOLELib.Attachment)attachmentFactory.AddItem("test");
attachment.Post();
The AddItem method on the second line keeps asking for "object ItemData" but I have no idea what that is exactly. HP has such poor documentation that there is really nothing explaining it. Does anyone know how to programatically using c# add a file attachment to a test run in HP ALM?
After much pain and research I have found an answer. I'm sure there are other ways of accomplishing this that are more efficient but since HP's documentation is the worst on the planet this is the best I could come up with. If anyone has a better way I would LOVE to see it so please post it!
I hope this helps!
try
{
if (qcConn.Connected)
{
string testFolder = #"Root\YourFolder";
TestSetTreeManager tsTreeMgr = (TestSetTreeManager)qcConn.TestSetTreeManager;
TestSetFolder tsFolder = (TestSetFolder)tsTreeMgr.get_NodeByPath(testFolder);
AttachmentFactory attchFactory = (AttachmentFactory)tsFolder.Attachments;
List tsList = tsFolder.FindTestSets("YourTestNameHere", false, null);
foreach (TestSet ts in tsList)
{
TestSetFolder tstFolder = (TestSetFolder)ts.TestSetFolder;
TSTestFactory tsTestFactory = (TSTestFactory)ts.TSTestFactory;
List mylist = tsTestFactory.NewList("");
foreach (TSTest tsTest in mylist)
{
RunFactory runFactory = (RunFactory)tsTest.RunFactory;
Run run = (Run)runFactory.AddItem("NameYouWantDisplayedInALMRuns");
run.CopyDesignSteps();
//runResult just tells me if overall my test run passes or fails - it's not built in. It was my way of tracking things though the code.
if(runResult)
run.Status = "Failed";
else
run.Status = "Passed";
run.Post();
//Code to attach an actual file to the test run.
AttachmentFactory attachmentFactory = (AttachmentFactory)run.Attachments;
TDAPIOLELib.Attachment attachment = (TDAPIOLELib.Attachment)attachmentFactory.AddItem(System.DBNull.Value);
attachment.Description = "Attach via c#";
attachment.Type = 1;
attachment.FileName = "C:\\Program Files\\ApplicationName\\demoAttach.txt";
attachment.Post();
//Code to attach a URL to the test run
AttachmentFactory attachmentFactory = (AttachmentFactory)run.Attachments;
TDAPIOLELib.Attachment attachment = (TDAPIOLELib.Attachment)attachmentFactory.AddItem(System.DBNull.Value);
//Yes, set the description and FileName to the URL.
attachment.Description = "http://www.google.com";
attachment.Type = 2;
attachment.FileName = "http://www.google.com";
attachment.Post();
//If your testset has multiple steps and you want to update
//them to pass or fail
StepFactory rsFactory = (StepFactory)run.StepFactory;
dynamic rdata_stepList = rsFactory.NewList("");
var rstepList = (TDAPIOLELib.List)rdata_stepList;
foreach (dynamic rstep in rstepList)
{
if (SomeConditionFailed)
rstep.Status = "Failed";
else
rstep.Status = "Passed";
rstep.Post();
}
else
{
rstep.Status = "No Run";
rstep.Post();
}
}
}
}
}
}
I have done something similar, but in Python and against Test Steps, so even if I don't have code you can copy & paste it, this might point you to the right direction.
Instead of calling:
attachmentFactory.AddItem( filename )
Call the function with no parameters (or a null paramer, can't tell since I never used the OTA API with C#):
file = attachmentFactory.AddItem()
Now assign the file to the attachment item, and the rest of its properties:
file.Filename = "C:\\Users\\myUser\\just\\an\\example\\path" + fileName
file.Description = "File description"
file.Type=1
file.Post()
The type specifies it's a local file, and not an URL.
If anyone is wondering how to do that on the requirement-module, here is the code:
Req req = Globals.Connection.ReqFactory.Item(*ID*));
VersionControl versionControl = ((IVersionedEntity)req).VC as VersionControl;
versionControl.CheckOut(string.Empty);
AttachmentFactory attFac = req.Attachments;
Attachment att = (Attachment)attFac.AddItem(System.DBNull.Value);
att.Description = "*Your description here";
att.Type = (int)TDAPI_ATTACH_TYPE.TDATT_FILE; //for URL, change here
att.FileName = "*Your path including filename here*";
att.Post();
versionControl.CheckIn("*Your check-in comment here*");
No valuable information on Internet!
After some digging on OTA documentation I have found this:
AttachmentFactory attachmentFactory = (AttachmentFactory)TstTest.Attachments;
TDAPIOLELib.Attachment attachment = (TDAPIOLELib.Attachment)attachmentFactory.AddItem("demoAttach.txt");
attachment.Description = "Bug Sample Attachment";
attachment.Post();
IExtendedStorage exStrg = attachment.AttachmentStorage;
exStrg.ClientPath = "E:\\TestData";
exStrg.Save("demoAttach.txt", true);
actually, was in VB script form but I managed to transform in C#.
OTA reference:
'-----------------------------------------
'Use Bug.Attachments to
' get the bug attachment factory.
Set attachFact = bugObj.Attachments
'Add a new extended storage object,an attachment
' named SampleAttachment.txt.
Set attachObj = attachFact.AddItem("SampleAttachment.txt")
' Modify the attachment description.
attachObj.Description = "Bug Sample Attachment"
' Update the attachment record in the project database.
attachObj.Post
' Get the bug attachment extended storage object.
Set ExStrg = attachObj.AttachmentStorage
'Specify the location of the file to upload.
ExStrg.ClientPath = "D:\temp\A"
'-----------------------------------------
'Use IExtendedStorage.Save to
' upload the file.
ExStrg.Save "SampleAttachment.txt", True
I have a SSIS package that eventually I would like to pass parameters too, these parameters will come from a .NET application (VB or C#) so I was curious if anyone knows of how to do this, or better yet a website with helpful hints on how to do it.
So basically I want to execute a SSIS package from .NET passing the SSIS package parameters that it can use within it.
For instance, the SSIS package will use flat file importing into a SQL db however the Path and name of the file could be the parameter that is passed from the .Net application.
Here is how to set variables in the package from code -
using Microsoft.SqlServer.Dts.Runtime;
private void Execute_Package()
{
string pkgLocation = #"c:\test.dtsx";
Package pkg;
Application app;
DTSExecResult pkgResults;
Variables vars;
app = new Application();
pkg = app.LoadPackage(pkgLocation, null);
vars = pkg.Variables;
vars["A_Variable"].Value = "Some value";
pkgResults = pkg.Execute(null, vars, null, null, null);
if (pkgResults == DTSExecResult.Success)
Console.WriteLine("Package ran successfully");
else
Console.WriteLine("Package failed");
}
Here's how do to it with the SSDB catalog that was introduced with SQL Server 2012...
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Data.SqlClient;
using Microsoft.SqlServer.Management.IntegrationServices;
public List<string> ExecutePackage(string folder, string project, string package)
{
// Connection to the database server where the packages are located
SqlConnection ssisConnection = new SqlConnection(#"Data Source=.\SQL2012;Initial Catalog=master;Integrated Security=SSPI;");
// SSIS server object with connection
IntegrationServices ssisServer = new IntegrationServices(ssisConnection);
// The reference to the package which you want to execute
PackageInfo ssisPackage = ssisServer.Catalogs["SSISDB"].Folders[folder].Projects[project].Packages[package];
// Add a parameter collection for 'system' parameters (ObjectType = 50), package parameters (ObjectType = 30) and project parameters (ObjectType = 20)
Collection<PackageInfo.ExecutionValueParameterSet> executionParameter = new Collection<PackageInfo.ExecutionValueParameterSet>();
// Add execution parameter (value) to override the default asynchronized execution. If you leave this out the package is executed asynchronized
executionParameter.Add(new PackageInfo.ExecutionValueParameterSet { ObjectType = 50, ParameterName = "SYNCHRONIZED", ParameterValue = 1 });
// Add execution parameter (value) to override the default logging level (0=None, 1=Basic, 2=Performance, 3=Verbose)
executionParameter.Add(new PackageInfo.ExecutionValueParameterSet { ObjectType = 50, ParameterName = "LOGGING_LEVEL", ParameterValue = 3 });
// Add a project parameter (value) to fill a project parameter
executionParameter.Add(new PackageInfo.ExecutionValueParameterSet { ObjectType = 20, ParameterName = "MyProjectParameter", ParameterValue = "some value" });
// Add a project package (value) to fill a package parameter
executionParameter.Add(new PackageInfo.ExecutionValueParameterSet { ObjectType = 30, ParameterName = "MyPackageParameter", ParameterValue = "some value" });
// Get the identifier of the execution to get the log
long executionIdentifier = ssisPackage.Execute(false, null, executionParameter);
// Loop through the log and do something with it like adding to a list
var messages = new List<string>();
foreach (OperationMessage message in ssisServer.Catalogs["SSISDB"].Executions[executionIdentifier].Messages)
{
messages.Add(message.MessageType + ": " + message.Message);
}
return messages;
}
The code is a slight adaptation of http://social.technet.microsoft.com/wiki/contents/articles/21978.execute-ssis-2012-package-with-parameters-via-net.aspx?CommentPosted=true#commentmessage
There is also a similar article at http://domwritescode.com/2014/05/15/project-deployment-model-changes/
To add to #Craig Schwarze answer,
Here are some related MSDN links:
Loading and Running a Local Package Programmatically:
Loading and Running a Remote Package Programmatically
Capturing Events from a Running Package:
using System;
using Microsoft.SqlServer.Dts.Runtime;
namespace RunFromClientAppWithEventsCS
{
class MyEventListener : DefaultEvents
{
public override bool OnError(DtsObject source, int errorCode, string subComponent,
string description, string helpFile, int helpContext, string idofInterfaceWithError)
{
// Add application-specific diagnostics here.
Console.WriteLine("Error in {0}/{1} : {2}", source, subComponent, description);
return false;
}
}
class Program
{
static void Main(string[] args)
{
string pkgLocation;
Package pkg;
Application app;
DTSExecResult pkgResults;
MyEventListener eventListener = new MyEventListener();
pkgLocation =
#"C:\Program Files\Microsoft SQL Server\100\Samples\Integration Services" +
#"\Package Samples\CalculatedColumns Sample\CalculatedColumns\CalculatedColumns.dtsx";
app = new Application();
pkg = app.LoadPackage(pkgLocation, eventListener);
pkgResults = pkg.Execute(null, null, eventListener, null, null);
Console.WriteLine(pkgResults.ToString());
Console.ReadKey();
}
}
}
So there is another way you can actually fire it from any language.
The best way I think, you can just create a batch file which will call your .dtsx package.
Next you call the batch file from any language. As in windows platform, you can run batch file from anywhere, I think this will be the most generic approach for your purpose. No code dependencies.
Below is a blog for more details..
https://www.mssqltips.com/sqlservertutorial/218/command-line-tool-to-execute-ssis-packages/
Happy coding.. :)
Thanks,
Ayan
You can use this Function if you have some variable in the SSIS.
Package pkg;
Microsoft.SqlServer.Dts.Runtime.Application app;
DTSExecResult pkgResults;
Variables vars;
app = new Microsoft.SqlServer.Dts.Runtime.Application();
pkg = app.LoadPackage(" Location of your SSIS package", null);
vars = pkg.Variables;
// your variables
vars["somevariable1"].Value = "yourvariable1";
vars["somevariable2"].Value = "yourvariable2";
pkgResults = pkg.Execute(null, vars, null, null, null);
if (pkgResults == DTSExecResult.Success)
{
Console.WriteLine("Package ran successfully");
}
else
{
Console.WriteLine("Package failed");
}