Invoking Azure Data Factory (V2) pipeline from Azure Functions - c#

Requirement: I am trying to invoke a ADFV2 pipeline from Azure Function. The ADFV2 pipeline is already created through .NET api.
Problem: Not able to resolve Microsoft.Azure.Management.DataFactory (this one was introduced newly for ADFV2) nuget package where else the nuget package Microsoft.Azure.Management.DataFactories (this one already exists from the timeline of ADFV1) gets successfully accepted.
Error - Unable to resolve 'Microsoft.Azure.Management.DataFactory (>= 0.2.1)' for '.NETFramework,Version=v4.5.2'.
Why through Azure Function and not Trigger or Custom Activity: The ADFV2 pipeline is dependent on another process (the copy activity source dataset might not be ready when the pipeline will be scheduled to run) which could be achieved by .NET custom activity in ADFV2 itself but thought to try in the other possible way around.
Note: Already went through the link which talks about the same thing only difference being its ADFV1.

Answering and providing some other information so that others who might get stuck on situations like the above would get benefited.
So, finally was able to invoke ADFV2 pipeline from Azure Functions but for some reason was not able to add Nuget packages(Microsoft.Azure.Management.DataFactory) by referring them in project.json or using them directly in run.csx. So, uploaded the dll in bin folder and referred them in my function app as if they were external assemblies which worked fine for my application and was able to invoke all the ADFV2 pipeline related methods (ex: DataFactoryManagementClient, CreateRunWithHttpMessagesAsync etc...).
Also, just started working on VS2017 (version 15.6.0 preview 1.0) and checked the Cloud->Azure Functions application where managing the above mentioned Nuget packages is working smoothly.

Related

XamarinAndroid task in Azure DevOps pipeline failing without reason

I have a Xamarin Forms app for Android, which works perfectly fine when I build it locally in VS. I've set up a pipeline to build it in Azure DevOps, too, but the XamarinAndroid#1 task keeps failing without any actual error messages; it just stops. The corresponding "error message" in the pipeline view is just "No message found for this issue". I even added a VSBuild#1 task before, which successfully compiles all projects (including unit tests, which all succeed).
The failing task itself is nothing special, pretty default stuff:
- task: XamarinAndroid#1
inputs:
projectFile: '**/*droid*.csproj'
outputDirectory: '$(outputDirectory)'
configuration: '$(buildConfiguration)'
jdkOption: 'JDKVersion'
It even used to work, too, before adding an indirect dependency to another library, which in turn has a few NuGet dependencies. Both NuGetToolInstaller#1 and NuGetCommand#2 also work perfectly fine in the beginning of the pipeline, finding, installing, and verifying all external packages. And as mentioned before, locally everything works fine, and even the plain VS build task has no problem building everything. I can even see in the build log of the XamarinAndroid-task that the newly added library builds without any errors. Only when it claims to build my actual Xamarin project does it silently die.
The last few lines of that build log are these:
...
CoreGenerateAssemblyInfo:
Skipping target "CoreGenerateAssemblyInfo" because all output files are up-to-date with respect to the input files.
CssG:
Skipping target "CssG" because it has no outputs.
CoreCompile:
C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\Bin\Roslyn\csc.exe /noconfig /unsafe- /checked- [... all the build options in the world ...]
Using shared compilation with compiler from directory: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\Bin\Roslyn
XamlC:
Compiling Xaml, assembly: obj\Release\netstandard2.0\[My Main Xamarin Project].dll
Finishing: XamarinAndroid
Any pointers to what could be the problem, where to look (is it my project, a Nuget dependency, or the pipeline definition itself?), or how to at least get an error message out of this?
So after the (semi-legal ^^) discussion in the comments with #jdweng I found the culprit: I was indeed using different frameworks in the various libraries, which somehow let the pipeline build fail.
Since DevOps agent jobs (to my understanding) always run on blank, fresh VMs, legacy files could not have been a problem (plus various un-/checkings of "clean" options didn't change anything, either). It's also important to note that everything was only about the pure building (a.k.a converting a *.sln file to an *.apk file), no actual deployment happened yet, and obviously no files were manually exchanged between local and remote machines, as everything was always built from scratch.
Detailed project structure before:
XamarinMainProject both .Net Standard 2.0
& XamarinMainProject.Android
'-> ViewModelLibrary .Net Framework 4.6.1
'-> OneCodeLibrary .Net Framework 4.6.1
'-> TheOtherLibrary .Net Standard 2.0
Interestingly everything worked fine (locally AND on DevOps) when TheOtherLibrary was not included in the build. But as soon as it was referenced, the DevOps Build failed as described above, while locally everything just kept on going as expected.
Now I have recreated the two 4.6.1 projects as .Net Standard 2.0 and the pipeline is happy, too.
This of course raises several questions still:
Why did it work locally (and even on DevOps with a VSBuild task), but not on DevOps for the XamarinAndroid task?
Almost more importantly: Why was there no error message at all (let alone a helpful one)?
Should this be reported as bug to Xamarin (and if so, any hints on where to do so)?

How do I pass assembly version from build step to next step

I have a simple windows application in my Azure DevOps repo. I'm trying to set a build for it so I can practice using the Azure Pipelines.
How would I pass the assembly version of the c# windows application which resides in the AssemblyInfo.cs file under the project properties to use in a next step of the pipeline?
First you need get assembly version. You can use powershell script to read that value from file. This post may help you with that
You need set variable to keep you assembly version. It also can be done in powershell script. You can read about this here
Write-Host "##vso[task.setvariable variable=AssemblyVersion;]$assemblyVersionFromFile"

MS Azure - C# WebJob Project Dependency

I am new to Azure WJs. I have 2 projects in one solution: the actual website - Project 2 and the WebJob - Project 1. The only task the WJ has is to call an exposed method from the public class of project 2 during the scheduled time.
When the WJ was created, the classes & methods of the Project 2 - website were added as a solution reference to the Project 1 - WebJob to make them accessible.
The problem I have is:
When the WebJob is build it compiles all the dependencies at the given time. When the final .zip is uploaded into Azure WebJob portal the webjob will execute with the compiled code version. This means that any new changes to Project 2 - website do not take effect until the WJ is re-build with the updated Project 2 - website dependencies and the .zip re-uploaded.
Is there a way to create WJ (as Project 1) which would call a specific exposed method from the Project 2 and be oblivious to the changes in the project 2 as long as the called method is present?
Example:
WebJob Code (project 1):
namespace SecondProject
{
class Program
{
static void Main()
{
var client = new WebClient();
secondProjectMethod();
}
}
}
Website Code (project 2):
namespace firstProject
{
public class someClass
{
public void secondProjectMethod()
{
// I want to make any code changes I want inside this
// method anytime and the WbJob should not care
// about these changes as long as this method name exist.
// Because all it should care about is that it should
// call this method name.
}
}
}
Since you are using a website, you can expose the method from Proj2 as an API, and make the WebJob invoke the API (instead of the method) at your scheduled interval. So even if you change the method in Proj2, as long as the exposed API is calling the updated method the WebJob will be independent.You can take a look at this link, for creating a basic API
at least two options come up in my mind:
Options 1: your WJ project do not directly reference your website project, instead, use reflection load the website dll in runtime. and use reflection to invoke the function you would like to call (assume signature will never change)
the challenge for this approach is your WJ somehow need to figure out the path to the website dll.
Options 2: since it is a website, why not expose the function you will like to call as a REST API, so your WJ just need to make a http call
The question doesn't seem to be a Web Job Specific one but anyways...
If you need the two projects to be independent, then you should not be using project references.
What I would suggest is to publish the Project2 shared class/methods as nuget packages and use the nuget instead, so any changes to the Project2 will not affect your WebJob until the point that you publish a new nuget package and use that your web job.
Alternatively you can have a folder for your dlls like "Shared Libraries" and reference the Project2 from that location instead of project to isolate the changes to your Web job from your Project reference..

The target “ResolveWebJobFiles” does not exist in the project in Azure Website

I have a Windows Azure project consisting of
ASP.NET project (using NancyFX) deployed as Azure Website
Business Logic class library
Unit test class library
2 Console apps
I want those 2 console app to be deployed as Azure WebJobs with the Azure Website. So I right clikec on the ASP.NET project and chose Add | Existing project as Azure WebJob and added both console apps with the right schedule.
Everything seems fine, the 2 console apps now contain a JSON with schedule and the ASP.NET project contains a JSON with WebJobs definition. All the project build just fine.
The problem is, when I try to deploy the ASP.NET project to Azure, I get
The target “ResolveWebJobFiles” does not exist in the project
I found this article when the author solved this problem by adding
<Import Project="..\packages\Microsoft.Web.WebJobs.Publish.1.0.2\tools\webjobs.targets" Condition="Exists('..\packages\Microsoft.Web.WebJobs.Publish.1.0.2\tools\webjobs.targets')" />
to his csproj file, but my csproj file for the ASP.NET project already contains this import.
Any ideas what the problem is?
even i faced this issue. I reinstalled the below package in my webjob to solve it
package id="Microsoft.Web.WebJobs.Publish" version="1.0.2"
Check to see if you have this file in your webjob project Properties folder:
webjob-publish-settings.json
If not, create it with this content, changing only the webJobName (for now)
{
"$schema": "http://schemastore.org/schemas/json/webjob-publish-settings.json",
"webJobName": "MyWebJobName",
"startTime": "2014-12-09T00:00:00-08:00",
"jobRecurrenceFrequency": "Hour",
"interval": 1,
"runMode": "Scheduled",
"stopping_wait_time": 3600
}
One primary reason could be missing "webjob-publish-settings.json" file in Properties.
Please double check this file exists and included in your project.
in my case it was this file, which had 1 invalid webjob associated with it, removed it and it started working
webjobs-list.json
I also ran into this. After updating all packages in my solution I started to get this. The <import /> was there but it was poiting to a different version. So check that too.
Also worth checking is the .net version of the WebJob project vs your Web app!
As stated in the documentation
A WebJobs-enabled project has the following items added to it:
- The Microsoft.Web.WebJobs.Publish NuGet package.
- A webjob-publish-settings.json file that contains deployment and scheduler settings.
it clearly says the project that needs to have a reference to Microsoft.Web.WebJobs.Publish Nuget package is the Webjob-enabled project application, not the web application. The web application only needs webjobs-list.json file.
The webjob console application also may have a webjob-publish-settings.json file
but only in case of non continuous jobs
I got this error recently and found the cause to be that [WebProject]/Properties/webjobs-list.json contained a reference to an old webjob that had been deleted from the solution. Deleting the reference from the json fixed the problem.
In Kudu (under "Advanced Tools" on your web app management page, under "Development Tools"), under "Debug console", select "Powershell", then navigate to the "site" folder, then the "repository" folder.
Enter the command "dotnet restore" at the command line, then redeploy.
Edit: Still works as of 26/01/2022.

SyncSvcUtilUI can't read its own sync configuration

I tried to use Microsoft's Syn Framework Toolkit: http://www.microsoft.com/en-us/download/details.aspx?id=23217
And as I intend to use an Android client, I also downloaded the Toolkit for it: http://code.msdn.microsoft.com/wpapps/Sync-Framework-Toolkit-4dc10f0e
I installed the SDK, then extracted the Toolkit and built it with VS.
Following the instruction in the help file, I used SyncSvcUtilUI to create a Sync configuration file. It went through as planned.
The next step would be to use that configuration file to provision the database. After I select the aforementioned Sync configuration file, I get this message:
Running SyncSvcUtil command...
Invalid parameter passed
Parameter name: Sync
Aren't the two applications compatible with each other? What should I check? The two executables are in the same directory.
The directory that contained SyncSvcUtilUI had space in its name. After moving it to a simple directory, it worked. Great software design...

Categories

Resources