Develop Reference

When using MSBuild to build a WPF project which files should I specify for compilation? does order matter?

I'm attempting to automate the build process of my WPF project and set up a CI system. I'm following details found in Continuous Integration in .NET by Marcin Kawalerowicz and Craig Bernston. Specifically, details in chapter 3 pertaining to build automation in MSBuild. The authors provide a sample build file for a project consisting of a single C# file, which looks similar to
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build;Deploy;Execute"
xmlns="http://schemas.microsoft.com/developer/msuild/2003">
<PropertyGroup>
<Debug Condition="'$(Delete)'==''">false</Debug>
<OutputFile>SampleMSBuild.exe</OutputFile>
<OutputDirectory>Output</OutputDirectory>
</PropertyGroup>
<ItemGroup>
**<CompileFile Include="SampleMSBuild.cs" />**
<DeleteFiles Include="SampleMSBuild.exe;SampleMSBuild.pdb" />
</ItemGroup>
<Target Name="Clean">
...
</Target>
<Target Name="Build" DependsOnTarget="Clean">
**<Csc Sources="#(CompileFiles)"**
OutputAssembly="$(OutputFile)"
EmitDebugInformation="$(Debug)" />
<Target Name="Deploy">
...
</Target>
<Target Name="Execute">
...
</Target>
</Project>
My confusion arises from the two lines of code I've surrounded in asterisks. Although I now know how to build a project with a single source file, I am unsure how to scale that up to a WPF project where there is a predefined template with many source files that are placed in different directories. Is it simply a matter of including every source file in the project like so
...
<CompileFile Include="source1.cs;source2.cs;source3.cs;...;sourceN.cs"
...
What about files in a project that are NOT source code? For example, files found in the References folder of a WPF project. I can't imagine all these are to be compiled as well. Does the complied source code communicate with the rest of the uncompiled files in the project? What exactly is the relationship between source files and other non-source files when it comes to build automation in MSBuild?

Make VisualStudio run a command to generate C# code

I'm trying to do something extremely simple, yet despite hours of scouring the Internet and desperately typing in every tiny example fragment I can find, I literally cannot make VS do what I want. Seriously frustrated here!
This is what I want:
When I do a build, I want VS to find every *.fubar file in the project.
For each *.fubar file, I want VS to execute fubar.exe with the input file as argument.
I want VS to compile the resulting *.cs files as usual.
I am damn-well convinced it must be possible to achieve this trivially simple task. I just need to figure out how.
So that's the problem. Let me explain what I've researched so far.
It seems that you can write a VS extension by implementing some COM interface. But that requires you to edit the Registry to tell VS where the plugin is. Obviously that's completely unacceptable; I should not have to reconfigure the OS just to run a command-line tool!
It appears that MSBuild is supposed to be this ultra-configurable build management tool where you can define an arbitrary set of build steps and their interdependencies. It seems like it should be trivial to add an extra task before the build C# compilation step that runs fubar.exe to generate the source code. And yet, after hours of trying, I cannot get a single C# file to appear.
Some documents talk about special pre-build and post-build steps. But it seems silly to need a special hook when the number and order of build steps is supposed to be arbitrary to begin with. Regardless, I tried it, and it still didn't work.
To be clear: Playing around with the project file makes VS display the item properties slightly differently, and doesn't make the build fail. But it also doesn't ever make any C# files appear. (At least, I can't find them anywhere on the disk. I can't tell if my tool is actually being run or not — I suspect "not".)
Can anybody tell me how to make this trivial, trivial thing work? There must be a way!

I agree that T4 may be a prettier solution, but the below MsBuild code works.
You can create a custom targets file that does the calling, I created one that simply copies files, but the process should be similar for invoking your executable.
Make sure you set your FuBar files to BuildAction "Fubars". The Build action has been defined in the targets file using the "AvailableItemName" property.
You may need to extend the below script with a couple of Conditions and Exists checks to make sure it runs when needed. I called it convert.targets and places it in the project directory.
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<!-- Ensure Visual Studio puts the FuBars item in the Build Action dropdown -->
<ItemGroup>
<AvailableItemName Include="FuBars" />
</ItemGroup>
<!-- Execute action on all items matching FuBar (only if output is missing or input has changed) -->
<Target Name="GenerateCodeFubar" Inputs="#(Fubars)" Outputs="$(MSBuildProjectDirectory)/obj/$(Configuration)/Fubars/%(Fubars.Identity).g.cs" >
<MakeDir Directories="$(MSBuildProjectDirectory)/obj/$(Configuration)/Fubars/%(Fubars.RelativeDir)" />
<Exec Command="cmd /c copy "%(FuBars.FullPath)" "$(MSBuildProjectDirectory)/obj/$(Configuration)/Fubars/%(FuBars.Identity).g.cs""/>
</Target>
<!-- Pick up generated code in the Compile group. Separated from the execute action so that the previous action only runs when files have changes -->
<Target Name="IncludeCodeFubarInCompile">
<ItemGroup>
<GeneratedFubars Include="$(MSBuildProjectDirectory)/obj/$(Configuration)/Fubars/%(FuBars.Identity).g.cs"/>
</ItemGroup>
<CreateItem Include="%(GeneratedFubars.FullPath)">
<Output TaskParameter="Include" ItemName="Compile" />
</CreateItem>
</Target>
<!-- Clean up for Rebuild or Clean -->
<Target Name="DeleteCodeFubar">
<RemoveDir Directories="$(MSBuildProjectDirectory)/obj/$(Configuration)/Fubars/" />
<Delete Files="$(MSBuildProjectDirectory)/obj/$(Configuration)/fubars/**/*.g.cs" />
</Target>
<!-- Instruct MsBuild when to call target on Build-->
<PropertyGroup>
<BuildDependsOn>
GenerateCodeFubar;
IncludeCodeFubarInCompile;
$(BuildDependsOn);
</BuildDependsOn>
</PropertyGroup>
<!-- Instruct MsBuild when to call target on Clean-->
<PropertyGroup>
<CleanDependsOn>
DeleteCodeFubar;
$(CleanDependsOn);
</CleanDependsOn>
</PropertyGroup>
</Project>
Include it in your .csproj after the import the last targets file:
</ItemGroup>
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
<!-- IMPORT HERE -->
<Import Project="convert.targets" />
<!-- END: IMPORT HERE -->
<!-- To modify your build process, add your task inside one of the targets below and uncomment it.
Other similar extension points exist, see Microsoft.Common.targets.
<Target Name="BeforeBuild">
</Target>
<Target Name="AfterBuild">
</Target>
-->
</Project>
You may have to tell Visual Studio to turn off the Host Compiler, as Visual Studio caches the contents of the <Compile> group for performance. Though a quick test shows this may not be needed.
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
<!-- The first property group should not have any condition on it -->
<PropertyGroup>
<!-- Include below item in the first PropertyGroup item:
<UseHostCompilerIfAvailable>FALSE</UseHostCompilerIfAvailable>
</PropertyGroup>

This is extremely easy.
In your project properties select the Build Events tab and enter a correct Pre-build event command line.
Without knowing what footer.exe is nor its command line args, I can't give you any specific help for that app. But there is a "Macros" button on the pre-build event dialog that can help you with various substitution variables.
Additionally, VS comes with the T4 templating system which might be a better fit than whatever footer.exe is.

How to include created database project created dacpac into a cs project

I have a separate database project that I’d like to build within the same solution and then reference the created dacpac. When I try adding the database project it builds fine and the dll is added to the secondary project file, but the dacpac is not.
Is there a way that I can have the dacpac copied into my main project through the msbuild? I keep on thinking that there should be a way to modify either the sqlproj file or csproj file so that the dacpac is included as one of the project outputs. My knowledge of msbuild is not extensive, I’ve not been able to figure it out.
It seems to me that I need to add the dacpac somehow to say the '#(ReferenceCopyLocalPaths)' item but I have not been able to figure it out. Any tips or suggestions would be appreciated.
I tried doing something a little like what is referenced here MSBuild - ItemGroup of all bin directories within subdirectories by doing:
<Target Name="AfterBuild">
<Message Text="#(MainAssembly)" />
<!--<DacPacs Include="%(ProjectReference.Directory)**" />-->
<ItemGroup>
<DacPacs Include="%(ProjectReference.Directory)**/*bin*/*.dac" />
</ItemGroup>
<Message Text="#(ReferenceCopyLocalPaths)" />
<Message Text="DacPacs: #(DacPacs)" />
<Message Text="Target Database: $(TargetDatabase)" />
</Target>
which gives nothing for DacPacs (when the wildcard is added). Also I tried referencing one of the item groups from the sqlproj file but it comes out empty to:

In the project properties, you can add a pre-build event command line to get a copy of the dacpac file.
Or you can just add it in the csproj :
<PropertyGroup>
<PreBuildEvent>copy "$(SolutionDir)DatabaseProject\bin\$(ConfigurationName)\DatabaseProject.dacpac" "$(ProjectDir)\DatabaseProject.dacpac"</PreBuildEvent>
</PropertyGroup>
This will only work if the database project was built first, so you should add a dependency. Right clic on the solution and select Project Dependencies..., then select the main project and check that it depends on the Database project.

Add this to your csproj:
<ItemGroup>
<Content Include="..\DatabaseProject\bin\Debug\DatabaseProject.dacpac">
<Link>DatabaseProject.dacpac</Link>
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</Content>
</ItemGroup>

This solution worked for me with a .NET 6 project which references a .sqlproj project using Micrsoft.Sql.Sdk. Assume there are two projects: DacpacDepender (.NET 6 project) and Dacpac (SQL project).
In DacpacDepender.csproj:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<!-- some properties redacted for brevity -->
<TargetFramework>net6.0</TargetFramework>
<RestorePackagesWithLockFile>true</RestorePackagesWithLockFile>
</PropertyGroup>
<ItemGroup>
<!-- Ensures build order, the DACPAC is _guaranteed_ to exist when included -->
<ProjectReference Include="..\Dacpac\Dacpac.sqlproj">
<!-- Companion to .sqlproj CopyBuildOutputToOutputDirectory=false -->
<ReferenceOutputAssembly>false</ReferenceOutputAssembly>
<!-- Suppresses a warning about this project not being compatible with SQL project -->
<SkipGetTargetFrameworkProperties>true</SkipGetTargetFrameworkProperties>
</ProjectReference>
</ItemGroup>
<ItemGroup>
<!-- Use `$(Configuration)` to reference the DACPAC output location regardless of build configuration. -->
<Content Include="$(ProjectDir)..\Dacpac\bin\$(Configuration)\Dacpac.dacpac" CopyToOutputDirectory="PreserveNewest" />
</ItemGroup>
</Project>
In Dacpac.sqlproj:
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build">
<Sdk Name="Microsoft.Build.Sql" Version="0.1.3-preview" />
<PropertyGroup>
<!-- some properties redacted for brevity -->
<!-- Prevents outputting the DLL (some may need it) -->
<CopyBuildOutputToOutputDirectory>false</CopyBuildOutputToOutputDirectory>
<CopyOutputSymbolsToOutputDirectory>false</CopyOutputSymbolsToOutputDirectory>
<Name>Dacpac</Name>
<NetCoreBuild>True</NetCoreBuild>
<RestorePackagesWithLockFile>true</RestorePackagesWithLockFile>
</PropertyGroup>
</Project>
Please note that <RestorePackagesWithLockFile /> is not strictly necessary but ensures dotnet restore --locked-mode works, which is a nicety for reproducible builds.

MSBuild task does not cause generated c# files to compile

I've been trying to write an MSBuild task to "compile" CoCo/R .ATG files into C# files which will then get compiled into the executable, this is to replace the pre-build event.
I've managed to get the .ATG -> .cs process working, however something is not right since the generates .cs files do not get compiled.
If I then modify the .ATG file again, the "old" .cs files seems to get compiled then new ones generated.
I'm quite sure I'm missing something that will inform the rest of the build process that these files have changed.
Here is the target definition that I have included in my Visual Studio 2010 project.
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<AvailableItemName Include="BuildATG" />
</ItemGroup>
<PropertyGroup>
<CoreBuildDependsOn>
BuildATGTarget;
$(CoreBuildDependsOn)
</CoreBuildDependsOn>
</PropertyGroup>
<Target Name="BuildATGTarget" Inputs="#(BuildATG)" Outputs="#(BuildATG -> '%(RelativeDir)Parser.cs')">
<Exec Command="Coco.exe %(BuildATG.Identity)" Outputs="%(BuildATG.RelativeDir)Parser.cs" />
</Target>
</Project>
I am completly new to MSBuild, so any advice / pointers would be appreciated.

One possible solution I have found is to do the following changes and add an ItemGroup inside the target.
<Target Name="BuildATGTarget" Inputs="#(BuildATG)" Outputs="#(BuildATG -> '%(RelativeDir)Parser.cs')">
<Exec Command="Coco.exe %(BuildATG.Identity)" Outputs="%(BuildATG.RelativeDir)Parser.cs" />
<ItemGroup>
<Compile Include="%(BuildATG.RelativeDir)Parser.cs" />
<Compile Include="%(BuildATG.RelativeDir)Scanner.cs" />
</ItemGroup>
</Target>
To avoid duplicate files in your build, you also need to mark the files (Parser.cs and Scanner.cs) in the Visual Studio 2010 project with Build Action: None

external file for build tasks

I am creating a .net web application and have some build tasks eg msbuild copy task. I add this by editing the project file for the application and adding the task.
While this works ok, is there any way I can use an external xml file to the project file and have my build tasks in this with the main project file referencing it? I would much prefer this as I wouldn't then have to edit the main project file and there is separation between the project file and the build tasks.

You can simply reference any external project or target file by adding an import to your main project file:
...
<!-- this is the default import for (c#) web project files -->
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
<Import Project="$(MSBuildExtensionsPath)\Microsoft\VisualStudio\v9.0\WebApplications\Microsoft.WebApplication.targets" />
<!-- import your custom project/target file here -->
<Import Project="MyCustom.targets" Condition="Exists('MyCustom.targets')" />
...
Adding the Condition will allow to build your main project even if your custom project/target file is missing because the build is run in a different environment.
I'm using this approach to run FxCop and StyleCop targets on my local machine but the same main project file can be built without any changes in my staging environment.
Update
Your comment suggests that you actually are looking rather for a solution that should work the other way round: You want to execute some steps before and after building your project without modifying the project configuration itself.
In that case the best way is to create your custom project to call the build of your web project. This could look something like this:
<Project DefaultTargets="MyTargetAfterBuild">
<!-- some Project attributes omitted for readability -->
<Target Name="MyTargetBeforeBuild" ContinueOnError="false">
<Exec Command="svn export" />
</Target>
<Target Name="Build" DependsOnTargets="MyTargetBeforeBuild">
<MSBuild Projects="MyWebProject.csproj" Targets="Build" Properties="Configuration=Release" >
</MSBuild>
</Target>
<Target Name="MyTargetAfterBuild" DependsOnTargets="Build">
<Exec Command="powershell.exe .\MyCustomScript.ps1" />
</Target>
</Project>
You might be interested in this answer on a similar scenario with a more detailed example.

You can call msbuild.exe, targetting your separate build file, in a pre-build or post-build event.