Develop Reference

Using MSBuild, how can you group .cs files with matching .g.cs files in another folder?

I have a .NET 6 project where I want some generated files - somewhere in the "Generated" folder structure - to be grouped with their non-generated equivalents. See dotnet/roslyn/issues/45645 for some related concepts.
How can you achieve this using MSBuild? Key challenges include:
Map all Generated\**\*.g.cs with their *.cs equivalents.
If necessary: Copy all .g.cs files to their siblings' locations, allowing for grouping.
Produce DependentUpon elements for the related files (.cs and .g.cs).
If necessary: Remove the copies from compilation as they are already part of compilation.
End result
The end result should look like below. Note that removing, hiding or clearing the "Generated" folder is acceptable (if not preferable).
My understanding is that DependentUpon is suitable for this, though (as far as I know) it will only work for files in the same folder.
So for our simple Console project, we would have something similar to this:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net6.0</TargetFramework>
</PropertyGroup>
<!-- End goal (not hard-coded, obviously): -->
<ItemGroup>
<Compile Update="Program.g.cs">
<DependentUpon>Program.cs</DependentUpon>
</Compile>
</ItemGroup>
</Project>
Steps to try this out
Create a .NET 6 Console app.
Create a "Generated" folder in the root.
Add Program.g.cs in the "Generated" folder.
Put MSBuild magic in the .csproj file.
// program.cs
partial class Program
{
public static void Main()
{
Foo();
}
}
// program.g.cs
partial class Program
{
public static void Foo() => Console.WriteLine("Generated");
}

I have an example of matching the files and adding the DependentUpon metadata. However, the Visual Studio Solution Explorer window will not show the files as nested. Just adding the metadata when the project is run is not sufficient. The project file that is saved to the file system must be updated to include the DependentUpon metadata.
Regardless here is a working example of matching the files and adding the metadata:
<Target Name="MatchAndAddDependentUpon">
<!-- The file suffix for generated files -->
<PropertyGroup>
<Suffix>.g</Suffix>
</PropertyGroup>
<!-- Example data -->
<ItemGroup>
<Compile Include="aaa.cs;aaa$(Suffix).cs;bbb.cs;ccc.cs;ccc$(Suffix).cs;ddd$(Suffix).cs" />
</ItemGroup>
<ItemGroup>
<!-- Create a set of generated files and a set of candiate source files (i.e. files not in the set of generated files) -->
<Generated Include="#(Compile)" Condition="$([System.String]::Copy('%(Filename)').EndsWith('$(Suffix)'))" />
<CandidateSource Include="#(Compile)" Exclude="#(Generated)" />
<!-- Create Match as a cartesian product of Generated and CandidateSource; then prune the 'matches' that are false -->
<Match Include="#(Generated)">
<DependentUpon>%(CandidateSource.Identity)</DependentUpon>
<DependentUponFileName>%(CandidateSource.Filename)</DependentUponFileName>
</Match>
<Match Remove="#(Match)" Condition="'%(DependentUponFileName)$(Suffix)' != '%(Filename)'" />
<!-- Remove the items in Match from Compile; then include them but without the 'DependentUponFileName' -->
<Compile Remove="#(Match)" />
<Compile Include="#(Match)" RemoveMetadata="DependentUponFileName" />
</ItemGroup>
<Message Text="#(Compile->'%(Identity) (DependentUpon %(DependentUpon))', '%0d%0a')" />
</Target>
The output will look like the following:
MatchAndAddDependentUpon:
aaa.cs (DependentUpon )
bbb.cs (DependentUpon )
ccc.cs (DependentUpon )
ddd.g.cs (DependentUpon )
aaa.g.cs (DependentUpon aaa.cs)
ccc.g.cs (DependentUpon ccc.cs)
Note that the code handles the case where 'ddd.g.cs' exists in 'Compile' and 'ddd.cs', the expected parent, does not.

Can an MSBuild Item use a Property set by a Target?

I'm trying to achieve the following with MSBuild: my main project (MyProject.csproj) should include a couple Reference items, but the path to one of those References is the value of the SomeProperty property, which is set by a Target. Specifically, the value for SomeProperty is parsed from a file using ReadLinesFromFileTask.
Here is the high-level structure of MyProject.csproj:
<Project>
<Target Name="CreateSomeProperty">
<!-- Tasks that ultimately set $(SomeProperty) by parsing a value with ReadLinesFromFileTask -->
</Target>
<ItemGroup>
<Reference Include="$(SomeProperty)" />
<!-- Other Reference items -->
</ItemGroup>
</Project>
Unfortunately, this setup is not working. I see those little yellow triangles under the Dependencies node of MyProject in the VS Solution Explorer, since the project is looking for a DLL at a path with missing characters. Similarly, when I build the project, I get a bunch of The type or namespace name could not be found errors, even though I still see the output from a Message Task inside my Target. Presumably, the CreatePathProperty Target is running during the execution phase, after the Reference items have already failed to load during the evaluation phase.
Is there a way to make a setup like this work? I've tried setting BeforeTargets="Build" in the Target element, and setting InitialTargets="CreateSomeProperty" in the Project element, but nothing seems to work. Any help would be much appreciated!

Can an MSBuild Item use a Property set by a Target?
Yes, I'm sure it's possible if you're in .net framework project with old csproj format and what you want is a supported scenario in VS2017(Only did the test in VS2017).
Tips:
Normally msbuild reads the Properties and Items before it executes your custom target. So we should use something like BeforeTargets="BeforeResolveReferences" to make sure the correct order in this scenario is custom target runs=>create the property=>msbuild reads the info about references and the property.
Otherwise the order(wrong order when BeforeTargets="Build" or what) should be: Msbuild reads the info about references(now the property is not defined yet)=>the target runs and creates the property.
Solution:
Add this script to the bottom of your xx.csproj.
<!-- Make sure it executes before msbuild reads the ItemGroup above and loads the references -->
<Target Name="MyTest" BeforeTargets="BeforeResolveReferences">
<ItemGroup>
<!-- Define a TestFile to represent the file I read -->
<TestFile Include="D:\test.txt" />
</ItemGroup>
<!-- Pass the file to read to the ReadLinesFromFile task -->
<ReadLinesFromFile File="#(TestFile)">
<!--Create the Property that represents the normal hintpath(SomePath\DLLName.dll)-->
<Output TaskParameter="Lines" PropertyName="HintPathFromFile" />
</ReadLinesFromFile>
<!-- Output the HintPath in output log to check if the path is right -->
<Message Text="$(HintPathFromFile)" Importance="high" />
<ItemGroup>
<Reference Include="TestReference">
<!--It actually equals:<HintPath>D:\AssemblyFolder\TestReference.dll</HintPath>-->
<HintPath>$(HintPathFromFile)</HintPath>
</Reference>
</ItemGroup>
</Target>
In addition:
I did the test with test.txt file whose content is:
I'm not sure about the actual content(and format) of your file, but if you only have path like D:\AssemblyFolder\ in that file, you should pass the D:\AssemblyFolder\+YourAssemblyName.dll to <HintPath> metadata. Cause the default reference format with hintpath looks like this:
<Reference Include="ClassLibrary1">
<HintPath>path\ClassLibrary1.dll</HintPath>
</Reference>
Pay attention to the path format! Hope my answer helps :)

In msbuild, static items can't depend on dynamic properties by design.
Accepted answer provides a work-around: use dynamic items, which, naturally, can depend on dynamic properties. But dynamic items don't show up in the IDE.
When dealing with files, more IDE-friendly approach is to create the item statically and update it when the target runs:
<!-- Static item definition -->
<ItemGroup>
<SomeItem Include="item_file" />
</ItemGroup>
<Target Name="...">
<!-- When the target runs, find the static item using Condition and change its properties -->
<ItemGroup>
<SomeItem Condition="'%(SomeItem.Identity)' == 'item_file'">
<SomeProperty>New Value</SomeProperty>
</SomeItem>
</ItemGroup>
</Target>
There's also an open msbuild issue (#889) to support improved syntax for this:
<Target Name="...">
<ItemGroup>
<!-- Update has the same syntax as Include -->
<SomeItem Update="item_file">
<SomeProperty>New Value</SomeProperty>
</SomeItem>
</ItemGroup>
</Target>

How to escape backslash in $(SolutionDir) in a VS project file?

I'm trying to write a post-build event for a C# project. And I'm using a custom console app (myTool.exe) to do that.
For example, the post-build event is
"$(SolutionDir)tools\myTool.exe" "$(SolutionDir)myProject\bin\" Debug
(All paths is quoted because they could contain whitespaces.)
Before escaping, $(SolutionDir) is D:\Some\MySystem\.
After escaping, it should become D:\\Some\\MySystem\\.
How to escape all the \ in $(SolutionDir) in this way in a csproj file?
I have tried to use this approach, but it seems not working for $(SolutionDir):
<PropertyGroup>
<EscapedSolutionDir>$(SolutionDir.Replace('\\', '\\\\'))</EscapedSolutionDir>
<EscapedTargetDir>$(TargetDir.Replace('\\', '\\\\'))</EscapedTargetDir>
</PropertyGroup>
<Target Name="PreBuild" BeforeTargets="PreBuildEvent">
<Exec Command=""$(EscapedSolutionDir)source\\Native\\Output\\NativeLibraryCopier.exe" "$(EscapedSolutionDir)" "$(ConfigurationName)" "$(PlatformName)" "$(TargetDir)"" />
</Target>
PS: See this for why I have to replace the backslash.

How to escape backslash in $(SolutionDir) in a VS project file?
Not sure why you want to replace the "\" with "\\" in the $(SolutionDir). That because this property is common macros for build commands and properties, You can use these macros anywhere in a project's Property Pages dialog box where strings are accepted.
But if you insist on replacing it, you should replace it with $(SolutionDir.Replace('\', '\\')) rather than $(SolutionDir.Replace('\\', '\\\\')). Because the path in the $(SolutionDir) is D:\Some\MySystem\, Only one backslash. When you replace it with double backslashes, MSBuild could not find the double backslashes in the $(SolutionDir).
So the scripts should be:
<PropertyGroup>
<EscapedSolutionDir>$(SolutionDir.Replace('\', '\\'))</EscapedSolutionDir>
<EscapedTargetDir>$(TargetDir.Replace('\', '\\'))</EscapedTargetDir>
</PropertyGroup>
Then I use a target to output the escaped value in the EscapedSolutionDir and EscapedTargetDir, both those value were escaped:
<Target Name="PreBuild" BeforeTargets="PreBuildEvent">
<Message Text="$(EscapedSolutionDir)" Importance="high">
</Message>
<Message Text="$(EscapedTargetDir)" Importance="high">
</Message>
</Target>
The output:

Be aware that some Environ Vars finish with a backslash....
add a "." and then when using them add the "\"
for example I was trying to get a meesage (#pragma message) with a file path and name...
I did it like this:
<PreprocessorDefinitions>INT_DIR=$(IntDir.Replace('', '\')).;%(PreprocessorDefinitions)</PreprocessorDefinitions>
And then
#define MyFile "File.ext"
#pragma message ( TO_STR(INT_DIR) "\\" MyFile "(0) : MESSAGE: File Included")

C# How to automatically include all .cs files in a folder for compiling?

I have a project containing a T4 template file which generates several .cs files based on the output (dll) of another project, which means the count & name of generated .cs files are varying. Now I want to:
Run T4 template to generate .cs files before build.
Include .cs files generated on point 1 in build.
Point 1 is archived by a before-build target. For point 2, I tried to use a wildcard to include all .cs files in the folder like that: <Compile Include="GeneratedCode\*.cs" />, but the problem is:
Any new generated .cs files are not included in the output dll for the first build, but are included for the second build.
VS doesn't refresh the project to show the new generated .cs files or to delete non-existing .cs files.
When I delete or add a .cs file under the folder in VS, VS replaces the wildcard with separated including command like <Compile Include="GeneratedCode\A.cs" />.
So what's the right way to include all .cs files in a folder for compiling to solve at least problem 1 & 3?
++ BeforeBuild target content
<PropertyGroup>
<_CommonProgramFiles>$([System.Environment]::GetEnvironmentVariable('CommonProgramFiles(x86)'))</_CommonProgramFiles>
<_CommonProgramFiles Condition=" '$(_CommonProgramFiles)' == '' ">$(CommonProgramFiles)</_CommonProgramFiles>
<TextTransformPath Condition="'$(TextTransformPath)' == ''">$(_CommonProgramFiles)\Microsoft Shared\TextTemplating\$(VisualStudioVersion)\TextTransform.exe</TextTransformPath>
<!-- Initial default value -->
<_TransformExe>$(TextTransformPath)</_TransformExe>
<_TransformExe Condition="!Exists('$(_TransformExe)')">$(_CommonProgramFiles)\Microsoft Shared\TextTemplating\10.0\TextTransform.exe</_TransformExe>
<_TransformExe Condition="!Exists('$(_TransformExe)')">$(_CommonProgramFiles)\Microsoft Shared\TextTemplating\11.0\TextTransform.exe</_TransformExe>
<_TransformExe Condition="!Exists('$(_TransformExe)')">$(_CommonProgramFiles)\Microsoft Shared\TextTemplating\12.0\TextTransform.exe</_TransformExe>
<_TransformExe Condition="!Exists('$(_TransformExe)')">$(_CommonProgramFiles)\Microsoft Shared\TextTemplating\13.0\TextTransform.exe</_TransformExe>
<_TransformExe Condition="!Exists('$(_TransformExe)')">$(_CommonProgramFiles)\Microsoft Shared\TextTemplating\14.0\TextTransform.exe</_TransformExe>
</PropertyGroup>
<Target Name="TransformOnBuild" AfterTargets="BeforeBuild">
<Error Text="Failed to find TextTransform.exe tool at '$(_TransformExe)." Condition="!Exists('$(_TransformExe)')"/>
<ItemGroup>
<_TextTransform Include="#(None)" Condition="'%(None.Generator)' == 'TextTemplatingFileGenerator'" />
</ItemGroup>
<!-- Perform task batching for each file -->
<Exec Command=""$(_TransformExe)" "%(_TextTransform.FullPath)""
Condition="'%(_TextTransform.Identity)' != ''"
ContinueOnError="ErrorAndContinue" />
</Target>

Msbuild copy and flatten with unique file names

I'm trying to copy multiple files from a deep source tree that have the same file name. For example TestResults.trx. I want to copy them into a single directory (i.e. flattened). Problem is they just overwrite each other and I just end up with a single TestResults.trx in the directory.
<ItemGroup>
<SilverlightTestResults Include=".\**\*.trx" Exclude=".\TestResults\*" />
</ItemGroup>
<Copy SourceFiles="#(SilverlightTestResults)" DestinationFolder=".\TestResults">
I thought I could do a transform using some well known metadata but there doesn't seem to be anything unique in there to do it (the test results I'm trying to copy live in directories like this: .\SomeProject\bin\debug\TestResults.trx).
Copying to a directory like this like this would be ideal:
.\TestResults\TestResults1.trx
.\TestResults\TestResults2.trx
.\TestResults\TestResults3.trx
I don't care about the actual names as long as they are unique.
Any ideas, looks like a custom task is required?

I can't provide a solution that just uses msbuild - you could either use msbuildtasks
to use the <Add /> task for incrementing a counter.
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Import Project="$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets"/>
<PropertyGroup>
<FileCounter>0</FileCounter>
</PropertyGroup>
<ItemGroup>
<MySourceFiles SilverlightTestResults Include=".\**\*.trx" Exclude=".\TestResults\*"/>
</ItemGroup>
<Target Name="CopyFiles">
<Math.Add Numbers="$(FileCounter);1">
<Output TaskParameter="FileCounter" PropertyName="FileCounter" />
</Math.Add>
<Copy
SourceFiles="#(MySourceFiles)"
DestinationFiles="#(MySourceFiles->'.\TestResults\%(Filename)_$(FileCounter)%(Extension)')"
/>
</Target>
However you might do better with a custom task or probably executing a powershell script.

Yes, a custom task will be required.
You could look to see what functionality the Move task from the community task project (link here) offers, but if it doesn't do what you want then it is open source, so it will be trivial to check the source and modify it to suit your needs.