Develop Reference

Is it possible to get a batch of text content through Azure DevOps REST API?

I need to get (not download) the content from 10.000~ manifest files within a project in Azure DevOps, but I don't manage to achieve this. I have found several ways to retrieve the content from one file at a time, but in this context, it is neither an efficient nor sustainable solution. I have managed to retrieve all files of a particular file type by checking if the file path ends with the name of the file, then using the TfvcHttpClientBase.GetItemsBatch method. However, this method does not return the item's content.
Program.cs
using Microsoft.TeamFoundation.SourceControl.WebApi;
AzureRest azureRest = new AzureRest();
var tfvcItems = azureRest.GetTfvcItems();
List<TfvcItemDescriptor> itemDescriptorsList = new List<TfvcItemDescriptor>();
foreach(var item in tfvcItems)
{
//Example manifest file .NET
if (item.Path.EndsWith("packages.config"))
{
var itemDescriptor = new TfvcItemDescriptor()
{
Path = item.Path,
RecursionLevel = VersionControlRecursionType.None,
Version = "",
VersionOption = TfvcVersionOption.None,
VersionType = TfvcVersionType.Latest
};
itemDescriptorsList.Add(itemDescriptor);
}
}
TfvcItemDescriptor[] itemDescriptorsArray = itemDescriptorsList.ToArray();
var itemBatch = azureRest.GetTfvcItemsBatch(itemDescriptorsArray);
foreach(var itemList in itemBatch)
{
foreach(var itemListList in itemList)
{
Console.WriteLine("Content: " + itemListList.Content); //empty/null
Console.WriteLine("ContentMetadata: " + itemListList.ContentMetadata); //not empty/null
}
}
AzureRest.cs
using Microsoft.TeamFoundation.SourceControl.WebApi;
using Microsoft.VisualStudio.Services.Common;
using Microsoft.VisualStudio.Services.WebApi;
public class AzureRest
{
const string ORG_URL = "https://org/url/url";
const string PROJECT = "Project";
const string PAT = "PersonalAccessToken";
private string GetTokenConfig()
{
return PAT;
}
private string GetProjectNameConfig()
{
return PROJECT;
}
private VssConnection Authenticate()
{
string token = GetTokenConfig();
string projectName = GetProjectNameConfig();
var credentials = new VssBasicCredential(string.Empty, token);
var connection = new VssConnection(new Uri(ORG_URL), credentials);
return connection;
}
public List<TfvcItem> GetTfvcItems()
{
var connection = Authenticate();
using (TfvcHttpClient tfvcClient = connection.GetClient<TfvcHttpClient>())
{
var tfvcItems = tfvcClient.GetItemsAsync(scopePath: "/Path", recursionLevel: VersionControlRecursionType.Full, true).Result;
return tfvcItems;
}
}
public List<List<TfvcItem>> GetTfvcItemsBatch(TfvcItemDescriptor[] itemDescriptors)
{
TfvcItemRequestData requestData = new TfvcItemRequestData()
{
IncludeContentMetadata = true,
IncludeLinks = true,
ItemDescriptors = itemDescriptors
};
var connection = Authenticate();
using (TfvcHttpClient tfvcClient = connection.GetClient<TfvcHttpClient>())
{
var tfvcItems = tfvcClient.GetItemsBatchAsync(requestData).Result;
return tfvcItems;
}
}
}
}

For reference:
I have tested the codes you shared and when debugging at "itemDescriptorsList" and have found that there is no content specified in it, so that's why you cannot get the txt content.
You should first check and add the content property into the "itemDescriptorsList".

Publish a project programmatically C#

I want to be able to publish a separate C# project programmatically. When I run the following code I want it to publish my Repair.csproj project.
What I'm doing here is I'm getting the version number from my code. I then increment it by one, delete everything in the publish folder and then try to publish my project.
However when I run this code it doesn't publish my Repair.csproj. I'm not sure where I'm going wrong.
My code is as follows
public const string publishLocation = #"C:\Workspace_User\Repair\MAIN\Publish";
public const string RepairLocation = #"C:\Workspace_User\Repair\MAIN\Repair.sln";
public const string repairproj = #"C:\Workspace_User\Repair\MAIN\Repair\Repair.csproj";
public const string r = #"C:\Workspace_User\Repair\MAIN\Repair\bin\Release\Repair.dll";
static void Main(string[] args)
{
Microsoft.Build.BuildEngine.Engine.GlobalEngine.BuildEnabled = true;
System.IO.DirectoryInfo di = new DirectoryInfo(publishLocation);
FileVersionInfo fvi = FileVersionInfo.GetVersionInfo(RepairLocation);
var version= AssemblyName.GetAssemblyName(r).Version;
var splitVersionNumber= version.ToString().Split('.');
var getNumber = splitVersionNumber[3];
var addInt=Convert.ToInt32(getNumber);
addInt++;
foreach (FileInfo file in di.GetFiles())
{
if (file.Name!="Working Folder")
{
file.Delete();
}
}
buildMethod();
}
public static object buildMethod()
{
var props = new Dictionary<string, string>();
props["Configuration"] = "Publish";
var request = new BuildRequestData(repairproj, props, null, new string[] { "Build" }, null);
var parms = new BuildParameters();
// parms.Loggers = ...;
var result = BuildManager.DefaultBuildManager.Build(parms, request);
Debug.Write(result);
return result.OverallResult == BuildResultCode.Success;
}

OpenXml Excel: throw error in any word after mail address

I read Excel files using OpenXml. all work fine but if the spreadsheet contains one cell that has an address mail and after it a space and another word, such as:
abc#abc.com abc
It throws an exception immediately at the opening of the spreadsheet:
var _doc = SpreadsheetDocument.Open(_filePath, false);
exception:
DocumentFormat.OpenXml.Packaging.OpenXmlPackageException
Additional information:
Invalid Hyperlink: Malformed URI is embedded as a
hyperlink in the document.

There is an open issue on the OpenXml forum related to this problem: Malformed Hyperlink causes exception
In the post they talk about encountering this issue with a malformed "mailto:" hyperlink within a Word document.
They propose a work-around here: Workaround for malformed hyperlink exception
The workaround is essentially a small console application which locates the invalid URL and replaces it with a hard-coded value; here is the code snippet from their sample that does the replacement; you could augment this code to attempt to correct the passed brokenUri:
private static Uri FixUri(string brokenUri)
{
return new Uri("http://broken-link/");
}
The problem I had was actually with an Excel document (like you) and it had to do with a malformed http URL; I was pleasantly surprised to find that their code worked just fine with my Excel file.
Here is the entire work-around source code, just in case one of these links goes away in the future:
void Main(string[] args)
{
var fileName = #"C:\temp\corrupt.xlsx";
var newFileName = #"c:\temp\Fixed.xlsx";
var newFileInfo = new FileInfo(newFileName);
if (newFileInfo.Exists)
newFileInfo.Delete();
File.Copy(fileName, newFileName);
WordprocessingDocument wDoc;
try
{
using (wDoc = WordprocessingDocument.Open(newFileName, true))
{
ProcessDocument(wDoc);
}
}
catch (OpenXmlPackageException e)
{
e.Dump();
if (e.ToString().Contains("The specified package is not valid."))
{
using (FileStream fs = new FileStream(newFileName, FileMode.OpenOrCreate, FileAccess.ReadWrite))
{
UriFixer.FixInvalidUri(fs, brokenUri => FixUri(brokenUri));
}
}
}
}
private static Uri FixUri(string brokenUri)
{
brokenUri.Dump();
return new Uri("http://broken-link/");
}
private static void ProcessDocument(WordprocessingDocument wDoc)
{
var elementCount = wDoc.MainDocumentPart.Document.Descendants().Count();
Console.WriteLine(elementCount);
}
}
public static class UriFixer
{
public static void FixInvalidUri(Stream fs, Func<string, Uri> invalidUriHandler)
{
XNamespace relNs = "http://schemas.openxmlformats.org/package/2006/relationships";
using (ZipArchive za = new ZipArchive(fs, ZipArchiveMode.Update))
{
foreach (var entry in za.Entries.ToList())
{
if (!entry.Name.EndsWith(".rels"))
continue;
bool replaceEntry = false;
XDocument entryXDoc = null;
using (var entryStream = entry.Open())
{
try
{
entryXDoc = XDocument.Load(entryStream);
if (entryXDoc.Root != null && entryXDoc.Root.Name.Namespace == relNs)
{
var urisToCheck = entryXDoc
.Descendants(relNs + "Relationship")
.Where(r => r.Attribute("TargetMode") != null && (string)r.Attribute("TargetMode") == "External");
foreach (var rel in urisToCheck)
{
var target = (string)rel.Attribute("Target");
if (target != null)
{
try
{
Uri uri = new Uri(target);
}
catch (UriFormatException)
{
Uri newUri = invalidUriHandler(target);
rel.Attribute("Target").Value = newUri.ToString();
replaceEntry = true;
}
}
}
}
}
catch (XmlException)
{
continue;
}
}
if (replaceEntry)
{
var fullName = entry.FullName;
entry.Delete();
var newEntry = za.CreateEntry(fullName);
using (StreamWriter writer = new StreamWriter(newEntry.Open()))
using (XmlWriter xmlWriter = XmlWriter.Create(writer))
{
entryXDoc.WriteTo(xmlWriter);
}
}
}
}
}

The fix by #RMD works great. I've been using it for years. But there is a new fix.
You can see the fix here in the changelog for issue #793
Upgrade OpenXML to 2.12.0.
Right click solution and select Manage NuGet Packages.
Implement the fix
It is helpful to have a unit test. Create an excel file with a bad email address like test#gmail,com. (Note the comma instead of the dot).
Make sure the stream you open and the call to SpreadsheetDocument.Open allows Read AND Write.
You need to implement a RelationshipErrorHandlerFactory and use it in the options when you open. Here is the code I used:
public class UriRelationshipErrorHandler : RelationshipErrorHandler
{
public override string Rewrite(Uri partUri, string id, string uri)
{
return "https://broken-link";
}
}
Then you need to use it when you open the document like this:
var openSettings = new OpenSettings
{
RelationshipErrorHandlerFactory = package =>
{
return new UriRelationshipErrorHandler();
}
};
using var document = SpreadsheetDocument.Open(stream, true, openSettings);
One of the nice things about this solution is that it does not require you to create a temporary "fixed" version of your file and it is far less code.

Unfortunately solution where you have to open file as zip and replace broken hyperlink would not help me.
I just was wondering how it is posible that it works fine when your target framework is 4.0 even if your only installed .Net Framework has version 4.7.2.
I have found out that there is private static field inside System.UriParser that selects version of URI's RFC specification. So it is possible to set it to V2 as it is set for .net 4.0 and lower versions of .Net Framework. Only problem that it is private static readonly.
Maybe someone will want to set it globally for whole application. But I wrote UriQuirksVersionPatcher that will update this version and restore it back in Dispose method. It is obviously not thread-safe but it is acceptable for my purpose.
using System;
using System.Diagnostics;
using System.Reflection;
namespace BarCap.RiskServices.RateSubmissions.Utility
{
#if (NET20 || NET35 || NET40)
public class UriQuirksVersionPatcher : IDisposable
{
public void Dispose()
{
}
}
#else
public class UriQuirksVersionPatcher : IDisposable
{
private const string _quirksVersionFieldName = "s_QuirksVersion"; //See Source\ndp\fx\src\net\System\_UriSyntax.cs in NexFX sources
private const string _uriQuirksVersionEnumName = "UriQuirksVersion";
/// <code>
/// private enum UriQuirksVersion
/// {
/// V1 = 1, // RFC 1738 - Not supported
/// V2 = 2, // RFC 2396
/// V3 = 3, // RFC 3986, 3987
/// }
/// </code>
private const string _oldQuirksVersion = "V2";
private static readonly Lazy<FieldInfo> _targetFieldInfo;
private static readonly Lazy<int?> _patchValue;
private readonly int _oldValue;
private readonly bool _isEnabled;
static UriQuirksVersionPatcher()
{
var targetType = typeof(UriParser);
_targetFieldInfo = new Lazy<FieldInfo>(() => targetType.GetField(_quirksVersionFieldName, BindingFlags.Static | BindingFlags.NonPublic));
_patchValue = new Lazy<int?>(() => GetUriQuirksVersion(targetType));
}
public UriQuirksVersionPatcher()
{
int? patchValue = _patchValue.Value;
_isEnabled = patchValue.HasValue;
if (!_isEnabled) //Disabled if it failed to get enum value
{
return;
}
int originalValue = QuirksVersion;
_isEnabled = originalValue != patchValue;
if (!_isEnabled) //Disabled if value is proper
{
return;
}
_oldValue = originalValue;
QuirksVersion = patchValue.Value;
}
private int QuirksVersion
{
get
{
return (int)_targetFieldInfo.Value.GetValue(null);
}
set
{
_targetFieldInfo.Value.SetValue(null, value);
}
}
private static int? GetUriQuirksVersion(Type targetType)
{
int? result = null;
try
{
result = (int)targetType.GetNestedType(_uriQuirksVersionEnumName, BindingFlags.Static | BindingFlags.NonPublic)
.GetField(_oldQuirksVersion, BindingFlags.Static | BindingFlags.Public)
.GetValue(null);
}
catch
{
#if DEBUG
Debug.WriteLine("ERROR: Failed to find UriQuirksVersion.V2 enum member.");
throw;
#endif
}
return result;
}
public void Dispose()
{
if (_isEnabled)
{
QuirksVersion = _oldValue;
}
}
}
#endif
}
Usage:
using(new UriQuirksVersionPatcher())
{
using(var document = SpreadsheetDocument.Open(fullPath, false))
{
//.....
}
}
P.S. Later I found that someone already implemented this pathcher: https://github.com/google/google-api-dotnet-client/blob/master/Src/Support/Google.Apis.Core/Util/UriPatcher.cs

I haven't use OpenXml but if there's no specific reason for using it then I highly recommend LinqToExcel from LinqToExcel. Example of code is here:
var sheet = new ExcelQueryFactory("filePath");
var allRows = from r in sheet.Worksheet() select r;
foreach (var r in allRows) {
var cella = r["Header"].ToString();
}

How to use the Microsoft Translator API over Windows Azure, for Windows Phone?

Previously there the bing translator was easily accessible with the SOAP interface. Now it has been migrated to Windows Azure. I have registered in the Azure marketplace for 10000 letters per month (free). How can I translate text through the translator api, for windows phone in C#? Please help. I am not sure how to use the BeginExecute and EndExecute for queries.
I have downloaded and added the TranslatorContainer.cs to my project. For now I am just trying to get the Languages with the GetLanguagesForTranslation method. This is the code which I have written.
public partial class PhonePage1 : PhoneApplicationPage
{
public PhonePage1()
{
InitializeComponent();
Translator transInstance = new Translator();
}
class Translator
{
private Uri service_root;
private TranslatorContainer context;
public Translator()
{
service_root = new Uri("https://api.datamarket.azure.com/Bing/MicrosoftTranslator/");
context = new TranslatorContainer(service_root);
context.Credentials = new NetworkCredential("ID","...........");
var query = context.GetLanguagesForTranslation();
query.BeginExecute(OnQueryComplete, query);
}
public void OnQueryComplete(IAsyncResult result)
{
var query = result as DataServiceQuery<Language>;
string langstring = "";
foreach (Language lang in query.EndExecute(result))
{
langstring += lang.Code + "\n";
}
MessageBox.Show(langstring);
}
}
}
In OnQueryComplete() the query is null even after the assignment. The result has the Properties IsCompleted as true, and statusCode is OK.
I am not able to figure out how to go about this. Please help.
Thank you

With help from Bing Translator team I got it working in my Silverlight Application:
UseDefaultCredentials needs to be turned off on the proxy
On the async callback, you were casting the result to a DSQ, but it’s the result’s AsyncState that needs to be casted. See below.
private void Button_Click_1(object sender, RoutedEventArgs e)
{
var serviceUri = new Uri("https://api.datamarket.azure.com/Bing/MicrosoftTranslator/");
var accountKey = "**********************"; //
var tcode = new Microsoft.TranslatorContainer(serviceUri);
tcode.Credentials = new NetworkCredential(accountKey, accountKey);
tcode.UseDefaultCredentials = false;
var query = tcode.GetLanguagesForTranslation();
query.BeginExecute(OnQueryComplete, query);
}
public void OnQueryComplete(IAsyncResult result)
{
var query = (DataServiceQuery<Microsoft.Language>)result.AsyncState;
var enumerableLanguages = query.EndExecute(result);
string langstring = "";
foreach (Microsoft.Language lang in enumerableLanguages)
{
langstring += lang.Code + "\n";
}
MessageBox.Show(langstring);
}
This way you can use BeginExecute() and BeginEnd() to get Async results.

I had exact same problem and I was suggested that the issue may be related with the how the Async results are return internally when calling GetLanguagesForTranslation, however I did not dig further and just used Execute() to get the list of Language as below:
var serviceUri = new Uri("https://api.datamarket.azure.com/Bing/MicrosoftTranslator/");
var accountKey = "***********************"; //
var tcode = new TranslatorContainer(serviceUri);
tcode.Credentials = new NetworkCredential(accountKey, accountKey);
var languages = tcode.GetLanguagesForTranslation().Execute().ToArray();
foreach (var i in languages)
{
Console.WriteLine(i.Code);
}
Not sure if that is what you are looking for but it worked in my case well.

How to avoid XmlSerializer failure if process environment is too large?

I ran into a strange problem last week. A call to new XMLSerializer(typeof(MyType)) crashed with an ExternalException, telling me that csc.exe could not be executed.
After some investigation I found that this exception only occurs if the process environment size reaches a "critical" limit. I created a little sample application to verify that reason.
namespace EnvironmentTester
{
public class Program
{
private static void Main(string[] args)
{
FillProcessEnvironmentBlock(false);
SerializeDataObject();
}
private static void SerializeDataObject()
{
var dto = new DataObject {Name = "MyDto"};
try
{
var xmlSerializer = new XmlSerializer(dto.GetType()); // throws exception
xmlSerializer.Serialize(TextWriter.Null, dto);
Console.WriteLine("No exception occured.");
}
catch(Exception e)
{
Console.WriteLine("Exception occured : " + e.GetType());
}
Console.ReadKey();
}
private static void FillProcessEnvironmentBlock(bool fillToMax)
{
var currentEnvVarIndex = 0;
var environmentSize = GetEnvironmentSize();
int criticalEnvironmentSize = fillToMax ? 30692 : 30691;
while (environmentSize < criticalEnvironmentSize)
{
var envVarName = "Env" + currentEnvVarIndex;
var envVarValueLength = (criticalEnvironmentSize - environmentSize - envVarName.Length - 2) % 32000;
Environment.SetEnvironmentVariable(envVarName, new string('a', envVarValueLength));
currentEnvVarIndex++;
environmentSize = GetEnvironmentSize();
}
}
private static int GetEnvironmentSize()
{
var envVars = Environment.GetEnvironmentVariables();
int environmentSize = 0;
foreach (string envKey in envVars.Keys)
{
environmentSize += envKey.Length;
}
foreach (string envVar in envVars.Values)
{
environmentSize += envVar.Length;
}
environmentSize += 2*envVars.Keys.Count; // add the '=' and the '\0'
return environmentSize;
}
public class DataObject
{
[XmlAttribute("ObjectName")]
public string Name { get; set; }
}
}
}
If FillProcessEnvironmentBlock is called with parameter false, the critical size is not reached, if it's called with true, the ExternalException is thrown. I tested it on two different WindowsXP 32bit SP2 machines, with the same result.
I know that csc.exe is called to create a temporary assembly used to read/write the xml file. But I don't know why the call to csc.exe fails if the process environment is too large.
Does anyone know the reason for this exception? And how can I work around it (if I don't want to write my own xml serialization)? Are there any other known problems causeed by a process environment that's too large?

You can precompile serializers using Sgen tool or MSBuild Sgen task