Develop Reference

Dependency logging with application insights does not log responses

We have set up application insights with our ASP.NET Core 6 application which uses application insights for logging. We use a RestSharp based HTTP client for executing HTTP requests which works fine.
When trying to debug calls made through the RestSharp library, I am not seeing any response bodies being logged, only the statuscode, while the requests are being (kind of) logged:
{
"name": "AppDependencies",
"time": "2023-02-02T06:05:04.6268266Z",
"tags": {
"ai.application.ver": "1.0.0.0",
"ai.cloud.roleInstance": "MY_NICE_PC",
"ai.user.id": "ltK4V",
"ai.operation.id": "11bf52695a8d8ea19f1cb7573f2b195b",
"ai.operation.parentId": "324234234234",
"ai.operation.name": "POST to/somewhere [v]",
"ai.location.ip": "::1",
"ai.internal.sdkVersion": "rdddsc:2.21.0-429",
"ai.internal.nodeName": "MY_NICE_PC"
},
"data": {
"baseType": "RemoteDependencyData",
"baseData": {
"ver": 2,
"name": "POST /none/of-your/business",
"id": "bfa554335eefae0b",
"data": "https://none.of.your.business/my-nice-api-0/my=1&nice&=2&querystring=3",
"duration": "00:00:04.8666247",
"resultCode": "422",
"success": false,
"type": "Http",
"target": "none.of.your.business",
"properties": {
"DeveloperMode": "true",
"AspNetCoreEnvironment": "localdev",
"_MS.ProcessedByMetricExtractors": "(Name:'Dependencies', Ver:'1.1')"
}
}
}
}
We are using ASP.NET Core 6.0 with the Microsoft.ApplicationInsights.AspNetCore version 2.2.21 and the following DI setup:
services.AddApplicationInsightsTelemetry(configure =>
{
configure.ConnectionString = "MySecretConnectionString";
configure.EnableAdaptiveSampling = false;
});
--- Edit:
I also made an implementation of the ITelemetryInitializer to capture all telemetry for all instances of DependencyTelemetry:
public class DependencyTelemetryInitializer : ITelemetryInitializer
{
public void Initialize(ITelemetry telemetry)
{
// SKIP for now
if (telemetry is DependencyTelemetry dependencyTelemetry)
{
}
}
}
This showed me that for every dependency call only the request is being captured, but the response is not.

That is by design, the dependency call basically logs the outgoing call, as that is what application insights has access to. It logs the duration, and also the response code (422 in your example.) I am also not sure what kind of information you would expect to see from the response.
If you want to get access to the response you can do so in a TelemetryInitializer:
public class CustomInitializer : ITelemetryInitializer
{
public void Initialize(ITelemetry telemetry)
{
if (telemetry is DependencyTelemetry dt
&& dt.Type.Equals("http", System.StringComparison.InvariantCultureIgnoreCase)
&& dt.TryGetHttpResponseOperationDetail(out var response))
{
...
}
}
}
You can then access the headers etc. Technically you can access the response content but since it is a stream by default it can only be read. So reading the full response content can't be done here withouth breaking the application afaik.
Also, even if you could, you should be carefull regarding logging the request and response playload. The size might be large, and Application Insights is not designed for that kind of log data. You might also hit the limits to the lenght of the custom properties and the logged payload will be cut off.

I came up with the following, since it is needed to buffer the response, so that it can be read again, I have added the LoadIntoBufferAsync. Also this seems be available as Async only.
public void Initialize(ITelemetry telemetry)
{
if (telemetry is DependencyTelemetry dependencyTelemetry &&
dependencyTelemetry.Type.Equals(DEPENDENCYTELEMETRY_TYPE_HTTP,
StringComparison.InvariantCultureIgnoreCase)
&& dependencyTelemetry.TryGetHttpResponseOperationDetail(out var response))
{
var task = Task.Run(async () => await response.Content.LoadIntoBufferAsync());
task. Wait();
var stream = response.Content.ReadAsStream();
using var reader = new StreamReader(
stream,
Encoding.UTF8,
detectEncodingFromByteOrderMarks: false,
bufferSize: 512, leaveOpen: true);
var responseBody = reader.ReadToEnd();
dependencyTelemetry.Properties.Add("responseBody", responseBody);
}
}
This only seems to be working with very small stream sizes (response sizes) less than 1K. I

How can i parse HttpResponse in .Net 6.0

I am sending request to an api with HttpClient. The response from the request works without any problem, but I cannot parse the key values I want in the response. According to the research I have done, I tried such a code, but the incoming data returns empty in this way. How can I get the values I want from the incoming data?
using LoggerApi.Methods;
using System;
using System.Net.Http;
using System.Text;
using Newtonsoft.Json;
using System.Linq;
namespace LoggerApi.Methods
{
public class ApiMethods
{
public async static Task<object> GetClientInformations(string authenticationCode, string username = "username")
{
var client = new HttpClient();
var userInformationEndpoint = new Uri("https://myurl.com/url");
var userInformationPayload = new UserInformationPayload()
{
Login = username
};
var serializePayload = JsonConvert.SerializeObject(userInformationPayload);
var payload = new StringContent(serializePayload, Encoding.UTF8, "application/json");
var res = await client.PostAsync(userInformationEndpoint, payload).Result.Content.ReadAsStringAsync();
var responseResultJson = JsonConvert.DeserializeObject<object>(res);
return responseResultJson;
}
}
}
this code output is empty looks like this
{
"HasError": [],
"AlertType": [],
"AlertMessage": [],
"ModelErrors": [],
"Data": [
[
[]
],
[
[
[
[
[]
],
[
[]
],
[
[]
],
[
[]
]
]
]
]
]
}
But when I return var res directly instead of var responseResultJson from the function, the result is like this. What I want to do here is to access values such as Login, FirstName, LastName, Id from the incoming data. How can I do that?
{"HasError":false,"AlertType":"success","AlertMessage":"Operation has completed successfully","ModelErrors":[],"Data":{"Count":1,"Objects":[{"Id":291031530,"CurrencyId":"TRY","FirstName":"Scott","LastName":"Marshall","MiddleName":"William","Login":"scotty3"}]}}

There are multiple possible solutions. Once of them is the following.
using System.Net.Http.Json;
using System.Text.Json;
namespace Program
{
class Program
{
// We need this to create a new ToDo
record TodoDto(int UserId, string Title, string Body);
// That's what's stored in the backend service and returned by the API
record TodoModel(int Id, int UserId, string Title, string Body);
public static async Task Main(string[] args)
{
// HTTP Client
var httpClient = new HttpClient();
// required as Properties are PascalCase in C# but JSON properties here are camelCase (that's usually the case)
var serializerOptions = new JsonSerializerOptions {
PropertyNamingPolicy = JsonNamingPolicy.CamelCase
};
// create new todo
var newTodo = new TodoDto(1, "My new todo", "I need to...");
// Add todo in the backend
var response = await httpClient.PostAsJsonAsync<TodoDto>("https://jsonplaceholder.typicode.com/posts", newTodo, serializerOptions);
if(response.IsSuccessStatusCode){
// if we have a succesful request we can deserialize the response
var newTodoModel = JsonSerializer.DeserializeAsync<TodoModel>(await response.Content.ReadAsStreamAsync(), serializerOptions);
Console.WriteLine(newTodoModel);
}
else
{
Console.WriteLine($"Request failed with status code {(int)response.StatusCode} ({response.StatusCode})");
}
}
}
Some remarks:
I am using JsonPlaceholder for some sample API endpoints to provide a working example here
Deserializing the API response to JSON could fail e.g. when the API returns something unexpected, so you should check for that especially if you don't control the API that you're calling.
For this to work you need to include the System.Net.Http.Json namespace at it contains the PostAsJsonAsync() extension method (see HttpClienJsonExtensions Class - Microsoft Learn for more nice and convenient extension methods).
There is no need for Newtonsoft JSON parser, as you can now use the built-in JSON Parser in the System.Text.Json namespace.
I prefer working with Streams as you can then use async all the way throughout your code which is a best practice (see Async best practices - Async All the Way)
By default a case-sensitive matching between your C# class and the JSON will be attempted. JSON usually uses camelCase and this API does so as well. Therefore we need to tell the (de-)serializer to use camelCase as well using JsonSerializerOptions.

Getting JsonElement into CosmosDB in ASP.NET Core 3.1

You'd think this would be easy as falling off a log, but I'm stumped. I have an ASP.NET 3.1 core web api (using System.Text.Json):
[HttpPost]
public async Task<ActionResult> CreateAsync(JsonElement dzc)
Which in turn calls a service:
public async Task AddItemAsync(JsonElement dzc)
{
var id = dzc.GetProperty("id").GetString(); // these are both valid
var userid = dzc.GetProperty("userid").GetString();
await this._container.CreateItemAsync<JsonElement>(dzc, new PartitionKey(userid));
}
This results in: Message: {"Errors":["The input content is invalid because the required properties - 'id; ' - are missing"]}
Or if I get the raw text:
public async Task AddItemAsync(JsonElement dzc)
{
var id = dzc.GetProperty("id").GetString(); // these are all valid
var userid = dzc.GetProperty("userid").GetString();
var raw = dzc.GetRawText();
await this._container.CreateItemAsync<string>(raw, new PartitionKey(userid));
}
and raw is:
{
"id": "foozy",
"userid": "foozy",
"name": "Jayb",
"fc": {
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"properties": {
"capacity": "10",
},
"geometry": {
"type": "Point",
"coordinates": [
-71.073283,
42.417500
]
}
}
]
}
}
This results in: Message: {"Errors":["One of the specified inputs is invalid"]}
All I'm really trying to do is shovel some Json directly from an API into CosmosDB without using C# classes. What is the magic incantation?

Short answer...you can't via CreateItemAsync. The documentation explicitly states that an id prop is required, which by definition does not exist on on either JsonElement or string.
A better option would be to use CreateItemStreamAsync which allows you to pass in the raw stream directly and appears to bypass an Id property requirement, per the example in the documentation.
using (Response response = await this.Container.CreateItemStreamAsync(partitionKey: new PartitionKey("streamPartitionKey"), streamPayload: stream))
{
using (Stream responseStream = await response.ContentStream)
{
//Read or do other operations with the stream
using (StreamReader streamReader = new StreamReader(responseStream))
{
string responseContentAsString = await streamReader.ReadToEndAsync();
}
}
}

Thanks David L, your suggestion worked great! Here's what I ended up with:
public async Task AddItemAsync(JsonElement dzc)
{
var id = dzc.GetProperty("id").GetString();
var userid = dzc.GetProperty("userid").GetString();
var raw = dzc.GetRawText();
byte[] bytes = Encoding.UTF8.GetBytes(raw);
using (var stream = new MemoryStream(bytes))
{
await this._container.CreateItemStreamAsync(stream, new PartitionKey(userid));
}
}
I'm curious if the copy can be avoided.

Can't Upload a File to QnaMaker Knowledge Base using SDK

I currently have an Azure Function that I would like to have update a QnaMaker Knowledge Base every day or so. Currently everything is connected and working fine, however I can only send Qna Objects (qna pairs) and not urls to files on a website of mine. So in the example I provided below, while it should populate the KB with 2 questions and the file from the url, it only populates the questions.
Currently this is not giving me any kind of error, in fact the response code from my call to the KB comes back as 204. So it it getting through, but still not adding the file to the KB as it should.
NOTE: The file being imported in this example (alice-I.html) is a random one for this demonstration (not mine, for security), but the issue is the same. If I directly add this file to the QnaMaker from the KB site itself it works fine, but it won't update from the Azure Function Code.
Any insights into what is happening would be great.
Content Being Sent To Knowledge Base
string replace_kb = #"{
'qnaList': [
{
'id': 0,
'answer': 'A-1',
'source': 'Custom Editorial',
'questions': [
'Q-1'
],
'metadata': []
},
{
'id': 1,
'answer': 'A-2',
'source': 'Custom Editorial',
'questions': [
'Q-2'
],
'metadata': [
{
'name': 'category',
'value': 'api'
}
]
}
],
'files': [
{
'fileName': 'alice-I.html',
'fileUri': 'https://www.cs.cmu.edu/~rgs/alice-I.html'
}
]
}";
Code Sending Content To Knowledge Base
using (var clientF = new HttpClient())
using (var requestF = new HttpRequestMessage())
{
requestF.Method = HttpMethod.Put;
requestF.RequestUri = new Uri(<your-uri>);
requestF.Content = new StringContent(replace_kb, Encoding.UTF8, "application/json");
requestF.Headers.Add("Ocp-Apim-Subscription-Key", <your-key>);
var responseF = await clientF.SendAsync(requestF);
if (responseF.IsSuccessStatusCode)
{
log.LogInformation("{'result' : 'Success.'}");
log.LogInformation($"------------>{responseF}");
}
else
{
await responseF.Content.ReadAsStringAsync();
log.LogInformation($"------------>{responseF}");
}
}

So I still don't know how to get the above working, but I got it to work a different way. Basically I used the UpdateKbOperationDTO Class listed here: class
This still isn't the perfect solution, but it allows me to update my KB with files using code instead of the interface.
Below is my new code:
QnAMakerClient qnaC = new QnAMakerClient(new ApiKeyServiceClientCredentials(<subscription-key>)) { Endpoint = "https://<your-custom-domain>.cognitiveservices.azure.com"};
log.LogInformation("Delete-->Start");
List<string> toDelete = new List<string>();
toDelete.Add("<my-file>");
var updateDelete = await qnaC.Knowledgebase.UpdateAsync(kbId, new UpdateKbOperationDTO
{
// Create JSON of changes ///
Add = null,
Update = null,
Delete = new UpdateKbOperationDTODelete(null, toDelete)
});
log.LogInformation("Delete-->Done");
log.LogInformation("Add-->Start");
List<FileDTO> toAdd = new List<FileDTO>();
toAdd.Add(new FileDTO("<my-file>", "<url-to-file>"));
var updateAdd = await qnaC.Knowledgebase.UpdateAsync(kbId, new UpdateKbOperationDTO
{
// Create JSON of changes ///
Add = new UpdateKbOperationDTOAdd(null, null, toAdd),
Update = null,
Delete = null
});
log.LogInformation("Add-->Done");

Http Request's Content Is Always Null When Making Request From Xamarin

I am making an app with Xamarin Forms and Azure. I am authenticating with Google and Facebook.
When requesting user's data, I use a HttpClient and make a request. Requesting from Google works fine but Facebook always returns null.
I have tested the Url in Postman with the same access token, it returns the correct data.
The Url is:
https://graph.facebook.com/v2.8/me/?fields=name,picture,locale,link,devices,email,first_name,last_name&access_token=
This returns:
{
"name": "Reece Russell",
"picture": {
"data": {
"height": 50,
"is_silhouette": false,
"url": "https://lookaside.facebook.com/platform/profilepic/?asid=2139651619601242&height=50&width=50&ext=1526558014&hash=AeSKoNrLS6O4UmMV",
"width": 50
}
},
"locale": "en_US",
"link": "https://www.facebook.com/app_scoped_user_id/YXNpZADpBWEhTLXBnRko4LWlBRUVzc0RBZAXhsc3R1eXpNNGZAxNmJMaXhWT013WTFCMVNnOFpLNE1jblh3dWJjLUwyRDhLQ0QyOHh6NmxnVGNyX2REMU9vUzNYTHFKNjlfN0J5R0lVbkV5ZA1V4aDRBZAVNDMnpwS0EZD/",
"devices": [
{
"hardware": "iPhone",
"os": "iOS"
}
],
"first_name": "Reece",
"last_name": "Russell",
"id": "2139651619601242"
}
But when making this request from Xamarin Forms using a HttpClient it always returns null.
The code I am using is below. I have also tried many ways of making the request including making the request from my server, which gets the correct data from FaceBook but returns null when I request from my server it still returns null.
using (var client = new HttpClient(new NativeMessageHandler()))
{
string url = "https://graph.facebook.com/v2.8/me/?fields=name,picture,locale,link,devices,email,first_name,last_name&access_token=" + Settings.AccessToken;
string json = await client.GetStringAsync(url);
FacebookUserProfile profile = JsonConvert.DeserializeObject<FacebookUserProfile>(json);
profile.PhotoUrl = profile.Picture.Data.Url;
profile.Name = profile.FirstName + " " + profile.LastName;
return profile;
}